Message boards :
Number crunching :
Lost "Ghost" task recovery protocol
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · Next
Author | Message |
---|---|
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
The problem comes from what the host advertises. One the magical things that the Lunatics installer does is preserve all the current cache of work on the host when converting to Lunatics apps. It does this by extensive rewriting of the client_state.xml file to change all the current entries for every task in the cache to the new plan_class and applications for the Lunatics apps. I think this is what confuses the server task scheduler. What it used to know about what the host advertised of its capabilities is all of a sudden completely different upon the first connection. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Ville Saari Send message Joined: 30 Nov 00 Posts: 1158 Credit: 49,177,052 RAC: 82,530 |
I upgraded Boinc to a newer version with the result that it ghosted everything I had as usual. I used the recovery protocol and again the server made everything expire instead of letting me redownload them :( Here is one example of those tasks: https://setiathome.berkeley.edu/result.php?resultid=8055686426 Task 8055686426 Name 13se19aa.6745.885.5.32.181.vlar_0 Workunit 3655915446 Created 17 Sep 2019, 6:44:38 UTC Sent 17 Sep 2019, 12:38:13 UTC Report deadline 17 Sep 2019, 13:17:16 UTC Received --- Server state Over Outcome No reply Client state New Exit status 0 (0x00000000) Computer ID 8652081 Run time CPU time Validate state Initial Credit 0.00 Device peak FLOPS 0.00 GFLOPS Application version SETI@home v8 Anonymous platform (CPU) So "deadline" less than one hour after initially receiving the task :( |
rob smith Send message Joined: 7 Mar 03 Posts: 22190 Credit: 416,307,556 RAC: 380 |
That is not a "Ghost" task, it is one that has failed to calculate in time. A "Ghost" task is one that left the servers but you never actually received. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Mr. Kevvy Send message Joined: 15 May 99 Posts: 3776 Credit: 1,114,826,392 RAC: 3,319 |
|
Ville Saari Send message Joined: 30 Nov 00 Posts: 1158 Credit: 49,177,052 RAC: 82,530 |
No need to wipe. Boinc wipes them on its own from the slightest suspicion of trouble.A "Ghost" task is one that left the servers but you never actually received.You can also "ghost" them by either wiping the files from the project folder, or badly editing app_info.xml so that the executable for a platform that has work units is invalid ie has a typo. In this case I just stopped the Boinc client, upgraded it and restarted it. It wiped everything on launch including the apps! And then downloaded nothing because it had no apps any more. App_info.xml was still intact, but all the apps it referred to had vanished. I kept copies elsewhere so I could easily restore them as this wasn't the first time boinc client has deleted my apps! |
Freewill Send message Joined: 19 May 99 Posts: 766 Credit: 354,398,348 RAC: 11,693 |
For those with many more than the current limit of 80 ghost tasks, the following may be helpful. I had quite a large number due to a disk failure. I discovered you can save repeating this procedure. If you leave no new tasks set for along time so that many more than 80 active tasks are returned and then follow Keith's procedure, you'll get 80 tasks resent at the first update and then it appears you'll get more resends at the next update (although it may not say so). Can anyone else confirm this is what's happening? Roger |
Freewill Send message Joined: 19 May 99 Posts: 766 Credit: 354,398,348 RAC: 11,693 |
Actually, you should be able to set it up to where instead of sending back 20 tasks it will 'Expire' all your 'Lost tasks' in one move. That worked like a charm! It moves the ghost tasks to the "error" section with a status of "timed out - no response" but at least they'll get resent to someone and not leave wing people hanging. Thanks, TBar! |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
One the magical things that the Lunatics installer does is preserve all the current cache of work on the host when converting to Lunatics apps. It does this by extensive rewriting of the client_state.xml file ...Actually, it doesn't. It doesn't even touch client_state.xml What it does is to create an app_info.xml file which covers every known combination of platform, version, and plan_class. Whatever you might have lurking in your client_state file - has a home to go to, and one of the Lunatics apps will pick it up and run with it. What this does mean is that if you write your own app_info.xml, and don't follow the established platform, version, and plan_class values, work assigned under that app_info file will probably become homeless after running the installer. |
rcthardcore Send message Joined: 23 Nov 08 Posts: 48 Credit: 1,306,006 RAC: 0 |
Maybe it is time for a function to be programmed into BOINC in order to allow us to download all of our ghost tasks. A button maybe. You would think that this would have already been done. |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Even with a button in the client, resending lost work has to be configured on every individual projects scheduler in the server software. So it would work on some projects and not on others depending on whether the project has that function enabled. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
I was talking in the recent past with Richard about this, an automated ghost recovery program. Seriously is not an complicated program, but the main problem is it could add an additional huge load to the servers something we all not want. In theory it will need to scrap all the db looking for all the Wu with destination of a particular host and check if they are on the host or no and make the adjusts on the client file and DL the WU to it. IIRC there are actually a subroutine on the server side who make something very similar, but is disabled exactly because the extra load. But there is the problem, imagine few hosts making that at the same time... I never tried but know few who does and say it works, i imagine the method described by Tbar on this thread is the easy way to do that. my 0.02 |
Lazydude Send message Joined: 17 Jan 01 Posts: 45 Credit: 96,158,001 RAC: 136 |
I I would like a button on the Computer page for resends. With restriction can do this only most every 3days and not on Tuesdays and a period of xx hours after outtake. Lazy |
Steven Gaber Send message Joined: 19 Jan 13 Posts: 111 Credit: 2,834,186 RAC: 11 |
"Thanks for this! The procedure seems clear and I tried it, as I appear to have about 60 ghost tasks. However, the server currently has no tasks to send. Would that cause it not to do the resends?" My account page says I have 64 tasks in progress, but my activity page only shows 15 , pus a bunch awaiting validation. I guess the end is beginning? Steven Gaber Oldsmar, FL |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
"Thanks for this! The procedure seems clear and I tried it, as I appear to have about 60 ghost tasks. However, the server currently has no tasks to send. Would that cause it not to do the resends?"That shouldn't be the case, as they are already Tasks that it thinks you have, so it should just resend those particular Tasks to your system again. It doesn't have to actually have any Tasks to send, or make any new ones up. Just re-send the existing ones. Grant Darwin NT |
Jiiimbooh Send message Joined: 1 Jul 09 Posts: 4 Credit: 321,586 RAC: 13 |
"Thanks for this! The procedure seems clear and I tried it, as I appear to have about 60 ghost tasks. However, the server currently has no tasks to send. Would that cause it not to do the resends?" The tasks page is not up to date right now. Judging by how many apparent ghost tasks I have, I'd say the page is about 4 days behind. |
BobMiller Send message Joined: 24 Jul 08 Posts: 32 Credit: 11,041,077 RAC: 129 |
Help, my tasks have disappeared again. |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Yours tasks shows: Outcome Abandoned Did you do reset the project or something like this? |
BobMiller Send message Joined: 24 Jul 08 Posts: 32 Credit: 11,041,077 RAC: 129 |
they disappeared with no action on my part. are they completely and absolutely lost if the project is reset |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
AFAIK yes they are completely lost from your host POV if abandoned. If you look at the WU the task is already send to another host. Maybe someone else knows a way to recover and post here for us to learn. |
BobMiller Send message Joined: 24 Jul 08 Posts: 32 Credit: 11,041,077 RAC: 129 |
Juan, Gracias, Me gusta a conocer usted |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.