Panic Mode On (78) Server Problems?

Message boards : Number crunching : Panic Mode On (78) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 22 · Next

AuthorMessage
Profile Jim Holly
Volunteer tester

Send message
Joined: 14 May 99
Posts: 11
Credit: 3,363,801
RAC: 0
United States
Message 1303868 - Posted: 9 Nov 2012, 5:23:45 UTC

Still no joy on getting more work units. Got those 44 a few days ago, and nothing since.
ID: 1303868 · Report as offensive
Profile edjcox
Avatar

Send message
Joined: 20 May 99
Posts: 96
Credit: 5,878,353
RAC: 0
United States
Message 1303875 - Posted: 9 Nov 2012, 5:42:17 UTC - in response to Message 1303868.  
Last modified: 9 Nov 2012, 5:58:22 UTC

My WU have piled up and can't be delivered either..
ID: 1303875 · Report as offensive
Profile S@NL - XP_Freak

Send message
Joined: 10 Jul 99
Posts: 99
Credit: 6,248,265
RAC: 0
Netherlands
Message 1303886 - Posted: 9 Nov 2012, 6:42:02 UTC

I have an i5 with 2 cores and 4 threads.
My cache is down to 155
I have no problem uploading or reporting work.

I expected to get new work when my cache was down to 200.
But still I get the following messages:

11/9/2012 7:33:22 AM Requesting new tasks for CPU
11/9/2012 7:33:35 AM Scheduler request completed: got 0 new tasks
11/9/2012 7:33:35 AM No tasks sent
11/9/2012 7:33:35 AM No tasks are available for SETI@home Enhanced
11/9/2012 7:33:35 AM This computer has reached a limit on tasks in progress
11/9/2012 7:33:35 AM Project has no tasks available

What is the limit for my system?

Goodbye Seti Classic
ID: 1303886 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1303896 - Posted: 9 Nov 2012, 7:18:51 UTC

The kitties are still working off cache. No new work issued to them since the 5th, due to the current limits imposed. If they run out of Seti, they will fall back to Einstein. If that starts to happen too much, they will shut down.

I'm not going to pay the power bills I do unless I am crunching for the project I hold most dear.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1303896 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1303902 - Posted: 9 Nov 2012, 7:34:40 UTC - in response to Message 1303763.  

Other than the limits imposed, I'm not having any trouble connecting with the servers.

Cheers.

I'm still getting the odd Scheduler timeout, but now they're the exception, not the rule.
Grant
Darwin NT
ID: 1303902 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1303905 - Posted: 9 Nov 2012, 7:43:02 UTC

I have an i5 with 2 cores and 4 threads.
My cache is down to 155
I have no problem uploading or reporting work.

I expected to get new work when my cache was down to 200.
But still I get the following messages:

11/9/2012 7:33:22 AM Requesting new tasks for CPU
11/9/2012 7:33:35 AM Scheduler request completed: got 0 new tasks
11/9/2012 7:33:35 AM No tasks sent
11/9/2012 7:33:35 AM No tasks are available for SETI@home Enhanced
11/9/2012 7:33:35 AM This computer has reached a limit on tasks in progress
11/9/2012 7:33:35 AM Project has no tasks available

What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time.
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1303905 · Report as offensive
PomperipossaSucker

Send message
Joined: 22 Apr 02
Posts: 1
Credit: 22,409,096
RAC: 9
Sweden
Message 1303924 - Posted: 9 Nov 2012, 10:16:38 UTC

All my computers are now zeroed out, that is, they have no tasks to crunch.
But that is only true locally. On Seti@Home side of the fence all computers have tons of work in progress, as can be seen on the Computer status page - tasks in progress. However, none has reached my computers.

I believe these are calles ghosts, and they are now piling up fast. Currently I have 1366 tasks marked as in progress, when however none have reached me to be crunched.
So I'm now throwing in my cents worth into the error mix discussion.


Could it be a network problem?

Shouldn't there be some sort of TCP Ack sent and received when download of task is complete in order to mark the tasks as sent and received? And only then should the task be marked as "In Progress"?
Could this be a BOINC bug, or is it a server side bug?

Perhaps the new Foundry switch is dropping Acks intermittently?
ID: 1303924 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1303927 - Posted: 9 Nov 2012, 10:43:50 UTC - in response to Message 1303924.  


Not sure what or why it's happening; i've got 4,230 WU's in total. 1150 are in progress, but 1955 are pending validation.
And my RAC is dropping like a stone to reflect all the pendings, and there's no backlog with the validators.
?
Grant
Darwin NT
ID: 1303927 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1303937 - Posted: 9 Nov 2012, 11:27:33 UTC - in response to Message 1303905.  



What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time. [/quote]

If this is true then my 8 core should have more than 89 tasks and my GPU (2 cards, 4 tasks) should have more than 110 tasks and all I get is "The computer has reached the limit on tasks in progress".
Now this computer does have over 3000 ghosts on it, I do not know if that affect anything or not, we have been told not but it sure looks like it.

Also another computer for whatever reason can not report its tasks, nor make contact with the server..no hint of anything wrong on this end.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1303937 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1303940 - Posted: 9 Nov 2012, 11:29:29 UTC - in response to Message 1303927.  


Not sure what or why it's happening; i've got 4,230 WU's in total. 1150 are in progress, but 1955 are pending validation.
And my RAC is dropping like a stone to reflect all the pendings, and there's no backlog with the validators.
?

Grant, I will bet that most of the pending are from the time period when all the ghosts were generated and these are sitting somewhere on the server right now, not being prcessed at all.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1303940 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1303945 - Posted: 9 Nov 2012, 11:38:12 UTC - in response to Message 1303942.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.


Now that is new.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1303945 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1303955 - Posted: 9 Nov 2012, 12:01:34 UTC - in response to Message 1303945.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.
ID: 1303955 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 9954
Credit: 103,452,613
RAC: 328
United Kingdom
Message 1303992 - Posted: 9 Nov 2012, 14:03:55 UTC - in response to Message 1303955.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Top left of server status page:

[As of 9 Nov 2012 | 14:00:07 UTC]

As Richard points out depending on when you access the page it can be up to 9:59 behind!
ID: 1303992 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1303999 - Posted: 9 Nov 2012, 14:26:25 UTC

Questions:

1. Would it be possible for Eric (or someone) to flush all the ghosts out of the scheduler? (I.e., somehow identify them as ghosts and deassign them so the scheduler isn't trying to keep track of so many.)

2. If it's possible, would it help the situation any to do it?

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1303999 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1304001 - Posted: 9 Nov 2012, 14:30:55 UTC - in response to Message 1303992.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Top left of server status page:

[As of 9 Nov 2012 | 14:00:07 UTC]

As Richard points out depending on when you access the page it can be up to 9:59 behind!

Or 19:59. Sometimes it switches to a 20 minute refresh cycle. Plus, it can take a minute or so to process the new snapshot and generate the report the page is based on. In my experience, it's very unlikely that you will ever see that page less than 50 seconds out of date.

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1304001 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1304005 - Posted: 9 Nov 2012, 14:49:13 UTC

Bruno's naps, or as Richard says, micro-naps, are so frequent I ignore them unless there's a drop on Cricket. Those naps usually are so short they usually don't cause a blip. Slavac's about to launch a fundraiser because the project wants to replace Bruno and Vader.

Questions:

1. Would it be possible for Eric (or someone) to flush all the ghosts out of the scheduler? (I.e., somehow identify them as ghosts and deassign them so the scheduler isn't trying to keep track of so many.)

2. If it's possible, would it help the situation any to do it?

Interesting thought. Not sure it would help if done before they find the fix since new ghosts would be generated by the timeouts, but if they can do it, it would help clean things up.
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1304005 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1304006 - Posted: 9 Nov 2012, 14:51:26 UTC - in response to Message 1303937.  
Last modified: 9 Nov 2012, 15:00:29 UTC



What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time.


If this is true then my 8 core should have more than 89 tasks and my GPU (2 cards, 4 tasks) should have more than 110 tasks and all I get is "The computer has reached the limit on tasks in progress".
Now this computer does have over 3000 ghosts on it, I do not know if that affect anything or not, we have been told not but it sure looks like it.

Also another computer for whatever reason can not report its tasks, nor make contact with the server..no hint of anything wrong on this end.


I have started to download CPU Ghost tasks which seem to be keeping my cache at 89 WUs. At least it seems to be working in some manner now.

added: You know I just re-read your answer....and you say 100 is the max as I have only one CPU here. Sorry to have missed that before. So the 89 is actually close to the 100 maximum on the CPU. However since I have two GPUs that should have close to 200 WUs and it is getting down to 100 shortly.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1304006 · Report as offensive
Cherokee150

Send message
Joined: 11 Nov 99
Posts: 192
Credit: 58,513,758
RAC: 74
United States
Message 1304014 - Posted: 9 Nov 2012, 15:09:23 UTC

With so many of our loyal SETI community upset and frustrated over the current situation, perhaps it would be a good time for one of the SETI staff to take a few minutes to let us know what their ideas are about the problem, why they have limited us so severely, and if they have been able to determine a way to fix things and release the restrictions.
ID: 1304014 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1304017 - Posted: 9 Nov 2012, 15:20:39 UTC
Last modified: 9 Nov 2012, 15:21:26 UTC

However since I have two GPUs that should have close to 200 WUs and it is getting down to 100 shortly.


There hasn't been any hard info on the limits, just going with what others are reporting. I got down to 100 cpu yesterday at about this time, and run on NNT most of the time i request work when I get down 15 or 20, and when I'm lucky and get a successful work request, the server has replenished me to exactly 100. Not down there on GPU yet, and there haven't been many posts to confirm that limit. Still assuming it is per gpu and your limit s/b 200 a. Not sure if ghosts are causing your problem.

With so many of our loyal SETI community upset and frustrated over the current situation, perhaps it would be a good time for one of the SETI staff to take a few minutes to let us know what their ideas are about the problem, why they have limited us so severely, and if they have been able to determine a way to fix things and release the restrictions.

+1. I'm sure they are working on it, but...
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1304017 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1304031 - Posted: 9 Nov 2012, 16:09:35 UTC - in response to Message 1303992.  
Last modified: 9 Nov 2012, 16:10:49 UTC



As Richard points out depending on when you access the page it can be up to 9:59 behind!

Not quite true.
The status page typically updates every 10 minutes...
But, for reasons unknown to me, it will sometimes only update every 20 minutes.

EDIT...
Ooops. Sorry, I see somebody already mentioned that.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1304031 · Report as offensive
Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 22 · Next

Message boards : Number crunching : Panic Mode On (78) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.