Panic Mode On (78) Server Problems?

Message boards : Number crunching : Panic Mode On (78) Server Problems?

To post messages, you must log in.

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 22 · Next

AuthorMessage
Lionel

Send message
Joined: 25 Mar 00
Posts: 665
Credit: 351,240,036
RAC: 139,312
Australia
Message 1303824 - Posted: 9 Nov 2012, 3:30:24 UTC - in response to Message 1303784.  

Thanks Richard, I am utilizing your suggestion .. the point i was trying to make was the amount of ghost units I have which at the rate I am going will probably end up in alot of "timed out" errors..over 3000 in web account and 0 in BOINC manager

11/8/2012 5:33:05 PM | SETI@home | work fetch suspended by user
11/8/2012 5:33:07 PM | SETI@home | update requested by user
11/8/2012 5:37:33 PM | SETI@home | Scheduler request failed: Timeout was reached
11/8/2012 5:37:36 PM | | Project communication failed: attempting access to reference site
11/8/2012 5:37:37 PM | | Internet access OK - project servers may be temporarily down.
am I missing something


Join the club...I have 1,000s that are not reporting (or reporting very slowly); and 1,000s that have not downloaded and timing out as they can't be done in time.




ID: 1303824 · Report as offensive
Profile Jim Holly
Volunteer tester

Send message
Joined: 14 May 99
Posts: 11
Credit: 3,363,801
RAC: 0
United States
Message 1303868 - Posted: 9 Nov 2012, 5:23:45 UTC

Still no joy on getting more work units. Got those 44 a few days ago, and nothing since.
ID: 1303868 · Report as offensive
Profile edjcox
Avatar

Send message
Joined: 20 May 99
Posts: 88
Credit: 4,593,084
RAC: 596
United States
Message 1303875 - Posted: 9 Nov 2012, 5:42:17 UTC - in response to Message 1303868.  
Last modified: 9 Nov 2012, 5:58:22 UTC

My WU have piled up and can't be delivered either..
ID: 1303875 · Report as offensive
Profile S@NL - XP_Freak

Send message
Joined: 10 Jul 99
Posts: 99
Credit: 5,799,647
RAC: 1,383
Netherlands
Message 1303886 - Posted: 9 Nov 2012, 6:42:02 UTC

I have an i5 with 2 cores and 4 threads.
My cache is down to 155
I have no problem uploading or reporting work.

I expected to get new work when my cache was down to 200.
But still I get the following messages:

11/9/2012 7:33:22 AM Requesting new tasks for CPU
11/9/2012 7:33:35 AM Scheduler request completed: got 0 new tasks
11/9/2012 7:33:35 AM No tasks sent
11/9/2012 7:33:35 AM No tasks are available for SETI@home Enhanced
11/9/2012 7:33:35 AM This computer has reached a limit on tasks in progress
11/9/2012 7:33:35 AM Project has no tasks available

What is the limit for my system?

Goodbye Seti Classic
ID: 1303886 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45941
Credit: 815,398,131
RAC: 124,791
United States
Message 1303896 - Posted: 9 Nov 2012, 7:18:51 UTC

The kitties are still working off cache. No new work issued to them since the 5th, due to the current limits imposed. If they run out of Seti, they will fall back to Einstein. If that starts to happen too much, they will shut down.

I'm not going to pay the power bills I do unless I am crunching for the project I hold most dear.
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1303896 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 7490
Credit: 91,162,254
RAC: 46,319
Australia
Message 1303902 - Posted: 9 Nov 2012, 7:34:40 UTC - in response to Message 1303763.  

Other than the limits imposed, I'm not having any trouble connecting with the servers.

Cheers.

I'm still getting the odd Scheduler timeout, but now they're the exception, not the rule.
Grant
Darwin NT
ID: 1303902 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1303905 - Posted: 9 Nov 2012, 7:43:02 UTC

I have an i5 with 2 cores and 4 threads.
My cache is down to 155
I have no problem uploading or reporting work.

I expected to get new work when my cache was down to 200.
But still I get the following messages:

11/9/2012 7:33:22 AM Requesting new tasks for CPU
11/9/2012 7:33:35 AM Scheduler request completed: got 0 new tasks
11/9/2012 7:33:35 AM No tasks sent
11/9/2012 7:33:35 AM No tasks are available for SETI@home Enhanced
11/9/2012 7:33:35 AM This computer has reached a limit on tasks in progress
11/9/2012 7:33:35 AM Project has no tasks available

What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time.
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1303905 · Report as offensive
PomperipossaSucker

Send message
Joined: 22 Apr 02
Posts: 1
Credit: 19,140,519
RAC: 0
Sweden
Message 1303924 - Posted: 9 Nov 2012, 10:16:38 UTC

All my computers are now zeroed out, that is, they have no tasks to crunch.
But that is only true locally. On Seti@Home side of the fence all computers have tons of work in progress, as can be seen on the Computer status page - tasks in progress. However, none has reached my computers.

I believe these are calles ghosts, and they are now piling up fast. Currently I have 1366 tasks marked as in progress, when however none have reached me to be crunched.
So I'm now throwing in my cents worth into the error mix discussion.


Could it be a network problem?

Shouldn't there be some sort of TCP Ack sent and received when download of task is complete in order to mark the tasks as sent and received? And only then should the task be marked as "In Progress"?
Could this be a BOINC bug, or is it a server side bug?

Perhaps the new Foundry switch is dropping Acks intermittently?
ID: 1303924 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 7490
Credit: 91,162,254
RAC: 46,319
Australia
Message 1303927 - Posted: 9 Nov 2012, 10:43:50 UTC - in response to Message 1303924.  


Not sure what or why it's happening; i've got 4,230 WU's in total. 1150 are in progress, but 1955 are pending validation.
And my RAC is dropping like a stone to reflect all the pendings, and there's no backlog with the validators.
?
Grant
Darwin NT
ID: 1303927 · Report as offensive
Profile Bill GProject Donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 576
Credit: 86,276,785
RAC: 61,774
United States
Message 1303937 - Posted: 9 Nov 2012, 11:27:33 UTC - in response to Message 1303905.  



What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time. [/quote]

If this is true then my 8 core should have more than 89 tasks and my GPU (2 cards, 4 tasks) should have more than 110 tasks and all I get is "The computer has reached the limit on tasks in progress".
Now this computer does have over 3000 ghosts on it, I do not know if that affect anything or not, we have been told not but it sure looks like it.

Also another computer for whatever reason can not report its tasks, nor make contact with the server..no hint of anything wrong on this end.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1303937 · Report as offensive
Profile Bill GProject Donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 576
Credit: 86,276,785
RAC: 61,774
United States
Message 1303940 - Posted: 9 Nov 2012, 11:29:29 UTC - in response to Message 1303927.  


Not sure what or why it's happening; i've got 4,230 WU's in total. 1150 are in progress, but 1955 are pending validation.
And my RAC is dropping like a stone to reflect all the pendings, and there's no backlog with the validators.
?

Grant, I will bet that most of the pending are from the time period when all the ghosts were generated and these are sitting somewhere on the server right now, not being prcessed at all.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1303940 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 38185
Credit: 21,406,493
RAC: 27,741
United Kingdom
Message 1303942 - Posted: 9 Nov 2012, 11:35:21 UTC

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

ID: 1303942 · Report as offensive
Profile Bill GProject Donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 576
Credit: 86,276,785
RAC: 61,774
United States
Message 1303945 - Posted: 9 Nov 2012, 11:38:12 UTC - in response to Message 1303942.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.


Now that is new.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1303945 · Report as offensive
Richard HaselgroveProject Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 11142
Credit: 83,824,125
RAC: 46,024
United Kingdom
Message 1303955 - Posted: 9 Nov 2012, 12:01:34 UTC - in response to Message 1303945.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.
ID: 1303955 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 8597
Credit: 43,164,195
RAC: 20,371
United Kingdom
Message 1303992 - Posted: 9 Nov 2012, 14:03:55 UTC - in response to Message 1303955.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Top left of server status page:

[As of 9 Nov 2012 | 14:00:07 UTC]

As Richard points out depending on when you access the page it can be up to 9:59 behind!
"Sometimes it is the people no one imagines anything of who do the things that no one can imagine."
ID: 1303992 · Report as offensive
David SProject Donor
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 17047
Credit: 20,956,022
RAC: 6,201
United States
Message 1303999 - Posted: 9 Nov 2012, 14:26:25 UTC

Questions:

1. Would it be possible for Eric (or someone) to flush all the ghosts out of the scheduler? (I.e., somehow identify them as ghosts and deassign them so the scheduler isn't trying to keep track of so many.)

2. If it's possible, would it help the situation any to do it?

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


ID: 1303999 · Report as offensive
David SProject Donor
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 17047
Credit: 20,956,022
RAC: 6,201
United States
Message 1304001 - Posted: 9 Nov 2012, 14:30:55 UTC - in response to Message 1303992.  

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Top left of server status page:

[As of 9 Nov 2012 | 14:00:07 UTC]

As Richard points out depending on when you access the page it can be up to 9:59 behind!

Or 19:59. Sometimes it switches to a 20 minute refresh cycle. Plus, it can take a minute or so to process the new snapshot and generate the report the page is based on. In my experience, it's very unlikely that you will ever see that page less than 50 seconds out of date.

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


ID: 1304001 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1304005 - Posted: 9 Nov 2012, 14:49:13 UTC

Bruno's naps, or as Richard says, micro-naps, are so frequent I ignore them unless there's a drop on Cricket. Those naps usually are so short they usually don't cause a blip. Slavac's about to launch a fundraiser because the project wants to replace Bruno and Vader.

Questions:

1. Would it be possible for Eric (or someone) to flush all the ghosts out of the scheduler? (I.e., somehow identify them as ghosts and deassign them so the scheduler isn't trying to keep track of so many.)

2. If it's possible, would it help the situation any to do it?

Interesting thought. Not sure it would help if done before they find the fix since new ghosts would be generated by the timeouts, but if they can do it, it would help clean things up.
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1304005 · Report as offensive
Profile Bill GProject Donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 576
Credit: 86,276,785
RAC: 61,774
United States
Message 1304006 - Posted: 9 Nov 2012, 14:51:26 UTC - in response to Message 1303937.  
Last modified: 9 Nov 2012, 15:00:29 UTC



What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time.


If this is true then my 8 core should have more than 89 tasks and my GPU (2 cards, 4 tasks) should have more than 110 tasks and all I get is "The computer has reached the limit on tasks in progress".
Now this computer does have over 3000 ghosts on it, I do not know if that affect anything or not, we have been told not but it sure looks like it.

Also another computer for whatever reason can not report its tasks, nor make contact with the server..no hint of anything wrong on this end.


I have started to download CPU Ghost tasks which seem to be keeping my cache at 89 WUs. At least it seems to be working in some manner now.

added: You know I just re-read your answer....and you say 100 is the max as I have only one CPU here. Sorry to have missed that before. So the 89 is actually close to the 100 maximum on the CPU. However since I have two GPUs that should have close to 200 WUs and it is getting down to 100 shortly.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1304006 · Report as offensive
Cherokee150

Send message
Joined: 11 Nov 99
Posts: 139
Credit: 34,341,654
RAC: 19,790
United States
Message 1304014 - Posted: 9 Nov 2012, 15:09:23 UTC

With so many of our loyal SETI community upset and frustrated over the current situation, perhaps it would be a good time for one of the SETI staff to take a few minutes to let us know what their ideas are about the problem, why they have limited us so severely, and if they have been able to determine a way to fix things and release the restrictions.
ID: 1304014 · Report as offensive
Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 22 · Next

Message boards : Number crunching : Panic Mode On (78) Server Problems?


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.