Panic Mode On (78) Server Problems?


log in

Advanced search

Message boards : Number crunching : Panic Mode On (78) Server Problems?

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 22 · Next
Author Message
Lionel
Send message
Joined: 25 Mar 00
Posts: 544
Credit: 216,082,043
RAC: 196,817
Australia
Message 1303824 - Posted: 9 Nov 2012, 3:30:24 UTC - in response to Message 1303784.

Thanks Richard, I am utilizing your suggestion .. the point i was trying to make was the amount of ghost units I have which at the rate I am going will probably end up in alot of "timed out" errors..over 3000 in web account and 0 in BOINC manager

11/8/2012 5:33:05 PM | SETI@home | work fetch suspended by user
11/8/2012 5:33:07 PM | SETI@home | update requested by user
11/8/2012 5:37:33 PM | SETI@home | Scheduler request failed: Timeout was reached
11/8/2012 5:37:36 PM | | Project communication failed: attempting access to reference site
11/8/2012 5:37:37 PM | | Internet access OK - project servers may be temporarily down.
am I missing something


Join the club...I have 1,000s that are not reporting (or reporting very slowly); and 1,000s that have not downloaded and timing out as they can't be done in time.




____________

Profile Jim Holly
Volunteer tester
Send message
Joined: 14 May 99
Posts: 11
Credit: 3,363,801
RAC: 0
United States
Message 1303868 - Posted: 9 Nov 2012, 5:23:45 UTC

Still no joy on getting more work units. Got those 44 a few days ago, and nothing since.
____________

Profile edjcox
Avatar
Send message
Joined: 20 May 99
Posts: 63
Credit: 3,967,042
RAC: 1,019
United States
Message 1303875 - Posted: 9 Nov 2012, 5:42:17 UTC - in response to Message 1303868.
Last modified: 9 Nov 2012, 5:58:22 UTC

My WU have piled up and can't be delivered either..

Profile S@NL - XP_Freak
Send message
Joined: 10 Jul 99
Posts: 99
Credit: 4,662,550
RAC: 1,944
Netherlands
Message 1303886 - Posted: 9 Nov 2012, 6:42:02 UTC

I have an i5 with 2 cores and 4 threads.
My cache is down to 155
I have no problem uploading or reporting work.

I expected to get new work when my cache was down to 200.
But still I get the following messages:

11/9/2012 7:33:22 AM Requesting new tasks for CPU
11/9/2012 7:33:35 AM Scheduler request completed: got 0 new tasks
11/9/2012 7:33:35 AM No tasks sent
11/9/2012 7:33:35 AM No tasks are available for SETI@home Enhanced
11/9/2012 7:33:35 AM This computer has reached a limit on tasks in progress
11/9/2012 7:33:35 AM Project has no tasks available

What is the limit for my system?
____________

Goodbye Seti Classic

msattler
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38154
Credit: 556,421,497
RAC: 597,128
United States
Message 1303896 - Posted: 9 Nov 2012, 7:18:51 UTC

The kitties are still working off cache. No new work issued to them since the 5th, due to the current limits imposed. If they run out of Seti, they will fall back to Einstein. If that starts to happen too much, they will shut down.

I'm not going to pay the power bills I do unless I am crunching for the project I hold most dear.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5685
Credit: 56,144,267
RAC: 49,755
Australia
Message 1303902 - Posted: 9 Nov 2012, 7:34:40 UTC - in response to Message 1303763.

Other than the limits imposed, I'm not having any trouble connecting with the servers.

Cheers.

I'm still getting the odd Scheduler timeout, but now they're the exception, not the rule.
____________
Grant
Darwin NT.

Profile Fred E.
Volunteer tester
Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,135,140
RAC: 4,337
United States
Message 1303905 - Posted: 9 Nov 2012, 7:43:02 UTC

I have an i5 with 2 cores and 4 threads.
My cache is down to 155
I have no problem uploading or reporting work.

I expected to get new work when my cache was down to 200.
But still I get the following messages:

11/9/2012 7:33:22 AM Requesting new tasks for CPU
11/9/2012 7:33:35 AM Scheduler request completed: got 0 new tasks
11/9/2012 7:33:35 AM No tasks sent
11/9/2012 7:33:35 AM No tasks are available for SETI@home Enhanced
11/9/2012 7:33:35 AM This computer has reached a limit on tasks in progress
11/9/2012 7:33:35 AM Project has no tasks available

What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time.
____________
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.

PomperipossaSucker
Send message
Joined: 22 Apr 02
Posts: 1
Credit: 18,760,151
RAC: 0
Sweden
Message 1303924 - Posted: 9 Nov 2012, 10:16:38 UTC

All my computers are now zeroed out, that is, they have no tasks to crunch.
But that is only true locally. On Seti@Home side of the fence all computers have tons of work in progress, as can be seen on the Computer status page - tasks in progress. However, none has reached my computers.

I believe these are calles ghosts, and they are now piling up fast. Currently I have 1366 tasks marked as in progress, when however none have reached me to be crunched.
So I'm now throwing in my cents worth into the error mix discussion.


Could it be a network problem?

Shouldn't there be some sort of TCP Ack sent and received when download of task is complete in order to mark the tasks as sent and received? And only then should the task be marked as "In Progress"?
Could this be a BOINC bug, or is it a server side bug?

Perhaps the new Foundry switch is dropping Acks intermittently?
____________

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5685
Credit: 56,144,267
RAC: 49,755
Australia
Message 1303927 - Posted: 9 Nov 2012, 10:43:50 UTC - in response to Message 1303924.


Not sure what or why it's happening; i've got 4,230 WU's in total. 1150 are in progress, but 1955 are pending validation.
And my RAC is dropping like a stone to reflect all the pendings, and there's no backlog with the validators.
?
____________
Grant
Darwin NT.

Profile Bill G
Avatar
Send message
Joined: 1 Jun 01
Posts: 347
Credit: 37,114,600
RAC: 74,902
United States
Message 1303937 - Posted: 9 Nov 2012, 11:27:33 UTC - in response to Message 1303905.



What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time. [/quote]

If this is true then my 8 core should have more than 89 tasks and my GPU (2 cards, 4 tasks) should have more than 110 tasks and all I get is "The computer has reached the limit on tasks in progress".
Now this computer does have over 3000 ghosts on it, I do not know if that affect anything or not, we have been told not but it sure looks like it.

Also another computer for whatever reason can not report its tasks, nor make contact with the server..no hint of anything wrong on this end.
____________

Profile Bill G
Avatar
Send message
Joined: 1 Jun 01
Posts: 347
Credit: 37,114,600
RAC: 74,902
United States
Message 1303940 - Posted: 9 Nov 2012, 11:29:29 UTC - in response to Message 1303927.


Not sure what or why it's happening; i've got 4,230 WU's in total. 1150 are in progress, but 1955 are pending validation.
And my RAC is dropping like a stone to reflect all the pendings, and there's no backlog with the validators.
?

Grant, I will bet that most of the pending are from the time period when all the ghosts were generated and these are sitting somewhere on the server right now, not being prcessed at all.
____________

Profile Chris S
Volunteer tester
Avatar
Send message
Joined: 19 Nov 00
Posts: 31007
Credit: 11,201,152
RAC: 19,710
United Kingdom
Message 1303942 - Posted: 9 Nov 2012, 11:35:21 UTC

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Profile Bill G
Avatar
Send message
Joined: 1 Jun 01
Posts: 347
Credit: 37,114,600
RAC: 74,902
United States
Message 1303945 - Posted: 9 Nov 2012, 11:38:12 UTC - in response to Message 1303942.

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.


Now that is new.
____________

Richard Haselgrove
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8374
Credit: 46,568,904
RAC: 14,319
United Kingdom
Message 1303955 - Posted: 9 Nov 2012, 12:01:34 UTC - in response to Message 1303945.

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 26 May 99
Posts: 6796
Credit: 24,465,780
RAC: 26,934
United Kingdom
Message 1303992 - Posted: 9 Nov 2012, 14:03:55 UTC - in response to Message 1303955.

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Top left of server status page:

[As of 9 Nov 2012 | 14:00:07 UTC]

As Richard points out depending on when you access the page it can be up to 9:59 behind!
____________


Today is life, the only life we're sure of. Make the most of today.

N9JFE David S
Volunteer tester
Avatar
Send message
Joined: 4 Oct 99
Posts: 10709
Credit: 13,395,541
RAC: 14,803
United States
Message 1303999 - Posted: 9 Nov 2012, 14:26:25 UTC

Questions:

1. Would it be possible for Eric (or someone) to flush all the ghosts out of the scheduler? (I.e., somehow identify them as ghosts and deassign them so the scheduler isn't trying to keep track of so many.)

2. If it's possible, would it help the situation any to do it?

____________
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


N9JFE David S
Volunteer tester
Avatar
Send message
Joined: 4 Oct 99
Posts: 10709
Credit: 13,395,541
RAC: 14,803
United States
Message 1304001 - Posted: 9 Nov 2012, 14:30:55 UTC - in response to Message 1303992.

The upload server is disabled, and it's only 3.30am in Berkeley. Ho hum.

Now that is new.

No, it's very common. If the upload server (or any server) happens to be taking a micro-nap at the precise instant that nurse comes round to take its temperature, it shows as disabled until the next 10-minute snapshot display of the Status page.

Top left of server status page:

[As of 9 Nov 2012 | 14:00:07 UTC]

As Richard points out depending on when you access the page it can be up to 9:59 behind!

Or 19:59. Sometimes it switches to a 20 minute refresh cycle. Plus, it can take a minute or so to process the new snapshot and generate the report the page is based on. In my experience, it's very unlikely that you will ever see that page less than 50 seconds out of date.

____________
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


Profile Fred E.
Volunteer tester
Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,135,140
RAC: 4,337
United States
Message 1304005 - Posted: 9 Nov 2012, 14:49:13 UTC

Bruno's naps, or as Richard says, micro-naps, are so frequent I ignore them unless there's a drop on Cricket. Those naps usually are so short they usually don't cause a blip. Slavac's about to launch a fundraiser because the project wants to replace Bruno and Vader.

Questions:

1. Would it be possible for Eric (or someone) to flush all the ghosts out of the scheduler? (I.e., somehow identify them as ghosts and deassign them so the scheduler isn't trying to keep track of so many.)

2. If it's possible, would it help the situation any to do it?

Interesting thought. Not sure it would help if done before they find the fix since new ghosts would be generated by the timeouts, but if they can do it, it would help clean things up.
____________
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.

Profile Bill G
Avatar
Send message
Joined: 1 Jun 01
Posts: 347
Credit: 37,114,600
RAC: 74,902
United States
Message 1304006 - Posted: 9 Nov 2012, 14:51:26 UTC - in response to Message 1303937.
Last modified: 9 Nov 2012, 15:00:29 UTC



What is the limit for my system?

100 per cpu and 100 per gpu. The # of cores is not a factor this time.


If this is true then my 8 core should have more than 89 tasks and my GPU (2 cards, 4 tasks) should have more than 110 tasks and all I get is "The computer has reached the limit on tasks in progress".
Now this computer does have over 3000 ghosts on it, I do not know if that affect anything or not, we have been told not but it sure looks like it.

Also another computer for whatever reason can not report its tasks, nor make contact with the server..no hint of anything wrong on this end.


I have started to download CPU Ghost tasks which seem to be keeping my cache at 89 WUs. At least it seems to be working in some manner now.

added: You know I just re-read your answer....and you say 100 is the max as I have only one CPU here. Sorry to have missed that before. So the 89 is actually close to the 100 maximum on the CPU. However since I have two GPUs that should have close to 200 WUs and it is getting down to 100 shortly.
____________

Cherokee150
Send message
Joined: 11 Nov 99
Posts: 103
Credit: 23,060,622
RAC: 28,238
United States
Message 1304014 - Posted: 9 Nov 2012, 15:09:23 UTC

With so many of our loyal SETI community upset and frustrated over the current situation, perhaps it would be a good time for one of the SETI staff to take a few minutes to let us know what their ideas are about the problem, why they have limited us so severely, and if they have been able to determine a way to fix things and release the restrictions.

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 22 · Next

Message boards : Number crunching : Panic Mode On (78) Server Problems?

Copyright © 2014 University of California