Panic Mode On (27) Server problems

Message boards : Number crunching : Panic Mode On (27) Server problems
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 10 · Next

AuthorMessage
Profile Smariga
Avatar

Send message
Joined: 13 Jun 99
Posts: 49
Credit: 30,454,070
RAC: 0
United States
Message 957783 - Posted: 21 Dec 2009, 13:38:46 UTC - in response to Message 957642.  

Its been disabled for a while. Have plenty of regular workunits to do, but of course am out of CUDA units and none are getting retrieved. With the algotithms for work the way they are, never seem to have enough CUDA to work on. The slightest blip, and my GPU is idle.
Alex
ID: 957783 · Report as offensive
PP

Send message
Joined: 3 Jul 99
Posts: 42
Credit: 10,012,664
RAC: 0
Sweden
Message 957790 - Posted: 21 Dec 2009, 14:16:53 UTC - in response to Message 957783.  

Modify work_fetch.cpp and recompile. Remove this line or set it to true:

if (p->too_many_uploading_results) return false;
ID: 957790 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 957793 - Posted: 21 Dec 2009, 14:27:36 UTC - in response to Message 957783.  


My PC run out of work with CPU and GPU tasks simultaneously.
Never only GPU tasks.

You should use the current recommended BOINC V6.10.18 .

If you ran out of CUDA work, you can DL the ReScheduler on the opt. crew site. Which can rename the WU - CPU -> GPU, GPU -> CPU. Then you can split your DLed WU cache.
Somewhere in the forum. Not in the DL area.
[http://lunatics.kwsn.net]




ID: 957793 · Report as offensive
Profile Mamluk
Avatar

Send message
Joined: 10 Sep 09
Posts: 80
Credit: 2,448,048
RAC: 0
South Africa
Message 957799 - Posted: 21 Dec 2009, 14:45:45 UTC - in response to Message 957779.  



The Berkeley crew is in vacation/holidays?


Sleeping I guess..



Then maybe someone of the cleaning crew of the campus came in the lab and on the server power switch?


Guess not as there are other processes on the same physical server that appear to be running.



It would be better if the Berkeley crew would announce this kind of activity..


Yes, for the few that go to the web-site, it would be nice
ID: 957799 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 957815 - Posted: 21 Dec 2009, 15:45:26 UTC - in response to Message 957783.  

Its been disabled for a while. Have plenty of regular workunits to do, but of course am out of CUDA units and none are getting retrieved. With the algotithms for work the way they are, never seem to have enough CUDA to work on. The slightest blip, and my GPU is idle.
Alex


Use the Rescheduler to move stuff from CPU to GPU. I do... It's available on the Lunatics site; version 1.9 is an EXE file, and works on my XP and Vista systems with no problems. You can run it manually, or set it to run every N hours to rebalance.

I rarely get sent any GPU WUs, so I use the Rescheduler to rebalance my workload (which in my case is 40% CPU/60% GPU).
ID: 957815 · Report as offensive
BarryAZ

Send message
Joined: 1 Apr 01
Posts: 2580
Credit: 16,982,517
RAC: 0
United States
Message 957835 - Posted: 21 Dec 2009, 16:31:47 UTC - in response to Message 957799.  

I'm guessing it to be an unplanned outage (perhaps a process crashed), which will be attended to when someone gets onsite this morning. Of course then we will have the standard post outage traffic jam as the backlog of uploads tries to come home.




The Berkeley crew is in vacation/holidays?


Sleeping I guess..



Then maybe someone of the cleaning crew of the campus came in the lab and on the server power switch?


Guess not as there are other processes on the same physical server that appear to be running.



It would be better if the Berkeley crew would announce this kind of activity..


Yes, for the few that go to the web-site, it would be nice


ID: 957835 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 957837 - Posted: 21 Dec 2009, 16:33:53 UTC

It's finally morning, I bet someone is there about to give things a good kick.
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 957837 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 957886 - Posted: 21 Dec 2009, 18:52:58 UTC

Yea! The uploader is online.

Boo! I can't talk to the server.

12/21/2009 1:38:14 PM Resuming network activity
12/21/2009 1:38:14 PM SETI@home Started upload of 19fe07aa.5326.231468.6.10.65_1_0
12/21/2009 1:38:14 PM SETI@home Sending scheduler request: To fetch work.
12/21/2009 1:38:14 PM SETI@home Requesting new tasks
12/21/2009 1:38:29 PM SETI@home Finished upload of 19fe07aa.5326.231468.6.10.65_1_0
12/21/2009 1:38:37 PM Project communication failed: attempting access to reference site
12/21/2009 1:38:40 PM Internet access OK - project servers may be temporarily down.
12/21/2009 1:38:40 PM SETI@home Scheduler request failed: Couldn't connect to server
12/21/2009 1:39:40 PM SETI@home Sending scheduler request: To fetch work.
12/21/2009 1:39:40 PM SETI@home Reporting 1 completed tasks, requesting new tasks
12/21/2009 1:40:02 PM Project communication failed: attempting access to reference site
12/21/2009 1:40:04 PM Internet access OK - project servers may be temporarily down.
12/21/2009 1:40:05 PM SETI@home Scheduler request failed: Couldn't connect to server
12/21/2009 1:41:05 PM SETI@home Sending scheduler request: To fetch work.
12/21/2009 1:41:05 PM SETI@home Reporting 1 completed tasks, requesting new tasks
12/21/2009 1:41:26 PM Project communication failed: attempting access to reference site
12/21/2009 1:41:30 PM Internet access OK - project servers may be temporarily down.
12/21/2009 1:41:30 PM SETI@home Scheduler request failed: Couldn't connect to server
12/21/2009 1:42:08 PM Suspending network activity - user request
12/21/2009 1:44:50 PM Resuming network activity
12/21/2009 1:44:50 PM SETI@home Sending scheduler request: To fetch work.
12/21/2009 1:44:50 PM SETI@home Reporting 1 completed tasks, requesting new tasks
12/21/2009 1:45:12 PM Project communication failed: attempting access to reference site
12/21/2009 1:45:14 PM Internet access OK - project servers may be temporarily down.
12/21/2009 1:45:15 PM SETI@home Scheduler request failed: Couldn't connect to server
12/21/2009 1:46:15 PM SETI@home Sending scheduler request: To fetch work.
12/21/2009 1:46:15 PM SETI@home Reporting 1 completed tasks, requesting new tasks
12/21/2009 1:46:37 PM Project communication failed: attempting access to reference site
12/21/2009 1:46:39 PM Internet access OK - project servers may be temporarily down.
12/21/2009 1:46:40 PM SETI@home Scheduler request failed: Couldn't connect to server


Just my luck.

Actually it doesn't matter. Wingman is yet another of a long line of faulty CUDA rigs or abandoned systems so I'll have to get yet another wingman before I receive any credit.

"Zathras is used to being beast of burden to other people's needs. Very sad life. Probably have very sad death. But, at least there is symmetry"
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 957886 · Report as offensive
Profile Keith

Send message
Joined: 19 May 99
Posts: 483
Credit: 938,268
RAC: 0
United Kingdom
Message 957890 - Posted: 21 Dec 2009, 19:07:52 UTC - in response to Message 957837.  
Last modified: 21 Dec 2009, 19:17:49 UTC

Hiamps

Seems we got the good kick just before 11am (7pm here).
But not quite hard enough -- another good wallop is needed!!!
My 20 completed tasks were uploaded, but none have yet been reported.

Keith

(One more added - 21 to be reported now.
And I may just about keep going with 7 hours more to crunch in my cache!!!)
ID: 957890 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 957893 - Posted: 21 Dec 2009, 19:13:11 UTC

I can confirm this. I got my ~40 tasks to upload with no failures. Took a look at the Cricket graph and noticed a huge spike on the blue line, so I resumed network activity on my two hosts. All went through.

Still waiting for the client to report them and/or request more work, but it looks like the Scheduling server has been disabled now.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 957893 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 957901 - Posted: 21 Dec 2009, 19:58:37 UTC

Panic may be over just reported 429 units...Mostly Cudas and no errors in the bunch.
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 957901 · Report as offensive
Profile Keith

Send message
Joined: 19 May 99
Posts: 483
Credit: 938,268
RAC: 0
United Kingdom
Message 957904 - Posted: 21 Dec 2009, 20:07:08 UTC
Last modified: 21 Dec 2009, 20:16:08 UTC

Yes.

I just had this welcome message:-

Mon 21 Dec 20:05:44 2009 SETI@home Reporting 22 completed tasks, not requesting new tasks


Keith

{But although I know these tasks were on work units with previously completed companion tasks, they have all gone to Pending Credits. So the hold up is not yet completely cleared.)
ID: 957904 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 957908 - Posted: 21 Dec 2009, 20:16:52 UTC
Last modified: 21 Dec 2009, 20:18:15 UTC

I also concur.

2009-12-21 14:29:20|SETI@home|Sending scheduler request: To report completed tasks. Requesting 0 seconds of work, reporting 24 completed tasks
2009-12-21 14:29:40|SETI@home|Scheduler request succeeded: got 0 new tasks


GMT -5
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 957908 · Report as offensive
Profile Keith

Send message
Joined: 19 May 99
Posts: 483
Credit: 938,268
RAC: 0
United Kingdom
Message 957910 - Posted: 21 Dec 2009, 20:24:34 UTC

Yes, but even though the sah validators are apparently working. all those reported tasks (even knowing the work units were completed for two tasks) are all now sitting in Pending Credits

Keith.
ID: 957910 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 957912 - Posted: 21 Dec 2009, 20:30:01 UTC - in response to Message 957910.  

Yes, but even though the sah validators are apparently working. all those reported tasks (even knowing the work units were completed for two tasks) are all now sitting in Pending Credits

Keith.

My RAC just went from 19,000 to 20,000 not to where it was but some are getting thru.
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 957912 · Report as offensive
Profile Keith

Send message
Joined: 19 May 99
Posts: 483
Credit: 938,268
RAC: 0
United Kingdom
Message 957914 - Posted: 21 Dec 2009, 20:40:16 UTC - in response to Message 957912.  
Last modified: 21 Dec 2009, 20:46:22 UTC

Hiamps

Sorry, but I have no reliance on RAC.
It's "formula" has been fiddled about so much that it is meaningless.
I rely on the kinks in my Total Credits line, taking care to avoid Pending Credits.
It is a little tedious, but does give reliable feedback.

The reason for my message is, without exception, my waiting tasks have gone to Pending Credits.
Although sah validators are all working, all work units are sitting there with 2 completed tasks in each!!!

Keith

(Working, that is, according to Server Status)
ID: 957914 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 957916 - Posted: 21 Dec 2009, 20:50:46 UTC - in response to Message 957914.  

Hiamps

Sorry, but I have no reliance on RAC.
It's "formula" has been fiddled about so much that it is meaningless.
I rely on the kinks in my Total Credits line, taking care to avoid Pending Credits.
It is a little tedious, but does give reliable feedback.

The reason for my message is, without exception, my waiting tasks have gone to Pending Credits.
Although sah validators are all working, all work units are sitting there with 2 completed tasks in each!!!

Keith

(Working, that is, according to Server Status)


The only pendings I see on your 3 computers are on 2881519 which just has normal pendings. I guess the others you were talking about have been cleared out by now.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 957916 · Report as offensive
Profile RMcCarthy

Send message
Joined: 1 Feb 08
Posts: 2
Credit: 5,539,477
RAC: 0
Ireland
Message 957920 - Posted: 21 Dec 2009, 20:53:57 UTC

Hi Guys,

Starting having issues last night with Seti. 100% complete, not uploading to site.
Temp failed to upload, http error, upload back off

sorry can't screen grab at moment.

I now have 4 machines with multi WU's waiting to upload.

Anyone any ideas whats going on??

Robbie
ID: 957920 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 957922 - Posted: 21 Dec 2009, 20:56:51 UTC - in response to Message 957920.  

Hi Guys,

Starting having issues last night with Seti. 100% complete, not uploading to site.
Temp failed to upload, http error, upload back off

sorry can't screen grab at moment.

I now have 4 machines with multi WU's waiting to upload.

Anyone any ideas whats going on??

Robbie


the upload server was turned off around noon S@H time and switch back on this morning. Right now there is a flood of everyone's machines trying to catch up.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 957922 · Report as offensive
Profile Keith

Send message
Joined: 19 May 99
Posts: 483
Credit: 938,268
RAC: 0
United Kingdom
Message 957923 - Posted: 21 Dec 2009, 21:04:39 UTC
Last modified: 21 Dec 2009, 21:10:04 UTC

Hiamps

The reason for my message is, without exception, my waiting tasks have gone to Pending Credits.
Although sah validators are all working, all work units are sitting there with 2 completed tasks in each!!!

Keith

(Working, that is, according to Server Status)


The only pendings I see on your 3 computers are on 2881519 which just has normal pendings. I guess the others you were talking about have been cleared out by now.


Okay, they were there and have now cleared (and I agree there are now 3 which were there originally), but no credit has been added to my Total Credits.
I assume this is the delay that the backup server has (from which we get our data to view).
So it should catch up very soon, otherwise it may mean I have lost the best part of a days crunching with 22 results completed.

Keith

(YES. THAT WAS IT.
I DID A PROJECT UPDATE.
THE CREDIT HAS NOW BEEN ADDED, SO ALL IS UP TO DATE NOW.
AND THE STRAIGHT LINE ON MY TOTAL CREDIT IS RESTORED!!!)
ID: 957923 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 10 · Next

Message boards : Number crunching : Panic Mode On (27) Server problems


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.