Panic Mode On (84) Server Problems?

Message boards : Number crunching : Panic Mode On (84) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 21 · Next

AuthorMessage
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8240
Credit: 14,654,533
RAC: 20
United States
Message 1373909 - Posted: 31 May 2013, 0:08:51 UTC - in response to Message 1373524.  

Is the reason for no work been split or very low amounts sub 10 per second to do with the rollout of the new application (version 7) or is there something more server related going on?

I suspect something's borked.

Or the server status page is not showing the activity of v7.......

Yes, exactly right, Mark. Same as when Astropulse V6 rolled out. The SSP continued to show only AP v505 until almost all of them were done ans validated, THEM they changed the scripts to track AP v6 generation and validation.

So for the next two-three months, the Sati@Home "Results Ready to Send" and "Creation Rate" numbers will be close to zero and not valid indicators of splitter performance.

I rather thought so this morning after looking at my 2 rigs which have successfully started running v7. No problems maintaining cache. Which did not fit with the figures that server status was showing.

Looks like they went ahead and updated the Server Status Page so the Splitter Status Table reflects V7 production.
Donald
Infernal Optimist / Submariner, retired
ID: 1373909 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1374028 - Posted: 31 May 2013, 6:32:56 UTC - in response to Message 1373863.  
Last modified: 31 May 2013, 6:35:09 UTC

After finally getting all of the v7 code downloaded, the scheduler shot off a big request for 99 new tasks "for CPU and NVIDIA". Unfortunately, the scheduler only loaded one CUDA task (SETI@home v7 v7.00 (cuda42)) and 98 SETI@Home v7 7.00) tasks.

I have three NVIDIA GPUs, yet only one GPU gets a single task (which it crunched in very short order) and the other two are totally latent. After the single CUDA task completed, all three GPUs are now dead in the water-- while the CPUs slowly grind away on 98 tasks.

Doesn't the scheduler appear to be a bit "unbalanced", or is something mis-configured?

Through some reading I've done in the past 24 hours..

There are several builds/types of GPU applications available. The server is going to be very cautious about sending you lots of GPU tasks and will try all the different builds/types on your GPUs until it decides which one is the best, and then you'll start getting loaded up with that one just fine.

You can probably help speed this process along by going into your preferences (links are in posts above/below..depending on your sort order..they were recent) and de-select "Use CPU" for the venue for that host. This will keep it from getting new CPU work and will kind of force you to be GPU-only until it stabilizes. Then you can go back to allowing CPU again and it will all balance itself out.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1374028 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1374049 - Posted: 31 May 2013, 7:04:31 UTC - in response to Message 1373888.  

Long response short, Preferences where Richard

As ever.

http://setiathome.berkeley.edu/home.php (Your account)
http://setiathome.berkeley.edu/prefs.php?subset=project (project preferences)


and you fell into your own trap ...

neither of these will solve the issue of AP on one box only (CPU only) and not on the other two boxes at all. the resolution has to be at per box level which neither of these do.

ID: 1374049 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1374067 - Posted: 31 May 2013, 7:32:50 UTC - in response to Message 1373517.  

And now to something completely different, but rather expected:

Where's my APs? :-)

I do not plan to go to MB v7 in the near future, unless AP dries out completely.


Sten, are your APs flowing through yet ??

ID: 1374067 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1374087 - Posted: 31 May 2013, 7:53:15 UTC - in response to Message 1374067.  

Do v7 GPU work units take longer to process than those that we processed before v7 ??
ID: 1374087 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1374088 - Posted: 31 May 2013, 7:54:33 UTC - in response to Message 1374087.  

Do v7 GPU work units take longer to process than those that we processed before v7 ??

Yes, they do.
More processing.

"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1374088 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14532
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1374103 - Posted: 31 May 2013, 8:31:05 UTC - in response to Message 1374049.  

Long response short, Preferences where Richard

As ever.

http://setiathome.berkeley.edu/home.php (Your account)
http://setiathome.berkeley.edu/prefs.php?subset=project (project preferences)


and you fell into your own trap ...

neither of these will solve the issue of AP on one box only (CPU only) and not on the other two boxes at all. the resolution has to be at per box level which neither of these do.

Yes it will. You have four complete sets of preferences available - default, home, work, school. Use two of them.
ID: 1374103 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1374172 - Posted: 31 May 2013, 10:38:25 UTC - in response to Message 1374088.  

Do v7 GPU work units take longer to process than those that we processed before v7 ??

Yes, they do.
More processing.


As I thought. The question is how much more...

and you beat me to where I was going by the way.

If we are now doing double the processing, then in theory we should be getting double the credit compared to before. This would bring us out at even. What I have noticed is that credit is a lot less per v7 GPU WU...so expect to see RAC plummet.

cheers mate

ps. hard typing over a maine coon lying in front of the keyboard


ID: 1374172 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1374174 - Posted: 31 May 2013, 10:40:27 UTC - in response to Message 1374103.  

Long response short, Preferences where Richard

As ever.

http://setiathome.berkeley.edu/home.php (Your account)
http://setiathome.berkeley.edu/prefs.php?subset=project (project preferences)


and you fell into your own trap ...

neither of these will solve the issue of AP on one box only (CPU only) and not on the other two boxes at all. the resolution has to be at per box level which neither of these do.

Yes it will. You have four complete sets of preferences available - default, home, work, school. Use two of them.


and if you have 4 or more computers ??????

ps. I do, just one of them is not doing any DC at the moment...but can guess where I'm going can't you ...

ID: 1374174 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 17862
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1374178 - Posted: 31 May 2013, 10:49:33 UTC

Having trouble d/loading these two files.

31/05/2013 11:40:43 | SETI@home | Started download of cudart32_50_35.dll
31/05/2013 11:40:43 | SETI@home | Started download of cufft32_50_35.dll
31/05/2013 11:40:44 | SETI@home | Temporarily failed download of cudart32_50_35.dll: can't resolve hostname
31/05/2013 11:40:44 | SETI@home | Backing off 23 min 27 sec on download of cudart32_50_35.dll
31/05/2013 11:40:44 | SETI@home | Temporarily failed download of cufft32_50_35.dll: can't resolve hostname
31/05/2013 11:40:44 | SETI@home | Backing off 9 min 58 sec on download of cufft32_50_35.dll

Is there a solution?
ID: 1374178 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13399
Credit: 208,696,464
RAC: 304
Australia
Message 1374188 - Posted: 31 May 2013, 11:07:31 UTC - in response to Message 1374178.  

Is there a solution?

Keep hammering retry.
It relates to the cache they're using to help reduce the load on the server while the new applications are being downloaded.
Apparently Eric disabled the cache on one of the servers, so if you keep hitting Retry eventually it should change to the uncached server & you'll be able to download the files.
It will be diabled completely in a few days once most people have managed to get the files.

Grant
Darwin NT
ID: 1374188 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1374222 - Posted: 31 May 2013, 12:48:07 UTC - in response to Message 1374178.  

Having trouble d/loading these two files.

31/05/2013 11:40:43 | SETI@home | Started download of cudart32_50_35.dll
31/05/2013 11:40:43 | SETI@home | Started download of cufft32_50_35.dll
31/05/2013 11:40:44 | SETI@home | Temporarily failed download of cudart32_50_35.dll: can't resolve hostname
31/05/2013 11:40:44 | SETI@home | Backing off 23 min 27 sec on download of cudart32_50_35.dll
31/05/2013 11:40:44 | SETI@home | Temporarily failed download of cufft32_50_35.dll: can't resolve hostname
31/05/2013 11:40:44 | SETI@home | Backing off 9 min 58 sec on download of cufft32_50_35.dll

Is there a solution?

Download the x41zc packs from Jason's site, and just drop the Cuda dll's in your project directory, then click retry.

Claggy
ID: 1374222 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 17862
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1374227 - Posted: 31 May 2013, 12:54:18 UTC - in response to Message 1374222.  

Having trouble d/loading these two files.

31/05/2013 11:40:43 | SETI@home | Started download of cudart32_50_35.dll
31/05/2013 11:40:43 | SETI@home | Started download of cufft32_50_35.dll
31/05/2013 11:40:44 | SETI@home | Temporarily failed download of cudart32_50_35.dll: can't resolve hostname
31/05/2013 11:40:44 | SETI@home | Backing off 23 min 27 sec on download of cudart32_50_35.dll
31/05/2013 11:40:44 | SETI@home | Temporarily failed download of cufft32_50_35.dll: can't resolve hostname
31/05/2013 11:40:44 | SETI@home | Backing off 9 min 58 sec on download of cufft32_50_35.dll

Is there a solution?

Download the x41zc packs from Jason's site, and just drop the Cuda dll's in your project directory, then click retry.

Claggy

Thanks, but got them from the beta site, now I can d/load tasks.
ID: 1374227 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1374293 - Posted: 31 May 2013, 14:21:10 UTC - in response to Message 1374285.  

Do v7 GPU work units take longer to process than those that we processed before v7 ??

Yes, they do.
More processing.


As I thought. The question is how much more...

and you beat me to where I was going by the way.

If we are now doing double the processing, then in theory we should be getting double the credit compared to before. This would bring us out at even. What I have noticed is that credit is a lot less per v7 GPU WU...so expect to see RAC plummet.

cheers mate

ps. hard typing over a maine coon lying in front of the keyboard




Maine Coons like doing that......
And many other thingys as well.
Very loving cats. Very intelligent cats.

They often find themselves trying to overcome our human idiocies all by themselves. I have one mixed breed almost 18yo, Lori has two that are purebreads, and they are both now over 2yo and over 20 pounds each. Both will look at you with such big eyes and such understanding that you cannot help but wonder what is in their loving looks.

"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1374293 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1374350 - Posted: 31 May 2013, 15:28:12 UTC - in response to Message 1374307.  

And now to something completely different, but rather expected:

Where's my APs? :-)

I do not plan to go to MB v7 in the near future, unless AP dries out completely.


Sten, are your APs flowing through yet ??


Oh yeah, I'm getting all I wan't, that is up to the 100 limit. I have nothing to complain about, other than that many of them are high blanked.

So you ARE somehow complaining, eh?

Life in the Seti lane.

"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1374350 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11172
Credit: 29,581,041
RAC: 66
United States
Message 1374375 - Posted: 31 May 2013, 16:01:10 UTC - in response to Message 1374351.  

That is why Norway built a new national zoo, they put a fence around Sweden.
ID: 1374375 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1374377 - Posted: 31 May 2013, 16:02:47 UTC - in response to Message 1374375.  
Last modified: 31 May 2013, 16:04:09 UTC

That is why Norway built a new national zoo, they put a fence around Sweden.


To contain the Seti crunchers??
Really?
Or just to restrain the APs from Sten?
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1374377 · Report as offensive
WezH
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 576
Credit: 67,033,957
RAC: 95
Finland
Message 1374427 - Posted: 31 May 2013, 17:12:09 UTC - in response to Message 1374416.  

That is why Norway built a new national zoo, they put a fence around Sweden.


Must be something funny somewhere in that sentence. I just can't find it....


But I did :)

Greetings from Eastern neighbor country :D
"Please keep Your signature under four lines so Internet traffic doesn't go up too much"

- In 1992 when I had my first e-mail address -
ID: 1374427 · Report as offensive
Kevin Benfield

Send message
Joined: 29 Dec 03
Posts: 39
Credit: 30,085,439
RAC: 0
United Kingdom
Message 1374487 - Posted: 31 May 2013, 19:18:53 UTC

Not been able to download any work units, ran out first thing this morning, project looks to have units available, nothing changed at my end, anyone any ideas?
ID: 1374487 · Report as offensive
Profile Michel448a
Volunteer tester
Avatar

Send message
Joined: 27 Oct 00
Posts: 1338
Credit: 2,970,814
RAC: 0
Canada
Message 1374495 - Posted: 31 May 2013, 19:27:39 UTC - in response to Message 1374487.  
Last modified: 31 May 2013, 19:29:04 UTC

i dunno i m still crunching the 99 WUs i ve got from yesterday (in 1 shot lol)

btw edit:

the 1st one was written cuda50, and all the 99 i ve got then after were cuda32... whats the difference, and why i didnt got all cuda50 ?
ID: 1374495 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 21 · Next

Message boards : Number crunching : Panic Mode On (84) Server Problems?


 
©2022 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.