Panic Mode On (84) Server Problems?

Message boards : Number crunching : Panic Mode On (84) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · 12 . . . 21 · Next

AuthorMessage
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1378138 - Posted: 7 Jun 2013, 16:51:49 UTC - in response to Message 1378130.  
Last modified: 7 Jun 2013, 16:54:22 UTC

Totaly out of MB work and no one left on cache, my RAC now will fall as an asteroid!

Panic Mode ON...

I belive is better to go to buy some more beer, this will be a long weekend...

You do have AP on GPU enabled, no?
I now have 943 of them....no rest for my GPUs.

I just see and 3 AP WU are DL now nothing before but at least i have few work now.

<edit> But just on one of my 3 fastest host, the other 2 are running empty no MB or AP Cuda Work.
ID: 1378138 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1378142 - Posted: 7 Jun 2013, 16:59:53 UTC - in response to Message 1378138.  
Last modified: 7 Jun 2013, 17:00:49 UTC

Totaly out of MB work and no one left on cache, my RAC now will fall as an asteroid!

Panic Mode ON...

I belive is better to go to buy some more beer, this will be a long weekend...

You do have AP on GPU enabled, no?
I now have 943 of them....no rest for my GPUs.

I just see and 3 AP WU are DL now nothing before but at least i have few work now.

<edit> But just on one of my 3 fastest host, the other 2 are running empty no MB or AP Cuda Work.

Dunno why you would not get more AP Cuda work....
Right now, I have 1,393 WUs in cache...946 of them are AP.
I know what the kitties are going to be working on for a while!!
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1378142 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1378149 - Posted: 7 Jun 2013, 17:07:46 UTC - in response to Message 1378142.  

Totaly out of MB work and no one left on cache, my RAC now will fall as an asteroid!

Panic Mode ON...

I belive is better to go to buy some more beer, this will be a long weekend...

You do have AP on GPU enabled, no?
I now have 943 of them....no rest for my GPUs.

I just see and 3 AP WU are DL now nothing before but at least i have few work now.

<edit> But just on one of my 3 fastest host, the other 2 are running empty no MB or AP Cuda Work.

Dunno why you would not get more AP Cuda work....
Right now, I have 1,393 WUs in cache...946 of them are AP.
I know what the kitties are going to be working on for a while!!

So the kitties must be very happy my friend, on my side still empty (the 3 hosts are inclusive in the same network, but la will leave that way, less heat at less, i realy don´t like to crunch AP. So let just wait (i have a lot of beers to help)

ID: 1378149 · Report as offensive
Profile Vipin Palazhi
Avatar

Send message
Joined: 29 Feb 08
Posts: 286
Credit: 167,386,578
RAC: 0
India
Message 1378190 - Posted: 7 Jun 2013, 18:14:30 UTC - in response to Message 1378149.  

Few of my crunchers have fallen back on Einstein. And looks as though the beer companies in Juan's neighborhood would be a happy lot.
______________

ID: 1378190 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22508
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1378210 - Posted: 7 Jun 2013, 18:39:29 UTC

I just hope they stuff some more tapes in the slot soon...
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1378210 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1378211 - Posted: 7 Jun 2013, 18:45:45 UTC - in response to Message 1378130.  

Totaly out of MB work and no one left on cache, my RAC now will fall as an asteroid!

Panic Mode ON...

I belive is better to go to buy some more beer, this will be a long weekend...

You do have AP on GPU enabled, no?
I now have 943 of them....no rest for my GPUs.


You might want to try putting some parameters in your ap_cmdline_win_x86_SSE2_OpenCL_NV.txt file, the following is a good start:

-unroll 10 -ffa_block 6144 -ffa_block_fetch 1536

Claggy
ID: 1378211 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1378212 - Posted: 7 Jun 2013, 18:46:15 UTC - in response to Message 1378210.  

I just hope they stuff some more tapes in the slot soon...

One just loaded.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1378212 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378266 - Posted: 7 Jun 2013, 20:25:59 UTC - in response to Message 1378212.  

I just hope they stuff some more tapes in the slot soon...

One just loaded.

Unfortunately nothing is being split.
Current result creation rate is 0.2658 (ie 0), and has been for several hours. Almost all of my requsts for work are for the last 5+ hours are resulting in "project has no tasks avalable" messages, with just the odd one or two getting work- often only 1 WU.
I think something's jammed up somewhere.
Grant
Darwin NT
ID: 1378266 · Report as offensive
Oddbjornik Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 220
Credit: 349,610,548
RAC: 1,728
Norway
Message 1378267 - Posted: 7 Jun 2013, 20:26:23 UTC - in response to Message 1378211.  
Last modified: 7 Jun 2013, 20:26:55 UTC


You might want to try putting some parameters in your ap_cmdline_win_x86_SSE2_OpenCL_NV.txt file, the following is a good start:

-unroll 10 -ffa_block 6144 -ffa_block_fetch 1536

Claggy


I've got a GTX 680 and a Quadro K2000M, each of which has loads and loads of memory.

I run two tasks simultaneously on both.

Is there any way I can put all that memory to use? I've set
-unroll 12 -ffa_block 8192 -ffa_block_fetch 4096
on both, as recommended in the readme.

Is there anything to gain from increasing those values further?
ID: 1378267 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1378284 - Posted: 7 Jun 2013, 21:31:46 UTC - in response to Message 1378212.  
Last modified: 7 Jun 2013, 21:35:57 UTC

I just hope they stuff some more tapes in the slot soon...

One just loaded.

Still no new AP or MB work, now all my 3 fastest hosts are running empty of GPU work. The good thing is my power drain drops more than 1.5 kw at least. So i could save some money to buy more beers.
ID: 1378284 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378285 - Posted: 7 Jun 2013, 21:31:52 UTC - in response to Message 1378266.  

Unfortunately nothing is being split.
Current result creation rate is 0.2658 (ie 0), and has been for several hours. Almost all of my requsts for work are for the last 5+ hours are resulting in "project has no tasks avalable" messages, with just the odd one or two getting work- often only 1 WU.
I think something's jammed up somewhere.


One good thing about the extra run time is that the limited cache lasts longer.
Even with that, at the present rate i'll be out of work before nightfall if the splitters don't start splitting again.
Grant
Darwin NT
ID: 1378285 · Report as offensive
Sir Mick
Volunteer tester

Send message
Joined: 18 Mar 13
Posts: 38
Credit: 106,756,204
RAC: 0
United States
Message 1378291 - Posted: 7 Jun 2013, 22:00:27 UTC

No work since this morning, any problems?
ID: 1378291 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378293 - Posted: 7 Jun 2013, 22:02:51 UTC - in response to Message 1378291.  

No work since this morning, any problems?

The splitters aren't splitting.
Grant
Darwin NT
ID: 1378293 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378311 - Posted: 7 Jun 2013, 22:41:37 UTC - in response to Message 1378285.  

One good thing about the extra run time is that the limited cache lasts longer.
Even with that, at the present rate i'll be out of work before nightfall if the splitters don't start splitting again.

Looks like i'll be out of GPU work by lunch time.
Won't help the plummeting RAC at all.
Grant
Darwin NT
ID: 1378311 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 36635
Credit: 261,360,520
RAC: 489
Australia
Message 1378314 - Posted: 7 Jun 2013, 22:50:56 UTC - in response to Message 1378311.  

One good thing about the extra run time is that the limited cache lasts longer.
Even with that, at the present rate i'll be out of work before nightfall if the splitters don't start splitting again.

Looks like i'll be out of GPU work by lunch time.
Won't help the plummeting RAC at all.

My 660's have been out of SETI work for 3hrs now and my 550Ti's just switched over to their backup project. :-(

Cheers.
ID: 1378314 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378325 - Posted: 7 Jun 2013, 23:00:41 UTC - in response to Message 1378314.  
Last modified: 7 Jun 2013, 23:00:56 UTC

If someone in the loop could let the staff know there's a problem with the splitters before they knock off for the weekend it would be appreciated.
Grant
Darwin NT
ID: 1378325 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378357 - Posted: 8 Jun 2013, 0:09:50 UTC - in response to Message 1378325.  
Last modified: 8 Jun 2013, 0:43:19 UTC

Just about out of GPU work. Didn't even make to close to lunch.

The splitters certainly are having issues, on the Server Status page sometimes all but one shows as green, then it's mostly red. Even when they are running 10/s is the most i've seen in the last 24hours or so. Even with the reduced load of v7 20/s seems to be the minimum to meet demand & still build up (although very slowly) a ready-to-send buffer.


Ever since the MB splitters were converted to PFB their output has been seriously limited. With several of them now repeatedly failing that limited output is making things even worse.



EDIT- i should have posted earlier.
The *very* second i hit OK for this post, i got 59 WUs for one of my systems GPUs. Another couple of hundred WUs all round & i should be good till tomorrow.
Grant
Darwin NT
ID: 1378357 · Report as offensive
spitfire_mk_2
Avatar

Send message
Joined: 14 Apr 00
Posts: 563
Credit: 27,306,885
RAC: 0
United States
Message 1378358 - Posted: 8 Jun 2013, 0:14:45 UTC

I did an upgrade to BOINC 7.0.64 earlier today. Imagine my surprise when I could not get any work afterward. However, I did got 4 AP tasks a few hours later.

(there is always Einstein work if you feel neglected by seti)
ID: 1378358 · Report as offensive
tbret
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 3380
Credit: 296,162,071
RAC: 40
United States
Message 1378816 - Posted: 9 Jun 2013, 6:25:01 UTC - in response to Message 1378358.  

I did an upgrade to BOINC 7.0.64 earlier today. Imagine my surprise when I could not get any work afterward. However, I did got 4 AP tasks a few hours later.

(there is always Einstein work if you feel neglected by seti)


I haven't read through this whole thread carefully, so ignore me if I say something stupid. I'd better amend that: Ignore me if I say something even more stupid than you would expect.

You did go to Preferences and enable v7 work, didn't you?
ID: 1378816 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1378820 - Posted: 9 Jun 2013, 6:34:16 UTC - in response to Message 1378358.  

I did an upgrade to BOINC 7.0.64 earlier today. Imagine my surprise when I could not get any work afterward. However, I did got 4 AP tasks a few hours later.

If you upgraded from BOINC v6 you need to swap the values around in the cache settings. They changed in v7.

Grant
Darwin NT
ID: 1378820 · Report as offensive
Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · 12 . . . 21 · Next

Message boards : Number crunching : Panic Mode On (84) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.