Panic Mode On (106) Server Problems?

Message boards : Number crunching : Panic Mode On (106) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 29 · Next

AuthorMessage
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873310 - Posted: 16 Jun 2017, 10:43:26 UTC - in response to Message 1873306.  

I think they are messing with the scheduler, InProgess is now dropping off.
With
Astro -yes
Seti - yes
Other - no

2 boxes are acting like "other" is stuck on.
- No tasks sent
- No tasks available for AP v7
ID: 1873310 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873312 - Posted: 16 Jun 2017, 10:49:41 UTC - in response to Message 1873306.  

And now thy flood gates have opened .... for now at least. 2 boxes just got big lumps in.
ID: 1873312 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34973
Credit: 261,360,520
RAC: 489
Australia
Message 1873314 - Posted: 16 Jun 2017, 11:06:56 UTC

No probs at all here, but then I use an old school version of BOINC. :-)

Cheers.
ID: 1873314 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873318 - Posted: 16 Jun 2017, 11:17:08 UTC - in response to Message 1873310.  

And it's back "No tasks available for AP v7 "
GGGRRRRR, I'm not set for AP only!
ID: 1873318 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13755
Credit: 208,696,464
RAC: 304
Australia
Message 1873321 - Posted: 16 Jun 2017, 11:22:35 UTC - in response to Message 1873314.  

No probs at all here, but then I use an old school version of BOINC. :-)

My C2D has the same version of BOINC as the i7, and continues to get work.
It really is rather perplexing.
Grant
Darwin NT
ID: 1873321 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34973
Credit: 261,360,520
RAC: 489
Australia
Message 1873334 - Posted: 16 Jun 2017, 12:46:06 UTC - in response to Message 1873321.  

No probs at all here, but then I use an old school version of BOINC. :-)

My C2D has the same version of BOINC as the i7, and continues to get work.
It really is rather perplexing.

Yet none of my 3 rigs have ever shown signs of that behaviour.

It's certainly seems to be a very perplexing problem for those with later BOINC versions at least (a code problem maybe?).

I wonder if it can be established what versions are being effected by that problem.

Cheers.
ID: 1873334 · Report as offensive
Keith J. LaGue
Volunteer tester
Avatar

Send message
Joined: 17 May 99
Posts: 59
Credit: 40,441,387
RAC: 0
United States
Message 1873386 - Posted: 16 Jun 2017, 16:29:43 UTC

Here's an old time favorite...

With only 28 GPU tasks onboard I now get...

"16 Jun 17 12:17:29 | SETI@home | This computer has reached a limit on tasks in progress"

Takes about nine hours for my rig to run 28 tasks...
ID: 1873386 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1873408 - Posted: 16 Jun 2017, 18:00:31 UTC

Having AstroPulse v7 7.0 in the CPU queue seems to limit the cache size considerably here.
Humans may rule the world...but bacteria run it...
ID: 1873408 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1873426 - Posted: 16 Jun 2017, 21:20:37 UTC

I'm having trouble getting work on my Windows 7 machines. Down below 100 tasks when it should be 200. Windows 10 machine seems to be staying up. From my daily driver Windows7 log.

Keith-Windows7

4828 SETI@home 6/16/2017 14:16:00 Sending scheduler request: To report completed tasks.
4829 SETI@home 6/16/2017 14:16:00 Reporting 4 completed tasks
4830 SETI@home 6/16/2017 14:16:00 Requesting new tasks for CPU and NVIDIA GPU
4831 SETI@home 6/16/2017 14:16:10 Scheduler request completed: got 0 new tasks
4832 SETI@home 6/16/2017 14:16:10 No tasks sent
4833 SETI@home 6/16/2017 14:16:10 No tasks are available for AstroPulse v7
4834 SETI@home 6/16/2017 14:16:10 This computer has reached a limit on tasks in progress
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1873426 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13755
Credit: 208,696,464
RAC: 304
Australia
Message 1873442 - Posted: 16 Jun 2017, 22:24:52 UTC - in response to Message 1873334.  

I wonder if it can be established what versions are being effected by that problem.

By running AP i'm not affected anywhere nearly as much as when I didn't.
Whatever it is, it all comes back to the late Dec borking of the application preferences after one of the weekly outages & server/Scheduler updates.
Grant
Darwin NT
ID: 1873442 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1873448 - Posted: 16 Jun 2017, 22:45:42 UTC - in response to Message 1873442.  

I wonder if it can be established what versions are being effected by that problem.

By running AP i'm not affected anywhere nearly as much as when I didn't.
Whatever it is, it all comes back to the late Dec borking of the application preferences after one of the weekly outages & server/Scheduler updates.

I never took Wiggo up on his offer to backlevel to a much earlier client to see if that fixed the problem. Every time it gets really bad and I've about had with it to actually pull the trigger on installing an 6.XX client, the problem disappears for a while. We'll see if this current episode finally gets me off my duff to try it. Toggling preferences in all permutations hasn't done anything for me.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1873448 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873450 - Posted: 16 Jun 2017, 23:13:42 UTC - in response to Message 1873386.  

Here's an old time favorite...

With only 28 GPU tasks onboard I now get...

"16 Jun 17 12:17:29 | SETI@home | This computer has reached a limit on tasks in progress"

Takes about nine hours for my rig to run 28 tasks...


. . Yep, I have been getting that for the last 12 hours or more. Only a partial cache but the servers say I am on the limit ... :(

Stephen

??
ID: 1873450 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873460 - Posted: 17 Jun 2017, 0:00:39 UTC
Last modified: 17 Jun 2017, 0:04:44 UTC

Running on fumes, I have only received 2 GPU tasks in the last 3 hours on my i7.

EDIT: just got 7, woohoo that will last 5 minutes more.
ID: 1873460 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873463 - Posted: 17 Jun 2017, 0:12:00 UTC - in response to Message 1873460.  

Running on fumes, I have only received 2 GPU tasks in the last 3 hours on my i7.

EDIT: just got 7, woohoo that will last 5 minutes more.


. . I did an experiment. I had a full CPU queue and an almost empty GPU queue. So I moved 10 WUs to the GPU q to make space and waited for the refresh. The CPU Q was topped u with Arecibo VLARs but nada, zip, nothing for the GPU. So I went to preferences and turned off Nvidia processing and did a refresh, waited a couple of minutes then went back in and turned Nvidia back on did another refresh. Next work request aI got a full load and GPU Q now full again. It may be coincidence but it may be worth a try.

Stephen

. . It seemed to behave like it had lost the "setting" to send me nvidia work :(

.
ID: 1873463 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873465 - Posted: 17 Jun 2017, 0:20:55 UTC - in response to Message 1873463.  
Last modified: 17 Jun 2017, 0:39:44 UTC

There may be something to that. I turned off NVidia and got 190 tasks. I know the setting doesn't actually appy until the NEXT request, but it may have been enough just to have the server access that table in prefs. will see what happens the next few requests ....

EDIT: ... Next request was for CPU only (as expected)
Set NVidia 'on', request was for CPU only (as expected)
Then request for NVidia/CPU tasks and 93 came in, cache full.

It also filled my i7-960 cache too (on the same profile)
So toggling NVidia IS an option to kick the server in the knackers.
I would guess toggling CPU would do the same since it is likely the same table to update.
ID: 1873465 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873475 - Posted: 17 Jun 2017, 0:53:16 UTC - in response to Message 1873465.  

There may be something to that. I turned off NVidia and got 190 tasks. I know the setting doesn't actually appy until the NEXT request, but it may have been enough just to have the server access that table in prefs. will see what happens the next few requests ....

EDIT: ... Next request was for CPU only (as expected)
Set NVidia 'on', request was for CPU only (as expected)
Then request for NVidia/CPU tasks and 93 came in, cache full.

It also filled my i7-960 cache too (on the same profile)
So toggling NVidia IS an option to kick the server in the knackers.
I would guess toggling CPU would do the same since it is likely the same table to update.


. . That was my experience as well. My 2 windows boxes are both on "Home" setting. When I did that from one box they both got full caches from it.

Stephen
ID: 1873475 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13755
Credit: 208,696,464
RAC: 304
Australia
Message 1873480 - Posted: 17 Jun 2017, 1:00:51 UTC - in response to Message 1873465.  

There may be something to that. I turned off NVidia and got 190 tasks. I know the setting doesn't actually appy until the NEXT request, but it may have been enough just to have the server access that table in prefs. will see what happens the next few requests ....

EDIT: ... Next request was for CPU only (as expected)
Set NVidia 'on', request was for CPU only (as expected)
Then request for NVidia/CPU tasks and 93 came in, cache full.

It also filled my i7-960 cache too (on the same profile)
So toggling NVidia IS an option to kick the server in the knackers.
I would guess toggling CPU would do the same since it is likely the same table to update.

My system has been keeping the cache full for that last hour or so.
Will give that option a go the next time it goes stupid on me.
Grant
Darwin NT
ID: 1873480 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1873506 - Posted: 17 Jun 2017, 3:49:45 UTC

I too tried the Nvidia toggle Stephen and it caused my lagging Windows7 crunchers to fill their caches. But my Windows10 cruncher is now lagging. I didn't perform a clean experiment though because I had also set work_fetch_debug in the log to try and and get the system to recognize the deficit on the GPU's. I am doing some memory tuning on the Ryzen system now so it is offline for the moment but I had set the work_fetch_debug in its log for a few cycles before I shut it down. We shall see if if picks itself back up after I bring it back online.

Thanks for tip and heads up. Definitely goes into the toolbag of tricks.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1873506 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1873515 - Posted: 17 Jun 2017, 4:31:38 UTC

My Windows10 Ryzen system immediately began picking up batches of GPU work as soon as I brought it back online. It has a full cache now along with the Windows7 systems. Further circumstantial evidence I think that goes back to the change for the AMD 8.23 app FUBAR in the schedulers. I think it messed with the Nvidia platforms too.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1873515 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13755
Credit: 208,696,464
RAC: 304
Australia
Message 1873522 - Posted: 17 Jun 2017, 4:57:46 UTC - in response to Message 1873515.  

My Windows10 Ryzen system immediately began picking up batches of GPU work as soon as I brought it back online. It has a full cache now along with the Windows7 systems. Further circumstantial evidence I think that goes back to the change for the AMD 8.23 app FUBAR in the schedulers. I think it messed with the Nvidia platforms too.

I seems to tie in with the work type as well- GBT v Arecibo.
It used to be you would get a fairly even split between then (1/3 Arecibo & 2/3 GBT when they had most of the Arecibo splitters disabled) these days I seem to get all Arecibo or all GBT a lot of the time, particularly when getting work is an issue and it finally arrives. Then at other times when the scheduler is behaving I usually get more of a mix of both.
Grant
Darwin NT
ID: 1873522 · Report as offensive
Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 29 · Next

Message boards : Number crunching : Panic Mode On (106) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.