Panic Mode On (108) Server Problems?

Message boards : Number crunching : Panic Mode On (108) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 25 · 26 · 27 · 28 · 29 · Next

AuthorMessage
Wild6-NJ
Volunteer tester

Send message
Joined: 4 Aug 99
Posts: 43
Credit: 100,336,791
RAC: 140
Message 1905720 - Posted: 8 Dec 2017, 21:07:40 UTC
Last modified: 8 Dec 2017, 21:17:34 UTC

Assimilators not running.
DB purge not running.
AP not getting split.
I have a bad feeling about this.



Edit:
Now the running gbt and pfb splitters are being culled.
ID: 1905720 · Report as offensive
Profile David@home
Volunteer tester
Avatar

Send message
Joined: 16 Jan 03
Posts: 750
Credit: 5,040,916
RAC: 28
United Kingdom
Message 1905729 - Posted: 8 Dec 2017, 21:42:39 UTC

Oh dear, regular "project has no tasks available" messages in the event log, cache below 200, weekend coming up.

Fingers crossed that the RAC will not take another dip, my stats graph is an amazing seesaw shape.
ID: 1905729 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13397
Credit: 208,696,464
RAC: 304
Australia
Message 1905737 - Posted: 8 Dec 2017, 22:35:21 UTC

Yeah, getting work is sporadic, the Assimilators are down, MB results awaiting validation are climbing (about 2 million higher than normal) and the Results & WU purge numbers are still flat-lined.
But at least the splitters picked up their act! (although a couple of PFB and GBT splitters aren't running they're still doing better than they were).
Grant
Darwin NT
ID: 1905737 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1905742 - Posted: 8 Dec 2017, 23:00:35 UTC - in response to Message 1905738.  

And therefore, BETA with tons of AP's, is now a good option.


. . Not if everyone scrambles over there and sucks them all out :(

Stephen

:)
ID: 1905742 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1905747 - Posted: 8 Dec 2017, 23:15:40 UTC - in response to Message 1905744.  

And therefore, BETA with tons of AP's, is now a good option.


. . Not if everyone scrambles over there and sucks them all out :(

Stephen

:)

But that won't happen of course, since so many people are credit wh**** :-)


. . I resemble that! (pun intended)

. . Actually I don't bother with APs these days ...

Stephen

:)
ID: 1905747 · Report as offensive
Profile David@home
Volunteer tester
Avatar

Send message
Joined: 16 Jan 03
Posts: 750
Credit: 5,040,916
RAC: 28
United Kingdom
Message 1905748 - Posted: 8 Dec 2017, 23:17:33 UTC

I assumed SETI Beta results didn't go into the master Database. Beta its for testing so wouldn't those results be invalid for the master DB?
ID: 1905748 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1905764 - Posted: 9 Dec 2017, 0:37:43 UTC - in response to Message 1905729.  

Oh dear, regular "project has no tasks available" messages in the event log, cache below 200, weekend coming up.

Fingers crossed that the RAC will not take another dip, my stats graph is an amazing seesaw shape.


. . Hi David,

. . I got the "no tasks available" message a couple of times but now the GPU queue is quite full, even though they are 90% guppis.

. . Better than nothing :)

Stephen

:)
ID: 1905764 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1905766 - Posted: 9 Dec 2017, 0:40:40 UTC - in response to Message 1905738.  
Last modified: 9 Dec 2017, 0:41:00 UTC

And therefore, BETA with tons of AP's, is now a good option.
Q? Why are there AP tasks on beta? Is a new app out to test? Or GBT tasks? I don't see any mention of it in the beta boards ...
ID: 1905766 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1905774 - Posted: 9 Dec 2017, 1:19:44 UTC

I'm still having troubles with the Linux cruncher. Only get a handful of tasks every 3rd or 4th request and no where near enough to prevent the cache from steadily dropping. Down about 140 tasks now from full.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1905774 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1905775 - Posted: 9 Dec 2017, 1:25:26 UTC - in response to Message 1905774.  

I'm still having troubles with the Linux cruncher. Only get a handful of tasks every 3rd or 4th request and no where near enough to prevent the cache from steadily dropping. Down about 140 tasks now from full.


. . Hi Keith,

. . Sorry to hear that the server issues are still plaguing you ...

. . Hopefully the work will pick up enough to get you through the weekend until the guys are back in at Berkeley on Monday.

Stephen

:(
ID: 1905775 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13397
Credit: 208,696,464
RAC: 304
Australia
Message 1905778 - Posted: 9 Dec 2017, 1:51:05 UTC - in response to Message 1905774.  

I'm still having troubles with the Linux cruncher. Only get a handful of tasks every 3rd or 4th request and no where near enough to prevent the cache from steadily dropping. Down about 140 tasks now from full.

TBar's triple update got my i7 picking up work again after it's cache started to run down.
Grant
Darwin NT
ID: 1905778 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1905788 - Posted: 9 Dec 2017, 2:24:00 UTC - in response to Message 1905778.  

What may I ask is "TBar's Triple Update"? The only trick I know that works most of the time is the "ghost recovery protocol" when the servers aren't responsive. Hasn't worked the last couple of times I've tried today.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1905788 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13397
Credit: 208,696,464
RAC: 304
Australia
Message 1905799 - Posted: 9 Dec 2017, 2:47:59 UTC - in response to Message 1905788.  

What may I ask is "TBar's Triple Update"?

Click on Update, and you'll get "Scheduler request pending, requested by user"
As soon as it changes to "Scheduler request pending, requested by user, request in progress" then click on Update again.
Wait for it to complete and as soon as you get the "Communication deferred 5:02" click on Update again.
Then just let it be and on the next automatic update, work will come down.

For the life of me I've no idea how or why it gets work, but It does. I used to muck around with the application settings and updating to get work- and even when that had no effect, the triple update would.
It'd be nice if the Application preferences issues would be fixed (it's been almost 12 months now), but it looks like there are plenty more serious issues to be resolved first.
Grant
Darwin NT
ID: 1905799 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1905827 - Posted: 9 Dec 2017, 4:45:49 UTC - in response to Message 1905799.  

What may I ask is "TBar's Triple Update"?

Click on Update, and you'll get "Scheduler request pending, requested by user"
As soon as it changes to "Scheduler request pending, requested by user, request in progress" then click on Update again.
Wait for it to complete and as soon as you get the "Communication deferred 5:02" click on Update again.
Then just let it be and on the next automatic update, work will come down.

For the life of me I've no idea how or why it gets work, but It does. I used to muck around with the application settings and updating to get work- and even when that had no effect, the triple update would.
It'd be nice if the Application preferences issues would be fixed (it's been almost 12 months now), but it looks like there are plenty more serious issues to be resolved first.

Thanks for the explanation. Do you have to set NNT before hitting Update?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1905827 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1905832 - Posted: 9 Dec 2017, 5:09:01 UTC

Scheduler is still messed up. Getting this on my most stable and consistent machine.
SETI@home v8 (anonymous platform, CPU)
Number of tasks completed	34861
Max tasks per day	8504
Number of tasks today	46
Consecutive valid tasks	8472
Average processing rate	54.47 GFLOPS
Average turnaround time	1.01 days
SETI@home v8 (anonymous platform, NVIDIA GPU)
Number of tasks completed	288510
Max tasks per day	115489
Number of tasks today	781
Consecutive valid tasks	115457
Average processing rate	349.68 GFLOPS
Average turnaround time	0.39 days

Pipsqueek

1625	SETI@home	12/8/2017 20:57:57	Scheduler request completed: got 0 new tasks	
1626	SETI@home	12/8/2017 20:57:57	No tasks sent	
1627	SETI@home	12/8/2017 20:57:57	This computer has reached a limit on tasks in progress


Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1905832 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13397
Credit: 208,696,464
RAC: 304
Australia
Message 1905836 - Posted: 9 Dec 2017, 5:17:39 UTC - in response to Message 1905827.  

Do you have to set NNT before hitting Update?

Nope, because you want new tasks, so you need to request them.
Grant
Darwin NT
ID: 1905836 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13397
Credit: 208,696,464
RAC: 304
Australia
Message 1905837 - Posted: 9 Dec 2017, 5:20:34 UTC - in response to Message 1905832.  
Last modified: 9 Dec 2017, 5:28:21 UTC

Scheduler is still messed up. Getting this on my most stable and consistent machine.

Does that machine do AP work?
I found with my system that not having AP installed & selected made it harder to get MB work.

Even now, generally when there is plenty of AP, Arecibo & GBT work I don't have an issue getting MB work.
When the AP work stops flowing, or most of the MB work is one type or the other, that's when I have problems getting work again. And it's mostly on my i7, the C2D generally chugs along (although it too occasionally has issues).

EDIT- having said all that, something is screwier than usual with the Scheduler allocating work as I've had to do the triple update 4 times so far today to keep the work coming.
And I wonder how long till we run out of disk space with all the WUs waiting validation & deletion accumulating? At least the rate of return for work has dropped down to 110,000/hr.
Grant
Darwin NT
ID: 1905837 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1905840 - Posted: 9 Dec 2017, 5:29:45 UTC
Last modified: 9 Dec 2017, 5:41:24 UTC

Actually getting the "reached a limit of tasks in progress" across all machines now. Caches falling ....... falllingg ...... fallliiinnnggg .... boom!

I haven't had to resort to toggling off the AP project for a long time now. Guess I try that next.

[Edit] Nope, that didn't work either.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1905840 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13397
Credit: 208,696,464
RAC: 304
Australia
Message 1905845 - Posted: 9 Dec 2017, 5:44:11 UTC - in response to Message 1905840.  
Last modified: 9 Dec 2017, 5:49:34 UTC

I haven't had to resort to toggling off the AP project for a long time now. Guess I try that next.

These days I just leave the "Use only selected..." and "if no work for selected" all on Yes.

EDIT- noticed the cache running down again, triple update & 14WUs on the next automatic request.
The way this issue manifests itself really is rather peculiar.
Grant
Darwin NT
ID: 1905845 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1905851 - Posted: 9 Dec 2017, 6:05:10 UTC - in response to Message 1905827.  

Thanks for the explanation. Do you have to set NNT before hitting Update?
It only takes 2 updates, but it must be 2 that are NOT returning tasks in a row. The third is if you need to report something. It can be a bit tricky at times with fast GPUs to find the right time to make the 'nudge'.

I generally don't have to nudge them often. When I seen your messages here I checked and was down >10% on them all. But they all recovered on their own since, some back down now. I very rarely see them drop below 66% without them recovering on their own.
ID: 1905851 · Report as offensive
Previous · 1 . . . 25 · 26 · 27 · 28 · 29 · Next

Message boards : Number crunching : Panic Mode On (108) Server Problems?


 
©2022 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.