Panic Mode On (113) Server Problems?

Message boards : Number crunching : Panic Mode On (113) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 26 · 27 · 28 · 29 · 30 · 31 · 32 . . . 37 · Next

AuthorMessage
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1859
Credit: 268,616,081
RAC: 1,349
United States
Message 1962606 - Posted: 31 Oct 2018, 5:47:58 UTC

I just got a dozen or so, 80% were noisy, 14 sec runtime.
ID: 1962606 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962610 - Posted: 31 Oct 2018, 6:00:50 UTC
Last modified: 31 Oct 2018, 6:03:47 UTC

Reporting work OK, but not able to get any so far.
Scheduler requests presently taking 20-30 secs instead of the usual 3sec.

And the forums are a bit sluggish at the moment.
Grant
Darwin NT
ID: 1962610 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1962613 - Posted: 31 Oct 2018, 6:06:21 UTC - in response to Message 1962606.  

I just got a dozen or so, 80% were noisy, 14 sec runtime.


. . I'm getting those HTTP errors and "Internet OK but servers may be down" messages. NO work for me :(

Stephen

:(
ID: 1962613 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962615 - Posted: 31 Oct 2018, 6:08:27 UTC - in response to Message 1962613.  

. . I'm getting those HTTP errors and "Internet OK but servers may be down" messages. NO work for me :(

Just got a couple of Scheduler errors, then able to contact it OK again on the next request.
Still no work though.
Grant
Darwin NT
ID: 1962615 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962620 - Posted: 31 Oct 2018, 6:17:10 UTC

A lot of the server processes info haven't started being updated either.
Grant
Darwin NT
ID: 1962620 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962623 - Posted: 31 Oct 2018, 6:34:23 UTC - in response to Message 1962620.  

A lot of the server processes info haven't started being updated either.

Looks like they're updating again, and for all the WUs supposedly ready-to-send, the Scheduler isn't giving any of them away.
Grant
Darwin NT
ID: 1962623 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962625 - Posted: 31 Oct 2018, 6:48:20 UTC
Last modified: 31 Oct 2018, 6:55:46 UTC

Well, the system that still has some work managed to pick up 50 new WUs.
The one that is out of work managed to pick 1 single solitary WU, that took 10 seconds to process.
Grant
Darwin NT
ID: 1962625 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 1962626 - Posted: 31 Oct 2018, 6:57:06 UTC - in response to Message 1962625.  

Well, the system that still has some work managed to pick up 50 new WUs.
The one that is out of work managed to pick 1 single solitary WU, that took 10 seconds to process.


I haven't managed to pickup anything on one system :(

Tom
A proud member of the OFA (Old Farts Association).
ID: 1962626 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1962627 - Posted: 31 Oct 2018, 7:05:41 UTC - in response to Message 1962615.  

. . I'm getting those HTTP errors and "Internet OK but servers may be down" messages. NO work for me :(

Just got a couple of Scheduler errors, then able to contact it OK again on the next request.
Still no work though.


. . I'm now getting some small amounts of work for the GPUs, but not seeing any for the CPUs.

Stephen

<shrug>
ID: 1962627 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1962628 - Posted: 31 Oct 2018, 7:07:28 UTC - in response to Message 1962625.  

Well, the system that still has some work managed to pick up 50 new WUs.
The one that is out of work managed to pick 1 single solitary WU, that took 10 seconds to process.


. . Saw that same thing on my most powerful unit, except I was lucky and it was NOT a noise bomb, it took 3 minutes to finsih :)

Stephen
ID: 1962628 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962629 - Posted: 31 Oct 2018, 7:14:04 UTC - in response to Message 1962627.  
Last modified: 31 Oct 2018, 7:51:43 UTC

. . I'm now getting some small amounts of work for the GPUs, but not seeing any for the CPUs.

Don't expect anything for the CPU till the GPUs have reached (or are very close to) the server side limits; although surprisingly I've managed to get some for my CPU in this most recent batch of work.

Just managed to pick up some work on the dry system- and it took over a minute for the downloads to finally start happening (and 4 of them so far have been noise bombs).
This is going to be a very messy recovery.
Grant
Darwin NT
ID: 1962629 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962632 - Posted: 31 Oct 2018, 7:51:25 UTC
Last modified: 31 Oct 2018, 8:30:04 UTC

I hope the splitters can get their act together soon, otherwise there won't be much work left when the Scheduler finally starts handing out work regularly.
Grant
Darwin NT
ID: 1962632 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1859
Credit: 268,616,081
RAC: 1,349
United States
Message 1962633 - Posted: 31 Oct 2018, 8:16:39 UTC

Doing much better.
2 of 4 caches full, and enough to get by on for the others ...
Last Einstein task crunching now ...
ID: 1962633 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1962634 - Posted: 31 Oct 2018, 8:33:53 UTC

A storm passed over here during the night and power went out, so still have some CPU work left this morning.
Otherwise "No tasks available".
Humans may rule the world...but bacteria run it...
ID: 1962634 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962636 - Posted: 31 Oct 2018, 9:04:59 UTC - in response to Message 1962634.  
Last modified: 31 Oct 2018, 9:07:45 UTC

A storm passed over here during the night and power went out, so still have some CPU work left this morning.
Otherwise "No tasks available".

And with the Ready-to-send buffer empty, and splitter output at 18/s, it's going to be that way for a while.

Edit- looks like the splitters have woken up. Now cranking out 80/s. As long as they can keep at 55 or better, then things will improve, eventually.
Grant
Darwin NT
ID: 1962636 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1962637 - Posted: 31 Oct 2018, 9:12:09 UTC - in response to Message 1962636.  


And with the Ready-to-send buffer empty, and splitter output at 18/s, it's going to be that way for a while.

Edit- looks like the splitters have woken up. Now cranking out 80/s. As long as they can keep at 55 or better, then things will improve, eventually.

Well something did wake up...suddenly my cache filled up, and I'm back to the steady hum of my GPU fans ;-)
Humans may rule the world...but bacteria run it...
ID: 1962637 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962639 - Posted: 31 Oct 2018, 9:27:05 UTC
Last modified: 31 Oct 2018, 9:28:53 UTC

And one of the splitters has started on a BLC01 file, so hopefully the number of noise bombs will start to decline as the BLC22 & BLC23 files are finally finished off, and the ther servers can clear the Validation/Assimilation backlog (now at 7.2/1.4 million).
Grant
Darwin NT
ID: 1962639 · Report as offensive
Kevin Olley

Send message
Joined: 3 Aug 99
Posts: 906
Credit: 261,085,289
RAC: 572
United Kingdom
Message 1962669 - Posted: 31 Oct 2018, 14:29:43 UTC - in response to Message 1962639.  

And one of the splitters has started on a BLC01 file, so hopefully the number of noise bombs will start to decline as the BLC22 & BLC23 files are finally finished off, and the ther servers can clear the Validation/Assimilation backlog (now at 7.2/1.4 million).


Just done a couple of them, they are fast, 55 seconds each compared with 90 seconds for normal BLC22 & BLC23 WU's.
Kevin


ID: 1962669 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1962750 - Posted: 31 Oct 2018, 22:29:29 UTC - in response to Message 1962669.  

And one of the splitters has started on a BLC01 file, so hopefully the number of noise bombs will start to decline as the BLC22 & BLC23 files are finally finished off, and the ther servers can clear the Validation/Assimilation backlog (now at 7.2/1.4 million).


Just done a couple of them, they are fast, 55 seconds each compared with 90 seconds for normal BLC22 & BLC23 WU's.


. . They are the "new" GBT format which first appeared in a Blc04 data series 12 months ago and became the norm by about Christmas last year or January this year. The Blc22/23 series we have been wading through, noise bombs and all, are the "old" format from before that. At least, they conform to the run times that identifies each of what I call format (for want of a better term).

. . I like the new format much better :)

Stephen

:)
ID: 1962750 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 1962791 - Posted: 1 Nov 2018, 5:21:17 UTC

I don't want to tempt fate, but I have to say i'm impressed with the servers at the moment. The recovery from the weekly outage wasn't all that great, but now they have recovered they're holding up well.
There's been a sustained return rate of 130k, and the splitters have still been able to meet the demand, and build up a Ready-to-send buffer as well. On top of that effort, they've also been able to put a dent in the Validation & Assimilation backlogs.
Grant
Darwin NT
ID: 1962791 · Report as offensive
Previous · 1 . . . 26 · 27 · 28 · 29 · 30 · 31 · 32 . . . 37 · Next

Message boards : Number crunching : Panic Mode On (113) Server Problems?


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.