The Server Issues / Outages Thread - Panic Mode On! (118)

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 15 · 16 · 17 · 18 · 19 · 20 · 21 . . . 94 · Next

AuthorMessage
6BQ5

Send message
Joined: 7 Dec 18
Posts: 29
Credit: 12,725,636
RAC: 357
United States
Message 2026045 - Posted: 3 Jan 2020, 0:43:01 UTC

I walked past my computer and heard it running dead quiet. At first I thought it crashed (again) due to overheating. It looks like there have been issues with the SETI servers feeding us work tasks. :(

I don't have any CPU tasks. Only my NVIDIA 1080 is getting some tasks to crunch on.

Hopefully the system comes back up soon! :)

-=- Boris
ID: 2026045 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2026048 - Posted: 3 Jan 2020, 1:06:01 UTC

nothing but http internal server error here.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2026048 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2026054 - Posted: 3 Jan 2020, 1:29:59 UTC

nothing but http internal server error here.

Ditto
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2026054 · Report as offensive
Niteryder
Volunteer tester

Send message
Joined: 1 Mar 99
Posts: 64
Credit: 22,663,988
RAC: 18
United States
Message 2026055 - Posted: 3 Jan 2020, 1:31:01 UTC

+1
ID: 2026055 · Report as offensive
Profile Oz
Avatar

Send message
Joined: 6 Jun 99
Posts: 233
Credit: 200,655,462
RAC: 212
United States
Message 2026062 - Posted: 3 Jan 2020, 2:03:55 UTC
Last modified: 3 Jan 2020, 2:18:29 UTC

I am getting the "Project has no tasks available" message .

Something is broken.


Almost as soon as I posted, workflow resumed - somewhat.
Member of the 20 Year Club



ID: 2026062 · Report as offensive
Tony Barnes

Send message
Joined: 19 Jun 12
Posts: 2
Credit: 1,540,918
RAC: 0
United States
Message 2026067 - Posted: 3 Jan 2020, 2:30:34 UTC

Since today's maintenance concluded I have not received any work items. I checked the logs and it says internal server error. Any ideas?

Doc
ID: 2026067 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2026069 - Posted: 3 Jan 2020, 2:44:28 UTC

I just re-set my "max reports" to 50, re-started Boinc and it "reported" 50 with a manual update and then promptly downloaded 50. Of which "several" were labeled as resends.

Tom
A proud member of the OFA (Old Farts Association).
ID: 2026069 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2026075 - Posted: 3 Jan 2020, 3:17:52 UTC - in response to Message 2026069.  

I've already reported everything on the daily driver and am only reporting currently finishing work. But I have yet to receive anything in return.

At least all the other hosts are able to report now instead of no connection. But nobody is getting any work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2026075 · Report as offensive
6BQ5

Send message
Joined: 7 Dec 18
Posts: 29
Credit: 12,725,636
RAC: 357
United States
Message 2026077 - Posted: 3 Jan 2020, 3:38:51 UTC

Sitting at my computer now....

My fans suddenly whirred on. BOINC says tasks have come through.

Hooray! :)

-=- Boris
ID: 2026077 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2026079 - Posted: 3 Jan 2020, 3:44:46 UTC
Last modified: 3 Jan 2020, 3:46:57 UTC

i've had a few bursts of new work, but it's too infrequent to keep the fast systems fed.

Thu 02 Jan 2020 10:29:38 PM EST | SETI@home | Scheduler request completed: got 61 new tasks


Thu 02 Jan 2020 10:39:36 PM EST | SETI@home | Scheduler request completed: got 38 new tasks

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2026079 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 2026085 - Posted: 3 Jan 2020, 4:31:47 UTC
Last modified: 3 Jan 2020, 4:32:51 UTC

Got home to find systems in extended backoffs, hit update & was able to report work, but "Project has no tasks available" was the general response. Then work started trickling in, a half dozen here & there. And now the downloads are just sitting there, timer counting away bt nothing happening (till just as I was about to post, then managed to get the downloads to finally start).
Will see what happens with the next Scheduler request.


Edit- back to "Project has no tasks available"
Grant
Darwin NT
ID: 2026085 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 2026097 - Posted: 3 Jan 2020, 6:08:53 UTC

Been almost 1.5 hrs since my Linux system got any work, and in that time my Window system has only gotten work once.

Larger caches do allow our systems to remain busy during outages, but it also means there's a bigger deficit to recover from after an outage. And the longer the outage, the bigger that bigger deficit is.
Almost 7 hours after the end of the outage, the return rate is still over 160k per hour. The splitters have finally started to produce some work, but the present deficit until caches are full again is around 800k WUs and still growing, as the return rate is still well above the new work production rate.

Hopefully the new NAS server will allow the splitters to get up to speed much sooner, and that speed to be much higher...
Grant
Darwin NT
ID: 2026097 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1646
Credit: 12,921,799
RAC: 89
New Zealand
Message 2026103 - Posted: 3 Jan 2020, 7:44:51 UTC - in response to Message 2026097.  

Grant in regards to the speed I am not sure there will be any increase as the hard drives are only 7200 rpm, in saying this I do not know what speed the drives are we are currently using. I am hopeful things will work faster. Only time will tell
ID: 2026103 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13904
Credit: 208,696,464
RAC: 304
Australia
Message 2026104 - Posted: 3 Jan 2020, 7:49:19 UTC - in response to Message 2026103.  

Grant in regards to the speed I am not sure there will be any increase as the hard drives are only 7200 rpm, in saying this I do not know what speed the drives are we are currently using.
The areal density of the new drives is much, much, much higher than the older ones, which means even for the same rotational speed their minimum & maximum data rates will be much higher.
Also being in a single enclosure, with much newer hardware (and more RAM for caching), the overall performance should be significantly better.
An All Flash Array would be better still (probably by an order of magnitude or more), but the cost of one with the same storage capacity would be considerably greater.
Grant
Darwin NT
ID: 2026104 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1646
Credit: 12,921,799
RAC: 89
New Zealand
Message 2026105 - Posted: 3 Jan 2020, 7:57:53 UTC - in response to Message 2026104.  

Thank you for the insight & I completely agree with you will be much faster.
ID: 2026105 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2026121 - Posted: 3 Jan 2020, 11:14:06 UTC - in response to Message 2026104.  
Last modified: 3 Jan 2020, 11:15:14 UTC

Grant in regards to the speed I am not sure there will be any increase as the hard drives are only 7200 rpm, in saying this I do not know what speed the drives are we are currently using.
The areal density of the new drives is much, much, much higher than the older ones, which means even for the same rotational speed their minimum & maximum data rates will be much higher.
Also being in a single enclosure, with much newer hardware (and more RAM for caching), the overall performance should be significantly better.
An All Flash Array would be better still (probably by an order of magnitude or more), but the cost of one with the same storage capacity would be considerably greater.


. . And that was in effect the new server which has died on Beta, Muarae2, all SSD drives.

Stephen

:(
ID: 2026121 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 37586
Credit: 261,360,520
RAC: 489
Australia
Message 2026124 - Posted: 3 Jan 2020, 11:42:26 UTC - in response to Message 2026121.  

Grant in regards to the speed I am not sure there will be any increase as the hard drives are only 7200 rpm, in saying this I do not know what speed the drives are we are currently using.
The areal density of the new drives is much, much, much higher than the older ones, which means even for the same rotational speed their minimum & maximum data rates will be much higher.
Also being in a single enclosure, with much newer hardware (and more RAM for caching), the overall performance should be significantly better.
An All Flash Array would be better still (probably by an order of magnitude or more), but the cost of one with the same storage capacity would be considerably greater.

. . And that was in effect the new server which has died on Beta, Muarae2, all SSD drives.

Stephen :(
Until the write/rewrite wear, cost and size factors have been eliminated then a mechanical drive is still the best outright bang for buck solution still under ideal conditions. ;-)

Cheers.
ID: 2026124 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2026146 - Posted: 3 Jan 2020, 16:24:16 UTC - in response to Message 2026124.  

. . And that was in effect the new server which has died on Beta, Muarae2, all SSD drives.

I haven't seen ANY post-mortem report on muarae2 over at Beta. What have you heard? All I heard was they had a problem with the database. Not any specifics.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2026146 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 2026148 - Posted: 3 Jan 2020, 17:38:23 UTC
Last modified: 3 Jan 2020, 17:40:28 UTC

The RTS is building and the out in the field is dropping. The greedy server isn't handing out WUs right now, even though it has some. I know at some point the dam will break and the WUs will flow, but I'm curious what coding bit is causing this hoarding.

edit: and just after posting. I get a bunch of WUs.
ID: 2026148 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11449
Credit: 29,581,041
RAC: 66
United States
Message 2026149 - Posted: 3 Jan 2020, 17:42:06 UTC - in response to Message 2026148.  

Posting about a problem often fixes it immediately
ID: 2026149 · Report as offensive
Previous · 1 . . . 15 · 16 · 17 · 18 · 19 · 20 · 21 . . . 94 · Next

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.