The Server Issues / Outages Thread - Panic Mode On! (118)

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 7 · 8 · 9 · 10 · 11 · 12 · 13 . . . 94 · Next

AuthorMessage
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2024815 - Posted: 24 Dec 2019, 11:37:11 UTC

Hope all will be fine and we could be back to run the anonymous apps after the outage.
Back in a couple of hours to see what we could expect.
Fingers crossed.
ID: 2024815 · Report as offensive
Profile NorthCup

Send message
Joined: 6 Jun 99
Posts: 108
Credit: 50,093,984
RAC: 5
Germany
Message 2024820 - Posted: 24 Dec 2019, 12:48:05 UTC
Last modified: 24 Dec 2019, 12:52:09 UTC

All Linux hosts are here without a job. I have reset BOINC at my last Win computer. He got 70 tasks without any problems and now needs twice the time per WU. I think it is time to finally optimize the clients. For all operating systems! We hav'nt money to throw it out the window. Merry Christmas to all Setians!
ID: 2024820 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14687
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2024821 - Posted: 24 Dec 2019, 13:01:09 UTC

I only set one Windows host back to stock so I could watch what was happening. I've now reported the last stock tasks in anticipation of an early outage, and restored the host to normal. It'll be patiently waiting for anonymous platform work whenever we come back online.

My one fast Linux 'special sauce' box has a full cache and should see out the outage (if it isn't excessively long). I'll restore that one before I leave for my own holiday break tomorrow, and it can recover by itself from there.
ID: 2024821 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13885
Credit: 208,696,464
RAC: 304
Australia
Message 2024826 - Posted: 25 Dec 2019, 1:50:00 UTC

Well, we're back.

25/12/2019 11:18:01 | SETI@home | Scheduler request failed: Couldn't connect to server
But it did only take 4 seconds to error out, which is better than 30sec to a couple of minutes.
Grant
Darwin NT
ID: 2024826 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2024830 - Posted: 25 Dec 2019, 1:56:22 UTC

Can't connect to server.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2024830 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13885
Credit: 208,696,464
RAC: 304
Australia
Message 2024831 - Posted: 25 Dec 2019, 1:59:39 UTC - in response to Message 2024830.  
Last modified: 25 Dec 2019, 2:00:28 UTC

Can't connect to server.
Yeah, but at least it only takes 4 secs now and not a couple of minutes...


How about stock hosts?
Grant
Darwin NT
ID: 2024831 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13885
Credit: 208,696,464
RAC: 304
Australia
Message 2024832 - Posted: 25 Dec 2019, 2:03:07 UTC

Just had a look at the server page. Transitioners are all down, Feeder is down.

And the Scheduling server is Disabled...
So we're not quite back yet.
Grant
Darwin NT
ID: 2024832 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2024833 - Posted: 25 Dec 2019, 2:03:43 UTC - in response to Message 2024831.  

Can't connect to server.
Yeah, but at least it only takes 4 secs now and not a couple of minutes...


How about stock hosts?


same on my stock hosts. cant connect, responds in a few seconds.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2024833 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13885
Credit: 208,696,464
RAC: 304
Australia
Message 2024835 - Posted: 25 Dec 2019, 2:27:47 UTC

Server status now showing almost all functions as Disabled.

Looks like the restart after the outage is having it's own issues.
Grant
Darwin NT
ID: 2024835 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2024837 - Posted: 25 Dec 2019, 2:36:16 UTC - in response to Message 2024836.  

At least we're back to Server version 709 now


+1

I am still crunching through a bunch of Linux stock GPU tasks so I will continue being NNT until I can re-start my AIO stuff.

Tom
A proud member of the OFA (Old Farts Association).
ID: 2024837 · Report as offensive
Profile Freewill Project Donor
Avatar

Send message
Joined: 19 May 99
Posts: 766
Credit: 354,398,348
RAC: 11,693
United States
Message 2024838 - Posted: 25 Dec 2019, 2:40:10 UTC

Glad to see ver 709! Like Tom, and perhaps many others, I have to clear my stock queue and report before I switch back to special sauce.

Happy Holidays and Happy Crunching! Hope it's all good in the morning. Time for bed here.
ID: 2024838 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2024840 - Posted: 25 Dec 2019, 2:42:49 UTC
Last modified: 25 Dec 2019, 2:49:45 UTC

Let`s wait a couple of hours for the system to stabilize and then we could think to power up our hungry hosts.

Time for Christmas dinner, wines & spend some time with the family.
ID: 2024840 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2024842 - Posted: 25 Dec 2019, 2:46:39 UTC - in response to Message 2024840.  

Let`s wait a couple of hours for the system to stabilize and then we could think to power up out hungry hosts.

Time for Christmas dinner, wines & spend some time with the family.



+1

Or maybe even till tomorrow! :)

Tom
A proud member of the OFA (Old Farts Association).
ID: 2024842 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2024843 - Posted: 25 Dec 2019, 2:47:18 UTC

my Anonymous platform host just got 14 new tasks :)
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2024843 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11447
Credit: 29,581,041
RAC: 66
United States
Message 2024846 - Posted: 25 Dec 2019, 2:56:28 UTC - in response to Message 2024843.  

my Anonymous platform host just got 14 new tasks :)

I would like some also.
ID: 2024846 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21570
Credit: 7,508,002
RAC: 20
United Kingdom
Message 2024848 - Posted: 25 Dec 2019, 2:59:08 UTC - in response to Message 2024759.  
Last modified: 25 Dec 2019, 2:59:56 UTC

Hence the statement "revert the database"...

Or more aptly:

Reverse the polarity of the neutron flow?

;-)

We have coinciding with the s@h server reversion... An astronomical real-world reversion:

Reversed polarity sunspots!


Enjoy! :-)

Happy Festivities!!

Keep searchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 2024848 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2024849 - Posted: 25 Dec 2019, 3:10:35 UTC - in response to Message 2024843.  

my Anonymous platform host just got 14 new tasks :)

Still either timing out on the request and backing off or 0 tasks received.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2024849 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1858
Credit: 268,616,081
RAC: 1,349
United States
Message 2024851 - Posted: 25 Dec 2019, 3:21:27 UTC - in response to Message 2024849.  

my Anonymous platform host just got 14 new tasks :)

Still either timing out on the request and backing off or 0 tasks received.

Ditto
ID: 2024851 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13885
Credit: 208,696,464
RAC: 304
Australia
Message 2024854 - Posted: 25 Dec 2019, 3:36:25 UTC
Last modified: 25 Dec 2019, 3:39:56 UTC

Server status is back to green, and making regular Scheduler contact here, but "Project has no tasks available" is the only response so far.
But at least the responses are coming within 2-3 seconds.

Given the effective length of this outage, and the fact the system has struggled to recover after much shorter outages, it could take a couple of days for things to fully recover.


Edit-
Spoke too soon. Just had a 40 second wait for a Scheduler response. Not a good sign.
Grant
Darwin NT
ID: 2024854 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13885
Credit: 208,696,464
RAC: 304
Australia
Message 2024857 - Posted: 25 Dec 2019, 3:55:44 UTC
Last modified: 25 Dec 2019, 3:57:52 UTC

Now back to how things were with the previous Scheduler version- extended Scheduler response times, with occasional errors instead of a valid response.


It would be rather disappointing if they've gone to all the effort to revert the Scheduler, and it doesn't fix the problem (as the bug that stops Anonymous platforms from getting work could have been in this Scheduler version as well, it's only the recent issue resulting in extended Scheduler responses that has made it more apparent).

Any Stock hosts getting work? Is Resend lost tasks still on?
Grant
Darwin NT
ID: 2024857 · Report as offensive
Previous · 1 . . . 7 · 8 · 9 · 10 · 11 · 12 · 13 . . . 94 · Next

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.