Panic Mode On (114) Server Problems?

Message boards : Number crunching : Panic Mode On (114) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 45 · Next

AuthorMessage
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1971886 - Posted: 24 Dec 2018, 23:36:29 UTC - in response to Message 1971864.  

I'm hoping that someone will do the Tuesday maintenance and it will help. I hope the Tuesday maintenance is mainly automated. It started really early the past couple of weeks.
I fear that Jeff has just proved that maintenance isn't completely automated. This just appeared on the front page:

Jeff Cobb wrote:
Because of the Christmas and New Years Day holidays, our weekly outage will be on Wednesday rather than the normal Tuesday both this week and next.


. . Cool! Knowing is enough :)

. . Thanks for the repost ...

Stephen

:)
ID: 1971886 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1971887 - Posted: 24 Dec 2018, 23:40:07 UTC - in response to Message 1971868.  

The validators are validating, the assimilators are assimilating, so the system is looking good until the Wednesday outage. I'm glad they let us know. I hope everyone has a good holiday!!!


. . And a peaceful

M E R R Y
. . . . . . . . . . . C H R I S T M A S

to everyone ...

Stephen

:)
ID: 1971887 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14020
Credit: 208,696,464
RAC: 304
Australia
Message 1971931 - Posted: 25 Dec 2018, 10:20:50 UTC - in response to Message 1971864.  

I'm hoping that someone will do the Tuesday maintenance and it will help. I hope the Tuesday maintenance is mainly automated. It started really early the past couple of weeks.
I fear that Jeff has just proved that maintenance isn't completely automated. This just appeared on the front page:

Jeff Cobb wrote:
Because of the Christmas and New Years Day holidays, our weekly outage will be on Wednesday rather than the normal Tuesday both this week and next.

The outage may not be automated, but it would appear the notice is.
Grant
Darwin NT
ID: 1971931 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51586
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1971972 - Posted: 25 Dec 2018, 21:16:03 UTC

Have not been able to connect with the servers for the last 20 minutes or so.
Anybody else?
Or a problem for the kitties to solve?

Meow?
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1971972 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1971973 - Posted: 25 Dec 2018, 21:18:31 UTC

. . Did someone let the servers know it is Christmas? They seem to be having a break by themselves.

. . Server with message forums, super slow. Upload/download servers very slow to respond (getting servers may be offline message) and then when a work request does get through ... "No tasks available" ...

. . I am beginning to think these servers have developed their own intelligence and are playing games with us. ...

. . Forgive the whimsy but hey ... what the heck?

Stephen

<shrug>
ID: 1971973 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1971974 - Posted: 25 Dec 2018, 21:21:00 UTC

I'm just acknowledging that there is an issue with the seti machines. It might be a temporary thing as we have noticed this happening for a couple of days now. It might fix itself, or it might not. I do not wish for anyone to be disturbed. Just wanted to mention it.

Merry Christmas!!!
ID: 1971974 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1971975 - Posted: 25 Dec 2018, 21:21:12 UTC

Web site is very slow right now. I'm not getting through now either
Serenity

103509 SETI@home 12/25/2018 1:17:21 PM Scheduler request failed: Timeout was reached
103510 SETI@home 12/25/2018 1:17:21 PM [sched_op] Deferring communication for 00:03:27
103511 SETI@home 12/25/2018 1:17:21 PM [sched_op] Reason: Scheduler request failed
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1971975 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51586
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1971976 - Posted: 25 Dec 2018, 21:21:53 UTC - in response to Message 1971973.  

. . Did someone let the servers know it is Christmas? They seem to be having a break by themselves.

. . Server with message forums, super slow. Upload/download servers very slow to respond (getting servers may be offline message) and then when a work request does get through ... "No tasks available" ...

. . I am beginning to think these servers have developed their own intelligence and are playing games with us. ...

. . Forgive the whimsy but hey ... what the heck?

Stephen

<shrug>

Thanks for the confirmation.
Hopefully just a chron job that will clear in a bit.

Meow.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1971976 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11451
Credit: 29,581,041
RAC: 66
United States
Message 1971977 - Posted: 25 Dec 2018, 21:28:00 UTC

Yep the servers are having indigestion.
ID: 1971977 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1971979 - Posted: 25 Dec 2018, 21:40:55 UTC

Tue 25 Dec 2018 04:38:09 PM EST |  | Project communication failed: attempting access to reference site
Tue 25 Dec 2018 04:38:09 PM EST | SETI@home | Scheduler request failed: Timeout was reached
Tue 25 Dec 2018 04:38:12 PM EST |  | Internet access OK - project servers may be temporarily down.


Servers Christmas hangover?
ID: 1971979 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51586
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1971980 - Posted: 25 Dec 2018, 21:51:11 UTC

Well, the kitties will send the word out.
If Eric or Jeff are about and wish to attend to it, fine.
If it doesn't get fixed before tomorrow's outage, that is just fine too.

Meow.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1971980 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1971982 - Posted: 25 Dec 2018, 22:07:53 UTC

Hosts are out of gpu work and not getting any. Time for backup projects.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1971982 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14020
Credit: 208,696,464
RAC: 304
Australia
Message 1971983 - Posted: 25 Dec 2018, 22:08:39 UTC

I see the Scheduler is having issues again, either "Couldn't connect to server" or when it does the often seen "Project has no tasks available".
Received-last-hour numbers have fallen off a cliff, and the Database Master-queries-per-second hit zero. They've recovered slightly, but are still well down on their usual 1.5k or so. And quite a few of the Server Status values haven't update for over an hour.

I think the staff should get at least 1 day a year off. We'll just have to wait till after tomorrows official outage, to recover from todays un-official one.
Grant
Darwin NT
ID: 1971983 · Report as offensive
Sleepy
Volunteer tester
Avatar

Send message
Joined: 21 May 99
Posts: 219
Credit: 98,947,784
RAC: 28,360
Italy
Message 1971986 - Posted: 25 Dec 2018, 22:29:57 UTC - in response to Message 1971983.  

Seems to me the flow has restarted.
ID: 1971986 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1861
Credit: 268,616,081
RAC: 1,349
United States
Message 1971987 - Posted: 25 Dec 2018, 22:36:08 UTC - in response to Message 1971986.  

Seems to me the flow has restarted.

+1, slowly but surely recovering. Caches refilled here.
Merry Christmas, all.
ID: 1971987 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14020
Credit: 208,696,464
RAC: 304
Australia
Message 1971989 - Posted: 25 Dec 2018, 22:40:56 UTC - in response to Message 1971986.  

Seems to me the flow has restarted.

Yep.
The servers live again.
Grant
Darwin NT
ID: 1971989 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1971990 - Posted: 25 Dec 2018, 22:41:29 UTC

RTS over 800K. We have seen this before where the splitter cutoff mechanism doesn't trip correctly and the splitters continue to split but the scheduler doesn't recognize requests for work and sends out no tasks available messages. Looks like the scheduler picked up the pieces again and tasks are going out to refill caches.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1971990 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 38734
Credit: 261,360,520
RAC: 489
Australia
Message 1971991 - Posted: 25 Dec 2018, 22:44:31 UTC - in response to Message 1971990.  

RTS over 800K. We have seen this before where the splitter cutoff mechanism doesn't trip correctly and the splitters continue to split but the scheduler doesn't recognize requests for work and sends out no tasks available messages. Looks like the scheduler picked up the pieces again and tasks are going out to refill caches.
That's the resends sending the RTS up Keith, not the splitters. ;-)

Cheers.
ID: 1971991 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1971992 - Posted: 25 Dec 2018, 22:58:23 UTC - in response to Message 1971991.  

RTS over 800K. We have seen this before where the splitter cutoff mechanism doesn't trip correctly and the splitters continue to split but the scheduler doesn't recognize requests for work and sends out no tasks available messages. Looks like the scheduler picked up the pieces again and tasks are going out to refill caches.
That's the resends sending the RTS up Keith, not the splitters. ;-)

Cheers.

Really? We get 300K resends for every hour??
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1971992 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1972005 - Posted: 25 Dec 2018, 23:54:53 UTC - in response to Message 1971976.  

Thanks for the confirmation.
Hopefully just a chron job that will clear in a bit.

Meow.


. . I don't think I have asked this before, but what is a Chron job?

Stephen

? ?
ID: 1972005 · Report as offensive
Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 45 · Next

Message boards : Number crunching : Panic Mode On (114) Server Problems?


 
©2026 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.