Panic Mode On (7) Server Problems! Closed for Renovation

Message boards : Number crunching : Panic Mode On (7) Server Problems! Closed for Renovation
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 12 · Next

AuthorMessage
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51555
Credit: 1,018,363,574
RAC: 1,004
United States
Message 733944 - Posted: 3 Apr 2008, 23:39:52 UTC - in response to Message 733943.  

Here's the new thread, May yer problems be brief ones.

Or preferably, none at all...
(But I look at the world with big kitty eyes........).
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 733944 · Report as offensive
Profile Logan
Volunteer tester
Avatar

Send message
Joined: 26 Jan 07
Posts: 743
Credit: 918,353
RAC: 0
Spain
Message 733946 - Posted: 3 Apr 2008, 23:43:56 UTC

ID: 733946 · Report as offensive
Profile Andy Lee Robinson
Avatar

Send message
Joined: 8 Dec 05
Posts: 630
Credit: 59,973,836
RAC: 0
Hungary
Message 733955 - Posted: 4 Apr 2008, 0:07:53 UTC

Hosed again!


(suggested new title) :-)
ID: 733955 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51555
Credit: 1,018,363,574
RAC: 1,004
United States
Message 733960 - Posted: 4 Apr 2008, 0:16:29 UTC - in response to Message 733955.  

Hosed again!


(suggested new title) :-)

How so hosed?
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 733960 · Report as offensive
Profile RandyC
Avatar

Send message
Joined: 20 Oct 99
Posts: 714
Credit: 1,704,345
RAC: 0
United States
Message 733977 - Posted: 4 Apr 2008, 0:48:46 UTC - in response to Message 733960.  
Last modified: 4 Apr 2008, 0:49:00 UTC

Hosed again!


(suggested new title) :-)

How so hosed?


Maybe rubber in a back room? ;)
ID: 733977 · Report as offensive
Profile Andy Lee Robinson
Avatar

Send message
Joined: 8 Dec 05
Posts: 630
Credit: 59,973,836
RAC: 0
Hungary
Message 733997 - Posted: 4 Apr 2008, 1:30:36 UTC - in response to Message 733960.  
Last modified: 4 Apr 2008, 1:31:12 UTC

Hosed again!


(suggested new title) :-)

How so hosed?


http://www.urbandictionary.com/define.php?term=hosed :-)

Says it all... I hope we can have an uninteruppted service one day...
ID: 733997 · Report as offensive
Profile John Clark
Volunteer tester
Avatar

Send message
Joined: 29 Sep 99
Posts: 16515
Credit: 4,418,829
RAC: 0
United Kingdom
Message 734057 - Posted: 4 Apr 2008, 6:44:02 UTC

Have you seen the level of demand on the Cricket graphs is still very high. This shows the system still settling down, and drawing 92+Mbps bandwidth.

Can this demand go on, or will the servers collapse under the strain?
It's good to be back amongst friends and colleagues



ID: 734057 · Report as offensive
Alinator
Volunteer tester

Send message
Joined: 19 Apr 05
Posts: 4178
Credit: 4,647,982
RAC: 0
United States
Message 734066 - Posted: 4 Apr 2008, 7:20:09 UTC
Last modified: 4 Apr 2008, 7:20:35 UTC

Offhand, I'd say this looks solid so far. The recent trend has been somewhat less than half the outage time at 90+ bandwidth before starting to tail off.

My recommendation; Panic Off for now. :-)

Alinator
ID: 734066 · Report as offensive
Profile Andy Lee Robinson
Avatar

Send message
Joined: 8 Dec 05
Posts: 630
Credit: 59,973,836
RAC: 0
Hungary
Message 734067 - Posted: 4 Apr 2008, 7:20:39 UTC - in response to Message 734057.  

Have you seen the level of demand on the Cricket graphs is still very high. This shows the system still settling down, and drawing 92+Mbps bandwidth.

Can this demand go on, or will the servers collapse under the strain?


If it lasted this long, then there's no reason to think that it'll fail because of overload... other factors seem to be the cause.

It's interesting, that the area of the peak above average should be roughly the same as the area of the gap in the chart...
ID: 734067 · Report as offensive
Profile AndyW Project Donor
Volunteer tester
Avatar

Send message
Joined: 23 Oct 02
Posts: 5862
Credit: 10,957,677
RAC: 18
United Kingdom
Message 734079 - Posted: 4 Apr 2008, 8:10:52 UTC

Looking at the graph the servers have coped remarkably well catching up with the backlog and only now is the throughout starting to reduce indicating the worst of it is over for them.
ID: 734079 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51555
Credit: 1,018,363,574
RAC: 1,004
United States
Message 734094 - Posted: 4 Apr 2008, 9:31:09 UTC

Wow........but that was an extended bit of bandwidth.....
Cricket Gone Wild......
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 734094 · Report as offensive
Profile RandyC
Avatar

Send message
Joined: 20 Oct 99
Posts: 714
Credit: 1,704,345
RAC: 0
United States
Message 734239 - Posted: 4 Apr 2008, 16:15:25 UTC - in response to Message 734226.  

Wow........but that was an extended bit of bandwidth.....
Cricket Gone Wild......

With two instances of about 100, One at or near and one slightly above 100, wow.


Interesting, even though we've dropped down quite a bit, we're still running above normal and increasing over the past week. We started at about 35MB after the recovery from an outage on 25 Mar, and are currently running 60-65MB (as of this post).

This link displays activity a little clearer.
ID: 734239 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 734265 - Posted: 4 Apr 2008, 17:09:51 UTC - in response to Message 734239.  

Wow........but that was an extended bit of bandwidth.....
Cricket Gone Wild......

With two instances of about 100, One at or near and one slightly above 100, wow.


Interesting, even though we've dropped down quite a bit, we're still running above normal and increasing over the past week. We started at about 35MB after the recovery from an outage on 25 Mar, and are currently running 60-65MB (as of this post).

This link displays activity a little clearer.

Actually, I would say that the current level is closer to 'normal'.

Coincident with the installation of the new router, Matt found a stash of tapes which split to provide almost exclusively long-running WUs. Now the shorties are back in the mix. We get through them quicker, so more have to be sent out to keep our shivering cores warm, so the bandwidth goes up - to about the level the old router maxxed out. Now, we have a bit of spare overhead for recovery too.
ID: 734265 · Report as offensive
Profile Clyde C. Phillips, III

Send message
Joined: 2 Aug 00
Posts: 1851
Credit: 5,955,047
RAC: 0
United States
Message 734268 - Posted: 4 Apr 2008, 17:13:49 UTC

Things look better at my computers. Fewer "Ready To Reports" and a fuller cache than yesterday.
ID: 734268 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13961
Credit: 208,696,464
RAC: 304
Australia
Message 734409 - Posted: 4 Apr 2008, 21:50:48 UTC


Minor problem- looking at Scarecrow's graphs only the Result's recieved in the last hour, Result turn around time & Result creation rate graphs are updating. Just had a look at the Server Status page & it's been 37 hours since the other stats were updated.
Grant
Darwin NT
ID: 734409 · Report as offensive
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 962
Credit: 537,293
RAC: 9
United Kingdom
Message 736208 - Posted: 8 Apr 2008, 15:28:23 UTC
Last modified: 8 Apr 2008, 15:29:05 UTC

08/04/2008 16:27:45|SETI@home|Sending scheduler request: To fetch work. Requesting 624 seconds of work, reporting 0 completed tasks
08/04/2008 16:27:50|SETI@home|Scheduler request succeeded: got 0 new tasks
08/04/2008 16:27:50|SETI@home|[sched_ops_debug] Server version 601
08/04/2008 16:27:50|SETI@home|Project requested delay of 11.000000 seconds
08/04/2008 16:27:50|SETI@home|[sched_op_debug] Deferring communication for 4 min 5 sec
08/04/2008 16:27:50|SETI@home|[sched_op_debug] Reason: no work from project
Sir Arthur C Clarke 1917-2008
ID: 736208 · Report as offensive
Scarecrow

Send message
Joined: 15 Jul 00
Posts: 4520
Credit: 486,601
RAC: 0
United States
Message 736304 - Posted: 9 Apr 2008, 2:17:24 UTC - in response to Message 734409.  
Last modified: 9 Apr 2008, 2:20:19 UTC


Minor problem- looking at Scarecrow's graphs only the Result's recieved in the last hour, Result turn around time & Result creation rate graphs are updating. Just had a look at the Server Status page & it's been 37 hours since the other stats were updated.


I reverse my stand on this issue... I liked it it better when the RTS was flat but at half a million. I hereby request you break it again. (just no pleasing some people, is there?) :)
ID: 736304 · Report as offensive
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 962
Credit: 537,293
RAC: 9
United Kingdom
Message 736463 - Posted: 9 Apr 2008, 14:25:51 UTC - in response to Message 736462.  

Yeah I'm seeing the stats both local to Seti and not so local not updating, I think something is broken again. :(


See Latest Tech News. Matt has explained the reason for the longer than usual outage yesterday, and there may be another one today. He also explained the reason that the validators are currently offline.

Stuff always seem to happen when I get close to a milestone, hoping to reach 50,000 SETI credits this month.
Sir Arthur C Clarke 1917-2008
ID: 736463 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19733
Credit: 40,757,560
RAC: 67
United Kingdom
Message 736818 - Posted: 10 Apr 2008, 11:42:23 UTC

The cricket graphs indicate the servers are not communicating again.

Daily Graph
ID: 736818 · Report as offensive
Profile David
Volunteer tester
Avatar

Send message
Joined: 19 May 99
Posts: 411
Credit: 1,426,457
RAC: 0
Australia
Message 736843 - Posted: 10 Apr 2008, 12:29:40 UTC - in response to Message 736818.  

The cricket graphs indicate the servers are not communicating again.

Daily Graph


Yeah, looks like we will have to wait until we hear from the staff about whats happened. Probably something simple I hope, but since I have a heap of WU's queued, probably no big deal here... For about 36 more hrs that is ;)

ID: 736843 · Report as offensive
1 · 2 · 3 · 4 . . . 12 · Next

Message boards : Number crunching : Panic Mode On (7) Server Problems! Closed for Renovation


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.