Panic Mode On (10) Server problems

Message boards : Number crunching : Panic Mode On (10) Server problems
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 9 · 10 · 11 · 12 · 13 · Next

AuthorMessage
Profile KWSN Ekky Ekky Ekky
Avatar

Send message
Joined: 25 May 99
Posts: 944
Credit: 52,956,491
RAC: 67
United Kingdom
Message 834531 - Posted: 26 Nov 2008, 16:32:31 UTC

Server page speak with forked tongue.

ID: 834531 · Report as offensive
-Bert-

Send message
Joined: 23 Mar 02
Posts: 152
Credit: 412,754
RAC: 0
Netherlands
Message 834532 - Posted: 26 Nov 2008, 16:41:19 UTC - in response to Message 834531.  

Why aren't the problems mentioned on the server status page?
ID: 834532 · Report as offensive
BarryAZ

Send message
Joined: 1 Apr 01
Posts: 2580
Credit: 16,982,517
RAC: 0
United States
Message 834535 - Posted: 26 Nov 2008, 16:57:18 UTC - in response to Message 834532.  

Possibly because one of the symptoms *IS* the status page??

The servers seem to have limited margin for error or extra load on them (or at least some of the servers) and so the most likely time for problems to surface is after the weekly maintenance. The servers also need close attention on a constant basis, so often enough problems surface at overnight or during the weekends when there is less constant server nursing available.

Why aren't the problems mentioned on the server status page?


ID: 834535 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65746
Credit: 55,293,173
RAC: 49
United States
Message 834536 - Posted: 26 Nov 2008, 16:59:15 UTC - in response to Message 834520.  

Same here folks....none of my rigs has been able to report any completed work for the last 3-1/2 hours or so....
Hopefully the boyz shall be arriving in the lab shortly with their server kicking boots laced up tight....LOL.

We had heavy rain out here last night and as usual I had to call Verizon and threaten to go to Time Warner before Verizon got a level2 tech on the line seems I had line trouble(an outage) and their putting someone on It after My line failed their line test and It started around this time:

11/26/2008 6:49:11 AM||Access to reference site succeeded - project servers may be temporarily down.
11/26/2008 6:49:12 AM|SETI@home|Computation for task 07oc08ac.4053.1704.3.8.193_0 finished
11/26/2008 6:49:12 AM|SETI@home|Starting 08oc08ah.12781.15205.4.8.234_1
11/26/2008 6:49:12 AM|SETI@home|Starting task 08oc08ah.12781.15205.4.8.234_1 using setiathome_enhanced version 528
11/26/2008 6:50:38 AM|SETI@home|Computation for task 07oc08ac.4053.1704.3.8.196_0 finished
11/26/2008 6:50:38 AM|SETI@home|Starting 11se08af.9257.72.15.8.254_3
11/26/2008 6:50:38 AM|SETI@home|Starting task 11se08af.9257.72.15.8.254_3 using setiathome_enhanced version 528
11/26/2008 6:50:51 AM|SETI@home|Computation for task 11se08af.9257.72.15.8.254_3 finished
11/26/2008 6:50:51 AM|SETI@home|Starting 08oc08ag.1970.4571.16.8.210_0
11/26/2008 6:50:51 AM|SETI@home|Starting task 08oc08ag.1970.4571.16.8.210_0 using setiathome_enhanced version 528
11/26/2008 6:51:39 AM||Project communication failed: attempting access to reference site
11/26/2008 6:51:39 AM|SETI@home|[file_xfer] Temporarily failed upload of 08oc08ah.19861.18068.3.8.203_0_0: HTTP error
11/26/2008 6:51:39 AM|SETI@home|Backing off 5 min 45 sec on upload of file 08oc08ah.19861.18068.3.8.203_0_0
11/26/2008 6:51:39 AM|SETI@home|[file_xfer] Started upload of file 08oc08aa.19792.25021.14.8.190_1_0
11/26/2008 6:52:00 AM||Access to reference site failed - check network connection or proxy configuration.
[snip]
11/26/2008 8:38:51 AM|SETI@home|[file_xfer] Started upload of file 08oc08ag.1970.7843.16.8.173_1_0
11/26/2008 8:38:53 AM||Project communication failed: attempting access to reference site
11/26/2008 8:38:53 AM|SETI@home|[file_xfer] Temporarily failed upload of 08oc08ah.12781.15205.4.8.143_0_0: connect() failed
11/26/2008 8:38:53 AM|SETI@home|Backing off 1 min 0 sec on upload of file 08oc08ah.12781.15205.4.8.143_0_0
11/26/2008 8:38:54 AM||Access to reference site succeeded - project servers may be temporarily down.
11/26/2008 8:38:54 AM|SETI@home|[file_xfer] Started upload of file 07oc08ac.4053.1704.3.8.140_0_0
11/26/2008 8:38:54 AM|SETI@home|[file_xfer] Finished upload of file 08oc08ag.1970.7843.16.8.173_1_0

This happens everytime It rains heavily out here as the cables are buried in the ground and being the soil here doesn't allow water to seep in quickly It takes a while to disappear unless It's Hot out and right now It's not too warm outside and rather damp too.

Their Tech support is nothing to be proud of and is not only irritating, But almost worse than useless and I wonder why I'm still out here(or rather now You know another reason why I wanted to move, sonic booms, excessive heat, this area is quiet and a bit neglected).
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 834536 · Report as offensive
Profile SATAN
Avatar

Send message
Joined: 27 Aug 06
Posts: 835
Credit: 2,129,006
RAC: 0
United Kingdom
Message 834537 - Posted: 26 Nov 2008, 17:03:55 UTC

Is it just me or does anyone else find it worrying, that the number of problems is slowly increasing. It seems that after the outage, the next three days are nothing but trouble. When the guys finally manage to sort those out it's time for the weekend. Which then bring the normal problems of one slip and the project is off until monday at least.

Hopefully this time it's a simple fix.


ID: 834537 · Report as offensive
BarryAZ

Send message
Joined: 1 Apr 01
Posts: 2580
Credit: 16,982,517
RAC: 0
United States
Message 834545 - Posted: 26 Nov 2008, 17:14:22 UTC - in response to Message 834537.  

Not just you - seems that way to me as well. This time we are headed for a holiday weekend with the server in an unhappy state. The longer any particular problem exists, the more stretched out the post fix server stress cycle is going to be. The thing is, the server maintenance cycle itself is a stressor.


Is it just me or does anyone else find it worrying, that the number of problems is slowly increasing. It seems that after the outage, the next three days are nothing but trouble. When the guys finally manage to sort those out it's time for the weekend. Which then bring the normal problems of one slip and the project is off until monday at least.

Hopefully this time it's a simple fix.



ID: 834545 · Report as offensive
Profile Ghery S. Pettit
Avatar

Send message
Joined: 7 Nov 99
Posts: 325
Credit: 28,109,066
RAC: 82
United States
Message 834546 - Posted: 26 Nov 2008, 17:15:10 UTC

11/26/2008 9:10:11 AM|SETI@home|Scheduler request failed: HTTP gateway timeout

I'm getting this all the time right now. I've got WU to report and need to download more. All is not well in Berkeley.
ID: 834546 · Report as offensive
Rudy
Volunteer tester

Send message
Joined: 23 Jun 99
Posts: 189
Credit: 794,998
RAC: 0
Canada
Message 834550 - Posted: 26 Nov 2008, 17:25:45 UTC - in response to Message 834532.  

Why aren't the problems mentioned on the server status page?


They are, it just not a bright red box.

If you look at the results recieved in the last hour it is down fom it's usual 50K to about 10K. People are having trouble downloading.

If you look at the astropulse read to send, the number is usualy close to zero. Since it is not people are not able to download work either.

Since the server statues look fine, it probably is some sort of network issue.

I find looking at the numbers on the status page (cricket, scarecrow graphs) much more useful that looking for red or orange boxes.
ID: 834550 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65746
Credit: 55,293,173
RAC: 49
United States
Message 834563 - Posted: 26 Nov 2008, 17:51:22 UTC - in response to Message 834550.  

Why aren't the problems mentioned on the server status page?


They are, it just not a bright red box.

If you look at the results received in the last hour it is down from it's usual 50K to about 10K. People are having trouble downloading.

If you look at the astropulse read to send, the number is usually close to zero. Since it is not people are not able to download work either.

Since the server statues look fine, it probably is some sort of network issue.

I find looking at the numbers on the status page (cricket, scarecrow graphs) much more useful that looking for red or orange boxes.

Speaking of the scarecrow, I mentioned Seti was having a bit of trouble over on the Boinc Forums(Berkeleys other doorway) and up He popped, Sorry Satan You didn't get a chance to pop up there.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 834563 · Report as offensive
Profile SATAN
Avatar

Send message
Joined: 27 Aug 06
Posts: 835
Credit: 2,129,006
RAC: 0
United Kingdom
Message 834569 - Posted: 26 Nov 2008, 18:25:40 UTC

Looks like things are slowly changing.

Wed Nov 26 18:48:27 2008|SETI@home|Sending scheduler request: Requested by user. Requesting 0 seconds of work, reporting 23 completed tasks
Wed Nov 26 18:49:32 2008|SETI@home|Scheduler request completed: got 0 new tasks
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 03no08af.27662.8657.9.8.139_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08aa.31968.22158.6.8.103_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.199_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 03no08af.27662.8657.9.8.166_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.161_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08aa.31968.22158.6.8.110_1 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 03no08af.27662.8657.9.8.99_1 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.201_1 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.164_1 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 03no08af.27662.8657.9.8.104_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08aa.31968.22158.6.8.113_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 03no08ad.28571.12342.13.8.12_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.243_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 03no08af.27662.8657.9.8.201_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.193_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008|SETI@home|Message from server: Completed result 04no08ab.28936.2117.5.8.245_0 refused: result already reported as success
Wed Nov 26 18:49:32 2008||General prefs: from SETI@home (last modified 26-Nov-2008 17:11:14)
Wed Nov 26 18:49:32 2008||Host location: none
Wed Nov 26 18:49:32 2008||General prefs: using your defaults
Wed Nov 26 18:49:32 2008||Reading preferences override file
Wed Nov 26 18:49:32 2008||Preferences limit memory usage when active to 10240.00MB
Wed Nov 26 18:49:32 2008||Preferences limit memory usage when idle to 10240.00MB
Wed Nov 26 18:49:32 2008||Preferences limit disk usage to 200.00GB
Wed Nov 26 18:49:32 2008||Preferences limit # CPUs to 2

ID: 834569 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 834576 - Posted: 26 Nov 2008, 18:44:03 UTC



see here: Message boards : Technical News : Nefarious Designs, Inc. (Nov 25 2008)



Message 834562 - Posted 26 Nov 2008 17:49:42 UTC - in response to Message 834555.

Yeah.. we can collect up to 100s of gigabytes a day. We'd need to be transporting data at (very) roughly 20Mbits a second 24/7 to keep up.

We don't have that spare bandwidth, and Arecibo surely does not.

About 10 years ago the ENTIRE connection between Arecibo and the rest of the world was a single 56.6 modem.

It's much better now, but still not quite up to state-of-the-art speed.

- Matt




BOINC Wiki . . .

Science Status Page . . .
ID: 834576 · Report as offensive
Profile [B^S] madmac
Volunteer tester
Avatar

Send message
Joined: 9 Feb 04
Posts: 1175
Credit: 4,754,897
RAC: 0
United Kingdom
Message 834579 - Posted: 26 Nov 2008, 18:48:37 UTC

Now getting upload failures connect failed messages and also servers may be temporarily down
ID: 834579 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 834580 - Posted: 26 Nov 2008, 18:50:24 UTC - in response to Message 834579.  
Last modified: 26 Nov 2008, 18:50:47 UTC

Now getting upload failures connect failed messages and also servers may be temporarily down


> same here

. . . been like that for awhile now - Internet [My Side] is fine - though domebody's goin' to kick the box soon ;)

ed.sp
BOINC Wiki . . .

Science Status Page . . .
ID: 834580 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65746
Credit: 55,293,173
RAC: 49
United States
Message 834582 - Posted: 26 Nov 2008, 18:58:15 UTC - in response to Message 834580.  

Now getting upload failures connect failed messages and also servers may be temporarily down


> same here

. . . been like that for awhile now - Internet [My Side] is fine - though somebody's goin' to kick the box soon ;)

ed.sp

Well We can always threaten to feed some raw Turkey to the server, It might reset all on Its own(Turkeys are reputed to be the dumbest birds on earth). ;)
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 834582 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 834585 - Posted: 26 Nov 2008, 19:04:42 UTC - in response to Message 834582.  

Now getting upload failures connect failed messages and also servers may be temporarily down


> same here

. . . been like that for awhile now - Internet [My Side] is fine - though somebody's goin' to kick the box soon ;)

ed.sp

Well We can always threaten to feed some raw Turkey to the server
It might reset all on Its own(Turkeys are reputed to be the dumbest birds on earth). ;)


;)))

eh Joker - 'ave a good Holiday . . .


BOINC Wiki . . .

Science Status Page . . .
ID: 834585 · Report as offensive
Profile dnolan
Avatar

Send message
Joined: 30 Aug 01
Posts: 1228
Credit: 47,779,411
RAC: 32
United States
Message 834588 - Posted: 26 Nov 2008, 19:08:35 UTC - in response to Message 834582.  

(Turkeys are reputed to be the dumbest birds on earth). ;)


This is really just a myth, at least according to this guy...

-Dave
ID: 834588 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65746
Credit: 55,293,173
RAC: 49
United States
Message 834605 - Posted: 26 Nov 2008, 20:04:41 UTC - in response to Message 834585.  

Now getting upload failures connect failed messages and also servers may be temporarily down


> same here

. . . been like that for awhile now - Internet [My Side] is fine - though somebody's goin' to kick the box soon ;)

ed.sp

Well We can always threaten to feed some raw Turkey to the server
It might reset all on Its own(Turkeys are reputed to be the dumbest birds on earth). ;)


;)))

eh Joker - 'ave a good Holiday . . .


Thanks Dr. C.E.T.I., I will indeed, As I don't have to cook on the 27th and I get some free food to bring home too(Leftovers, Yum). :)
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 834605 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 834606 - Posted: 26 Nov 2008, 20:11:02 UTC - in response to Message 834478.  

Last night around 9 PM MST, the whole website was unavailable, I am guessing that something that directs traffic for the site went down.
...

That was very interesting, all attempts to get something from http://setiathome.berkeley.edu/ were being redirected to https://setisvn.ssl.berkeley.edu/ and getting a security failure.
                                                                Joe
ID: 834606 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8962
Credit: 12,678,685
RAC: 0
United States
Message 834613 - Posted: 26 Nov 2008, 20:25:05 UTC

Switching to ABC for a cpl days to offer some relief to the server. My due dates aren't until December anyways


ID: 834613 · Report as offensive
Alinator
Volunteer tester

Send message
Joined: 19 Apr 05
Posts: 4178
Credit: 4,647,982
RAC: 0
United States
Message 834617 - Posted: 26 Nov 2008, 20:37:55 UTC

Hmmmm...

This doesn't look like a great way to be leading into a 4 day weekend! :-(

History has shown that even when the project is running stable going into Thanksgiving, it usually is a mess by the Monday after. ;-)

I don't even want to think about what next week may bring! :-D

Alinator

ID: 834617 · Report as offensive
Previous · 1 . . . 9 · 10 · 11 · 12 · 13 · Next

Message boards : Number crunching : Panic Mode On (10) Server problems


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.