Panic Mode On (10) Server problems

Message boards : Number crunching : Panic Mode On (10) Server problems
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 13 · Next

AuthorMessage
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 829606 - Posted: 12 Nov 2008, 13:39:26 UTC


. . . Database/file status [right cha' be Mark . . .]

Results ready to send 0 0 16m
Current result creation rate 12.8182/sec 0.0769/sec 7m
Results out in the field 3,531,444 172,540 16m
Results received in last hour 57,337 809 0m
Result turnaround time (last hour average) 53.49 hours 215.60 hours 0m
Results returned and awaiting validation 2,510,686 163,047 16m
Workunits waiting for validation 53 1 16m
Workunits waiting for assimilation 533 8 16m
Workunit files waiting for deletion 621,494 88 16m
Result files waiting for deletion 622,890 60 16m
Workunits waiting for db purging 274,889 9,035 16m
Results waiting for db purging 1,207,101 24,167 16m
Transitioner backlog (hours) 0 0m

BOINC Wiki . . .

Science Status Page . . .
ID: 829606 · Report as offensive
Rudy
Volunteer tester

Send message
Joined: 23 Jun 99
Posts: 189
Credit: 794,998
RAC: 0
Canada
Message 829633 - Posted: 12 Nov 2008, 16:03:46 UTC - in response to Message 829594.  

There is a problem right now......
Server status shows 0 ready to send, and the Scarecrow Graphs also show ready to send has dwindled and the splitter activity has slowed way down......

Something has tanked again....probably until today's maintenance outage.


The problem started on the 10th when something went wrong in the deletion process. It just took until now for the queues to fill and the splitters to slow down.

Server Bruno who has been acting up since his OS upgrade a few weeks back. All the details are in the tech news threads.

It likely will extend the shutdown since I recall reading that they like the queues to be empty before they do a backup during the shutdown.
ID: 829633 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 829634 - Posted: 12 Nov 2008, 16:11:10 UTC - in response to Message 829633.  

There is a problem right now......
Server status shows 0 ready to send, and the Scarecrow Graphs also show ready to send has dwindled and the splitter activity has slowed way down......

Something has tanked again....probably until today's maintenance outage.


The problem started on the 10th when something went wrong in the deletion process. It just took until now for the queues to fill and the splitters to slow down.

Server Bruno who has been acting up since his OS upgrade a few weeks back. All the details are in the tech news threads.

It likely will extend the shutdown since I recall reading that they like the queues to be empty before they do a backup during the shutdown.

You could indeed be correct on both counts......
The 7 Day Scarecrow Graphs do show the 'awaiting deletion' stats continuously rising since sometime on the 10th....and there is only so much storage space available....
Good observation.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 829634 · Report as offensive
Profile Byron S Goodgame
Volunteer tester
Avatar

Send message
Joined: 16 Jan 06
Posts: 1145
Credit: 3,936,993
RAC: 0
United States
Message 829689 - Posted: 13 Nov 2008, 1:14:43 UTC

Just curious if anyone has started getting work yet since after the outage?
ID: 829689 · Report as offensive
JohnAlton
Avatar

Send message
Joined: 28 Aug 01
Posts: 54
Credit: 164,417,653
RAC: 369
United States
Message 829719 - Posted: 13 Nov 2008, 2:33:24 UTC - in response to Message 829689.  

Not a drop here...
ID: 829719 · Report as offensive
JohnAlton
Avatar

Send message
Joined: 28 Aug 01
Posts: 54
Credit: 164,417,653
RAC: 369
United States
Message 829743 - Posted: 13 Nov 2008, 3:16:29 UTC - in response to Message 829719.  
Last modified: 13 Nov 2008, 3:16:46 UTC

Not a drop here...


UPDATE: 10 drops received. Well, it's a start.
ID: 829743 · Report as offensive
Profile Byron S Goodgame
Volunteer tester
Avatar

Send message
Joined: 16 Jan 06
Posts: 1145
Credit: 3,936,993
RAC: 0
United States
Message 829747 - Posted: 13 Nov 2008, 3:21:21 UTC - in response to Message 829743.  

Not a drop here...


UPDATE: 10 drops received. Well, it's a start.



Flowing here again.

ID: 829747 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 829755 - Posted: 13 Nov 2008, 4:32:25 UTC

Not a drop available for my iMac.

ID: 829755 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 829759 - Posted: 13 Nov 2008, 5:10:58 UTC

Just got a few for one of my rigs, and the demand shown on the Cricket Graph seems to be slowing a bit....

Sooooooooo....hopefully that means the splitters can play a little catch-up and WUs will be in a bit more abundant supply soon.....
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 829759 · Report as offensive
Profile [B^S] madmac
Volunteer tester
Avatar

Send message
Joined: 9 Feb 04
Posts: 1175
Credit: 4,754,897
RAC: 0
United Kingdom
Message 829781 - Posted: 13 Nov 2008, 8:16:55 UTC

Now having problems uploading getting connect failed messages or project servers may be temporarily down messages.
ID: 829781 · Report as offensive
Profile KWSN Ekky Ekky Ekky
Avatar

Send message
Joined: 25 May 99
Posts: 944
Credit: 52,956,491
RAC: 67
United Kingdom
Message 829798 - Posted: 13 Nov 2008, 9:51:10 UTC - in response to Message 829781.  

Yes. Upload problems look pretty general at present. Downloads have been going well.
No alarms on the server status page, though.

ID: 829798 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 829800 - Posted: 13 Nov 2008, 10:01:24 UTC

Have a look at the graph in message 825377 (this thread, 31 October). We're up against the top limit on downloads again (Matt found some AP data in a cellar somewhere) - you can never get uploads through cleanly when the pipe is full of downloads. Just try getting into a football stadium when a match has just ended.
ID: 829800 · Report as offensive
Profile [B^S] madmac
Volunteer tester
Avatar

Send message
Joined: 9 Feb 04
Posts: 1175
Credit: 4,754,897
RAC: 0
United Kingdom
Message 829822 - Posted: 13 Nov 2008, 12:06:14 UTC
Last modified: 13 Nov 2008, 12:56:55 UTC

must be unlucky with my timing then, now have 4 waiting to be uploaded even though one says 100%. Will wait they will go up sometime hopefully today.It is amazing that this message can get through but not my wus
ID: 829822 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 829824 - Posted: 13 Nov 2008, 12:08:41 UTC - in response to Message 829822.  
Last modified: 13 Nov 2008, 12:09:26 UTC

must be unlucjy with my timing then, now have 4 waiting to be uploaded even though one says 100%. Will wait they will go up sometime hopefully today.It is amazing that this message can get through but not my wus

The forums are on a different server than the up/downloads.......thank goodness.
At least then we can still compare notes about what the workload servers are doing........
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 829824 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 830769 - Posted: 15 Nov 2008, 7:57:09 UTC

Oops......
According to the server status page, Vader has left the building.........just in time for the weekend........
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 830769 · Report as offensive
Profile Vipin Palazhi
Avatar

Send message
Joined: 29 Feb 08
Posts: 286
Credit: 167,386,578
RAC: 0
India
Message 830782 - Posted: 15 Nov 2008, 9:20:05 UTC

Just wondering... Why is it that the servers always go down during the weekends? It cant be a mere coincidence. Any comments?

Uploads and downloads seem to be ok, however, the pending credits list is building up. I am up to 22k now. Not really bothered though. This is not the first and this wont be the last, hehe...

Thinking about it, this is similar to the wedding vows: "To have and to hold, from this day forward (the day we signed up for seti), for better for worse (success in finding an ET signal), for richer or poorer (in terms of credits), in sickness and in health (server status), to love and to cherish (continue to crunch), till death us do part (hope the death is only on our part and not seti)."
______________

ID: 830782 · Report as offensive
Iona
Avatar

Send message
Joined: 12 Jul 07
Posts: 790
Credit: 22,438,118
RAC: 0
United Kingdom
Message 830795 - Posted: 15 Nov 2008, 11:26:56 UTC

I thought something had gone wrong, when a few results were reported at around 0630 local time and no credits were returned after about 20 minutes. Oh well, its better than having WUs sent out to a 3rd machine. Perhaps the reason that the servers seem to go wrong at the weekend, is because they miss people being there!



Don't take life too seriously, as you'll never come out of it alive!
ID: 830795 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 830797 - Posted: 15 Nov 2008, 11:50:29 UTC


. . . Database/file status

Results ready to send - 0 - 0 - 28m
Current result creation rate - NULL/sec - NULL/sec - 6m
Results out in the field - 3,400,242 - 148,672 - 28m
Results received in last hour - 46,894 - 694 - 0m
Result turnaround time (last hour average) - 66.04 hours - 210.31 hours - 0m
Results returned and awaiting validation - 2,748,804 - 159,540 - 28m
Workunits waiting for validation - 0 - 0 - 28m
Workunits waiting for assimilation - 40 - 0 - 28m
Workunit files waiting for deletion - 12 - 0 - 28m
Result files waiting for deletion - 0 - 0 - 28m
Workunits waiting for db purging - 469,262 - 7,124 - 28m
Results waiting for db purging - 966,038 - 19,881 - 28m
Transitioner backlog (hours) - 6 - 0m


BOINC Wiki . . .

Science Status Page . . .
ID: 830797 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 830801 - Posted: 15 Nov 2008, 12:39:08 UTC

The only part of Vader that is currently running is the download server, which does no good as there are no units to be sent out.

ID: 830801 · Report as offensive
Zap de Ridder
Volunteer tester

Send message
Joined: 9 Jan 00
Posts: 227
Credit: 1,468,844
RAC: 1
Netherlands
Message 830827 - Posted: 15 Nov 2008, 15:11:34 UTC - in response to Message 830801.  
Last modified: 15 Nov 2008, 15:18:18 UTC

I was just looking and while doing so the ones with two results ( validate state: initial ) got validated some minutes ago.
ID: 830827 · Report as offensive
Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 13 · Next

Message boards : Number crunching : Panic Mode On (10) Server problems


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.