Panic Mode On (59) Server problems?

Message boards : Number crunching : Panic Mode On (59) Server problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 12 · Next

AuthorMessage
Blake Bonkofsky
Volunteer tester
Avatar

Send message
Joined: 29 Dec 99
Posts: 617
Credit: 46,383,149
RAC: 0
United States
Message 1164918 - Posted: 24 Oct 2011, 4:18:54 UTC - in response to Message 1164917.  

You mean all of the DB queries and such? I had that pop up for about 30 seconds. Possibly the connection to the DB was broken for a moment?
ID: 1164918 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1164920 - Posted: 24 Oct 2011, 4:21:13 UTC - in response to Message 1164918.  

You mean all of the DB queries and such? I had that pop up for about 30 seconds. Possibly the connection to the DB was broken for a moment?

Dunno, but glad somebody else witnessed it.....
I was pretty sure it was not on my end.

"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1164920 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1164935 - Posted: 24 Oct 2011, 5:11:23 UTC - in response to Message 1164894.  

That was a close one! Almost fell onto the second page!

Yeah......
But there's something funky goin' about.
There has been AP data available to split for a while now, but the AP splitters seem to have had trouble picking up on that. Usually, bandwidth maxxes out as soon as AP can be split.
And the last server status update shows MB ready to send dropping, yet some MB splitters are no longer active.

So......something's afoot, and we may have more to talk about soon other than the dang cache limits.

The file sizes look suspiciously like they're again trying to clean up the scraps...
                                                                   Joe
ID: 1164935 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1164938 - Posted: 24 Oct 2011, 5:16:14 UTC - in response to Message 1164935.  

That was a close one! Almost fell onto the second page!

Yeah......
But there's something funky goin' about.
There has been AP data available to split for a while now, but the AP splitters seem to have had trouble picking up on that. Usually, bandwidth maxxes out as soon as AP can be split.
And the last server status update shows MB ready to send dropping, yet some MB splitters are no longer active.

So......something's afoot, and we may have more to talk about soon other than the dang cache limits.

The file sizes look suspiciously like they're again trying to clean up the scraps...
                                                                   Joe

Ahh......see what you mean, Joe.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1164938 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1165320 - Posted: 25 Oct 2011, 18:37:18 UTC - in response to Message 1164938.  


So, did we have an early outage or was it another glitch?
Grant
Darwin NT
ID: 1165320 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1165344 - Posted: 25 Oct 2011, 20:56:16 UTC - in response to Message 1165321.  


So, did we have an early outage or was it another glitch?


I'd say it was an early and short outage, and not a glitch.

A long time ago Matt mentioned in one of his posts that Jeff was an early riser. My guess is the early start of the outage indicates he's tending things alone again.
                                                                   Joe
ID: 1165344 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22534
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1165425 - Posted: 26 Oct 2011, 7:16:12 UTC

Looking at the server stats page a few minutes ago (8am UK time, 26 October) I noticed the file sizes. Now I understood that the "tapes" were a standard 50GB, but this lot range from 0GB to 50GB. Are they clearing the shelves of all the odd bits or what??

01ap10ep 0.00 GB
01dc10ab 8.78 GB
01ja11ac 8.78 GB
02se10ad 8.53 GB
03ap10ak 7.91 GB
04mr11ae 8.78 GB
07ja10ae 8.53 GB
07no10aa 8.78 GB
08ap11ai 4.02 GB
08se11ab 50.20 GB
09mr11ab 8.78 GB
09se11ad 50.20 GB
09se11ae 50.20 GB
10mr11ad 8.03 GB
10se10ae 8.78 GB
14mr11ad 8.78 GB
14se10aa 8.53 GB
14se11ab 50.20 GB
15se11ad 50.20 GB
15se11ae 50.20 GB
17se11ab 41.66 GB
17se11ac 50.20 GB
19jl10aa 8.66 GB
19jl10ab 2.26 GB
20ap11ae 4.39 GB
20se11aa 50.20 GB
21au10ad 7.28 GB
21dc10ab 0.00 GB
22jl10ab 6.27 GB
23se11aa 50.20 GB
23se11ab 50.20 GB
25ap10ad 8.78 GB
25se11ab 50.20 GB
28mr10af 8.78 GB
29ap11af 4.77 GB
29au10ah 0.00 GB
30mr11ae 8.78 GB
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1165425 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1165430 - Posted: 26 Oct 2011, 7:24:57 UTC - in response to Message 1165425.  

Looking at the server stats page a few minutes ago (8am UK time, 26 October) I noticed the file sizes. Now I understood that the "tapes" were a standard 50GB, but this lot range from 0GB to 50GB. Are they clearing the shelves of all the odd bits or what??

I believe they could be clearing out ends of left over tapes. I'm wondering how it will take to burn through the tapes
ID: 1165430 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1165431 - Posted: 26 Oct 2011, 7:28:37 UTC - in response to Message 1165430.  

Looking at the server stats page a few minutes ago (8am UK time, 26 October) I noticed the file sizes. Now I understood that the "tapes" were a standard 50GB, but this lot range from 0GB to 50GB. Are they clearing the shelves of all the odd bits or what??

I believe they could be clearing out ends of left over tapes. I'm wondering how it will take to burn through the tapes

Might just be some housekeeping clearing out the offline storage...dunno.
V7 is gonna be rolled out at some point, but I am not sure that is really reliant on the datasets available to process, but rather the way in which they will be processed.

I have seen no news indicating that current data collection from Arecibo is not proceeding as usual.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1165431 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22534
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1165433 - Posted: 26 Oct 2011, 7:39:43 UTC

If v7 is as big a step as some suggest it would certainly make sense to clear the decks of the odds an sods.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1165433 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1165437 - Posted: 26 Oct 2011, 7:51:20 UTC

Perhaps data what wasn't able to be pulled off before is getting a run though.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1165437 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1165859 - Posted: 27 Oct 2011, 21:01:53 UTC - in response to Message 1165834.  

Replica DB is falling badly behind all of a sudden. As of now it's 2,406 seconds behind, and it increases fast.

Nobody dares mention the yellow one please.....

It's only 14 seconds behind now. Main DB is 648/sec as of 20:50:06
ID: 1165859 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1165860 - Posted: 27 Oct 2011, 21:06:56 UTC - in response to Message 1165859.  
Last modified: 27 Oct 2011, 21:10:36 UTC

Replica DB is falling badly behind all of a sudden. As of now it's 2,406 seconds behind, and it increases fast.

Nobody dares mention the yellow one please.....

It's only 14 seconds behind now. Main DB is 648/sec as of 20:50:06

Generation of daily stats dump files, anyone? Those timestamps are in local time, i.e. about 2 hours ago.

You can usually see those daily spikes in queries and replica backlog on Scarecrow's graphs, but he seems to be a bit busy right now.
ID: 1165860 · Report as offensive
Profile Link
Avatar

Send message
Joined: 18 Sep 03
Posts: 834
Credit: 1,807,369
RAC: 0
Germany
Message 1165955 - Posted: 28 Oct 2011, 9:53:55 UTC

I'm getting quite a few MD5 errors recently, my 3G connection might be part of the problem, but I can't change that... and everything else works, just some downloads from SETI don't. Anyway, is there any way to tell BOINC to retry to download the file on such error? I couldn't find any cc_config option for that...
ID: 1165955 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1165957 - Posted: 28 Oct 2011, 10:33:28 UTC - in response to Message 1165955.  
Last modified: 28 Oct 2011, 10:37:20 UTC

I'm getting quite a few MD5 errors recently, my 3G connection might be part of the problem, but I can't change that... and everything else works, just some downloads from SETI don't. Anyway, is there any way to tell BOINC to retry to download the file on such error? I couldn't find any cc_config option for that...

No, But as long as you don't report the failed task, you can shut down Boinc, edit your client_state.xml and reset it back to downloading, first of all delete the failed download from your project folder,
then make the <file_info> section look like this (the status will need to be 0, and you'll need to delete the MD5_FAILED message):

<file_info>
    <name>17jl11ac.2358.153186.13.10.196</name>
    <nbytes>375345.000000</nbytes>
    <max_nbytes>0.000000</max_nbytes>
    <md5_cksum>3626d3dc15ec5ff72f49f892f8493bb3</md5_cksum>
    <status>0</status>
    <url>http://boinc2.ssl.berkeley.edu/sah/download_fanout/1e/17jl11ac.2358.153186.13.10.196</url>
</file_info>


The result section will need to be edited until it looks like this:

<result>
    <name>17jl11ac.2358.153186.13.10.196_2</name>
    <final_cpu_time>0.000000</final_cpu_time>
    <final_elapsed_time>0.000000</final_elapsed_time>
    <exit_status>0</exit_status>
    <state>2</state>
    <platform>windows_intelx86</platform>
    <version_num>610</version_num>
    <plan_class>cuda_fermi</plan_class>
    <wu_name>17jl11ac.2358.153186.13.10.196</wu_name>
    <report_deadline>1321982214.000000</report_deadline>
    <received_time>1319028235.992689</received_time>
    <file_ref>
        <file_name>17jl11ac.2358.153186.13.10.196_2_0</file_name>
        <open_name>result.sah</open_name>
    </file_ref>
</result>


Edit: Advanced Users Only.

Claggy
ID: 1165957 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1165958 - Posted: 28 Oct 2011, 10:33:48 UTC

Have the Crickets run out of green pixels?
ID: 1165958 · Report as offensive
Blake Bonkofsky
Volunteer tester
Avatar

Send message
Joined: 29 Dec 99
Posts: 617
Credit: 46,383,149
RAC: 0
United States
Message 1165959 - Posted: 28 Oct 2011, 10:41:17 UTC - in response to Message 1165958.  

I was thinking the same thing! My ups and downs are still going through just fine, but the cricket is just, blank!
ID: 1165959 · Report as offensive
__W__
Avatar

Send message
Joined: 28 Mar 09
Posts: 116
Credit: 5,943,642
RAC: 0
Germany
Message 1165960 - Posted: 28 Oct 2011, 11:00:14 UTC - in response to Message 1165958.  

Have the Crickets run out of green pixels?

Seems to be a problem of the Cricket software or the connection to the webinterface and gladly not a problem of the routers. At all monitored routers Cricket comes to a stop at the same time.

__W__
_______________________________________________________________________________
ID: 1165960 · Report as offensive
Profile Link
Avatar

Send message
Joined: 18 Sep 03
Posts: 834
Credit: 1,807,369
RAC: 0
Germany
Message 1165961 - Posted: 28 Oct 2011, 11:05:28 UTC - in response to Message 1165957.  

I've done that few times like that, but in normally BOINC asks for new tasks after such failed download ASAP, so you have just five minutes (actually less, specially if there was a HTTP error) to catch that... so not really possible for people who do not watch BOINC Manager all the time.

Well, I'll have to live with that I guess...
ID: 1165961 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 1166019 - Posted: 28 Oct 2011, 17:09:08 UTC - in response to Message 1165958.  

Have the Crickets run out of green pixels?

The blue pixels are missing as well. ;-)
ID: 1166019 · Report as offensive
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 12 · Next

Message boards : Number crunching : Panic Mode On (59) Server problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.