Panic Mode On (88) Server Problems?

Message boards : Number crunching : Panic Mode On (88) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 3 · 4 · 5 · 6 · 7 · 8 · 9 . . . 21 · Next

AuthorMessage
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8240
Credit: 14,654,533
RAC: 20
United States
Message 1515381 - Posted: 13 May 2014, 7:14:10 UTC

Looks like the Transitioners on Vader have crashed.
Other functions on Vader show green.
Reboot it in the morning, I guess.
Donald
Infernal Optimist / Submariner, retired
ID: 1515381 · Report as offensive
Profile S@NL Etienne Dokkum
Volunteer tester
Avatar

Send message
Joined: 11 Jun 99
Posts: 212
Credit: 43,822,095
RAC: 0
Netherlands
Message 1515398 - Posted: 13 May 2014, 8:22:59 UTC - in response to Message 1515381.  

Looks like the Transitioners on Vader have crashed.
Other functions on Vader show green.
Reboot it in the morning, I guess.


they had the outage yesterday because the guys were out today so could be a while...
ID: 1515398 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13739
Credit: 208,696,464
RAC: 304
Australia
Message 1515399 - Posted: 13 May 2014, 8:27:35 UTC - in response to Message 1515381.  

Looks like the Transitioners on Vader have crashed.
Other functions on Vader show green.
Reboot it in the morning, I guess.

Maybe what's causing the lack of work production?
Plenty to be split, and splitters show green, but even though the ready-to-send buffer is falling steadily & it's well below the usual threshold for pumping out new work, nothing is being split.
Grant
Darwin NT
ID: 1515399 · Report as offensive
Dave Stegner
Volunteer tester
Avatar

Send message
Joined: 20 Oct 04
Posts: 540
Credit: 65,583,328
RAC: 27
United States
Message 1515402 - Posted: 13 May 2014, 8:42:51 UTC

If I remember correctly, split work does not become "ready to send" without the Transitioners.
Dave

ID: 1515402 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13739
Credit: 208,696,464
RAC: 304
Australia
Message 1515407 - Posted: 13 May 2014, 8:58:31 UTC - in response to Message 1515402.  

If I remember correctly, split work does not become "ready to send" without the Transitioners.

That'd do it then.
Grant
Darwin NT
ID: 1515407 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14652
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1515424 - Posted: 13 May 2014, 10:54:06 UTC

Looks like an automated recovery script has started to kick in. Unless one of the out-of-town trips has gone to a completely different timezone, or somebody has insomnia.
ID: 1515424 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1515449 - Posted: 13 May 2014, 13:45:44 UTC

The SSP has always confused me, but now I have a good reason to ask about it.

The AP splitters all show not running, but the AP files still show green. What I don't get is what the number in parentheses after each file means. Is it how many channels are left to split in that file, how many are in progress, or how many are done?

The five AP files that don't say "done" all have a (14) after them.
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1515449 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14652
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1515453 - Posted: 13 May 2014, 14:02:30 UTC - in response to Message 1515449.  

The SSP has always confused me, but now I have a good reason to ask about it.

The AP splitters all show not running, but the AP files still show green. What I don't get is what the number in parentheses after each file means. Is it how many channels are left to split in that file, how many are in progress, or how many are done?

The five AP files that don't say "done" all have a (14) after them.

There's a legend at the bottom of the tape list:

[dark green]: channels in progress
[light green]: completed channels
[grey]: channels with errors
(each file contains data for 14 channels)

{gripe: why doesn't IMG accept gifs?}

'in progress' means 'started, but not finished' - like a client task waiting to run, they can sometimes stay like that for days. An outrage often seems to interrupt the tapes in progress, and the splitters start on new ones when the project is woken up again. I'm guessing these (14) just need to do a bit of final housekeeping before they're properly finished. The number seems to be the number of channels - out of 14 - which have at least been started - after all, it says (1) as soon as the first channel goes dark green.
ID: 1515453 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1515492 - Posted: 13 May 2014, 16:21:07 UTC - in response to Message 1515453.  
Last modified: 13 May 2014, 16:22:05 UTC

The SSP has always confused me, but now I have a good reason to ask about it.

The AP splitters all show not running, but the AP files still show green. What I don't get is what the number in parentheses after each file means. Is it how many channels are left to split in that file, how many are in progress, or how many are done?

The five AP files that don't say "done" all have a (14) after them.

There's a legend at the bottom of the tape list:

[dark green]: channels in progress
[light green]: completed channels
[grey]: channels with errors
(each file contains data for 14 channels)

{gripe: why doesn't IMG accept gifs?}

'in progress' means 'started, but not finished' - like a client task waiting to run, they can sometimes stay like that for days. An outrage often seems to interrupt the tapes in progress, and the splitters start on new ones when the project is woken up again. I'm guessing these (14) just need to do a bit of final housekeeping before they're properly finished. The number seems to be the number of channels - out of 14 - which have at least been started - after all, it says (1) as soon as the first channel goes dark green.

I am certainly aware of the legend, but it doesn't adequately answer the question. "At least started" does seem to fit the evidence, but rather limits the usefulness of it.
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1515492 · Report as offensive
Profile KWSN Ekky Ekky Ekky
Avatar

Send message
Joined: 25 May 99
Posts: 944
Credit: 52,956,491
RAC: 67
United Kingdom
Message 1515685 - Posted: 13 May 2014, 21:28:58 UTC

Upload problems at present? Everything looks OK but may be a temporary problem?

ID: 1515685 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 783
Credit: 348,560,338
RAC: 223
United Kingdom
Message 1515688 - Posted: 13 May 2014, 21:39:34 UTC - in response to Message 1515685.  
Last modified: 13 May 2014, 22:01:50 UTC

Upload problems at present? Everything looks OK but may be a temporary problem?

Might be regional, I don't see anything obvious on the cricket graphs but my last contact was 20:59:47 UTC.
[Edit] Tja! Things started uploading 5 seconds after my original post! [/Edit]
ID: 1515688 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14652
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1515700 - Posted: 13 May 2014, 22:10:45 UTC - in response to Message 1515688.  

I came home to find a few tasks had failed to upload: they cleared at the first retry attempt. One host also hadn't reported for a while: again, that cleared by itself the next time it tried.

Seems to have been a short-lived network glitch, out of the way now.
ID: 1515700 · Report as offensive
Profile KWSN Ekky Ekky Ekky
Avatar

Send message
Joined: 25 May 99
Posts: 944
Credit: 52,956,491
RAC: 67
United Kingdom
Message 1515723 - Posted: 13 May 2014, 23:19:39 UTC

Upload problems seem to be intermittent. As suggested, probably local.

ID: 1515723 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1515886 - Posted: 14 May 2014, 8:36:50 UTC

So this one was interesting.. wu1497896368. It is a B3_P1. When I get those, I throw them at 7-zip to see what their compression level is. If it ends up compressing to under 70% of the original size.. then it has always been 100% blanked. This WU though, compressed to.. 40-something% of the original size, but the stderr shows ZERO blanking. Weeeeird. Maybe those six repetitive pulses were identical and gave LZMA something to compress.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1515886 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1516034 - Posted: 14 May 2014, 16:33:25 UTC - in response to Message 1515886.  

So this one was interesting.. wu1497896368. It is a B3_P1. When I get those, I throw them at 7-zip to see what their compression level is. If it ends up compressing to under 70% of the original size.. then it has always been 100% blanked. This WU though, compressed to.. 40-something% of the original size, but the stderr shows ZERO blanking. Weeeeird. Maybe those six repetitive pulses were identical and gave LZMA something to compress.

But it's not significantly compressible with zip or gzip or V.42bis modem compression. The difference appears to be because 7-zip normally uses a dictionary size of 1M or larger. Even bzip2 achieves some compression using 900k, but 7-zip in its "Fastest" mode with a 64k dictionary size doesn't.

The AP apps actually only consider extremely short 32 byte (128 samples) sequences when checking whether blanking is needed. That's very different from how a lossless compression algorithm deals with the data, so having the correlation break down sometimes is inevitable.
                                                                   Joe
ID: 1516034 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1516141 - Posted: 14 May 2014, 19:09:19 UTC - in response to Message 1516034.  

Noted. I was just pointing out that it was the first time I had seen high compression on a B3_P1 that ended up not only not being 100% blanked, but had zero blanking.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1516141 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34802
Credit: 261,360,520
RAC: 489
Australia
Message 1517748 - Posted: 18 May 2014, 1:37:46 UTC

I see that the transitioners have been down again for a while again and the ready to send is dropping quickly.

Cheers.
ID: 1517748 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13739
Credit: 208,696,464
RAC: 304
Australia
Message 1517767 - Posted: 18 May 2014, 2:11:48 UTC - in response to Message 1517748.  

Hopefully whatever kicked them back to life the last time around will do so again. Even with the recently increased GPU work cache, it won't take long to run out of work once the ready-to-send buffer is all gone.
Grant
Darwin NT
ID: 1517767 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1517778 - Posted: 18 May 2014, 3:00:53 UTC

Yeah, the transitioners haven't been right for more than a week, in one way or another. Case in point, there are 5 MB splitters, but the status shows 10 channels being split. There are 7 AP splitters, which are shown not running, but the status shows 7 channels in progress of splitting. Whenever it all messed up, a week ago or whenever it was, those channels being split at the time have shown as orphans ever since?
ID: 1517778 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1517787 - Posted: 18 May 2014, 4:13:25 UTC
Last modified: 18 May 2014, 4:17:39 UTC

Transitioners are running again as there are now over 350,000 with ready to send and it is splitting at 230.2308 per second. The automatic restart script or whatever it is must have kicked in. In regards to the APs I think you are right Jim. I am sure this will be sorted when you tapes are added
ID: 1517787 · Report as offensive
Previous · 1 . . . 3 · 4 · 5 · 6 · 7 · 8 · 9 . . . 21 · Next

Message boards : Number crunching : Panic Mode On (88) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.