Panic Mode On (117) Server Problems?

Message boards : Number crunching : Panic Mode On (117) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 38 · Next

AuthorMessage
Stephen "Heretic" Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 4780
Credit: 164,066,427
RAC: 240,255
Australia
Message 2015388 - Posted: 14 Oct 2019, 5:46:15 UTC

. . Hmmm, I don't suppose there is anyone who would care to do something with those 4 ancient tapes that have been hanging about on the splitters since Noah wore shorts? Two very old Blc22 and 2 x very old Blc34 tapes that have shown no sign of starting in the many months they have been loitering on the system. Maybe it is time to either kick start them or toss them to the weeds.

. . Just a thought ...

Stephen

? ?
ID: 2015388 · Report as offensive     Reply Quote
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 2833
Credit: 907,761,829
RAC: 1,735,708
Canada
Message 2015535 - Posted: 15 Oct 2019, 19:55:17 UTC
Last modified: 15 Oct 2019, 19:56:12 UTC

Dr. Korpela indicated that he just may have found and fixed the Sunday unresponsive scheduler issue. Let's hope we're back to merely one weekly outage...
“Never doubt that a small group of thoughtful, committed citizens can change the world; indeed, it's the only thing that ever has.”
---Margaret Mead
ID: 2015535 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 8246
Credit: 518,116,307
RAC: 396,154
Panama
Message 2015536 - Posted: 15 Oct 2019, 19:58:43 UTC - in response to Message 2015535.  

Dr. Korpela indicated that he just may have found and fixed the Sunday unresponsive scheduler issue. Let's hope we're back to merely one weekly outage...


ID: 2015536 · Report as offensive     Reply Quote
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 620
Credit: 2,036,031
RAC: 2,649
United States
Message 2015537 - Posted: 15 Oct 2019, 20:28:14 UTC - in response to Message 2015535.  

Dr. Korpela indicated that he just may have found and fixed the Sunday unresponsive scheduler issue. Let's hope we're back to merely one weekly outage...


YES!! good work Dr. Korpela. and thanks to Mr. Kevvy for bringing us the news.
ID: 2015537 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 3906
Credit: 231,215,255
RAC: 376,465
United States
Message 2015549 - Posted: 15 Oct 2019, 22:34:49 UTC - in response to Message 2015536.  

Dr. Korpela indicated that he just may have found and fixed the Sunday unresponsive scheduler issue. Let's hope we're back to merely one weekly outage...



+1
A proud member of the OFA (Old Farts Association)
"Over the hill? WHAT Hill? I don't REMEMBER any hill...." (from a bumper sticker I bought at a truck stop).
ID: 2015549 · Report as offensive     Reply Quote
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1537
Credit: 193,809,506
RAC: 475,388
United States
Message 2015555 - Posted: 15 Oct 2019, 23:24:00 UTC - in response to Message 2015536.  

Dr. Korpela indicated that he just may have found and fixed the Sunday unresponsive scheduler issue. Let's hope we're back to merely one weekly outage...


As Einstein goes into mourning .... :)
ID: 2015555 · Report as offensive     Reply Quote
Stephen "Heretic" Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 4780
Credit: 164,066,427
RAC: 240,255
Australia
Message 2015557 - Posted: 16 Oct 2019, 0:01:47 UTC - in response to Message 2015535.  

Dr. Korpela indicated that he just may have found and fixed the Sunday unresponsive scheduler issue. Let's hope we're back to merely one weekly outage...


. . Hip, Hip, Hooray!

Stephen

:)
ID: 2015557 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11865
Credit: 184,124,184
RAC: 234,190
Australia
Message 2015581 - Posted: 16 Oct 2019, 8:34:23 UTC

The file deleters don't appear to be doing much since the outage- their backlog continues to grow , and the splitter output continues to be poor.
Grant
Darwin NT
ID: 2015581 · Report as offensive     Reply Quote
Stephen "Heretic" Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 4780
Credit: 164,066,427
RAC: 240,255
Australia
Message 2015661 - Posted: 16 Oct 2019, 23:19:01 UTC

. . I haven't been monitoring deletions but since the big Sunday outage nearly two weeks ago my uploads/ downloads and work requests have been really, really slllloooooowwww! Completely stalled now

Stephen

:(
ID: 2015661 · Report as offensive     Reply Quote
Profile Wiggo "Democratic Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 17200
Credit: 240,022,955
RAC: 176,198
Australia
Message 2015681 - Posted: 17 Oct 2019, 3:26:46 UTC - in response to Message 2015661.  

. . I haven't been monitoring deletions but since the big Sunday outage nearly two weeks ago my uploads/ downloads and work requests have been really, really slllloooooowwww! Completely stalled now

Stephen :(
No such problems at all like that here, have you done a modem/router/switch reboot since then?

Cheers.
ID: 2015681 · Report as offensive     Reply Quote
Stephen "Heretic" Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 4780
Credit: 164,066,427
RAC: 240,255
Australia
Message 2015692 - Posted: 17 Oct 2019, 7:42:38 UTC - in response to Message 2015681.  

. . I haven't been monitoring deletions but since the big Sunday outage nearly two weeks ago my uploads/ downloads and work requests have been really, really slllloooooowwww! Completely stalled now

Stephen :(
No such problems at all like that here, have you done a modem/router/switch reboot since then?

Cheers.


. . It's worst on the Windows box, seriously thinking of mutating it into a Linux box as well. It's the only one that stalls. I have done a complete shutdown of boinc/manager and it seems a little better now. I will monitor progress ...

Stephen

:(
ID: 2015692 · Report as offensive     Reply Quote
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1381
Credit: 10,859,076
RAC: 6,253
New Zealand
Message 2015695 - Posted: 17 Oct 2019, 7:51:10 UTC - in response to Message 2015692.  

. . I haven't been monitoring deletions but since the big Sunday outage nearly two weeks ago my uploads/ downloads and work requests have been really, really slllloooooowwww! Completely stalled now

Stephen :(
No such problems at all like that here, have you done a modem/router/switch reboot since then?

Cheers.


. . It's worst on the Windows box, seriously thinking of mutating it into a Linux box as well. It's the only one that stalls. I have done a complete shutdown of boinc/manager and it seems a little better now. I will monitor progress ...

Stephen

:(

Pleased things are looking on the up for you. If you were to convert the machine to Linux it would certainly boost your RAC.
ID: 2015695 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11865
Credit: 184,124,184
RAC: 234,190
Australia
Message 2015820 - Posted: 18 Oct 2019, 5:42:31 UTC

The servers have certainly been having issues lately.
After the outage the deleters weren't doing much at all. Once they got their act together and cleared the backlog, the file purger had issues clearing that backlog. After a day or so they managed to clear their backlog, and now the Awaiting validation is becoming backlogged (along with WU awaiting assimilation).
Grant
Darwin NT
ID: 2015820 · Report as offensive     Reply Quote
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 620
Credit: 2,036,031
RAC: 2,649
United States
Message 2015822 - Posted: 18 Oct 2019, 5:56:07 UTC - in response to Message 2015820.  

The servers have certainly been having issues lately.
After the outage the deleters weren't doing much at all. Once they got their act together and cleared the backlog, the file purger had issues clearing that backlog. After a day or so they managed to clear their backlog, and now the Awaiting validation is becoming backlogged (along with WU awaiting assimilation).


I've seen this too. I don't see any reason for it though, the hourly return rate is within reason. Something is off.
ID: 2015822 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11865
Credit: 184,124,184
RAC: 234,190
Australia
Message 2016454 - Posted: 24 Oct 2019, 5:08:27 UTC

Looks like another issue with the Validators again- a huge backlog there.
Grant
Darwin NT
ID: 2016454 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11865
Credit: 184,124,184
RAC: 234,190
Australia
Message 2016459 - Posted: 24 Oct 2019, 7:41:09 UTC

Validator backlog continues to grow- 500k over it's usual level & rising sharply.
Grant
Darwin NT
ID: 2016459 · Report as offensive     Reply Quote
Profile Sebastian M. Bobrecki
Volunteer tester

Send message
Joined: 7 Feb 02
Posts: 22
Credit: 35,695,670
RAC: 193,751
Poland
Message 2016492 - Posted: 24 Oct 2019, 16:47:54 UTC

If I remember correctly "Results returned and awaiting validation" are results awaiting for quorum. So it is not a server problem. I'm guessing that this could be due to lots of Arecibo shorties, which ends with "-9 result_overflow" and don't match between opt. and normal app., so they requires another wingman.
ID: 2016492 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11865
Credit: 184,124,184
RAC: 234,190
Australia
Message 2016545 - Posted: 25 Oct 2019, 5:25:05 UTC - in response to Message 2016492.  

If I remember correctly "Results returned and awaiting validation" are results awaiting for quorum.

It's a server issue- there's no way there are almost a million shorties out there waiting to be validated. For as much as we rage against results that take forever to be returned, the vast majority are returned within a few days.
The backlog has started to clear, but it's got a long way to go before it gets back down to it's usual levels.


Arecibo shorties, which ends with "-9 result_overflow"
Those aren't shorties, they are noise bombs, because they bomb out early (in a matter of seconds). Shorties are processed all the way through, but don't take nearly a long as an average mid range WU to process.
Grant
Darwin NT
ID: 2016545 · Report as offensive     Reply Quote
Profile Sebastian M. Bobrecki
Volunteer tester

Send message
Joined: 7 Feb 02
Posts: 22
Credit: 35,695,670
RAC: 193,751
Poland
Message 2016550 - Posted: 25 Oct 2019, 6:15:00 UTC - in response to Message 2016545.  

It's a server issue- there's no way there are almost a million shorties out there waiting to be validated. For as much as we rage against results that take forever to be returned, the vast majority are returned within a few days.
The backlog has started to clear, but it's got a long way to go before it gets back down to it's usual levels.
I think the behavior match with what I think before. I looked at logs of my clients and yesterday was a lot more Arecibo tasks than usual. Actually more Arecibo than blc. According to haveland it was about 500-600k more "awaiting validation" than usual. With over 120k results returned per hour it's not that much. I also saw yesterday that my validation inconclusive increased from usual below 100 to over 150. And I'm not a mega rac cruncher. Today I see it starts dropping with some increase in invalids.


Those aren't shorties, they are noise bombs, because they bomb out early (in a matter of seconds). Shorties are processed all the way through, but don't take nearly a long as an average mid range WU to process.
This is more a matter of naming convention. For me naming those few seconds tasks noise bombs makes a negative feelings (as noise is something we don't want). And actually they could be what we are all looking for. But as I wrote this is just a matter of naming convention.
ID: 2016550 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11865
Credit: 184,124,184
RAC: 234,190
Australia
Message 2016552 - Posted: 25 Oct 2019, 6:43:17 UTC - in response to Message 2016550.  

Those aren't shorties, they are noise bombs, because they bomb out early (in a matter of seconds). Shorties are processed all the way through, but don't take nearly a long as an average mid range WU to process.
This is more a matter of naming convention. For me naming those few seconds tasks noise bombs makes a negative feelings (as noise is something we don't want). And actually they could be what we are all looking for. But as I wrote this is just a matter of naming convention.
And it is important to follow convention in order to avoid confusion.
Grant
Darwin NT
ID: 2016552 · Report as offensive     Reply Quote
Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 38 · Next

Message boards : Number crunching : Panic Mode On (117) Server Problems?


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.