Panic Mode On (100) Server Problems?

Message boards : Number crunching : Panic Mode On (100) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 29 · 30 · 31 · 32 · 33 · 34 · 35 · Next

AuthorMessage
Cavalary

Send message
Joined: 15 Jul 99
Posts: 71
Credit: 6,931,652
RAC: 2,565
Romania
Message 1731559 - Posted: 3 Oct 2015, 17:32:17 UTC

Yep, working for me now too.
ID: 1731559 · Report as offensive
Profile Kibble (KB7TIB)
Avatar

Send message
Joined: 6 Dec 99
Posts: 27
Credit: 9,868,303
RAC: 931
United States
Message 1731566 - Posted: 3 Oct 2015, 17:45:28 UTC - in response to Message 1731535.  

Two of three machines have been able to make contact this morning. Looks like Einstein is getting the benefits from one. Oh, well.[/size]
ID: 1731566 · Report as offensive
Profile Ageless
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 14848
Credit: 4,053,932
RAC: 1,016
Netherlands
Message 1731572 - Posted: 3 Oct 2015, 18:19:26 UTC

It's as if it's on a timer. What kind of timer does not allow access to the network for 18-20 hours a day?
Jord

According to Giorgo of the Ancient Astronaut Theorists I do not help with tech questions via private message. He's right: please use the forums for that.
ID: 1731572 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 25390
Credit: 48,631,406
RAC: 25,410
United States
Message 1731583 - Posted: 3 Oct 2015, 18:51:41 UTC - in response to Message 1731572.  

It's as if it's on a timer. What kind of timer does not allow access to the network for 18-20 hours a day?

A RAM buffer overflow timer!

With the system working
$ traceroute -I boinc.berkeley.edu
traceroute to boinc.berkeley.edu (208.68.240.115), 64 hops max, 72 byte packets
 1  * * *
 2  l100.lsanca-dsl-20.verizon-gni.net (71.108.177.1)  23.359 ms  23.182 ms  23.112 ms
 3  p0-2-2-5.lsanca-lcr-21.verizon-gni.net (130.81.35.32)  26.164 ms  28.747 ms  35.141 ms
 4  ae1-0.lax01-bb-rtr1.verizon-gni.net (130.81.199.90)  26.092 ms  24.303 ms  24.625 ms
 5  * * *
 6  0.ae5.br1.lax15.alter.net (140.222.225.135)  24.722 ms  24.374 ms  24.884 ms
 7  ae6.edge1.losangeles9.level3.net (4.68.62.169)  24.365 ms  24.890 ms  24.662 ms
 8  ae-3-80.ear1.losangeles1.level3.net (4.69.144.146)  25.726 ms  25.891 ms  25.620 ms
 9  cenic.ear1.losangeles1.level3.net (4.35.156.66)  25.590 ms  25.382 ms  25.355 ms
10  dc-svl-agg4--lax-agg6-100ge.cenic.net (137.164.11.1)  36.270 ms  35.459 ms  36.197 ms
11  dc-oak-agg4--svl-agg4-100ge.cenic.net (137.164.46.144)  39.906 ms  36.472 ms  36.677 ms
12  ucb--oak-agg4-10g.cenic.net (137.164.50.31)  34.949 ms  38.679 ms  35.176 ms
13  t2-3.inr-201-sut.berkeley.edu (128.32.0.37)  38.471 ms  39.041 ms  39.848 ms
14  et3-48.inr-311-ewdc.berkeley.edu (128.32.0.101)  39.286 ms  39.073 ms  38.682 ms
15  isaac.ssl.berkeley.edu (208.68.240.115)  39.068 ms  38.789 ms  38.355 ms
$

ID: 1731583 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 4826
Credit: 30,725,064
RAC: 448
United States
Message 1731587 - Posted: 3 Oct 2015, 19:00:52 UTC
Last modified: 3 Oct 2015, 19:13:24 UTC

All systems running smoothly..............I smell a rat......

[edit] WoW, did that ready to send buffer drain quickly.......

"Sour Grapes make a bitter Whine." <(0)>
ID: 1731587 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1366
Credit: 154,261,956
RAC: 247,045
United States
Message 1731606 - Posted: 3 Oct 2015, 20:54:07 UTC - in response to Message 1731587.  
Last modified: 3 Oct 2015, 20:54:34 UTC

Very interesting that with things working "properly", a

tracert setiboinc.ssl.berkeley.edu

now yields a timeout. Better than not found, but still ???
ID: 1731606 · Report as offensive
Herb Smith
Volunteer tester

Send message
Joined: 28 Jan 07
Posts: 76
Credit: 31,615,205
RAC: 0
United States
Message 1731639 - Posted: 3 Oct 2015, 22:19:07 UTC

Went out to see The Martian and came home to find my caches full and things looking more normal. Oh, and it is a good movie also. Two good things today.

Herb
ID: 1731639 · Report as offensive
Profile betreger
Avatar

Send message
Joined: 29 Jun 99
Posts: 9337
Credit: 24,606,610
RAC: 20,818
United States
Message 1731645 - Posted: 3 Oct 2015, 22:32:41 UTC

SSP shows RTS = 0 but I won't panic because I have 5 days or so of APs left to process.
ID: 1731645 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11476
Credit: 167,270,000
RAC: 101,640
Australia
Message 1731657 - Posted: 3 Oct 2015, 23:32:40 UTC - in response to Message 1731645.  

Looking at my log, for the last 7.5 hours I haven't had any Scheduler contact issues.

The only issue at the moment is with the splitters, once again their output has dropped way off & they're not able to keep up with demand, let alone build up the ready-to-send buffer.
09ap11aa has 2 splitters on the one file. Sticky file? Even so, 1 or 2 splitters down shouldn't result in running out of ready-to-send work unless there's a shorty storm, and that isn't the case.


What's really causing concern for me right now- the Haveland page isn't displaying anything at the moment.
Grant
Darwin NT
ID: 1731657 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 13022
Credit: 142,386,079
RAC: 177,094
United Kingdom
Message 1731664 - Posted: 3 Oct 2015, 23:52:19 UTC - in response to Message 1731657.  

What's really causing concern for me right now- the Haveland page isn't displaying anything at the moment.

It's showing normally for me, with none of the gaps that usually occur when it can't pull server stats from the status page.

The only oddity I see is a full RTS (and matching low creation rate, because of high water mark) until 18:00 his time - which I think is one hour the other side of UTC from me. Then, a dramatic draining of RTC over about 3 hours, which the creation rate - now uninhibited - wasn't able to keep up with.
ID: 1731664 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11476
Credit: 167,270,000
RAC: 101,640
Australia
Message 1731673 - Posted: 4 Oct 2015, 0:29:08 UTC - in response to Message 1731664.  
Last modified: 4 Oct 2015, 0:31:18 UTC

What's really causing concern for me right now- the Haveland page isn't displaying anything at the moment.

It's showing normally for me, with none of the gaps that usually occur when it can't pull server stats from the status page.

*sighs in relief*
The page is now coming up for me as well (mostly, some of the graphs aren't loading). Before it was just a blank white page.


The only oddity I see is a full RTS (and matching low creation rate, because of high water mark) until 18:00 his time - which I think is one hour the other side of UTC from me. Then, a dramatic draining of RTC over about 3 hours, which the creation rate - now uninhibited - wasn't able to keep up with.

Yep, 18:00 there was a huge surge in returned work (250,000 per hour), Ready-to-send buffer drained, and at that time the splitter output wasn't inhibited, but it is now.
It was peaking at 38 with lows of 25, then at approx. 22:30hrs it dropped to a max of 30 & minimum below 20.
It's just now coming off the sub 20 minimum & is still just on 30.

For some reason received in the last hour is still sitting around 100,000- in my cache & the odd amount of work I'm able to get seems to be a reasonable mix of VLARs and shorties. Possibly as I get more work, there will be more shorties than VLARs, hence the current Returned-in-the-last-hour numbers.
Grant
Darwin NT
ID: 1731673 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11476
Credit: 167,270,000
RAC: 101,640
Australia
Message 1731677 - Posted: 4 Oct 2015, 0:51:53 UTC - in response to Message 1731673.  

Looking at that surge in returned work, and my client logs for Scheduler contact issues, it appears my Scheduler contact issues were resolved pretty much around the same time the upload avalanche began.
Looks like they may have fixed the network issues.
Grant
Darwin NT
ID: 1731677 · Report as offensive
Profile betreger
Avatar

Send message
Joined: 29 Jun 99
Posts: 9337
Credit: 24,606,610
RAC: 20,818
United States
Message 1731686 - Posted: 4 Oct 2015, 1:22:07 UTC - in response to Message 1731677.  

Looks like they may have fixed the network issues.

If so we will never know what was wrong.
ID: 1731686 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11476
Credit: 167,270,000
RAC: 101,640
Australia
Message 1731698 - Posted: 4 Oct 2015, 2:39:01 UTC - in response to Message 1731673.  

For some reason received in the last hour is still sitting around 100,000- in my cache & the odd amount of work I'm able to get seems to be a reasonable mix of VLARs and shorties. Possibly as I get more work, there will be more shorties than VLARs, hence the current Returned-in-the-last-hour numbers.


Still getting a few VLARs, but everything else is a shortie.
100,000/h continue to come in, and even with the splitters finally picking up the pace they still can't keep up with the demand.
Grant
Darwin NT
ID: 1731698 · Report as offensive
bluestar

Send message
Joined: 5 Sep 12
Posts: 3320
Credit: 1,973,323
RAC: 184
Message 1731704 - Posted: 4 Oct 2015, 3:03:39 UTC
Last modified: 4 Oct 2015, 3:08:40 UTC

Why all those complaints?

At least I am getting my share of tasks without too much problems.

Everyone knows that Seti@home is not the same as the Seti Institute.

If there are any problems when it comes to downloading or uploading of tasks, such problems should be the responsibility of the lab and noone else.

If someone wishes me to not say anything, it probably is still the correct answer.
ID: 1731704 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11476
Credit: 167,270,000
RAC: 101,640
Australia
Message 1731757 - Posted: 4 Oct 2015, 8:59:32 UTC - in response to Message 1731704.  

Huge number of results being returned, and once again the splitters have dropped their bundle.
For a few hours there they were managing to average around 30/s; but once again they've dropped back to less than 20/s with the odd burst to almost (but not quite) 30/s.
Grant
Darwin NT
ID: 1731757 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2766
Credit: 519,530,857
RAC: 803,949
Canada
Message 1731772 - Posted: 4 Oct 2015, 11:57:27 UTC - in response to Message 1731757.  

Grant, Why is it the every time you are complaining about the splitters, Your computers are at least 80% full?

Please quit complaining about splitters until you are EMPTY!
ID: 1731772 · Report as offensive
Profile Clyde "Liberal" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Aug 99
Posts: 15240
Credit: 45,292,834
RAC: 742
United States
Message 1731787 - Posted: 4 Oct 2015, 13:29:19 UTC

Patience, Patience, Patience.

SETI@Home is operating on a Shoestring Budget.

BTW: Would like to see a 'Green Star' next to every Poster's name.
Those who cannot remember the past are condemned to repeat it.
--- George Santayana

Power tends to corrupt and absolute power corrupts absolutely.
--- Lord Acton
ID: 1731787 · Report as offensive
Profile Louis Loria II
Volunteer tester
Avatar

Send message
Joined: 20 Oct 03
Posts: 259
Credit: 9,171,532
RAC: 0
United States
Message 1731793 - Posted: 4 Oct 2015, 13:49:34 UTC - in response to Message 1731787.  

Patience, Patience, Patience.

SETI@Home is operating on a Shoestring Budget.

BTW: Would like to see a 'Green Star' next to every Poster's name.


Patience is good. So many people complaining about a volunteer effort...
Looking forward to my Green Star.
ID: 1731793 · Report as offensive
Profile Akio
Avatar

Send message
Joined: 18 May 11
Posts: 375
Credit: 32,129,242
RAC: 0
United States
Message 1731798 - Posted: 4 Oct 2015, 13:58:56 UTC - in response to Message 1731787.  

SETI@Home is operating on a Shoestring Budget.

BTW: Would like to see a 'Green Star' next to every Poster's name.


+1
ID: 1731798 · Report as offensive
Previous · 1 . . . 29 · 30 · 31 · 32 · 33 · 34 · 35 · Next

Message boards : Number crunching : Panic Mode On (100) Server Problems?


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.