Panic Mode On (83) Server Problems?

Message boards : Number crunching : Panic Mode On (83) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 15 · 16 · 17 · 18 · 19 · 20 · 21 · Next

AuthorMessage
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1367239 - Posted: 12 May 2013, 18:29:32 UTC - in response to Message 1367236.  
Last modified: 12 May 2013, 18:30:00 UTC

The splitters are showing green, and there's plenty of data to split, but no work has been produced for several hours (2-4/s doesn't count). Another couple of hours & we'll be out of work.

Yeah, something odd going on there...been seeing it for a few hours.

AP stopped pretty much dead in it's tracks a couple of hours ago, even though splitters show on and data is there.
And the MB splitters have been dawdling as well.
No smoking guns that I can find though.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1367239 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1367270 - Posted: 12 May 2013, 19:53:57 UTC

Well...
Looks like it's hit bottom.
My caches just started to run down.

My only guess is something DB related, as the languishing splitters are on several different servers.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1367270 · Report as offensive
ExchangeMan
Volunteer tester

Send message
Joined: 9 Jan 00
Posts: 115
Credit: 157,719,104
RAC: 0
United States
Message 1367289 - Posted: 12 May 2013, 20:15:00 UTC - in response to Message 1367270.  

Well...
Looks like it's hit bottom.
My caches just started to run down.

My only guess is something DB related, as the languishing splitters are on several different servers.

Ya, it's a shame. Things were running really good. Nice mix of MBs and APs and just a few shorties.

ID: 1367289 · Report as offensive
tbret
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 3380
Credit: 296,162,071
RAC: 40
United States
Message 1367337 - Posted: 12 May 2013, 22:08:07 UTC - in response to Message 1367236.  

The splitters are showing green, and there's plenty of data to split, but no work has been produced for several hours (2-4/s doesn't count). Another couple of hours & we'll be out of work.


So, so frustrating.

My triple 670 rig is dry of GPU work; others to follow shortly.


ID: 1367337 · Report as offensive
bill

Send message
Joined: 16 Jun 99
Posts: 861
Credit: 29,352,955
RAC: 0
United States
Message 1367338 - Posted: 12 May 2013, 22:27:33 UTC

Anyway that the earthquake shown on the daily graph
might have had something to do with the current circumstances?
ID: 1367338 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1367339 - Posted: 12 May 2013, 22:35:29 UTC - in response to Message 1367338.  

Same here. Running down quickly. It's just a pity that the limits are so low and we don't have caches otherwise we could keep working whist it was sorted out ...


ID: 1367339 · Report as offensive
Terror Australis
Volunteer tester

Send message
Joined: 14 Feb 04
Posts: 1817
Credit: 262,693,308
RAC: 44
Australia
Message 1367364 - Posted: 13 May 2013, 2:10:00 UTC - in response to Message 1365998.  

They probably were just on the edge of being VLARs. I think that next time this happens I might run "reschedule" and see if they get shunted to the cpu ... just curious to see what happens ...

With "Fred's Rescheduler"you can adjust the AR of what to count as "shorties" and VLAR's by plugging the values into the config.xml file.

Here is the one I use, as you can see, I count anything less than AR=0.25 as a VLAR. I think the official value is around AR=0.15.

config.xml file for Fred's Rescheduler.
Place in the Resched's home directory.

<config>
   <seti>
      <vlar>0.25</vlar>   
      <vhar>1.127</vhar>
   </seti>
<debug>
      <log_all_tasks>1</log_all_tasks>
      <rsc_fpops>1</rsc_fpops>
   </debug>
</config>


T.A.
ID: 1367364 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1367403 - Posted: 13 May 2013, 6:49:15 UTC - in response to Message 1367364.  


MB just started flowing through again ...
ID: 1367403 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1367518 - Posted: 13 May 2013, 16:09:05 UTC

Man, I like the random number generator the past few days.

Before, my APs were typically getting between 650-725, but the past 3 days, the average is 857. There's two in the low 700s, but most are in 800s, with three in 900 and one at 1004.88. I got a batch of high-blanked tasks last week though and I'm just now starting to chew through them, so that's got a bit to do with it. Run times aren't even that much longer than usual..not even +5%. I'm fine with that.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1367518 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1367595 - Posted: 13 May 2013, 18:24:58 UTC - in response to Message 1367592.  

I will only say it once today:

Give me AP's. I need AP's.

The kitties are crunching up all the MBs they can to clear the decks for ya.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1367595 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1367651 - Posted: 13 May 2013, 22:36:26 UTC - in response to Message 1367596.  


... and how a cache as well ... just enough to get us through the next outage would be fine ...
ID: 1367651 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1367656 - Posted: 13 May 2013, 23:08:35 UTC - in response to Message 1367595.  

I will only say it once today:

Give me AP's. I need AP's.

The kitties are crunching up all the MBs they can to clear the decks for ya.

I'm helping to by crunching as fast as I can. I'm doing three at a time on my 660 TI they are averaging 17 minutes and 41 seconds. I'm using the latest GPU optimised app.
ID: 1367656 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1367732 - Posted: 14 May 2013, 5:32:19 UTC - in response to Message 1367656.  


Just had several uploads backup, not to mention quite a few Scheduler requests resulting in "Couldn't connect to server" responses.
Hopefully just a passing glitch, not to be repeated.

Grant
Darwin NT
ID: 1367732 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22190
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1367735 - Posted: 14 May 2013, 5:44:58 UTC

Thankfully the blockage on uploads only lasted a few minutes.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1367735 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22190
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1367952 - Posted: 14 May 2013, 20:59:14 UTC

\Looking at the sate of the tapes being split it won't be very long before a new batch are loaded and Sten will have his APs to chew on.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1367952 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1368064 - Posted: 15 May 2013, 6:08:18 UTC - in response to Message 1367954.  


So, anyone got any theories about the present network traffic?
Inbound 35+Mb/s, and outbound 550+Mb/s- sustained for several hours now.
And after the outage yet another record for outbound traffic- 664M/s.
Grant
Darwin NT
ID: 1368064 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1368083 - Posted: 15 May 2013, 6:53:51 UTC - in response to Message 1367952.  

\Looking at the sate of the tapes being split it won't be very long before a new batch are loaded and Sten will have his APs to chew on.

Looks like lotsa AP set to split now....
Hope Sten can grab enough of them to make his day.


"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1368083 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1368087 - Posted: 15 May 2013, 6:58:42 UTC - in response to Message 1368064.  


So, anyone got any theories about the present network traffic?
Inbound 35+Mb/s, and outbound 550+Mb/s- sustained for several hours now.
And after the outage yet another record for outbound traffic- 664M/s.

AP's being thrown out as fast as they're made?

They do take up a lot of bandwidth when there is a glut of them.

Cheers.
ID: 1368087 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1368450 - Posted: 16 May 2013, 14:24:45 UTC

I don't know what befuddles me more: the two steep, cliff-like drop-offs in the green part of the cricket graph, or the fact that I'm making the first post in this thread (WITH the graph looking like that, no less) in 28 hours.

I could understand the drops if we had run out of ready to send, but there are plenty of both MB and AP.

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1368450 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1368454 - Posted: 16 May 2013, 14:29:06 UTC - in response to Message 1368450.  

I don't know what befuddles me more: the two steep, cliff-like drop-offs in the green part of the cricket graph, or the fact that I'm making the first post in this thread (WITH the graph looking like that, no less) in 28 hours.

I could understand the drops if we had run out of ready to send, but there are plenty of both MB and AP.

I don't understand the sharp drop in the Cricket graph either...
But, that notwithstanding, my caches have remained full almost up to the limits, so no panic here.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1368454 · Report as offensive
Previous · 1 . . . 15 · 16 · 17 · 18 · 19 · 20 · 21 · Next

Message boards : Number crunching : Panic Mode On (83) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.