Panic Mode On (83) Server Problems?

Message boards : Number crunching : Panic Mode On (83) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 9 · 10 · 11 · 12 · 13 · 14 · 15 . . . 21 · Next

AuthorMessage
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1363836 - Posted: 3 May 2013, 8:03:47 UTC - in response to Message 1363791.  
Last modified: 3 May 2013, 8:04:39 UTC

This post from Matt seems pretty explicit to me ...

However that post was on March 8, the problems we're having occured about a week, week and a half after that.
If it is related to that post, it would be nice to have confirmation, as things were running OK with a ready-to-send buffer.

I suspect this shall either pass or become the new normal, although right now it is being brought to bear due to the shorties being sent out.

We are dealing with quite a different animal since the move to the new digs for the servers. The amazing bandwidth is a two edged sword. Cuts both ways.

It has allowed the servers to send and receive all the data possible in both directions most of the time. This is a very grand thing!

What has happened, is that now other limitations present themselves. Splitting capacity. Database capacity. I/O capacity.
Still, a very good thing.

What do you think is worse, not being able to get work because the servers could not parse it out over 100Mb bandwidth, or not being able to get work at times because the servers are handing it out as fast as they can feed it to the new pipe?

I vote for the current situation.

The day may well come when Eric or Matt have to post and say..'We have no more data from Arecibo to split today.'

And that, believe it or not, would be the best situation the project could be in.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1363836 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1363838 - Posted: 3 May 2013, 8:11:13 UTC - in response to Message 1363836.  

I suspect this shall either pass or become the new normal, although right now it is being brought to bear due to the shorties being sent out.

Unfortunately it was also the case for several hours where there were hardly any shorties about.

Grant
Darwin NT
ID: 1363838 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1363842 - Posted: 3 May 2013, 8:16:57 UTC - in response to Message 1363838.  

I suspect this shall either pass or become the new normal, although right now it is being brought to bear due to the shorties being sent out.

Unfortunately it was also the case for several hours where there were hardly any shorties about.

Well, either there is something left to be sorted, or this is just the way it shall be.

I am just now looking at the Killawatt on one of my best rigs. 350 watts.
It should be drawing about 650 watts. It will keep sniffing for work, and will get a hit here and there.

I DO still wish the limits were raised, would keep my fast crunchers in work for a bit longer when the ready to send crashes.

But, I am in the same boat as everybody else. And will keep rowing as hard as I can.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1363842 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1363849 - Posted: 3 May 2013, 9:01:50 UTC

I belive the problem is the way the pfb_splitter works, seems they are slower than the old mb_splitter used until few days ago and the lack of AP work who makes we all asking for MB work only and some hidden limit added by Matt. Sooner or later we will know the answer. Until that is fixed our power bill would be smaller...
ID: 1363849 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1363853 - Posted: 3 May 2013, 9:32:16 UTC - in response to Message 1363842.  

I DO still wish the limits were raised, would keep my fast crunchers in work for a bit longer when the ready to send crashes.

Even that wouldn't help in this case as the output isn't sufficient to meet demand, let alone produce a ready-to-send buffer.

Ideally, they'll sort out the splitters so they can maintain a ready-to-send buffer.
Then sort out the database issues so we can increase the limits so we can get enough work to last more than a couple of hours.


If we run out of work, we run out of work. I can live with that (it'd be disappointing, but if there isn't any to process then there's no point getting worked up about it).
But when there is work there to be done, but it's not available, that's frustrating.
Grant
Darwin NT
ID: 1363853 · Report as offensive
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8240
Credit: 14,654,533
RAC: 20
United States
Message 1364063 - Posted: 3 May 2013, 18:06:01 UTC - in response to Message 1363791.  
Last modified: 3 May 2013, 18:15:43 UTC

This post from Matt seems pretty explicit to me ...

However that post was on March 8, the problems we're having occured about a week, week and a half after that.
If it is related to that post, it would be nice to have confirmation, as things were running OK with a ready-to-send buffer.

No, Grant, it is from April 8, immediately after the move to the CoLo facility. We have been here less than 1 month, and for the last 2 weeks the data being split has been mostly shorties. Absent any new word from Matt, I must presume that those throttles he mentioned are still in place, and for the reasons he stated. We are still establishing the new "normal". As always with this project, patience is not just a virtue, it is a requirement.

Edit: added link to Matt's Tech News message.
Donald
Infernal Optimist / Submariner, retired
ID: 1364063 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1364076 - Posted: 3 May 2013, 18:50:05 UTC
Last modified: 3 May 2013, 18:55:05 UTC

I just wish for what I came here for.
Un ending WUs for me to process in the name of the un endeding searrcy.

I have come and gone.

You can not sidetrack me any more.

Ya know, friends..........I been wrong as hell sometimes.
NOt on this one.

\Hit the road, Jack.
\

I really don't think some of you realize just who you really dealing with.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1364076 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1364128 - Posted: 3 May 2013, 21:16:50 UTC - in response to Message 1364063.  

This post from Matt seems pretty explicit to me ...

However that post was on March 8, the problems we're having occured about a week, week and a half after that.
If it is related to that post, it would be nice to have confirmation, as things were running OK with a ready-to-send buffer.

No, Grant, it is from April 8, immediately after the move to the CoLo facility.

Typo on my part- i meant April. The rest of my statement in that post is correct.

Grant
Darwin NT
ID: 1364128 · Report as offensive
Profile Akio
Avatar

Send message
Joined: 18 May 11
Posts: 375
Credit: 32,129,242
RAC: 0
United States
Message 1364185 - Posted: 3 May 2013, 23:58:46 UTC - in response to Message 1364076.  

I really don't think some of you realize just who you really dealing with.


No one is any bit scared of you, so knock it off.

Can't seem to get any work for the GPU. Anyone else with the same issue?

ID: 1364185 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1364202 - Posted: 4 May 2013, 0:20:53 UTC - in response to Message 1364185.  
Last modified: 4 May 2013, 0:21:15 UTC

Can't seem to get any work for the GPU. Anyone else with the same issue?

Nope.

"Project has no tasks available" is the usual response for any type of work request over the last week or so, then every now & then some work gets allocated. Depending on the backoffs for CPU or GPU work one or the other may miss out on that work allocation because it wasn't asked for due to the backoff.
Grant
Darwin NT
ID: 1364202 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 1364216 - Posted: 4 May 2013, 1:02:04 UTC - in response to Message 1364185.  

I really don't think some of you realize just who you really dealing with.


No one is any bit scared of you, so knock it off.

Can't seem to get any work for the GPU. Anyone else with the same issue?

I'm offline until 7pm, but this might explain the lack of work.

http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=/router-interfaces/inr-211/gigabitethernet6_17&ranges=d%3Aw&view=Octets
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1364216 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1364229 - Posted: 4 May 2013, 1:27:36 UTC - in response to Message 1364216.  

ID: 1364229 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 1364237 - Posted: 4 May 2013, 1:45:49 UTC - in response to Message 1364229.  

ID: 1364237 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1364240 - Posted: 4 May 2013, 1:54:36 UTC - in response to Message 1364237.  
Last modified: 4 May 2013, 1:56:23 UTC

I'm offline until 7pm, but this might explain the lack of work.

http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=/router-interfaces/inr-211/gigabitethernet6_17&ranges=d%3Aw&view=Octets

?

The wu output seems kind of low.

That just shows network traffic.

This gives a better idea of WU generation.
http://setistats.haveland.com/cgi/munin-cgi-graph/setiathome/setiathome/sah_creation-day.png

and this shows you how much work is ready-to-send & how much is being returned per hour.
http://setistats.haveland.com/cgi/munin-cgi-graph/setiathome/setiathome/sah_results-day.png
Grant
Darwin NT
ID: 1364240 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1364245 - Posted: 4 May 2013, 2:01:54 UTC - in response to Message 1364237.  

I'm offline until 7pm, but this might explain the lack of work.

http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=/router-interfaces/inr-211/gigabitethernet6_17&ranges=d%3Aw&view=Octets

?

The wu output seems kind of low.

Work out is the blue line this represents the work been returned to the servers. I think this was sitting about normal 18.19 MB as I write, reason not why it has been so high yesterday Berkely time is because I think they were transferring more data sets to the servers for the splitters to be able to work on. As I write the bits in data been sent to clients is at 189.06 MB I don't consider this to be low unless it becomes under 100 MB.


ID: 1364245 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1364394 - Posted: 4 May 2013, 14:55:02 UTC

Not so good here right now.
I have a few powerful rigs left sucking big wind for the lack of tasks sent.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1364394 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1364397 - Posted: 4 May 2013, 14:59:15 UTC - in response to Message 1364394.  

Not so good here right now.
I have a few powerful rigs left sucking big wind for the lack of tasks sent.

Sorry guys ;)

04/05/2013 15:55:05 | SETI@home | Scheduler request completed: got 48 new tasks
ID: 1364397 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1364401 - Posted: 4 May 2013, 15:03:51 UTC - in response to Message 1364397.  
Last modified: 4 May 2013, 15:07:18 UTC

Not so good here right now.
I have a few powerful rigs left sucking big wind for the lack of tasks sent.

Sorry guys ;)

04/05/2013 15:55:05 | SETI@home | Scheduler request completed: got 48 new tasks

The kitties gracefully sidestep so's Richard can get some work.

You are quite welcome, Richard.

Maybe my turn next.

And see/?
My big hitter just got a 51 WU hit.

Should be good to go for about 20 minutes or so...LOL.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1364401 · Report as offensive
ExchangeMan
Volunteer tester

Send message
Joined: 9 Jan 00
Posts: 115
Credit: 157,719,104
RAC: 0
United States
Message 1364409 - Posted: 4 May 2013, 15:14:25 UTC - in response to Message 1364397.  

Not so good here right now.
I have a few powerful rigs left sucking big wind for the lack of tasks sent.

Sorry guys ;)

04/05/2013 15:55:05 | SETI@home | Scheduler request completed: got 48 new tasks

I would occasionally get a burst of work units like that, but I spin through them rather quickly. Then there might not be anything for hours. There hasn't been any AP splitting for days.

The funny thing is that before they moved to the colo facility, I actually got more work units even with the retries, frustration and all.

ID: 1364409 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1364411 - Posted: 4 May 2013, 15:17:09 UTC - in response to Message 1364409.  
Last modified: 4 May 2013, 15:18:48 UTC

Not so good here right now.
I have a few powerful rigs left sucking big wind for the lack of tasks sent.

Sorry guys ;)

04/05/2013 15:55:05 | SETI@home | Scheduler request completed: got 48 new tasks

I would occasionally get a burst of work units like that, but I spin through them rather quickly. Then there might not be anything for hours. There hasn't been any AP splitting for days.

The funny thing is that before they moved to the colo facility, I actually got more work units even with the retries, frustration and all.

Well, that's the breaks, as they say.

I am OK with the servers being all that they can be for now.

It's a nice change of pace for them at least being able to send what they can send.

The Cricket is doing a rather steady 200Mb/sec.....so work is flowing.

Much better than with the old 100Mb cap.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1364411 · Report as offensive
Previous · 1 . . . 9 · 10 · 11 · 12 · 13 · 14 · 15 . . . 21 · Next

Message boards : Number crunching : Panic Mode On (83) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.