Panic Mode On (60) Server problems?

Red Alert, servers are down at this moment.....
____________

Red Alert, servers are down at this moment.....

Red Alert, servers are down at this moment.....

Oh, you mean more down than they have been for the last hour or two?

Red Alert, servers are down at this moment.....

Oh, you mean more down than they have been for the last hour or two?

More like the last 6 hours or so.

That is when the upload server started misbehaving.

Red Alert, servers are down at this moment.....

Oh, you mean more down than they have been for the last hour or two?

More like the last 6 hours or so.

That is when the upload server started misbehaving.

And kinda looks like it's gonna be rough ridin' until somebody can get in to da lab to set things straight.

Eric was trying to remote boot thingys when he got home, but I don't think it quite worked out yet.

Sometimes the servers doin' 'most alright...
And sometimes I tink dey ain't.
Once in a while dey get a few bits out,
but most da time they cain't.

Red Alert, servers are down at this moment.....

Oh, you mean more down than they have been for the last hour or two?

More like the last 6 hours or so.

That is when the upload server started misbehaving.

And kinda looks like it's gonna be rough ridin' until somebody can get in to da lab to set things straight.

Eric was trying to remote boot thingys when he got home, but I don't think it quite worked out yet.

Sometimes the servers doin' 'most alright...
And sometimes I tink dey ain't.
Once in a while dey get a few bits out,
but most da time they cain't.

I just allowed Milkyway for Nvidia for a little while to see how fast the GTX560 can crunch a standard unit. Most likely way slower than my HD5830 in the same machine.

Sometimes the servers doin' 'most alright...
And sometimes I tink dey ain't.
Once in a while dey get a few bits out,
but most da time they cain't.

Sometimes the servers doin' 'most alright...
And sometimes I tink dey ain't.
Once in a while dey get a few bits out,
but most da time they cain't.

Hey Mark.
As the author of this original work, may I please have your permission to print it out and stick it on the wall of the server room at work ?
:-)

T.A.

I just allowed Milkyway for Nvidia for a little while to see how fast the GTX560 can crunch a standard unit. Most likely way slower than my HD5830 in the same machine.

Well, y'all enjoy......
The kitties are gonna proceed to crunch what they got, and hope things can get fixed before the kibble bowls run dry.

Meow meow meow!

I just allowed Milkyway for Nvidia for a little while to see how fast the GTX560 can crunch a standard unit. Most likely way slower than my HD5830 in the same machine.

Well, y'all enjoy......
The kitties are gonna proceed to crunch what they got, and hope things can get fixed before the kibble bowls run dry.

Meow meow meow!

My CUDA kibble bowl is dry, I ran through those 140 units almost as fast as I could download them.

The other machine still has a good supply though.

Uploads have been down for 6 to 7 hours now but the green on the Cricket Graphs is still maxxed out.

Shows how many downloads must have been backed up. (Unfortunately none of them are mine.) :(

T.A.

Uploads have been down for 6 to 7 hours now but the green on the Cricket Graphs is still maxxed out.

Shows how many downloads must have been backed up. (Unfortunately none of them are mine.) :(

T.A.

There is still a bit of work being issued on the few scheduler requests that make it in and out of the black hole.

6.12.33 is showing it's limits again. The machine with 6.10.58 continues to occasionally report & get new work, but 6.12.33 backs off so far with each failed attempt that it hasn't reported or received work for hours.

I can't report or request now.

And in reply to the last two post, I am sure the extra long back-offs in 6.12.nn are actually causing more problems than they are curing. Very noticable if you abuse the buttons.

Would you believe that in the distant past, I actually gave written warnings to people who played with switches and twiddled with variable controls for no good reason.

Looking at the crickets there have been a series of drop-outs for the last few hours, every couple of hours the throughput has dropped by about 10%.
Coupled with a very poor upload performance its obvious that the servers are less than happy. We're going to have to wait a few hours until anyone in Berkeley is awake, or maybe a couple of days until someone gets in to work on Monday.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

I can't report or request now.

And in reply to the last two post, I am sure the extra long back-offs in 6.12.nn are actually causing more problems than they are curing. Very noticable if you abuse the buttons.

Would you believe that in the distant past, I actually gave written warnings to people who played with switches and twiddled with variable controls for no good reason.

Work outside the project indicates that long back-off are actually counter productive in terms of overall throughput on saturated data links. This is particularly so when you have a long first back-off. If you must use back-off then a short, random initial back-off is far more effective in load spreading.

Far more effective is load throttling, where you reduce the data rate to each of your concurrent clients by a small faction.
So, if you have 100 concurrent clients who would normally each have a 1% share of the available bandwidth and you are suffering signs of congestion (increases packet drop for example) you reduce the bandwidth share of each client by 1%, that is from 1% of total to 0.99% of total. This has virtually no effect on the data rate to the user, but does reduce the instantaneous data rate enough to reduce the number of destructive collisions, so reducing packet loss, and the client actual observes an INCREASE in effective data rate at their end of the bit of wire, but you are only using 99% of the available bandwidth.

Obviously if you have a situation where the feed server is off-line (for maintenance, or it has crashed) then you have to implement a message that says so, and a realistic "wait for" delay.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

I can't report or request now.

And in reply to the last two post, I am sure the extra long back-offs in 6.12.nn are actually causing more problems than they are curing. Very noticable if you abuse the buttons.

Would you believe that in the distant past, I actually gave written warnings to people who played with switches and twiddled with variable controls for no good reason.