Panic Mode On (78) Server Problems?

Yay! Back from normal Tuesday time-out. (btw, people in lab are really morning people...)

Yes, they took it down just before 6am California time.

Scheduler request failed: Error 403.


____________

I was just talking about one of the rigs that recently got CPU units. You still had around 1500 GPU units for the 3 GPUs. @500 seconds per GPU unit that's nearly 3 days worth left. Even if you get down to 100 per GPU that's still a half a day's worth. What did you normally run your queue as? 10 days.

It doesn't make a difference in bandwidth usage in the long run once the whole seti@home ecosystem hits steady state, it'll just mean that when a super cruncher's nVidia card goes off the rails they can only shaft at most 100 wingman per GPU as oppose to thousands. (Please check your, not directed at you msattler just nVidia users in general, results daily to catch when you system starts to produce mostly inconclusive/error/invalid GPU results.)

Each 690 crunch a WU in less than 7 min runing 3 WU at time on each GPU (it have 2) about 48 per hour or more, so in a big cruncher (3x690) a 100 WU cache is simpy ridiculous, not last for 1 hour. I have 2x690 sleeping on a bed waiting they rissing the limits, with the actual limits is a waste of time/resources put them to work, simply they will not receive the WU they need to work.

Scheduler request failed: Error 403.

these are the fore runners to the GTX7xx series ...

http://www.dvhardware.net/article56628.html

if you think there are problems at the moment with the scheduler, wait till the GTX7xx cards become wide spread ... a GTX780 is about 40-50% faster than a GTX680 or roughly about equal to 0.75 times a GTX690 card ... personally I think these cards are going to hum ...

cheers
____________

I am getting "scheduler request: timeout was reached" again. All of the last 5 requests since 08:32 UTC.

No luck since AP splitters are on-line again... just timeouts even with NNT :-(
____________

Same here - back to "Not now, honey, I've got a headache" mode.
Seems pretty clear from here that the ap's are holding the smoking gun.
____________

Same here - back to "Not now, honey, I've got a headache" mode.
Seems pretty clear from here that the ap's are holding the smoking gun.

Same here - back to "Not now, honey, I've got a headache" mode.
Seems pretty clear from here that the ap's are holding the smoking gun.

APs may not be the only problem, but they're certainly making it worse.

I noticed something that may be very significant.

In looking back at what SETI was doing with my computers just before the current problems began, I discovered that, while everything was deteriorating, SETI was sending all four of my machines enough units for a -month- of processing each! Even after squeezing every last cycle I can into processing, I still have around ten days worth of GPU units and quite a few days of CPU units left to process on most of my hosts.

While I did have my cache set to 10 days to give me a cushion for emergencies, I have -never- been sent too many units before.

If many of us were getting the same overload by SETI, it would most likely explain many of the symptoms we were seeing. The throughput demand to crank thousands of us up to caches of that size would most certainly run the SETI servers into the ground.

Perhaps this will help shed light on the problem, and also on why the SETI staff has (temporarily, I hope), limited us to 100/CPU and 100/GPU.

I am getting "scheduler request: timeout was reached" again. All of the last 5 requests since 08:32 UTC.

I'm still getting the odd one here & there, but mostly i'm getting a response within a minute or so.