Panic Mode On (80) Server Problems?


log in

Advanced search

Message boards : Number crunching : Panic Mode On (80) Server Problems?

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 25 · Next
Author Message
Mark Lybeck
Send message
Joined: 9 Aug 99
Posts: 176
Credit: 45,945,686
RAC: 169,223
Finland
Message 1323763 - Posted: 2 Jan 2013, 20:56:04 UTC - in response to Message 1323749.


I have over 150 machines that have not got any tasks today and they are getting tasks from Einstein which has a resource of 0.

Any idea why polling less than 303 seconds does not return the normal error about the request being too recent?


Um.. How do you fit all 150 machines at home? Or did you hijack a corporate server farm? How do you finance the powerbill for those 150 machines?

With a moderate assumption of 150W/machine would mean 22,5kW. At a 0,1€/kWh this would cost you 2,25€/hour -> 2,25€*24 = 54€/day -> 365* 54€/day = 19710€/year to run.

With the power youre spending you could trade in your car for a new one every year.

____________

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 3277
Credit: 76,686,050
RAC: 76,623
United States
Message 1323765 - Posted: 2 Jan 2013, 21:02:00 UTC - in response to Message 1323763.


I have over 150 machines that have not got any tasks today and they are getting tasks from Einstein which has a resource of 0.

Any idea why polling less than 303 seconds does not return the normal error about the request being too recent?


Um.. How do you fit all 150 machines at home? Or did you hijack a corporate server farm? How do you finance the powerbill for those 150 machines?

With a moderate assumption of 150W/machine would mean 22,5kW. At a 0,1€/kWh this would cost you 2,25€/hour -> 2,25€*24 = 54€/day -> 365* 54€/day = 19710€/year to run.

With the power youre spending you could trade in your car for a new one every year.

IIRC he owns a server farm. Which he runs projects on when he doesn't have it leased out.
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

Profile ivan
Volunteer tester
Avatar
Send message
Joined: 5 Mar 01
Posts: 382
Credit: 93,314,906
RAC: 118,679
United Kingdom
Message 1323768 - Posted: 2 Jan 2013, 21:07:19 UTC - in response to Message 1323757.

Guess I will just have to find some alcohol to get some patience, :^}

Yep, I know that feelin'... :-0!
____________

mikeej42
Send message
Joined: 26 Oct 00
Posts: 83
Credit: 702,289,211
RAC: 1,563,741
United States
Message 1323773 - Posted: 2 Jan 2013, 21:22:28 UTC - in response to Message 1323763.

LOL
Yes these are servers at my work that I can use when we are not doing their "real work". I use them in between test cycles. They are HP c7000 Blade Chassis mounted 4 to a rack and the company was actually experimenting with special cooling systems for the equipment racks that the company was going to sell. There are 3 160KW cooling systems in our lab. The cooling system alone could handle several large homes. The company has since abandoned the cooling project.

____________

Profile ivan
Volunteer tester
Avatar
Send message
Joined: 5 Mar 01
Posts: 382
Credit: 93,314,906
RAC: 118,679
United Kingdom
Message 1323789 - Posted: 2 Jan 2013, 21:38:23 UTC - in response to Message 1323773.

LOL
Yes these are servers at my work that I can use when we are not doing their "real work". I use them in between test cycles. They are HP c7000 Blade Chassis mounted 4 to a rack and the company was actually experimenting with special cooling systems for the equipment racks that the company was going to sell. There are 3 160KW cooling systems in our lab. The cooling system alone could handle several large homes. The company has since abandoned the cooling project.

I'm working on a project that's tooling up to use carbon-dioxide mixed-phase cooling in its system (particle detectors, not CPUs) but I'd love to commercialise it for processor cooling. Probably too expensive for home computing (a hefty compressor is needed to liquify the CO2) but maybe for server farms?
____________

Profile Siran d'Vel'nahr
Volunteer tester
Avatar
Send message
Joined: 23 May 99
Posts: 5623
Credit: 3,511,241
RAC: 2,979
United States
Message 1323866 - Posted: 2 Jan 2013, 22:18:51 UTC

Greetings,

Dang near 5,000,000 WUs between MB and AstroPulse and my BOINC client event log says project has no new tasks?

What gives?

Keep on BOINCing...! :)

____________
CMDR Siran d'Vel'nahr XO
USS Vre'kasht NCC-33187

Siran's website: [ ONLINE! ]

Profile Qui-Gon
Volunteer tester
Avatar
Send message
Joined: 15 May 99
Posts: 2888
Credit: 5,494,950
RAC: 2,601
United States
Message 1323926 - Posted: 2 Jan 2013, 23:21:49 UTC - in response to Message 1323866.

Greetings,

Dang near 5,000,000 WUs between MB and AstroPulse and my BOINC client event log says project has no new tasks?

What gives?

Keep on BOINCing...! :)

Same thing happening to me. I took a break from crunching over the holidays so my hoppers are empty. Now that we're back up again, I can't get any work? That is disappointing.

PS: Happy New Year Siran

Profile Bernie Vine
Volunteer tester
Avatar
Send message
Joined: 26 May 99
Posts: 5671
Credit: 14,216,855
RAC: 51,033
United Kingdom
Message 1323928 - Posted: 2 Jan 2013, 23:33:00 UTC

Just got 24 CPU tasks when I want GPU, when I looked all 24 are VLARS!!!


____________

mikeej42
Send message
Joined: 26 Oct 00
Posts: 83
Credit: 702,289,211
RAC: 1,563,741
United States
Message 1323952 - Posted: 3 Jan 2013, 0:43:24 UTC - in response to Message 1323926.
Last modified: 3 Jan 2013, 1:02:40 UTC

Greetings,

Dang near 5,000,000 WUs between MB and AstroPulse and my BOINC client event log says project has no new tasks?

What gives?

Keep on BOINCing...! :)

Same thing happening to me. I took a break from crunching over the holidays so my hoppers are empty. Now that we're back up again, I can't get any work? That is disappointing.

PS: Happy New Year Siran

It appears that there are so many hosts with empty caches all trying to get tasks at the same time that the system is having troubles getting the tasks delivered. Tasks are coming out very slowly. I have gotten a few hundred between all my hosts today. We just need to remain patient while the servers get caught up.

Enjoy a mug of your favorite beverage and have a happy new year.
____________

clive G1FYE
Volunteer moderator
Send message
Joined: 4 Nov 04
Posts: 1284
Credit: 14,941,395
RAC: 2,739
United Kingdom
Message 1323966 - Posted: 3 Jan 2013, 1:32:00 UTC
Last modified: 3 Jan 2013, 1:42:03 UTC

The cricket graph is well short of max,
somthing iz still broken.
the work cant get to the pipe to go slowly through it.
There shurly cant be that many VLARs not going to GPUs ??!!
I would realy like to get back to crunching VLARs on my ATI GPU but Eric an Co wont let me, [/whinge]

VBobier
Avatar
Send message
Joined: 30 Nov 03
Posts: 40790
Credit: 32,089,497
RAC: 19,414
United States
Message 1323970 - Posted: 3 Jan 2013, 1:43:47 UTC - in response to Message 1323966.

The cricket graph is well short of max,
somthing iz still broken.
the work cant get to the pipe to go slowly through it.
There shurly cant be that many VLARs not going to GPUs ??!!
I would realy like to get back to crunching VLARs on my ATI GPU but Eric an Co wont let me, [/whinge]

Well I for one don't want a VLAR to impale My gpus...
____________
BSG Anthem
My Facebook page


Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5059
Credit: 40,813,367
RAC: 52,696
Australia
Message 1323991 - Posted: 3 Jan 2013, 3:47:43 UTC - in response to Message 1323966.
Last modified: 3 Jan 2013, 3:49:47 UTC

somthing iz still broken.

Yep.
If there are that many WUs ready to send, and everything was working as it should then the traffic should be maxed out. It's not, and most requests for work result in the No tasks available message.


EDIT- one of my systems just scored 38 WUs for the GPU. They're not shorties, so it should last for a few hours before they're all done.
____________
Grant
Darwin NT.

ExchangeMan
Volunteer tester
Send message
Joined: 9 Jan 00
Posts: 67
Credit: 41,542,352
RAC: 232,274
United States
Message 1323995 - Posted: 3 Jan 2013, 4:08:15 UTC - in response to Message 1323991.

I just got 44 GPU work units - unfortunately all shorties.

____________

Profile Peter M. Ferrie
Volunteer tester
Send message
Joined: 28 Mar 03
Posts: 85
Credit: 8,805,610
RAC: 23,427
United States
Message 1323998 - Posted: 3 Jan 2013, 4:27:58 UTC

if there really are 5,000,000 + workunits ready to send ... it will be the great wu flood of 2013 .. head for higher cpu's
____________

Keith White
Avatar
Send message
Joined: 29 May 99
Posts: 271
Credit: 1,581,079
RAC: 3,904
United States
Message 1324000 - Posted: 3 Jan 2013, 4:40:47 UTC
Last modified: 3 Jan 2013, 4:42:24 UTC

I've gotten 26 so far.

Back of the envelope calculations suggest at a rate of 25 units per second could be downloaded, limited due to the pipe's bandwidth. This guesstimate doesn't include any outgoing bandwidth used to for control and status requests.

Now since a significant number of us are using GPUs to help do the heavy lifting, a two day outage would have wiped out most if not all of our 100 unit queue so everybody wants 100 GPU units. That comes out to only filling 900 hosts per hour, best case. So how many active hosts out there crunching with one or more GPUs? 10,000? 20,000? So it's going to take a dozen hours or two to handle demand and that's doesn't include the fact that for the fastest among us could crunch 100 GPU units in under an hour and a good number of us in under a day. It'll take a bit.

And since I've started writing this, I got 41 more.
____________
"Life is just nature's way of keeping meat fresh." - The Doctor

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5059
Credit: 40,813,367
RAC: 52,696
Australia
Message 1324007 - Posted: 3 Jan 2013, 5:18:24 UTC - in response to Message 1324000.

So it's going to take a dozen hours or two to handle demand and that's doesn't include the fact that for the fastest among us could crunch 100 GPU units in under an hour and a good number of us in under a day. It'll take a bit.

My GTX 560Ti does roughly 200 WUs per day- if they are all 22min crunching time. Some are longer, most are shorter.
On average it probably does closer to 300WU per day. When it can get them.

____________
Grant
Darwin NT.

Wiggo
Send message
Joined: 24 Jan 00
Posts: 1488
Credit: 62,571,035
RAC: 87,469
Australia
Message 1324036 - Posted: 3 Jan 2013, 7:53:19 UTC - in response to Message 1324007.

It seems that the feeder is having a bit of a problem feeding us.

Cheers.
____________

Profile Bernie Vine
Volunteer tester
Avatar
Send message
Joined: 26 May 99
Posts: 5671
Credit: 14,216,855
RAC: 51,033
United Kingdom
Message 1324041 - Posted: 3 Jan 2013, 8:14:06 UTC

Still just getting VLARS for the CPU. Never seen this many at once. Nothing for GPU.
____________

Keith White
Avatar
Send message
Joined: 29 May 99
Posts: 271
Credit: 1,581,079
RAC: 3,904
United States
Message 1324051 - Posted: 3 Jan 2013, 9:35:52 UTC - in response to Message 1324041.

Well so far I've gotten all of 67 MB units. 56 GPU, 11 CPU, none VLAR.

I'm now at 99 GPU and 46 CPU.

Like I said before, best super ideal guesstimate is that maybe 900 hosts can be fed 100 units per hour. At the current cricket levels I'll guess maybe 500 hosts per hour.

So how many hosts want GPU units? How many do an Oliver Twist and ask for another full reload in 3/6/12/24 hours?
____________
"Life is just nature's way of keeping meat fresh." - The Doctor

msattler
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 34138
Credit: 343,667,357
RAC: 541,784
United States
Message 1324052 - Posted: 3 Jan 2013, 9:42:36 UTC - in response to Message 1324041.

Still just getting VLARS for the CPU. Never seen this many at once. Nothing for GPU.

I set 'use nVidia GPU' to no for the night, will try to build some CPU cache.
Otherwise the rigs were just asking for GPU and getting NOTHING.
I'll enable GPU in the morning and see if anything's changed.
Thanks for the heads up.
____________
******

"Ask not, what your kitty can do for you. Ask what you can do for your kitty."

As it is kitten, so shall it be done.



Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 25 · Next

Message boards : Number crunching : Panic Mode On (80) Server Problems?

Copyright © 2013 University of California