Short WU : Good or Evil ?


log in

Advanced search

Message boards : Number crunching : Short WU : Good or Evil ?

1 · 2 · Next
Author Message
Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1301020 - Posted: 1 Nov 2012, 17:24:48 UTC

Why do shorties burn more electricity ? ...power load averages show a 40 watt increase when short WUs are processing on my gpus.

msattler
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 37382
Credit: 500,085,307
RAC: 524,056
United States
Message 1301022 - Posted: 1 Nov 2012, 17:34:33 UTC - in response to Message 1301020.

Why do shorties burn more electricity ? ...power load averages show a 40 watt increase when short WUs are processing on my gpus.


For the kitties, that means the opti app is processing them more efficiently and doing more work in a shorter period of time.
I can watch the killawatt meters on a few of my rigs and can tell right away when they are crunching shorties.
Luv 'em, when AP is not clogging up the pipes and the servers can keep up with the increased demand that shorties create.
____________
******************
Crunching Seti, loving all of God's kitties.

I have met a few friends in my life.
Most were cats.

Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1301044 - Posted: 1 Nov 2012, 18:20:31 UTC - in response to Message 1301022.

Why do shorties burn more electricity ? ...power load averages show a 40 watt increase when short WUs are processing on my gpus.


For the kitties, that means the opti app is processing them more efficiently and doing more work in a shorter period of time.
I can watch the killawatt meters on a few of my rigs and can tell right away when they are crunching shorties.
Luv 'em, when AP is not clogging up the pipes and the servers can keep up with the increased demand that shorties create.



Speaking of the kitties, didn't you have some photos posted, showing your cooling setup ?..
love to see that again. looking into whipping up a similar system for this next project o'mine.
Are the chilly kitties fed from one compressor or do they each have their own?

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 3571
Credit: 98,087,550
RAC: 77,373
United States
Message 1301058 - Posted: 1 Nov 2012, 19:03:45 UTC - in response to Message 1301020.
Last modified: 1 Nov 2012, 19:04:22 UTC

Why do shorties burn more electricity ? ...power load averages show a 40 watt increase when short WUs are processing on my gpus.


I see about a 3% drop is power usage when my 24 core box is chewing through a queue of shorties.

Good, bad, or otherwise. It is all data that needs to be processed.
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

Richard Haselgrove
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8275
Credit: 44,967,775
RAC: 13,837
United Kingdom
Message 1301062 - Posted: 1 Nov 2012, 19:11:46 UTC

OK, I'll venture an opinion:

Evil, on two grounds -
1) There is very little chance of detecting ET with the very limited signal analysis done on these rapid sky surveys.
2) They clog up every part of SETI processing chain (storage, database, communications) to no very good purpose.

Note that I have no opinion on either power consumed or credit awarded ;)

alan
Avatar
Send message
Joined: 18 Feb 00
Posts: 131
Credit: 401,606
RAC: 22
United Kingdom
Message 1301066 - Posted: 1 Nov 2012, 19:17:13 UTC

Evil.

More bandwidth required to send the large numbers of little tasks.

Poor usage of the client, the same overhead as a large task but less data.

Huge numbers of results clog up the databases, validators, schedulers et al.

And they pay back worst of all in terms of credits per CPU-second.
____________

Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1301067 - Posted: 1 Nov 2012, 19:18:13 UTC

The power consumption bugs me because I am riding on the line with the power supply I have ,
once I add this next double 460, my little 850w will be peaking out, a random 100 watt increase will definitely cause errors .

msattler
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 37382
Credit: 500,085,307
RAC: 524,056
United States
Message 1301179 - Posted: 2 Nov 2012, 6:09:36 UTC - in response to Message 1301044.

Why do shorties burn more electricity ? ...power load averages show a 40 watt increase when short WUs are processing on my gpus.


For the kitties, that means the opti app is processing them more efficiently and doing more work in a shorter period of time.
I can watch the killawatt meters on a few of my rigs and can tell right away when they are crunching shorties.
Luv 'em, when AP is not clogging up the pipes and the servers can keep up with the increased demand that shorties create.



Speaking of the kitties, didn't you have some photos posted, showing your cooling setup ?..
love to see that again. looking into whipping up a similar system for this next project o'mine.
Are the chilly kitties fed from one compressor or do they each have their own?
Only one rig is phase change cooled with a compressor, all the others are air cooled.
I would not recommend this kind of cooling to anybody these days. When the compressor finally wears out, I will not go there again. Maybe some form of liquid cooling, perhaps with a phase change assist to get a bit below ambient, but never again cool a CPU to -28c.....
It's really hard to do without getting condensation in the CPU socket, which really kills things.
I could repost a few picks in another thread if folks really wanna see it again, but it would be too far off topic in this thread.

____________
******************
Crunching Seti, loving all of God's kitties.

I have met a few friends in my life.
Most were cats.

WinterKnight
Volunteer tester
Send message
Joined: 18 May 99
Posts: 8219
Credit: 21,805,161
RAC: 11,922
United Kingdom
Message 1301187 - Posted: 2 Nov 2012, 6:22:58 UTC

credits/time, in comparison to other longer tasks, is quite good when done on the CPU but terrible on the GPU.

And yes they clog up the comms system, especially when AP's are available.

Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1301480 - Posted: 2 Nov 2012, 23:32:48 UTC - in response to Message 1301179.

.
It's really hard to do without getting condensation in the CPU socket, which really kills things.
I could repost a few picks in another thread if folks really wanna see it again, but it would be too far off topic in this thread.


I experimented with this same problem while trying to ice cool a p4 ..
A heaping glob of silicone dielectric compound under the cpu and all around the socket.
A basic soft potting of the immediate area, stops all condensation from forming in places it cant be dealt with. keeping the air out is the key.

alan
Avatar
Send message
Joined: 18 Feb 00
Posts: 131
Credit: 401,606
RAC: 22
United Kingdom
Message 1305409 - Posted: 12 Nov 2012, 17:24:30 UTC

While the AP splitters have been demonised for causing the problems a lot of people are having, I think the huge number of shorties that have been sent out recently are the real problem.

Each one is about ΒΌ of the workload, resulting in a 4 x increase in network traffic as the results come in, and a corresponding increase in the load on the database servers.

As far as I know the basic SETI workunit hasn't changed in size for many years. I'd say it was about time that larger workunits were created, and some way was found to eliminate shorties altogether.

I appreciate that this may not be simple, but unless the workunits are somehow made to take up more of the crunchers time, the bandwidth and the database servers will get more and more pressure.

I can remember workunits taking days at a time in SETI Classic. It's about time that they were made bigger again.
____________

Profile Jaye Ellen
Send message
Joined: 29 Nov 08
Posts: 24
Credit: 5,013,573
RAC: 5,196
United States
Message 1305418 - Posted: 12 Nov 2012, 17:46:53 UTC - in response to Message 1305409.

Hi --- my question has to do with an extremely large number of tasks being run in "high priority" --- what prompts SETI to require older tasks needing to be run like this? Just curious, you know?

Jaye Ellen

Profile Mike
Volunteer tester
Avatar
Send message
Joined: 17 Feb 01
Posts: 22411
Credit: 29,371,652
RAC: 24,886
Germany
Message 1305420 - Posted: 12 Nov 2012, 17:49:52 UTC - in response to Message 1305418.
Last modified: 12 Nov 2012, 17:50:59 UTC

Hi --- my question has to do with an extremely large number of tasks being run in "high priority" --- what prompts SETI to require older tasks needing to be run like this? Just curious, you know?

Jaye Ellen


That happens when the work is getting close to the deadline (shorties) or your computer is over committed.
____________

Richard Haselgrove
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8275
Credit: 44,967,775
RAC: 13,837
United Kingdom
Message 1305429 - Posted: 12 Nov 2012, 18:02:30 UTC - in response to Message 1305409.

I think we need to distinguish between two different types of problem.

Shorties are primarily a communications problem: they tend to be easy to get allocated, but slow to download through our over-congested pipe. We've actually been having something of a 'shorty storm' for the last 36 hours, during the time that people have been saying, in general, that the servers are getting back to normal.

What happened last weekend (3-4 November) was completely difficult. Getting work downloaded (after it had been allocated) was no more difficult than normal: the new problem - and it was new, I've not seen it in many years of posting on these boards - was getting notification of work allocations in the first place.

Although I did see one post speculating that a fault on a router handling the scheduler IP address might have opened a black hole in the scheduler reply network path - and that deserves consideration - the 'scheduler reply timeout problem' didn't feel to me like a communications problem. Instead, it felt as if the scheduler wasn't assembling and despatching the reply messages at all. I think that's where the finger pointing at the AP splitters comes from: the splitter processes themselves constipating the server, rather than the data they produce constipating the comms link.

This morning, Brad commented that 'The Lab has Synergy loaded down heavy here of late': I think it would be helpful to at least try the system with a lighter load on whichever server runs the scheduling service for the duration.

msattler
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 37382
Credit: 500,085,307
RAC: 524,056
United States
Message 1305458 - Posted: 12 Nov 2012, 18:52:00 UTC - in response to Message 1305429.

I think we need to distinguish between two different types of problem.

Shorties are primarily a communications problem: they tend to be easy to get allocated, but slow to download through our over-congested pipe. We've actually been having something of a 'shorty storm' for the last 36 hours, during the time that people have been saying, in general, that the servers are getting back to normal.

What happened last weekend (3-4 November) was completely difficult. Getting work downloaded (after it had been allocated) was no more difficult than normal: the new problem - and it was new, I've not seen it in many years of posting on these boards - was getting notification of work allocations in the first place.

Although I did see one post speculating that a fault on a router handling the scheduler IP address might have opened a black hole in the scheduler reply network path - and that deserves consideration - the 'scheduler reply timeout problem' didn't feel to me like a communications problem. Instead, it felt as if the scheduler wasn't assembling and despatching the reply messages at all. I think that's where the finger pointing at the AP splitters comes from: the splitter processes themselves constipating the server, rather than the data they produce constipating the comms link.

This morning, Brad commented that 'The Lab has Synergy loaded down heavy here of late': I think it would be helpful to at least try the system with a lighter load on whichever server runs the scheduling service for the duration.

I agree, Richard.
The ghost attack was NOT purely a bandwidth issue, although I was fooled into thinking it was at the time.
The ghost generation simply made the bandwidth issue worse.
I think we know now that the issue was the server was very overtaxed and lost it's marbles. It could complete the first half of scheduler requests, up to assigning work. But then, it never answered the hosts back. And the cycle then repeated itself over and over again.

I have started to receive new work for the first time since the 5th.
I could not say if that is because something became 'unbroken' or if my caches finally became depleted enough to generate work requests. Or if the imposed limits have been changed.
I have killawatts on 3 rigs I can see from where I am sitting, and they are all pegged at their usual limits. So, crunching is still fine here.
____________
******************
Crunching Seti, loving all of God's kitties.

I have met a few friends in my life.
Most were cats.

msattler
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 37382
Credit: 500,085,307
RAC: 524,056
United States
Message 1305462 - Posted: 12 Nov 2012, 18:55:08 UTC

And my belief is that new up/download servers will fix a lotta woes.
There are just too many demands being placed on the hardware in place at the moment.
Ya gotta give them some breathing room.
____________
******************
Crunching Seti, loving all of God's kitties.

I have met a few friends in my life.
Most were cats.

Profile Jaye Ellen
Send message
Joined: 29 Nov 08
Posts: 24
Credit: 5,013,573
RAC: 5,196
United States
Message 1305526 - Posted: 12 Nov 2012, 20:22:51 UTC - in response to Message 1305420.

All right, next question is (I run SETI on 4 different laptops) what can I do to my settings to more equitably perform the tasks in a timely manner? I had always thought my settings had nothing to do with the actual scheduling, but apparently that's just one MORE example of my ignorance !!! Also, lately, the amount of work stored on my various machines is dwindling. I run an i-7, an i-5, an Intel Win7 and a win Vista. The Win7 machine seems to be the slowest of the collection. As far as workload, SETI@Home is the main application I run. My partner is an avid Gamer with her i-7 and her machine is set to stop BOINC when she is gaming but except for that SETI has the run of all 8 processes. We just topped 1.9 million work units (It was 1.2 million till I put the i-7 online maybe 3 months ago.) Anyway, I like to watch the units running on the computers and watching the BOINC screen saver. I'm a medically retired Air Force member, last involved with Satellite Communications via a 60 foot dish in Sunnyvale, CA, in Silicon Valley.
____________

N9JFE
Volunteer tester
Avatar
Send message
Joined: 4 Oct 99
Posts: 9313
Credit: 11,933,233
RAC: 13,425
United States
Message 1305531 - Posted: 12 Nov 2012, 20:26:06 UTC

I first said it a couple weeks ago: fewer AP splitters, more MB splitters, especially since the APs get through their work faster anyway.

I still support the first half of that. Shut down the AP splitter processes on synergy altogether. Only run the ones on vader.

This will A: reduce the amount of work synergy has to do; B: reduce the amount of AP available to send at any given time, which should help with the bandwidth problem; C: better balance the amount of AP and MB work. (I thought I had a better C: than that, but now I can't remember it if I did.)

____________
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1305553 - Posted: 12 Nov 2012, 21:38:12 UTC

N9JFE wrote:
I first said it a couple weeks ago: fewer AP splitters, more MB splitters, especially since the APs get through their work faster anyway


I think the problem is something more than astropulse wu's , there have been many times in the past where AP's were split and distributed without problem.
Not to say that AP's aren't contributing to the clog, I still think it is a hardware issue with synergy.

rob smith
Volunteer moderator
Send message
Joined: 7 Mar 03
Posts: 7678
Credit: 44,907,205
RAC: 75,944
United Kingdom
Message 1305565 - Posted: 12 Nov 2012, 21:53:32 UTC

I very much doubt that it is the AP stream that is causing the problem just now - there haven't been any APs split for a couple of days, and I'm still getting a fairly poor response from the servers - quite a high time-out count, downloads hanging around for hours, uploads going into re-try and taking a long time, not to mention an increasing number of tasks awaiting validation.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

1 · 2 · Next

Message boards : Number crunching : Short WU : Good or Evil ?

Copyright © 2014 University of California