Observation of CreditNew Impact (2)

Message boards : Number crunching : Observation of CreditNew Impact (2)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 20 · Next

AuthorMessage
Michael Becker

Send message
Joined: 25 May 01
Posts: 8
Credit: 6,140,575
RAC: 0
Germany
Message 1387797 - Posted: 4 Jul 2013, 20:56:07 UTC - in response to Message 1387022.  

GPUs typically have a higher (10-100X) peak FLOPS than CPUs. However, application efficiency is typically lower (very roughly, 10% for GPUs, 50% for CPUs).

efficiency?
10% for GPU?
90% are only necessary to perform the necessary calculations, and are NOT needed to solve the quest?
Great, Haha, I piss myself fully.
If you expect that you will take each company against the wall.
Solely the material costs are taken into account.
Development, purchasing, shipping, work preparation, personel, .... are worth nothing.
10 or 50% efficiency, that's shit, because the rest is work that needs to be done in order to work, and this also takes time, and must be compensated.
The drive to the customer, no matter how much that costs something, and work, no matter where, it costs so much that the last idiot of a company also can make a living.
I'll look for a topic and will become Dr., Prof., or anything else arrogant, or to become a optimist developer.
i burn 100$ to earn 1$.
yeha
sorry, it's translated by 'translate.google' so it's not exactley what i want to say.
ID: 1387797 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20982
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1387843 - Posted: 5 Jul 2013, 0:31:16 UTC - in response to Message 1387797.  
Last modified: 5 Jul 2013, 0:35:52 UTC

GPUs typically have a higher (10-100X) peak FLOPS than CPUs. However, application efficiency is typically lower (very roughly, 10% for GPUs, 50% for CPUs).

efficiency?

[...]

i burn 100$ to earn 1$.
yeha
sorry, it's translated by 'translate.google' so it's not exactley what i want to say.

Sounds a bit like Monty Python: I spit in the eye of efficiency!


We are still in the 'early days' of utilizing GPUs. Note that they are primarily specialized for rendering graphics to display pretty pictures to instead perform for s@h more general computation algorithms.

Hence, we can expect that GPUs are less than 100% efficient for working through the s@h tasks. It takes an awful lot of effort to make the s@h algorithms fit the available GPU capabilities. Just ask a Lunatic! ;-)


Meanwhile, GPUs are very good at producing pretty graphics at near 100% efficiency!

Even with far less than 100% efficiency, the computational power of GPUs can be expected to far exceed anything possible by using a CPU alone.


Happy faster crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1387843 · Report as offensive
Ulrich Metzner
Volunteer tester
Avatar

Send message
Joined: 3 Jul 02
Posts: 1256
Credit: 13,565,513
RAC: 13
Germany
Message 1387851 - Posted: 5 Jul 2013, 1:02:20 UTC

All this cr*p about efficiency is only "hot air talking" business - like it's "en vogue" our days... - ...it's simply annoying! :þ

If a problem takes 124 minutes to process, it doesn't matter, if it is processed on a CPU or on a GPU - it simply takes 124 minutes to process - BASTA!

Ok, on a very fast machine it might take only 46 minutes to process, but come on, it's the same amount of work worked in a shorter time, cause the owner of the "faster" processor(s) invested more money. Even on an ancient machine, if it takes 555 minutes to process, it is still the same amount work, that's been done, so there is absolutely ***NO*** discussion! It's the same work and should be awarded the same credit! *B*A*S*T*A* !!!

This whole babbling around this esoteric credit-algorithm-sh*t is just dazzling the masses from reality...

...my 2 cents...
Aloha, Uli

ID: 1387851 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1387868 - Posted: 5 Jul 2013, 3:00:38 UTC - in response to Message 1386773.  
Last modified: 5 Jul 2013, 3:01:52 UTC


I also agree that running v7 on a cpu is now virtually not worth it. v7 has taken this over the tipping point. You are far better running AP on a cpu and getting at least twice the credit per core per hour.



Quite strange conclusion considering elative speeds of stock CPU (and even opt CPU) AP and GPU AP.
I would recommend directly opposite config: to crunch MB on CPU and AP on GPU. At least for ATi. For NV hosts better choice would be MB-only setup of CPU and CUDA opt apps. And avoid CPU AP for both configs.

EDIT: and this recommendation has nothing with definitely broken credits awarding. It bases on relative apps performance per se.


Raistmer,

A little while ago I ceased processing MB v7 WUs and only processed AP WUs as you may be aware by what I have said previously. I recently restarted processing MB v7 WUs on both CPU and GPU. I have a Q9450, Q6600 and i7 3903K all running MB v7 at present. I moved the boxes back over, over 2-3 July, so any credits granted for MB should be based on the most recent methodology for granting credit (lets assume that this is the case).

Attached below is the run times and credits of the last 25 WUs completed on the CPU of the Q9450. Average Run Time is 11,683 seconds, with average Credit of 97 per WUs. Translating this to hours and cores, gives a rate of 29.877 Credits per hour per core, or 2,868 credits per day for all 4 cores of a Q9450 running at 3.2GHz.

As you can see, granted credit is fairly low. This is why I have said that "running v7 on a cpu is now virtually not worth it". We have passed the tipping point where CPU participation on 4 core processors in processing WUs adds little benefit.

The Q6600 will be slightly less than the above average, and the i7 due to the extra 2 cores and AVX will be higher.

You are right in that running AP on GPU is better as I can obtain the same credit as above within an hour (approximately) through doing AP on 2 GTX295s, but there is a dearth of AP at present.


Data:

    Run Time CPU Time Credit
    12007 8403 100
    11978 8861 93
    11715 8315 94
    11965 8733 103
    9449 6864 78
    11035 8403 99
    11912 8905 104
    11934 8826 103
    11943 8794 101
    12197 8547 96
    11651 8856 103
    6549 4784 55
    12919 8917 95
    11155 8363 87
    13272 8943 106
    11786 8866 95
    12484 8858 102
    13312 8905 100
    12282 8918 101
    12262 8803 105
    11906 8825 95
    11870 8741 102
    11663 8418 100
    10937 8343 106
    11890 8858 101


ID: 1387868 · Report as offensive
Mark Lybeck

Send message
Joined: 9 Aug 99
Posts: 245
Credit: 216,677,290
RAC: 173
Finland
Message 1387913 - Posted: 5 Jul 2013, 7:00:50 UTC - in response to Message 1387195.  


Grrnffx32(nonoggin) becomes grrrf(kibblenoggin) with DCT kibbles in the bowl.

Meow?


The O() notation is what mathematicians and computer scientists use to describe how "hard" a problem is.

https://en.wikipedia.org/wiki/Big_O_notation

O(log n) means as a problem gets bigger the time needed to solve it increases with the log of the size. This is usually considered "good" behavior.

Something like O(n^2) or O(2^n) is considered "bad", it wouldn't be hard to find a problem too big to solve with real-world hardware. Although for small problems the O(2^n) algorithm might be better than the O(log n) algorithm, and you might use it.

If you have time on your hands you can also look in to the related P versus NP problem.


You can also check for example: "Data Structures and Algorithm Analysis in C" from Mark Allen Weis (Second Edition) 1997, Chapter 2 Algorithm Analysis.
ISBN 0-201-49840-5



ID: 1387913 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13835
Credit: 208,696,464
RAC: 304
Australia
Message 1387933 - Posted: 5 Jul 2013, 7:49:15 UTC - in response to Message 1387913.  


It would appear my falling RAC has stopped falling. It hasn't climbed any, just hit bottom.
From 31,500 down to 19,000.
Grant
Darwin NT
ID: 1387933 · Report as offensive
Thomas
Volunteer tester

Send message
Joined: 9 Dec 11
Posts: 1499
Credit: 1,345,576
RAC: 0
France
Message 1387957 - Posted: 5 Jul 2013, 9:26:53 UTC - in response to Message 1387933.  

Good news Grant ! :)
ID: 1387957 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13835
Credit: 208,696,464
RAC: 304
Australia
Message 1389385 - Posted: 10 Jul 2013, 6:15:21 UTC - in response to Message 1387933.  

It would appear my falling RAC has stopped falling. It hasn't climbed any, just hit bottom.
From 31,500 down to 19,000.


Well, mostly stopped falling.
It has slipped down just a little more over the last few days.
Grant
Darwin NT
ID: 1389385 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13835
Credit: 208,696,464
RAC: 304
Australia
Message 1390022 - Posted: 12 Jul 2013, 8:30:45 UTC - in response to Message 1389385.  

It would appear my falling RAC has stopped falling. It hasn't climbed any, just hit bottom.
From 31,500 down to 19,000.


Well, mostly stopped falling.
It has slipped down just a little more over the last few days.

Still falling, slowly.
Grant
Darwin NT
ID: 1390022 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1390071 - Posted: 12 Jul 2013, 12:52:01 UTC
Last modified: 12 Jul 2013, 12:55:40 UTC

Follow the Doctor´s orders: Try to crunch few AP... it´s good for the RAC...

By accident they discover a way to make us all crunch AP´s... That´s calls Creditnew...

Don´t ask me why, but if you want a better RAC after the V7/Creditnew mess, you need to crunch AP´s... Ok i agree it´s totaly insane!
ID: 1390071 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1390190 - Posted: 12 Jul 2013, 17:10:14 UTC
Last modified: 12 Jul 2013, 17:19:31 UTC

The number of credits we receive for a WU seems to have been restored to pre V7 levels.

So now the problem left is the time it takes to process a WU has increased. This is because of 1) for CPUs the fact we, the Lunatic optimized crowd, can't use the Intel highly optimized libraries anymore and 2) the addition of the auto-correlation code. And the thing is, 2) is seriously impacting completion time for GPU WUs.

Now for me, the difference between CPU apps is about 37% longer to process. But for GPU, which is where all the big crunchers are getting a ton of credits, it's now taking, at least for me in some cases, nearly three times as long. Shorties that took 15-20 minutes on the GPU before are now taking 50 minutes. Regular length work units that took about an 65-75 minutes are now taking 90-110 minutes.

Now I'm running a fairly weak AMD GPU and it was doing an equal amount of credits per hour as three of my CPU cores. That's not the case any longer especially with shorties. YMMV but I imagine for a cruncher using better hardware and maybe multiple cards, the difference is amplified since it appears that the auto-correlation code isn't something that scales to the number of GPU cores you have.

That's my current take on the issue. It's not CreditNew anymore. WU X on standard app is back to delivering the same credit as it did before. We are all now just suffering from longer processing times and the faster the GPU you have, the more GPUs you have crunching, the greater the impact.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1390190 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1390897 - Posted: 15 Jul 2013, 14:05:46 UTC - in response to Message 1390190.  


Now for me, the difference between CPU apps is about 37% longer to process. But for GPU, which is where all the big crunchers are getting a ton of credits, it's now taking, at least for me in some cases, nearly three times as long. Shorties that took 15-20 minutes on the GPU before are now taking 50 minutes. Regular length work units that took about an 65-75 minutes are now taking 90-110 minutes.

Just a tip that maybe help, if you run more than 1 WU at a time try a little less, if you run 3 try 2 etc. not sure the right number in you GPU i only use Nvidia´s. The best number depends on each host combination, but as a general roule if you run 3 in the past now 2 is better... Only the fastest GPU´s actualy can handle 3 or more WU at a time due the correlation overhead. You could even have the memory to run more WU at a time but the actual performance is worst with to many running at a time due the overhead the task switch involves. And leave few cores avaiable to feed the GPU, that specialy important if you use AMD CPU (see you GPU usage to find the best combination).
ID: 1390897 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1391014 - Posted: 15 Jul 2013, 18:48:28 UTC - in response to Message 1390897.  

In my case, I know my GPU isn't up to snuff so I only run 1 GPU WU at a time.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1391014 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1391056 - Posted: 15 Jul 2013, 20:39:15 UTC - in response to Message 1391014.  

In my case, I know my GPU isn't up to snuff so I only run 1 GPU WU at a time.

And you mantain one core free to feed the GPU?

ID: 1391056 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1391439 - Posted: 16 Jul 2013, 23:08:30 UTC - in response to Message 1391056.  
Last modified: 16 Jul 2013, 23:09:17 UTC

In my case, I know my GPU isn't up to snuff so I only run 1 GPU WU at a time.

And you mantain one core free to feed the GPU?

Yep. Both before and after the switch to V7.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1391439 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1391905 - Posted: 18 Jul 2013, 8:45:49 UTC
Last modified: 18 Jul 2013, 8:49:05 UTC

Could anyone realy explain why GPU AP crunching (about 1 credit/each 5 secs of processing time by my last validated WU) is actualy paying about 2x more credit per crunching time than GPU MB (about 1 credit/each 10 sec)? Almost 2 months pass after the introduction of V7 so the theory who say that we need to wait the slowest crunchers report their tasks to clear the problems is allready obsolete, i belive we need a new theory. Maybe some flopcounter (or whatever else creditnew use to calculate the credits) is not working fine on MB crunching?
ID: 1391905 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1391945 - Posted: 18 Jul 2013, 11:04:36 UTC - in response to Message 1391905.  
Last modified: 18 Jul 2013, 11:37:01 UTC

Could anyone realy explain why GPU AP crunching (about 1 credit/each 5 secs of processing time by my last validated WU) is actualy paying about 2x more credit per crunching time than GPU MB (about 1 credit/each 10 sec)? Almost 2 months pass after the introduction of V7 so the theory who say that we need to wait the slowest crunchers report their tasks to clear the problems is allready obsolete, i belive we need a new theory. Maybe some flopcounter (or whatever else creditnew use to calculate the credits) is not working fine on MB crunching?

It's Because of the relative inefficiency of the v7 apps, especially the autocorrelation portion, when Jason and Raistmer produce faster GPU apps, Credit will start to improve, and because the apps are faster you'll also get more work done,

NewCredit doesn't use flopcounters, it uses the runtime statistics of all the Stock users to generate app_version.pfc_avg and host_app_version.pfc_avg averages, it then normalises them for cross version, and also normalises cross hosts,

Finally after all the normalisation, Credit will either be:

if app.min_avg_pfc is defined
C = app.min_avg_pfc*wu.fpops_est*cobblestone_scale
else
C = wu.fpops_est * cobblestone_scale

What Eric might be doing to recover some of the Credits, is to Bump up the v7 Wu's wu.fpops_est (<rsc_fpops_est>) value, effectively making it a bigger Wu. (It was suggested at Beta, I have no idea whether it was acted on)

If we were to introduce the Optimised Astropulse CPU app as Stock, most of the CPU times would fall by 2 or 3 times, the Statistics would change, the AP GPU apps wouldn't seem so fast any more, and Credit per Wu would suffer, a lot.

Claggy
ID: 1391945 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1391974 - Posted: 18 Jul 2013, 13:00:37 UTC
Last modified: 18 Jul 2013, 13:21:09 UTC

Thanks Claggy

If you tell i´m sure this is the right explanation but totaly "bugs my mind".

Can´t understand why: "relative inefficiency of the v7 apps" i allways hear, optimized app´s are faster and more optimized... x41zc was a success when Jason allow us to use, we all was very happy at that time, all working fine and our hosts produces a lot more science (less time to process a WU so our GPU´s actualy crunches more per day) and a lot more credit (at least on Kepplers GPU´s with cuda50)... Then why now that apps are less eficient?

Have you any ideia how much time still needed for "normalisation"? Almost 2 months allready pass.

You last paragraph (about Optimized AP apps) makes me entering in "panic mode", that indicates another large falling in our RACs...

Ok i agree credits have no real value, but is the only way we have to measure our real production, the way creditnew actualy "credits" the MB done after v7 is realy a "pain in the A...", is dificult to understand why a WU with more processing time receives less credit than a faster to process WU as i post few examples on the threads, yes i know is because the wingmans efect, but is wierd anyway. Now by just imagine that could be happening with AP... i need a beer!

Hope Eric make the adjust fast, to be fair, MB and AP must "paid" the relative same number of meaningless credits to allow us crunch both.

In my particular case crunching AP is complicated because i normaly uses I5 (4 cores so a maximum of 4 AP could runs on that host) and the 690 (actualy a double GPU) and more than 1 GPU per host so when i run 4 AP on that host actualy 1 or 2 GPU are waiting for a free core, that not happening with MB. So i´m in a cross road, if i run AP to get more credit 1/2 of my GPU´s are useless and all PU WU crunching is suspended, if i run MB i make all working but i receive 1/2 of the credit... at the end is a waste of processing time/power.

Yes i need to frequent more beta to keep posted on last news.

Thanks for your time and pacience and don´t forget to join us in the Wow Event.
ID: 1391974 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1392089 - Posted: 18 Jul 2013, 17:42:16 UTC - in response to Message 1391974.  

...Can´t understand why: "relative inefficiency of the v7 apps" i allways hear, optimized app´s are faster and more optimized... x41zc was a success when Jason allow us to use, we all was very happy at that time, all working fine and our hosts produces a lot more science (less time to process a WU so our GPU´s actualy crunches more per day) and a lot more credit (at least on Kepplers GPU´s with cuda50)... Then why now that apps are less eficient?...


That part's only a factor, optimisation related, probably best in its own thread where I could illustrate with analogies if that method works for you.

Have you any ideia how much time still needed for "normalisation"? Almost 2 months allready pass.
Not sure anyone knows this for sure, but many are saying it has normalised.

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1392089 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1392190 - Posted: 18 Jul 2013, 23:48:36 UTC - in response to Message 1392089.  

Not sure anyone knows this for sure, but many are saying it has normalised.


In looking at tasks going through the i7, enhanced wus are still receiving significantly more credit than v7 wus (even though enhanced wus take ~50% less time to crunch). Enhanced and AP seem to be roughly equivalent whilst v7 stands out like the proverbial sore thumb. Simple logic would dictate that if a wu takes twice as long to crunch then one should expect to see twice the credit. I am not seeing this with v7 wus compared to enhanced. Given we have scientists running the project, I would ask where is the offline analysis which supports the proposition that there is nothing wrong with the credit system ???








ID: 1392190 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 20 · Next

Message boards : Number crunching : Observation of CreditNew Impact (2)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.