Observation of CreditNew Impact (2)

Message boards : Number crunching : Observation of CreditNew Impact (2)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 20 · Next

AuthorMessage
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1393034 - Posted: 20 Jul 2013, 22:00:27 UTC - in response to Message 1392762.  
Last modified: 20 Jul 2013, 22:11:17 UTC


Which shows that the system is broken.
The whole point of Credits was to recognise work done.
Since the stock application is now using optimisations it should result in the stock application now earning similar credit to what the previous optimised application did. It shouldn't result in less credit being granted for the same work being done, it should have resulted in more credit being granted as the stock application is now doing more work.
The stock application is now doing more work- it should provide more credit to recognise that. For those that were previously using the optimised application the amount of credit received should have remained about the same (the newly introduced auto correlations would probably have resulted in some drop in credit- not the 40% or more that has occured). Credit deflation for those that used the optimised application should not have been the result.


Even in V6 a standard app is rewarded the same credit as an optimized app, not more, not less, the same. We had large RACs ... because we could do more WUs in the same amount of time, nothing more. Now that advantage is largely gone. Our apps got slower while standard apps got faster.

Now however you want to set up a reward structure at some point you have to define a "gold standard". This valid WU processed with the "gold" standard app reports it took Y flops to do so it's worth X credits. Doesn't matter what app is used or what platform it's run on, it's worth X credits.

CreditNew attempts to do this and at it's root is still flops rate, based on total flops reported and elapsed time. The raw data is sanity checked and bounded to reduce bad data or cheaters and an exponentially-weighted average of flops rate of each host is calculated, normalized against the average reported flop rate of all hosts using the same CPU or GPU. At the end of the day your host ends up with a normalized scale factor that adjusts the flops reported and that is used to calculate credits.

This all relies on the reported flop count from the standard app. So here's the question, did this value change significantly between V6 and V7? It seems like it should have due to the auto-correlation function but what if the the total flops used for all the old analysis functions went down due to a better method to estimate flops? And since our optimized apps still use the baseline code from the standard app that change is reflected in our app as well. Or maybe it's as simple as the auto-correlation function doesn't add to the reported flop count? or by enough to compensate for the addition time it took?

The key is the flops count reported by the standard app and it's elapsed time that sets everything else up. If it hasn't changed significantly, our credits for a valid app won't change significantly and our slower processing rate compared to V6 means our RAC will be lower.

Info on CreditNew.

As for "recognize work done", well in terms of WUs per day we are now doing fewer so we are rewarded for work done, with fewer credits.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1393034 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13832
Credit: 208,696,464
RAC: 304
Australia
Message 1393129 - Posted: 21 Jul 2013, 2:33:50 UTC - in response to Message 1393034.  
Last modified: 21 Jul 2013, 2:34:21 UTC

As for "recognize work done", well in terms of WUs per day we are now doing fewer so we are rewarded for work done, with fewer credits.

Which is not what the credit system was designed for.
It was designed to recognise the amount of work done- eg, more credit for a VLAR as it required more processing than a shortie.
I may be doing less WUs per day, but my hardware has not changed, the amount of work i do per day has not changed, therefore the amount of credit awarded should not have changed- No more, nor less. But it has been cut, significantly.
Grant
Darwin NT
ID: 1393129 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1393188 - Posted: 21 Jul 2013, 8:13:31 UTC
Last modified: 21 Jul 2013, 8:27:47 UTC

My point is, no mather what is the base line and the credit given per unit of processing, both credits must be the same MB & AP. The way there are now (AP paid´s 2 x more than MB) is not good. On the same host 1 hr of processing an AP must receive the same credit as 1 hour of processing MB or the balance will be broke.

It´s simple: if a MB WU that takes for example 20 min to process now receive 100 credits, a AP WU that takes 2 hours to process (6 times more processing time on the same hosts) must receive 600 credits nothing more or less so the balance is keeped.

Your RAC can´t be determinated by the type of WU (AP or MB) you crunch.

BTW Someone know why most of the MB WU crunched by a GPU with the AR of about 1.5 paids so few credits? about 40 or less.
ID: 1393188 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34345
Credit: 79,922,639
RAC: 80
Germany
Message 1393192 - Posted: 21 Jul 2013, 8:33:00 UTC

First of all you cant compare V7 and AP`s.
This will never happen.

A VHAR unit will always receive less credits because of less science.
And its processed faster by CPU and GPU.
Each unit above 0.7 will get lesser credits than a 0.4 and lower AR.



With each crime and every kindness we birth our future.
ID: 1393192 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1393227 - Posted: 21 Jul 2013, 10:44:38 UTC - in response to Message 1393129.  
Last modified: 21 Jul 2013, 11:08:31 UTC

As for "recognize work done", well in terms of WUs per day we are now doing fewer so we are rewarded for work done, with fewer credits.

Which is not what the credit system was designed for.
It was designed to recognise the amount of work done- eg, more credit for a VLAR as it required more processing than a shortie.
I may be doing less WUs per day, but my hardware has not changed, the amount of work i do per day has not changed, therefore the amount of credit awarded should not have changed- No more, nor less. But it has been cut, significantly.


In BOINC "work" is defined as reported flop count, plain and simple. Yes VLARs report a higher flop count than shorties thus they get more credits, that hasn't changed. The method used to calculate credits has not changed from V6 to V7. And apparently the flop count being reported by the official standard V7 app hasn't changed much when compared to the official standard V6 app. Flop count isn't CPU time.

So "the problem" has nothing to do with CreditNew and everything to do with flop count as reported by the official standard app.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1393227 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1393240 - Posted: 21 Jul 2013, 11:55:14 UTC - in response to Message 1393192.  
Last modified: 21 Jul 2013, 12:35:56 UTC

First of all you cant compare V7 and AP`s.
This will never happen.

A VHAR unit will always receive less credits because of less science.
And its processed faster by CPU and GPU.
Each unit above 0.7 will get lesser credits than a 0.4 and lower AR.

I not comparing AP with MB, i comparig the credit gived by each one, a totaly diferent thing.

If credit is related to the processing power used to crunch the WU, yes you can compare the results.

As all about credit makes no sense this makes no sense to, let try to explain:

If my hosts spend 1 hour to crunch a AP WU it´s receive a "paid" of 800 credits for this hour of work.

If that is ok then: Why now (after V7) if the same hosts spend the same hour to crunch MB WU it receive only 400 credits?

That´s what i talk about, we the "almost normal" humans allways try to find a balance in anything, is the way the brain works.

ABout Vlars, my nvidias did not receive vlars anymore, thanks to our complains in the forum someone (Eric) hear that and make the changes, so can´t say nothing about them. I know the roule bigger the AR less the credit due the processig time need to crunch and that is totaly right no questions about.

I was talking is why now a 12 min to process MB WU receives 40 credit against 100 of a 18 min to process MB (simple math about 66% of the time but only 40% of the credit is given). Before the diference was not so big.

OK i close my participation on this thread, have no meaning spend more time to try to show something nobody wants to see, or maybe, just maybe, the way it´s working now, that makes us all crunch a lot more AP, could be good from the projects point of view and we will never reach the "balance".

Have a good day/night and please take no ofense on my posts.
ID: 1393240 · Report as offensive
Sleepy
Volunteer tester
Avatar

Send message
Joined: 21 May 99
Posts: 219
Credit: 98,947,784
RAC: 28,360
Italy
Message 1393265 - Posted: 21 Jul 2013, 15:24:50 UTC - in response to Message 1393227.  
Last modified: 21 Jul 2013, 15:25:56 UTC

In BOINC "work" is defined as reported flop count, plain and simple. Yes VLARs report a higher flop count than shorties thus they get more credits, that hasn't changed. The method used to calculate credits has not changed from V6 to V7. And apparently the flop count being reported by the official standard V7 app hasn't changed much when compared to the official standard V6 app. Flop count isn't CPU time.

So "the problem" has nothing to do with CreditNew and everything to do with flop count as reported by the official standard app.

This would put everything under another light.
It would mean that, possibly due to the new correlation task, the present optimised application is not so much optimised yet.
Therefore, being the application less efficient, it generates less RAC.
We have this far discussed under the assumption that the V7 applications, stock or optimised, are roughly as efficient as the old V6, only taking longer due to more work to be done. This may prove wrong.

Therefore, we would have that those who ran on stock had no big bumps, but those who were on optimised are now on a (evidently, much) less optimised application and therefore their RAC falls.
The developers may need some time to optimise also this (no blame on them! This is not the reason for this post!).

Do we have any other confirmation (but the flop count figure may well be a first one)?

Sleepy
ID: 1393265 · Report as offensive
Profile Link
Avatar

Send message
Joined: 18 Sep 03
Posts: 834
Credit: 1,807,369
RAC: 0
Germany
Message 1393310 - Posted: 21 Jul 2013, 16:35:45 UTC - in response to Message 1392934.  
Last modified: 21 Jul 2013, 16:37:07 UTC

If you have more powerfull hardware, or a more effcicent application, you should receive more credit per hour.

You do. If you use better optimized applications than stock, you receive more credit per hour than with stock.



A more optimised stock application should not result in a drop in credit for those that have always been using such applications, but more credit for the stock application- reflecting the greater amount of work done.

Yes, it should. With more efficient application less computing is needed to do the same job. Less computing, less credit.



I may be doing less WUs per day, but my hardware has not changed, the amount of work i do per day has not changed, therefore the amount of credit awarded should not have changed- No more, nor less. But it has been cut, significantly.

It should not have been cut if you were using stock apps and in case of properly working credit system. Opt apps make your hardware virtually faster than it actually is, it makes for example a 2GHz CPU as fast as 3GHz CPU on stock (don't care if the numbers are exact). Now with v7 opt apps don't do it as much as v6 did, because v7 stock apps are better optimized, so the 2GHz CPU running optimized might be just as fast as a 2.5GHz CPU running stock. So in this example the hardware got virtually slower by 0.5GHz.
ID: 1393310 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1393324 - Posted: 21 Jul 2013, 16:59:09 UTC

The kitties say......
"Dead Issue."

It's not going to be changed, other than whatever small auto adjustments remain.

So, give it up folks.

Not even on my radar anymore.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1393324 · Report as offensive
Profile janneseti
Avatar

Send message
Joined: 14 Oct 09
Posts: 14106
Credit: 655,366
RAC: 0
Sweden
Message 1393334 - Posted: 21 Jul 2013, 17:25:15 UTC - in response to Message 1393310.  
Last modified: 21 Jul 2013, 17:26:29 UTC

If you have more powerfull hardware, or a more effcicent application, you should receive more credit per hour.

You do. If you use better optimized applications than stock, you receive more credit per hour than with stock.


This is not true. A single unit gives the same credit to all wingmans involved regardless of there applications.
The credit for a unit is also predetermined by the server.
ID: 1393334 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1393339 - Posted: 21 Jul 2013, 17:37:16 UTC - in response to Message 1393334.  

If you have more powerfull hardware, or a more effcicent application, you should receive more credit per hour.

You do. If you use better optimized applications than stock, you receive more credit per hour than with stock.


This is not true. A single unit gives the same credit to all wingmans involved regardless of there applications.
The credit for a unit is also predetermined by the server.

Yes, but the statement meant that by running optimized, you can do more work per hour. Not get more credits per WU.

And not sure how true that statement is anymore, as the stock app is now the opti app.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1393339 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13832
Credit: 208,696,464
RAC: 304
Australia
Message 1393352 - Posted: 21 Jul 2013, 18:06:34 UTC - in response to Message 1393227.  

In BOINC "work" is defined as reported flop count, plain and simple.

Not so. Credits are meant to indicate work done.
The FLOP count (of work done, not the estimate for the hardware) is the method used, and it used to work before Credit New.
Grant
Darwin NT
ID: 1393352 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1393376 - Posted: 21 Jul 2013, 20:32:30 UTC - in response to Message 1393352.  

In BOINC "work" is defined as reported flop count, plain and simple.

Not so. Credits are meant to indicate work done.
The FLOP count (of work done, not the estimate for the hardware) is the method used, and it used to work before Credit New.


CreditNew was being used in V6. The CreditNew algorithm hasn't changed for V7.

CreditNew relies on the Flop Count as reported in the results as calculated by the app. That value is scaled to handle differences between app (GPU and CPU) Flop Counts as well as hosts. The scaled value of Flop Count is used to calculate credits reported.

So if the methodology, the algorithm behind CreditNew hasn't changed then it must be the data it's being fed is "wrong". That the Flop Count being reported isn't reflecting the amount of actual "work" being done.

Or is your contention that CreditNew's scaling mechanism will scale out the additional "work" (Flop Count) over V6 and thus in the end use a scaled value that's close to what it was in V6?

"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1393376 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1393377 - Posted: 21 Jul 2013, 20:37:32 UTC - in response to Message 1391945.  


If we were to introduce the Optimised Astropulse CPU app as Stock, most of the CPU times would fall by 2 or 3 times, the Statistics would change, the AP GPU apps wouldn't seem so fast any more, and Credit per Wu would suffer, a lot.

Claggy


And exactly this part makes CreditNew pure evil for our project at least.
Project improvement will hurt credit-dependent participant of project. It's non-sense for prarticipation stimulating feature like whole this "credits" idea is.

SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1393377 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1393385 - Posted: 21 Jul 2013, 21:02:24 UTC - in response to Message 1393377.  

There's one other thing that impacts overall performance. The apps could just be slower.

Once upon a time I had a RAC of 4200 or so. My GPU MB task was, IIRC r390 for ATI/AMD. Then word came out that there was a "new" version of the ATI/AMD app so I dutifully installed it, r1817 and my RAC dropped to 3900. Why? Because my GPU WUs now took longer to process.

It wasn't doing any AC that V7 does, it was just slower. Once Lunatics put out their V7 package it got upgraded to r1843 and is even slower now since it does the AC pass.

So some loss of performance may be due to code changes to support V7, a different compiler and optimized FFT libraries.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1393385 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13832
Credit: 208,696,464
RAC: 304
Australia
Message 1393510 - Posted: 22 Jul 2013, 7:14:06 UTC - in response to Message 1393376.  

So if the methodology, the algorithm behind CreditNew hasn't changed then it must be the data it's being fed is "wrong". That the Flop Count being reported isn't reflecting the amount of actual "work" being done.

No, what is wrong is the way credit new works. It's the methodology that is wrong, or it's iplementation.
As i mentioned in an earlier post- improving the efficiency of the stock application should not penalise those that have been using a more efficient application previously. Their level of credit should have remained unchanged, the level of the credit for the stock application should have increased, as it is a more efficient one, and they are doing more work. That didn't happen. Those that were using the optimised application now get less credit for doing work, the stock one remains unchanged, even though it is doing more work & doing it more efficiently. Deflation, on a grand scale.

Grant
Darwin NT
ID: 1393510 · Report as offensive
Profile Link
Avatar

Send message
Joined: 18 Sep 03
Posts: 834
Credit: 1,807,369
RAC: 0
Germany
Message 1393537 - Posted: 22 Jul 2013, 10:46:33 UTC - in response to Message 1393510.  

As i mentioned in an earlier post- improving the efficiency of the stock application should not penalise those that have been using a more efficient application previously.

It does not penalise anybody, only the benefit of using optimized applications is not as high as it was before. If the stock applications were even better, everyone would be runnng stock and get stock credit, just like on many other projects, that don't even accept anonymous platform. If SETI would not have accepted anonymous platform and the apps became more efficient at one point, than the credit per WU would have gone down as well while the credit per day should have stay at the same level.

Read my previous post, optimized apps make your computer faster than it actually is, v7 opt apps don't do it as much as v6 did. Consider the additional credits you got for running optimized as a bonus for installing the optimized applications. If one day the stock apps be full optimized, you will not need to install anything but you will also not get that bonus.
ID: 1393537 · Report as offensive
Keith White
Avatar

Send message
Joined: 29 May 99
Posts: 392
Credit: 13,035,233
RAC: 22
United States
Message 1393548 - Posted: 22 Jul 2013, 11:14:36 UTC - in response to Message 1393510.  
Last modified: 22 Jul 2013, 11:42:13 UTC

An optimized app shouldn't produce more credit than the standard app on the same WU. The optimized app produces more credits by simply doing more WUs in the same amount of time.

The measurement of work done is the FLOPS Count. Not FLOPS/S, total number of floating point operations. Always has.

In days of old it was simply the whetstone benchmark of the host times elapsed CPU time. This is were the notion of "takes longer so more credits" come from.

Then it was reported FLOPS total by the app. This meant the non-standard apps had to report similar results to get the same credit or used a coded scale factor. If the standard app changed the non standard app might have to be rebuilt with a new scale factor.

So they came up with a way calculate this scale factor on the fly, comparing the running average of FLOPS count per WU of each host to the global running average of FLOPS count per WU as done by the standard app. That's all CreditNew is trying to do.

I think you believe that optimized apps report a higher FLOPS count or credits than the standard app but it doesn't. Less than a one percent difference (more like less than a 1/2 of a percent) in FLOPS count. A more efficient app lowers elapse time, not increase FLOPS count.

None of this should have impacted us as long as reported FLOPS count increased somewhat proportionally to the increased time to process for V7 over V6. It didn't. The thing is though, there is nowhere that says it must. If the AC function is less efficient (as measured in FLOPS/S) than the analysis common between V6 and V7, then the total FLOP count doesn't increase proportionally to elapse time.

Example of what I mean (values made up). Lets say the common analysis that's done in both V6 and V7 reports 6 GFLOPS and takes 60 minutes and the new AC analysis found only in V7 reports 100 MFLOPS and takes an additional 10 minutes. So V6 reports 6 GFLOPS in 60 minutes while V7 reports 6.1 GFLOPS in 70 minutes (or 5.23 GLOPS per 60 minutes).

OMG my RAC went down it's broken.

We could simply be looking at an inefficient algorithm for AC. Something as simple as stepping through memory in a way that ends up largely negating the CPU's memory cache as well as the memory controller's prefetch mechanism can significantly impact performance. CPUs are very memory bandwidth dependent and system memory is very slow compared to the needs of a CPU. Easiest way to throttle a CPU's performance is to make it wait for a memory fetch to complete, repeatedly.

Not saying that's the problem but it could be in addition to reporting a low ball FLOPS count by the AC portion of V7.
"Life is just nature's way of keeping meat fresh." - The Doctor
ID: 1393548 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20930
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1393680 - Posted: 22 Jul 2013, 18:14:18 UTC - in response to Message 1392739.  
Last modified: 22 Jul 2013, 18:14:44 UTC

[...]

We've gotten such a sense of credit rate elitism after years of being significantly faster than the standard app that when the standard app catches up to us we get all boo-hooey over our perceived loss of superiority. We are just angry that we are no longer so much better than someone running a standard app. Well we have to learn to get over it.

More a case of the people who have gone to the trouble to install optimsed apps are precisely those people that are most likely the 'hard core' on these forums...


Edit: Also I believe that the auto-correlation code does not add to the reported flop count. So even though the app is doing more, it's not reported as doing more.

If so, then that would explain the lack of credits for the computation time (RAC) compared to previously. It would also explain how "CreditNew" has been befuddled due to erroneous FLOP counts...


Happy fast crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1393680 · Report as offensive
EdwardPF
Volunteer tester

Send message
Joined: 26 Jul 99
Posts: 389
Credit: 236,772,605
RAC: 374
United States
Message 1393709 - Posted: 22 Jul 2013, 20:23:51 UTC - in response to Message 1393680.  

argumentum ad absurdum

I'm so confused ...

in 2015 when intel adds its new instructions including the

SETI R1,R2,R3

instruction that can run a typical WU in 60 sec's using the 1 atomic instruction

SETI R1,R2,R3

should the WU be given

1) More credit for being more efficient
2) less credit for using only 1 atomic Flop
3) credit calculated by some other method

Ed F

(maybe the moderator should just remove this comment :-)

Ed F
ID: 1393709 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 20 · Next

Message boards : Number crunching : Observation of CreditNew Impact (2)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.