credits and workunits: a fair system needed for Boinc


log in

Advanced search

Message boards : Number crunching : credits and workunits: a fair system needed for Boinc

1 · 2 · 3 · Next
Author Message
nemesis
Avatar
Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131785 - Posted: 25 Jul 2011, 16:53:56 UTC

continue the discussions please
____________

Sirius B
Volunteer tester
Avatar
Send message
Joined: 26 Dec 00
Posts: 10843
Credit: 1,577,456
RAC: 2,300
Israel
Message 1131788 - Posted: 25 Jul 2011, 17:01:57 UTC
Last modified: 25 Jul 2011, 17:02:35 UTC

Thanks Phud.

Seti Classic pulled in a lot of crunchers & with each level of achievements (set by Seti project) one was able to print/save a certificate showing that achievement.

There were no credits involved, yet people crunched the project & at a pace that would not be imaginable today with the current state of technology. Anyone for 56k dial-up?

Personally, I used to enjoy watching the hours accumulate especially as I only used to use the Net from a Fri evening at 6pm until 8am Mon morning as BT had a rate of 0.01p a minute.

Thought for the day.

With some of the crunching beasts on the projects, I wonder what total hrs will be racked up per day?

Would be nice to find out. (not by using a speadsheet guys, by crunching...lol)
____________

Profile Bill Walker
Avatar
Send message
Joined: 4 Sep 99
Posts: 3350
Credit: 2,030,576
RAC: 1,905
Canada
Message 1131789 - Posted: 25 Jul 2011, 17:02:13 UTC
Last modified: 25 Jul 2011, 17:03:27 UTC

A bazillion credits, plus a buck fifty, will buy you a cup of coffee.

Credits have some meaning to an individual, to provide some measure of crunching power as affected by settings, new hardware, time dedicated, etc. Beyond that they have no meaning. None. Zero. Zip.

One user can work a lot harder to get 50 credits than another user does to get 50 million. Who is the better cruncher? What is a better cruncher? Both questions defy a meaningful answer. Unless you are ready to personally fund an entire project, the 50 RAC user is just as meaningful as the 50 million RAC user. They are both just somebody doing what they can.

Therefore, do what you like with credit, just don't keep changing it. If double credit makes you feel good, good for you. It just confuses me.
____________

Sirius B
Volunteer tester
Avatar
Send message
Joined: 26 Dec 00
Posts: 10843
Credit: 1,577,456
RAC: 2,300
Israel
Message 1131790 - Posted: 25 Jul 2011, 17:08:30 UTC

But that's the point Bill. Until the economy went pearshape, I used to run a nice little farm (still do but not as large) with it being mainly used for business purposes.

During the time the full farm ran, it was nice to see the credits mount up & the regular competition between team members on my team forum.

Since the credit situation has been changed & not for the 1st time, they have begun to become totally meaningless & the competition seems to have jaded & that's a shame.

The science is still getting done, but to me at least, it seems it's beginning to lack the fun of participating.
____________

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4055
Credit: 111,100,872
RAC: 147,763
United States
Message 1131791 - Posted: 25 Jul 2011, 17:08:47 UTC

Personally I liked the old system of counting the number of tasks a system did. I can think of a "credit system" that would work for S@H, but I don't know how it work work for other projects.
With this project tasks in the .42 AR are "normal" with higher and lower AR taking more or less time. So credit in the "normal range would be 1. Then as the AR get closer to each side of the VLAR and VHAR credits would be higher and lower. With 0.01 being the lower cap and an appropriate value being the high cap. Perhaps 2.0 being the max would be a good artificial limit, but whatever number it might be should be based on some kind of scale that someone without a degree in mathematics can figure out in a few minutes.

I imagine most BOINC projects wouldn't work for a system like this. That is why they came up with a very complex system that few understand. That way no one can tell if it is actually working. :)
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

rob smithProject donor
Volunteer tester
Send message
Joined: 7 Mar 03
Posts: 8294
Credit: 54,938,790
RAC: 74,859
United Kingdom
Message 1131792 - Posted: 25 Jul 2011, 17:09:58 UTC

Any credits system must take account of the actual processing rate, normalise it, then allocate a fair score per normalised second. Thus if you have a mega fast processor which turns round WU in a very short time you will get (or should get) the same number of brownie points as a person with a very pedestrian processor - for a given "standard" WU.

The problem is defining the standard WU, and countering those who gain advantage by moving WU between processors. Neither of these is an easy problem to solve. Particularly when one considers that a given cruncher might be crunching 24/7 for a few weeks then doing 0.5/7 for the next month just because that's the owner's lifestyle.

Currently the credits model uses "peak flops" which I don't think is the right model, because it is very easy to get a very high peak flops while having a very low "average flops". One way around this might be to periodically distribute a standard WU and use that to calibrate each cruncher. But this has flaws - what happens if you change the hardware, or software, between calibration runs? One solution is to rerun the calibration if a "significant anomaly" is detected, and all results for that cruncher be frozen until a calibration cycle is completed.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

nemesis
Avatar
Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131793 - Posted: 25 Jul 2011, 17:10:12 UTC

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.

the recent "doubling" and "pay for doubling"
does nothing but show how the credit system can be corrupted
for no real reason.

if you have to resort to cheap gimmicks to keep a project afloat
what does that really say about the honesty of the project.
____________

Sirius B
Volunteer tester
Avatar
Send message
Joined: 26 Dec 00
Posts: 10843
Credit: 1,577,456
RAC: 2,300
Israel
Message 1131797 - Posted: 25 Jul 2011, 17:16:12 UTC - in response to Message 1131793.

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.

the recent "doubling" and "pay for doubling"
does nothing but show how the credit system can be corrupted
for no real reason.

if you have to resort to cheap gimmicks to keep a project afloat
what does that really say about the honesty of the project.



That is the best statement I've seen regarding all the credit debates I've read for the past 4 years.
____________

Profile Bill Walker
Avatar
Send message
Joined: 4 Sep 99
Posts: 3350
Credit: 2,030,576
RAC: 1,905
Canada
Message 1131799 - Posted: 25 Jul 2011, 17:25:55 UTC - in response to Message 1131793.

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.


That sort of worked, until GPUs came along. It also gets very meaningless very quickly if you start comparing a S@H shorty WU (minutes to complete) to a CPD WU (months to complete on the same machine). Does the relative completion time reflect the relative value of the science? Now there's a question to debate.

Despite my somewhat cynical earlier post, I do realize that credit is one way of attracting new crunchers, and one way of encouraging crunchers to upgrade hardware. We have to recognize that, and that is where things get complicated. Personally, I'm in it for the science. And the free toaster.
____________

Sten-Arne
Volunteer tester
Send message
Joined: 1 Nov 08
Posts: 3360
Credit: 19,542,264
RAC: 18,565
Sweden
Message 1131801 - Posted: 25 Jul 2011, 17:33:21 UTC

I kind of started to like this CreditNew system. It's hilarious actually, One AP WU can give 5 in credits, and another with similar blanking up to 5000.

Forrest Gump mode ON CreditNew is like a box of chocolates. You never know what you're gonna get." Forrest Gump mode OFF
____________

nemesis
Avatar
Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131805 - Posted: 25 Jul 2011, 17:37:46 UTC - in response to Message 1131799.

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.


That sort of worked, until GPUs came along. It also gets very meaningless very quickly if you start comparing a S@H shorty WU (minutes to complete) to a CPD WU (months to complete on the same machine). Does the relative completion time reflect the relative value of the science? Now there's a question to debate.

Despite my somewhat cynical earlier post, I do realize that credit is one way of attracting new crunchers, and one way of encouraging crunchers to upgrade hardware. We have to recognize that, and that is where things get complicated. Personally, I'm in it for the science. And the free toaster.


if you look at boincstats for instance....
you see a breakdown of credits achieved at different projects.
the same could happen with a wu completion + hours processed.

say something like this

seti 100 wu's completed, 1000 hours processor time.
einstein@home 101 wu's completed, 956 hours processor time.

it would give meaning to each projects processing needs.
a processor intensive project would essentially be shown by its
ratio of wu's/processor time...
____________

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5783
Credit: 57,832,379
RAC: 48,350
Australia
Message 1131813 - Posted: 25 Jul 2011, 17:48:23 UTC - in response to Message 1131805.

it would give meaning to each projects processing needs.
a processor intensive project would essentially be shown by its
ratio of wu's/processor time...

Which is what Credits do.
____________
Grant
Darwin NT.

nemesis
Avatar
Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131816 - Posted: 25 Jul 2011, 17:52:14 UTC - in response to Message 1131813.

really?

how so?
____________

Profile Bill Walker
Avatar
Send message
Joined: 4 Sep 99
Posts: 3350
Credit: 2,030,576
RAC: 1,905
Canada
Message 1131819 - Posted: 25 Jul 2011, 17:53:42 UTC - in response to Message 1131805.
Last modified: 25 Jul 2011, 17:57:28 UTC

I'm beginning to see some merit in Phud's idea. The individual user can track hours per WU to see if their tweaks and hardware upgrades have the intended results. Changes in WU size and/or complexity (which will continue to happen) will just require you to "wait a bit" till the numbers settle down again.

Still not sure about cross project comparisons though. Anything done will be subject to "players". If Project X make WUs shorter, will that attract some users? Probably, but nothing is gained for the science. If an improvement in Project X science results in longer WUs, will that motivate some users to switch to another project? Probably yes again, despite an improvement in science.

This is my final offer. Do what you like, I'm here for science. If we scare off some users, [shrug]oh well[/shrug]. I haven't seen a forum yet at any project with people complaining about too many users (except occasionally here at S@H). All I ever see is people demanding more WUs. Whether their motive is more science or more credit is really irrelevant, as long as more science gets done.
____________

Profile Tazz
Volunteer tester
Avatar
Send message
Joined: 5 Oct 99
Posts: 137
Credit: 34,342,390
RAC: 0
Canada
Message 1131820 - Posted: 25 Jul 2011, 17:55:32 UTC - in response to Message 1131793.

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.
<SNIP>


Waaaaay back in '99 when I started crunching it was because I found the xx.xx.SETI usenet newsgroup. In it people were posting their times per wu; Guy1-"I can do one wu in 3hrs and 10 min.". Guy-2"Well I can do one in 2hrs and 58 min.". Guy-1"What have you got (motherboard setting) set to?". And so on.

I had just spent a load of money for a 233 MMX PC and wanted to make sure it was running comparable to other 233s. Running a wu, checking the time, changing some settings, getting another wu ... It was fun. But I digress.

When we put a new GPU through it's paces how to we usually share the results? One normal AR wu in xx minutes. Simply because it takes a while to get an inaccurate RAC that most people only see as 'higher is better'.

I'm all for the hours spent on wu's completed.


SETI@home classic workunits 12,500
SETI@home classic CPU time 54,781 hours
____________
</Tazz>

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 1131837 - Posted: 25 Jul 2011, 18:22:51 UTC

The best credit system we've had was the original benchmark * time.

Because bechmarking is an inexact science, claimed credit wasn't really consistent, but requiring three results for a quorum, and throwing out high and low produced a fair and reasonably accurate result.

(Fair means "equitable" -- everyone got a bonus sometimes, but the odds of a bonus, or a low result, were constant)

Unfortunately, that doesn't work for GPUs unless BOINC has benchmarks for CUDA, OpenGL, or whatever.

The big problem we have when we leave benchmark * time is a lack of *comparable* credit from project to project. Science applications vary, and projects have to determine a scaling factor to convert "flops" into credit.

Some projects don't care, and don't put much effort into awarding credit. Others attract credit hounds by awarding more credit than they should.

In an idea world, credits would scale automatically between projects.

Still, it's amazing how "nothings" turn into "money," and people spend actual money to collect "BOINC money."

Profile James SotherdenProject donor
Avatar
Send message
Joined: 16 May 99
Posts: 8673
Credit: 32,948,927
RAC: 55,662
United States
Message 1131851 - Posted: 25 Jul 2011, 18:52:09 UTC

Interesting read, and some good solid points. Id go for the hours and work units crunched. But really I crunch because I believe in Seti@Home. Yes I have back up projects, But only use them when we are down for days and Im out of work.
If Dr. A. says no more credit will be awarded I will still crunch. I dont care what other projetcs pay in credit. But maybe we can have Dr. A. make a picture of a Seti@Home toaster:)
____________

Old James

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12372
Credit: 6,674,046
RAC: 8,749
United States
Message 1131865 - Posted: 25 Jul 2011, 19:15:34 UTC

I'm not sure there is any fair way to compare across projects. The issue is that some projects use only integer math and others only floating point math. At present only the floating point speed benchmark is used in calculations.

An arbitrary conversion can be done saying that 5 integer adds = 1 floating point add, but is that fair across all processors?

I think people are forgetting the original reason they went from units to credits. In the old system a shorty counted as much as a long. Not fair. But CPU time was also counted. That took the short/long into account, but did not take into account processor speed.

The credit system was supposed to take both into account and come up with a single number. They tried benchmarks to see how fast the processor ran. What they got was a very crude system that is only fair if the science is the benchmark.

As I understand creditnew it is more to prevent user fraud in open source than it is a serious attempt to even out credits. It addresses a optimized app that reports 100x work by comparing the claimed credit numbers with other machines. It can't do much about a project that claims 2X for every unit.

Now perhaps if the credit software consulted manufacturers data sheets on the number of cycles for each instruction and the speed of the clock on the system something close to fair could be implemented. Then each science application would have to be counted as to how many of each type of instruction was run. Not going to happen.

So what has happened is that each project admin has been left on his word that his correction factor is honest. As crunchers say different projects pay different amounts, we know that the correction factors are not set uniformly, can't say if that is intentional or not. I think most people believe it is intentional in some cases. I don't know how the BOINC software can catch that unless it debugs the science application to count instructions (which would make it crawl) Or do science applications need to be registered with a central authority, which would end optimized apps.

This isn't ever going away.

____________

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4055
Credit: 111,100,872
RAC: 147,763
United States
Message 1131875 - Posted: 25 Jul 2011, 19:33:55 UTC

I do miss the old days of seeing how much CPU time I had contributed.
Seeing the stats as:

Results received: 93865 Total CPU time: 98.567 years Average CPU time per work unit: 9 hr 11 min 55.7 sec

This gave me several goals to work towards. One of the big ones I was working on was an average time under 10 hours. I was hoping to end with 100,000 wu's and 100 years of CPU time, but I missed that mark.

With the current credit system(s) I can really only work towards credit number goals. Like "try to make these machines to make 100,000 credits this month". Which doesn't really tell me how much work the machine is doing, just that it gets granted that amount of credit.

Maybe they could implement these values again. Not to replace the current credit system, but to give us something else to track for fun. I imagine not all of this information has been stored since the switch over to BOINC. SO they would have to pick an arbitrary date to implement it. Much liek they do for the change in the credit system(s).
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

nemesis
Avatar
Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131879 - Posted: 25 Jul 2011, 19:37:24 UTC

"I think people are forgetting the original reason they went from units to credits. In the old system a shorty counted as much as a long. Not fair. But CPU time was also counted. That took the short/long into account, but did not take into account processor speed."

this is why a simpler method is better...

if you're running a pentium 4 and getting 1 wu every 12 hours and you see
someone else getting 48 wu's in 12 hours you'll look and see someone running
opt's apps and a faster cpu/gpu.
is that discrimination? no...you choose to crunch on what you have at your disposal.

that very scenario could make the p4 user look into opt'd apps..
and maybe upgrade.

i'm not saying its perfect to run wu's completed/hours processed, but it'd be a
lot less corrupted that the system is now.


____________

1 · 2 · 3 · Next

Message boards : Number crunching : credits and workunits: a fair system needed for Boinc

Copyright © 2014 University of California