credits and workunits: a fair system needed for Boinc

Message boards : Number crunching : credits and workunits: a fair system needed for Boinc
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
nemesis
Avatar

Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131785 - Posted: 25 Jul 2011, 16:53:56 UTC

continue the discussions please
ID: 1131785 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24905
Credit: 3,081,182
RAC: 7
Ireland
Message 1131788 - Posted: 25 Jul 2011, 17:01:57 UTC
Last modified: 25 Jul 2011, 17:02:35 UTC

Thanks Phud.

Seti Classic pulled in a lot of crunchers & with each level of achievements (set by Seti project) one was able to print/save a certificate showing that achievement.

There were no credits involved, yet people crunched the project & at a pace that would not be imaginable today with the current state of technology. Anyone for 56k dial-up?

Personally, I used to enjoy watching the hours accumulate especially as I only used to use the Net from a Fri evening at 6pm until 8am Mon morning as BT had a rate of 0.01p a minute.

Thought for the day.

With some of the crunching beasts on the projects, I wonder what total hrs will be racked up per day?

Would be nice to find out. (not by using a speadsheet guys, by crunching...lol)
ID: 1131788 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 1131789 - Posted: 25 Jul 2011, 17:02:13 UTC
Last modified: 25 Jul 2011, 17:03:27 UTC

A bazillion credits, plus a buck fifty, will buy you a cup of coffee.

Credits have some meaning to an individual, to provide some measure of crunching power as affected by settings, new hardware, time dedicated, etc. Beyond that they have no meaning. None. Zero. Zip.

One user can work a lot harder to get 50 credits than another user does to get 50 million. Who is the better cruncher? What is a better cruncher? Both questions defy a meaningful answer. Unless you are ready to personally fund an entire project, the 50 RAC user is just as meaningful as the 50 million RAC user. They are both just somebody doing what they can.

Therefore, do what you like with credit, just don't keep changing it. If double credit makes you feel good, good for you. It just confuses me.

ID: 1131789 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24905
Credit: 3,081,182
RAC: 7
Ireland
Message 1131790 - Posted: 25 Jul 2011, 17:08:30 UTC

But that's the point Bill. Until the economy went pearshape, I used to run a nice little farm (still do but not as large) with it being mainly used for business purposes.

During the time the full farm ran, it was nice to see the credits mount up & the regular competition between team members on my team forum.

Since the credit situation has been changed & not for the 1st time, they have begun to become totally meaningless & the competition seems to have jaded & that's a shame.

The science is still getting done, but to me at least, it seems it's beginning to lack the fun of participating.
ID: 1131790 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1131791 - Posted: 25 Jul 2011, 17:08:47 UTC

Personally I liked the old system of counting the number of tasks a system did. I can think of a "credit system" that would work for S@H, but I don't know how it work work for other projects.
With this project tasks in the .42 AR are "normal" with higher and lower AR taking more or less time. So credit in the "normal range would be 1. Then as the AR get closer to each side of the VLAR and VHAR credits would be higher and lower. With 0.01 being the lower cap and an appropriate value being the high cap. Perhaps 2.0 being the max would be a good artificial limit, but whatever number it might be should be based on some kind of scale that someone without a degree in mathematics can figure out in a few minutes.

I imagine most BOINC projects wouldn't work for a system like this. That is why they came up with a very complex system that few understand. That way no one can tell if it is actually working. :)
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1131791 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22442
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1131792 - Posted: 25 Jul 2011, 17:09:58 UTC

Any credits system must take account of the actual processing rate, normalise it, then allocate a fair score per normalised second. Thus if you have a mega fast processor which turns round WU in a very short time you will get (or should get) the same number of brownie points as a person with a very pedestrian processor - for a given "standard" WU.

The problem is defining the standard WU, and countering those who gain advantage by moving WU between processors. Neither of these is an easy problem to solve. Particularly when one considers that a given cruncher might be crunching 24/7 for a few weeks then doing 0.5/7 for the next month just because that's the owner's lifestyle.

Currently the credits model uses "peak flops" which I don't think is the right model, because it is very easy to get a very high peak flops while having a very low "average flops". One way around this might be to periodically distribute a standard WU and use that to calibrate each cruncher. But this has flaws - what happens if you change the hardware, or software, between calibration runs? One solution is to rerun the calibration if a "significant anomaly" is detected, and all results for that cruncher be frozen until a calibration cycle is completed.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1131792 · Report as offensive
nemesis
Avatar

Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131793 - Posted: 25 Jul 2011, 17:10:12 UTC

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.

the recent "doubling" and "pay for doubling"
does nothing but show how the credit system can be corrupted
for no real reason.

if you have to resort to cheap gimmicks to keep a project afloat
what does that really say about the honesty of the project.
ID: 1131793 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24905
Credit: 3,081,182
RAC: 7
Ireland
Message 1131797 - Posted: 25 Jul 2011, 17:16:12 UTC - in response to Message 1131793.  

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.

the recent "doubling" and "pay for doubling"
does nothing but show how the credit system can be corrupted
for no real reason.

if you have to resort to cheap gimmicks to keep a project afloat
what does that really say about the honesty of the project.



That is the best statement I've seen regarding all the credit debates I've read for the past 4 years.
ID: 1131797 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 1131799 - Posted: 25 Jul 2011, 17:25:55 UTC - in response to Message 1131793.  

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.


That sort of worked, until GPUs came along. It also gets very meaningless very quickly if you start comparing a S@H shorty WU (minutes to complete) to a CPD WU (months to complete on the same machine). Does the relative completion time reflect the relative value of the science? Now there's a question to debate.

Despite my somewhat cynical earlier post, I do realize that credit is one way of attracting new crunchers, and one way of encouraging crunchers to upgrade hardware. We have to recognize that, and that is where things get complicated. Personally, I'm in it for the science. And the free toaster.

ID: 1131799 · Report as offensive
nemesis
Avatar

Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131805 - Posted: 25 Jul 2011, 17:37:46 UTC - in response to Message 1131799.  

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.


That sort of worked, until GPUs came along. It also gets very meaningless very quickly if you start comparing a S@H shorty WU (minutes to complete) to a CPD WU (months to complete on the same machine). Does the relative completion time reflect the relative value of the science? Now there's a question to debate.

Despite my somewhat cynical earlier post, I do realize that credit is one way of attracting new crunchers, and one way of encouraging crunchers to upgrade hardware. We have to recognize that, and that is where things get complicated. Personally, I'm in it for the science. And the free toaster.


if you look at boincstats for instance....
you see a breakdown of credits achieved at different projects.
the same could happen with a wu completion + hours processed.

say something like this

seti 100 wu's completed, 1000 hours processor time.
einstein@home 101 wu's completed, 956 hours processor time.

it would give meaning to each projects processing needs.
a processor intensive project would essentially be shown by its
ratio of wu's/processor time...
ID: 1131805 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13835
Credit: 208,696,464
RAC: 304
Australia
Message 1131813 - Posted: 25 Jul 2011, 17:48:23 UTC - in response to Message 1131805.  

it would give meaning to each projects processing needs.
a processor intensive project would essentially be shown by its
ratio of wu's/processor time...

Which is what Credits do.
Grant
Darwin NT
ID: 1131813 · Report as offensive
nemesis
Avatar

Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131816 - Posted: 25 Jul 2011, 17:52:14 UTC - in response to Message 1131813.  

really?

how so?
ID: 1131816 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 1131819 - Posted: 25 Jul 2011, 17:53:42 UTC - in response to Message 1131805.  
Last modified: 25 Jul 2011, 17:57:28 UTC

I'm beginning to see some merit in Phud's idea. The individual user can track hours per WU to see if their tweaks and hardware upgrades have the intended results. Changes in WU size and/or complexity (which will continue to happen) will just require you to "wait a bit" till the numbers settle down again.

Still not sure about cross project comparisons though. Anything done will be subject to "players". If Project X make WUs shorter, will that attract some users? Probably, but nothing is gained for the science. If an improvement in Project X science results in longer WUs, will that motivate some users to switch to another project? Probably yes again, despite an improvement in science.

This is my final offer. Do what you like, I'm here for science. If we scare off some users, [shrug]oh well[/shrug]. I haven't seen a forum yet at any project with people complaining about too many users (except occasionally here at S@H). All I ever see is people demanding more WUs. Whether their motive is more science or more credit is really irrelevant, as long as more science gets done.

ID: 1131819 · Report as offensive
Profile Tazz
Volunteer tester
Avatar

Send message
Joined: 5 Oct 99
Posts: 137
Credit: 34,342,390
RAC: 0
Canada
Message 1131820 - Posted: 25 Jul 2011, 17:55:32 UTC - in response to Message 1131793.  

i think that in its pure form
just stating the hours spent plus wu's completed
were the honest way to go.
<SNIP>


Waaaaay back in '99 when I started crunching it was because I found the xx.xx.SETI usenet newsgroup. In it people were posting their times per wu; Guy1-"I can do one wu in 3hrs and 10 min.". Guy-2"Well I can do one in 2hrs and 58 min.". Guy-1"What have you got (motherboard setting) set to?". And so on.

I had just spent a load of money for a 233 MMX PC and wanted to make sure it was running comparable to other 233s. Running a wu, checking the time, changing some settings, getting another wu ... It was fun. But I digress.

When we put a new GPU through it's paces how to we usually share the results? One normal AR wu in xx minutes. Simply because it takes a while to get an inaccurate RAC that most people only see as 'higher is better'.

I'm all for the hours spent on wu's completed.


SETI@home classic workunits 12,500
SETI@home classic CPU time 54,781 hours
</Tazz>
ID: 1131820 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 1131837 - Posted: 25 Jul 2011, 18:22:51 UTC

The best credit system we've had was the original benchmark * time.

Because bechmarking is an inexact science, claimed credit wasn't really consistent, but requiring three results for a quorum, and throwing out high and low produced a fair and reasonably accurate result.

(Fair means "equitable" -- everyone got a bonus sometimes, but the odds of a bonus, or a low result, were constant)

Unfortunately, that doesn't work for GPUs unless BOINC has benchmarks for CUDA, OpenGL, or whatever.

The big problem we have when we leave benchmark * time is a lack of *comparable* credit from project to project. Science applications vary, and projects have to determine a scaling factor to convert "flops" into credit.

Some projects don't care, and don't put much effort into awarding credit. Others attract credit hounds by awarding more credit than they should.

In an idea world, credits would scale automatically between projects.

Still, it's amazing how "nothings" turn into "money," and people spend actual money to collect "BOINC money."
ID: 1131837 · Report as offensive
Profile James Sotherden
Avatar

Send message
Joined: 16 May 99
Posts: 10436
Credit: 110,373,059
RAC: 54
United States
Message 1131851 - Posted: 25 Jul 2011, 18:52:09 UTC

Interesting read, and some good solid points. Id go for the hours and work units crunched. But really I crunch because I believe in Seti@Home. Yes I have back up projects, But only use them when we are down for days and Im out of work.
If Dr. A. says no more credit will be awarded I will still crunch. I dont care what other projetcs pay in credit. But maybe we can have Dr. A. make a picture of a Seti@Home toaster:)
[/quote]

Old James
ID: 1131851 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30923
Credit: 53,134,872
RAC: 32
United States
Message 1131865 - Posted: 25 Jul 2011, 19:15:34 UTC

I'm not sure there is any fair way to compare across projects. The issue is that some projects use only integer math and others only floating point math. At present only the floating point speed benchmark is used in calculations.

An arbitrary conversion can be done saying that 5 integer adds = 1 floating point add, but is that fair across all processors?

I think people are forgetting the original reason they went from units to credits. In the old system a shorty counted as much as a long. Not fair. But CPU time was also counted. That took the short/long into account, but did not take into account processor speed.

The credit system was supposed to take both into account and come up with a single number. They tried benchmarks to see how fast the processor ran. What they got was a very crude system that is only fair if the science is the benchmark.

As I understand creditnew it is more to prevent user fraud in open source than it is a serious attempt to even out credits. It addresses a optimized app that reports 100x work by comparing the claimed credit numbers with other machines. It can't do much about a project that claims 2X for every unit.

Now perhaps if the credit software consulted manufacturers data sheets on the number of cycles for each instruction and the speed of the clock on the system something close to fair could be implemented. Then each science application would have to be counted as to how many of each type of instruction was run. Not going to happen.

So what has happened is that each project admin has been left on his word that his correction factor is honest. As crunchers say different projects pay different amounts, we know that the correction factors are not set uniformly, can't say if that is intentional or not. I think most people believe it is intentional in some cases. I don't know how the BOINC software can catch that unless it debugs the science application to count instructions (which would make it crawl) Or do science applications need to be registered with a central authority, which would end optimized apps.

This isn't ever going away.

ID: 1131865 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1131875 - Posted: 25 Jul 2011, 19:33:55 UTC

I do miss the old days of seeing how much CPU time I had contributed.
Seeing the stats as:
Results received:		93865
Total CPU time:			98.567 years
Average CPU time per work unit:	9 hr 11 min 55.7 sec 	

This gave me several goals to work towards. One of the big ones I was working on was an average time under 10 hours. I was hoping to end with 100,000 wu's and 100 years of CPU time, but I missed that mark.

With the current credit system(s) I can really only work towards credit number goals. Like "try to make these machines to make 100,000 credits this month". Which doesn't really tell me how much work the machine is doing, just that it gets granted that amount of credit.

Maybe they could implement these values again. Not to replace the current credit system, but to give us something else to track for fun. I imagine not all of this information has been stored since the switch over to BOINC. SO they would have to pick an arbitrary date to implement it. Much liek they do for the change in the credit system(s).
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1131875 · Report as offensive
nemesis
Avatar

Send message
Joined: 12 Oct 99
Posts: 1408
Credit: 35,074,350
RAC: 0
Message 1131879 - Posted: 25 Jul 2011, 19:37:24 UTC

"I think people are forgetting the original reason they went from units to credits. In the old system a shorty counted as much as a long. Not fair. But CPU time was also counted. That took the short/long into account, but did not take into account processor speed."

this is why a simpler method is better...

if you're running a pentium 4 and getting 1 wu every 12 hours and you see
someone else getting 48 wu's in 12 hours you'll look and see someone running
opt's apps and a faster cpu/gpu.
is that discrimination? no...you choose to crunch on what you have at your disposal.

that very scenario could make the p4 user look into opt'd apps..
and maybe upgrade.

i'm not saying its perfect to run wu's completed/hours processed, but it'd be a
lot less corrupted that the system is now.


ID: 1131879 · Report as offensive
Profile SciManStev Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Jun 99
Posts: 6657
Credit: 121,090,076
RAC: 0
United States
Message 1131889 - Posted: 25 Jul 2011, 19:55:48 UTC
Last modified: 25 Jul 2011, 19:57:00 UTC

I agree with Bill Walker in that the science comes first. That is the reason I joined, and is the reason I will stay until I drop dead. How credits are calculated means less to me, as long as we are all using the same system. I do enjoy seeing how my rig compares against others, as it is natural human competition. As far as cross projects go, I see it as "When in Rome, do as the Romans do." What ever system is employed at each project gives one a sense of how ones rig is performing against others doing the same tasks. To me it doesn't matter about total BOINC credits or rankings, as the credit systems are different. They make the most sense in the project they are in.

Recently my rig was in the #3 spot of top computers. This was partially due to the amount of data it crunched, and partially due to what others did not crunch. Back during the Great Outage, I crunched for Einstein, and also brought my rig up to the #3 spot. In Einstein I got further with CPU power and hyperthreading. With Seti I get further with GPU power, and no hyperthreading. For what ever credit system is used, I learned to tune my system to get the most from it based on RAC and credits. In both cases, I did quite a lot of science, and that pleased me no end.

Steve
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website
ID: 1131889 · Report as offensive
1 · 2 · 3 · Next

Message boards : Number crunching : credits and workunits: a fair system needed for Boinc


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.