As a longtime seti@home user, and a relatively new BOINC user, I am disappointed by the BOINC credit system. It is simply too complex. I have sat around and had a few beers with buddies while they ribbed me about how few units I've processed because we can all relate to units! However, it is very difficult to relate to the abstract, almost arbitrary, nature of cobblestones. What's telling to me is when I try to introduce non-techies who are happily crunching away with SETI to BOINC -- their first reaction is disappointment and confusion about the credit system! Their eyes glaze over when talk of quorums and reference computers begins.

Some of you will say "its not about the credit, its about the science". Yes, it is about the science, so lets not forget that there are a lot of people who are contributing that care about both the science and the credits, and even more that care only about the credits -- that doesn't make their contribution to the science any less valuable as long as they don't cheat.

So to fix the system, I would make sure it was simple, simple, simple! On the assumption that most people are not cheaters, go back to giving "units" of work for units processed. Anyone can understand this. Forget about the whole concept of pending credit -- increment unit counts as soon as units are received.

What about cheaters affecting the quality of the science? The servers can still maintain the concept of quorum and validating results, but this would be invisible to users. Also, the server-side process that is handling quorum/validation can also check for hosts that have an unusually high percentage of bad returns (say, anything over 1-2 percent), and a) send an email to the user to warn them that either they are cheating or that something may be wrong with their computer / network link, and direct them to the message boards or an FAQ for support, and b) in extreme cases, reduce the number of units processed. Yes, in this system, some more units may be assigned than are due (e.g. if someone's computer returns the odd invalid result), but so what? Its about the science right?

What about researchers and other people who are interested in the amount of distributed work done by BOINC, or other analysis that requires more accurate measures of processing? Keep the current cobblestones measure, but make it user transparent. Then, the measure could even be changed/improved/tweaked over time to meet the needs of researchers without raising an uproar.

What about the differences in "units" between the BOINC projects? Keep the units processed separate! For users and interested parties who want aggregates, it should be easy to accurately weight the amount of work done for units between different projects once each project has a few hundred thousand results on which to do the statistical analysis. This measure would be the only "complex" one that is user-visible, though even this is far easier to understand for the vast unwashed masses than the current system.

Other than the obvious lack of time and manpower, what are the arguments against such an approach?

---

maybe helpfull pages<p>
http://homepage.mac.com/pauldbuck/index.html
<p>
http://users.iafrica.com/c/ch/chrissu/boinc-README.html
<p>

---

BUMP ruleZ

---

I am totally with you on this one. As a non-scientist, but interested party, I can find no rational connection between what my computers do and how much credit I have been granted. The formula is so complex that all meaning has been lost in the mishmash of calculations that go into it. I am not a cheater, but cannot in any reasonable way tell if I am being cheated. I would like to be able to audit my credits, but as a lay person without the technical knowledge to even understand how credits are granted, I am in a position where BOINC adminstration has in effect said, "We are scientists, trust us." My response is BS. I don't even understand what use the credits are to anyone. I do understand the general concepts of credit being granted on how my computers perform workunits in relation to system benchmarks. SO WHAT! In 90% of the cases, the credit granted to me is based on what someone else's computer has done. I'm not far from discontinuing my participation in SETI, and I have been a user for over four years, just because of this issue. I have three issues:

1. I have BOINC workunits going back to May and June of this year which are still showing PENDING status and the validator queue is no showing zero.

2. In many cases, I have been granted zero credit, when there are three results returned, and the other two results received credit.

3. Also there are many cases where the software has made an error, and I was granted no credit. You know, all I did was load BOINC's software on my system. Why should I be penalized, when their software made a mistake? I had nothing to do with it, and I have no way of fixing it.

So here's my conclusions:

1. Of what use is the credit system to anyone? What does it really show? If my computers' performances are not really reflected in the total credit granted to me, then why have it at all? Does my credit granted show me anything of use TO ANYONE, and if not, why have it?

2. Basically all I'm interested in, is how many workunits have I processed and how do I compare with others. I have two computers running workunits so I'll never be top of the heap, and I have no desire to be there, but I do want some relevant and understandable way of tracking what I have done. If the scientists want to keep track of the all the other junk that the credit system generates, that would be fine with me, but it has no meaning to me...in fact being rated on the current system angers me more than anything else.

3. The appeal of setiathome, I believe, is that us lay people could participate in science without having to be scientists. If credit is to granted at all, it shouldn't be so complex so that all meaning is lost in the process.

---

Hi!

Regarding credits: idea is fine, but implemetation...

Benchmarking produces random results, so same WU on different boxes
produce very different result. What is 21 on mine, is 65 on somebodies elses PC.
(Linux on all my crunchers).
The only time I've been credited by my number was when another one run Solaris,
and poor Windows guy lost 20 credits (I've been the middle one).

It is temptation to recompile and set it to report double values, but this
is not a sollution.

On the end, statistcs work. It is number of WUs multiplied by average credits
per WU. Easy to calculate for every machine returning 100 WUs.
Same accuracy as old seti.

BR Iztok

> Regarding credits: idea is fine, but implemetation...

Agreed

> It is temptation to recompile and set it to report double values, but this
> is not a sollution.

Actually this IS a solution. It is well known that the linux core client is not very optimized. This causes the benchmarks to return much lower numbers than the windows client on the same hardware. The solution can be found here. This only affects the core client so yo won't be messing with the science code at all so you don't have to worry about screwing that up.

Raman: So you would like to go back to using 'work units'? Even within seti, that is kind of worthless as some units take longer to process than others. Also for cross project stats sites such as mine, you suggest making rough equivalences from other projects work units to seti? That is even more arbitrary than the cobblestone. Especially when you have projects like LHC which sometimes give out work units that finish in mere seconds, others that take 30 minutes and still others that take 8+ hours on the same machine.

I wouldn't be opposed to listing CPU time in the stats but it certainly shouldn't be used as the primary count of work completed otherwise my P3-500 MHz would get the same amount of credit as the newest AMD 64 which obviously isn't fair.

No, in a cross project environment you MUST report credits in a portable format (not bound to a specific project) that takes total number crunching power into account. This is exactly waht the cobblestone is designed for. As I said above, there are some problems with its implementation but 1) such is life and 2) they are still working on refining it.

I think your problem with introducing new people to the system is that you are trying to explain too much. And certainly don't use technical terms such as "quorum". Just say "the work unit has to be verified before you are granted credit". With seti@home's validators being caught up, there should be a lot less confusion on the issue as credits will start rolling in much faster for new users. If they really want more info, point them to Paul's website. (link posted above I believe)

---

A member of The Knights Who Say NI!
For rankings, history graphs and more, check out:
My BOINC stats site

>

---

I believe a misunderstanding has occurred in what a cobblestone actually is. It is just as in SETI classic the amount of work that it takes to process a unit. Every single work unit should have a single value no matter who processes and no matter what CPU or OS is used to process it. The problem is the code and the way to measure the actual work done. It is more of an estimate. The cobblestones are just representations of how much work (just like in SETI classic) you have done. The 4 minute work units I do get a very small amount of credit. The long full run work units get plenty of credit and yes, it is true I have processed for hours on end and received no credit for my work. However just because I did the work did I really accomplish anything. No I had a bad work unit and contributed nothing toward the science so really I did not deserve the work.

Get over the part about preventing cheating. I think one person said that pre-beta and no one has forgotten it. Yes it does prevent cheating but it is not the only thing that verification does. It makes sure the science is accurate. I really have not seen any major hack on BOINC as of yet and I have been around since BETA first opened up. There are the people that download their max amount of work and done do any processing but all they do is slow down you getting your credit. The system should reach equilibrium. My two computers that have been on SETI BOINC for a long time now have been consistent in their credit. It has just now reached a consistent number because of the hardware problems and the 3 project divide I have, it just takes more time.

The problem of credits is very complex. It is in place a fair system, although has its errors in the granting of credit in some cases, that accurately represents how much work is done.

It in essence measures how much CPU time in conjunction with the benchmark of your computer and gives you a score for each work unit and then grants the median of that work unit for the other people that processed it. It does everything but the number just does not turn out to be a nice even number that goes up right after you upload. It will however consistently go up if things say the way they are now with the validator.



The graph is going up, what is there not to see?


Just look at my signature. It has my RAC, total credit, and rank. You can even click on it to get more information about my computing. I don’t see how this is so hard to see my progress.

---

<a href="http://www.boincstats.com/stats/boinc_user_graph.php?id=877f93559fda9f7c5a65f974a8763090"><img src="http://www.boincstats.com/stats/banner.php?cpid=877f93559fda9f7c5a65f974a8763090"></a>

> > It is temptation to recompile and set it to report double values, but
> this
> > is not a sollution.
>
> Actually this IS a solution. It is well known that the linux core client is
> not very optimized. This causes the benchmarks to return much lower numbers
> than the windows client on the same hardware. The solution can be found <a> href="http://forums.pcper.com/showthread.php?t=349185">here[/url]. This only
> affects the core client so yo won't be messing with the science code at all so
> you don't have to worry about screwing that up.

This is the kind of thing that annoys me about the current system. It just shows how complex this approach really is. Users should not have to care about this type of stuff!

> Raman: So you would like to go back to using 'work units'? Even within seti,
> that is kind of worthless as some units take longer to process than others.
> Also for cross project stats sites such as mine, you suggest making rough
> equivalences from other projects work units to seti? That is even more
> arbitrary than the cobblestone. Especially when you have projects like LHC
> which sometimes give out work units that finish in mere seconds, others that
> take 30 minutes and still others that take 8+ hours on the same machine.

Yes, I would like to go back to using 'work units'... at least then I can relate the credit given to some simple real measure, not an abstract one. Agree that different units take different times, but statistically that should all average out in the end. Especially when calculating equivalences between projects, after a few hundred thousand units have been processed the "rough equivalence" will rather be statistically quite accurate, regardless of large differences between individual units (remember that this statistical analysis would still use the Cobblestone or something like it to compare the actual work done). As many people have said, no system is perfect, including work units, but I believe it beats the pants off of the current system, especially when combined with the Cobblestone for the salient statistical analysis.

> I wouldn't be opposed to listing CPU time in the stats but it certainly
> shouldn't be used as the primary count of work completed otherwise my P3-500
> MHz would get the same amount of credit as the newest AMD 64 which obviously
> isn't fair.

Agreed, this measure should be informational only, just as in classic SETI.

> No, in a cross project environment you MUST report credits in a portable
> format (not bound to a specific project) that takes total number crunching
> power into account. This is exactly waht the cobblestone is designed for. As
> I said above, there are some problems with its implementation but 1) such is
> life and 2) they are still working on refining it.

I think I see what you're saying here -- if the credits were not cross-project, many people might process units for the projects where the units were the least work, rather than the ones for which they most wanted to contribute to the science. This is a definite disadvantage of using simple work units. However, my take on this is "so be it". The people that want to just get units get what they want (it is their computing power after all, and their choice about how they want to assign it), and they happen to contribute to some science as well, so wonderful. The advantages of the simple system I feel still outweigh the disadvantages.

> I think your problem with introducing new people to the system is that you are
> trying to explain too much. And certainly don't use technical terms such as
> "quorum". Just say "the work unit has to be verified before you are granted
> credit". With seti@home's validators being caught up, there should be a lot
> less confusion on the issue as credits will start rolling in much faster for
> new users. If they really want more info, point them to Paul's website. (link
> posted above I believe)

I could probably explain it in a less technical way, but it cannot be denied that the complexity still must underly the explanation -- most of the people I talk to agree with hamcons -- all they really care about is how many units they've processed. When they ask why they can't have that simple measure, it requires a bit of detail to explain the advantages of the cobblestone. I've looked at Paul's site, and no offense intended, but it seems a bit heavy on the "its really about the science, so you shouldn't care about the credit anyway" -- this is tantamount to telling people that they shouldn't, and shouldn't want to, get anything out of contributing their computing power except warm fuzzies for being so altruistic.

What it comes down to I think is this: most of the people I talk to just don't have as much fun with cobblestones as work units, and that will keep a lot of them away from BOINC.

---

> I believe a misunderstanding has occurred in what a cobblestone actually is.
> [snip]

No, I understand what a cobblestone is. Unfortunately, the reality is that most people do not. Exactly why we need a simpler method.

> Get over the part about preventing cheating. I think one person said that
> pre-beta and no one has forgotten it. Yes it does prevent cheating but it is
> not the only thing that verification does. It makes sure the science is
> accurate.
> [snip]

I'm not saying take away the verification. My initial post specifically said to keep the verification explicitly to make sure the science was accurate. My initial post also said that I agreed most people were not cheaters -- thats why the system should not be geared to them, which it partly seems to be now -- why else would the BOINC system withhold credit until it was verified? So what if occasionally somebody gets credit for an invalid result?

> The problem of credits is very complex. It is in place a fair system,
> although has its errors in the granting of credit in some cases, that
> accurately represents how much work is done.

Exactly! Its too complex! It needs to be something simple like work units. Everyone always says its about the science, not about the credits, so why does everyone put so much emphasis on accurately and fairly representing the work done using the credit system? For most people I've talked to, credits are just for fun, as they should be!

> It in essence measures how much CPU time in conjunction with the benchmark of
> your computer and gives you a score for each work unit and then grants the
> median of that work unit for the other people that processed it. It does
> everything but the number just does not turn out to be a nice even number that
> goes up right after you upload. It will however consistently go up if things
> say the way they are now with the validator.

OK, so your graph goes up. Great. You're a techie -- someone who understands cobblestones and why they make sense from a technical perspective. Show this to a non-techie and ask them what it means that you went from 13500 credit to 27000 credit over about a month and a half. Nothing because they have no idea what 13500 credit means! But if you showed them a graph where they had 45 units and then a month and a half later they had 65 units -- now that means something!

> Just look at my signature. It has my RAC, total credit, and rank. You can
> even click on it to get more information about my computing. I don’t see how
> this is so hard to see my progress.

Sure, you're making progress, but on an abstract measure that to most people will seem arbitrary. Yeah, it might be fairer (though a lot of people seem to think not), but its just not as much fun!

Cheers,
Raman

---

> OK, so your graph goes up. Great. You're a techie -- someone who understands
> cobblestones and why they make sense from a technical perspective. Show this
> to a non-techie and ask them what it means that you went from 13500 credit to
> 27000 credit over about a month and a half. Nothing! But if you showed them
> a graph where they had 45 units and then a month and a half later they had 65
> units -- now that means something!

This is what I believe our fundamental difference is. The number of credits is very big in BOINC. I have been crunching for a while now and have accumulated a considerable amount of credit, although next to nothing compared to other people, and it is hard to memorize how much work I have done. I believe that the large numbers should remain but a decent compromise would be to have a ladder system where it simplifies down to basic whole numbers for so many cobblestones. Say for instance that 1000 or any arbitrary number equaled a "Process Unit" or whatever nice and fancy name you want to call it. Then I would have under my made up system 7 process units. It is a bit easier to remember small numbers and I think a simplified equivalent of the current system could be a method to summarize the current process.

There is no changing the current system. It gives a good guess at the work done and is fairly true for computers both fast and slow. Just like the old system slow computers get less work than fast ones. The credit just adds up to be a bit bigger numbers but I would not call them "abstract" just large. What they represent is the work done. That is all.

I believe this would be a case where the solution would be to just make it more complex by having a parallel credit system or a "Credit for dummies" way and then the "Credit for Techies and anyone else who has a week to learn how credit works under BOINC" :)


On a separate note I notice people doing two things:
1) Monitoring BOINC too closely and watching and waiting for their credits like it was Christmas.
2) Not monitoring it close enough to notice that a large amount of their work units have gone past the deadline.

I know it is not every one but there have been reports of plenty of people doing what I have just mentioned.


By the way by “The problem of credits is very complex.” I meant that in general there is no system that has been made that is really very simple and is truly accurate to the amount of work being done. If you did assign just the number 1 to ever wu then CPU time is not taken into account. If you just go by CPU time then the amount of work is not taken into account. The current system blends both of them into a new, more accurate system.

---

<a href="http://www.boincstats.com/stats/boinc_user_graph.php?id=877f93559fda9f7c5a65f974a8763090"><img src="http://www.boincstats.com/stats/banner.php?cpid=877f93559fda9f7c5a65f974a8763090"></a>

> I believe this would be a case where the solution would be to just make it
> more complex by having a parallel credit system or a "Credit for dummies" way
> and then the "Credit for Techies and anyone else who has a week to learn how
> credit works under BOINC" :)

Yup, makes sense to me. That way people who want to compare things by the "accurate" and "fair" measure of cobblestones can do so, and other people who just want to see something simple like work units (however unfair) can do that too.

> By the way by “The problem of credits is very complex.” I meant that in
> general there is no system that has been made that is really very simple and
> is truly accurate to the amount of work being done. If you did assign just
> the number 1 to ever wu then CPU time is not taken into account. If you just
> go by CPU time then the amount of work is not taken into account. The current
> system blends both of them into a new, more accurate system.

I'm not arguing that the current system is not more fair or accurate. I agree that it probably is. However, I feel that to get the mass appeal of classic SETI, which in the end is what caused it to be so successful, the BOINC credit system will need to be simplified (at least in parallel, as you suggested above).

Cheers,
Raman

---

I would suggest you write an email to one of the developers suggesting of a parallel system of credit to be considered. It would be an easy way to compare apples and apples. Or whatever they are going to be called. ;)

What ever it would be it would need to be a simplified version of the cobblestones. Cobblestones were one day going to include your storage used, bandwidth, and processing power, not to add to any of the current complexity.

If SETI and other projects want to get a more "user friendly" face easy credit and simple few step setup are essential. They have yet to reach this point. They are still in unofficial BETA and are geared toward people who are more knowledgeable.

It would be nice to see people be able to be introduced to, download, install, run, and see their credits only a few days later.

---

<a href="http://www.boincstats.com/stats/boinc_user_graph.php?id=877f93559fda9f7c5a65f974a8763090"><img src="http://www.boincstats.com/stats/banner.php?cpid=877f93559fda9f7c5a65f974a8763090"></a>

Credits are complex by necessity. You have multiple projects doing all kinds of oddball things, but all using the same base platform. You can't do that "1 WU=1 credit" thing because of projects like CPDN that have credits that take weeks to crunch.

I wouldn't worry about fixing credits at this point. There are other more important issues to tackle.

---

Hmm... yeah, I guess there is no denying the fact that cobblestones are more complex - although I'm not sure how much I buy the whole 'more abstract' line. I mean... how many non-techies out there know what in heaven's name a "work unit" is until you expain to them about the radio telescope recording signals and breaking them up into crunchable packets? Don't their eyes glaze over at that point as well? I see no difference in going from 10,000 to 15,000 cobblestones in a week and going from 100 to 150 work units in a week except for a couple of zeros. I definitely think they should have dropped a couple of powers. Maybe when the numbers start getting too large, I will divide the numbers on my stats site by 1,000 and call them "KiloCobbles" - has a nice ring to it, don't you think? :)

I don't know about others but who cares about the actual numbers... Just point them to graphs like these and say "more is better - stay ahead of the people around you!" :) FYI that graph is for teams only but I am working on adding users as well :)

So I guess some of your points are valid but the fact of the matter is that it is HIGHLY unlikely to change so we might as well make the best of it. :)

---

A member of The Knights Who Say NI!
For rankings, history graphs and more, check out:
My BOINC stats site

KiloCobbles... That is great.

Someone start a petition to have KiloCobbles be used once numbers get so large;) just as numbers are reduced in measurements in size for storage and bandwidit.

Maybe even MegaCobbles GigaCobbles for the project totals.

Just be sure to use a power of 1000 and not 1024. :) Hang on.. Then you would need to explain why you used that number to divide and not the other.

So I would have 7 KiloCobbles for SETI but The overall statistics for SETI would be 231 MegaCobbles.

You could even use KC, MC, and GC for short.

I like that. :)

---

<a href="http://www.boincstats.com/stats/boinc_user_graph.php?id=877f93559fda9f7c5a65f974a8763090"><img src="http://www.boincstats.com/stats/banner.php?cpid=877f93559fda9f7c5a65f974a8763090"></a>

> As a longtime seti@home user, and a relatively new BOINC user, I am
> disappointed by the BOINC credit system. It is simply too complex. I have sat
> around and had a few beers with buddies while they ribbed me about how few
> units I've processed because we can all relate to units! However, it is very
> difficult to relate to the abstract, almost arbitrary, nature of cobblestones.
> What's telling to me is when I try to introduce non-techies who are happily
> crunching away with SETI to BOINC -- their first reaction is disappointment
> and confusion about the credit system! Their eyes glaze over when talk of
> quorums and reference computers begins.
>
> Some of you will say "its not about the credit, its about the science". Yes,
> it is about the science, so lets not forget that there are a lot of people who
> are contributing that care about both the science and the credits, and even
> more that care only about the credits -- that doesn't make their contribution
> to the science any less valuable as long as they don't cheat.
>
> So to fix the system, I would make sure it was simple, simple, simple! On the
> assumption that most people are not cheaters, go back to giving "units" of
> work for units processed. Anyone can understand this. Forget about the whole
> concept of pending credit -- increment unit counts as soon as units are
> received.
>
> What about cheaters affecting the quality of the science? The servers can
> still maintain the concept of quorum and validating results, but this would be
> invisible to users. Also, the server-side process that is handling
> quorum/validation can also check for hosts that have an unusually high
> percentage of bad returns (say, anything over 1-2 percent), and a) send an
> email to the user to warn them that either they are cheating or that something
> may be wrong with their computer / network link, and direct them to the
> message boards or an FAQ for support, and b) in extreme cases, reduce the
> number of units processed. Yes, in this system, some more units may be
> assigned than are due (e.g. if someone's computer returns the odd invalid
> result), but so what? Its about the science right?
>
> What about researchers and other people who are interested in the amount of
> distributed work done by BOINC, or other analysis that requires more accurate
> measures of processing? Keep the current cobblestones measure, but make it
> user transparent. Then, the measure could even be changed/improved/tweaked
> over time to meet the needs of researchers without raising an uproar.
>
> What about the differences in "units" between the BOINC projects? Keep the
> units processed separate! For users and interested parties who want
> aggregates, it should be easy to accurately weight the amount of work done for
> units between different projects once each project has a few hundred thousand
> results on which to do the statistical analysis. This measure would be the
> only "complex" one that is user-visible, though even this is far easier to
> understand for the vast unwashed masses than the current system.
>
> Other than the obvious lack of time and manpower, what are the arguments
> against such an approach?
>
>
>This is an Idea that makes sense. But you will see alot of the I do it for the science credits dont mean anything Yeah . I like this idea.

---

If people want their WU counts back. If I go to my computer summary in my own account data, there's a number there counting my "results" (right after the download rate. This is a pretty good indication of the number of WUs, right? Actually it's more like how many you've downloaded, so invalid WUs and WUs that got lost due to a reset or so also get counted, but if I read remarks like "So what if occasionally somebody gets credit for an invalid result?" that may not be so much of a problem. Just as long as someone doesn't constantly reset his project because that would get him/her another cache full of instant "results credit". So basically we already have a parallel system in place.

---

I do find the system a little more complicated, but have to say I still prefer it over the old system. While the old system of counting units made it easy to keep track of, I was disappointerd when I later learned that I might not actually have been doing anything.

The new system gives a better sense of accomplishment. I'm not redoing a work unit for the tenth time for no reason, and I only get credit for a work unit if my computers actually do something useful. Under the old system I got credit for my work even if it was wasted time or an unusable result.

I think as the development team iron out the difficulties and everyone gets more comfortable with the new terms, it will become less of an issue.

---

The biggest objection I have to the count of the work units is this ...

For cp.net I am only comming up to completing my second work unit ... :)

---