Unfair granting credits

Message boards : Number crunching : Unfair granting credits
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Longus

Send message
Joined: 28 Jan 04
Posts: 9
Credit: 28,483
RAC: 0
Poland
Message 38798 - Posted: 21 Oct 2004, 13:37:07 UTC
Last modified: 21 Oct 2004, 13:45:43 UTC

At the very first: I do not care about the amount of credits I get !

But if it is said on official web site that "BOINC assigns a variable amount of credit per completed work unit, based on your CPU speed and the CPU time used." - let it be true !

Example 1:
WU is sent to three computers of the same type, let's say Pentium 4 3GHz. Each computer is claiming similiar credit, about 42. Similiar value is granted to all of them, right ? So it's OK.

Example 2:
WU is sent to Pentium 4 3GHz, Celeron 2.4GHz and Athlon XP 1900+. First one is claiming 41.61, second 86.92 and third 81.76. All are granting 81.67 - Pentium almost 2 times more than claimed !!! Unbelivable ? Please see http://setiweb.ssl.berkeley.edu/workunit.php?wuid=3132116

If you are lucky, you can get 2 times more credits than your friend with the same CPU speed and time !!!

What do you think ?
<p><img src="http://boinc.mundayweb.com/seti2/stats.php?userID=1721&amp;trans=off">   <img src="http://seti.mundayweb.com/stats.php?userID=512&amp;trans=off">
ID: 38798 · Report as offensive
Profile slavko.sk
Avatar

Send message
Joined: 27 Jun 00
Posts: 346
Credit: 417,028
RAC: 0
Slovakia
Message 38802 - Posted: 21 Oct 2004, 13:42:55 UTC
Last modified: 21 Oct 2004, 13:44:06 UTC

Yes, it is the known feature, many time and still discuessed. I started some discussion a long time ago, but then I got only replies that it's OK. So I don't care about this credit granting at all. And I'm still thinking that slower computer will produce more credit then faster.
Ypu may see that in Hosts charts, 4 processors computers have less credit then 2 processors computer and even then 1processor coputer, sometime, not always.
It is strange, at least.
ALL GLORY TO THE HYPNOTOAD!
Potrebujete pomoc?
My Stats
ID: 38802 · Report as offensive
Longus

Send message
Joined: 28 Jan 04
Posts: 9
Credit: 28,483
RAC: 0
Poland
Message 38804 - Posted: 21 Oct 2004, 13:48:48 UTC - in response to Message 38802.  

I am new on the forum and haven't seen your thread, sorry for raising a problem again :-)

If it is OK - I don't care either.
ID: 38804 · Report as offensive
Petit Soleil
Avatar

Send message
Joined: 17 Feb 03
Posts: 1497
Credit: 70,934
RAC: 0
Canada
Message 38807 - Posted: 21 Oct 2004, 13:55:28 UTC
Last modified: 21 Oct 2004, 14:00:01 UTC

I never really understood why they made such a complex, confusing
and apparently unfair credits system. I think it's OK for WU to
be validated three times in regards to the science aspect, but
not for the credit.

It would be better and more simple to get credit for the work that
is being done, no matter if the results failed for various reasons
wich are not under the control of the users.

My PC has a benchmark of B and it took T times to process so I receive
a certain amount of credit, no matter what the two other results were.

That's in my opinion of how it should be.

ID: 38807 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 38808 - Posted: 21 Oct 2004, 14:02:09 UTC - in response to Message 38807.  

They did it in part to try to reduce or eliminate a possible source of cheating.

Jim


> I never really understood why they made such a complex, confusing
> and apparently unfair credits system. I think it's OK for WU to
> be validated three times in regards to the science aspect, but
> not for the credit.

ID: 38808 · Report as offensive
Profile JigPu
Avatar

Send message
Joined: 16 Feb 00
Posts: 99
Credit: 2,513,738
RAC: 0
Message 38812 - Posted: 21 Oct 2004, 14:35:04 UTC

I don't want to sound like I'm crunching SETI only for the credit, but this credit system is broken and needs fixing.

We were promised a credit system that would grant the same credit (within a range) for a WU to all computers crunching it. It was designed to put to rest concerns about fast computers being forced to claim less credit should a slow box also complete the unit. However, currently the system is causing slower boxes to ask for significantly more credit than faster boxes. This is just as broken as a system where the faster boxes as for significantly more.

Certianly this issue of tweaking how credit is calculated is not a priority one issue by any means, but there is no reason this bug should not be fixed when the SETI folks finaly have a nice long vacation from the myriad of hardware/software failures that have been plaguing them.
ID: 38812 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 38820 - Posted: 21 Oct 2004, 14:57:02 UTC - in response to Message 38812.  

> I don't want to sound like I'm crunching SETI only for the credit, but this
> credit system is broken and needs fixing.

I think most of us agree on that. :)

> Certianly this issue of tweaking how credit is calculated is not a priority
> one issue by any means, but there is no reason this bug should not be fixed
> when the SETI folks finaly have a nice long vacation from the myriad of
> hardware/software failures that have been plaguing them.

When the cross-platform GUI is out we may very well see some changes and improvements in the client side (which is where most of the error sources are, but that is only my opinion).
ID: 38820 · Report as offensive
Cryz

Send message
Joined: 22 Feb 02
Posts: 46
Credit: 9,737
RAC: 0
Belgium
Message 38826 - Posted: 21 Oct 2004, 15:20:11 UTC

I haven’t read all past discussions on the subject but the difference in claimed credit, in my opinion, is cause by the benchmark results as the claimed credit is based on the amount of calculations don by you system. That amount is based on how long it took and the benchmark results (how many calculations a second). In other words, if the benchmark function was perfect, every system would claim the same credits.
<a href="http://www.boinc.dk/index.php?page=user_statistics&project=sah&userid=1288413"><img border=0 width="280" height="70" src="http://1288413.sah.sig.boinc.dk?79"></a>
ID: 38826 · Report as offensive
Profile JavaPersona
Volunteer tester

Send message
Joined: 4 Jun 99
Posts: 112
Credit: 471,529
RAC: 0
United States
Message 38853 - Posted: 21 Oct 2004, 17:27:34 UTC - in response to Message 38826.  

> I haven’t read all past discussions on the subject but the difference in
> claimed credit, in my opinion, is cause by the benchmark results as the
> claimed credit is based on the amount of calculations don by you system. That
> amount is based on how long it took and the benchmark results (how many
> calculations a second). In other words, if the benchmark function was perfect,
> every system would claim the same credits.
>
I tend to think this is correct. Although my machine's bechmarks don't seem to fit their relative computing power, over the long haul, the faster machines are granted more credit than the slower ones, in line with their ability to complete more work units for a given time.
ID: 38853 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 38866 - Posted: 21 Oct 2004, 18:59:49 UTC

Absolutely correct Cryz,

I got a little annoyed by all the delay so I changed the source code.

My systems now report same credit for same average completion time (ie: average of all CPU times for given machine, that is average WU time for it, credit claimed is identical to 'average' for other PCs)

ID: 38866 · Report as offensive
Petit Soleil
Avatar

Send message
Joined: 17 Feb 03
Posts: 1497
Credit: 70,934
RAC: 0
Canada
Message 38867 - Posted: 21 Oct 2004, 19:06:20 UTC - in response to Message 38853.  
Last modified: 21 Oct 2004, 19:07:23 UTC

> I tend to think this is correct. Although my machine's bechmarks don't seem
> to fit their relative computing power, over the long haul, the faster machines
> are granted more credit than the slower ones, in line with their ability to
> complete more work units for a given time.

It is absolutely true but it has nothing to do with their ability to
complete more work units. Well yes but not as number of WU but as
higher Benchmark on a given crunching time.

You have two machine, one has benchmark of 1000 and the other 2000 and
they are both crunching for 24/7. At the end of the week the 2000 Box
will claim twice the credit claimed by the 1000 box, no matter if it is
2,5,or 178 WU

ID: 38867 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 38873 - Posted: 21 Oct 2004, 19:48:15 UTC

This then is the question...

Is the "credit": Credit for a particular machine, capable of performing xx benchmark score for yy percent of the day?

Or is credit: For a particular machine, capable of doing [zz] identical WUs for the science of seti?

I would vote for the 2nd version. This would also apply to other projects.
Example: [zz] identical WUs of project [abc].

Identical WUs = WUs with identical # of Integer and FP OPS to one another.

ID: 38873 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 38878 - Posted: 21 Oct 2004, 19:56:37 UTC - in response to Message 38873.  

> This then is the question...
>
> Is the "credit": Credit for a particular machine, capable of performing xx
> benchmark score for yy percent of the day?
>
> Or is credit: For a particular machine, capable of doing [zz] identical WUs
> for the science of seti?
>
> I would vote for the 2nd version. This would also apply to other projects.
> Example: [zz] identical WUs of project [abc].
>
> Identical WUs = WUs with identical # of Integer and FP OPS to one another.
>
The problem then comes back to potential cheating! If I can do zz units and send 1000 of them in then and get crdit for all of them, whether they stand up to peer review or not, because that is what you are saying, then whats the point?!!

ID: 38878 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 38884 - Posted: 21 Oct 2004, 20:10:35 UTC - in response to Message 38807.  

> My PC has a benchmark of B and it took T times to process so I receive
> a certain amount of credit, no matter what the two other results were.
>
> That's in my opinion of how it should be.
>
So I can overclock my machine so it runs at like a 10 ghz speed demon, yes it CAN be done, just not anywhere near economically or reliably, and the results are useless, you are saying I should still get credit for crunching?! AND I should get the credits I am asking for, not the ones that the peer review process suggests I should get?!
On with the overclocking, where is that bottle of liquid nitrogen? You can run a unit in ram and then the whole board in liquid nitrogen and overclock to your hearts delight, the numbers produced are just worthless. BUT the system WILL run!



ID: 38884 · Report as offensive
Petit Soleil
Avatar

Send message
Joined: 17 Feb 03
Posts: 1497
Credit: 70,934
RAC: 0
Canada
Message 38887 - Posted: 21 Oct 2004, 20:25:50 UTC - in response to Message 38884.  

> So I can overclock my machine so it runs at like a 10 ghz speed demon, yes it
> CAN be done, just not anywhere near economically or reliably, and the results
> are useless, you are saying I should still get credit for crunching?! AND I
> should get the credits I am asking for, not the ones that the peer review
> process suggests I should get?!
> On with the overclocking, where is that bottle of liquid nitrogen? You can run
> a unit in ram and then the whole board in liquid nitrogen and overclock to
> your hearts delight, the numbers produced are just worthless. BUT the system
> WILL run!

I get you point and there is a very simple solution to avoid that.
validation of the 3 WU sent before credit is given. Oh no... it can't
work because as you say if your machine is sending crap results, the
two other users would be penallysed because of your nitrogen machine.

There's got to be a way to get out of this vicious circle and having
a more efficient and fair credit system.
ID: 38887 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 38893 - Posted: 21 Oct 2004, 20:42:53 UTC

Helloooo...anybody home??

When I say xx units of WU, of course I mean cross checked xx units of WU vs other machines results.

Yes they require variable ammounts of time, and should be granted appropriately scaled ammounts of credit.

And if things work properly, these other peoples machines should claim same credit as my machines for each WU.

If on ONE machine, average WU takes 3:00 hours, and a particular WU (on that machine) takes 0:10 (10 minutes), then of course credit claimed should be 10 / 180 of a regular WU.

The point being, identical WUS (as are seen in "results for host" page http://setiweb.ssl.berkeley.edu/results.php) should claim identical credit. And if cross verified, be granted identical credit.

ID: 38893 · Report as offensive
Cryz

Send message
Joined: 22 Feb 02
Posts: 46
Credit: 9,737
RAC: 0
Belgium
Message 38899 - Posted: 21 Oct 2004, 21:05:31 UTC

Maybe I’m blind, but I fail to see the unfairness in the current system. In theory you should get credits for every floating point or integer operation of you CPU. Unlike with the classic version, 2 identical systems should about claim the same amount of credit in the same time frame. With classic the one system could get 50 short wu’s and the other 25 longer ones and as a result the first system would have twice the credit of the second one for doing the ‘same’ work. With boinc, no matter what system, for the same amount of work (x floating point operations) you (should) get y credits. And As I said before, the big difference is caused by the benchmarks, because they determine how many floating point operation boinc thinks that were needed to complete the wu.
Also the validation is a big improvement because now you need to produce usable results before you get credits. And in my humble opinion, all this is what the developers mean with a ‘more fair credit system’.

<a href="http://www.boinc.dk/index.php?page=user_statistics&project=sah&userid=1288413"><img border=0 width="280" height="70" src="http://1288413.sah.sig.boinc.dk?79"></a>
ID: 38899 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 38903 - Posted: 21 Oct 2004, 21:24:16 UTC
Last modified: 21 Oct 2004, 21:27:08 UTC

Hi All,
I don't know if I am missing the point or I am mistakenly making light of something really serious. I was under the impression that if you have a fast processor that you would crunch more WU's thus recieving less credit for actual processor time Correct? So that would mean that those people that have the slower processors that take much longer to crunch one Wu get more credit for the WU that they are crunching but they are doing far less than the faster processors Correct? What is th problem here.? The only problem I see is if people are in individual competetion with each other based on performance and not on hardware capabilities. Everything else seems to be a wash. Benchmarks aside. Maybe they should never have been mentioned because all they have done, have enraged those that know nothing about them, angered those that know a little about them, and made mad those that are well versed about them. Please don't get me wrong, I am not trying to upset anyone right now, I want everyone to get along. It would probably be well advised for those that know little and those that know nothing (Like Myself) about benchmarks to listen to what those that know what they are talking about have to say. And correspond accordingly. This is the only way we learn and grow as a project and as a computing community.

Have A Great Day A Better Tomorrow!

It's Great To Be Back!

Regards,

Rocky Cudd
www.boincsynergy.com


ID: 38903 · Report as offensive
Ron Roe
Volunteer tester

Send message
Joined: 28 Feb 02
Posts: 156
Credit: 24,124
RAC: 0
United States
Message 38913 - Posted: 21 Oct 2004, 21:55:44 UTC
Last modified: 22 Oct 2004, 13:38:12 UTC

ID: 38913 · Report as offensive
Ron Roe
Volunteer tester

Send message
Joined: 28 Feb 02
Posts: 156
Credit: 24,124
RAC: 0
United States
Message 38914 - Posted: 21 Oct 2004, 21:58:27 UTC - in response to Message 38893.  
Last modified: 26 Oct 2004, 18:58:59 UTC

ID: 38914 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Unfair granting credits


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.