Get rid of credit system!

Message boards : Number crunching : Get rid of credit system!
Message board moderation

To post messages, you must log in.

AuthorMessage
Killeroy

Send message
Joined: 1 Jun 03
Posts: 3
Credit: 161,500
RAC: 0
United States
Message 41451 - Posted: 30 Oct 2004, 14:08:16 UTC

The Seti "experience" told some things:
-If there is a way to cheat, there will be cheat.
-Cheating (at least some form) hurts the science/project

I am rather new to BOINC/Seti so I was wondering how the credit system works. Here's my take:

-Each client claims a credit. That claim is proportional to the CPU time needed to crunch and the benchmark numbers which can be changed by the user either by blatant editing or clever compiler options for the client in order to increase them.

-A credit is granted if 3 clients submit results and claim credit. The credit system takes the middle value of those 3 claims (note: not the mean value!) and grants that to all 3 clients.

Example:
3 clients (all Windows clients) submit claims
Client 1: AthlonXP 3000+, claimed credit: 64
Client 2: P4 3.2, claimed credit: 37
Client 1: P4 3.0, claimed credit: 32

The middle claim is 37. That's what will get awarded to all 3.

Note the distinctive difference in claims between the P4 and the Athlon?

A person with too much time on his hands could now easily go and
a) increase his benchmark numbers to claim higher credit. That way he will make sure to elevate the granted credit for all of the 3 clients that submit for one workunit.
b) Scan through all of the workunits assigned to him and discard those that show "unfavourable" clients (with usually low claims). That would create damage to the project because assigned work will not get done.

So looking at this my conclusion and recommendation would be to get either rid of the credit system completely or make it watertight. A way to do this would be to not make it dependant on the benchmark results and also to not show the distribution of a given workunit to several clients.

Leap of Faith
ID: 41451 · Report as offensive
Profile Toby
Volunteer tester
Avatar

Send message
Joined: 26 Oct 00
Posts: 1005
Credit: 6,366,949
RAC: 0
United States
Message 41518 - Posted: 30 Oct 2004, 19:06:52 UTC

This topic has already been exhaustively discussed here

Bottom line: this is the way it is and the chances of it changing are somewhere between bad and non-existent.
A member of The Knights Who Say NI!
For rankings, history graphs and more, check out:
My BOINC stats site
ID: 41518 · Report as offensive
Killeroy

Send message
Joined: 1 Jun 03
Posts: 3
Credit: 161,500
RAC: 0
United States
Message 41539 - Posted: 30 Oct 2004, 20:32:39 UTC

Thanks for the link!

I am actually more concerned that the whole system will suffer if people start scrapping WU's that have a low probable granted credit.

This is open for misuse due to two issues:
-The website actually lets you see who the other client machines are that are crunching your WU. By checking out their history you can see easily the typically claimed credit.
The "easy" fix would be to not display that publically on the website.

-The claimed credit varies greatly between different machines for similar work. Here are some approximate examples of typical claimed credits (Windows):
P4 3.2: 37
Athlon XP 3000+: 63
Athlon64 3200: 65
Athlon XP 2500+: 55
AMD K6: 90
Mac G4: 33 ("Darwin")

So chances are that if in the group of 3 clients there are two clients of either P4 or G4, the granted credit will be almost half of what it could be if there were all AMD machines. Note that this has nothing to do if one or the other CPU is better/faster/stronger/whatever. This could only be fixed by introducing a "Seti" client specific benchmark which I agree could be impossible.

The problem is that once people start discarding work units, other "honest" people would be cheated from getting any credit at all. Not to mention the effects on the science.

Leap of Faith.
ID: 41539 · Report as offensive
Profile Toby
Volunteer tester
Avatar

Send message
Joined: 26 Oct 00
Posts: 1005
Credit: 6,366,949
RAC: 0
United States
Message 41559 - Posted: 30 Oct 2004, 21:50:32 UTC

As was pointed out in the above linked thread, the benchmark system is in need of some improvement. The low claimed credit has more to do with operating system, not CPU. The default linux client that they give us is horrible when it comes to benchmarks and it will claim about half the credit of the same hardware running windows. I have recompiled my linux clients with better optimizations and am now claiming credit much closer to what windows boxes are claiming. I'm hoping they will be improving the benchmark code in version 4.5 which is being tested in alpha right now. If the benchmarks were more reliable, then all computers, no matter their hardware, would claim approximately equal credit.
A member of The Knights Who Say NI!
For rankings, history graphs and more, check out:
My BOINC stats site
ID: 41559 · Report as offensive
Profile Sir Ulli
Volunteer tester
Avatar

Send message
Joined: 21 Oct 99
Posts: 2246
Credit: 6,136,250
RAC: 0
Germany
Message 41560 - Posted: 30 Oct 2004, 21:54:53 UTC
Last modified: 30 Oct 2004, 22:14:23 UTC

there is also a seceret in this, for people who builds to fast Benchmarks, they dont can connect to the Servers...

lo one test from me

--- - 2004-08-22 15:43:56 - Benchmark results:
--- - 2004-08-22 15:43:56 - Number of CPUs: 1
--- - 2004-08-22 15:43:56 - 3118 double precision MIPS (Whetstone) per CPU
--- - 2004-08-22 15:43:56 - 25645 integer MIPS (Dhrystone) per CPU

no connect to the Berkeley Servers

Greetings from Germany NRW
Ulli S@h Berkeley's Staff Friends Club m7 ©




ID: 41560 · Report as offensive
Pascal, K G
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 2343
Credit: 150,491
RAC: 0
United States
Message 41573 - Posted: 30 Oct 2004, 23:01:19 UTC - in response to Message 41539.  

> Thanks for the link!
>
> I am actually more concerned that the whole system will suffer if people start
> scrapping WU's that have a low probable granted credit.
>
> This is open for misuse due to two issues:
> -The website actually lets you see who the other client machines are that are
> crunching your WU. By checking out their history you can see easily the
> typically claimed credit.
> The "easy" fix would be to not display that publically on the website.
>
> -The claimed credit varies greatly between different machines for similar
> work. Here are some approximate examples of typical claimed credits
> (Windows):
> P4 3.2: 37
> Athlon XP 3000+: 63
> Athlon64 3200: 65
> Athlon XP 2500+: 55
> AMD K6: 90
> Mac G4: 33 ("Darwin")
>
> So chances are that if in the group of 3 clients there are two clients of
> either P4 or G4, the granted credit will be almost half of what it could be if
> there were all AMD machines. Note that this has nothing to do if one or the
> other CPU is better/faster/stronger/whatever. This could only be fixed by
> introducing a "Seti" client specific benchmark which I agree could be
> impossible.
>
> The problem is that once people start discarding work units, other "honest"
> people would be cheated from getting any credit at all. Not to mention the
> effects on the science.
>
> Leap of Faith.
>


AS I understand it the WUWUs are resent up to 8 times, so I think things will be OK.........How would one know a low from a high wuwu before crunching?
Semper Eadem
So long Paul, it has been a hell of a ride.

Park your ego's, fire up the computers, Science YES, Credits No.
ID: 41573 · Report as offensive
Killeroy

Send message
Joined: 1 Jun 03
Posts: 3
Credit: 161,500
RAC: 0
United States
Message 41596 - Posted: 31 Oct 2004, 1:09:02 UTC

Well, you just look at who is also crunching your WU and then look at the past claimed credits for those other machines (the WU's that those machines already have crunched). If they are comparatively low, chances are you will be granted a low credit compared to what you claim no matter how "difficult" that particular WU is.

Leap of Faith.
ID: 41596 · Report as offensive

Message boards : Number crunching : Get rid of credit system!


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.