Credits Proposal

Message boards : Number crunching : Credits Proposal
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Mike Gelvin
Avatar

Send message
Joined: 23 May 00
Posts: 92
Credit: 9,298,464
RAC: 0
United States
Message 125002 - Posted: 18 Jun 2005, 21:09:43 UTC
Last modified: 18 Jun 2005, 21:11:00 UTC

I throw this into the public discussion area… for whatever its worth.

I know there has been a lot of discussion about credits, and benchmarks, and making credits across projects consistent with the amount of work (science) done.

I suspect the solution is in concept simple, but would require some changes to the server side to implement.

Facts:
For a given work unit, the amount of science is equal across all platforms, regardless of speed performed, provided the answer is consistent across all platforms.
Different computers will complete a given work unit in differing speeds. This is dependant on the platform itself, as well as the workload the platform is being subjected to from time to time

Things that wont work
The benchmark approach will not work due to:
1) it is a snapshot and does not represent the true picture of the loading of the platform. 2) no benchmark can truly be representative of the science (especially across all projects).

Things that might work
Sample work units down loaded to the clients and run which are in and of themselves benchmarks. This has the objection that it could be time consuming to run a representative work unit for a specific project, and it is desirable to run this frequent enough to catch system changes (but could never be frequent enough to catch differences in system loading). Running benchmarks is NOT science and actually detracts from the final objective.

A simple approach
The projects themselves know how much science is being done.
For all of the work units distributed by a project, an average amount of time to complete a work unit can be calculated (in reality, you would want a sliding average of say 100,000 units?) Call this number ADVTime. Further, lets say that ADVTime is in seconds.

To equalize projects ADVTime should be combined with a number that would indicate what a portion of time that this project takes to process one work unit. I shall call this PRJTime and is equal to ADVTime/ (Seconds in a day) = ADVTime/86400. This will have the effect of making a SETI work unit more valuable than say a Predictor work unit, because SETI takes more time to evaluate on average.

For a given work unit the amount of credit a platform would receive would be proportional to the time taken in relation to ADVTime.

WUCredit = ADVTime/WUTime
If the work unit took more than the average time, than you get less credit due to the fact of applying less computing power to the problem.

Overall credit for a work unit would then be multiplied by the project adjustment:

Credit = WUCredit * PRJTime


The entire equation looks like:

Credit = (ADVTime/WUTime) * (ADVTime/86400)

Or

Credit = (ADVTime * ADVTime) / (WUTime * 86400)

I perceive the following benefits:
All of this could easily be calculated by the servers when a work unit is returned.
Each platform would receive its share immediately.
Credits would be distributed evenly across all projects.
And credits would never be subject to the luck of the draw on what platforms (and their speeds) a work unit is distributed with.
Credits value will slide up and down depending on the required science for the representative number of work units that ADVTime is taken from.
NO benchmarks are required, there are no wasted CPU cycles out at the clients.
There is no alteration to the client software at all… all changes would be done at the server end.


A small problem with the above might dictate a slightly more complicated approach.

It can be argued that within a project there is wildly differing work unit times, and hence results. This would have the effect that if a work unit were a “short” one, it would be perceived by the servers as being done blazingly fast and hence would get a totally disproportional share of the credits.

I suspect that the actual science returned by these anomalous work units could be perceived by the servers and adjusted for. If the work units (after a result was returned) could be classified as to the amount of science it contained, then the above mentioned algorithm could be maintained separately for each class of work unit. I suspect this classification is already being done at some level by the servers to either count events, or in some other way quantify the value of the result. Hence this would (I’m guessing here) be a simple extension of that process.


ID: 125002 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 125020 - Posted: 18 Jun 2005, 21:57:09 UTC

We could also just do away with the credits and just crunch for the science.
ID: 125020 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 125021 - Posted: 18 Jun 2005, 21:59:07 UTC - in response to Message 125002.  

Hmm... and where do you get WUTime from...


ID: 125021 · Report as offensive
Profile Kajunfisher
Volunteer tester
Avatar

Send message
Joined: 29 Mar 05
Posts: 1407
Credit: 126,476
RAC: 0
United States
Message 125025 - Posted: 18 Jun 2005, 22:04:39 UTC

Would that be =CPU Time?
ID: 125025 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 125084 - Posted: 18 Jun 2005, 23:42:39 UTC - in response to Message 125002.  

I throw this into the public discussion area… for whatever its worth.
A simple approach
The projects themselves know how much science is being done.
For all of the work units distributed by a project, an average amount of time to complete a work unit can be calculated (in reality, you would want a sliding average of say 100,000 units?) Call this number ADVTime. Further, lets say that ADVTime is in seconds.

So if I understand you correctly you want each project to keep 100,000 units in storage at any one time just so they can average the time it took to process them? Doesn't that mean a ton of storage space all by itself, and then require a lot of processing power every day, every hour, every whatever, to then take all those times and average them? That seems like a lot of work just to come up with an average. What about the smaller projects, the ones that are trying to run Boinc on a shoestring? I think this part needs some work.
It could also encourage people to either hold their units or send in their units asap depending on how often the average is done, and how they think it will affect their credit.

ID: 125084 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 125097 - Posted: 19 Jun 2005, 0:09:35 UTC - in response to Message 125025.  

Would that be =CPU Time?


Not if this credit-proposal should be an improvement...


If not overlooked something, if you uses cpu-time from client, this proposal basically means a fast computer crunching a wu in 1 hour will claim 24x as much credit for a wu as a slow computer using 24 hours on the same wu will claim...

ID: 125097 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 125103 - Posted: 19 Jun 2005, 0:14:39 UTC - in response to Message 125097.  

I want to bet that if you put his proposal through a calculator that each outcome would be one. 1 credit then. ;)
ID: 125103 · Report as offensive
j2satx
Volunteer tester

Send message
Joined: 2 Oct 02
Posts: 404
Credit: 196,758
RAC: 0
United States
Message 125105 - Posted: 19 Jun 2005, 0:17:54 UTC


What's wrong with that?

Seems the fairest to me............you crunch one WU, you get one credit. Isn't that how it was in the original SETI?

ID: 125105 · Report as offensive
Profile Dorsai
Avatar

Send message
Joined: 7 Sep 04
Posts: 474
Credit: 4,504,838
RAC: 0
United Kingdom
Message 125107 - Posted: 19 Jun 2005, 0:21:53 UTC - in response to Message 125084.  

(snip)So if I understand you correctly you want each project to keep 100,000 units in storage at any one time just so they can average the time it took to process them? (snip)


Unless I am mistaken, and I probably am....

S@H has about 132,000 users. About 43% are active = about 55,000. I think it's safe to assume that S@H has more than 2 completed results on file for each active user.

So S@H already have well over 100,000 completed units in storage.

(snip).require a lot of processing power every day, every hour, every whatever, to then take all those times and average them?(snip)

Yes, probably beyond current capabilities.

The results are there, but the processing powere is not.


Foamy is "Lord and Master".
(Oh, + some Classic WUs too.)
ID: 125107 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 125115 - Posted: 19 Jun 2005, 0:33:37 UTC - in response to Message 125103.  

I want to bet that if you put his proposal through a calculator that each outcome would be one. 1 credit then. ;)


Hmm... digs out calculator, and let's throw some real numbers at it...

Recent Average cpu-time per wu in SETI@home "classic" is... 5h36m42.3s = 20202.3s

For fast machine, 1h-wu:
credit = 20202.3s * 20202.3s / (3600s * 86400s) = 1.31

For slow machine, 24h-wu:
credit = 20202.3 * 20202.3 / (86400s * 86400s) = 0.0547

0.0547 * 24 = 1.31...
ID: 125115 · Report as offensive
Profile Kajunfisher
Volunteer tester
Avatar

Send message
Joined: 29 Mar 05
Posts: 1407
Credit: 126,476
RAC: 0
United States
Message 125116 - Posted: 19 Jun 2005, 0:35:10 UTC

As soon as my wu has finished crunching I am sending results in, as I have no idea if i am going to be first, second, third, or possibly even last in the quota... as slow as this thing crunches and of course it gets slower the more projects i attach to MY problem gets worse. I don't expect any of the projects to keep 10,000 of my results "on hand" just to figure an average. Hopefully they can get that with the last 2 or 3. If they need any further info i'd be happy to oblige (keystrokes, apps ran, etc.):-)
ID: 125116 · Report as offensive
Profile LarryB56
Volunteer tester
Avatar

Send message
Joined: 22 Apr 02
Posts: 73
Credit: 4,402,310
RAC: 0
United States
Message 125127 - Posted: 19 Jun 2005, 0:55:45 UTC

Honestly now!!! I care about statistics, but I would much rather know when one of my computers runs across something Interesting!!! That is the thing I miss about Classic Seti.

What'd the guy say??? My 2¢...
LarryB56
ID: 125127 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 125148 - Posted: 19 Jun 2005, 1:44:40 UTC - in response to Message 125115.  
Last modified: 19 Jun 2005, 1:47:38 UTC

For fast machine, 1h-wu:
1.31

For slow machine, 24h-wu:
1.31...

Err... I was kinda kidding. ;)

Also in reply to post125105
What's wrong with that?

Seems the fairest to me............you crunch one WU, you get one credit. Isn't that how it was in the original SETI?

It was, but how would you feel if my computer chugs out 12 units per 24 hours, while yours takes 24 days for one unit? That was Classic. Both of the computers got 1 credit per unit. That's 288 against 1, to make things clear. (p.s. the example given was just as: an example)

Seeing how not everyone can either get a farm of computers together or upgrade every 2 months to the next best thing, Boinc is more fair in these things.
ID: 125148 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 125162 - Posted: 19 Jun 2005, 2:09:54 UTC

More to the point, if I get a series of 5 minute noise WUs, should they be given the same credit as a normal S@H WU? As a fully completed CPDN WU? BOINC has several different projects with different processing requirements, and the measure should be cross project.


BOINC WIKI
ID: 125162 · Report as offensive
Profile Kajunfisher
Volunteer tester
Avatar

Send message
Joined: 29 Mar 05
Posts: 1407
Credit: 126,476
RAC: 0
United States
Message 125174 - Posted: 19 Jun 2005, 2:41:04 UTC - in response to Message 125162.  

More to the point, if I get a series of 5 minute noise WUs, should they be given the same credit as a normal S@H WU? As a fully completed CPDN WU? BOINC has several different projects with different processing requirements, and the measure should be cross project.


For S@H you may have been given approx. 20
For E#H approx. 90
CPDN-Complete- 6800-6900

But, since it's noisy wu's, nada, sorry, i don't make the rules, i just follow them ;-)

(If you really have 5 noisy wu's, I feel your pain)
ID: 125174 · Report as offensive
Profile Pooh Bear 27
Volunteer tester
Avatar

Send message
Joined: 14 Jul 03
Posts: 3224
Credit: 4,603,826
RAC: 0
United States
Message 125192 - Posted: 19 Jun 2005, 3:56:48 UTC - in response to Message 125105.  


What's wrong with that?

Seems the fairest to me............you crunch one WU, you get one credit. Isn't that how it was in the original SETI?


As soon as other observatories come on line, their data will be of differing sizes. No longer will 1 WU = other WUs anymore. Other projects are doing the same with different sized WUs.

I think the whole 1 thing has been beaten with a stick too many times. It's not going to happen, let it rest.



My movie https://vimeo.com/manage/videos/502242
ID: 125192 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 125217 - Posted: 19 Jun 2005, 4:34:45 UTC - in response to Message 125127.  

Honestly now!!! I care about statistics, but I would much rather know when one of my computers runs across something Interesting!!! That is the thing I miss about Classic Seti.

What'd the guy say??? My 2¢...


Use one of the SETI logging tools out there ... there are several and will do plots like of old ...

Don't know how current that list is though ...
ID: 125217 · Report as offensive
Mike Gelvin
Avatar

Send message
Joined: 23 May 00
Posts: 92
Credit: 9,298,464
RAC: 0
United States
Message 125239 - Posted: 19 Jun 2005, 5:22:53 UTC - in response to Message 125084.  
Last modified: 19 Jun 2005, 5:26:26 UTC

So if I understand you correctly you want each project to keep 100,000 units in storage at any one time just so they can average the time it took to process them? Doesn't that mean a ton of storage space all by itself, and then require a lot of processing power every day, every hour, every whatever, to then take all those times and average them? That seems like a lot of work just to come up with an average. What about the smaller projects, the ones that are trying to run Boinc on a shoestring? I think this part needs some work.
It could also encourage people to either hold their units or send in their units asap depending on how often the average is done, and how they think it will affect their credit.


Not at all... it would take about 400K in memory (RAM or hard drive space in a DB?) and about 3 integer math operations per returned work unit to maintain. Small change really. The average would constantly be update, hence there would be absolutely NO advantage as to when you sent a work unit back.


ID: 125239 · Report as offensive
Mike Gelvin
Avatar

Send message
Joined: 23 May 00
Posts: 92
Credit: 9,298,464
RAC: 0
United States
Message 125241 - Posted: 19 Jun 2005, 5:24:46 UTC - in response to Message 125162.  
Last modified: 19 Jun 2005, 5:33:26 UTC

More to the point, if I get a series of 5 minute noise WUs, should they be given the same credit as a normal S@H WU? As a fully completed CPDN WU? BOINC has several different projects with different processing requirements, and the measure should be cross project.


No, thats what the across project adjustments would be for. Seti WU would be more valuable than Predictor, and CPDN would be more valuable than Seti. (See PRJTime)



ID: 125241 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 125345 - Posted: 19 Jun 2005, 14:05:56 UTC - in response to Message 125239.  

[quote]So if I understand you correctly you want each project to keep 100,000 units in storage at any one time just so they can average the time it took to process them? Doesn't that mean a ton of storage space all by itself, and then require a lot of processing power every day, every hour, every whatever, to then take all those times and average them? That seems like a lot of work just to come up with an average. What about the smaller projects, the ones that are trying to run Boinc on a shoestring? I think this part needs some work.
It could also encourage people to either hold their units or send in their units asap depending on how often the average is done, and how they think it will affect their credit.

Not at all... it would take about 400K in memory (RAM or hard drive space in a DB?) and about 3 integer math operations per returned work unit to maintain. Small change really. The average would constantly be update, hence there would be absolutely NO advantage as to when you sent a work unit back.

I don't see how that can be, EACH wu for Seti is 353K. You are saying 400K TOTAL! That is only 47 more K than one unit all by itself!!!!! The OP was talking about storing 1000,000 units in ONE db!
I think more calculations are in order.

ID: 125345 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Credits Proposal


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.