Why is team RAC so far off?

Message boards : Number crunching : Why is team RAC so far off?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile FloridaBear
Avatar

Send message
Joined: 28 Mar 02
Posts: 117
Credit: 6,480,773
RAC: 0
United States
Message 103892 - Posted: 26 Apr 2005, 1:41:42 UTC
Last modified: 26 Apr 2005, 1:42:29 UTC

Just using Overclockers.com as an example (my team), Berkeley is giving the team RAC as 19,528; the actual rate calculated by summing the RAC of all of the members is more like 84,080. That's a huge discrepancy. The top 25 members (by RAC) total 42,808 RAC. What's going on here? Apologies if this has been discuessed before; my visits to the board have been pretty infrequent.

ID: 103892 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 103981 - Posted: 26 Apr 2005, 7:38:18 UTC - in response to Message 103892.  

> Just using Overclockers.com as an example (my team), Berkeley is giving the
> team RAC as 19,528; the actual rate calculated by summing the RAC of all of
> the members is more like 84,080. That's a huge discrepancy. The top 25 members
> (by RAC) total 42,808 RAC. What's going on here? Apologies if this has been
> discuessed before; my visits to the board have been pretty infrequent.

You cannot add the team participants RAC and then divide by the total number of participants and then compare to the team RAC.

RAC is very dependent on the time of the grants. If you look in the glossary there is material there that attempts to explain the RAC calculation.

Anyway, each member submits a Result and makes a claim. At that point their RAC is updated and so is the teams. The teams RAC is not derived from the participant's RAC, but on the timeing of all the Result submissions.

ID: 103981 · Report as offensive
shady

Send message
Joined: 2 Feb 03
Posts: 40
Credit: 2,640,527
RAC: 0
United Kingdom
Message 104028 - Posted: 26 Apr 2005, 12:19:48 UTC

The Team RAC calculation is broken and has been for a while (at least for some teams).


Shady

<img src='http://www.boincsynergy.com/images/stats/comb-1527.jpg'>
ID: 104028 · Report as offensive
Profile FloridaBear
Avatar

Send message
Joined: 28 Mar 02
Posts: 117
Credit: 6,480,773
RAC: 0
United States
Message 104046 - Posted: 26 Apr 2005, 13:59:00 UTC - in response to Message 104028.  
Last modified: 26 Apr 2005, 13:59:26 UTC

> The Team RAC calculation is broken and has been for a while (at least for some
> teams).
>

Well, thanks for confirming what I already suspected.

Paul, you should be able to roughly add the participants' RAC to get the team RAC, in my opinion. If user 1 has an RAC of 1000 and user 2 has an RAC of 750, together they should have an RAC of 1750. There shouldn't be anything complicated about that. Despite teams producing steady credits for a long time now, team RAC for most top teams has failed to rise to even 25% of the "real" value. A team that is producing 80,000 credits per day should not have an RAC of 19,000. The only real explanation is shady's--"it's broken."

ID: 104046 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 104074 - Posted: 26 Apr 2005, 15:25:24 UTC - in response to Message 104046.  

> Paul, you should be able to roughly add the participants' RAC to get
> the team RAC, in my opinion. If user 1 has an RAC of 1000 and user 2 has an
> RAC of 750, together they should have an RAC of 1750. There shouldn't be
> anything complicated about that. Despite teams producing steady credits for a
> long time now, team RAC for most top teams has failed to rise to even 25% of
> the "real" value. A team that is producing 80,000 credits per day should not
> have an RAC of 19,000. The only real explanation is shady's--"it's broken."

You can see the same effect if you compare the RAC for your computers and for your account. I know it is counter intuitive, but, RAC is a calculation that is based in part on the exact moment in time when a credit grant is given.

The only part of the RAC calculation that I know is broken is the part where non-contributing systems will retain the last RAC calculated. For example, in my account (if you look at my computers) there is one named 'Mariciel' and it has a RAC of about 136. Yet that system has not returned a result in a long time. Because of that, the RAC should be 0, but isn't.

ID: 104074 · Report as offensive
Profile Pooh Bear 27
Volunteer tester
Avatar

Send message
Joined: 14 Jul 03
Posts: 3224
Credit: 4,603,826
RAC: 0
United States
Message 104083 - Posted: 26 Apr 2005, 15:46:27 UTC

I agree with Paul. This is an "Recent Average Credit", not a Combined Credit. I can see how this would be confusing.

On a side note, Paul your documentaions Rock! Keep up the good work. Glad someone has the time to create all the needed helpful insights people need.



My movie https://vimeo.com/manage/videos/502242
ID: 104083 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 104130 - Posted: 26 Apr 2005, 17:39:39 UTC - in response to Message 104083.  

> On a side note, Paul your documentaions Rock! Keep up the good work. Glad
> someone has the time to create all the needed helpful insights people need.

Thank you ... :)

But a lot of it is gathering of other peoples work too ...

ID: 104130 · Report as offensive
Profile Bruno G. Olsen & ESEA @ greenholt
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 875
Credit: 4,386,984
RAC: 0
Denmark
Message 104185 - Posted: 26 Apr 2005, 20:18:38 UTC - in response to Message 104074.  

> my account (if you look at my computers) there is one named 'Mariciel' and it

you know he can't see the computer name ;) (I may have some questions about RAC as well later - as I've already read the doc about it a couple of times ;))


ID: 104185 · Report as offensive
Profile FloridaBear
Avatar

Send message
Joined: 28 Mar 02
Posts: 117
Credit: 6,480,773
RAC: 0
United States
Message 104191 - Posted: 26 Apr 2005, 20:28:07 UTC - in response to Message 104074.  
Last modified: 26 Apr 2005, 20:38:26 UTC

> You can see the same effect if you compare the RAC for your computers and for
> your account. I know it is counter intuitive, but, RAC is a calculation that
> is based in part on the exact moment in time when a credit grant is given.
>
> The only part of the RAC calculation that I know is broken is the part where
> non-contributing systems will retain the last RAC calculated. For example, in
> my account (if you look at my computers) there is one named 'Mariciel' and it
> has a RAC of about 136. Yet that system has not returned a result in a long
> time. Because of that, the RAC should be 0, but isn't.

Well, our team RAC used to be 70-80K, but is now "stuck" around 18K. For awhile, I thought it was because of the interruptions and such, but it has not recovered, even though all of the team members' RAC's have recovered nicely. I still stand by my statement that team RAC should be roughly the sum of its members' RAC.

Your example, if anything, would give a higher RAC to the user vs. what it should be, so it does not explain this problem at all.

Team RAC is either broken or it's not broken but completely useless, since it should estimate roughly how many credits per day the team is generating.

[EDIT]: In fact, our team RAC has gone DOWN to 15,500 or so; it's so broken that I can't even begin to speculate what's wrong!
ID: 104191 · Report as offensive
Profile Keck_Komputers
Volunteer tester
Avatar

Send message
Joined: 4 Jul 99
Posts: 1575
Credit: 4,152,111
RAC: 1
United States
Message 104226 - Posted: 26 Apr 2005, 21:43:14 UTC - in response to Message 104191.  

> Well, our team RAC used to be 70-80K, but is now "stuck" around 18K.
> For awhile, I thought it was because of the interruptions and such, but it has
> not recovered, even though all of the team members' RAC's have recovered
> nicely. I still stand by my statement that team RAC should be roughly the sum
> of its members' RAC.
>
This is only true if the member roster has not changed in at least a month. Any change will make the total less accurate because the individual RAC is not transfered to the new team.

> Your example, if anything, would give a higher RAC to the user vs. what
> it should be, so it does not explain this problem at all.
>
It explains this exactly. Since the team RAC is updated more often than the users RAC the users RAC will always total more than the team RAC.

> Team RAC is either broken or it's not broken but completely useless, since it
> should estimate roughly how many credits per day the team is generating.
>
The RAC is a measure of the rate of credit granted. Since it is exponetially reduced it does not really translate into credit per week exactly.

> [EDIT]: In fact, our team RAC has gone DOWN to 15,500 or so; it's so broken
> that I can't even begin to speculate what's wrong!
>
I'm not trying to be mean or anything but the main thing that is wrong is your understanding of RAC. It is a hard thing to get your head around, I'm not even sure I have it down completely.

I did start an investigation into this on the alpha site around the begining of april. It seems to be tracking fairly well there allready. Sum of members = 864.12 vs team RAC = 873.08. After the first week of may we will see if there is a problem.
BOINC WIKI

BOINCing since 2002/12/8
ID: 104226 · Report as offensive
shady

Send message
Joined: 2 Feb 03
Posts: 40
Credit: 2,640,527
RAC: 0
United Kingdom
Message 104253 - Posted: 26 Apr 2005, 23:26:47 UTC
Last modified: 26 Apr 2005, 23:44:43 UTC

I agree with your Floridabear that either the team RAC calculation is broken or that it is a pointless calculation. I go with it being broken because it used to be much more representative of the average amount of credit that a team was doing at the time. When seti had an outage some time ago , the top teams RAC's went into meltdown and never recovered.

If teams are producing 50k credits day in day out without fail , then why would the rac calculaton be showing in the 10k-20k mark and why was this not the case prior to the seti outage ?

perhaps Paul could comment on the RAC that is showing for the top teams, in particular note that Seti Germany produce on average at least 150k a day yet their RAC shows as 15K. Yet if you look at the RAC's of their individual team members then these are a close representation of their average daily credits.

http://www.boincsynergy.com/stats/teams.php?project=sah


Shady
<img src='http://www.boincsynergy.com/images/stats/comb-1527.jpg'>
ID: 104253 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 104444 - Posted: 27 Apr 2005, 11:11:47 UTC - in response to Message 104253.  
Last modified: 27 Apr 2005, 11:17:16 UTC

> perhaps Paul could comment on the RAC that is showing for the top teams, in
> particular note that Seti Germany produce on average at least 150k a day yet
> their RAC shows as 15K. Yet if you look at the RAC's of their individual team
> members then these are a close representation of their average daily credits.

The problem is that RAC is not calculated as a value on a daily basis. It is calculated each and every time credit is granted. This means that the "daily" number you think exists, doesn't exist at all.

Part of the reason for this method of calculation, for better or worse, is that to do it on any other interval would involve multiple full table scans. This is way too expensive of an operation for a number of limited value.

The RAC values on the the stat sites may be more "accurate" in that they could/would reflect values based on calculations made at intervals other than the grants. I did a partial example some time ago (which I cannot find again) but it was an attempt to demonstrate this "oddness".

Someone with a low number of computers may be better suited to coming up with an example by comparing and contrasting the values shown for their computers vs. what is shown in the account.

So, the fundamental problem is that RAC is a rate at which credit is earned, but the time interval is NOT fixed, nor is it stated. The RAC number on the team page is not 15K Cobblestones per hour, nor 15K Cobblestones per Day [edit] it is simply 15K.

Even more fundamental is the fact that as the time dependency for the team is on shorter intervals than the interval for any one individual, this is also going to affect the values ... in fact, I would expect that the Team RAC would be higher than expected as the "hits" are occuring for the team on a shorter time interval (then again, I could be wrong about that too ... but the function is a decreasing exponential with recent time values having more influence).
ID: 104444 · Report as offensive
Profile FloridaBear
Avatar

Send message
Joined: 28 Mar 02
Posts: 117
Credit: 6,480,773
RAC: 0
United States
Message 104528 - Posted: 27 Apr 2005, 16:12:52 UTC
Last modified: 27 Apr 2005, 16:16:29 UTC

One more post and I will drop this issue. Here is a quick sketch of reported RAC vs. average credit per day (calculated as average over the last 7 days) for the top 20 teams:

Top Teams

Here are my points:

1. The RAC and "true" average are way, way different.
2. They're not even correct in a relative sense; compare Czech National Team to SETI.Germany.

C'mon, Paul, just admit they're broken! ;)


ID: 104528 · Report as offensive
N/A
Volunteer tester

Send message
Joined: 18 May 01
Posts: 3718
Credit: 93,649
RAC: 0
Message 104541 - Posted: 27 Apr 2005, 16:36:32 UTC - in response to Message 104528.  
Last modified: 27 Apr 2005, 16:37:22 UTC

Looks more like a feature to me, invented so that we have something to squabble about.
ID: 104541 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 104553 - Posted: 27 Apr 2005, 17:12:45 UTC - in response to Message 104528.  

> C'mon, Paul, just admit they're broken! ;)

Ok, they are broken.

The user understanding function is not working as expected ... :)
ID: 104553 · Report as offensive
Profile FloridaBear
Avatar

Send message
Joined: 28 Mar 02
Posts: 117
Credit: 6,480,773
RAC: 0
United States
Message 104562 - Posted: 27 Apr 2005, 17:29:04 UTC - in response to Message 104553.  

> > C'mon, Paul, just admit they're broken! ;)
>
> Ok, they are broken.
>
> The user understanding function is not working as expected ... :)
>

Heh. My point is that RAC is supposed to reflect some kind of rate (which it does pretty well for users), but for the teams, it is doing no such thing. Therfore, I conclude that it is broken.
ID: 104562 · Report as offensive
Profile Bruno G. Olsen & ESEA @ greenholt
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 875
Credit: 4,386,984
RAC: 0
Denmark
Message 104627 - Posted: 27 Apr 2005, 21:15:14 UTC

just a thought I throw into the discution hehe: maybe team rac WAS broken, but has been fixed *lol*


ID: 104627 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 104637 - Posted: 27 Apr 2005, 22:06:19 UTC - in response to Message 104528.  

> Here are my points:
>
> 1. The RAC and "true" average are way, way different.
> 2. They're not even correct in a relative sense; compare Czech National Team
> to SETI.Germany.

You seem to neglect (or not read) all of the answers given.
If one of your users hasn't run for a week, then started crunching again, his RAC will go down, the team RAC will immediately go down by a lot more.
If one of your team members has detached for whatever reason, waited a month to start crunching again, the team RAC will go down a lot.
If one of your team members has had 16 different accounts for one computer and put them all together just 3 days ago, it'll affect his RAC and the team RAC.

But hey, why neglect all the posts in your thread and just ask of everyone to say that they are wrong, when you have had your preliminary answer here already and aren't awaiting the outcome?
ID: 104637 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 104642 - Posted: 27 Apr 2005, 22:34:03 UTC - in response to Message 104627.  

> just a thought I throw into the discution hehe: maybe team rac WAS broken, but
> has been fixed *lol*

"Fixed", like my puppy? :)

No, it is the way it has been since Beta days ... I am trying to see if I can put together an example that is useful ... probably by the time I get it done no one will care anymore ...

ID: 104642 · Report as offensive
Profile Bruno G. Olsen & ESEA @ greenholt
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 875
Credit: 4,386,984
RAC: 0
Denmark
Message 104672 - Posted: 28 Apr 2005, 0:33:15 UTC - in response to Message 104642.  
Last modified: 28 Apr 2005, 0:35:14 UTC

> > just a thought I throw into the discution hehe: maybe team rac WAS
> broken, but
> > has been fixed *lol*
>
> "Fixed", like my puppy? :)

:D

> No, it is the way it has been since Beta days

Well, it was just a thought ;) Sometimes people can get so used to an error in a program that, when the error is fixed, they think the outcome of that fix is due to a new bug (I think we're on Animal Planet here *lol*). That's why I threw the thought in there, as it seems any changes are blamed on bugs/unintended/unwanted behaviour and not fixes to bugs/unintended/unwanted behaviour ;)

As when the benchmarks are altered to be better, it seems the outcome (usually lower numbers) makes many believe/asume it's a bug

[edit]forgot to mention that at least I'll care, as I'm very interested in understanding this RAC much better than I do at the moment[/edit]


ID: 104672 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Why is team RAC so far off?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.