* The script calculate_credit_multiplier (expected to be run daily as a config.xml task) looks at the ratio of granted credit to CPU time for recent results for each app. Multiplier is calculated to cause median hosts granted credit per cpu second to equal to equal that expected from its benchmarks. This is 30-day exponentially averaged with the previous value of the multplier and stored in the table credit_multplier.

I'd have to read the code in this script to find the definition of recent. I mostly do embedded stuff -- not SQL or Perl. I don't see a reference to RAC or Recent Average Credit.

In the script "recent" is defined as up to 10,000 results which have been granted credit and assimilated in the last 24 hours. The timings are defined at the top of the file, so if a project wants a 60 day moving average or 100,000 results or results returned in the last 48 hours, that change is easy to make. There are some instances where changes like that would be required. If an project gets less than a 10 results a day, the multiplier will bounce around by 10 percent or so. In that case you'd want to look at the last week's worth of results and you might want to go with a 60 day average.

If there are so few hosts (<1000) that the median changes drastically from day to day, that could also be a problem. You'd address that in the same way.

It also doesn't make sense to run it if you just started your project and you don't expect the first result to come back for a year or so. Although if you have trickles, you might want to try to calculate a multiplier based upon your grants of credit for trickles.

It says it looks at 10,000 results picked from results returned that day. I'm not sure how it uses the data from the result, or how much comes from the host table.

The CPU time and granted credit come from the result table. The host benchmarks come from the host table.

Eric

... and all of this data is for the given project, not any other, because it's working against the result table and host table for that project, right?

Thanks for the clarifications. I do mostly embedded systems these days, and that doesn't give much opportunity to use SQL or Perl.

---

Out of the most recent 10,000 returns, pick the median host for Credits granted / (benchmarks * time). Base the multiplier of the FLOPS estimate on this host Note that it throws out all of the high and low benchmarks and under requesting earlier hosts. This removes much of the variability caused by high and low benchmarks and allows the project to correct their estimate.

Given the popularity of Core2 Duo / Quad, the most recent 10,000 will be heavily tilted for high benchmarks and lower times, making the median machine more of a relatively new machine. To illustrate, if a Quad returns 20 results at a time (not unheard of), then only 500 machines doing that make up 10,000 retuns.

Don't think the core2's have taken over the BOINC world yet. The majority of cpu's, by a long way, are P4's between 2.8 and 3.2 GHz. BoincStats Seti CPU Breakdown Which have low benchmarks (BM) and long times. The high end X2's have high BM's, higher than Q6600's, but longer times.
Also, if others have the same experience as my core2's they usually need to request more work before they get anywhere near reporting 20 units, connect interval 0.1days. Five or six tasks reported is the usual max for MB tasks.

If this is the case, those of us with non-Intel and/or single-core Intel machines could be pushed to below median fairly regularly... This could then initiate "slow-host-oriented project shopping", bringing to fruition another fear of people shopping for credit, with a new twist. I'm too tired to think through the math at the moment though...

Given the popularity of Core2 Duo / Quad, the most recent 10,000 will be heavily tilted for high benchmarks and lower times, making the median machine more of a relatively new machine. To illustrate, if a Quad returns 20 results at a time (not unheard of), then only 500 machines doing that make up 10,000 retuns.

If this is the case, those of us with non-Intel and/or single-core Intel machines could be pushed to below median fairly regularly... This could then initiate "slow-host-oriented project shopping", bringing to fruition another fear of people shopping for credit, with a new twist. I'm too tired to think through the math at the moment though...

High benchmarks and lower time do go hand in hand, demonstrating that there is a correlation between the benchmarks and actual work capability. If the benchmarks were perfect for our purposes, it would be a fixed ratio. Eric is taking the median of the actual ratios as representative.

For this project, the most recent 10000 results is about a 10 or 15 minute snapshot just before the script is run. Because most hosts are using always on connections so report any time of day, that's almost certainly a decent sample. There could be some small geographical effect from those like me who only connect about once a day, though. If the script is run during my sleep period my hosts will never contribute, but there are enough similar hosts I don't see any problem in that.

=====================

The comparisons based on the <credit_per_cpu_second> values in the host stats do have some significant differences from the adjustment method. Each host is equally weighted no matter how much work it's doing, but I don't know what criteria the stats sites choose to keep the comparison among "active" hosts so the minimum to be included is uncertain. Basically, the method identifies the set of hosts which are active and common to the two projects, the sum of all those host's <credit_per_cpu_second> values for each project is computed and the ratio taken between those two totals. When I looked at the code for <credit_per_cpu_second> some time ago, I estimated it has about a 2.5 day time constant. It's 2.5 days of CPU time, though, and a quad can have up to 4 days of CPU time in 1 day of wall time. That makes it change fairly quickly with changes in work characteristics. Because the host stats are only dumped once a calendar day some of those variations won't be seen, it's effectively a snapshot.

                                                             Joe

Given the popularity of Core2 Duo / Quad, the most recent 10,000 will be heavily tilted for high benchmarks and lower times, making the median machine more of a relatively new machine. To illustrate, if a Quad returns 20 results at a time (not unheard of), then only 500 machines doing that make up 10,000 retuns.

If this is the case, those of us with non-Intel and/or single-core Intel machines could be pushed to below median fairly regularly... This could then initiate "slow-host-oriented project shopping", bringing to fruition another fear of people shopping for credit, with a new twist. I'm too tired to think through the math at the moment though...

High benchmarks and lower time do go hand in hand, demonstrating that there is a correlation between the benchmarks and actual work capability. If the benchmarks were perfect for our purposes, it would be a fixed ratio. Eric is taking the median of the actual ratios as representative.

For this project, the most recent 10000 results is about a 10 or 15 minute snapshot just before the script is run. Because most hosts are using always on connections so report any time of day, that's almost certainly a decent sample. There could be some small geographical effect from those like me who only connect about once a day, though. If the script is run during my sleep period my hosts will never contribute, but there are enough similar hosts I don't see any problem in that.

=====================

The comparisons based on the <credit_per_cpu_second> values in the host stats do have some significant differences from the adjustment method. Each host is equally weighted no matter how much work it's doing, but I don't know what criteria the stats sites choose to keep the comparison among "active" hosts so the minimum to be included is uncertain. Basically, the method identifies the set of hosts which are active and common to the two projects, the sum of all those host's <credit_per_cpu_second> values for each project is computed and the ratio taken between those two totals. When I looked at the code for <credit_per_cpu_second> some time ago, I estimated it has about a 2.5 day time constant. It's 2.5 days of CPU time, though, and a quad can have up to 4 days of CPU time in 1 day of wall time. That makes it change fairly quickly with changes in work characteristics. Because the host stats are only dumped once a calendar day some of those variations won't be seen, it's effectively a snapshot.

                                                             Joe

This project still has a high concentration of Pentium 4 systems. The "bad egg" projects have significantly higher ratios of Core2-based systems. The effect I am thinking about will be dependent upon that ratio.

As for the stat site portion, the thing I'm curious about is if I do not do any work at all on my Pentium 4, does my cpcs value go down, or will it only go down once I have another task validate and be granted credit. If it does not decay, either at the project itself (origin point for the data) or have a weighting in the cross-project calculation for that chart, then the chart is not reliable.

Out of the most recent 10,000 returns, pick the median host for Credits granted / (benchmarks * time). Base the multiplier of the FLOPS estimate on this host Note that it throws out all of the high and low benchmarks and under requesting earlier hosts. This removes much of the variability caused by high and low benchmarks and allows the project to correct their estimate.

Given the popularity of Core2 Duo / Quad, the most recent 10,000 will be heavily tilted for high benchmarks and lower times, making the median machine more of a relatively new machine. To illustrate, if a Quad returns 20 results at a time (not unheard of), then only 500 machines doing that make up 10,000 retuns.

Don't think the core2's have taken over the BOINC world yet. The majority of cpu's, by a long way, are P4's between 2.8 and 3.2 GHz. BoincStats Seti CPU Breakdown Which have low benchmarks (BM) and long times. The high end X2's have high BM's, higher than Q6600's, but longer times.
Also, if others have the same experience as my core2's they usually need to request more work before they get anywhere near reporting 20 units, connect interval 0.1days. Five or six tasks reported is the usual max for MB tasks.

As I just mentioned to Joe, what you are saying is true for this specific project, but not necessarily other projects. This is a pitfall of trying to force one project's particular dynamics onto another.

Out of the most recent 10,000 returns, pick the median host for Credits granted / (benchmarks * time). Base the multiplier of the FLOPS estimate on this host Note that it throws out all of the high and low benchmarks and under requesting earlier hosts. This removes much of the variability caused by high and low benchmarks and allows the project to correct their estimate.

Given the popularity of Core2 Duo / Quad, the most recent 10,000 will be heavily tilted for high benchmarks and lower times, making the median machine more of a relatively new machine. To illustrate, if a Quad returns 20 results at a time (not unheard of), then only 500 machines doing that make up 10,000 retuns.

Don't think the core2's have taken over the BOINC world yet. The majority of cpu's, by a long way, are P4's between 2.8 and 3.2 GHz. BoincStats Seti CPU Breakdown Which have low benchmarks (BM) and long times. The high end X2's have high BM's, higher than Q6600's, but longer times.
Also, if others have the same experience as my core2's they usually need to request more work before they get anywhere near reporting 20 units, connect interval 0.1days. Five or six tasks reported is the usual max for MB tasks.

As I just mentioned to Joe, what you are saying is true for this specific project, but not necessarily other projects. This is a pitfall of trying to force one project's particular dynamics onto another.

If, as has been suggested, other projects are unhappy at Seti because it is paying above the going rate, then I cannot see how it can be said Seti is trying to force its attempt at adjustment on others. Especially as it should be a self correcting mechanism.
How it works when the median "computer" is different on different projects/application I am not sure. It is almost certain, unless the requirements are changed from the initial proposal, that the median computer for AP tasks will be significantly different from the median computer for MB.

And, to answer your question to Joe, if I understand correctly this method is only about active computers. So if a computer has not been active in a particular project then the figures for that computer are excluded from the calculation.

How it works when the median "computer" is different on different projects/application I am not sure.

I think the basic idea is to first find the "median" computer.

Then, looking at the individual result that "picked" the median computer, you know the duration, and the number of credits granted for the work unit.

From the host record, you have the benchmarks.

You can now calculate what the benchmark * time credit would have been.

The new scaling factor is the ratio between the two.

As this is the median, it should be a "middle-of-the-road" architecture.

That ratio goes into a weighted average with the result from the 29 previous days, so that one "odd" pick won't distort the value too much. I'd also expect some odd picks to be "high" and others "low."

---

And, to answer your question to Joe, if I understand correctly this method is only about active computers. So if a computer has not been active in a particular project then the figures for that computer are excluded from the calculation.

Most of the sites delcare "active" as credit within the past 30 days. With folks that are very bent on having that number be 0.8 to 1.2, there could be pressure to initiate a second reduction before the true effect of the first reduction is completely known.

Basically, Cosmology is the "worst offender" of the active projects as the chart would lead you to believe. If you are not familiar with all of the turmoil going on over there, you (general sense) might still be inclined to rail against them...

On a different subject, some time ago my gut feeling was that the Linux application was faster on identical hardware over at Einstein. I couldn't prove it, as I wasn't willing to go with a dual-boot and the overhead of the VM made the comparison meaningless. Fortunately, Tony came along by chance with some dual-boot systems and by chance with his various charts proved I was right. Gary Roberts, a forum moderator there, has a thread over there in which he explicitly states that the Linux app is significantly faster than the Windows app.

I'm unclear on how something like that is accounted for in this schema. Right now I see it as a "that's just too bad" situation that would have to be addressed by the project. What if the project is either unwilling or cannot? Not speaking specifically about Einstein, but in general. Yes, I know that would be the case no matter the credit scheme, but I'm pointing out known problems that are masked by the current situation in that Windows is the dominant OS. If things were to be "fair" and "equal", then why would that specific situation, where the only difference was the OS and the performance delta was 10% or more, not qualify for the disadvantaged host to have its' own separate "compensation adjustment"?

A "compensation adjustment" was a real-life thing that was done for me in my prior job, as I was paid below the stated minimum pay range for the job. At each review for 3 years, I had access to a separate pool of money that was earmarked for gradually adjusting people in my situation up. I would've rather had it all at once, obviously, but at least it was done rather than saying "that's just too bad". Irony of all ironies was, the same year that I finally made it to the bottom of the pay range, I was also laid off...
On that note, must sleep....

This project still has a high concentration of Pentium 4 systems. The "bad egg" projects have significantly higher ratios of Core2-based systems. The effect I am thinking about will be dependent upon that ratio.

Well lets hope the other projects make the same plots I have (I will suggest that they do, or a stat site could since all the info is in the hosts.xml file). As I mentioned earlier, our current median is a dual processor (or dual core, no way to tell the difference) with 1.5 GFLOP/s per core, which sounds the machine on my desktop at work. It's only marginally slower than the median in beta (which tends to attract higher end machines.)


The good thing about medians is that the middle of a distribution tends to vary far less than the average, especially when exponential growth is involved. If machines get replaced on a 3 year cycle, our median machine is probably about 1.5 years old. (Like the one on my desktop at work).

This method doesn't rely on the median machines being the same across projects. It assumes that on average the work done by a machine is proportional to its benchmark scores.

More tomorrow.... Or today if you're across the pond.

---

How it works when the median "computer" is different on different projects/application I am not sure.

I think the basic idea is to first find the "median" computer.

Then, looking at the individual result that "picked" the median computer, you know the duration, and the number of credits granted for the work unit.

From the host record, you have the benchmarks.

You can now calculate what the benchmark * time credit would have been.

The new scaling factor is the ratio between the two.

As this is the median, it should be a "middle-of-the-road" architecture.

That ratio goes into a weighted average with the result from the 29 previous days, so that one "odd" pick won't distort the value too much. I'd also expect some odd picks to be "high" and others "low."

I know that's a general idea of statistical sampling, but you're talking to someone who has only won something in any raffle contest ONCE in my entire life, and it isn't for lack of trying... (not counting scratch tickets, but I'm in the negative on those, so it isn't a "win", per se). I'd be the sap that could pick the 3 strike chips in a row out of that bag in the game on The Price is Right...

Anyway, I have my doubts about this, but I know the clamoring for it is unending and is fairly high up the food chain...

Like I said, good luck with the quest...

This project still has a high concentration of Pentium 4 systems. The "bad egg" projects have significantly higher ratios of Core2-based systems. The effect I am thinking about will be dependent upon that ratio.

Well lets hope the other projects make the same plots I have (I will suggest that they do, or a stat site could since all the info is in the hosts.xml file). As I mentioned earlier, our current median is a dual processor (or dual core, no way to tell the difference) with 1.5 GFLOP/s per core, which sounds the machine on my desktop at work. It's only marginally slower than the median in beta (which tends to attract higher end machines.)


The good thing about medians is that the middle of a distribution tends to vary far less than the average, especially when exponential growth is involved. If machines get replaced on a 3 year cycle, our median machine is probably about 1.5 years old. (Like the one on my desktop at work).

This method doesn't rely on the median machines being the same across projects. It assumes that on average the work done by a machine is proportional to its benchmark scores.

More tomorrow.... Or today if you're across the pond.

How much is the OS going to play in all of this. I notice you have a E6600 running Linux, Measured floating point speed 1696.48 million ops/sec, and I have one running WinXP same stepping etc, Measured floating point speed 2684.68 million ops/sec.

edit] And if it has no effect here on the credit adjustment will it have an effect on deciding which computers are offered AP work? [/edit

How much is the OS going to play in all of this. I notice you have a E6600 running Linux, Measured floating point speed 1696.48 million ops/sec, and I have one running WinXP same stepping etc, Measured floating point speed 2684.68 million ops/sec.

edit] And if it has no effect here on the credit adjustment will it have an effect on deciding which computers are offered AP work? [/edit

Then it comes to being assigned work or not, the benchmark is being used, yes, but, it's being adjusted by the duration_correction_factor.

That can be a problem, especially if you're running optimized seti, is that the duration_correction_factor is too low for that it should have been with Astropulse-work. If you've also got a large cache-setting, you risk being assigned too much Astropulse-work than can be finished by deadline...

---

"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."

How much is the OS going to play in all of this. I notice you have a E6600 running Linux, Measured floating point speed 1696.48 million ops/sec, and I have one running WinXP same stepping etc, Measured floating point speed 2684.68 million ops/sec.

edit] And if it has no effect here on the credit adjustment will it have an effect on deciding which computers are offered AP work? [/edit

Then it comes to being assigned work or not, the benchmark is being used, yes, but, it's being adjusted by the duration_correction_factor.

That can be a problem, especially if you're running optimized seti, is that the duration_correction_factor is too low for that it should have been with Astropulse-work. If you've also got a large cache-setting, you risk being assigned too much Astropulse-work than can be finished by deadline...

There are indications that the normal method is not going to be only factors in determining AP compatablity. Beta - AP release soon post 34251

I know that's a general idea of statistical sampling, but you're talking to someone who has only won something in any raffle contest ONCE in my entire life, and it isn't for lack of trying...

Which, if you understand statistical sampling, makes sense.

Raffles and Lotteries are not trying to find the median.

---

I know that's a general idea of statistical sampling, but you're talking to someone who has only won something in any raffle contest ONCE in my entire life, and it isn't for lack of trying...

Which, if you understand statistical sampling, makes sense.

Raffles and Lotteries are not trying to find the median.

My point was not about finding the median, but in the fact that if someone is going to get short-changed / not win, I have a higher statistical "batting average" in that category...

Re: Cosmology & the BOINC Combined Statistics chart

According to the chart, the ratio now stands at:

1.928 (3852)

Recently Bernd over at Einstein has said that they are granting 20-25% "too much" compared to SETI.

Per BOINCstats, my Avg. Credit per CPU Second values for my AMD system are as follows:

Einstein - 0.011674
SETI - 0.009661
Cosmology - 0.012499

Clearly if Einstein is deemed 20-25% over SETI, which is true per the cpcs values between the two, then Cosmology is not 92.8% over SETI.

I am unaffected by the 0 credit issues that I brought up in the informational post to Eric. I have only had a total of two of the longer-running 70-credit tasks be figured into the cpcs calculation. I have 13 more coming, all of them running at real (cpu clock) cr/hr rates of around 12-14 cr/hr.

More on the "worst offender" - Cosmology...

Current stat according to "the chart": 1.890 (3831)
Original stat that Joe posted: 2.023 (3884)
Percentage decrease from original: 6.57% (in only 2 days)

According to my own estimates, if my AMD was getting 2x SETI before the change, with runtime multi of 4.0 and credit multi of 1.4, this means that the ratio of credit as compared to SETI becomes:

2.0 / (4.0/1.4) -> 2.0 / 2.9 = 0.69

Current stat for LHC: 0.693 (1183)

At Cosmology, a certain individual (Saenger) asserts that the credit is "just right".

Adjusting stats to reflect 15% reduction here:

1.0 -> 0.85

0.69 is still around 18-19% less than 0.85.

Thus, regardless of the reduction, there is still a significant difference in the amount granted per project.

The question that needs to be asked is: What is parity?

In defining that, one would come up with the maximum allowable differential between projects. In my mind, I would define it as no more than 5%, preferably no more than 3%. After that, you have to define who moves to meet who.

---

More on the "worst offender" - Cosmology...

Current stat according to "the chart": 1.890 (3831)
Original stat that Joe posted: 2.023 (3884)
Percentage decrease from original: 6.57% (in only 2 days)

According to my own estimates, if my AMD was getting 2x SETI before the change, with runtime multi of 4.0 and credit multi of 1.4, this means that the ratio of credit as compared to SETI becomes:

2.0 / (4.0/1.4) -> 2.0 / 2.9 = 0.69

Current stat for LHC: 0.693 (1183)

At Cosmology, a certain individual (Saenger) asserts that the credit is "just right".

Adjusting stats to reflect 15% reduction here:

1.0 -> 0.85

0.69 is still around 18-19% less than 0.85.

Thus, regardless of the reduction, there is still a significant difference in the amount granted per project.

The question that needs to be asked is: What is parity?

In defining that, one would come up with the maximum allowable differential between projects. In my mind, I would define it as no more than 5%, preferably no more than 3%. After that, you have to define who moves to meet who.

I just looked up these credit/cpu sec on BOINCStats
BOINC 0.003009
Seti 0.003628
Cosmology 0.009987
They show that Cosmology grants 3.319 * more that BOINC and 2.753 * more than Seti.

Do these figures tell the truth? Well yes, probably at this moment in time,
but do they show the whole truth? No.

They are in fact the figures for one type of cpu, and are therefore not representative of any average or median when looking at cross project parity.

The cpu type, well yours of course, an AMD Athlon 64 3700+.

I just looked up these credit/cpu sec on BOINCStats
BOINC 0.003009
Seti 0.003628
Cosmology 0.009987
They show that Cosmology grants 3.319 * more that BOINC and 2.753 * more than Seti.

Do these figures tell the truth? Well yes, probably at this moment in time,
but do they show the whole truth? No.

They are in fact the figures for one type of cpu, and are therefore not representative of any average or median when looking at cross project parity.

The cpu type, well yours of course, an AMD Athlon 64 3700+.

The median or average does not matter to me, personally. What matters to me is my own stats. I tend to think that is the case for most people as well, as generally speaking, most people are not philanthropic / altruistic / empathetic when it comes to the expenditure of their resources.

Not only that, but my CPU might be classed by name as a 3700+, but it is overclocked to a performance level somewhere near a default-clocked FX-57. How many others in the 3700+ group are overclocked? Unknown. How many are underclocked? Unknown. Is the "average" or "median" sample within that subset actually clocked appropriately (2200MHz)? Unknown. Is the default clock rate 2200MHz? Yes, for Socket 939, but no for socket 754 (default 2400).

All I can do as an individual is look at my own stats. They indicate how things are for me, personally. As they say, YMMV... One person's "parity" can be another person's "inequity"... Thus, taking this to the logical conclusion, there will still be a significant chance for "credit shopping"... Meanwhile, there will be high incidences of angering people who are not engaged in the credit shopping...

---

The question that needs to be asked is: What is parity?

In defining that, one would come up with the maximum allowable differential between projects. In my mind, I would define it as no more than 5%, preferably no more than 3%. After that, you have to define who moves to meet who.

Right now, we're looking at a range of about 400% in really round numbers.

... and I'd think that reducing that by an order of magnitude would be a really good first goal.

Note that the change that started this thread only adjusts SETI@Home, and does not reference any project.

---