I was dinking around yesterday on one of my AMD basket computers and bumped up the FSB a bit, ran ran SANDRA and BOINC benchmarks... all seemed about right based on what a Mhz or two has done in the past (I have it in a .xls). This basket runs BOINC v4.05. I let it run overnight and no problems so applied the same changes to the another basket (both AMD XPs, same mobo). This machine is running 4.09. While the SANDRA benchmarks show about the same increase as the other basket computer this one showed a drop in the BOINC floating point benchmark. After spending an hour or so double checking all my BIOS settings between the two computers it dawned on me that these two had different versions of BOINC on them and it was only the BOINC benchmark that seemed off. I now wish I'd checked the benchmarks right after installing v4.09 to be sure. However, while both of the CPUs where AMD XP 2000+ (overclocked to 2400/2600)one is a Thoroughbred core and one is the Thorton core). In the past they both produced about the same numbers on all benchmarks. But it seems to me that v4.09 benchmarking routines must have changes that, at least on this CPU, result in lower floating point numbers. Can anyone confirm this code change?

Thanks, Skip

I did some comparisons on my Win XP Pro SP1 PC running on a P4 2.4@2.88GHz.
There is definitely a difference. The floating point operations dropped down some 20% and the integer speed went up some 4%.

Greetings from Belgium.

pre 4.09 they were having problems with compilers 'optimizing' the benchmark code and essentially chopping out a good chunk of it because it "didn't do anything" as far as the compilers could see. Essentially the compiler saw the program doing some operation on a variable a few thousand times but noticed that the variable was then never used so it just took out the few thousand loops to 'speed up' the program. Unfortunately, this is one instance where you don't want things to run faster :)

I never actually bothered to look and compare benchmarks but I immagine most people's will fall. Not sure where the 4% increase in integer speed is coming from but I'm sure they made several modifications to the benchmark code.


---------------------------------------
- A member of The Knights Who Say NI!
Possibly the best stats site in the universe:
http://boinc-kwsn.no-ip.info

Skip

Everyone, regardless of processor or OS, is getting lower benchmarks with the v4.09. There has been quite a bit of discussion on the change in Benchmark results in the newer 4.09v being lower due to the compiler change.

The benchmarks for my P4 HT systems were 18.8% lower on the Whetstone but were only a few points lower on the Dhrystone.
----------



Join the Overclockers.com SETI Team!

OK! Thanks ya'll for all the good info and I'll not worry about testing further to confirm that it was v4.09.

Skip

> OK! Thanks ya'll for all the good info and I'll not worry about testing
> further to confirm that it was v4.09.

Skip,

Actually, they fixed the benchmark. One of the problems with benchmarks is that optimization frequently "improves" the performance by trimming out code that it determines that is not needed.

Yet, the code is there to "test" the speed, not to get an answer that is used. So, they changed the code so that the optimizations would not remove necessary code.

<p>
For BOINC Documentaion: Click Me!

> Actually, they fixed the benchmark. One of the problems with
> benchmarks is that optimization frequently "improves" the performance by
> trimming out code that it determines that is not needed.
>
> Yet, the code is there to "test" the speed, not to get an answer that is used.
> So, they changed the code so that the optimizations would not remove
> necessary code.
>

Paul,

What was this a fix to - compiling the benchmark or the benchmark determining the processors speed? Maybe the compilation part is "fixed" but it certainly doesn't work for estimating the completion time for workunits. And that time is used for other parts of the project, like how many workunits to download, scheduling between multiple projects and determining how much credit a completed workunit returns.

For 4.05/4.06, whatever value the benchmarks computed came pretty close to the actual CPU time for WU's. But for 4.09, not only are the benchmark numbers low, but that turns into artifically longer times for WU estimates.

On my Win98 system (700Mhz PIII, 256M RAM), WU's take around 9 hours to complete. BOINC 4.06 estimate is pretty close, the WORK tab shows around 9 hours for Ready to Run WU's. This estimate is a bit high, they usually take almost 20 minutes longer. (Pretty close, thats less tahn 4% off).

But when I ran 4.09, the benchmark numbers (whetstones) are lower by like 30%, and the estimated completion time jumped to well over 12 hours. Even though they still take the same 9 or so hours to complete (some are quicker). And off enough that I've returned to an earlier version.

From what I see, benchmarks for 4.09 are broken.

> But for 4.09, not only are the benchmark numbers
> low, but that turns into artifically longer times for WU estimates.

Looking to the completion time for both S@H and LHC@Home, the time is some 30% too high. This would mean the benchmark are 42.8% too low.

Two solutions for the project are possible...one is better.

1. Change estimation code.
or
2. Multiply the benchmark by some #


Estimation code is kinda tricky, but it should be possible to average all WUs on a given machine (after at least one is complete) as most take about the same time.

Occasionally you will get a WU with data differences...such that other functions are called to see if that was ET calling. These take longer.
Usually when it does take longer, ET is using Do Not Disturb.

Walt,

> What was this a fix to - compiling the benchmark or the benchmark determining
> the processors speed? Maybe the compilation part is "fixed" but it certainly
> doesn't work for estimating the completion time for workunits. And
> that time is used for other parts of the project, like how many workunits to
> download, scheduling between multiple projects and determining how much credit
> a completed workunit returns.

Since the compiler was optimizing out some of the steps for the benchmark, they came up with a way to prevent that. Therefore, the numbers comming out of the benchmark should be more representitive of what they originally intended. Unfortunately, it caused in most cases a decrease in the reported numbers.

> For 4.05/4.06, whatever value the benchmarks computed came pretty close to the
> actual CPU time for WU's. But for 4.09, not only are the benchmark numbers
> low, but that turns into artifically longer times for WU estimates.

On the limited number of times I have watched the processing of work I have not seen this. I could be mistaken of course, but I have not seen this effect.

> On my Win98 system (700Mhz PIII, 256M RAM), WU's take around 9 hours to
> complete. BOINC 4.06 estimate is pretty close, the WORK tab shows around 9
> hours for Ready to Run WU's. This estimate is a bit high, they usually take
> almost 20 minutes longer. (Pretty close, thats less tahn 4% off).
>
> But when I ran 4.09, the benchmark numbers (whetstones) are lower by like 30%,
> and the estimated completion time jumped to well over 12 hours. Even though
> they still take the same 9 or so hours to complete (some are quicker). And
> off enough that I've returned to an earlier version.
>
> From what I see, benchmarks for 4.09 are broken.

And you may well be correct. As with all of the other aspects of BOINC this is a hot button issue. We struggled with it in the Beta test. And I suspect that we will be struggling with it for some time to come. :(

But, I will stick to my song .. :)

This is still a cosmetic issue. I thnk the cross-platform GUI is far more important. Then these kinds of issues can be addressed. We are accompplishing science now. But we still have legacy projects that can be retired freeing resources that can then be used to improve the running environment we are in now.
<p>
For BOINC Documentaion: Click Me!