Benchmark results don't seem right. Will this affect my scores?

Questions and Answers : Windows : Benchmark results don't seem right. Will this affect my scores?
Message board moderation

To post messages, you must log in.

AuthorMessage
Mark Stevens

Send message
Joined: 15 Feb 01
Posts: 28
Credit: 177,705
RAC: 0
United Kingdom
Message 4199 - Posted: 4 Jul 2004, 23:02:15 UTC

I have 2 PCs. Both Dell Precision 450 twin CPUs with Windows XP Pro SP1. One has 2 x 2.66GHz Xeons the other 2 x 3.06GHz.

When running the benchmark the double precision MIPs on both is always about the same but the integer MIPs varies quite a bit. eg. I've had between 2379 and 3697 on one while the dp MIPs were always around 1726. Each time I run this the result varies by quite a margin. I always make sure nothing else is running.

If these benchmark are incorrect, as it seems, will this affect my score calculations?
ID: 4199 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 4267 - Posted: 5 Jul 2004, 3:32:59 UTC

Yes incorrect benchmarks will affect your credit requests. Wait for a quiet moment on your computer, and re-benchmark. In the GUI, it is in the File menu.

ID: 4267 · Report as offensive
EclipseHA

Send message
Joined: 28 Jul 99
Posts: 1018
Credit: 530,719
RAC: 0
United States
Message 4276 - Posted: 5 Jul 2004, 3:56:57 UTC - in response to Message 4267.  

> Yes incorrect benchmarks will affect your credit requests. Wait for a quiet
> moment on your computer, and re-benchmark. In the GUI, it is in the File
> menu.
> <a> href="http://www.boinc.dk/index.php?page=user_statistics&userid=9915">
>
>

Basically, benchmarks are really screwed up right now anyway. If not, my Celeron 2.2g machine (linux) wouldn't be request 2 times the work of my 2.40GHz Pentium4 (win2k) using the same preferences! (yes, they been benchmarked no less than 3 times since this mess went live!)

The whole benchmarking mess was completely redone a short time before the launch and is still way off....

I'm suprised that VII didn't know this! (well, he only posts positive things...)

This ALSO realy impacts claimed vs granted credits, as "claimed credit" will vary a great amount on the same HW if it's windows or linux!

I laugh every time I read a UCB chearleader claiming that "science" is the top goal! It's clear that "computer science" was not considered!
ID: 4276 · Report as offensive
Mark Stevens

Send message
Joined: 15 Feb 01
Posts: 28
Credit: 177,705
RAC: 0
United Kingdom
Message 4307 - Posted: 5 Jul 2004, 6:18:29 UTC - in response to Message 4276.  

> > Yes incorrect benchmarks will affect your credit requests. Wait for a
> quiet
> > moment on your computer, and re-benchmark. In the GUI, it is in the
> File
> > menu.
> > <a>
> href="http://www.boinc.dk/index.php?page=user_statistics&userid=9915">
> >
> >
>
> Basically, benchmarks are really screwed up right now anyway. If not, my
> Celeron 2.2g machine (linux) wouldn't be request 2 times the work of my
> 2.40GHz Pentium4 (win2k) using the same preferences! (yes, they been
> benchmarked no less than 3 times since this mess went live!)
>
> The whole benchmarking mess was completely redone a short time before the
> launch and is still way off....
>
> I'm suprised that VII didn't know this! (well, he only posts positive
> things...)
>
> This ALSO realy impacts claimed vs granted credits, as "claimed credit" will
> vary a great amount on the same HW if it's windows or linux!
>
> I laugh every time I read a UCB chearleader claiming that "science" is the top
> goal! It's clear that "computer science" was not considered!

Thanks, not good news.

I'm running the benchmarks manually as you'd expect with everything that could affect it shut down. The integer results vary by as much as 100% which is crazy. Also, it makes you wonder if you'd get a fair result when it's run automatically - anything else could be running then.

I really hope they sort this out. The description of the credit system talks about the way it's done as being fair. I think they're falling well short of their objective at present. Does anyone know if this is on the to be fixed list?
ID: 4307 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 4702 - Posted: 6 Jul 2004, 1:40:48 UTC - in response to Message 4307.  

> > > Yes incorrect benchmarks will affect your credit requests. Wait for
> a
> > quiet
> > > moment on your computer, and re-benchmark. In the GUI, it is in
> the
> > File
> > > menu.
> > > <a>
> >
> href="http://www.boinc.dk/index.php?page=user_statistics&userid=9915"> [/url]
ID: 4702 · Report as offensive
EclipseHA

Send message
Joined: 28 Jul 99
Posts: 1018
Credit: 530,719
RAC: 0
United States
Message 4750 - Posted: 6 Jul 2004, 4:09:12 UTC - in response to Message 4702.  


> I believe that the development team is aware of the issue. Some of them have
> had the problem of benchmarks that differ by a very large margin. The main
> problem is that benchmarks are run right after program installation, and some
> machine reboots when the most things are interfeering with the benchmark.


Maybe they shouldn't have completely changed the benchmark method a few days before going live! (a big impact to the crunchers) The router problem meant that the beta users had little chance to test before that! (I was able to get 3.18 the same day that Live went live!)

The software was not ready to go "live" there VII... You know it as well as I do!

The benchmarks are bad, and it effects the credits that all users have gotten since 6/22!

ID: 4750 · Report as offensive

Questions and Answers : Windows : Benchmark results don't seem right. Will this affect my scores?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.