How Are Benchmarks Calculated?

Message boards : Number crunching : How Are Benchmarks Calculated?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 24282 - Posted: 10 Sep 2004, 0:37:42 UTC - in response to Message 24240.  

> Thanks Rom.
>
> To all Celeron users !!! In order to complete your work in time, be sure to
> run the benchmarks manually with your box fully loaded. Launch a defrag or
> play doom while you BOINC benchmarking...
>
> Friendly
> Marc
>
Im not sure how that would work since benchmarking suspends all other CPU activity. I cant even open up a new window let alone start an application.
ID: 24282 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 24312 - Posted: 10 Sep 2004, 2:42:34 UTC

Hi all,
Thanks Rom for the input on the Benchmarks. I appreciate all you do.

>BTW, MLCudd.... Aim High! Go USAF! ;) (you know a zoomie's gotta poke a little fun at a squid) :)

Hey Thunder,
I amied higher than the Air Force could Fly. We flew F-14's, The old obselete
F-4 Phantoms, and I was lucky enough to be part of the testing on the YF-17, Which became the Navy's Best Fighter, The F-18 Hornet. The Air Force only wanted me to work on the Engines. :-( no fun in that!!

And Thanks everybody who had input on this thread.

Have A Great Day.

Warm Regards To All,

Rocky Cudd
ID: 24312 · Report as offensive
Profile Razorirr
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 92
Credit: 7,414
RAC: 0
United States
Message 24346 - Posted: 10 Sep 2004, 3:56:25 UTC

want to know somthing weird i actually was able to get my to completion times to get into the 1 day mark. my computer usually says they will take 8-9 hrs but i ran a bench mark while running a bunch of computer sterss test at the same time( hot cpu tester lite edition, the tourture test that comes on gimps, etc.) that was bad. another thing i dont run any thing on this computer except boinc unless im on these forums. the last wu seti ran took 22 hrs when it said it would take 8.5. the scary thing is if its taking almost 3 times longer than it should how long will my cpdn one take its competion time is 2352:33:16. if it ran the same rate thats 7000 to 7100 hrs. :>
ID: 24346 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 24356 - Posted: 10 Sep 2004, 4:07:01 UTC
Last modified: 10 Sep 2004, 4:11:08 UTC

Hi Roland,

>>how long will my cpdn one take its competion time is 2352:33:16. if it ran the same rate thats 7000 to 7100 hrs. :>




A short and Sweet Answer

A Long Long Time!! :-) :-) :-)


Happy crunching!!

Regards,

Rocky
ID: 24356 · Report as offensive
Profile Razorirr
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 92
Credit: 7,414
RAC: 0
United States
Message 24363 - Posted: 10 Sep 2004, 4:14:53 UTC - in response to Message 24356.  

> Hi Roland,
> A short and Sweet Answer
>
> A Long Long Time!! :-) :-) :-)
>
>
> Happy crunching!!
>
> Regards,
>
> Rocky
> <img> src="http://boinc.mundayweb.com/seti2/stats.php?userID=948&useCached=true">
>
hey theres a good thing to it il go from worst to first in one wu except for the fact that as i was writing this it said this
climateprediction.net - 2004-09-10 00:07:33 - Unrecoverable error for result 1y22_100111719_0 (CreateProcess() failed - The paging file is too small for this operation to complete. (0x5af))
climateprediction.net - 2004-09-10 00:07:33 - CreateProcess() failed - The paging file is too small for this operation to complete. (0x5af)
im going to their message boards right now to post this if it already hasent happened
ID: 24363 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 24367 - Posted: 10 Sep 2004, 4:19:10 UTC

Roland,


>>Unrecoverable error for result 1y22_100111719_0 (CreateProcess() failed - The paging file is too small for this operation to complete. (0x5af))


That is the exact reason why I detached from CPDN until all the bugs are worked out of Boinc. From what I got from the other message posts, It has something to do with I believe Server tuning or timing. When I find the Threads, I will forward them


Have A Better Tomorrow,

Rocky Cudd
ID: 24367 · Report as offensive
Profile Rachel
Avatar

Send message
Joined: 13 Apr 02
Posts: 978
Credit: 449,704
RAC: 0
United Kingdom
Message 24370 - Posted: 10 Sep 2004, 4:21:26 UTC - in response to Message 24356.  

Hello
my pc is a Dell 2.8ghz with pentuim 4 processors with ht and 520 ram with 80gb hard drive and I am doing two seti@home boinc wu's in 3 hours and 20 mins, so around one hour and 40 mins per unit.
ID: 24370 · Report as offensive
Profile Razorirr
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 92
Credit: 7,414
RAC: 0
United States
Message 24378 - Posted: 10 Sep 2004, 4:26:29 UTC - in response to Message 24367.  
Last modified: 10 Sep 2004, 4:35:41 UTC

> Roland,
>
>
> >>Unrecoverable error for result 1y22_100111719_0 (CreateProcess()
> failed - The paging file is too small for this operation to complete.
> (0x5af))
>
>
> That is the exact reason why I detached from CPDN until all the bugs are
> worked out of Boinc. From what I got from the other message posts, It has
> something to do with I believe Server tuning or timing. When I find the
> Threads, I will forward them
>
>
> Have A Better Tomorrow,
>
> Rocky Cudd
> <img> src="http://boinc.mundayweb.com/seti2/stats.php?userID=948&useCached=true">
>
i just did that i will try them again next year. i startedd with them on the 23 of august and havent ever got to a trickle but ive had five wus
1. boinc crashed and lost it
2. tryed to go to 4.08 and crashed it
3. same as two
4. paging error
5. the one it downloaded before i detached
ID: 24378 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 24383 - Posted: 10 Sep 2004, 4:35:49 UTC

Rach,
You have a nice fast box. I will gladly trade you 2 of mine for it.

Are you on Dial-up?

Being new, I would investigate the links that you were sent previously. They will be of tremendous help. Paul Buck is the master of documentation.


Like I said just holler, someone will be there to help.

Regards,

Rocky
ID: 24383 · Report as offensive
Profile Rachel
Avatar

Send message
Joined: 13 Apr 02
Posts: 978
Credit: 449,704
RAC: 0
United Kingdom
Message 24386 - Posted: 10 Sep 2004, 4:40:10 UTC - in response to Message 24383.  

> Rach,
> You have a nice fast box. I will gladly trade you 2 of mine for it.
>
> Are you on Dial-up?
>
Hello

Got my pc last week.Got broadband last week.Before that I had an 8 year old pc which was 333mhz and only 2.9gb hard drive and 250 ram.It took 24 hours to do one seti@home classic screensaver ,lol.It would crash 3 times a day.I bought my new pc as the other one was unusable.It's nice having something that actually works.My dial up I had was so slow and it was only £3 a month more to have 10x broadband.Well worth an extra £3 a month I say.
ID: 24386 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 24388 - Posted: 10 Sep 2004, 4:40:58 UTC

Hey Roland,
I don't think it will be necessary to wait until next year to restart, I have a feeling that all the bugs will be resolved in the next 30 days.

By the way, the new program LHC opens to the public around the 29th of Sept. It is on the Boinc Platform. I will look for you there.

Regards,

Rocky
ID: 24388 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 24464 - Posted: 10 Sep 2004, 10:21:25 UTC - in response to Message 24383.  

Rocky,

> Being new, I would investigate the links that you were sent previously. They
> will be of tremendous help. Paul Buck is the master of documentation.

Come on, you are making me blush!

Thank you though... If I just help one, that is the reward.

Rachel, not to scare you off, but I do have about 230 pages of stuff you can read. Lots of examples. And my friends John and John keep me honest and right (well, I think they are my friends, I am not sure if the feeling is mutual, I never asked!)

:)
<p>
Click Me!


ID: 24464 · Report as offensive
Profile mlcudd
Volunteer tester
Avatar

Send message
Joined: 11 Apr 03
Posts: 782
Credit: 63,647
RAC: 0
United States
Message 24538 - Posted: 10 Sep 2004, 14:01:31 UTC

Paul,
Your Documentation has been a tremendous help for me, and I AM SURE OTHERS!!

Thank You.

Warm Regards,

Rocky Cudd
ID: 24538 · Report as offensive
Profile Razorirr
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 92
Credit: 7,414
RAC: 0
United States
Message 24868 - Posted: 11 Sep 2004, 1:08:05 UTC - in response to Message 24388.  

> Hey Roland,
> I don't think it will be necessary to wait until next year to restart, I
> have a feeling that all the bugs will be resolved in the next 30 days.
>
> By the way, the new program LHC opens to the public around the 29th of Sept.
> It is on the Boinc Platform. I will look for you there.
>
> Regards,
>
> Rocky
>
yah i found out about LHC the day after they were open but they all ready filled up. ill have to wait for the final front ear to start a team there though.
ID: 24868 · Report as offensive
Profile dazphotog

Send message
Joined: 13 Mar 02
Posts: 73
Credit: 99,224
RAC: 0
United States
Message 24870 - Posted: 11 Sep 2004, 1:13:53 UTC - in response to Message 24868.  

I have a 2.0 Ghz Celeron with 256meg of Ram...and the benchmarks say a workunit should take 6.5 hours and most units take 6.5 to 6.75 hours.

Don
ID: 24870 · Report as offensive
Profile Christopher Hauber
Avatar

Send message
Joined: 10 Feb 01
Posts: 196
Credit: 71,611
RAC: 0
United States
Message 24939 - Posted: 11 Sep 2004, 3:49:51 UTC - in response to Message 24870.  

Hmm.. I knew the Celeron was the "value" line but I didn't realize just how inferior to the Pentiums they really were. My P3 1.0 GHz with 512 MB RAM only takes about an hour longer than your 1.0 GHz Celeron. I'm surprised there isn't a bigger difference there.

As an added comparison my "2800+" (2400+ overclocked) processes work in about 2:50 on the low end of the 2800+ range (I say "range" because I have lowered the speed a couple times recently without it changing the 2800+ designation).

Chris

> I have a 2.0 Ghz Celeron with 256meg of Ram...and the benchmarks say a
> workunit should take 6.5 hours and most units take 6.5 to 6.75 hours.
>
> Don
>
>
ID: 24939 · Report as offensive
Profile Razorirr
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 92
Credit: 7,414
RAC: 0
United States
Message 24956 - Posted: 11 Sep 2004, 4:50:23 UTC

i got the same processor and ram as he did but my computer must be slow they used to say 8 hours. they wuold take 22 though. now after increasing my virtual paging file from 200 mgs to 3 gigs of memory i dropped to 6.5 hrs but i havent goten to run one from start to finish. when i get enough cash im going to take the inspiron 1100 notebook in to get a bigger ram chip and swap out the celeron with a mobile p4 with hyperthreading tech. that should make it faster.
ID: 24956 · Report as offensive
Profile Celeron300A@450

Send message
Joined: 16 May 99
Posts: 2
Credit: 318,306
RAC: 0
Malaysia
Message 52402 - Posted: 9 Dec 2004, 11:01:52 UTC - in response to Message 24030.  

> about 3.8 hours, and the WU's recieved fit that box. However my Intel Celeron
> 2.8 box, says that I should be crunching WU's in 4.22 hours. Problem is, it
> takes exactly double that time. When I ran a new set of Benchmarks yesterday,
> the WU's I just got, are for the exact same amount of time 4.22 hours. I did

well, my celeron 2.4ghz is crunching wu's in under 4 hours, when boinc says it should take more than 8 hours.

im not sure if these are factors, but ill list them for analysis :

1) using single stick of cheap unbranded ram. 512mb shared with on board agp chip

2) mobo chipset is single channel via

3) swap file is set to 1024mb

4) usage load is basically for surfing, email, streaming radio.

i do have concerns that i may run out of work. not so much that i wont finish work in time.
ID: 52402 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 52457 - Posted: 9 Dec 2004, 17:49:18 UTC
Last modified: 9 Dec 2004, 17:56:53 UTC

Hi all,

I'm the Ben Rom was referring to.

As for how benchmarks are calculated...

Whetstone is the name of the benchmark that is reported on your [Show computers] web page as "Measured floating point speed"
Dhrystone is the name of the benchmark used for "Measured integer speed".
Float = 1.48283 or 3.141592, Integer = 1 or 2 or 938283 or 2004.
Floats have a decimal point, integers are also called whole numbers.

Whetstone
Whetstone does 8 different groups of tests (repeatedly of course), times how long they took to finish, and pops out a number. [ops performed]/[time].

These tests all use floating point math operations of the CPUs being tested. Some of them are simple math plus, minus, times, divide and others check trigonometry feaures sine, cosine, tangent, exponent.

Dhrystone
Dhrystone checks repeated integer operations, and several operating system file handling operations.

Neither of the tests really checks how well/fast a system can access memory, and seti (for example) acesses memory a lot.

Here is an example of memory introducing a delay. A pentium 4 CPU of any speed, can calculate the sine of an angle in approximately 170 tics of its internal clock. It could have performed 170 regular integer additions in this time.

But if it wanted to do an integer addition on a number somewhere out in memory (say it was working on a table of numbers), the CPU might have to wait for this memory integer to be delivered to the CPU as much as 260 tics. So a badly timed integer+memory operation would take far longer than a sine calculation.

This is where Celeron CPUs can really slow down. Pentium has many more features to predict when the CPU might be getting memory, and begins getting it long before the CPU actually calculates with it. Thus much less delay for most memory operations = faster.

Predicted time vs actual
Each WU delivered to your machine includes an estimated number of floating point (FP) calculations. BOINC divides this by the FP benchmark number to estimate completion time. Seti's WUs estimate number is currently allways 27.9 trillion (american), however the actuall number of FP ops varies greatly which is why they don't all take the same ammount of time.

A better estimate might be calculated on the servers, but it would probably take so long looking through the WU data that much fewer WUs would be generated and host machines wouldn't get WUs.

Seti vs Benchmarks
Seti uses almost all "single" precision floating point math, whetstone is all "double" precision math. On Intel x86 processors the speed difference in calculating single vs. double isn't very large.

Seti uses mostly add, sub, multiply and divide. About 20% of its time is spent in trigonometry. Almost all the time in Whetstone is used for trigonometry.

Memory access speed and trigonometry are the two major reasons that the benchmark results and seti processing speed don't match up on many systems.
ID: 52457 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 52492 - Posted: 9 Dec 2004, 22:38:09 UTC

Ben,

Nice explanation!

I've been thinking of adding a small faq to my site relating to common questions concerning some of the more technical aspects of the credit system as it seems to relate to my optimised boinc clients (benchmark scores).

I've noticed you've posted some nice explanations such as the one above together with a previous explanation of the formula to calculate claimed credit that also caught my eye. I was wondering if you'd mind if I lifted/borrowed your excellent explanations and used them on my site, acknowledging yourself, of course, as the source? This is the type of info you don't come across to often and I feel having it available on my site may be helpful to others.

Thanks, and I await your thoughts,

Ned


*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 52492 · Report as offensive
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : How Are Benchmarks Calculated?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.