Lunatics benchmark, 8.04r3306_sse2-linux64

Message boards : Number crunching : Lunatics benchmark, 8.04r3306_sse2-linux64
Message board moderation

To post messages, you must log in.

AuthorMessage
Gene Project Donor

Send message
Joined: 26 Apr 99
Posts: 150
Credit: 48,393,279
RAC: 118
United States
Message 1756585 - Posted: 15 Jan 2016, 15:48:23 UTC

I picked, at random, a v8 work unit and ran it in benchmark mode comparing the current stock x86_64 (linux) with the Lunatics sse2 optimized app. Here's the bottom line:
stock app: 12267 seconds
Lunatics: 7356 seconds ---> relative 166% speed.

This was on an AMD FX-4300 CPU while running three other stock instances in the background. (A reminder regarding this CPU -- it has 4 cores but only 2 FPU's.) I'm assuming that running the benchmark without the background tasks would be faster (less FPU contention) but maybe the same relative speed would result. In any case, multiple concurrent tasks is my normal configuration.

The result comparison was "strongly similar" with Q=99.98%. I did a "diff" on the result files and found all values to differ only at the 7th or 8th significant digit. But... what does this mean?
<setiathome_version>7.99</setiathome_version>

This line appears at the end of the result.sah (xml) file generated by the Lunatics app, just before the closing </result> tag. I was expecting to see either 8.00 or 8.04. I am holding off on configuring to run the optimized application live until someone can (1) reassure me that the version number is harmless, or (2) advise me to wait for a fix in the optimized app.

Thanks to the Lunatics team for their efforts. They do make a difference!
ID: 1756585 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1756629 - Posted: 15 Jan 2016, 17:39:43 UTC - in response to Message 1756585.  

That was news to me, as well! I've been looking as .res files till I'm blue in the face, and I'd never noticed it - though in my defence, I'm mainly using automated tools which either compare or summarise the signal blocks.

I've consulted one of our developers, and it's a new feature this time, to help defeat the silly b****rs who use completely the wrong app. For the moment, I think it's safe to say that any version number is a safe indicator of an application built to v8 standards - though obviously, you should switch to a finalised version reporting version 8.xx as soon after an officially announced v8 app is released as possible. That should happen shortly after v8 GPU applications are transferred the the Main project.

I'd judge that v7.99 is safe to run, but it probably indicates a Beta version, so normal precautions apply. Keep watching for announcements of updated versions, and consider removing the Lunatics app for the time being (reverting to stock) if you're not happy with the number of inconclusive validations you see.
ID: 1756629 · Report as offensive
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 388
Credit: 1,857,738
RAC: 0
Finland
Message 1756696 - Posted: 15 Jan 2016, 21:46:09 UTC - in response to Message 1756629.  

Not a very reliable method to determine the actual version used though. 8.00 here and 8.04 and 8.05 at Beta all report themselves as 8.00.
ID: 1756696 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1756722 - Posted: 15 Jan 2016, 23:12:29 UTC - in response to Message 1756696.  

Not a very reliable method to determine the actual version used though. 8.00 here and 8.04 and 8.05 at Beta all report themselves as 8.00.

I think you're looking at two different places.

What's displayed to even the most casual observer looking at the 'Application' column in BOINC Manager is the <version_num> from <app_version> - and if that <app_version> is defined in an app_info.xml file, it can be set to anything the user desires. The server (especially the validator) will ignore it.

The <setiathome_version> found by Gene embedded in the uploaded result file (which the user usually never sees) is - I'm told - hard-coded into the application binary at compilation. That again is meaningless at an open source project - any fool with a compiler or a Hex editor can change it - but it's slightly less trivial than text-editing an xml file.
ID: 1756722 · Report as offensive
Gene Project Donor

Send message
Joined: 26 Apr 99
Posts: 150
Credit: 48,393,279
RAC: 118
United States
Message 1756771 - Posted: 16 Jan 2016, 3:42:25 UTC - in response to Message 1756722.  

Further note: The stock MB v8 app does properly set the <setiathome_version> tag to 8.00.

I have held off reconfiguring (anonymous platform) for the Lunatics 8.04 version just because of the ambiguity it might introduce. I think I can download the Lunatics AVX optimized app and run it in benchmark to see if it gives the same output. In previous AVX benchmarks they have run slower than the SSE2 versions so I wasn't really planning to use it. But it might be a useful clue to the developers to know if this is a common (to AVX and SSE2) issue or just to one.

Yes, Richard, the result files compare in the benchmark with no serious problems noted; but, being a curious sort, I used the (Linux) diff utility just to see how closely the numeric values matched and, of course, the version number popped up entirely serendipitously. From all appearances it is hard coded into the app. My suspicion (highly speculative) is that the developers put in the "magic" value 7.99 to mark the output as a "pre-release" version and somehow overlooked changing it when the app was put in the Lunatics download tree. Not a big deal - and should be easy enough to fix.

Meanwhile, I'll keep chugging away with the MB v8 stock app.

Gene;
ID: 1756771 · Report as offensive
Gene Project Donor

Send message
Joined: 26 Apr 99
Posts: 150
Credit: 48,393,279
RAC: 118
United States
Message 1756801 - Posted: 16 Jan 2016, 7:06:05 UTC - in response to Message 1756771.  

O.K., here's the result with the Lunatics 8.04r3306_avx_Linux64 benchmark:
yes, the same <setiathome_version>7.99</setiathome_version> appears at the end of the result.sah file.

Otherwise, "strongly similar" comparison with Q=99.99% and execution time:
Lunatics AVX 7651 seconds --> relative speed 160%
(for comparison with original post, and run with same system conditions).

It is hard for me to find the equivalent "result.sah" file as produced by normal boinc/seti stock applications. The file exists in the "slot/N" directory while the task is running but is apparently deleted very soon after the result is uploaded. It is, in fact, a soft link to a file named like the work unit but with extra stuff appended. I suspect the "version" tag is appended just as the task completes, the </result> tag is written, the file is closed and uploaded and deleted immediately.

Gene;
ID: 1756801 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1756841 - Posted: 16 Jan 2016, 11:09:43 UTC - in response to Message 1756801.  

It is hard for me to find the equivalent "result.sah" file as produced by normal boinc/seti stock applications. The file exists in the "slot/N" directory while the task is running but is apparently deleted very soon after the result is uploaded. It is, in fact, a soft link to a file named like the work unit but with extra stuff appended. I suspect the "version" tag is appended just as the task completes, the </result> tag is written, the file is closed and uploaded and deleted immediately.

Actually, it's very easy. Use the 'Activity' menu in BOINC Manager to (temporarily) "Suspend network activity" when a task is close to completion. You'll find the full result file in the main project directory: you can inspect it or copy it for as long as you wish, before finally allowing it to upload and go to meet its maker.

I've just done that, and it appears I need to get busy with a Hex editor to remove evidence of the Beta heritage of the next generation of v8 applications.
ID: 1756841 · Report as offensive
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 388
Credit: 1,857,738
RAC: 0
Finland
Message 1756913 - Posted: 16 Jan 2016, 18:36:23 UTC - in response to Message 1756722.  

I was thinking about how useful it is as a debugging feature.

Since you are testing development and alpha and beta versions I imagine you might find it useful to know which app version produced any particular result file. Right now you need to check the version from database or email or post-it note or whatever. I think it would have been handy if the "metadata" is included in the result file.

For comparison, here's what Einstein's BRP app adds to the result file:

% User: 0 (Juha)
% Host: 11695455 (abc)
% Date: 2015-03-04T09:13:30+00:00
% Exec: einsteinbinary_BRP4_1.06_arm-unknown-linux-gnueabihf
% ERP git id: 38c433526387ff03e8b6900495de4a17adc82494
% BOINC rev.: ba6ebc657c24a07d8d073972649e0c46a630f5ae


That makes it pretty easy to tell exactly what code produced that result.

Of course I have no idea if you have any use for such feature but if you do then all I'm saying is don't trust the version in the result file even if the file came from an otherwise trustworthy person and app.

Malicious people and what the validator can and can't trust are then different matter.
ID: 1756913 · Report as offensive
Gene Project Donor

Send message
Joined: 26 Apr 99
Posts: 150
Credit: 48,393,279
RAC: 118
United States
Message 1760607 - Posted: 30 Jan 2016, 6:29:04 UTC - in response to Message 1756913.  

All the optimized (Lunatics) CPU versions supported by my AMD FX-4300, i.e. AVX, SSE2, SSSE3, SSE4.1, SSE4.2, tag the "result" file with <setiathome_version>7.99</setiathome_version>. They all seem to be the r3306 code.

All the above MBv8... applications ran o.k. in Benchmark environment.

--off topic-- I can't register as a user at lunatics.kwsn.info . Is it for developers only? Otherwise I would have posted this comment there.

G;
ID: 1760607 · Report as offensive
Urs Echternacht
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 692
Credit: 135,197,781
RAC: 211
Germany
Message 1761153 - Posted: 31 Jan 2016, 23:58:53 UTC - in response to Message 1760607.  

All the optimized (Lunatics) CPU versions supported by my AMD FX-4300, i.e. AVX, SSE2, SSSE3, SSE4.1, SSE4.2, tag the "result" file with <setiathome_version>7.99</setiathome_version>. They all seem to be the r3306 code.
...
G;

Than check lunatics downloads again AVX has been updated lately to r3345
_\|/_
U r s
ID: 1761153 · Report as offensive
JLDun
Volunteer tester
Avatar

Send message
Joined: 21 Apr 06
Posts: 573
Credit: 196,101
RAC: 0
United States
Message 1761202 - Posted: 1 Feb 2016, 3:23:19 UTC

I don't know, for your specific case, but... When the v8 apps started being tested at Beta (with the current version being anywhere btwn 8.01 and a8.05), the very first one to be built was 7.99 and otherwise titled v8.
ID: 1761202 · Report as offensive

Message boards : Number crunching : Lunatics benchmark, 8.04r3306_sse2-linux64


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.