[LINUX] A Faster SETI app to go with my optimized boinc apps

Message boards : Number crunching : [LINUX] A Faster SETI app to go with my optimized boinc apps
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · Next

AuthorMessage
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 102934 - Posted: 23 Apr 2005, 10:54:00 UTC

The 7.1MB client is from Berkeley, not me. My static client is 2.1MB and my non-static version is about 600K. Yes, I'm sure it's static.

You can check it's truely static by using the ldd command to see what libraries it's linked against:

ldd setiathome_4.7_i686-pc-linux-gnu

If it gives a list of lib files then it's not static.

We've now very nearly finished our final testing - just a few more results to come back, and will be in a position to build our final ultra clients this week :)

From the test results we've had back, it looks like you can expect a 30-35% performance increase over the standard native berkeley client on Athlon XP processors.

Ned

*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 102934 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20518
Credit: 7,508,002
RAC: 20
United Kingdom
Message 102944 - Posted: 23 Apr 2005, 11:30:24 UTC - in response to Message 102934.  
Last modified: 23 Apr 2005, 11:31:50 UTC

> The 7.1MB client is from Berkeley, not me. My static client is 2.1MB and my
> non-static version is about 600K. Yes, I'm sure it's static.

Just checked and sure enough:

$ ldd setiathome_4.7_i686-pc-linux-gnu
not a dynamic executable

$ ldd setiathome_4.02_i686-pc-linux-gnu
linux-gate.so.1 => (0xffffe000)
libz.so.1 => /lib/libz.so.1 (0x40027000)
libnsl.so.1 => /lib/libnsl.so.1 (0x40038000)
libdl.so.2 => /lib/libdl.so.2 (0x4004b000)
libm.so.6 => /lib/tls/libm.so.6 (0x4004e000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x40072000)
libc.so.6 => /lib/tls/libc.so.6 (0x4007b000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

OK, so yours is static (:-))



> From the test results we've had back, it looks like you can expect a 30-35%
> performance increase over the standard native berkeley client on Athlon XP
> processors.

Good work.

For mine at the moment, boinc is still working through WU's downloaded for the 4.02 client. I'll look to see when it picks up your 4.7. I look forward to the uber-optimised FFTW client!

Thanks,
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 102944 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 102948 - Posted: 23 Apr 2005, 11:45:20 UTC

Details of how to get it to compile statically can be found towards the end of this thread:

http://forums.pcper.com/showthread.php?t=385265

Ned

*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 102948 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 103009 - Posted: 23 Apr 2005, 17:21:33 UTC - in response to Message 102948.  
Last modified: 23 Apr 2005, 17:27:03 UTC

> Details of how to get it to compile statically can be found towards the end of
> this thread:
>
> http://forums.pcper.com/showthread.php?t=385265
>
> Ned
>
>

But do you really need statically linked executable?

and yes, Ned's page is very useful for those who want to build their own optimized seti@home client. But it's for athlon-xp...for P4, I used CFLAGS and CXXFLAGS as

export CFLAGS='-march=i686 -mmmx -msse -msse2 -O3 -falign-functions=4 -fprefetch-loop-arrays -fomit-frame-pointer -funroll-loops -fforce-addr -ftracer -ffast-math'
export CXXFLAGS=$CFLAGS

Somehow "-march=pentium4" gave slower results. Furthur discussion should be in Ned's forum.

I got a nice (to me) results with my optimization.....it takes only 82.8% time to crunch the reference_workunit...a bit faster than Windows seti client. Maybe you may be able to make a faster client using FFTW library, which is mentioned in Ned's forum, but I didn't do it because of the complexity :)

So I guess the slowness of Linux seti client is due to inappropriate (or generic) optimization. You may want optimization for your cpu to cruch faster.
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 103009 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 103015 - Posted: 23 Apr 2005, 18:02:09 UTC

I'm guessing that the large majority of the perfomance increase you're seeing is due to -ffast-math.

For Intel processors, you really should try Metod's clients here:

http://mkx.feridot.com/boinc/

They really are a LOT faster than anything you'll easily compile yourself just using gcc.

For Athlon-XP's, use my client for best results.

I'm now looking at doing a generic client for older AMD chips.

Ned

*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 103015 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 103441 - Posted: 24 Apr 2005, 20:40:39 UTC - in response to Message 103015.  

> I'm guessing that the large majority of the perfomance increase you're seeing
> is due to -ffast-math.
>
> For Intel processors, you really should try Metod's clients here:
>
> http://mkx.feridot.com/boinc/
>
> They really are a LOT faster than anything you'll easily compile yourself just
> using gcc.
>
> For Athlon-XP's, use my client for best results.
>
> I'm now looking at doing a generic client for older AMD chips.
>
> Ned
>
>

Thanks. now I'm about to test one for p4, but I wonder it's really version 4.07.....isn't it 4.70? The filename is setiathome-4.07-fftw.i686-p4-linux-gnu
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 103441 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 103455 - Posted: 24 Apr 2005, 21:52:41 UTC - in response to Message 103441.  
Last modified: 24 Apr 2005, 21:53:20 UTC

My new optimized SETI clients are now posted to my website for download (Click the link in my signiture).

I've not had a chance to test them yet so please report if they are broken.

Special thanks to Chris Bosshard for compiling and supplying the x86-64 client (this one has been tested).

And thanks to everyone who's help test and develop these clients!

We are seeing performance increases of about +32-35% for the athlon-xp client and about +40% for the 64-bit client over the native berkeley linux client.

These should be our final releases for the time being (bug fixes only now if anything is broken).

Enjoy,

Ned


>
> Thanks. now I'm about to test one for p4, but I wonder it's really version
> 4.07.....isn't it 4.70? The filename is
> setiathome-4.07-fftw.i686-p4-linux-gnu
>

Yes.
*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 103455 · Report as offensive
Metod, S56RKO
Volunteer tester

Send message
Joined: 27 Sep 02
Posts: 309
Credit: 113,221,277
RAC: 9
Slovenia
Message 103664 - Posted: 25 Apr 2005, 12:33:27 UTC - in response to Message 103441.  
Last modified: 25 Apr 2005, 12:33:51 UTC

> Thanks. now I'm about to test one for p4, but I wonder it's really version
> 4.07.....isn't it 4.70? The filename is
> setiathome-4.07-fftw.i686-p4-linux-gnu

There's some confusion about what the version number should be. Compilation process (configure and make) creates file with 4.7 in its name while binary if run using -version switch says it's 4.07.
Metod ...
ID: 103664 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 103676 - Posted: 25 Apr 2005, 13:46:53 UTC - in response to Message 103664.  
Last modified: 25 Apr 2005, 13:50:57 UTC

> There's some confusion about what the version number should be. Compilation
> process (configure and make) creates file with 4.7 in its name while
> binary if run using -version switch says it's 4.07.
>

yes, that's exactly where I was at a loss. I suspected it should be 4.70 because there is already version 4.09 for Windows. The most recent nightly build makes setiathome-4.7.i686-pc-linux-gnu. I suspect "4.70" should be truncated to 4.7 during compilation.

Now I found the correct source files for fftw3 (mentioned in Chris' page), and I'm trying gcc-3.4 fftw3 vs. intel fftw3 on my P4 :)

I didn't know Intel C++ compiler was free for non-commercial use.


So far....crunching times are:

official 4.02 15915 secs
my gcc-3.4 build (no fftw3) 13415 secs
Chris' intel build (with fftw3) 8538 secs

awesome....
ID: 103676 · Report as offensive
Metod, S56RKO
Volunteer tester

Send message
Joined: 27 Sep 02
Posts: 309
Credit: 113,221,277
RAC: 9
Slovenia
Message 103683 - Posted: 25 Apr 2005, 14:02:31 UTC - in response to Message 103676.  
Last modified: 25 Apr 2005, 14:05:05 UTC

> > There's some confusion about what the version number should be.
> Compilation
> > process (configure and make) creates file with 4.7 in its name
> while
> > binary if run using -version switch says it's 4.07.
> >
>
> yes, that's exactly where I was at a loss. I suspected it should be 4.70
> because there is already version 4.09 for Windows. The most recent nightly
> build makes setiathome-4.7.i686-pc-linux-gnu. I suspect "4.70" should be
> truncated to 4.7 during compilation.

I'm perfectly fine with version being 4.07 since everything done on windows app is connected to display of graphics ... they pulled back the 4.10 version due to bugs and I suspect they'll re-release it when they make it work.

Actually, here's an excerpt from configure script:

echo "--- Configuring SETI_BOINC 4.07 (${configured_to_build}) ---"
major_version=`echo 4.07 | sed 's/..*//'`
minor_version=`echo 4.07 | sed 's/.*.//' | sed 's/^0//'`
MAJOR_VERSION=$major_version

MINOR_VERSION=$minor_version


cat >>confdefs.h <<_ACEOF
#define VERSION_MAJOR $major_version
_ACEOF


cat >>confdefs.h <<_ACEOF
#define VERSION_MINOR $minor_version
_ACEOF


You can see that the leading 0 from minor_version is explicitly ripped off.
Metod ...
ID: 103683 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 103695 - Posted: 25 Apr 2005, 14:47:40 UTC - in response to Message 103455.  
Last modified: 25 Apr 2005, 14:50:07 UTC

> My new optimized SETI clients are now posted to my website for download (Click
> the link in my signiture).
>
> I've not had a chance to test them yet so please report if they are broken.
>
> Special thanks to Chris Bosshard for compiling and supplying the x86-64 client
> (this one has been tested).
>


you really really really need to have the {executable/} statement in the app_info.xml file for the x86_64 client (replace with proper brackets) otherwise people will bork up their entire cache of WU's with -123 errors
ID: 103695 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 103709 - Posted: 25 Apr 2005, 15:35:54 UTC - in response to Message 103683.  

> I'm perfectly fine with version being 4.07 since everything done on windows
> app is connected to display of graphics ... they pulled back the 4.10 version
> due to bugs and I suspect they'll re-release it when they make it work.
>
> Actually, here's an excerpt from configure script:
> .........

thx.

oh...I also found in seti_boinc/config.h

#define PACKAGE_STRING "setiathome 4.07"

regards....
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 103709 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 103744 - Posted: 25 Apr 2005, 17:48:20 UTC
Last modified: 25 Apr 2005, 17:49:30 UTC

Hi, crunchers,

As I wrote here before, I began compiling seti@home and core client with Intel C++ compiler as well as gcc/g++. My executables compiled with gnu compilers work fine, but I have a problem with Intel compiler regardless of fftw3 library....compile went w/o problem and core client works, but the setiathome-4.7.i686-pc-linux-gnu doesn't work....

for further details see http://softwareforums.intel.com/ids/board/message?board.id=16&message.id=2604 in Intel's forum (my nickname is "maverick6664").

Thanks in advance. I will send email to Chris....

Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 103744 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 103787 - Posted: 25 Apr 2005, 20:11:17 UTC - in response to Message 103695.  

>
> you really really really need to have the {executable/} statement in the
> app_info.xml file for the x86_64 client (replace with proper brackets)
> otherwise people will bork up their entire cache of WU's with -123 errors
>

spacemeat,

I'm not sure I understand?

If you could explain, or e-mail me (nedslider[at]f2s[dot]com) an example app_info.xml showing what you mean, I'd be happy to impliment.

I've not tried an in situ install, only a fresh install of this client so was not aware of this. Thanks for bringing it up.

Ned

*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 103787 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 103788 - Posted: 25 Apr 2005, 20:13:12 UTC

We now have some data in on the generic i686 client; it's showing about +31% performance increases on both AMD and Intel processors.

Ned

*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 103788 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 103803 - Posted: 25 Apr 2005, 20:56:25 UTC - in response to Message 103787.  
Last modified: 25 Apr 2005, 21:03:06 UTC

> >
> > you really really really need to have the {executable/} statement in the
> > app_info.xml file for the x86_64 client (replace with proper brackets)
> > otherwise people will bork up their entire cache of WU's with -123
> errors
> >
>
> spacemeat,
>
> I'm not sure I understand?
>
> If you could explain, or e-mail me (nedslider[at]f2s[dot]com) an example
> app_info.xml showing what you mean, I'd be happy to impliment.
>
> I've not tried an in situ install, only a fresh install of this client so was
> not aware of this. Thanks for bringing it up.
>
> Ned
>
>

all it means is the app_info.xml is missing a line. line 7 (the "executable/" tag) from the app_info.xml skeleton in
http://boinc.berkeley.edu/anonymous_platform.php
is omitted from the tarball copy.
im not sure if this is just in the x86_64 download or if others have similar copies.

if boinc doesn't treat the seti client as executable, you will get client errors returned for every cached WU left in the setiathome directory in about 5 seconds flat as i discovered.

i'm going to try your i686 client later as i have some old pentiums that need a boost. thanks for putting these together
ID: 103803 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 103864 - Posted: 26 Apr 2005, 0:11:03 UTC - in response to Message 103803.  


> all it means is the app_info.xml is missing a line. line 7 (the "executable/"
> tag) from the app_info.xml skeleton in
> http://boinc.berkeley.edu/anonymous_platform.php
> is omitted from the tarball copy.
> im not sure if this is just in the x86_64 download or if others have similar
> copies.
>
> if boinc doesn't treat the seti client as executable, you will get client
> errors returned for every cached WU left in the setiathome directory in about
> 5 seconds flat as i discovered.
>
> i'm going to try your i686 client later as i have some old pentiums that need
> a boost. thanks for putting these together
>

Thank you. All three fixed and new tarballs uploaded :)
*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 103864 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 103865 - Posted: 26 Apr 2005, 0:14:11 UTC - in response to Message 103803.  

>
> i'm going to try your i686 client later as i have some old pentiums that need
> a boost. thanks for putting these together
>

As I mentioned before, for Intel processors you may like to also check out Metod's excellent clients here:

http://mkx.feridot.com/boinc/

His clients appear to be faster than mine on INTEL processors, although mine should also give a good performance boost. Mine should be faster on AMD processors.

Or try both with the reference client and see which performs best for you :)

Ned

*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 103865 · Report as offensive
Metod, S56RKO
Volunteer tester

Send message
Joined: 27 Sep 02
Posts: 309
Credit: 113,221,277
RAC: 9
Slovenia
Message 103972 - Posted: 26 Apr 2005, 6:30:19 UTC - in response to Message 103744.  

> As I wrote here before, I began compiling seti@home and core client with Intel
> C++ compiler as well as gcc/g++. My executables compiled with gnu compilers
> work fine, but I have a problem with Intel compiler regardless of fftw3
> library....compile went w/o problem and core client works, but the
> setiathome-4.7.i686-pc-linux-gnu doesn't work....

I can't reply in that forum, so I'll do it here ...

Occasionally I ran into the same problem. A cure (for me) was simply to delete the final binary and redo the linking. And no, you don't have to disable IPO in the linking stage. In my understanding, IPO only works in final (linking) stage,
it's running between files.

If you're not into compiling seti crunchers for fun, you might want to get my executables here.
Metod ...
ID: 103972 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 104017 - Posted: 26 Apr 2005, 9:59:50 UTC - in response to Message 103972.  
Last modified: 26 Apr 2005, 10:06:32 UTC

> I can't reply in that forum, so I'll do it here ...
>
> Occasionally I ran into the same problem. A cure (for me) was simply to delete
> the final binary and redo the linking. And no, you don't have to disable IPO
> in the linking stage. In my understanding, IPO only works in final (linking)
> stage,
> it's running between files.
>

Thanks..but I still cannot get it to work...same problem...if you have spare time, will you test the executable seti_boinc (3947291 bytes)?
This has a weird problem:
----------------
maverick:~/tmp% ./seti_boinc
zsh: no such file or directory: ./seti_boinc
maverick:~/tmp% file seti*
seti_boinc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), not stripped
maverick:~/tmp% ldd seti_boinc
/usr/bin/ldd: line 1: ./seti_boinc: No such file or directory
----------------
The first and third error messages show seti_boinc is a shell script, while the second shows it's a binary.


Intel C++ compiler takes time, but it seems to optimize a LOT...(or it is just complaining many things?)

> If you're not into compiling seti crunchers for fun, you might want to get my
> executables here.
>

Yes, I've read it began this by myself :) So building it with Intel compiler is just for fun to me... As a matter of fact, I have been using that binary since yesterday and getting nice results.
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 104017 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · Next

Message boards : Number crunching : [LINUX] A Faster SETI app to go with my optimized boinc apps


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.