New Linux client compiled with Intel C++ compiler?

Message boards : Number crunching : New Linux client compiled with Intel C++ compiler?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 109987 - Posted: 11 May 2005, 4:00:01 UTC
Last modified: 11 May 2005, 4:01:30 UTC

A few weeks ago I saw Metod's page and began using his client and at the same time began to compile by mysalf. His client is the best so far and I cannot beat his one. Just can make the one with same speed.

It's already fast enough (takes about 53% time of the original) but found another way to optimize it called PGO (profile guided optimization)....but it takes time to optimze; it needs test run (called "instrumented execution") of the test binary..

I don't know how fast it will be, but I'll try.....Intel's compiler for Linux is completely free for non-commercial use while the one for WIndows doesn't have option for "non-commercial" use; only for "full licence" or "evaluation for 30 days."

Just a notice...I must be speed-maniac.
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 109987 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21763
Credit: 7,508,002
RAC: 20
United Kingdom
Message 110059 - Posted: 11 May 2005, 10:13:19 UTC - in response to Message 109987.  
Last modified: 11 May 2005, 10:14:13 UTC

<blockquote>A few weeks ago I saw Metod's page and began using his client and at the same time began to compile by mysalf. His client is the best so far and I cannot beat his one. Just can make the one with same speed.

It's already fast enough (takes about 53% time of the original) but found another way to optimize it called PGO (profile guided optimization)....but it takes time to optimze; it needs test run (called "instrumented execution") of the test binary...</blockquote>
TMR, go for it!
(:-))

It will also be interesting to compare against what the linux gcc can achieve for the Intel Pentiums and AMD Athlons.

And thanks for your efforts so far. It has made interesting reading (:-))

Regards,
Martin

(Powered by AMD CPUs!)
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 110059 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 110061 - Posted: 11 May 2005, 10:37:26 UTC
Last modified: 11 May 2005, 10:39:04 UTC

Martin,

Chris Bosshard and myself have been working on ggc compiled Linux clients for AMD processors. So far, using gcc seems to give faster results than icc whereas icc seems to produce faster clients for Intel CPUs. We are currently testing the latest revision as Chris has found some nice extra performace increases over our current versions. You can expect the new improved versions to be available next week maybe after we've finished our final testing :)

Current versions for i686, Athlon XP and Athlon64 Processors are available from my website (Link in my signiture). However, Metod's (and Maverick's) ICC versions are likely to give better results on Intel processors :)

The plan is for Metod and Maverick to continue to develop using icc for Intel processors whist Chris and myself will continue to develop with gcc for AMD processors. There are just so many different options to experiment with to squeeze the last drop of performance out of the client, it's just too much for one person to do alone. Big thanks to everyone that's helped test new clients so far :)

Ned
*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 110061 · Report as offensive
Profile Neil Walker
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 288
Credit: 18,101,056
RAC: 0
United Kingdom
Message 110213 - Posted: 11 May 2005, 18:53:03 UTC - in response to Message 110061.  

<blockquote>Martin,

Current versions for i686, Athlon XP and Athlon64 Processors are available from my website (Link in my signiture). However, Metod's (and Maverick's) ICC versions are likely to give better results on Intel processors :)

Ned</blockquote>

Metod's setiathome-4.07-fftw.i686-p4-linux-gnu is substantially faster on my AMD64. Times for the reference WU:

setiathome-4.07-fftw.i686-p4-linux-gnu 5184 secs
setiathome_4.70.2-x86-64-fftw3-static 6016 secs


Be lucky

Neil



ID: 110213 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21763
Credit: 7,508,002
RAC: 20
United Kingdom
Message 110266 - Posted: 11 May 2005, 23:03:54 UTC - in response to Message 110061.  
Last modified: 11 May 2005, 23:04:31 UTC

<blockquote>Martin,

Chris Bosshard and myself have been working on ggc compiled Linux clients for AMD processors. So far, using gcc seems to give faster results than icc whereas icc seems to produce faster clients for Intel CPUs. We are currently testing the latest revision as Chris has found some nice extra performace increases over our current versions. You can expect the new improved versions to be available next week maybe after we've finished our final testing :)</blockquote>
Yahoo!
And sorry for the wrong impression. I'm already using your improved client for the AthlonXP and it works fine and fast. (:-))

Keep hacking,
Regards,
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 110266 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 110472 - Posted: 12 May 2005, 14:28:49 UTC
Last modified: 12 May 2005, 15:24:03 UTC

I made a PGO enabled binary after two instrumented executions, and found it's only 1 or 2 % faster than the one w/o PGO. So now I am doing another instrumented execution to make it faster...

And I found another way to boost SETI (and other programs also)....compile the kernel with ICC!! It's possible, but so far I can find only kernel 2.6.5 patch. I want 2.6.11 or at least 2.6.10. I am looking for the solution on mailing lists/forums or other means.... But it doesn't look so effective. If it's too complicated, I won't do this. Am I too maniac?

best....

PS: I happened to see "Keep hacking," in Martin's signature, and remembered if one can make a good cookie-stealer on this site you can login as another person; the account id is directly in the cookie, "auth". It looks somewhat insecure..

EDIT: I personally came to think optimizing kernel won't benefit....so I don't think I'll do it.
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 110472 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21763
Credit: 7,508,002
RAC: 20
United Kingdom
Message 110486 - Posted: 12 May 2005, 15:39:34 UTC - in response to Message 110472.  
Last modified: 12 May 2005, 15:40:30 UTC

<blockquote>I made a PGO enabled binary... only 1 or 2 % faster... make it faster...</blockquote>
Interesting, but I would expect you to get diminishing returns to gain any further improvement...

<blockquote>... Am I too maniac?</blockquote>
Yes, but in the right way! (:-))

<blockquote>...[site] "auth". It looks somewhat insecure.</blockquote>
You'd better email the webmaster, and suggest an alternative? Forged postings could get annoying.

<blockquote>EDIT: I personally came to think optimizing kernel won't benefit....so I don't think I'll do it.</blockquote>
Good to not waste your time.

I think better improvements might be gained by improving the library code or use of the library code. Ensure that the debug code is excluded at least!

Good hackings (;-))
Regards,
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 110486 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 110491 - Posted: 12 May 2005, 15:50:09 UTC

<blockquote>EDIT: I personally came to think optimizing kernel won't benefit....so I don't think I'll do it.


Good to not waste your time.</blockquote>

Hey heck, why not build up the entire OS with optimizations...

Just install a system with Gentoo Linux. Where the installation process:
A) downloads source for GCC, and compiles that with your specified optimizations...and then
B) downloads the entire O/S and compiles it with your optimizations ;)

ID: 110491 · Report as offensive
Profile spacemeat
Avatar

Send message
Joined: 4 Oct 99
Posts: 239
Credit: 8,425,288
RAC: 0
United States
Message 110494 - Posted: 12 May 2005, 15:58:08 UTC - in response to Message 110491.  

<blockquote><blockquote>EDIT: I personally came to think optimizing kernel won't benefit....so I don't think I'll do it.


Good to not waste your time.</blockquote>

Hey heck, why not build up the entire OS with optimizations...

Just install a system with Gentoo Linux. Where the installation process:
A) downloads source for GCC, and compiles that with your specified optimizations...and then
B) downloads the entire O/S and compiles it with your optimizations ;)

</blockquote>

i was going to say Maverick was sounding a lot like a gentoo user...
this might also be of use in that case ->
http://gentoo-wiki.com/HOWTO_ICC_and_Portage
ID: 110494 · Report as offensive
Profile Neil Walker
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 288
Credit: 18,101,056
RAC: 0
United Kingdom
Message 110495 - Posted: 12 May 2005, 15:59:07 UTC - in response to Message 110491.  

<blockquote><blockquote>EDIT: I personally came to think optimizing kernel won't benefit....so I don't think I'll do it.


Good to not waste your time.</blockquote>

Hey heck, why not build up the entire OS with optimizations...

Just install a system with Gentoo Linux.
</blockquote>

He he. I'll second that. :) All my systems run on Gentoo. ;)


Be lucky

Neil



ID: 110495 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 110532 - Posted: 12 May 2005, 19:56:12 UTC

My original clients pre-FFTW3 that used the libm maths library showed some benefit when run on an optimised gentoo system as libm would also be optimised, but that advantage is negated when using optimised FFTW3 libs. Personally, I saw no benefit compiling my kernel for athlon-xp over i686.

Ned


*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 110532 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 110589 - Posted: 12 May 2005, 23:04:52 UTC - in response to Message 110494.  
Last modified: 12 May 2005, 23:05:42 UTC

<blockquote>
i was going to say Maverick was sounding a lot like a gentoo user...
this might also be of use in that case ->
http://gentoo-wiki.com/HOWTO_ICC_and_Portage</blockquote>

Thank you for the nice hints, Benher and spacemeat.

I'm a debian user for portability..I don't know much about gentoo.
But building the whole gentoo system with Intel C++ sounds very attractive....My problem is portability and upgrading. Debian has a lot of nice packages.

So I'll install gentoo on vmware, compile whole system with icc and compile SETI on it link statically. Try gentoo for a while on vmware and decide whether I will install gentoo on the base or not. Or is it possible to install icc-based glibc/kernel on debian? If it's possible, it is best.

best regards,

Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 110589 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 110654 - Posted: 13 May 2005, 2:33:26 UTC

PGO optimized client benchmark test has finished (1 vs 1 comparison; run 2 clients at the same time for an accurate result)...but only a small difference has been found; 9101 seconds (no PGO) vs. 9028 seconds (w PGO). PGO is working, but little bit.

I did PGO for all object files in boinc, fftw-3.0.1 and seti_boinc directories, so it won't be better with PGO.

Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 110654 · Report as offensive
Profile Steve Cressman
Volunteer tester
Avatar

Send message
Joined: 6 Jun 02
Posts: 583
Credit: 65,644
RAC: 0
Canada
Message 110684 - Posted: 13 May 2005, 3:16:11 UTC

Here is some info I just found:

Another thing the Intel compiler does work well for AMD too. But you may have to force it to recognize it as AMD to turn on even more optimizations. Apparently you get a boost as a nonIntel CPU, but if you disable the "intel-only" detection [google.ca] you can get even more of a boost.

Not sure if this applies to Linux, Windows or both
98SE XP2500+ @ 2.1 GHz Boinc v5.8.8

And God said"Let there be light."But then the program crashed because he was trying to access the 'light' property of a NULL universe pointer.
ID: 110684 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 110702 - Posted: 13 May 2005, 4:18:32 UTC - in response to Message 110654.  

<blockquote>PGO optimized client benchmark test has finished (1 vs 1 comparison; run 2 clients at the same time for an accurate result)...but only a small difference has been found; 9101 seconds (no PGO) vs. 9028 seconds (w PGO). PGO is working, but little bit.

I did PGO for all object files in boinc, fftw-3.0.1 and seti_boinc directories, so it won't be better with PGO.
</blockquote>

self-reply...but I will continue optimizing :) and found gentoo very useful;there are lots of packages and those packages are always up to date. At first I will try it on vmware and when I am familialized with it, install it on the base.

Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 110702 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 110722 - Posted: 13 May 2005, 5:39:55 UTC
Last modified: 13 May 2005, 5:55:10 UTC

As for AMD, I hope Intel compiler would work for Athlons with proper options (for ex, -xW for Athlon64)..but it won't support 3dnow. I don't have AMD processors and cannot try by myself. Loop vectorization and interprocedural optimization are common in AMD as well as in Intel.


Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 110722 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 111061 - Posted: 14 May 2005, 12:38:48 UTC

I tried gentoo, but found the gnome was a bit buggy. I like the scheme of installation (though it takes a bit time :) ), but as I want apps to run normally (especially desktop usability), I abandoned gentoo and returned to debian :)

I may try sourcemage....masochistic? :) but it looks like it dones't support so many packages.

After all, SETI's speed depends mostly on the SETI client, not on libraries, kernel or distros, because SETI spends most of the time in the SETI client.
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 111061 · Report as offensive
Ned Slider

Send message
Joined: 12 Oct 01
Posts: 668
Credit: 4,375,315
RAC: 0
United Kingdom
Message 111082 - Posted: 14 May 2005, 13:53:38 UTC - in response to Message 111061.  

<blockquote>I tried gentoo, but found the gnome was a bit buggy. I like the scheme of installation (though it takes a bit time :) ), but as I want apps to run normally (especially desktop usability), I abandoned gentoo and returned to debian :)

I may try sourcemage....masochistic? :) but it looks like it dones't support so many packages.

After all, SETI's speed depends mostly on the SETI client, not on libraries, kernel or distros, because SETI spends most of the time in the SETI client.</blockquote>

Yes, the main lib being used was the libm maths lib, but this functionality has been replaced by FFTW3 which can be optimized.

Don't worry too much about AMD - we (Chris and I) have new revisions of our AMD clients coming out next week that are about twice as fast as our current AMD clients. The athlon XP client is about +59% compared to the current client which is about +32%. We found some nice new speed increases that were quite unexpected :)

Ned


*** My Guide to Compiling Optimised BOINC and SETI Clients ***
*** Download Optimised BOINC and SETI Clients for Linux Here ***
ID: 111082 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21763
Credit: 7,508,002
RAC: 20
United Kingdom
Message 111087 - Posted: 14 May 2005, 14:19:25 UTC - in response to Message 111082.  

<blockquote>[...]
Don't worry too much about AMD - we (Chris and I) have new revisions of our AMD clients coming out next week that are about twice as fast as our current AMD clients. The athlon XP client is about +59% compared to the current client which is about +32%. We found some nice new speed increases that were quite unexpected :)

Ned</blockquote>PHEW!
What ya found?
Do tell.

Whatever, it certainly sounds impressive.

Happy FAST crunchin',
Regards,
Martin

(On an AMD AthlonXP Palomino core, hint, hint, (;-))
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 111087 · Report as offensive
Tetsuji Maverick Rai
Volunteer tester
Avatar

Send message
Joined: 25 Apr 99
Posts: 518
Credit: 90,863
RAC: 0
Japan
Message 111117 - Posted: 14 May 2005, 15:40:04 UTC
Last modified: 14 May 2005, 15:48:34 UTC

ohh AMD project (should I call it so?) is making progress :) nice to hear that. it's a BIG progress isn't it? gcc-4.0? If so, it's more generic than Intel and maybe easy to handle. Intel's object file format isn't compatible with gcc by default.

I'll give gentoo a try once more. Apart from SETI, this distro is also as attractive as debian. So I want to investigate more and get accustomed to it. At least many people are using it happily. But during my distro-switching, my server is completely down.....sorry, fortunately another faster server is always running ;)

BTW since I began to use the new Windows SETI client, my credit is climbing very fast. I have 17182 credits (including Einstein and Predictor) since Apr 25 (less than a month). Thanks Ned, Chris and Metod and other guys here and Ned's forum!!

About Windows/Linux clients by intel, I'm willing to answer any questions if possible, but because of my distro switching, my reply may be late. sorry for your inconvenience. (email: maverick31337@grad.com)

BTW Intel also provides GUI based integrated developing environment called eclipse on Linux. Anyone did try it? How useful is it? I'm not sure it's so nice...I prefer command line environment.

regards,

-Mav
Luckiest in the world. WMD = Weapon of Mass Distraction.
Click this table.
ID: 111117 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : New Linux client compiled with Intel C++ compiler?


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.