Compiling Applications for Linux

Message boards : Number crunching : Compiling Applications for Linux
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5

AuthorMessage
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1521761 - Posted: 28 May 2014, 4:29:09 UTC - in response to Message 1521709.  

Ok, got all three. Got them unzipped. It's not obious where to go from here.

Some blank directories: APPS, KWSN-Bench-Linux-MBv7_v2.01.08, old, REF_APPS, testData

V7_MBtestWUs1 (V7_MBtestWUs1.7z created this directory and put some wu's in there)
V7_WU_FGset.7z did not create a directory and unzipped in the directory where it was.

Assuming test work units all go in /testWUs? So that's where I put them to clean up the directory.

Two binaries: benchmark and rescmpv5_1

comlineoptions.txt with some very basic instructions and a default "setiathome_7.01_x86_64-pc-linux-gnu -st -verb -nog
(I'm assuming I replace this with the name of my freshly compiled app? Don't know what the switches do)

init_data.xml.template -- I'm assuming I have to make some changes in here?

------------------

It's looking like I place my newly complied app in /APPS?
Then, change comlineoptions.txt to match the name of my app?
Then, is there anything I must change in init_data.xml.template? Other than possibly just renaming it init_data.xml?
Then, ./ one of the binaries by itself? Or with some switches?

Step by step sure would help me here.

First of all you'll want a Reference app, either the i686 or x86_64 app, put it in the REF_APPS folder (just one reference app, the Linux Bench program can't cope with any more, unlike the Windows version),
You'll want to put any apps under test in the APPS folder (You can put a number of them in there to test),
You'll want to some Wu's to test, the quickest and easiest to use are the four PG series of Wu's, PG0009_v7.wu, PG0395_v7.wu, PG0444_v7.wu and PG1327_v7.wu, these are partial Wu's that only take a few minutes to run (unlike the most of the others), put these four in the testWUs folder,
you'll also need the Wisgen Wu's, just copy _WisgenA.wu to the testWUs folder, make a second copy of there, and rename them A1_WisGenA.wu and A2_WisGenA.wu (the objective is to make them alphabetically before the other Wu's, so the Wisgen Wu's are run before the test Wu's),
Now Suspend of Shutdown Boinc, and run (in terminal) benckmark program,
the ref apps will run first (and cached, so you won't need to run them next time), then the apps under test will be run, terminal will exit when finished, results are in the testDATA folder, the text file you need to look for is systemname.testlog.datetime.txt
Restart Boinc, or unsuspend.

Claggy
ID: 1521761 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1521897 - Posted: 28 May 2014, 12:57:25 UTC - in response to Message 1521709.  

comlineoptions.txt with some very basic instructions and a default "setiathome_7.01_x86_64-pc-linux-gnu -st -verb -nog
(I'm assuming I replace this with the name of my freshly compiled app? Don't know what the switches do)

init_data.xml.template -- I'm assuming I have to make some changes in here?

------------------

It's looking like I place my newly complied app in /APPS?
Then, change comlineoptions.txt to match the name of my app?
Then, is there anything I must change in init_data.xml.template? Other than possibly just renaming it init_data.xml?
Then, ./ one of the binaries by itself? Or with some switches?

You don't need to change anything in init_data.xml, you don't need to rename it or do anything to it.

You don't have to put anything in the comlineoptions.txt, you can put the reference app file name in there, and put -verb in there (makes the Stock app produce verbose output), -st is for standalone, -nog is for No Graphics,
for apps under text you'll need to check the readme's for what commandline parameters are suitable for each app.

Claggy
ID: 1521897 · Report as offensive
Wedge009
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 451
Credit: 431,396,357
RAC: 553
Australia
Message 1522133 - Posted: 28 May 2014, 22:21:57 UTC
Last modified: 28 May 2014, 22:22:35 UTC

Sounds like you've done very well.

According to NV's notes, your GT 620 can go up to sm_21 and your GTX 650 Ti can use sm_30 (assuming I'm understanding the notation correctly).

For comparison with a reference, I suppose you could try testing against the old x41g compilation. This is what I'm using at the moment and it's really slow, even when compared with x41g on Windows.
Soli Deo Gloria
ID: 1522133 · Report as offensive
Wedge009
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 451
Credit: 431,396,357
RAC: 553
Australia
Message 1522218 - Posted: 29 May 2014, 1:48:21 UTC

That's encouraging: results are closely similar as well as the performance improving a bit. On a full-size WU, the improvement could be significant.
Soli Deo Gloria
ID: 1522218 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1522219 - Posted: 29 May 2014, 1:48:30 UTC - in response to Message 1522201.  
Last modified: 29 May 2014, 1:54:13 UTC

A little bit of difference. This is what I got on a GT620:


In terms of dealing with [the vagaries of] floating point precision, Those Q's indicate identical results.

2-11% performance based on straight Cuda version swap is something, though you can expect more down the road, since those times would appear to indicate Linux is headed down the heavyweight high latency driver road that MS and Mac have. (inevitable I suppose with all those new features). From there better performance and utilisation requires new tools and techniques.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1522219 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1522825 - Posted: 30 May 2014, 22:41:51 UTC
Last modified: 30 May 2014, 22:49:35 UTC

*should* be OK, and I doubt you'd run into any issues.

Most likely, IMO, the *glitches* are actually the CUda32 one being a bit old & cranky now, e.g. missing its best signals in that first one, because some synchronisation has changed since way back when. If you wanted to be sure, you could run reference CPU app runs on the two problem ones under controlled conditions, though I suspect it'd come up clean.

Speed wise, The low angle ranges are showing the effects measured elsewhere of increased driver latencies in Later Cuda version paths (despite the same code, same driver, same card). The Big K and Maxwell architectures are too big/fast for the small kernels being thrown at them, so there results upwards of 40% underutilisation at times, directly attributable to driver latency. That's the big part of recent research, finding that the whole pulsefinding (dominant in VLAR like those) needs to be transposed, and the newer latency hiding mechanisms need to be marshalled throughout the application.

To perfect that in a general way with so many different architectures now online & coming, is requiring me to build some special tools. More on that when I'm a little further along, as those tools will be cross platform.

Manual targeted (to a specific device + Cuda version) latency hiding approaches are possible, though that's proven a dead end for me, as I need to support every Cuda enabled GPU. Instead I'm building tools to make the apps more self-scaling/optimising (i.e. decide to feed larger chunks of work, and choose and configure appropriate latency hiding mechanisms automatically where needed).
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1522825 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1523008 - Posted: 31 May 2014, 15:05:19 UTC - in response to Message 1522788.  

Excellent experimenting... I may try spinning up an attempt on Gentoo from your series & The Lunatics...


(sorry, looks like bbcode 'code' removes blanks spaces...)


Try the "pre" codes?:

1 space
2  spaces
3   spaces
4    more... ;-)

All from [ pre ] ... [ /pre ]



Happy fast kool crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1523008 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1528082 - Posted: 14 Jun 2014, 20:11:22 UTC - in response to Message 1517927.  

Okey doke, seemed to get 7.28 (stock multibeam) building here (untested binary) Ubuntu 12.04 LTS x64, fully patched.

Biggest issue here was that Ubuntu's repository fftw3 doesn't appear to put fftw libraries in the standard places, so wasn't being picked up by MB 7's .configure script. Rather than dig for it to make links to an old lib, I just built it to the standard lcoation as per fftw docs instead. I did both the double precision and and single float ones just for completion.

Ignoring checking out / extracting Sequence was:

#1) make and install fftw libraries and headers: ( location I used ~/seti/fftw)
./configure
make
sudo make install
make clean
./configure --enable-float --enable-type-prefix
make
sudo make install

Trying to build fftw 3.3.4 libs, but don't understand what the --enable-type-prefix is supposed to do.

Claggy
ID: 1528082 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 783
Credit: 348,560,338
RAC: 223
United Kingdom
Message 1528105 - Posted: 14 Jun 2014, 21:04:17 UTC - in response to Message 1528082.  

make clean
./configure --enable-float --enable-type-prefix
make
sudo make install

Trying to build fftw 3.3.4 libs, but don't understand what the --enable-type-prefix is supposed to do.

Claggy

--enable-type-prefix Adds a `d' or `s' prefix to all installed libraries and header files to indicate the floating-point precision.

http://www.fftw.org/fftw2_doc/fftw_6.html, or perhaps
 ./configure --help

ID: 1528105 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1528107 - Posted: 14 Jun 2014, 21:07:23 UTC - in response to Message 1528082.  
Last modified: 14 Jun 2014, 21:20:46 UTC

Okey doke, seemed to get 7.28 (stock multibeam) building here (untested binary) Ubuntu 12.04 LTS x64, fully patched.

Biggest issue here was that Ubuntu's repository fftw3 doesn't appear to put fftw libraries in the standard places, so wasn't being picked up by MB 7's .configure script. Rather than dig for it to make links to an old lib, I just built it to the standard lcoation as per fftw docs instead. I did both the double precision and and single float ones just for completion.

Ignoring checking out / extracting Sequence was:

#1) make and install fftw libraries and headers: ( location I used ~/seti/fftw)
./configure
make
sudo make install
make clean
./configure --enable-float --enable-type-prefix
make
sudo make install

Trying to build fftw 3.3.4 libs, but don't understand what the --enable-type-prefix is supposed to do.

Claggy


FFTW libraries come in single and double precision flavours. The --enable-float config option is fairly obvious (we want single precision floats). The type prefix relates to the [name clashes that would occur if you need both libraries] . enabling that [for headers and lib naming], and the floats [ultimately] prefixes the calls with fftwf_ internally (for these single float versions), which will be the format multibeam will be expecting. Without [both] those I'd expect link time issues.

[Edit:] note to check the multibeam generated makefiles too, Eric may or may noy still be using the headers and libs the old way (without explicit prefix, but specifying float, not sure what's current with the Android messing around going on)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1528107 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1528126 - Posted: 14 Jun 2014, 22:39:59 UTC - in response to Message 1528105.  
Last modified: 14 Jun 2014, 22:50:30 UTC

make clean
./configure --enable-float --enable-type-prefix
make
sudo make install

Trying to build fftw 3.3.4 libs, but don't understand what the --enable-type-prefix is supposed to do.

Claggy

--enable-type-prefix Adds a `d' or `s' prefix to all installed libraries and header files to indicate the floating-point precision.

http://www.fftw.org/fftw2_doc/fftw_6.html, or perhaps
 ./configure --help


I got the following error when i tried that option:

configure: WARNING: unrecognized options: --enable-type-prefix

and the first thing i did was ./configure --help

There was no mention of --enable-type-prefix

and there is no mention of it here:

http://fftw.org/fftw3_doc/Installation-on-Unix.html#Installation-on-Unix

Claggy
ID: 1528126 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1528192 - Posted: 15 Jun 2014, 6:41:07 UTC - in response to Message 1528126.  
Last modified: 15 Jun 2014, 7:07:44 UTC

I got the following error when i tried that option:

configure: WARNING: unrecognized options: --enable-type-prefix

and the first thing i did was ./configure --help

There was no mention of --enable-type-prefix

and there is no mention of it here:

http://fftw.org/fftw3_doc/Installation-on-Unix.html#Installation-on-Unix

Claggy


probably removed in that version of fftw then :). I think I did 3.3.2 (will check a bit later)

[edit:] Nope, I did 3.3.4, but it's running configure and ignoring that option anyway, with a warning [i.e. not an error]. You can leave it out, as it's obviously from old versions.

[edit2:] you'll probably want to enable the vector instructions for whatever you're compiling for as well , e.g. --enable-sse --enable sse2 --enable-avx , anyway checking the generated wisdom on a multibeam build run will tell you what fftw kernels it's used to check you get what you want. 64 bit x86 should imply SSE2+ already, but never know what matteo's done unless you look.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1528192 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1532028 - Posted: 25 Jun 2014, 21:31:26 UTC - in response to Message 1528192.  

I did manage to build and install fftw 3.3.4, and manage to build a 7.28 app with fftw 3.3.4, it was a little faster than the last build with fftw 3.3,
But still slower than the Stock 7.01 app on the PG0444_v7.wu:

KWSN-Linux-MBbench v2.1.08
Running on stephen-SR700 at Sun 15 Jun 2014 14:34:27 UTC
----------------------------------------------------------------
Starting benchmark run...
----------------------------------------------------------------
Listing wu-file(s) in /testWUs :
0001_WisGenA.wu
0002_WisGenA.wu
PG0009_v7.wu
PG0395_v7.wu
PG0444_v7.wuPG0444_v7.wu
PG1327_v7.wu
refquick_v7.wu

Listing executable(s) in /APPS :
setiathome-7.28.x86_64-pc-linux-gnu

Listing executable in /REF_APPS :
setiathome_7.01_x86_64-pc-linux-gnu
----------------------------------------------------------------
Current WU: 0001_WisGenA.wu

----------------------------------------------------------------
Running default app with command :... setiathome_7.01_x86_64-pc-linux-gnu -st -verb -nog
./setiathome_7.01_x86_64-pc-linux-gnu -st -verb -nog 23.69 sec 19.30 sec 1.92 sec
Elapsed Time: ....................... 24 seconds

----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 25.67 sec 20.27 sec 2.60 sec
Elapsed Time : ...................... 26 seconds
Speed compared to default : ......... 92 %
-----------------
Comparing results
Result : Strongly similar, Q= 100.0%

----------------------------------------------------------------
Done with 0001_WisGenA.wu

====================================================================
Current WU: 0002_WisGenA.wu

----------------------------------------------------------------
Running default app with command :... setiathome_7.01_x86_64-pc-linux-gnu -st -verb -nog
./setiathome_7.01_x86_64-pc-linux-gnu -st -verb -nog 24.36 sec 19.68 sec 2.34 sec
Elapsed Time: ....................... 24 seconds

----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 24.91 sec 20.11 sec 2.69 sec
Elapsed Time : ...................... 25 seconds
Speed compared to default : ......... 96 %
-----------------
Comparing results
Result : Strongly similar, Q= 99.99%

----------------------------------------------------------------
Done with 0002_WisGenA.wu

====================================================================
Current WU: PG0009_v7.wu

----------------------------------------------------------------
Skipping default app setiathome_7.01_x86_64-pc-linux-gnu, displaying saved result(s)
Elapsed Time: ....................... 852 seconds
----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 823.58 sec 818.04 sec 2.91 sec
Elapsed Time : ...................... 824 seconds
Speed compared to default : ......... 103 %
-----------------
Comparing results
Result : Strongly similar, Q= 100.0%

----------------------------------------------------------------
Done with PG0009_v7.wu

====================================================================
Current WU: PG0395_v7.wu

----------------------------------------------------------------
Skipping default app setiathome_7.01_x86_64-pc-linux-gnu, displaying saved result(s)
Elapsed Time: ....................... 918 seconds
----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 873.07 sec 867.72 sec 2.66 sec
Elapsed Time : ...................... 873 seconds
Speed compared to default : ......... 105 %PG0444_v7.wu
-----------------
Comparing results
Result : Strongly similar, Q= 100.0%

----------------------------------------------------------------
Done with PG0395_v7.wu

====================================================================
Current WU: PG0444_v7.wu

----------------------------------------------------------------
Skipping default app setiathome_7.01_x86_64-pc-linux-gnu, displaying saved result(s)
Elapsed Time: ....................... 817 seconds
----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 833.72 sec 828.47 sec 2.61 sec
Elapsed Time : ...................... 834 seconds
Speed compared to default : ......... 97 %
-----------------
Comparing results
Result : Strongly similar, Q= 99.98%

----------------------------------------------------------------
Done with PG0444_v7.wu

====================================================================
Current WU: PG1327_v7.wu

----------------------------------------------------------------
Skipping default app setiathome_7.01_x86_64-pc-linux-gnu, displaying saved result(s)
Elapsed Time: ....................... 628 seconds
----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 616.19 sec 609.17 sec 4.16 sec
Elapsed Time : ...................... 616 seconds
Speed compared to default : ......... 101 %
-----------------
Comparing results
Result : Strongly similar, Q= 100.0%

----------------------------------------------------------------
Done with PG1327_v7.wu

====================================================================
Current WU: refquick_v7.wu

----------------------------------------------------------------
Skipping default app setiathome_7.01_x86_64-pc-linux-gnu, displaying saved result(s)
Elapsed Time: ....................... 167 seconds
----------------------------------------------------------------
Running app with command : .......... setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose
./setiathome-7.28.x86_64-pc-linux-gnu -standalone -verbose 173.92 sec 169.23 sec 2.56 sec
Elapsed Time : ...................... 174 seconds
Speed compared to default : ......... 95 %
-----------------
Comparing results
Result : Strongly similar, Q= 99.97%

----------------------------------------------------------------
Done with refquick_v7.wu

====================================================================
Hosts CPU data ...
model name : Intel(R) Core(TM)2 Duo CPU T8100 @ 2.10GHz
cpu cores : 2
cpu MHz : 2101.000
cache size : 3072 KB
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida dtherm tpr_shadow vnmi flexpriority

Done with Benchmark run! Removing temporary files!

Claggy
ID: 1532028 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5

Message boards : Number crunching : Compiling Applications for Linux


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.