Suggestions to further improve my performance?

Message boards : Number crunching : Suggestions to further improve my performance?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Dr.Diesel Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 14 May 99
Posts: 41
Credit: 123,695,755
RAC: 139
United States
Message 1898444 - Posted: 1 Nov 2017, 0:13:39 UTC

My main crunch box is named seti, computers are visible. This box runs Linux, has (3) GTX 1060s, a Xeon E3-1270 v3 @ 3.50GHz CPU, and TBars latest "special" app with a SETI v of 7.8.3. Looks like my GPU average WU duration is around ~3.5min, with CPU WUs running much wider duration, hard to specify. Currently it's limited to (3) CPU cores, but that is leftover from the default app when GPUs caused a bit of CPU tax, might be able to dedicate the last core to crunching as well?

One big question, how to cache enough GPU work to sustain the Tuesday outages? The past two weeks I've run out of GPU work in the first ~3 hours of the outage. I've got my settings set to store 9 days of work with an additional 5 days.

Any other suggestions to further optimize my performance? app_config file is stock TBar.

This is a dedicated crunch box, so no need to spare the end user.

Many thanks,
ID: 1898444 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11358
Credit: 29,581,041
RAC: 66
United States
Message 1898449 - Posted: 1 Nov 2017, 0:23:23 UTC - in response to Message 1898444.  

I've run out of GPU work in the first ~3 hours of the outage. I've got my settings set to store 9 days of work with an additional 5 days.

IMO nope, Seti will only f=give you max 100 tasks per GPU.
IIRC I read somewhere in these fora that Pettri has done some trick to fool Boinic into thinking he has many more GPUs so he doesn't run dry.
ID: 1898449 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1898452 - Posted: 1 Nov 2017, 0:48:55 UTC - in response to Message 1898444.  

The only way to get through the outage with the special app is to do some bunkering of tasks before the start by rescheduling. You can bunker up to 1000 tasks and not exceed the BOINC limit.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1898452 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1898512 - Posted: 1 Nov 2017, 13:52:00 UTC - in response to Message 1898452.  

The only way to get through the outage with the special app is to do some bunkering of tasks before the start by rescheduling. You can bunker up to 1000 tasks and not exceed the BOINC limit.


So could you point me to a url/message that describes "bunkering"?

Thanks,
Tom Miller
A proud member of the OFA (Old Farts Association).
ID: 1898512 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1898534 - Posted: 1 Nov 2017, 16:19:11 UTC - in response to Message 1898512.  

You would start with the Rescheduler thread. Variations on Mr. Kevvy's original program by Jimbocous, Laurent and Petri have been created. You need to use Laurent's CPU2GPU script to properly bunker for Tuesday.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1898534 · Report as offensive
Profile Dr.Diesel Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 14 May 99
Posts: 41
Credit: 123,695,755
RAC: 139
United States
Message 1898543 - Posted: 1 Nov 2017, 18:07:57 UTC - in response to Message 1898534.  

Thank you Keith, that is very helpful.
ID: 1898543 · Report as offensive
Profile Dr.Diesel Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 14 May 99
Posts: 41
Credit: 123,695,755
RAC: 139
United States
Message 1904426 - Posted: 2 Dec 2017, 16:22:18 UTC - in response to Message 1898543.  

So I ran across this post:

https://setiathome.berkeley.edu/forum_thread.php?id=82127&postid=1899214

That mentioned tweaking the sbs value to 1024 and got me poking around. I'm running Tbars special app and noticed that I only have a single .txt config file:

ap_cmdline_7.08_x86_64-pc-linux-gnu__opencl_nvidia_100.txt

With the following contents:

-unroll 12 -sbs 256 -ffa_block 3072 -ffa_block_fetch 1536

So I take that (new to this please bare with me) that I'm running no special options except for the Astropulse Nvidia WUs. Is this because Tbar has already built the options into the binaries, or do I have some optimizations still left on the table? Could I benefit from changing the sbs value above or additional config options?

Many thanks for the help.
ID: 1904426 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1904442 - Posted: 2 Dec 2017, 17:19:41 UTC - in response to Message 1904426.  

I'm running Tbars special app and noticed that I only have a single .txt config file:

Tbar's special application uses CUDA. The command line you are using, and values you are using, are for OpenCL applications. Completely different.

Check your directories for a readme file, it should show what command line options there are, and which file to place them in.
As it is, Tbar's application has an auto tune option by default that does a pretty good job of getting the most from supported hardware.
Grant
Darwin NT
ID: 1904442 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1904489 - Posted: 2 Dec 2017, 20:59:39 UTC - in response to Message 1904426.  

If you used TBar's all-in-one installer, he provided the Linux OpenCL AP app and the AP txt config file in the distribution. The default you posted would be a good starting point for the 1060 but you could get more aggressive for the tuning parameters for the 1070. The tuning parameters for the AP OpenCL app is the same for the app in Windows.

I couldn't find any tuning parameters for the CUDA 9.0 app other than the -unroll and -nobs parameters in the included docs. I think more were defined in the CUDA 8.0 app docs. You can use the same parameters that were available in the x41zc CUDA app docs other than process priority control. So -pfp and -pfb are available for modification.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1904489 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1904498 - Posted: 2 Dec 2017, 21:26:45 UTC - in response to Message 1904489.  

If you used TBar's all-in-one installer, he provided the Linux OpenCL AP app and the AP txt config file in the distribution. The default you posted would be a good starting point for the 1060 but you could get more aggressive for the tuning parameters for the 1070. The tuning parameters for the AP OpenCL app is the same for the app in Windows.

Doh!
Missed the AP bit. Ignore my previous post.
Grant
Darwin NT
ID: 1904498 · Report as offensive

Message boards : Number crunching : Suggestions to further improve my performance?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.