Taking over SETI rig management for my dad, settings and configurations?

Message boards : Number crunching : Taking over SETI rig management for my dad, settings and configurations?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

AuthorMessage
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1918311 - Posted: 11 Feb 2018, 19:44:50 UTC - in response to Message 1918309.  

Can only speak for myself. I go for high end GPUs as the time to complete is much faster than a 750Ti.

Did I have a system with risers? Yup. I start off with some 650 and work my way up. At one point I had 4 750Ti in a case with Risers. Then I started to upgrade the cards. Once I hit the 780Tis, I found the cards weren't working with the Risers anymore. Tried different configurations. Eventually made the decision to just go for a extended motherboard with high end CPUs and lots of lanes. Stuck in a huge PSU and there you go.


high end cards is fine. my inquiry is more about people seeming to be adverse to high GPU count, regardless of GPU used. your setup seems to be in the minority at 4 cards. I expect that's partially due to the cost involved with buying a board that supports 4x GPUs. that board alone is like $500.

if seti truly doesn't see much performance hit running on 1x links, you can take a board like whats in my miner

board - $100
risers - $60
m.2 pcie adapters - $13

and be up and running with 7-8 GPUs at a cheaper upfront cost and lower power consumption. though, density wont be there unless you run a rig like mine. 8 GPUs wouldn't fit in a normal desktop type case.

Think the issue with the early versions of risers is that the additional power connections for the higher wattage cards wasn't thought out or engineered very well. Seems the miners came up with adequate hardware and power solutions to use lots of X1 risers. The motherboard manufacturers are releasing specifically designed mining motherboards with dozens of X1 slots right on the motherboard ready for X1 riser cables to be plugged in.


honestly, on the mining front, you don't even need a board with lots of PCIe slots. my first iteration of my mining rig had a mATX board that only 3 PCIe sots. i used some 4-in-1 PCIe expenders to plug multiple cards into a single slot. the bandwidth requirements for mining are incredibly low so it doesnt matter. i had 7x GPUs mining in this configuation. that was even cheaper to setup because the motherboard and processsor were old gen stuff.



i just couldn't get 8 gpus working because the board wouldnt post with more than 7 attached. likely due to memory allocation problems which only newer gen boards are equipped to handle.

but yes, riser technology has improved a lot since 3-4 years ago. everyone uses the risers that have a small PCB attached to the 16x connector of the card, slot power provided by a 6-pin PCIe, SATA, or MOLEX connectors, then data transferred over a USB 3.0 cable (which is really only carrying the PCIe data, no power). they are fairly robust at this point. no more ribbon cable risers.

another thought:

can there be a system that is "too fast"? maybe SETI only allocates a max number of WUs to each system and the extra power would somehow be wasted if say a system had to sit idle for half the day waiting for the next day's work? i thought i read something to that effect.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1918311 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1918326 - Posted: 11 Feb 2018, 20:49:04 UTC - in response to Message 1918311.  

can there be a system that is "too fast"? maybe SETI only allocates a max number of WUs to each system and the extra power would somehow be wasted if say a system had to sit idle for half the day waiting for the next day's work? i thought i read something to that effect.


Petri can answer that one all too well. The server is unable to meet the demand of his system so he has to fool it into thinking he has more GPUs than he actually does. That way he can keep a constant supply of work to feed his machine.

For the rest of us, it's usually server day when we run out of work and don't have a big enough cache to ride it out.

The bandwidth required by different projects necessitated the move to larger boards with more bandwidth. Einstein at home is a perfect example, GPUGrid is another. Their work units are heavily dependent on BUS speed and volume to get the data crunched. Depending on PCIe speed will significantly impact the time it takes to finish 1 work unit. That is why some of us moved to larger boards. Since some of us migrate between projects, we need machines that will function exceptional across all projects. If I only wanted to crunch Seti, then I could go it with lesser GPUs and cheaper Mobo. That is mostly due to the exceptional coding done by the Lunatics crew that wrote almost all of Seti's current applications. They have tailored it to such a point it is very efficient when it comes to bandwidth usage.
ID: 1918326 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1918336 - Posted: 11 Feb 2018, 21:17:09 UTC - in response to Message 1918326.  

can there be a system that is "too fast"? maybe SETI only allocates a max number of WUs to each system and the extra power would somehow be wasted if say a system had to sit idle for half the day waiting for the next day's work? i thought i read something to that effect.


Petri can answer that one all too well. The server is unable to meet the demand of his system so he has to fool it into thinking he has more GPUs than he actually does. That way he can keep a constant supply of work to feed his machine.

For the rest of us, it's usually server day when we run out of work and don't have a big enough cache to ride it out.

The bandwidth required by different projects necessitated the move to larger boards with more bandwidth. Einstein at home is a perfect example, GPUGrid is another. Their work units are heavily dependent on BUS speed and volume to get the data crunched. Depending on PCIe speed will significantly impact the time it takes to finish 1 work unit. That is why some of us moved to larger boards. Since some of us migrate between projects, we need machines that will function exceptional across all projects. If I only wanted to crunch Seti, then I could go it with lesser GPUs and cheaper Mobo. That is mostly due to the exceptional coding done by the Lunatics crew that wrote almost all of Seti's current applications. They have tailored it to such a point it is very efficient when it comes to bandwidth usage.


ah, makes a lot of sense. i didn't think about other projects.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1918336 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1918403 - Posted: 12 Feb 2018, 0:39:00 UTC - in response to Message 1918156.  
Last modified: 12 Feb 2018, 0:41:50 UTC

Nice rack! You should compare your rack to Al's or RuieKe's.
Ya ya ya... I'll take your rack, and raise you 2 Dewalts! lol

ID: 1918403 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919060 - Posted: 15 Feb 2018, 20:02:14 UTC

woo, cracked 100,000 RAC.

looking at credit earned the past few days, looks like I'll level off around 140-150k RAC, and a short term goal to hit 100,000,000 pts in 6mo!
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919060 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1919075 - Posted: 15 Feb 2018, 21:31:48 UTC - in response to Message 1919060.  

If you go to your BoincStats account Detailed Stats page, it has two calculators that allow you to input where you will be on a date or put in a desired credit target and it will calculate when you will hit it in the future. Would allow you to pinpoint when you will crack the 100MM mark.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1919075 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1919078 - Posted: 15 Feb 2018, 21:52:53 UTC

I was having a look at your GPU results and wondering why your completion times were so atrocious and then I came across,
i did create cc_config and app_config XML files with info from some random threads i was able to find when i wanted to change Summit2 and Summit3 to use 2 WU per GPU.
, which explains those times. Highend GPU's can get away with those settings but yours can't and you'll get better results returning to 1 w/u per GPU.

Cheers.
ID: 1919078 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919150 - Posted: 16 Feb 2018, 2:45:28 UTC

hmm interesting. reading old posts seemed to indicate that the 760 and 750ti should handle 2x WU just fine?

currently all 7xx cards are running 2x WUs and the 1050tis are actually running 3x.

i gauged this based on GPU load as reported by GPUz. running 1x work unit on the 700 cards ran 50-60% load. so i figured 2x was a better fit (bumped it to 99-100% on 2x). and with 2x on the 1050ti, i was seeing 67% load, so i threw on 3x to get it to max out.

do you think i should lower the 1050tis to 2x? or its ok crunching 3x?
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919150 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1919162 - Posted: 16 Feb 2018, 3:32:38 UTC - in response to Message 1919150.  
Last modified: 16 Feb 2018, 3:41:19 UTC

hmm interesting. reading old posts seemed to indicate that the 760 and 750ti should handle 2x WU just fine?
i gauged this based on GPU load as reported by GPUz. running 1x work unit on the 700 cards ran 50-60% load. so i figured 2x was a better fit (bumped it to 99-100% on 2x). and with 2x on the 1050ti, i was seeing 67% load, so i threw on 3x to get it to max out.

What is important is how many WUs you process per hour.
% GPU use, power use, memory use, bus use etc are all irrelevant- what is the point of having those at their maximum value, if you are only doing a 1/4 of the work you could be doing???

For SoG, unless you are running highend hardware, 1 GPU WU at a time will result in the most work being done per hour, day, week, month, year, etc.
Making use of command line values, and reserving a CPU core for each GPU WU being processed will furtherincrease the amount of work done per hour (as well as GPU usage, power usage etc).
If running the LINUX Special CUDA application, letting it make use of it's auto tune & running only 1 WU at a time- regardless of the GPU hardware- is best.
If running the older stock CUDA applications, 2 WU at a time is the maximum, unless running highend hardware, then 3 at a time may (or may not) give more work per hour.

Edit
Do you think i should lower the 1050tis to 2x? or its ok crunching 3x?

If you are going to stick with CUDA, no more than 2.
If you want to process as much work as possible, re-run the Lunatics installer & select SoG for Nvidia, run 1 WU at a time & make use of command line values.
Grant
Darwin NT
ID: 1919162 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919167 - Posted: 16 Feb 2018, 3:41:30 UTC

so for the recent tasks across all computers, i'll try to post here how long it takes each WU on the different systems. probably easier than trying to flip back and forth in my profile.

Sierra Summit:
1x GTX 760
2x WUs per card
Linux, SoG
avg run time of GPU WU = ~1450s

Sierra Spare:
2x GTX 750ti
2x WU per card
Linux, SoG
avg run time of GPU WU = ~1650s

Sierra Summit 2:
2x GTX 760
2x WUs per card
Windows 7 64 SP1, Lunatics mod
avg run time of GPU WU = ~2750s

Sierra-Prize2:
2x GTX 760
2x WU per card
Windows 7 64 SP1, Lunatics mod
avg run time of GPU WU = ~2750s

Sierra Summit 3:
2x GTX 1050ti
3x WU per card
Windows 7 64 SP1, Lunatics mod
avg run time of GPU WU = ~3400s

-------------------------------------------------------------------

So one thing is clear, linux, even SoG runs a lot better than even modded Windows, at least for points. the 760s under linux are running 2x faster, though i cant say with certainty that its running the same kind of WUs because the linux machines havent been modded with the same Lunatics stuff and i dont know how to tell what kind of WU they are to compare if its really the same. maybe someone can take a look.

also, with the 1050tis, sure they are running slower than the 760s, but its only running ~24% slower per WU but crunching 50% more WUs, so overall thats ahead right?

but maybe moving all 700 cards to 1x and the 1050 to 2x will actually come out ahead as far as work/credit goes? i'll have to make the change and see.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919167 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1919170 - Posted: 16 Feb 2018, 3:45:09 UTC

hmm interesting. reading old posts seemed to indicate that the 760 and 750ti should handle 2x WU just fine?

currently all 7xx cards are running 2x WUs and the 1050tis are actually running 3x.


Key point. All those old posts were referenced against the old CUDA apps.

The SoG app is another beast altogether. You need to use one full cpu core to support each gpu task. On the lower end hardware, one SoG task will fully load the hardware. Not even the 1050Ti can get away with more than 1 task per card and maintain the best daily throughput.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1919170 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919171 - Posted: 16 Feb 2018, 3:45:26 UTC - in response to Message 1919162.  


For SoG, unless you are running highend hardware, 1 GPU WU at a time will result in the most work being done per hour, day, week, month, year, etc.
Making use of command line values, and reserving a CPU core for each GPU WU being processed will furtherincrease the amount of work done per hour (as well as GPU usage, power usage etc).


were do i set command line arguments in the boinc application? on all machines seti is running via boinc manager, not CLI. ive seen command line values mentioned but where do i actually put them in the application?
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919171 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919172 - Posted: 16 Feb 2018, 3:46:44 UTC - in response to Message 1919170.  

hmm interesting. reading old posts seemed to indicate that the 760 and 750ti should handle 2x WU just fine?

currently all 7xx cards are running 2x WUs and the 1050tis are actually running 3x.


Key point. All those old posts were referenced against the old CUDA apps.

The SoG app is another beast altogether. You need to use one full cpu core to support each gpu task. On the lower end hardware, one SoG task will fully load the hardware. Not even the 1050Ti can get away with more than 1 task per card and maintain the best daily throughput.


i have allocated 1 core/thread per GPU WU on all machines. i did this via the app_config file.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919172 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1919173 - Posted: 16 Feb 2018, 3:48:34 UTC - in response to Message 1919167.  

So one thing is clear, linux, even SoG runs a lot better than even modded Windows, at least for points. the 760s under linux are running 2x faster, though i cant say with certainty that its running the same kind of WUs because the linux machines havent been modded with the same Lunatics stuff and i dont know how to tell what kind of WU they are to compare if its really the same. maybe someone can take a look.

For Windows, run the Lunatics Installer- select AVX for the CPU if it supports it. Select SoG for Nvidia. Make sure only 1 WU at a time is running. Then we can give you some command line values to improve their performance. It doesn't matter how many WUs you run at a time with CUDA- it will always produces less work per hour than SoG running 1 WU at a time.
Even under Linux, SoG outperforms the stock CUDA applications with the right command lines, and running only 1 WU at a time.
If you go the Special application route, the output will be about 2-3 times higher still- all while running 1 WU at a time.
Grant
Darwin NT
ID: 1919173 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1919174 - Posted: 16 Feb 2018, 3:51:54 UTC - in response to Message 1919171.  
Last modified: 16 Feb 2018, 3:56:19 UTC

were do i set command line arguments in the boinc application? on all machines seti is running via boinc manager, not CLI. ive seen command line values mentioned but where do i actually put them in the application?

There is no point in doing so as the application you arte running can't take advantage of them.
You need to re-run the Lunatics installer & select SoG for Nvidia.


Edit- your Linux system appears to have settled on SoG as best
I don't know if the names are the same for Windos & Linux. In Windows in the Project folder
C:\ProgramData\BOINC\projects\setiathome.berkeley.edu
is the file
mb_cmdline_win_x86_SSE3_OpenCL_NV_SoG,txt

These are very aggressive settings, but they worked OK with my GTX 750Tis on a dedicated cruncher.
-tt 1500 -hp -period_iterations_num 1 -high_perf -sbs 2048 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64
Once the file is saved, those values will be used when the next WU starts processing.
Grant
Darwin NT
ID: 1919174 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919176 - Posted: 16 Feb 2018, 3:53:07 UTC - in response to Message 1919173.  

So one thing is clear, linux, even SoG runs a lot better than even modded Windows, at least for points. the 760s under linux are running 2x faster, though i cant say with certainty that its running the same kind of WUs because the linux machines havent been modded with the same Lunatics stuff and i dont know how to tell what kind of WU they are to compare if its really the same. maybe someone can take a look.

For Windows, run the Lunatics Installer- select AVX for the CPU if it supports it. Select SoG for Nvidia. Make sure only 1 WU at a time is running. Then we can give you some command line values to improve their performance. It doesn't matter how many WUs you run at a time with CUDA- it will always produces less work per hour than SoG running 1 WU at a time.
Even under Linux, SoG outperforms the stock CUDA applications with the right command lines, and running only 1 WU at a time.
If you go the Special application route, the output will be about 2-3 times higher still- all while running 1 WU at a time.


wow i had no idea. when running the lunatics installer i chose the highest cuda option(50 i think), as the wording seemed to suggest that was the best way "recommended for GTX6xx and newer" or something like that.

i guess i'll switch the windows machines back to SoG with 1x WU each
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919176 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1919177 - Posted: 16 Feb 2018, 3:54:10 UTC - in response to Message 1919174.  

were do i set command line arguments in the boinc application? on all machines seti is running via boinc manager, not CLI. ive seen command line values mentioned but where do i actually put them in the application?

There is no point in doing so as the application you arte running can't take advantage of them.
You need to re-run the Lunatics installer & select SoG for Nvidia.


understood, i plan to switch it back to SoG, but I still dont know where the option is even located, so when the time comes i'd like to know how to access it.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1919177 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1919178 - Posted: 16 Feb 2018, 3:55:18 UTC - in response to Message 1919172.  

In the same place, app_config.xml. Just add in the recommended command line parameters.

Look at the ReadMe_MultiBeam_OpenCL_NV_SoG.txt in the /docs folder in the Seti project folder that the Lunatics installer created when installed. The document at the end shows the best command line tuning parameters for each class of Nvidia hardware.

Your command line should look like this:

<cmdline>-sbs 192 -spike_fft_thresh 2048 -tune 1 64 1 4</cmdline>


added to the MB section of the app_config file.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1919178 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1919179 - Posted: 16 Feb 2018, 3:58:40 UTC - in response to Message 1919178.  

added to the MB section of the app_config file.

Or use the command line file as I suggested.
Either one will work.
My method, the values will be used when the next WU starts. The other method, you have to exit/restart BOINC, or Options, "Read config files" in the Manager for the settings to take effect.
Grant
Darwin NT
ID: 1919179 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1919180 - Posted: 16 Feb 2018, 3:59:57 UTC - in response to Message 1919177.  

understood, i plan to switch it back to SoG, but I still dont know where the option is even located, so when the time comes i'd like to know how to access it.

From memory it's on the same page as the 3 CUDA applications, you just need to select it.
Grant
Darwin NT
ID: 1919180 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

Message boards : Number crunching : Taking over SETI rig management for my dad, settings and configurations?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.