How to use multiple GPU's

Questions and Answers : GPU applications : How to use multiple GPU's
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Profile Bravo6 Project Donor
Avatar

Send message
Joined: 28 Oct 99
Posts: 41
Credit: 11,740,506
RAC: 41,934
United States
Message 1953751 - Posted: 5 Sep 2018, 1:35:22 UTC - in response to Message 1953516.  

I think I have covered most of the bases, thinking about this project, but one last question popped up before starting. Given the example of the gtx 1050 series, is it sufficient to match model and and memory size, and what would be the effect of straying too far in matching the cards in a system?
Thanx
"Don't worry about it, nothing is gonna be O.K. anyway.........."
ID: 1953751 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 1327
Credit: 30,352,983
RAC: 142,573
United States
Message 1953797 - Posted: 5 Sep 2018, 5:00:26 UTC - in response to Message 1953751.  

I think I have covered most of the bases, thinking about this project, but one last question popped up before starting. Given the example of the gtx 1050 series, is it sufficient to match model and and memory size, and what would be the effect of straying too far in matching the cards in a system?
Thanx


One effect of having different cards in one system is it is harder to run the higher end cards at full speed because unless you can do some fancy tinkering (that I don't know how to do) you will end up using the same command line on all cards of the same brand (eg. NVidia and AMD). And if you can run 3 tasks on say your Gtx 1080Ti but you also have a Gtx 1050Ti that you can't run three tasks on, you would have to see if you could figure out a way to give each card a different "app_config.xml" parameter. Which is something else I don't know how to do.

So in general, running multiples of the exact same card even if they are different brands is easier (at least for me).

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1953797 · Report as offensive     Reply Quote
Profile Ageless
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 14568
Credit: 3,723,160
RAC: 968
Netherlands
Message 1953828 - Posted: 5 Sep 2018, 12:38:36 UTC - in response to Message 1953797.  

...you would have to see if you could figure out a way to give each card a different "app_config.xml" parameter. Which is something else I don't know how to do.
Run multiple clients. But that's Astrophysics level BOINC.
Jord

Ancient Astronaut Theorists can tell you that I do not help with tech questions via private message. Please use the forums for that.
ID: 1953828 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 1327
Credit: 30,352,983
RAC: 142,573
United States
Message 1953855 - Posted: 5 Sep 2018, 15:14:16 UTC - in response to Message 1953828.  

...you would have to see if you could figure out a way to give each card a different "app_config.xml" parameter. Which is something else I don't know how to do.
Run multiple clients. But that's Astrophysics level BOINC.



Thank you for the reminder. I had forgotten.

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1953855 · Report as offensive     Reply Quote
Profile Bravo6 Project Donor
Avatar

Send message
Joined: 28 Oct 99
Posts: 41
Credit: 11,740,506
RAC: 41,934
United States
Message 1953960 - Posted: 6 Sep 2018, 1:49:52 UTC - in response to Message 1953855.  

Ok makes sense. While I used to install and manage multiple server level systems, it is amazing to me the change/advances since the" dark ages " (for me that's 2005). I am afraid my level of expertise has degenerated to "not too dumb, considering". Now we will see............
Thanx.....
😨
"Don't worry about it, nothing is gonna be O.K. anyway.........."
ID: 1953960 · Report as offensive     Reply Quote
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16479
Credit: 340,358,761
RAC: 225,349
United Kingdom
Message 1954077 - Posted: 6 Sep 2018, 16:19:41 UTC

A couple of tips if you are contemplating running mixed AMD & nVidia GPUS - get each type stable and working independently and save the configuration files before trying to get the other one working. For each type start with one GPU the build up to the "final fleet" size". Then when you start to think seriously about the mixed environment start with a small count and build up.
From memory there is flag in BOINC that needs to be set to allow multiple instances to run, which you will need to do for a successful mixed fleet. There has been some discussion in the "Number Crunchers" forum inside SETI (https://setiathome.berkeley.edu/forum_forum.php?id=10) that covers this sort of activity.
Take your time to get each step right before moving on to the next, and keep copious notes of what you've done so you can roll back when you get things wrong or want to try something slightly different.

Good Luck
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1954077 · Report as offensive     Reply Quote
Profile Bravo6 Project Donor
Avatar

Send message
Joined: 28 Oct 99
Posts: 41
Credit: 11,740,506
RAC: 41,934
United States
Message 1954130 - Posted: 6 Sep 2018, 22:44:08 UTC - in response to Message 1954077.  

I wasn't thinking so much of different cards so much as different versions of same card. For example the gtx 1050s, and 1050to with different men size, different DirectX , versions and the other little things different for production run to production run. I Am going to start with two Hewlett packard refurbs, one I3 and one I5 and use 1050s in one and 1050ti in the other. ( i have access to several of each). Just waiting on delivery from #$%&%$ china. Did not realize that was where stuff i ordered was going to come from. I like newegg , but ya gotta read the source print........
😒
"Don't worry about it, nothing is gonna be O.K. anyway.........."
ID: 1954130 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 1327
Credit: 30,352,983
RAC: 142,573
United States
Message 1954142 - Posted: 7 Sep 2018, 0:17:52 UTC - in response to Message 1954130.  

I wasn't thinking so much of different cards so much as different versions of same card. For example the gtx 1050s, and 1050to with different men size, different DirectX , versions and the other little things different for production run to production run. I Am going to start with two Hewlett packard refurbs, one I3 and one I5 and use 1050s in one and 1050ti in the other. ( i have access to several of each). Just waiting on delivery from #$%&%$ china. Did not realize that was where stuff i ordered was going to come from. I like newegg , but ya gotta read the source print........
😒


My experience with various brands of the same models of Nvidia (mostly Gtx 750Ti's and 3 Gtx 1060 3GB) is that you can use the same parameters for the same models even if the performance is different. I have had two 750's and two 1060's running under Windows where I could use the GPU-Z utility on them. Inspite of being the same model cards there was differences in speed and some differences in productivity.

My Gtx 750Ti's have (I think) 2 GB of memory. And the 1050's apparently start at 2GB and go up.

I hope you ordered "brand name" cards (eg. ASUS, EGA etc.) not just "generic" Nvidia cards. My experience with "generic" Nvidia cards even from the US is they are not the actual product you thought you were getting.

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1954142 · Report as offensive     Reply Quote
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 4734
Credit: 302,576,472
RAC: 319,230
United States
Message 1954193 - Posted: 7 Sep 2018, 3:56:15 UTC - in response to Message 1954130.  

I wasn't thinking so much of different cards so much as different versions of same card. For example the gtx 1050s, and 1050to with different men size, different DirectX , versions and the other little things different for production run to production run. I Am going to start with two Hewlett packard refurbs, one I3 and one I5 and use 1050s in one and 1050ti in the other. ( i have access to several of each). Just waiting on delivery from #$%&%$ china. Did not realize that was where stuff i ordered was going to come from. I like newegg , but ya gotta read the source print........
😒



You are always going to be limited by the lowest card. That mean the one that is the slowest or has the least amount of Ram. To that end, your commandline and setting will always be dictated by that card. Example. If you had a 1050 and a 1070, you would be restricted by the 1050. If you had 2 1050s but one had 2 GB of Memory and the other 6GB, you will not be able to craft your commandline to take advantage of the extra memory. So it's best to always have similar cards. If you chose to use different series of cards, like a 1050 and 1070, then you will be forced to use a cc_config.xml to force bonic to use both cards. Otherwise it will ignore the 1050 and only use the 1070.

Lastly, there is a limit on how much RAM of a card you can use for OpenCl. Not such a problem with cuda but depending on which application you choose to run, it will also limit how you can tailor your system.
ID: 1954193 · Report as offensive     Reply Quote
Profile Bravo6 Project Donor
Avatar

Send message
Joined: 28 Oct 99
Posts: 41
Credit: 11,740,506
RAC: 41,934
United States
Message 1954537 - Posted: 9 Sep 2018, 16:46:53 UTC - in response to Message 1954193.  
Last modified: 9 Sep 2018, 16:55:51 UTC

ok added a gtx 1050ti to one on my newer refurbs got it all recognized yada yada and the work units i had waiting dissappeared and i am back to only getting the work units the system can handle at one time. I understand (i thjink) that the system will start letting this machine download up to a hundred wu s after a while, but this machine had about a hundred wu s waiting to run. Did adding a card the a system already getting many work units downloaded put it status back and orphan the wu's already waiting?
I am working thru adding multiple gpu's to several systems, and the new card here may be temporarily removed while building out. Am i going to have more problems like this? and am i causing problems for the larger system? ShouId i get the system i am adding to down to not having workunits actively downloaded and then add the gpu? don t want to mess up beacoup wu's while "messing around"
I guess i m still ahving problems, i just noticed that after running a few wu's, the new gpu is now not seen by seti pgm...... sound familiar to anyone? lokks like lotsa fun coming up.......
Just re-ran the cpu benchmarks and the new cpu showed up again......... will watch it..... looks like I ll need lots of notes!!
thanx
"Don't worry about it, nothing is gonna be O.K. anyway.........."
ID: 1954537 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 1327
Credit: 30,352,983
RAC: 142,573
United States
Message 1954565 - Posted: 9 Sep 2018, 22:30:00 UTC - in response to Message 1954537.  
Last modified: 9 Sep 2018, 22:31:08 UTC

For your A10 you have a special case that doesn't apply directly to your Intel cpus as long as you are not using their internal gpu's to process seti tasks.

I would either shutdown the AMD gpu tasks or throttle the cpus to 75% leaving 1 "clear". Either of these will speed up the whole system processing.

If you turn off the AMD gpu tasks, they will have to "clear out" or be aborted before you will see any improvement.

If you use the local configuration to reduce the # of cpus from 100% to 75% that will drop out one cpu, allowing the internal gpu to process at "full speed" while the other 3 cpu's also process at full speed.

I don't have any empirical evidence about how the GTX 1050Ti will run if you throttle the cpu's. But I am hopeful it will run at "full speed".

HTH,
Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1954565 · Report as offensive     Reply Quote
Profile Bravo6 Project Donor
Avatar

Send message
Joined: 28 Oct 99
Posts: 41
Credit: 11,740,506
RAC: 41,934
United States
Message 1954587 - Posted: 10 Sep 2018, 3:09:06 UTC - in response to Message 1954565.  
Last modified: 10 Sep 2018, 3:28:21 UTC

Don t know much about the throttling options, but the AMD solved the problem temp by dying, I think it is either a glitch in the power supply or a dead fan. It is a cyberpower trinity, so I have to do some deconstructing....😞
I m not sure that I understand what you are saying about throttling the cp u s down from 100% to 75%? Where can I do that?
Is there somewhere a decent tutorial on the deeper options of the seti software, I tend to like getting in the weeds when I have spare time, but haven't found a documented source yet, is there one around?
Also I am assuming that the computing options are individual per computer and not "groupwise" is that correct? I am using the local config files (I think). The more I read the more complexities i am seeing.....
Thanx
"Don't worry about it, nothing is gonna be O.K. anyway.........."
ID: 1954587 · Report as offensive     Reply Quote
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16479
Credit: 340,358,761
RAC: 225,349
United Kingdom
Message 1954598 - Posted: 10 Sep 2018, 5:20:05 UTC

Last question first...
It all depends where you set your "computer options", and how you've got other flags set.
If you set the computer options on the website, and do not have the "use local prefs" flag set then settings work at "group" level.
If you do them using the various configuration files on one computer then then they always only apply to that PC.
If you set them using the BOINC manager interface then they can only apply to the local computer, but, in many cases only if the "use local prefs" flag is set.

Throttling of a CPU or GPU reduces the amount of work it is able to do, thus its running temperature. There are various ways of doing it (didn't you guess?). There are tools like "Tthrottle" that allow you to control clock speeds and the like, but they aren't normally needed unless your cool is very marginal, or you are seriously overclocking (so why not just down clock a bit and reduce the temps - that's all these tools do). Or you can tell BOINC not to use all the CPU cores (that can be set using the BOINC manager advanced view) - simple and immediate. Or in the case of an A10 you can opt not to use the APU for processing.

People have provided "mixed reviews" of the performance of the APU in the Ax family from AMD - the A10 is the top of the tree, so has quite a fair performance, certainly better than a CPU core, but not as good as a GTX750. It will need a bit of care when it comes to setting up the drivers as AMD GPUs (and by inference APUs) are "very fussy" over driver setting compared to the nVidia family.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1954598 · Report as offensive     Reply Quote
Profile Bravo6 Project Donor
Avatar

Send message
Joined: 28 Oct 99
Posts: 41
Credit: 11,740,506
RAC: 41,934
United States
Message 1954671 - Posted: 10 Sep 2018, 14:42:49 UTC - in response to Message 1954598.  
Last modified: 10 Sep 2018, 14:45:21 UTC

O. K, a big "duh" is due from me. My brain (unique in its own right) wrapped around those options wrongly and was thinking "time" not "horsepower". Should have known better but was probably in a hurry.
Have to start looking for documentation on the more arcane options....... Want to make my setup as efficient as my poor brain allows......
Thanx
"Don't worry about it, nothing is gonna be O.K. anyway.........."
ID: 1954671 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 1327
Credit: 30,352,983
RAC: 142,573
United States
Message 1954736 - Posted: 11 Sep 2018, 1:11:32 UTC - in response to Message 1954671.  

Bravo6,

My discussion about dropping 1 cpu off the A10 was based on some extensive testing done on another website (I think I read this on someplace on Lunatics where petri got very serious and took lots of notes).

It also is something that is specific to the AMD "All in One" aka: A-x to A-10. I believe that the "choking" problem that I have experienced using the A-series is specific to that design and should not be generalized to all systems, either AMD or Intel.

As Bob mentioned, a way to reduce the number of cores/threads being run on a cpu is to use the individial configuration options that are available in the Boinc manager. Since you appear to be running only one A-10, I would go that route rather than setting it "group wide" from the website.

I understand something about the A-10 has died. Hope it isn't the motherboard/cpu :(

I expect trying to boot it without a video card and with a known good PSU will answer the question of if the MB has bit the dust.

I had a lot of trouble with my A-10. Its driver regularly crashed and when it restarted, the Seti gpu processing didn't restart. I would go with the external card and forget the internal one.

Good luck!

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1954736 · Report as offensive     Reply Quote
Darrell Wilcox
Volunteer tester

Send message
Joined: 11 Nov 99
Posts: 211
Credit: 125,071,961
RAC: 97,996
Vietnam
Message 1957420 - Posted: 27 Sep 2018, 4:44:01 UTC - in response to Message 1954193.  

@Zalster

I run a mixed GTX750Ti/GTX1070 in a couple of systems, and have
found at least some of the parameters can be specified for the
LARGER GPU, and the smaller GPU uses its maximum.

For example, I specify SBS 2048, and the GTX750Ti says
Used GPU device parameters are:
Number of compute units: 5
Single buffer allocation size: 512MB
Total device global memory: 2048MB

while the GTX1070 says
Used GPU device parameters are:
Number of compute units: 15
Single buffer allocation size: 1024MB
Total device global memory: 8192MB

This is with the SOG (stock) version.

I haven't messed with other variables so if you do, let the community know how it goes.
ID: 1957420 · Report as offensive     Reply Quote
Kissagogo27 Special Project $75 donor

Send message
Joined: 6 Nov 99
Posts: 258
Credit: 5,433,738
RAC: 3,867
France
Message 1957437 - Posted: 27 Sep 2018, 8:27:53 UTC

some Raistmer research about Igpu limiting cpu computation

http://lunatics.kwsn.info/index.php?topic=1735.0
ID: 1957437 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 1327
Credit: 30,352,983
RAC: 142,573
United States
Message 1957443 - Posted: 27 Sep 2018, 11:18:56 UTC - in response to Message 1957437.  

some Raistmer research about Igpu limiting cpu computation

http://lunatics.kwsn.info/index.php?topic=1735.0


Thank you. I was looking for that URL!

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1957443 · Report as offensive     Reply Quote
Kissagogo27 Special Project $75 donor

Send message
Joined: 6 Nov 99
Posts: 258
Credit: 5,433,738
RAC: 3,867
France
Message 1957454 - Posted: 27 Sep 2018, 14:49:50 UTC

;o)
ID: 1957454 · Report as offensive     Reply Quote
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 4734
Credit: 302,576,472
RAC: 319,230
United States
Message 1957554 - Posted: 28 Sep 2018, 1:42:50 UTC - in response to Message 1957420.  

@Zalster

I run a mixed GTX750Ti/GTX1070 in a couple of systems, and have
found at least some of the parameters can be specified for the
LARGER GPU, and the smaller GPU uses its maximum.

For example, I specify SBS 2048, and the GTX750Ti says
Used GPU device parameters are:
Number of compute units: 5
Single buffer allocation size: 512MB
Total device global memory: 2048MB

while the GTX1070 says
Used GPU device parameters are:
Number of compute units: 15
Single buffer allocation size: 1024MB
Total device global memory: 8192MB

This is with the SOG (stock) version.

I haven't messed with other variables so if you do, let the community know how it goes.


That's fine but you have to understand how memory is set by Nvidia. Only 26% of total ram is available for OpenCl use. Period _iterations is normally 50. I would not decrease that value as the 750 might have problems.

Here's an example of a commandline.

-sbs 1024 -period_iterations_num 10 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64

The ones in yellow are all you really need. All the rest can be used but that is for a high end GPUs. I never used them on my cards. I would stay away from -hp since there is a 750 in that case.
ID: 1957554 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Questions and Answers : GPU applications : How to use multiple GPU's


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.