A Question About Memory Support for GPU WUs

Message boards : Number crunching : A Question About Memory Support for GPU WUs
Message board moderation

To post messages, you must log in.

AuthorMessage
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1854121 - Posted: 9 Mar 2017, 16:15:04 UTC

I seem to remember some talk in the fora about using a PCIe x1 slot extender cable to run GPUs. I have a MB with 7 PCIe x16 (physical) slots. 1,3,5 and 7 have 8 or 16 lanes for data; 2, 4 and 6 have 1 for data. I was planning on running 3 dual slot cards in 1, 4 and 7 so there would be space between them so that heat is less of a problem (my Big32 machine has 3 980 reference card running adjacent to one another, and the 2 top cards run considerably hotter than the bottom card due to lack of side ventilation).

My question is: with the middle card on my new machine being in slot 4 (1x electrical), will that have a significant effect on the speed of the card as data is shuffled around?

Has anyone made any measurements of this effect?

Also, I am using SoG; does that minimize the amount of communication between the GPU and CPU? Or should I use CUDA? Or is the relative effect negligible between the two apps?

(Too bad BOINC doesn't allow tuning by card rather than machine, but that's a gripe for another thread!)
ID: 1854121 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1854124 - Posted: 9 Mar 2017, 16:38:15 UTC - in response to Message 1854121.  

I'm thinking the consensus here on the forum is that PCIe X1 slots have adequate data transfer speed for what we are doing with BOINC projects. There is very little information going across the bus from CPU to GPU with our tasks. I believe lots of users have been using PCIe X1 risers to connect to outboard graphics cards with no issues. You should have no problems I believe with your configuration.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1854124 · Report as offensive
Mark Stevenson Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 8 Sep 11
Posts: 1736
Credit: 174,899,165
RAC: 91
United Kingdom
Message 1854136 - Posted: 9 Mar 2017, 17:40:48 UTC - in response to Message 1854121.  
Last modified: 9 Mar 2017, 17:44:02 UTC

I'm using a couple of x1 to x16 powerd pci-e extenders

https://www.amazon.co.uk/dp/B01EM4MIHC/ref=wl_it_dp_o_pC_S_ttl?_encoding=UTF8&colid=1MO3Q5XHSHB61&coliid=I1MVSEROM5S9K0

In machine ID: 7998866 with 4 750ti cards , there's no difference in the speed of one card using the extender compaired to the other 3 using the mobo pci-e slots 2 x x16 & 1 x x8 , i would go with a powerd connector only a couple of £'s more expensive and haven't had any problems with them at all

edit : 1 extender in that machine and another in another machine but no difference or problems with either :-)
Life is what you make of it :-)

When i'm good i'm very good , but when i'm bad i'm shi#eloads better ;-) In't I " buttercups " p.m.s.l at authoritie !!;-)
ID: 1854136 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1854148 - Posted: 9 Mar 2017, 19:42:34 UTC

Thanks for those two answers, guys. That makes feel a whole lot better about what I am doing.

BTW: I don't need an extender, just using the MB slots as is.
ID: 1854148 · Report as offensive
W3Perl Project Donor
Volunteer tester

Send message
Joined: 29 Apr 99
Posts: 251
Credit: 3,696,783,867
RAC: 12,606
France
Message 1854159 - Posted: 9 Mar 2017, 21:01:43 UTC - in response to Message 1854121.  

I seem to remember some talk in the fora about using a PCIe x1 slot extender cable to run GPUs. I have a MB with 7 PCIe x16 (physical) slots.


Just curious to know your MB reference .....
ID: 1854159 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1854165 - Posted: 9 Mar 2017, 21:16:38 UTC - in response to Message 1854159.  

Curious, myself. Have seen many with that many PCIe slots, but not any with that many X16 physical slots.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1854165 · Report as offensive
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1854168 - Posted: 9 Mar 2017, 21:31:21 UTC

Hi folks,

The x1 will do just fine.

1) The task is read into the RAM of the computer and some pre-processing is done
2) The data is copied into the GPU once. YES just once. No feeding done.
3) The data is processed by the GPU with the guidance of the CPU that is mostly waiting (sometimes actively in a loop depending of the driver and the application) for the GPU to decide and tell if there is something to download from the GPU back to the CPU/RAM.
3.5) The spikes are processed by the CPU in CUDA versions while the GPU is busy doing other stuff.
4) The real data that is meaningful for the project is seldom found (the spikes, gaussians, pulses, autocorrelations, triplets). At the very beginning of a task there is some 'best' pulses etc data coming from the GPU. Most of the time nothing comes out except a status flag (found/not found).
5) Do not worry.

Petri

p.s. One could try and put all GPU tasks to a) one core only or b) to the HT cores with a suitable add on program, and run the CPU tasks only on real cores.
p.p.s. I've done that.
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1854168 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1854226 - Posted: 10 Mar 2017, 1:04:09 UTC - in response to Message 1854159.  
Last modified: 10 Mar 2017, 1:05:32 UTC

Just curious to know your MB reference .....



It's an MSI X79 (skt 2011) board: MS-7737 Big Bang Xpower II. (God bless Craigslist!) Runs the extra E5-2670 v1 I have, so 16 threads possible. MB has 2 8pin power sockets for the CPU (?) and a PCIe 6pin socket to help power the 7 slots, so the MB doesn't burn up like the X58 one I had die a few days ago.


Petri: Most Excellent! Just perfect for me...
ID: 1854226 · Report as offensive

Message boards : Number crunching : A Question About Memory Support for GPU WUs


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.