Message boards :
Number crunching :
Question about back-to-back-to-back GPUs
Message board moderation
Author | Message |
---|---|
Cruncher-American Send message Joined: 25 Mar 02 Posts: 1513 Credit: 370,893,186 RAC: 340 |
I am currently running a machine with 3 GTX 980s in it ("Big32"). Due to the fact that the MB has only 5 PCIe slots, they are quite close to each other ( slot 1, 3, 5) so there is little, if any, air flow between the cards. Luckily, they are all reference-type cards, with blower fans and large openings in the ends of the cards, so cooling is not too bad. The bottom card runs around 60-65C, and the two cards immediately above run 70-75C. I have another machine running 2 GTX 1080s in slots 1 and 5, so there is a 2-slot gap between them. So no cooling problem (the bottom card runs 56C or so, and the top card runs around 58C or so). Note that these have the more standard side fans on them, so I would assume they would be more affected by being adjacent, as in the other machine. My question is: if I insert another 1080 with side fans between them in slot 3, how will that affect the temps? I know these run cooler than the 980s, but I am wondering if the lack of clean air flow will badly affect them. I don't want to buy another card only to find out I can't use it for heat reasons (too expensive!). In all my machines I am running 3 WUs/GPU, and therefore, for support I need 9 CPU cores for them. Big32 is a dual EP-2670 with 32 HT cores, so no problem there. So I won't do the extra 1080 until I get an X5650 to run in the 1080 machine (it's an X58 i7-950 - "arf-PC") to provide me with 12 HT cores. But it's on the way! |
Shaggie76 Send message Joined: 9 Oct 09 Posts: 282 Credit: 271,858,118 RAC: 196 |
I know the 1070's are a bit less power-hungry but I've got 3 of them sandwiched together in 2 different machines with no problems (not founders edition but still reference blower designs). I do ensure they're fed with cool air from the front of the case, though. |
Zalster Send message Joined: 27 May 99 Posts: 5517 Credit: 528,817,460 RAC: 242 |
I would ask, are these the EVGA cards? Are they ACX design? Do they have backplates on them? If the answer to all three of these questions is yes, then I would tell you not to put a third card in between the 2 existing cards. Why? There is a little description in the pdf that comes with the EVGA ACX card that says... there is the possibility that the fans of an adjoining cards may strike the backplate of the GPU below it if placed next to each other. They recommend the removal of the backplate if that occurs. What they are really saying is the ACX with backplate is slightly larger than the standard 2 pcie slot design. Not quite the 2.1 of some cards but close enough that the fans of the superior card will hit the backplate and (best case just stop turning, or worse case shatter and come flying out) I figured this out with my Mega Cruncher that had 4 ACX 1070s in it. I resolved my issue by places spacers between the GPUS that forced them apart ( but be careful as you can bend the piece that goes into the pcie slot) Even with that, I discovered the temperatures extremely high ( I set a throttle temp of 80C) which 3 of the 4 hit regularly. I can't remember where I saw it but I believe there is a change in how heat is removed from the GPU with the new ACX design. Before the heat was removed to the front and back of the GPUs. But somewhere I saw that the new design actually pushes the heat down away from the card and onto the GPU below it. ( I could very well be wrong in this part since I can't find the source video that I saw about this again but it made sense to me when I saw the video) So what to do? I converted my 1070s to hybrids. By doing so I actually reduced the height of the cards so they no longer touch each other. I also shifted where the heat was exhausted to. Temps are now in the upper 30Cs on the 4 cards Zalster |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13736 Credit: 208,696,464 RAC: 304 |
My question is: if I insert another 1080 with side fans between them in slot 3, how will that affect the temps? As long as there is physical clearance between them (so they don't foul each other as Zalster pointed out) the cards will still run cooler than the reference cards, however the one in the middle will be somewhat warmer than the other 2, and the one closest to the CPU warmer than the coolest one, but still cooler than the one in the middle (all depending on your fan speed settings of course). in all my machines I am running 3 WUs/GPU, and therefore, for support I need 9 CPU cores for them. Big32 is a dual EP-2670 with 32 HT cores, so no problem there. Have you checked to see if that gives the best throughput? For my system (GTX 750Ti and a GTX 1070), running 2 SoG WUs at a time gave only slightly better throughput than 1 WU with 1 Guppie & 1 Arecibo on the same card dragging out the Arecibo WU a huge amount. Running 1 WU at a time I've found to be good enough with SoG. With your GTX 1080s 2 WU per card might be worth the overhead. Grant Darwin NT |
woohoo Send message Joined: 30 Oct 13 Posts: 972 Credit: 165,671,404 RAC: 5 |
I tried running quad cards with backplates and no gaps and temperatures would just skyrocket. I tried two risers but that wasn't reliable for me. So now I'm running triple with one slot gap in between each of the cards, 91C/72C/84C |
Cruncher-American Send message Joined: 25 Mar 02 Posts: 1513 Credit: 370,893,186 RAC: 340 |
I know the 1070's are a bit less power-hungry but I've got 3 of them sandwiched together in 2 different machines with no problems (not founders edition but still reference blower designs). I do ensure they're fed with cool air from the front of the case, though. That's how my 980s roll. I would ask, are these the EVGA cards? Are they ACX design?X No - ASUS Strix; the middle card would be an MSI Do they have backplates on them? Yes Have you checked to see if that gives the best throughput? No - it takes about a week to get anything like a settled-down RAC (after a couple of days wait after making a change to let RAC approach a steady state in terms of validated and pending-validation counts). I tried running quad cards with backplates and no gaps and temperatures would just skyrocket. I tried two risers but that wasn't reliable for me. So now I'm running triple with one slot gap in between each of the cards, 91C/72C/84C I am afraid that will be my case, too. I will have to get a larger (more PCIex16 slots) MB. But I will try it first. Perhaps the 1080s have better thermal properties; they do dissipate less power, after all.[/quote] And thanks for your answers, all! |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13736 Credit: 208,696,464 RAC: 304 |
Have you checked to see if that gives the best throughput? A week and more. I've always just manually checked the runtimes for the various angle ranges. A much more immediate (and due to the randomness of credit, accurate) way of seeing what gives the best throughput. Grant Darwin NT |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
My question is, If you have room to add a third card in between your other two, Why not just move a card and see how it works? |
Zalster Send message Joined: 27 May 99 Posts: 5517 Credit: 528,817,460 RAC: 242 |
hmmm...I've had questions about those cards. Looking at the head on they appear to be slightly larger than 2 pcie slots. 2.1 I know the specs say 2 but you can see there is overhang to either side of the plate where the video plugs are. If you do put them in, just check to make sure they aren't making contact before turning them on. |
Cruncher-American Send message Joined: 25 Mar 02 Posts: 1513 Credit: 370,893,186 RAC: 340 |
My question is, But I can't. There are 7 slots on the MB; only 1,3 and 5 (counting down from the CPU) are physically PCIe x16. So if I have a card in 1 and 5, the only other place for the 3rd card is 3. If you mean just have (2) cards in 1 and 3 (say), the cards still each have a completely exposed side; no card is "covered" on both sides. THAT'S the case I am worried about. I know in the case 1,3 there will be some temp rise, but the openness on one side will prevent catastrophic overheating. Or so I believe, not having tried it. Perhaps I should, but I should be in bed now. I'll try that tomorrow and see what happens. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13736 Credit: 208,696,464 RAC: 304 |
If you mean just have (2) cards in 1 and 3 (say), the cards still each have a completely exposed side; no card is "covered" on both sides. THAT'S the case I am worried about. Yep, but it will allow you to see just how much room there is, and how warm a card will run with another right up against it's intake. And pretty much all the after market cards are cooled much more efficiently than those with the reference coolers. Grant Darwin NT |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
If your worried it will run wild when your away you can set a limit in Tthottle or better yet Percision X to prevent that. But it's certainly worth a try to test the waters. PercisionX works great. after my 1070 "incident" set a 75C limit on it with a low fan speed and it purred right along without throttling my 750 - as Ttrottle will affect both. Now it's quite happy with 2x120mm and 60mm fans on it -- well it works at 68C :D |
Cruncher-American Send message Joined: 25 Mar 02 Posts: 1513 Credit: 370,893,186 RAC: 340 |
Brent - in fact I do use Precision to control fans, so I will look at limiting temps directly with it. I don't use tthrottle; I tried it once and didn't like how it worked (IIRC). It didn't really throttle cores as such, but just turned them off and on very rapidly (or so it seemed in Task Manager). |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
Just set your temperature as the priority, not the usage. It will go to that temp then ease off on it's power levels. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.