---Edit-------
I just re-read your previous message which addressed my question(s)
-----------------------------
I guess I need to borrow that Kiliwatt meter and see how far an 850 watt PSU will go in this gtx 1060 3GB/gtx 750Ti experiment. And see how to rig my cables to both drive the top power cable connectors as well as the expander board(s) power.

Tom

---

A proud member of the OFA (Old Farts Association).

I noted the outlet meltdown mentioned earlier in the post. I recently added two new 20 amp circuits (in 2 different buildings) to handle the growth in my stable of systems. Standard residential duplex outlets come in different ratings (15, 20 and 30 amp are common). Most tract residential construction has used 15 amp rated outlets, and they look identical (from the front) to 20 amp duplexes. The only way to distinguish between then is the rating which is cast or stamped on the outlet reverse side. Also, the outlet needs to be consistent with the circuit breaker on that branch circuit. Putting a 20 amp outlet on a 15 amp breaker is a recipe for trouble. Switching out to a higher breaker won't always solve the problem, even if you match the breaker and outlet, because the conductor needs to be sized to the load. Many older tract homes might only use 14 gauge wire as opposed to 12 gauge, because it was less expensive. Depending on the length of the wire run, the gauge then becomes a limiting factor. You burn the wire in the wall while the outlet and breaker chug along just fine. When in doubt, consult an electrician.

---

Awesome post with some great info. With the collapse of all the 'coin' mining, this type of hardware is pretty cheap on ebay.

---

Well, I got up to 3 GPUs on what is a 2 full slot MB (with two short slots). And then noticed I don't have enough places to plug in my jury-rigged power cables :)

Guess I am going to have to borrow the PSU from my 2990wx rig after the CPU finishes its auction on eBay (Thursday) or earlier after I get done processing all the tasks in the cache.

In theory, I can try up to 6 GPUs if I can get the power for them. The other PCIe extender hasn't come in yet :(

Since this is not a Miner motherboard I don't know what the results will be until I plug it together and try.

Once I have gotten a "final" version up and running and the table it is sitting on cleaned off, I promise a picture ;)

This is a box that runs a mix of BOINC projects (Seti, Seti beta, World Community Grid, Rosetti@Home, Mind Modeling and Einstein@home (sometimes)). The gpu's are Seti only.

https://setiathome.berkeley.edu/show_host_detail.php?hostid=8565979

Tom

---

A proud member of the OFA (Old Farts Association).

Well, I got up to 3 GPUs on what is a 2 full slot MB (with two short slots). And then noticed I don't have enough places to plug in my jury-rigged power cables :)

Guess I am going to have to borrow the PSU from my 2990wx rig after the CPU finishes its auction on eBay (Thursday) or earlier after I get done processing all the tasks in the cache.

In theory, I can try up to 6 GPUs if I can get the power for them. The other PCIe extender hasn't come in yet :(

Since this is not a Miner motherboard I don't know what the results will be until I plug it together and try.

Once I have gotten a "final" version up and running and the table it is sitting on cleaned off, I promise a picture ;)

This is a box that runs a mix of BOINC projects (Seti, Seti beta, World Community Grid, Rosetti@Home, Mind Modeling and Einstein@home (sometimes)). The gpu's are Seti only.

https://setiathome.berkeley.edu/show_host_detail.php?hostid=8565979

Tom

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

---

A proud member of the OFA (Old Farts Association).

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

Looks like it is a hard limit. Not sure if it is MB hard limit or bios hard limit. Since this is a "generic" X79 (or something like that) MB I don't think I am going to be able to find a bios update.

So I guess the next question is just how high the Seti RAC will go with 5 GPUs and a mixed load (including Seti) on the cpu.

Tom

---

A proud member of the OFA (Old Farts Association).

I think you can count on one hand the number of hosts running multiple gpu card systems over the normal four on mining boards or on extender cables and have them running normally. At least that is what it appears to be the case from my quick perusal of the first few pages in the Top 100 list. Seems to be a difficult task to achieve.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

This thread needs more love!!!

Sorry for the HUGE pictures, I'm not sure how to re-size for forum use?
Also, if you think cable management is important, consider this your "trigger" warning!

ASUS X99-E-10G WS Motherboard
*Motherboard has 7 PCIe slots & a PCIe M.2 slot, hence the ability to run 8 GPU simultaneously.
*When the M.2 slot is used with a GPU, this is the card that drives the monitor. Unsure why.
Intel i7-5960x - 8/16 - @ ~4.0GHZ (water cooled)
2x EVGA Titan X Hybrid GPU (maxwell)
6x EVGA 1070SC Gaming GPU (pascal)
EVGA 1600 T2 PSU
I forget the H2O system, RAM, and other component stats, as this was a couple years ago.
Windows 10 and Stock SETI App.
Various Configurations (it continuously evolved):




Same rig with seven Zotac GT1030 single slot cards. This short-lived test was quite important for what, at the time, was my eventual goal for my dedicated SETI rig.


Rig as it sits now (finally back up and running!).
ASUS X99-E-10G WS Motherboard
Intel i7-5960x - 8/16 - @ ~3.8GHz (air cooled)
4x NVIDIA Titan Xp GPU (pascal) with EVGA 1080Ti Hybrid Water Cooler Kits
Linux Lubuntu 16.04 & CUDA90 App


A few notes*
The 8-GPU rig very nearly set my house on fire. If you are going to try a setup like this, make sure not only your PSU (or two) is capable of the stress, but your home wiring is up to the task as well. No joke, there were flames and a melted outlet, I was lucky to be home to witness the event and stop it immediately. Air cooled CPU and GPU are LOUD, Very loud. Not much you can do about thermal output, but switching to water-cooled CPU and GPU drops the noise pollution significantly. This brings me to the seven GPU test. I bought this motherboard specifically for an incredibly insane idea. Mount 7 GPU to the MOBO, all in a water loop, to have a ridiculous SETI rig, while maintaining a small physical and audible footprint. My goal was to use SEVEN NVIDIA Titan Xp GPU, like the four in the last picture above. These cards, when equipped with a water plate, physically become single-slot cards. To make a long story short, this is a good example of what I was after, before I went broke... HAHA:

https://rawandrendered.com/octane-render-hepta-gpu-build

That was my eventual goal, but I ran out of money so 4x Titan Xp will have to do haha! Even If I had the money to complete my build, I would still have (as Ian&Steve C. mentioned) power delivery issues. I'll have to wait until I own a house and can make some specific modifications to the electrical system before I feel comfortable adding much to this unit. A custom water loop with an external (read: outdoor) radiator is also on the list, but that's another insane idea that is just going to have to wait.

Let's see some more crazy rigs!!!!

I was browsing on Youtube for SETI videos and your computer setup with 4 titan Xps being converted to water cooling came up. I watched it. It looked like a neat project!

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

Looks like it is a hard limit. Not sure if it is MB hard limit or bios hard limit. Since this is a "generic" X79 (or something like that) MB I don't think I am going to be able to find a bios update.

So I guess the next question is just how high the Seti RAC will go with 5 GPUs and a mixed load (including Seti) on the cpu.

Tom

Just stop trying to use that chinese “x79” board for anything but a basic setup. It’s going to be nothing but problems.

If you want a multi-GPU setup, you’re better off just using a standard Z270 board like I am. Or get a mining board if you want more than like 7 GPUs.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

Looks like it is a hard limit. Not sure if it is MB hard limit or bios hard limit. Since this is a "generic" X79 (or something like that) MB I don't think I am going to be able to find a bios update.

So I guess the next question is just how high the Seti RAC will go with 5 GPUs and a mixed load (including Seti) on the cpu.

Tom

Just stop trying to use that chinese “x79” board for anything but a basic setup. It’s going to be nothing but problems.

If you want a multi-GPU setup, you’re better off just using a standard Z270 board like I am. Or get a mining board if you want more than like 7 GPUs.

If I want to re-use the e50270 v1 cpu (Lga 2011) and maybe go with a miners rack so there is no size constraint have you had better luck with anything?

Like this one: https://www.ebay.com/itm/Supermicro-EATX-Extended-ATX-DDR3-1600-LGA-2011-Motherboard-X9DRH-ITF-O/192735767457?epid=23026376286&hash=item2cdff26ba1:g:bm4AAOSwa~Fb-VB9:rk:9:pf:0&LH_BIN=1? It has 7 PCIe slots.

Tom

---

A proud member of the OFA (Old Farts Association).

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

Looks like it is a hard limit. Not sure if it is MB hard limit or bios hard limit. Since this is a "generic" X79 (or something like that) MB I don't think I am going to be able to find a bios update.

So I guess the next question is just how high the Seti RAC will go with 5 GPUs and a mixed load (including Seti) on the cpu.

Tom

Just stop trying to use that chinese “x79” board for anything but a basic setup. It’s going to be nothing but problems.

If you want a multi-GPU setup, you’re better off just using a standard Z270 board like I am. Or get a mining board if you want more than like 7 GPUs.

If I want to re-use the e50270 v1 cpu (Lga 2011) and maybe go with a miners rack so there is no size constraint have you had better luck with anything?

Like this one: https://www.ebay.com/itm/Supermicro-EATX-Extended-ATX-DDR3-1600-LGA-2011-Motherboard-X9DRH-ITF-O/192735767457?epid=23026376286&hash=item2cdff26ba1:g:bm4AAOSwa~Fb-VB9:rk:9:pf:0&LH_BIN=1? It has 7 PCIe slots.

Tom

You know. It just dawned on me I have a high-end server motherboard running here that all I have to do is upgrade the PSU and I can try playing with at least 7 GPUs.

First I have to install my 1600 watt psu and then set it up on the bench with the side off so I can get at one of the slots. It has run 3 gpus and left a slot uncovered before. Hmmmmm. I already have another gtx 1070Ti on order for it. I wonder if I can get two 1070Ti's and 5 1060 3GB's to play nicely. :)

Probably not. But at least all I have to do is spend some time not yet more money :)

Tom

---

A proud member of the OFA (Old Farts Association).

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

Looks like it is a hard limit. Not sure if it is MB hard limit or bios hard limit. Since this is a "generic" X79 (or something like that) MB I don't think I am going to be able to find a bios update.

So I guess the next question is just how high the Seti RAC will go with 5 GPUs and a mixed load (including Seti) on the cpu.

Tom

Just stop trying to use that chinese “x79” board for anything but a basic setup. It’s going to be nothing but problems.

If you want a multi-GPU setup, you’re better off just using a standard Z270 board like I am. Or get a mining board if you want more than like 7 GPUs.

If I want to re-use the e50270 v1 cpu (Lga 2011) and maybe go with a miners rack so there is no size constraint have you had better luck with anything?

Like this one: https://www.ebay.com/itm/Supermicro-EATX-Extended-ATX-DDR3-1600-LGA-2011-Motherboard-X9DRH-ITF-O/192735767457?epid=23026376286&hash=item2cdff26ba1:g:bm4AAOSwa~Fb-VB9:rk:9:pf:0&LH_BIN=1? It has 7 PCIe slots.

Tom

You know. It just dawned on me I have a high-end server motherboard running here that all I have to do is upgrade the PSU and I can try playing with at least 7 GPUs.

First I have to install my 1600 watt psu and then set it up on the bench with the side off so I can get at one of the slots. It has run 3 gpus and left a slot uncovered before. Hmmmmm. I already have another gtx 1070Ti on order for it. I wonder if I can get two 1070Ti's and 5 1060 3GB's to play nicely. :)

Probably not. But at least all I have to do is spend some time not yet more money :)

Tom

I did install the 1600 watt PSU and after a lot of variations on the theme it only runs 3 discrete gpu cards w/without the internal gpu enabled/disabled.

So that box is now running 2 Gtx 1070Ti's and a Gtx 1060 3GB. Without trying a new MB the only upgrade left on it will be a 3rd gtx 170Ti (Zotac, because they are light and the slot is a 1x).
http://setiathome.berkeley.edu/show_host_detail.php?hostid=8610031

Since I actually managed to get more gpus (5) running on my generic x79 I may go back and put that into production if I can figure out the power wiring solution that will plug into the PSU. I don't want to use my 2nd 1600 Watt PSU on it.

Tom

---

A proud member of the OFA (Old Farts Association).

Well, it is running 5 gpus. The bios popped up and said "insufficient PCIe resources, please remove 1 card. Have disabled one card". If you want to continue booting do save/discard exit. I have 6 GPUs plugged in 2 on the MB and 4 via the expansion board.
Tried enabling the "use over 64" for decoding, apparently didn't help.

It is funny really. The PSU (1600 watts) didn't start spinning till it got slightly warm :)

Well, I wanted to "bump my nose" against the gpu limit that everyone else has been experiencing and I think I just did.

I will try doing a reset to "default" (aka: "load optimized settings") and see if that helps.

Tom

I set out to recreate the above box with an 850 watt PSU because I didn't want to tie down my 2nd 1600 watt PSU.

No dice. It won't boot with more than 2 Gtx 1060's installed. Guess I will have to use my 2nd 1600 watt PSU if I want to go back up to 5 gpus on this box https://setiathome.berkeley.edu/show_host_detail.php?hostid=8565979

Probably don't have time till Tuesday.

Tom

---

A proud member of the OFA (Old Farts Association).

Ok, finally adding a picture of my 9 gpu hardware setup. I am perfectly aware that this is not a pretty rig.



Here is a link https://drive.google.com/open?id=1PWJ5gCp2vWIz9kZ-w4FAWs835AI3wRZSRQ

Apparently getting it to display requires something I am not figuring out yet.

Tom

---

A proud member of the OFA (Old Farts Association).

Apparently getting it to display requires something I am not figuring out yet.

I'd like to figure it out too. I used to be able to post an image on a hosting site, but not now apparently.

All you see is the placeholder icon now. I did find out all you have to do to open the image is right-click the placeholder icon in the post and open it in another tab.

But why did the behavior change? Something to do with the forum software or the browser I assume.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

All the pics I host on imgur seem to post properly. Maybe something to do with the image being hosted on google drive? Or not using the proper link. You have to use the img tags with the url to the direct image, not the gallery in google

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Ok, finally adding a picture of my 9 gpu hardware setup. I am perfectly aware that this is not a pretty rig.



Here is a link https://drive.google.com/open?id=1PWJ5gCp2vWIz9kZ-w4FAWs835AI3wRZSRQ

Apparently getting it to display requires something I am not figuring out yet.

Tom

fixed it for you.

it's because of Google.

to find the direct img URL:

http://drive.google.com/uc?export=view&id=PASTE YOU ID HERE

you'll be redirected to the direct image. copy and paste it from the address bar

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Actually when using the image tags your link must end with a .jpg/.bmp/.png/etc for the image to be displayed. ;-)

Cheers.

Actually when using the image tags your link must end with a .jpg/.bmp/.png/etc for the image to be displayed. ;-)

Cheers.

the image i just posted does not end in those file types. google does things differently with direct links to images.

here is the direct image URL that is inside the image tags in my previous post. and is displaying just fine in my Firefox browser (doesnt seem to load in iOS/Safari though)

https://doc-00-9c-docs.googleusercontent.com/docs/securesc/i7qnpvbfjt17088uv19pcui53oq9i6r6/5abesao3a7jq51pc2td2u0kfrvc0mpu1/1551060000000/17402175493652966844/16007457166002018431/1PWJ5gCp2vWIz9kZ-w4FAWs835AI3wRZSRQ?e=view&nonce=mqkrfhum3r3r6&user=16007457166002018431&hash=jrmojpun93fhfm5js9gsjn4mm3a8c218

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Ok, finally adding a picture of my 9 gpu hardware setup. I am perfectly aware that this is not a pretty rig.



Here is a link https://drive.google.com/open?id=1PWJ5gCp2vWIz9kZ-w4FAWs835AI3wRZSRQ

Apparently getting it to display requires something I am not figuring out yet.

Tom

fixed it for you.

it's because of Google.

to find the direct img URL:

http://drive.google.com/uc?export=view&id=PASTE YOU ID HERE

you'll be redirected to the direct image. copy and paste it from the address bar

At least on the Foxfire I am using under Linux, the image didn't display. But if I can get a direct link with the file name ending in a graphics file it will display? Hmmmmmm......

Let me look around.

Tom

---

A proud member of the OFA (Old Farts Association).