Tesla Personal Supercomputing Gpu's

Tesla GPU are not for all, they are so expensive and the price/performance for crunching is not good. Go for Testa only if your software realy need one. There target is the corporate market not the end user market.

For crunching, in the Nvidia universe the high end GPUs GTX690 have a far better price/performance tag. But something you need to know. 4 GTX 690 does not work on the same MB, it´s a windows limitation, they use to many resources to work.

Of course, unless you are "crazy or extramely rich person" and you want to go to a BIOS re-writing of a >US$ 1000 GPU. There are a way to do that, but i know at least one (old and highly trained cruncher) who try and almost "loose" his board, even with the Nvidia suport helping.

If you can handle the mecanical and hot air flow problems, the real actual maximum capacitiy is 3x690 + 1x680 on a MB like yours (actualy 3x680+1x590 works), or if you could find 4x590 but that will produce a lot more heat and drain a lot more power. The actual 3 GPU limit only apply to the 690 model.

But to put that quantity of GPUs on a MB, you will need a ultra Big PSU (each 690 drains 300W, 680 about 170W and a 590 even more >350W not sure) so is 1100W just for the GPUs, + 300W for the rest of the host we are talking about 1400W. You could think a 1500W PSU will drive that, NO you are wrong, there a so many other things to consider. PSU eficience, W/VA conversion, Capacitor/Components Lifetine, ect. so at least you need no less than a 2000W PSU or even bigger if you are thinking to use the 24/7.

In this cases is better go to a hibrid solution, 2 PSU on the same hosts, something like a big 1250 or more PSU for the MB and 2x290 + 1 1000W PSU for the rest 690+680 (there are a lot more combinations possible). You could thing i´m crazy, but that configuration will make you run the PSU on a confortable margin to avoid any trouble.

Of course if you could go to water cooling, if no, prepare to use fans all over the case, these host will produce a lot of heat (with rear and front hot air exaust), more than normal fans can hand. You will need to study your case to maximixe the air flow and heat exaust = more big fans.

One last thing, you will need a high end CPU to handle this beast, go for a top of the line I7 nothing less, big GPUs need a big CPU to feed them.

If all works you will reach no less than >200K RAC on this host actualy bearing >250K.

I build one like this few months ago (with another less expensive MB of course), but i never realy use, works but is to hot to handle, to over the edge componets to fail, then i decide to split in two smaller hosts (with a less expensive I5 CPU, one the 2x690 and the other with 1x690+1x590). An in the end 2 heavy hosts actualy work faster than 1 ultra heavy host)

Good luck, and post a photo if you build one.
____________

Tesla Kepler are they worth the price and for SETI? Any sugestions?

The three 690-cards-per-host limitation, that is strictly due to Windows?

Where in Windows is the bottleneck exactly, just curious...

The three 690-cards-per-host limitation, that is strictly due to Windows?

Where in Windows is the bottleneck exactly, just curious...

DOn´t know why, when i ask the answer was, no mather the MB have suport for 4 GPUS the 690 is realy a 2xGPU on one, for make it work it uses a lot of resources and 4 of them on one host exausts the windows resources. What resources i don cleary Know. Just know because when i try to build a supercruncher, does not work and i ask help from Nvidia suport. But at the time the win 8 was not avaiable, so i don´t know if that changes, in win 7/64 or XP not works.

On the other hand, there is a fix avaiable from EVGA (never see it realy work or not), but is realy complicated and requires a compleate rewrite of the GPU BIOS, even if you know how to do that, is something very dangerous to do with a high price GPU. I know someone who try and need to send the GPU back to EVGA... for repairs... I nerver try i could be crazy but not totaly madness to play with fire.

That limitation only apears on the 690, you could use for example 2x690 + 2x680 on a single host without problem or even 4x590 if you have the way to feed them and take out the heat.

The three 690-cards-per-host limitation, that is strictly due to Windows?

Where in Windows is the bottleneck exactly, just curious...

DOn´t know why, when i ask the answer was, no mather the MB have suport for 4 GPUS the 690 is realy a 2xGPU on one, for make it work it uses a lot of resources and 4 of them on one host exausts the windows resources. What resources i don cleary Know. Just know because when i try to build a supercruncher, does not work and i ask help from Nvidia suport. But at the time the win 8 was not avaiable, so i don´t know if that changes, in win 7/64 or XP not works.

On the other hand, there is a fix avaiable from EVGA (never see it realy work or not), but is realy complicated and requires a compleate rewrite of the GPU BIOS, even if you know how to do that, is something very dangerous to do with a high price GPU. I know someone who try and need to send the GPU back to EVGA... for repairs... I nerver try i could be crazy but not totaly madness to play with fire.

That limitation only apears on the 690, you could use for example 2x690 + 2x680 on a single host without problem or even 4x590 if you have the way to feed them and take out the heat.

I can confirm the power draw and the heat at my 4x590 rig.

I use 2 x1200 watt psus. One for the system and the 2x590, and one for the other 2x590.

Now about the limitations.... i don't know...Maybe it is driver issue and not windows?
My rig working perfect with 275.33 driver, but i have problems with the latest drivers, like restarts, blue screens etc.

The issue is not limited to windows : It is a CPU /MB capacity issue.
The dual CPU asus WS MoBo's may support all 4 690's as long as both CPU are installed.
And, if there is adequate PCI resources available on the motherboard.

What motherboard is Tim running on this host ?

@ Juan, 1) how do you have your 4 580's setup?

2) Would you recommend 2 680's or 1 690 for the best performance? The current price for either setup is similar.

3) Looking at eBay it also looks like 2 590s is possible for the price of a single 690, do you think the 690 is still going to be the best bet?

Thanks for the great information. Power isn't cheap in Germany as well, so sounds like a 690 would be the best choice for me.

One of my rigs has 3 295's in it and your right, its been a pain to get running stable. They put out a ton of heat. Going to give it another try, but at this point I'm thinking it might be better to just sell them off and use the money towered a 690, I bet it would be a lot trouble overall.

Looks like tpl is also running a couple 295 rigs. One has 2 cards, but is only showing 3 cores which is an issue I've had as well with one my cards.

If you were to use an a10 5800k fm2 motherboard 8 gig of pc1866 ddr3 memory a 650 watt power supply and 2 hd6870 video cards plus 1 scrooged case.
for about $560 you would have a 3 gpu system that ran about 4.7 teraflops.
don't overclock you do not need to build another.
save the killer server for controlling the network and being a sweat gaming/ business box.
server load the boxes they don't need monitors, keyboards mice, or hard drives.
I would also spend $30 on a cpu coolers it will run hot.
for prospective 1982 the record of 800 MFLOPS was set by Cray X-M
my current rig runs 4.1 teraflops and cost a hole lot more.
____________

The issue is not limited to windows : It is a CPU /MB capacity issue.
The dual CPU asus WS MoBo's may support all 4 690's as long as both CPU are installed.
And, if there is adequate PCI resources available on the motherboard.

What motherboard is Tim running on this host ?

The issue is not limited to windows : It is a CPU /MB capacity issue.
The dual CPU asus WS MoBo's may support all 4 690's as long as both CPU are installed.
And, if there is adequate PCI resources available on the motherboard.

What motherboard is Tim running on this host ?

I am using the Asus P6T7 WS SuperComputer.

http://www.asus.com/Motherboards/Intel_Socket_1366/P6T7_WS_SuperComputer/