Well, here goes... I've had a browser tab always open with a search for GTX295s that I refresh every so often to keep up on what they sell for, and of course being on the lookout for good deals, so yesterday afternoon I hit it and up popped a seller that had 4 working EVGA GTX295 Hydro cards that were working system pulls. He wanted $150 plus $25 shipping on each. I thought about it for a few minutes, and decided that this was too good a deal to pass up. I know the 5xx series of cards have better performance potential, but at this point in time, I don't believe I can touch the level of performance of these cards for anywhere near $175.

So, I now owned 4 of these cards, but was thinking what do I put them in? I suppose I could build 2 systems with 2 cards each in them, but what fun/challenge would that be? I hit Neweggs site, and checked out motherboards, when I came across this board, the EVGA X58 Classified 4-Way SLI. Even better, as it seems that they are discontinuing them, they were on sale for $379, with a $100 rebate. I read many of the reviews, and most people liked them, though most recently there were some complaints of opened packages or seals on the boards packaging. Of course they didn't have any, so I went to CompUSA.com and found this deal. I know it isn't the latest and greatest board, but it should work perfectly for my intended use. Plus, they guaranteed me that they are new factory sealed pieces, and if it was opened, they would pay to ship it back to them and send me out another one on their dime, so I figured I couldn't lose.

I have a PC Power & Cooling 1200w power supply, which should be enough to get it all up and running, just need to find some splitters for the power cables to go to all the cards. Next I need to decide what proc to get, and if I want to water cool that too. Which brings me to my next dilemma, how to water cool this beast.

I am going to start searching around EVGAs forums to get some ideas as to the type of kit I'll need to get to make this thing work well, as well as the best way to set it up for the most efficient cooling. I know that this is going to be an ambitious project, but should end up in a decent crunching rig.

Lots to learn, but I look at it as an adventure, which will have ups and downs, challenges, etc. It sure will be fun when it's up and running, that I am counting on! If anyone has experience with this type of setup, or has any suggestions/ideas to do it properly or any gotchas they have run into when setting it up, as well as links to sites that could assist me, I would like to hear them all.

---

Well, the main issue with water cooling such a massive number of components is that the water will heat up over time, which will decrease cooling effectiveness. So spend money to get GOOD radiators. When I say good, you are looking for radiators that are THICK and can hold 3 120mm fans. I'd say you will need at least 2, perhaps 3 of those. You are going to be dumping a huge amount of heat with all those cards running 24/7. You're also going to need a very powerful (or even 2 very powerful) pumps to keep the water flowing. As the water passes through each component, there will be a drop in the flow rate, naturally, and most of the waterblocks have a recommended flow rate.

As far as your hardware components go, that sounds like a top 10 system to me. 8 GPUs plus 8-12 CPU cores, depending on what CPU model you go with. I would recommend something pretty beefy to feed all those GPUs.

You may have a bit of trouble finding water blocks for the 295s these days, but someone like dangerden or frozencpu would likely have something that would work. They also have tons of other water cooling materials that you'll need to do the job right. I've dealt with both companies in the past when building water cooled computers for myself and others and they really know what they're talking about. If you're not really sure what to do, I'd say fire off an email to one, or both, of those companies and see what they recommend.

You're probably looking at $600+ for all the waterblocks, tubing, reservoir (should you decide to use one, and I strongly recommend it), fans, radiators, pumps and accessories. You can make this system quiet, but I wouldn't hold my breath that you'd be able to enjoy television with that system in the same room. I'm guessing they'll recommend 6 fans per radiator in a push-pull configuration for maximum airflow and that's gonna be loud, no matter what fans you use. That being said, with water cooling, your components will stay cooler and last longer, so it's a good investment, especially when going with such a high-end system.

Make no mistake, it's a lot of work to set up a proper water cooling system, but the rewards are more than worth the time and effort. Plus, it looks really cool if done right.

http://www.youtube.com/user/Trubyd44#p/u/33/EZRjj2-hQvs
This guy does some custom water-cooling systems on multiple types of CPUs/motherboards/GPUs, you might get a few ideas from him. This wasn't the video I was looking for of his, but it's one of his better ones. He talks about the case modifications he has to do and the issues he's run into with water cooling, so it may be something from which you can learn.

I look forward to seeing your system when it's done, please keep us posted on how it's coming along.

---

-baron_iv
Proud member of:
GPU Users Group

As the water passes through each component, there will be a drop in the
flow rate, naturally, and most of the waterblocks have a recommended flow rate.

I think you meant the heat transfer rate will drop through each unit as the
water warms up on each stage, assuming a serial circuit. The water flow rate
will be the same everywhere, as it is not compressible.

If that is a problem, then a custom solution could use 4 circuits
running through one larger radiator, or even better, an insulated
underground water tank, as that heat could be recovered later.
25kW per day is not a small amount of energy just to throw away!

Hydros come with water blocks baron_iv, So Al doesn't have to worry about that, He just needs to connect them together, For My project with 6 GTX295 cards(I have 5 of 6 now and still need 2 blocks, They'll be sold online by somebody eventually) I had to choose 2 psus, a 650w Thermaltake Drive Bay psu and a 1500w Silverstone psu and an EVGA P55 Classified 200 motherboard. But It's taking a back seat as I'm still trying to buy a house.

---

The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's

As the water passes through each component, there will be a drop in the
flow rate, naturally, and most of the waterblocks have a recommended flow rate.

I think you meant the heat transfer rate will drop through each unit as the
water warms up on each stage, assuming a serial circuit. The water flow rate
will be the same everywhere, as it is not compressible.

If that is a problem, then a custom solution could use 4 circuits
running through one larger radiator, or even better, an insulated
underground water tank, as that heat could be recovered later.
25kW per day is not a small amount of energy just to throw away!

In the old days, when the water just passed through/over a piece of copper, there would be no pressure/flow differences between the beginning of the block (cpu block, for example) and the exit. These days, there are very intricate "patterns" inside the copper part of the cold plate that touches the CPU. Some have pins, like the CPU pins on an old amd/intel CPU, others have grooves, others have waves, but all three greatly disturb the flow of the water inside the block. It's done this way to provide more surface area for heat dissipation into the water. Because of these new designs, the water flow and pressure decrease from one side of the block to the other, even at very high flow rates.

It's really amazing how far we've come in terms of waterblock technology. The old ones were basically a slab of copper that allowed water to flow over it. After many years of improvements, there are some extremely intricate designs inside of a CPU waterblock. On the best waterblocks these days, there's less than a millimeter of copper between the water and the CPU, but there are many times more surface area than there were in the beginning, so the performance has increased dramatically.

---

-baron_iv
Proud member of:
GPU Users Group

I don't deny that there must be corrugations and obstacles to
generate turbulence to maximise heat transfer, which creates
resistance and pressure differences but the inescapable fact
is that flow rate in == flow rate out!

I don't deny that there must be corrugations and obstacles to
generate turbulence to maximise heat transfer, which creates
resistance and pressure differences but the inescapable fact
is that flow rate in == flow rate out!

http://www.clunk.org.uk/martins-liquid-lab-articles/thermalright-xwb-01-water-block-flow-rate-testing.html

"...
The most scientific way to determine a blocks hydraulic resistance is to test pressure drop. Pressure drop is a measurement of pressure loss across a block that varies with flow rate. This is basically a measurement of energy loss, and directly influences how much flow rate you will have.
..."

Perhaps I'm missing something, I am not a scientist or engineer, but the difference in pressure would also mean a difference in flow, correct? A lot of the blocks use jet impingement systems to purposely increase the pressure by limiting flow to a certain degree, then causing it to hit the cooling plate with greater force.

If you look over reviews for water blocks, there is almost a section dedicated to pressure drop (at least on the good reviews). Can you have a drop in pressure without a change in flow? (not trying to be a smarty here, just trying to figure this out in my little brain). ;)

I'm envisioning pouring water through a funnel here, if you try to force too much water, the top of your funnel will run over. Ideally, the perfect flow rate would be the same amount of water in as out, but it rarely happens like that, and when it comes to CPU water blocks, I can't see in my mind how it would work with the exact same flow rate in and out, with the jet impingement and turbulence.

The one way I could think of to agree with you is if you do a comparison over time. Obviously, whatever goes into the block must come out the other side, given enough time, unless there is a leak, but it's not a straight-through trip for the water, so there must be pressure drop on the exit side of the block, therefore a (temporary) difference in flow from one side to the other.

Al, I'm sorry for hijacking your thread here. Perhaps we can all learn something in the process though, so it won't be a total loss. ;)

---

-baron_iv
Proud member of:
GPU Users Group

I don't deny that there must be corrugations and obstacles to
generate turbulence to maximise heat transfer, which creates
resistance and pressure differences but the inescapable fact
is that flow rate in == flow rate out!

Roughly correct, but the temperature gain from input to output reduces the density of the water, hence flow rate out in terms of volume per unit time is actually higher than flow rate in (by a very small factor). In terms of mass per unit time they are absolutely identical, of course.

OTOH, water isn't totally incompressible. A sound wave travels through water as peaks and troughs of pressure, ultrasound cleaners and poorly designed ship propellers can even create troughs sufficient to temporarily convert to gas phase. Turbulence also requires density differences, and amounts to very localized different flow rates.

                                                                Joe

Joe, thanks for the clarification.

Thanks for your encouragement, and thoughts guys. I have a couple advantages with this setup, it is going to be a bench rig, no case/space worries, and for a radiator, I work in appliance repair, so was thinking maybe of a evap from a subzero fridge or freezer, or maybe a small car rad (think ford fiesta, yugo, along those lines). That would provide a decent amount of volume, and living here in MN, I plan on taking advantage of the natural 'cooling' outside, and ducting in some outside air to help cool it in the winter. Kind of like a poor mans makeup air.

I am still looking at pumps and hose, trying to decide which size to go with, as with smaller size you get less flow but the water has a chance to stay in the heat sink longer to pick up more heat, while larger had a better flow rate, but too large and the water can't pick up the the maximum amount of heat and thus is less efficient. Is there some type of calculator to help a person pick the idea size, and maybe even give suggestions of other component sizes and quantities based upon user input? I still have a lot to read, and research to do on that front.

I checked out their CPU compatibility list, and am trying to decide which way to go. I know that it would be nice to have the 980X, but at a grand, well, that probably isn't going to happen, but given the list below, which would be a persons best choice, there's a few diff flavors of Xeons and reg i7's, now sure which has the best perf, but if I could find a used high end Xeon, that would be nice. If I remember correctly, the X series of Xeon are the highest end ones, but would they be the smartest choice in this rig for price/performance? I presume bigger/faster is always better...

Gulftown
Core i7 980X - 3.33 GHz
Bloomfield
Core i7 975 - 3.33 GHz
Core i7 965 - 3.20 GHz
Core i7 960 - 3.20 GHz
Core i7 950 - 3.06 GHz
Core i7 940 - 2.93 GHz
Core i7 930 - 2.80 GHz
Core i7 920 - 2.66 GHz
Xeon W3580 - 3.33 GHz
Xeon W3570 - 3.20 GHz
Xeon W3540 - 2.93 GHz
Xeon W3530 - 2.80 GHz
Xeon W3520 - 2.66 GHz
Gainestown
Xeon W5590 - 3.33 GHz
Xeon W5580 - 3.20 GHz
Xeon L5530 - 2.40 GHz
Xeon E5507 - 2.27 GHz
Xeon E5503 - 2.00 GHz
Nehalem
Xeon X5550 - 2.66 GHz
Xeon L5520 - 2.26 GHz
Xeon L5506 - 2.13 GHz
Xeon E5540 - 2.53 GHz
Xeon E5530 - 2.40 GHz
Xeon E5520 - 2.26 GHz
Xeon E5506 - 2.13 GHz
Xeon E5504 - 2.00 GHz
Westmere
Xeon X5680 - 3.33 GHz
Xeon X5677 - 3.46 GHz
Xeon X5670 - 2.93 GHz
Xeon X5667 - 3.06 GHz
Xeon X5660 - 2.80 GHz
Xeon X5650 - 2.66 GHz
Xeon L5640 - 2.27 GHz
Xeon L5630 - 2.13 GHz
Xeon E5649 - 2.53 GHz
Xeon E5645 - 2.40 GHz
Xeon E5640 - 2.67 GHz
Xeon E5630 - 2.53 GHz
Xeon E5620 - 2.40 GHz
Xeon E5607 - 2.27 GHz
Xeon E5606 - 2.13 GHz
Xeon E5603 - 1.60 GHz


Thanks again guys, I will take pics now and then as my build progresses, hope to have it all together in a month or two, but I am pretty busy at work, and with an active 4 yo, spare/free time is often in short supply, so we'll see how it goes. Plus, I have a bit of research to do on this to make sure I do it right. Don't want to put all that time and money into it to find that I messed up something basic like tube size or such.

---

http://www.overclock.net/water-cooling/749693-review-22-cpu-waterblocks-tested-roundup.html

Here's a good benchtest of many of the top CPU waterblocks, with specs, flow rates, and all other pertinent information. At the bottom of the review is a list of how they perform.

As far as flow rate and tube size, bigger is better, to an extent. I think most end up with 3/4" tubing, some with 5/8ths. If you do a search for individual waterblocks that peak your interest from the list above, you can often find reviews to see how they perform at different flow rates. With very few exceptions, higher flow rate = better cooling. I've always used 1/2 ID (inner diameter tubing), but I never cooled as much as you need to cool, so I don't know how well that would work for you. 5/8 or 3/4 will likely be best, but asking the guys at dangerden might be the best idea, if nobody else here responds.

One thing I did that you can learn from is about tubing. When I built my first watercooled system, the stores were mostly out of the good half inch id tubing, so I got the brilliant idea to go to home depot and buy tubing...sounds clever, right? Wrong. The tubing at home depot had very thin walls, which caused it to kink and pinch off flow in placves, even though I was using the plastic swirly stuff that goes around the outside of tubing to prevent that. So don't skimp on good tubing! ;)

---

-baron_iv
Proud member of:
GPU Users Group

http://www.overclock.net/water-cooling/749693-review-22-cpu-waterblocks-tested-roundup.html

Here's a good benchtest of many of the top CPU waterblocks, with specs, flow rates, and all other pertinent information. At the bottom of the review is a list of how they perform.

As far as flow rate and tube size, bigger is better, to an extent. I think most end up with 3/4" tubing, some with 5/8ths. If you do a search for individual waterblocks that peak your interest from the list above, you can often find reviews to see how they perform at different flow rates. With very few exceptions, higher flow rate = better cooling. I've always used 1/2 ID (inner diameter tubing), but I never cooled as much as you need to cool, so I don't know how well that would work for you. 5/8 or 3/4 will likely be best, but asking the guys at dangerden might be the best idea, if nobody else here responds.

One thing I did that you can learn from is about tubing. When I built my first watercooled system, the stores were mostly out of the good half inch id tubing, so I got the brilliant idea to go to home depot and buy tubing...sounds clever, right? Wrong. The tubing at home depot had very thin walls, which caused it to kink and pinch off flow in placves, even though I was using the plastic swirly stuff that goes around the outside of tubing to prevent that. So don't skimp on good tubing! ;)

I knew about kinking, What I didn't know is that It can happen even with anti-kink coils(for lack of the right word), Me I have 16' of Swiftech 1/2"ID Hose(3/4"OD), I have two 120x360mm Swiftech radiators, I may have to get a 3rd rad, Maybe an XSPC 360...

---

The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's

I have also heard of people using heater cores. They have a ton of capacity. I think that was what he was referring to in his last post. I still have one from an 80s oldsmobile, which is massive...would hold like 6 120mm fans on each side, made entirely of copper, heavy. Quite ugly though, if you are concerned about looks. Got it at auto zone. lol

I did some crazy stuff when I first got into water-cooling.

---

-baron_iv
Proud member of:
GPU Users Group

I'm posting this here, As It pertains to running four GTX295 cards, I just talked to a person a Silverstonetek about their Strider SST-ST1500 1500w psu and 4 GTX295 cards, I was told It would only run 3, Of course He did leak that their introducing a New 1350w psu in a couple of months(Thermaltake beat them to this wattage size of course and theirs has 60A on each of the two 12v rails).

So anyone who's running 4 GTX295 cards off a single psu, What are You using?

I'm looking at eventually maybe buying an Enermax Galaxy EVO 1250w psu to replace the unopened, unwanted and brand New Silverstone SST-ST1500 1500w power supply as I can't get a refund or an exchange from Performance-PCs at all as I waited too long I was told.

---

The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's

VW

Cooler master Silent Pro Gold 1200 with i7 930 and 4 295 works fine

had it running for several weeks till one card decided to not play nicely

but will be back upto 4 cards again in a few days

---

Tim

I'm posting this here, As It pertains to running four GTX295 cards, I just talked to a person a Silverstonetek about their Strider SST-ST1500 1500w psu and 4 GTX295 cards, I was told It would only run 3, Of course He did leak that their introducing a New 1350w psu in a couple of months(Thermaltake beat them to this wattage size of course and theirs has 60A on each of the two 12v rails.

Strange how a 1500 watt PS won't support that load, but a 1350 (10% less rated power) would? Maybe it's not really 1500 watts, or else it isn't rated properly? Weird.

---

Maybe I'll just get an Enermax 1250w Galaxy EVO, As It has 30A on each of Its 12v rails or 104A total, It's not cheap, But at least I can swap cables between the 1050w and the 1250w.

Cooler Master 1200 / 98.0A Single 12v rail
Corsair AX1200 /100.4A Single 12v rail
Sparkle 1250 /100.0A Six 20A 12v rails
Enermax Galaxy EVO 1250 /104.0A Six 30A 12v rails
Thermaltake Toughpower 1350w Two 60A 12v rails
Thermaltake Toughpower XT Gold 1275w & 1475w Too New to tell yet.

---

The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's

I'm posting this here, As It pertains to running four GTX295 cards, I just talked to a person a Silverstonetek about their Strider SST-ST1500 1500w psu and 4 GTX295 cards, I was told It would only run 3, Of course He did leak that their introducing a New 1350w psu in a couple of months(Thermaltake beat them to this wattage size of course and theirs has 60A on each of the two 12v rails.

Strange how a 1500 watt PS won't support that load, but a 1350 (10% less rated power) would? Maybe it's not really 1500 watts, or else it isn't rated properly? Weird.

I know, Sounds weird to Me too, I looked at selling the 1500w on ebay, But found almost no demand for them, So I ate $374.95, Of course If I bought a 2nd 1500w psu I'd have enough power for all six, But then I'd have no place for the 2nd/3rd Radiator inside the case, My last post shows the choices available of course, So I gather You have a 1350w Thermaltake Toughpower? Sounds Beefy.

---

The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's

I'm posting this here, As It pertains to running four GTX295 cards, I just talked to a person a Silverstonetek about their Strider SST-ST1500 1500w psu and 4 GTX295 cards, I was told It would only run 3....

http://www.jonnyguru.com/modules.php?name=NDReviews&op=Story&reid=175

me thinks the guy you talked to is wrong, see above link.

i believe my thermaltake 1200w in theory would have enough capacity! i7 + 3x gtx295's pulls about 1100 max from the wall, at 83% eff, that is only 913 watts from the power supply. you don't think 500 watts is enough to run one graphics card???

---

I'm posting this here, As It pertains to running four GTX295 cards, I just talked to a person a Silverstonetek about their Strider SST-ST1500 1500w psu and 4 GTX295 cards, I was told It would only run 3....

http://www.jonnyguru.com/modules.php?name=NDReviews&op=Story&reid=175

me thinks the guy you talked to is wrong, see above link.

i believe my thermaltake 1200w in theory would have enough capacity! i7 + 3x gtx295's pulls about 1100 max from the wall, at 83% eff, that is only 913 watts from the power supply. you don't think 500 watts is enough to run one graphics card???

Jonny Guru said It should, Not that It would, But then the ST1500 does support 4 GTX580 cards according to Silverstone, I'll take a chance on It, Right now I have no other choice as house hunting is paramount(I made an offer on one today).

---

The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's