Message boards :
Number crunching :
A Multi-GPU Cruncher for the Less Affluent Among Us
Message board moderation
Author | Message |
---|---|
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
Being rather frugal and with apparently somewhat shallower pockets than some crunchers, I read with some bemusement the threads detailing the building of crunch-monsters with the latest and greatest top-of-the-line processors, motherboards, GTX 10xx GPUs, and cooling solutions that rival the refrigerator in my kitchen. All with apparently little regard to the dollars/euros/pounds/yen/rupees that go out the door. My approach tends to be a bit different, recycling/reusing/repurposing pre-owned components, primarily from sources on eBay and elsewhere online. Just about everything I currently crunch with, except for my daily driver, has come to life via that route. A recent plan to start upgrading my GTX 660s to GTX 960s, now that the folks on the bleeding edge were starting to dump their barely used 9xx cards in favor of the new 10xx series, instead evolved into the assembly of an entirely new multi-GPU cruncher, at a cost that, at least to me, was relatively affordable. The new machine is host 8064262. It's an HP xw9400 Workstation that arrived at my door with the following specifications: Dual Quad Core AMD Opteron 2380 2.5GHz Processors HP OEM Liquid Cooler 16GB ECC RAM Two 300GB SAS Hard Drives 1050-Watt PSU NVIDIA Quadro FX3700 Graphics Card After pulling half the RAM, one of the HDDs and the Quadro FX3700, I gradually added one GPU at a time to get to the cruncher that's now been happily humming along for a couple of weeks. At what cost, you may ask. Well, even if you didn't, that's the whole point of this post. Here's the tale of the [cash register] tape (all prices are in US$ and include any S/H charges): HP xw9400 Workstation =============== $170.00 Windows 7 Pro 64-bit ================= 34.94 HP (OEM) GTX 960 Reference GPU =========== 119.00 Gigabyte GTX 960 GPU ================= 122.50 EVGA GTX 960 SSC GPU ================= 122.50 Gigabyte GTX 960 GPU =================== 120.00 PCIe x16 12-inch Riser Cable =============== 10.32 Two PCIe 12-inch 6-pin Extension Cables ======= 13.98 PCIe x16 7-inch Riser Cable (from my parts bin) === N/C KVM Cable (from my parts bin) =============== N/C ------------------------------------------------------------- TOTAL Cost ============================= $713.24 That cost could also be slightly offset by the extra items (8GB ECC Memory, 1 300GB SAS HDD, Quadro FX3700) pulled from the machine. In fact the FX3700 has already been sold for a net return of $10.52. So, with only a couple weeks of occasionally intermittent crunching, that box has already reached a rack of 20K and I'm hoping to see it reach somewhere close to 35K by the time it stabilizes (not running 24/7 but rather about 143 hours per week). Does that compare with any of the "latest and greatest" boxes built by the deep pockets guys? Probably not, but I still have money to spend on other things, like food. So, for those of you with a few cobwebs in your wallet, you can see that there are actually some fairly heavy-duty crunching possibilities out there that don't require a second mortgage! ;^) |
Al Send message Joined: 3 Apr 99 Posts: 1682 Credit: 477,343,364 RAC: 482 |
Jeff, I guess I'm one of those with the crunching monsters you're referring to. I actually have been swinging both ways in this regard, yes, I have put together a couple fairly over the top machines recently, but I also have on tap a couple that would definitely fall into the recycling/reusing/repurposing pre-owned components side of things as well. For example, I am re-purposing a couple of HP Z800 motherboards, which weren't terribly expensive, under $125, which is less than many new motherboards out there. Also using 4-5 generation old Xeon procs in them, which again is less expensive than many new I series procs available today. Not to mention that I am also most recently buying $80-90 (after rebate) 750Ti cards, as the 10*0 series are still way out of line for the purchasing more than one or 2, at least for me until they have a significant price drop, or I win the lottery. Neither of which I am counting on any time soon. I guess the way I look at it, is this is one of my few hobbies, that I enjoy spending my time and $ on. I don't go to bars and drink (and then literally p*ss) away my money, I don't do lavish vacations for weeks at at time every year, I've been pretty good at saving for quite a while, and have had decent jobs over the years. I guess to me it's more along the lines of spending your money on whatever brings you enjoyment today, as long as you're being responsible of course. And, preferably if it's legal. ;-) So, you'll probably see a few more of the older systems coming on line in the not too distant future in my computer listing, because that is where I usually live, I just thought it would be fun to toss together a couple crazy machines and see how they go. It's all in good fun, as it should be for everyone. So, keep on crunchin', hopefully one of is will have that WoW moment we're all searching for, because in the end, that's why I'm here! :-) |
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
Yeah, I have nothing at all against those with the money to spend on the shiniest stuff. I get a bit envious sometimes, too. But since it seems like there's been so much emphasis on the high-end toys lately, especially since the 10xx series was launched, I thought it worthwhile to remind those who are itching to increase their contribution to the project that there can be more economical ways to do it, too. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13736 Credit: 208,696,464 RAC: 304 |
I thought it worthwhile to remind those who are itching to increase their contribution to the project that there can be more economical ways to do it, too. Lots of GTX 750Tis available at very reasonable prices, and as Shaggie76's charts show they're no slouch when it comes to work done for power consumed. Still keeping pace with the latest & greatest & leading all the older cards. Grant Darwin NT |
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
I thought it worthwhile to remind those who are itching to increase their contribution to the project that there can be more economical ways to do it, too. My other xw9400 (with an 800w PSU) has been running for a couple years now with 2 GTX 750Tis and 2 GTX 660s. The GTX 660s are more productive overall but do use more power. I went ahead and stuck with the 960s on the new xw9400 because I figured the 1050w PSU gave me more headroom and I went for higher total output instead of output per watt. As it is, the new build actually seems to be drawing slightly less total power than that first xw9400, and my preliminary estimate is for at least 15% more crunching. I also think that, although the newer Maxwell cards aren't as productive as they could be with the current apps, I've gotten the impression that once Jason and Petri dish out that Special app to the general public, that whole scenario will improve. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13736 Credit: 208,696,464 RAC: 304 |
I also think that, although the newer Maxwell cards aren't as productive as they could be with the current apps, I've gotten the impression that once Jason and Petri dish out that Special app to the general public, that whole scenario will improve. Oh yes. As it is, using SoG with the example command line gives a huge boost to their output. Grant Darwin NT |
tazzduke Send message Joined: 15 Sep 07 Posts: 190 Credit: 28,269,068 RAC: 5 |
Greetings Jeff, That is a nice economical build, on your side of the world, but here in Australia, those 960's would be pulling AUD$ 200 plus. But I am in the way of reuse/recylce, my 3 crunchers are old LGA775 P45 Boards running a GTX 580, GTX 760 and GTX 770. Yep 3 really power munchers. They are not mega crunchers, and mostly part time GPU crunch (WOW Event exception - max throttle) I have over the last 12 months rotated a few GPU's through the machines, and still have 3 sitting in the computer crate (in shed) for when these ones start to show signs of wear and tear lol. Happy Crunching Regards |
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
A PM pointed out to me that I omitted the ongoing electricity usage for this rig. Kill-A-Watt is showing me a usage fluctuating from about 510 to 550 watts. That's with all 4 GPUs and 8 CPUs averaging about a 95% load. I don't know how that wattage compares to some of the big rigs with the latest and greatest hardware, but it's the upfront cost that I was focusing on, inasmuch as some people here seem to have very low electric rates, while others trend toward the high end. |
Al Send message Joined: 3 Apr 99 Posts: 1682 Credit: 477,343,364 RAC: 482 |
Jeff, as a (sort of) comparison, my 6 GPU dual Xeon rig, with 5 750Ti's and a 980Ti thrown in for good measure, when I just now checked it, was pulling between 680-800 watts (it fluctuates constantly, though looked like the average while watching it for a minute or 2 was in the 725-755 range) out of the wall. I'm sure if I pulled the 980 out of the mix it'd probably drop by a good 100 watts or so. It will interesting to see the watts/work produced ratio once it's had a chance to stabilize a bit. |
petri33 Send message Joined: 6 Jun 02 Posts: 1668 Credit: 623,086,772 RAC: 156 |
My i7 draws 140W, mobo 50W, 4 pcs of 1080's draw a total of 4x140W (overestimated avg). Sum is 750W. The room is much cooler now than with the 780's or 980's. To overcome Heisenbergs: "You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones |
Al Send message Joined: 3 Apr 99 Posts: 1682 Credit: 477,343,364 RAC: 482 |
Lol, Petri, that is pretty much a perfect world scenario, in terms of power and especially output the 1080 seems to be a fairly primo. Now if we could all win the Powerball, well, load em up, boys! lol |
Zalster Send message Joined: 27 May 99 Posts: 5517 Credit: 528,817,460 RAC: 242 |
@Al, go for the hybrid 1080s if you win, that 1080 reference and/or gamers seem underpowered, at least that is my opinion... |
Al Send message Joined: 3 Apr 99 Posts: 1682 Credit: 477,343,364 RAC: 482 |
Naa, I think I'll bide my time and wait till Dec or so when the Ti versions supposedly might be coming out, and preferably Hydro Cooled versions! Hey, the things you can do when money is no object, eh? ;-) |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.