Message boards :
Number crunching :
Multi-GPU hardware setups: My method (feel free to share your own!)
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 8 · Next
Author | Message |
---|---|
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
Ok, finally adding a picture of my 9 gpu hardware setup. I am perfectly aware that this is not a pretty rig. Ok, now it is showing up in this browser, but will it show up in someone else's browser. The resulting string looks a lot like the one Ian&SteveC created. Tom A proud member of the OFA (Old Farts Association). |
Ian&Steve C. Send message Joined: 28 Sep 99 Posts: 4267 Credit: 1,282,604,591 RAC: 6,640 |
Let’s see. Can you see this one? Change “open†to “uc†https://drive.google.com/uc?id=1PWJ5gCp2vWIz9kZ-w4FAWs835AI3wRZSRQ Seti@Home classic workunits: 29,492 CPU time: 134,419 hours |
Ian&Steve C. Send message Joined: 28 Sep 99 Posts: 4267 Credit: 1,282,604,591 RAC: 6,640 |
Ok, finally adding a picture of my 9 gpu hardware setup. I am perfectly aware that this is not a pretty rig. I think that method is generating a one time image link (nonce in the url) which is why we can only see the images we post but no one else. Seti@Home classic workunits: 29,492 CPU time: 134,419 hours |
Bill G Send message Joined: 1 Jun 01 Posts: 1282 Credit: 187,688,550 RAC: 182 |
and this message is not wrapping properly on my machine. SETI@home classic workunits 4,019 SETI@home classic CPU time 34,348 hours |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Let’s see. Can you see this one? This is the only post [Message 1982081] in this thread that displays an image in Chromium browser. Appears that changing “open†to “uc†is the trick to making the image display with the Google Drive shareable link. Just tried a post with that trick and it displays. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Wiggo Send message Joined: 24 Jan 00 Posts: 36853 Credit: 261,360,520 RAC: 489 |
...This is the only post [Message 1982081] in this thread that displays an image in Chromium browser.You probably did what I did Keith and opened that pic up through the url at some stage and it now shows, but opening the same page on my other PC (not logged in) just display busted links. ;-) But I will repeat what I said before, "Actually when using the image tags your link must end with a .jpg/.bmp/.png/etc for the image to be displayed." Cheers. |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
OK, thanks Wiggo for the clarification. I think you are correct. Google Drive does not give any explicit naming for images that have a normal image extension. I think that is the difference. When I hosted images on Flickr, the images had the normal image extensions and when I grabbed a link, the link still had the explicit naming convention of the image. Those worked. I will have to start reusing Flickr for hosted images again I think. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Can I suggest and ask please that contributors to this thread post their total system cost and monthly power usage and cost. Total System cost for 5 crunching computers = $19715 Monthly power usage = 3000 kWh Monthly cost = $600 Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Sirius B Send message Joined: 26 Dec 00 Posts: 24913 Credit: 3,081,182 RAC: 7 |
£460 per month equates to £5,520 per annum...£15,000? |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
Let’s see. Can you see this one? Yes :) Tom A proud member of the OFA (Old Farts Association). |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
Can I suggest and ask please that contributors to this thread post their total system cost and monthly power usage and cost. Ah, I would have to do some calculations on the power bill question. I am on a "level pay" plan that allows me to setup my banks bill paying service and forget it. Yes, it has gone down. I can give you a fairly good ballpark on the costs for my biggest box. Please understand that the cpus I am running are significantly more expensive than a pair of Intel e5-2670 v1's. The 2670's started hitting the used market when the data centers went through an upgrade cycle. They are now under $100USD per cpu on eBay. I paid about $450USD for the pair of cpus I have now (e5-2690 v2). The motherboard is an ASROCK that I paid under $300USD for. I got my 3 gtx 1070Ti's used for about $300 a piece on eBay. The gtx 1060 3GB's cost anyway from $250 down to $130 or less on eBay (used). The case is about $100 new. The PSU was used at $160 but new on sale it is ~ $300 The riser card kits and 1 to 4 expander boards totaled about $100~ Almost forgot the ram. Since this is DDR3 it is a bunch less expensive than DDR4 (as well as slower). And since I bought a lot of this system one piece at a time the upgrades were much less noticeable. Looks like I could re-create this system for about $2500+ or less. If I went with the cheaper e5-2670 v1's and nothing but gtx 1060 3GB's I should be able to do it from around $1,500+. It would cause me a pause if I forked out this cash at once. But since it was a "one piece at a time" type of project followed by "re-allocating" the hardware it was much less visibly painful. Tom A proud member of the OFA (Old Farts Association). |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
In English money that is £15,000 and £460 a month. When you are running AMD motherboards with DDR4 ram plus high end gpus (when new, especially) it is a bunch more expensive to buy. I have been consolidating all my gpus onto basically one box. So my major expense is in fact the gpus on one box. I think most of us have been upgrading one system at a time. Because of that, the cost of the "installment loan payment" makes the cost(s) of the system(s) less noticeable to our cash flow. I know that Ian&SteveC's last upgrade was mainly a new gpu-server motherboard (very expensive and rare) plus a miners rig platform with an extra sheet of Lucite. Pretty much everything else was parts from an existing system(s). Tom A proud member of the OFA (Old Farts Association). |
Ian&Steve C. Send message Joined: 28 Sep 99 Posts: 4267 Credit: 1,282,604,591 RAC: 6,640 |
besides the fact that the price of a system is irrelevant to this thread, it would only serve the purpose for some people to gloat about how they spent less. rather boring. RAC more or less scales to money invested. but a reasonable (mid-high end consumer GPUs, not enterprise tier or the majorly overpriced Titans) GPU focused system is both more cost and power efficient than a RAC-equivalent CPU based setup. think antony's 500+ systems are using less electricity or cost less than a GPU setup? doubt it. I don't have time to add up every little expense, and I don't really track my electric use, simply because this is a hobby and I can afford it. that's all that matters. no one in the top list of contributors is being financially reckless. but rough price list for the main system, which is inching towards being a 1,000,000 RAC system (one host, i'll post pics here later). many parts bought used and at various times. for stuff i cant remember price, i'll just put what they are currently worth in parentheses () mining rack/case/frame - $36 Acrylic sheet to custom mount MB - $18 Supermicro X9DRX+-F motherboard - $575 2x Xeon E5-2680v2 CPUs - ($300) 32GB (8x4GB) DDR3 Registered ECC ram - ($65) OS drive - 2x 250GB (RAID1, you can easily use just one drive) samsung SSDs I had laying around - ($100) 1000w Corsair PSU - $100 2x 1200w HP server PSU - $110 2x breakout boards with cables - $40 3x RTX 2070 - $1500 6x GTX 1080ti - $3000 1x GTX 1060 3GB - ($120) Total = $5864, or €5167 , or £4487 power use is estimated at ~1750w average $0.11/kWh in my area = $138.60 (€122.14/£106.10) per month prices can vary depending on the deals you find and the conditions of your market and how much your electric costs. Seti@Home classic workunits: 29,492 CPU time: 134,419 hours |
Sirius B Send message Joined: 26 Dec 00 Posts: 24913 Credit: 3,081,182 RAC: 7 |
besides the fact that the price of a system is irrelevant to this thread, it would only serve the purpose for some people to gloat about how they spent less. rather boring.Best comment seen on these boards in a long time. |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Hi Ian Just curiosity, you need 2 Xeon's to power all the GPU's or you could run with only one? Considering no CPU task are running on the hosts. |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
besides the fact that the price of a system is irrelevant to this thread, it would only serve the purpose for some people to gloat about how they spent less. rather boring. RAC more or less scales to money invested. but a reasonable (mid-high end consumer GPUs, not enterprise tier or the majorly overpriced Titans) GPU focused system is both more cost and power efficient than a RAC-equivalent CPU based setup. think antony's 500+ systems are using less electricity or cost less than a GPU setup? doubt it. I didn't think about the "gloating" aspect. So are you "gloaters" out there, you have been WARNED!!!! :/ No gloating allowed on this thread. Given how fast anything from a gtx 750Ti running Windows on up, processes compared to cpu-only processing I would be hard put to make a case for cpu only processing unless you were playing with "free" cpu/memory/motherboard hardware and solar-powered daytime only crunching. (aka: eng4hire). Tom A proud member of the OFA (Old Farts Association). |
Ian&Steve C. Send message Joined: 28 Sep 99 Posts: 4267 Credit: 1,282,604,591 RAC: 6,640 |
Hi Ian I'm am running CPU tasks on this host. but only 26 threads. 80% total CPU use between the CPU and GPU jobs but 2 CPUs are needed on this board to use all the PCIe slots/lanes. about half of the slots wouldn't work if you didnt have one of the CPUs installed. i could use 2x 4-core/8-thread Xeons and be fine with 16 total threads, but i already had these chips so i used them. i went this route because with -nobs, you need one full CPU thread, and all of the mining boards only support up to a 4-core/8-thread chip. running 7 GPUs pretty much maxes it out when running -nobs. Seti@Home classic workunits: 29,492 CPU time: 134,419 hours |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Yes I believe in the project, been here since 2001 continuously. I never intended to build this high a production farm, but I wanted to try my hand with Linux again so I built a computer to learn. Then a free workstation motherboard ended up in my lap from a donation from RueiKe. So that got built out. Then another free workstation motherboard ended up here again as a donation from RuieKe. That too got built out. Yes, I can afford the costs. This crunching farm just morphed into my latest hobby. I haven't sunk any recent monies into my other hobbies of high-end audio or astrophotography. Those had much higher purchase costs. So crunching is a "cheap" hobby compared to those I would say. I happen to know the exact cost of my hosts because I maintain their construction bill on Rigbuilder on Overclockers.net website. So easy to keep track of the build cost. I participate in the AMD threads there. https://www.overclock.net/forum/rigbuilder.php?do=rig&action=my-rigs I know my power usage and cost since I have extensive power generation and consumption monitoring with my PV solar plant. The generation is not near enough to cover the consumption now. So the much higher monthly power bill than when I started. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
TBar Send message Joined: 22 May 99 Posts: 5204 Credit: 840,779,836 RAC: 2,768 |
The point people should consider is how much can be Saved by running Multiple GPUs on One System. Cost of the GPUs are irrelevant since they are the same no matter which machine runs them. Consider My 12 GPU system. All 12 GPUs are running on just One machine instead of 3 or 4 different machines. Big savings.... The One machine was relatively cheap; CPU = $220 Power Supplies = $170 Ram = $120 Motherboard = $80 GPU Connectors = $100 Case = Ancient Compaq - Free Total = ~$690 The CPU is running around 50%, using about 50 watts to run the 12 GPU.... Big Savings. It's currently producing a RAC of around 387k from a machine with just One CPU running at half speed. |
Tom M Send message Joined: 28 Nov 02 Posts: 5126 Credit: 276,046,078 RAC: 462 |
The point people should consider is how much can be Saved by running Multiple GPUs on One System. Cost of the GPUs are irrelevant since they are the same no matter which machine runs them. Right on there. More bang for the buck. That is why I have been playing with more gpus on a single motherboard and/or box. I would not have gone out and bought this all at once. Just incremented my way into it. Tom A proud member of the OFA (Old Farts Association). |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.