Message boards :
Number crunching :
Buying a Supercomputer?
Message board moderation
Author | Message |
---|---|
C Horse Send message Joined: 25 Jul 18 Posts: 5 Credit: 10,361,509 RAC: 1 |
Hello, I'd like to buy a "supercomputer" for the main purpose of contributing to research like SETI and medical projects. I'm also somewhat interested in using it for VR, because why not. I have an approximate budget of 20-30k, and have a few questions: -How wasteful of my funds is this? Would building my own Linux cluster be 2x cheaper? 5x? -What computer should i buy to get the most bang for my buck? I was looking at the "Silverdraft Power Resolve Workstation"... but not sure if I could stretch my dollars further with a different computer. -Does this invite a security risk? I was once told by a friend that having such a powerful computer can attract the attention of hackers who would like to take control of the machine for mining bitcoin or whatever else. Thanks for the help! C |
arkayn Send message Joined: 14 May 99 Posts: 4438 Credit: 55,006,323 RAC: 0 |
Hello, Looking at that machine that they want $33,500+Tax for, I decided to price out the components and add a few other items as well. I can build pretty much that same computer, add two 31.5" 4K monitors, and faster ram for a quarter of the price they want. Buy another of the same case, more GPU's and another PSU and you still will not be even half their price. |
Zalster Send message Joined: 27 May 99 Posts: 5517 Credit: 528,817,460 RAC: 242 |
+1 |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Excellent choises i only wish to add: - Instead of the Air cooled EVGA 1080Ti, use Hybrids of the same model: https://www.amazon.com/EVGA-GeForce-HYBRID-GAMING-Technology/dp/B074D7S8HR/ref=sr_1_2?ie=UTF8&qid=1534673406&sr=8-2&keywords=evga+hybrid+ftw3+1080ti&dpID=512khJjnmQL&preST=_SY300_QL70_&dpSrc=srch They cost a little more but runs a lot cooler and with less air flow problems due the use of the 4 GPU's. Their radiators fits perfect on the top of the Thermaltake X9 Cube making your entire system running a lot cooler. or you can wait until the next week announcement of the RTX2080 series...... who are expected to be a lot faster GPU. - You realy not need more than 16 GB to put this baby to crunch SETI, specialy on Linux (top SETI crunchers runs on Linux), so 64 GB are not realy necesary , unless you have another use for it. - A Top SETI cruncher runs on Linux, so you not need to buy a Windows 10 unless you plan to use it for another use. My 0.02 |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
I was going to say that 4 air cooled cards will never fit in 2.0 slot width, without water blocks, HyroCopper or AIO cards. Full dual loop water cooling with 3 x 360 rads (or 4) would be nice and cool. But add ~4k to that bill. |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
Hello, The first thing I would like to ask is exactly which Medical Projects? A purely Seti oriented machine would be heavily biased towards Gpu processing because we have really fast processing across multiple OS as well as things like the CUDA90 project aka: secret sauce. Nearly all the "medical projects" I have looked at are CPU-based. So the more fast cores/hyper-threading you have the more production you would get. You need to ask yourself if you are a "price is no object" person or if you are a "I want the most bang for the least buck" person. This is a link from the Seti website to the top producing machines as well as the top producing individuals. https://setiathome.berkeley.edu/stats.php In the other responses you received a top end (price is no object) system was proposed. You may find that used locked slower cpus with high core counts (8c/16t or 10c/20t or 12c/24 etc) will actually give you excellent results. Used gtx 1080Ti's can be had on eBay for $500-$600. Used e5-2670v1 2.6Ghz CPU's (8c/16t) can be had for $100~ on eBay Run them on dual CPU (server) ~$400 MB's (Please note, this is my "hobby" horse so I am offering biased opinion here). My own experience with SSD has been a massive speed up in booting up speeds. If you want it to run as cool as possible you may want to go with an "open air" Coin mining "case." You can buy them or get busy and make them out of wood. This would also allow you to run (probably) more than 4 gpus. They don't look especially pretty however. You can also buy "turn key" machines like you previously mentioned. These have the benefit of you not having to assemble all the parts and some kind of warranty. There are "server" companies out there that will build a custom server to your specifications. --------------------------------------- You asked about security and being a target for hackers etc. If you don't go around bragging about your Super Computer (or Super Computer Cluster) and you use adequate security firewalls or even "security appliances" you should be OK. A lot of trouble is caused by going to "bad" websites and gaining an infection from them. --------------------------------------- VR is certainly interesting but you may find that your machine is just a little stressed when you try that at the same time you are Seti processing. Btw, you should NOT setup multiple gpus in "Crossfire" (I think that is one name for linking gpus for better gaming) for Seti processing. It is possible that VR benefits from "Crossfire" so you would have to choose. The "Crossfire" I am thinking of is purely hardware based with links between the gpus. You may find yourself with high electrical bills. HTH, Tom A proud member of the OFA (Old Farts Association). |
C Horse Send message Joined: 25 Jul 18 Posts: 5 Credit: 10,361,509 RAC: 1 |
Thank you all so much for the help. I think i'll probably look into some local help assembling the pieces, since these computers can be built for 1/4 the price of of buying one pre-assembled one (as mentioned in the first response I received). I'm going to do some research into who might be able to assemble something in my neighborhood. Thanks again C |
C Horse Send message Joined: 25 Jul 18 Posts: 5 Credit: 10,361,509 RAC: 1 |
I'm mostly interested in SETI. Running VR was an after thought, and it's ok if the machine isn't optimal for medical research. I look for server companies like you mentioned. Hopefully they aren't as much of a rip off. This is the first one I found: https://www.pssclabs.com/servers/hpc-servers/?gclid=Cj0KCQjwquTbBRCSARIsADzW88x1B9Xsg9Yaoi0ChnxpgNKKpBEKiO43qkBRWP-61_OeqeRsL1Koh_AaApHXEALw_wcB If you were to build an ideal machine for SETI, that is likely in the neighborhood of 20-30k, what components would you choose? I'm hoping im not paying a 4x premium from these guys... I'm fine with paying a little extra to avoid building it, but 4x seems a bit much. Thanks for the help! |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
You could likely get 80% of what you need right from EVGA.com, but that Thremaltake X9 case is a goodie for a big build. Also a good liquid cooler for a big CPU if you go huge. AIO coolers are fine, but not really designed for high power 24/7. |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
I'm mostly interested in SETI. Running VR was an after thought, and it's ok if the machine isn't optimal for medical research. How about multiple machines that are "a bit less than $20 a pop" and then buy/make several :) I remember someone proposing a seti oriented box that had a fast i5 4 core no hyper-threading and 4 gtx 1080Ti's plus all the stuff you need like a big box, lots of cooling, big PSU etc. Possibly as little as 16 GB of memory? Basically it is recommend you use one core per video card in the NVidia world. Now the 1080's would be running 3 tasks in parallel each. So that would give you 12 gpu-based processing routes. To make it run "red hot" (aka really, really fast) you would need to be trying to run the "all in one Cuda" that Tbar has. I don't think it is bleeding edge being 2-3 minor updates behind but it has been shown to be easily installed under Lubuntu (that is what I am running and will be running for my bigger box once its fixed) and several other versions of Linux. If you are a skilled/experience Linux person you shouldn't have a problem running it (yes I saw Darwin). I personally would prefer a lot of cores as well as a lot of gpu's. That way if I want to start doing much medical processing I have the cores to do it and if not, yes Seti CPU processing is still a major stream in the entire programs production. I have see several 32 core machines running with no GPU at all ("headless"?). They were "suffering" but still generating respectable numbers. I am guessing if you wanted to buy a single machine you either need to go with the top end Threadripper 2 MB cpu with 32c/64t or investigate the dual cpu MB for the very expensive/high end Intel cpus. What is confusing is the really high count CPU's that Intel sells mostly live in "Server Racks" where they have no space for a GPU at all. The server company URL you posted has that issue. Its boxes don't have any place for a Gpu so you would need to see how many gpu slots they have and run the gpus via "riser" cables (not necessarily big flat ribbons). Would it be possible to repurpose a Bit Coin Mining MB? ASUS has released one. https://www.newegg.com/Product/Product.aspx?Item=N82E16813119028&ignorebbr=1&nm_mc=KNC-GoogleAdwords-PC&cm_mmc=KNC-GoogleAdwords-PC-_-pla-_-Motherboards+-+Intel-_-N82E16813119028&gclid=EAIaIQobChMInOiV8Zn63AIVELnACh0hFwjYEAYYASABEgJPRPD_BwE&gclsrc=aw.ds I do wonder if you could really run Seti GPU processing using the 19 gpu MB from Asus. ( As I understand it, you can only run 16 gpus (8 Nvidia, 8 AMD/Ati) because of the driver limits. The cpu socket is late model so you could run a high end, fast, large core count intel cpu. But you might need to spread the gpus over more than one core. Lets see. 6 X Gtx 1080 Ti X 3 parallel tasks = 18 parallel tasks. 6 X AMD Radeon (TM) R9 390 Series X 3? parallel tasks = 18 parallel tasks plus probably some CPU tasks. Its enough to make you drool ;) You might go with the original proposed system above except buy used Gtx 1080 Ti for near $500 rather than list price new ones at $1,000+ HTH, Tom A proud member of the OFA (Old Farts Association). |
Ian&Steve C. Send message Joined: 28 Sep 99 Posts: 4267 Credit: 1,282,604,591 RAC: 6,640 |
I was going to say that 4 air cooled cards will never fit in 2.0 slot width, without water blocks, HyroCopper or AIO cards. you can do it with all 4 cards in a single loop, with a single radiator. Watercool MO-RA3 (either 360 [9x120] or 420 [9x140] version) I'm using one to cool 2x1080ti and 2xE5-2697v2 12-core all in the same loop. ---------------------------------------------------------------------------------------------------------------------- but in response to the general replies of "build your own" here: looks into used server hardware. server hardware is extremely reliable, and it can be had cheap on the secondary markets (ebay, reddit, etc) for CHEAP because of lack of demand. if you want to run 4 cards plugged into a motherboard, you need water cooling ideally, or at the least some hybrid AIO cards. if you try to run anything more powerful than a 1050ti in a 4x config stacked next to one another, you will roast the cards. i have 2 systems with 2x1060s size by side and i have to use extremely powerful and load server exhaust fans + 80+% fan speed on the top card just to keep it below 80C. it will only get worse with more powerful cards. if you really want to maximize SETI output from a single box, at maybe the expense of speed on other projects (depending how much they care about PCIe bandwidth), just run an open air rig, and put the cards on risers. you can get 6-8 cards on a single board. might need to run multiple power supplies and it wont look pretty. i have no problems feeding SETI WUs all the way down to a PCIe x1 interface, even on a 1080ti. like i said, other projects might care more about it, but SETI does not. my most powerful machine is actually a system with Linux OS + optimized SETI app a single 4c/8t xeon e3 CPU single stick of 8GB ram cheap server case (but long) Rosewill RSV-L4500 1x1080ti 5x1060 mining risers carrying PCIe signals over USB 3.0 cable to a 1x interface 2x HP 750w server PSUs to power the cards and CPU 8-pin plug 1x 160w picoPSU to power the MB 24-pin 6x noctua iPPC 2000 fans to pull air across the GPUs basically this: it does like 300,000 credits/day Seti@Home classic workunits: 29,492 CPU time: 134,419 hours |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
I think Rosetta@Home is the only cpu only based medical distributed computing project still around. Folding@Home, World Community Grid and GPUGrid.net for example have gpu based applications as well as cpu. Gpu applications are much more productive on a watt per hour basis compared to cpu unless you factor in low power computing on Raspberry Pi type of hardware. So I only can see a mega core cruncher being useful at Rosetta while a gpu based cruncher would be useful at many more medical focused projects. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
mmonnin Send message Joined: 8 Jun 17 Posts: 58 Credit: 10,176,849 RAC: 0 |
I think Rosetta@Home is the only cpu only based medical distributed computing project still around. Folding@Home, World Community Grid and GPUGrid.net for example have gpu based applications as well as cpu. Gpu applications are much more productive on a watt per hour basis compared to cpu unless you factor in low power computing on Raspberry Pi type of hardware. So I only can see a mega core cruncher being useful at Rosetta while a gpu based cruncher would be useful at many more medical focused projects. WCG does not have a GPU app. Years ago it did but not any more. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13736 Credit: 208,696,464 RAC: 304 |
If you were to build an ideal machine for SETI, that is likely in the neighborhood of 20-30k, what components would you choose? That would build up to 10 high performance systems. No single system is worth that much money unless it is being used in a data centre with very specific duties. Grant Darwin NT |
tullio Send message Joined: 9 Apr 04 Posts: 8797 Credit: 2,930,782 RAC: 1 |
GPUGRID is a medical project and makes use of GPUs (nVidia) both in Windows and Linux. Besides this,it runs Computer Chemistry tasks on CPUs, only in Linux. Tullio |
Ghia Send message Joined: 7 Feb 17 Posts: 238 Credit: 28,911,438 RAC: 50 |
Wouldn't a NVMe disk (or two) be better than SATA ? The EVGA X299 Dark has room for 2 of them...I know because I have that mobo. And btw...would you recommend 1 large disk, or 2 smaller ? Like 1 for the OS and one for apps ? Humans may rule the world...but bacteria run it... |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
Wouldn't a NVMe disk (or two) be better than SATA ? When I can afford it, I like to run paired disks on Raid1. Then when one dies its less trouble to "rebuild" the drive. Tom A proud member of the OFA (Old Farts Association). |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
+1 A proud member of the OFA (Old Farts Association). |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
basically this: it does like 300,000 credits/day Thank you for the picture. Now I get it. That "partition" in the middle of the case is the fans "drawing air from the gpus" that you referenced. It must have hit a "bump" its doing "only" 150,000+- RAC at the moment ;) And its not listing the Gtx 1080Ti at all? Tom A proud member of the OFA (Old Farts Association). |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
If you were to build an ideal machine for SETI, that is likely in the neighborhood of 20-30k, what components would you choose? +1 A proud member of the OFA (Old Farts Association). |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.