Message boards :
Number crunching :
Building a 32 thread xeon system doesn't need to cost a lot
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 12 · Next
Author | Message |
---|---|
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
Fortunately or unfortunately depending on how you look at it, they are mostly Intel boards (Sturdy/stable, but feature poor), and the extent of their BIOS adjustments is pretty much setting the fan to either normal or aggressive. Nothing in the way of fine tuning there, and of course I set them to aggressive, since they are running 2-3 of 4 cores full bore 24x7 so I'd think the cooler the better. Yeah some motherboards don't have a lot in the way of fan configuration in the BIOS. Sometimes you don't even get Auto or Manual settings. For CPU fans I generally don't run them run any faster than is needed. AS I found that the old way of running high volume fans at full tilt constantly really didn't accomplish anything. Other than making a lot of noise. Now I tend to go with low RPM high volume fans. For the dual Xeon X5470 that I have half put together. Once I have the heatsinks mounted so they properly draw heat from the CPUs I'm planning to remove the fan from the CPU coolers. Then use either 2 140mm or 3 120mm fans to blow air across the MB. Possibly with a shroud, likely made from cardboard, to better channel the air over the bits that need cooling. Similar to the layout in 1U & 2U rack servers. At the moment I am using several 120mm or 140mm NZXT fans in my systems. Which run 1200 RPM, 59 CFM, & 27 dBA at full tilt. They spend most of their time around 800 RPM. As that seems to be as slow as they operate. SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Al Send message Joined: 3 Apr 99 Posts: 1682 Credit: 477,343,364 RAC: 482 |
I've been recently playing around with that as well, starting the GPU fan at 100%, and seeing under load what the operating temps are, and then backing them down maybe 5-10% per step and noting the temp differences. I've found in my limited testing so far that I can drop it from 100% to 70% and only pick up a degree or 2, and at 50% maybe 5-10 degrees, so I've decided to try running them between 60-70% for the time being, as the sound levels at 75% and below are much less noticeable. I've done the wind tunnel thing with thin cardboard on my Megahelms that I had bought 5 years ago when I started this farm project, I think they do make a little difference, but it's been a while since I've played with them. Well, if nothing else, this is fun playing with new systems and processors. As hobbies go, this one isn't too bad in either terms of initial or longer term outlays, but as the farm comes online, my opinion on that may change... |
Jimbocous Send message Joined: 1 Apr 13 Posts: 1853 Credit: 268,616,081 RAC: 1,349 |
Jim, I have both Coretemp64 and Speedfan running on this system, have all the fans cranked up to 11(!), and am not very impressed with HP's extremely expensive (for what it appears you get) CPU coolers. Maybe the AS5 thermal paste needs to set up a little, but in core temp, they are running at 71-77* on both procs. This seems quite high to me, compared to other systems I have running, but to be honest, this is the only one in a case, the rest are just boards on trays out in the open. Oh, one other thing I should add, Speedfan and Coretemp don't agree on the CPU temps, Coretemp is giving the 70* readings on the cores, Speedfan only lists one temp, about 55* but doesn't break it out per CPU/core like Coretemp does, so I guess I don't know what to believe... The Z600 has 2xE5620 @ 2.40GHz. HWinfo reports Core Max average at 63 and 55, with max of 69 and 62. First core will always seem to run about 5 higher, as the second core cooler blows right into it. When I got the machine, I did remove the fan from each cooler and pull out all the carpet fuzz that had built up over the years, and hit the vanes with compressed air. BTW, it is possible to remove the fans without removing the cooler from the CPU. Remove the top screws, loosen the lower ones if possible, then rotate the vertical bars enough out of the way that the fan can be slid up. Could try replacing the fans on the factory heat pipes, perhaps with Arctic PWM fans @$20 ea. My Z400's, unfortunately, are running in the hi 70's consistently. TJmax on these Xeons is somewhere north of 95c, so I'm not too concerned about anything under 80. I have these in the basement for the summer, and fans are cranked to the max in BIOS. In the winter, they heat my living room:) |
Jimbocous Send message Joined: 1 Apr 13 Posts: 1853 Credit: 268,616,081 RAC: 1,349 |
I wonder if HP has some utility that I can try tuning the fans with? I'll have to put that on the list to look into. I've looked, so far without success. Ever find anything, let me know. Seems silly that there isn't, but HP has the BIOS locked down so tight on these it's lucky we can look at it ;) |
Jimbocous Send message Joined: 1 Apr 13 Posts: 1853 Credit: 268,616,081 RAC: 1,349 |
I'm running Malik's HWInfo64, as it lets me watch all machines across the network and gives me a bunch of info about each machine, including GPU data. Might be worth a look. Very decent, especially for a freebie. I especially like that I can turn on widgets for the tool bar with color-coded data for each machine. |
Al Send message Joined: 3 Apr 99 Posts: 1682 Credit: 477,343,364 RAC: 482 |
My Z400's, unfortunately, are running in the hi 70's consistently. TJmax on these Xeons is somewhere north of 95c, so I'm not too concerned about anything under 80. I can certainly believe that, I can't believe how much heat that thing puts out. Maybe it is just appearances because it is concentrated coming out the back side of the case instead of sitting in the open, but even my dual X5690s don't feel like they are generating that much heat. Now that it is apparently working properly, I think I am going to NNT it for now, and when it's dry shut her down for the season. Once the free cooling starts again in about 5-6 months, I'll fire the ol girl back up and add her back into the mix. |
Phud Redux Send message Joined: 20 Apr 16 Posts: 270 Credit: 2,976,272 RAC: 1 |
i bought one E5-2660 over a month ago for 54 bucks. three weeks later i got a second one for 49 bucks. as i see it the only drawback is finding an X79/C602/C606 board. the seti system is paired with one Gtx 760. which is pumping out really well. |
Phud Redux Send message Joined: 20 Apr 16 Posts: 270 Credit: 2,976,272 RAC: 1 |
going to start the other E5-2660 on sunday, i think. our niece is graduating from university tomorrow, so sunday will be better. |
Zombu2 Send message Joined: 24 Feb 01 Posts: 1615 Credit: 49,315,423 RAC: 0 |
just grabbed a 96 core opteron system on ebay runs good and stable I came down with a bad case of i don't give a crap |
ML1 Send message Joined: 25 Nov 01 Posts: 20267 Credit: 7,508,002 RAC: 20 |
just grabbed a 96 core opteron system on ebay runs good and stable Yeah!!! Way to go!! Slightly jealous... I'm running some 32 core opteron systems to good effect. Very solid and consistent and reliable for server work even if not the hottest for number crunching. But then again: GPUs are always going to be the winners for number compute power... Happy fast crunchin Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13732 Credit: 208,696,464 RAC: 304 |
just grabbed a 96 core opteron system on ebay runs good and stable What's it's power load? Grant Darwin NT |
Zombu2 Send message Joined: 24 Feb 01 Posts: 1615 Credit: 49,315,423 RAC: 0 |
just grabbed a 96 core opteron system on ebay runs good and stable yeah but still 32 cores is nice too i got the newer 16 core opterons coming 2.5ghz 16 cores whith a grand total of 128 cores they where cheap too 35$ a piece ...yay for amd products and their rapid depriciation power load is 2.8kw at 100% cpu but the psu's are 2 phase 220v 9.8Amp per psu I came down with a bad case of i don't give a crap |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13732 Credit: 208,696,464 RAC: 304 |
power load is 2.8kw at 100% cpu but the psu's are 2 phase 220v 9.8Amp per psu 2.8kW! Is that actual load when running, or maximum rating for the unit? Grant Darwin NT |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
power load is 2.8kw at 100% cpu but the psu's are 2 phase 220v 9.8Amp per psu The spec sheet for the Dell PowerEdge C6145 says it can be equipped with either 1100W or 1400W redundant PSUs. I'm guessing they have the 1400w PSUs. All of my Dell servers typically would run 50-60% of the PSU rating when at full load & filled with HDDs. That is normally in the peak efficiency range for PSUs. So it makes sense the power supplies would be spec'd to run in that range. SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13732 Credit: 208,696,464 RAC: 304 |
The spec sheet for the Dell PowerEdge C6145 says it can be equipped with either 1100W or 1400W redundant PSUs. I'm guessing they have the 1400w PSUs. All of my Dell servers typically would run 50-60% of the PSU rating when at full load & filled with HDDs. That is normally in the peak efficiency range for PSUs. So it makes sense the power supplies would be spec'd to run in that range. So the actual load would be up to around 840W. That's still a lot of power for a single system. Before one of my video cards died, I had 2 systems with 4 video cards & 3 monitors; total load through the UPS was around 500W max. Grant Darwin NT |
Jimbocous Send message Joined: 1 Apr 13 Posts: 1853 Credit: 268,616,081 RAC: 1,349 |
Pretty wild when we get to where we lowly number crunchers have more under the hood than that project servers do :) |
bloodrain Send message Joined: 8 Dec 08 Posts: 231 Credit: 28,112,547 RAC: 1 |
wow price to performance is. very very good. |
Jimbocous Send message Joined: 1 Apr 13 Posts: 1853 Credit: 268,616,081 RAC: 1,349 |
wow price to performance is. very very good. ?? |
George 254 Send message Joined: 25 Jul 99 Posts: 155 Credit: 16,507,264 RAC: 19 |
Very tempted to get a tower unit instead of a stack of laptops. Yes purchase price to performance is good, but compared to laptops, running costs are high. My stack of 6 laptops use between 3 & 20 watts so with average of say 15 watts, that should give total usage of around 90 watts. Tasks vary between 30 and 65 per day with credits between 3,600 and 3,900 per day. Two machines run Rosetta as well as SETI@Home and I have just added Enigma to a separate machine as well. |
Cruncher-American Send message Joined: 25 Mar 02 Posts: 1513 Credit: 370,893,186 RAC: 340 |
Very tempted to get a tower unit instead of a stack of laptops. Yes - but one 32-thread machine (with no GPU work at all) will run 500+ WUs/day, drawing about 400 watts. So roughly 10 times the work done for about 5 times the cost. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.