Building a 32 thread xeon system doesn't need to cost a lot

Message boards : Number crunching : Building a 32 thread xeon system doesn't need to cost a lot
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 12 · Next

AuthorMessage
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1784040 - Posted: 1 May 2016, 18:21:29 UTC - in response to Message 1784017.  

Oh, and a little OT, but Jason, could you head over to this thread and check out my post here and comment there on your thoughts of what might be going on? It seems a little weird, with such a large difference in systems that they would be so close in RAC. Thanks!


Done!
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1784040 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22200
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1784041 - Posted: 1 May 2016, 18:22:50 UTC

A lot of folks say that the 750 is a better cruncher than the 950 - I guess that the 750 is one of those "sweet spot" devices that come along every now and then.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1784041 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1784043 - Posted: 1 May 2016, 18:31:41 UTC - in response to Message 1784036.  
Last modified: 1 May 2016, 18:34:19 UTC

Hal, thanks for the advice, I will look into both the adapters I'd need to pull this off, and those cards you linked to. So this pretty much limits me to a single powered GPU, plus another like the bottom end 950 or 750.

Maybe it's because it is is the Z600, which is the little brother to the Z800, and has available a 1100w PSU. I have never seen the inside of that one, but I'm pretty sure that they have more GPU power options available, otherwise, why bother, eh? ;-)

It might be worth checking to see if the Z600 & Z800 use the same PSU design. You might be able to swap in the 850w or the 1100w PSU from a Z800.

EDIT: Scratch that. I just has a closer look at the images for both.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1784043 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1784048 - Posted: 1 May 2016, 18:44:15 UTC
Last modified: 1 May 2016, 18:46:20 UTC

The problem with the Z600 is the proprietary nature of the PSU-MB connections. I had a Z800 board connected to a standard PSU via an adapter I bought on eBay for south of $10 shipped, so I had all the PCIe plugs from my PSU available. If you can find one for the Z600, you will be golden. Of course, you will also need a standard case big enough for the MB (I used a Xigmatek Elysium).

For some info on this, http://andybrown.me.uk/2014/11/01/z800/

It was fun BUT: HP are a bunch of cheapskates - for the Z800 MB, although you can SLI WS graphics cards, you can't do it for regular gaming cards - they didn't buy a license for it. If you are thinking of using the Z600 part time as a gamer, better check to see if it will let you SLI GTX cards. Of course, I was able to run SETI on 2 GTX 960s, as SETI doesn't involve SLI.
ID: 1784048 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1784072 - Posted: 1 May 2016, 19:59:42 UTC

Wow, not much I can add to all the above, Al. But I think you'd be fine using a splitter adapter to power a second GPU. You can always wye a drive power lead so both legs aren't off the same source.
Wasn't an issue in my case, in that I was using 2x GTX750tiSC, which don't need added power and are surprisingly low in power consumption. Max consumption seems to range from 20-35w per 750. That 650w supply gives a fair amount of head room.
The old Quadros that came with these boxes (570, 3700, 3800) I simply sold off, as they're too weak to bother with.
ID: 1784072 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1784111 - Posted: 1 May 2016, 22:56:19 UTC

Good ideas, and Nope, if I plan on doing any gaming, it will be on my i7 CAD machine with the 980 and 700 series cards in it. This one is just for crunching. Interesting link about setting up a HP system board with a normal PSU. They make cables now specifically for that, though you have to get the 003 version of the board to be sure that you can run X56xx series procs guaranteed.

Was reading a little more about that idea, and came across an opposing viewpoint on it, by MusclesLinguine in the comments section of this thread. He made some pretty valid points, so I guess it depends on your ability to pick up the correct HP system board at the right price, otherwise it's just as cost effective and probably less hassle and frustration going the way he did.

As to my situation, it was mentioned that the 750 may be more effective crunching than the 950? If so, that is something I will have to look into.

ID: 1784111 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1784138 - Posted: 2 May 2016, 2:06:04 UTC - in response to Message 1784111.  

Good ideas, and Nope, if I plan on doing any gaming, it will be on my i7 CAD machine with the 980 and 700 series cards in it. This one is just for crunching. Interesting link about setting up a HP system board with a normal PSU. They make cables now specifically for that, though you have to get the 003 version of the board to be sure that you can run X56xx series procs guaranteed.

Great info about the power supply adapter cable. Was able to find a current listing from Aliexpress for the cable for $8. Good to know also that I've seen working Z600 power supplies used on eBay in the $20-40 range ...

Was reading a little more about that idea, and came across an opposing viewpoint on it, by MusclesLinguine in the comments section of this thread. He made some pretty valid points, so I guess it depends on your ability to pick up the correct HP system board at the right price, otherwise it's just as cost effective and probably less hassle and frustration going the way he did.

Not sure if that woulld really be the best approach, as opposed to the deals one can find on the full box. In my case, I wouldn't have the 2 Z400s and the Z600 had I not found full-system prices that were to good to beat. Just not worth the HP oddness factor otherwise.

As to my situation, it was mentioned that the 750 may be more effective crunching than the 950? If so, that is something I will have to look into.

I've got 8 750s spread across 4 PCs. If you take a peek at my stats, should give you a pretty good idea. Only the weakest box (the HP-8000CMT) does anything other than crunch 24x7, as it's my primary workstation for browsing and Solitaire. :)
ID: 1784138 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1784142 - Posted: 2 May 2016, 2:41:54 UTC - in response to Message 1784111.  

As to my situation, it was mentioned that the 750 may be more effective crunching than the 950? If so, that is something I will have to look into.

I was going to suggest you check the Stats tab on the SAH Main Web page, but it seems to be exploded right now (see Panic Mode thread) ...
ID: 1784142 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1784143 - Posted: 2 May 2016, 3:30:01 UTC - in response to Message 1784142.  

As to my situation, it was mentioned that the 750 may be more effective crunching than the 950? If so, that is something I will have to look into.

I was going to suggest you check the Stats tab on the SAH Main Web page, but it seems to be exploded right now (see Panic Mode thread) ...

The GTX950 is rated at 90W, the GTX750 at 55W & the GTX750Ti at 60W (there are meant to be some GTX950s rated at 75W but they appear to be rather thin on the ground).
I'd expect the GTX950 to do more work per hour than the 750 or even the 750Ti, but allowing for power consumption I'd expect the GTX750/750Ti to still be the leaders for power efficiency (the 75W GTX 950 would probably be pretty close to matching then).
Grant
Darwin NT
ID: 1784143 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1784147 - Posted: 2 May 2016, 3:50:04 UTC - in response to Message 1784143.  
Last modified: 2 May 2016, 3:55:21 UTC

As to my situation, it was mentioned that the 750 may be more effective crunching than the 950? If so, that is something I will have to look into.

I was going to suggest you check the Stats tab on the SAH Main Web page, but it seems to be exploded right now (see Panic Mode thread) ...

The GTX950 is rated at 90W, the GTX750 at 55W & the GTX750Ti at 60W (there are meant to be some GTX950s rated at 75W but they appear to be rather thin on the ground).
I'd expect the GTX950 to do more work per hour than the 750 or even the 750Ti, but allowing for power consumption I'd expect the GTX750/750Ti to still be the leaders for power efficiency (the 75W GTX 950 would probably be pretty close to matching then).

Gotta wonder about those power ratings. Assuming Malik's HWInfo is a reliable indicator, I've never seen any of my 750s exceed 40w power consumption at 100% GPU Core utilization. So either gaming loads these more than number crunching does, or they're very conservative on their power ratings ...

Hmmm, maybe I'll trot out some SOG Beta stuff and look at the loading under that ...

[Edit] Nope, still under 30w [/edit]
ID: 1784147 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1784161 - Posted: 2 May 2016, 5:59:32 UTC - in response to Message 1784147.  

I'd expect the GTX950 to do more work per hour than the 750 or even the 750Ti, but allowing for power consumption I'd expect the GTX750/750Ti to still be the leaders for power efficiency (the 75W GTX 950 would probably be pretty close to matching then).

Gotta wonder about those power ratings. Assuming Malik's HWInfo is a reliable indicator, I've never seen any of my 750s exceed 40w power consumption at 100% GPU Core utilization. So either gaming loads these more than number crunching does, or they're very conservative on their power ratings ...

They're the maximum supported power rating.
My GTX 750Tis, running at around 85-90% GPU load pull between 55-75% of their maximum power loading. One is clocked slightly higher than the other, so even though it's GPU & memory load is similar, it pulls another 4-6% more power than the slower clocked unit.

I'd expect a card running at normal clock speeds, with 100% GPU & Memory Controller loads would still pull less than the maximum possible power rating as the Video Engine section is doing nothing at all- and given they are crunching numbers, and not doing video work then other sections of the hardware that would normally be used during video gaming won't be in use for crunching.

If a card is overclocked, then you'll start running in to the power limits, even if GPU & Memory controller loads are less than 100% and other sections of the card are not in use.
Grant
Darwin NT
ID: 1784161 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 1784238 - Posted: 2 May 2016, 15:26:53 UTC
Last modified: 2 May 2016, 15:27:38 UTC

I have installed a Geforce GTX 750 OC on my Windows 10 PC and it works both in SETI@home and Einstein@home. An attempt to install a HD 7770 on my SuSE Linux Leap 42.1 box ended in disaster. It was not able to boot, after carefully following instructions. I had to change the disk, which luckily was an old 160 GB Hitachi disk and I installed a modern WD 1 TB disk, loading the OS from a DVD. This PC is an old SUN workstation of 2008 vintage, and probably its BIOS is too old. But I am running mostly CERN projects with Virtual Box on it, and I don't need a GPU.
Tullio
ID: 1784238 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1784451 - Posted: 3 May 2016, 7:30:04 UTC - in response to Message 1784161.  

I'd expect the GTX950 to do more work per hour than the 750 or even the 750Ti, but allowing for power consumption I'd expect the GTX750/750Ti to still be the leaders for power efficiency (the 75W GTX 950 would probably be pretty close to matching then).

Gotta wonder about those power ratings. Assuming Malik's HWInfo is a reliable indicator, I've never seen any of my 750s exceed 40w power consumption at 100% GPU Core utilization. So either gaming loads these more than number crunching does, or they're very conservative on their power ratings ...

They're the maximum supported power rating.
My GTX 750Tis, running at around 85-90% GPU load pull between 55-75% of their maximum power loading. One is clocked slightly higher than the other, so even though it's GPU & memory load is similar, it pulls another 4-6% more power than the slower clocked unit.

I'd expect a card running at normal clock speeds, with 100% GPU & Memory Controller loads would still pull less than the maximum possible power rating as the Video Engine section is doing nothing at all- and given they are crunching numbers, and not doing video work then other sections of the hardware that would normally be used during video gaming won't be in use for crunching.

If a card is overclocked, then you'll start running in to the power limits, even if GPU & Memory controller loads are less than 100% and other sections of the card are not in use.

Good info...
ID: 1784451 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1784819 - Posted: 4 May 2016, 22:57:41 UTC

Well, got that special SATA male to Molex female adapter today, and got the Z600 fired up with the 770 and a 750 installed, and best of all, no smoke was let out! Installed the latest vid drivers which then removed the older ones the seller had installed for the Quadro card, only installed the driver and the HD sound, nothing else, and it removed older versions of some ancillary EVGA stuff during the install that I didn't want polluting the system.

Installed Precision X16, it seems to be working properly, as I can control both cards, and just did a Disk Cleanup which removed 660 meg of unneeded crap off the hard drive. Defragging it now, then going to install BOINC and see what this gal's got in her. Fun Stuff!

ID: 1784819 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1784887 - Posted: 5 May 2016, 5:19:19 UTC - in response to Message 1784819.  

Well, ... got the Z600 fired up with the 770 and a 750 installed... Fun Stuff!

Nice!
ID: 1784887 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1784968 - Posted: 5 May 2016, 15:43:41 UTC

Jim, I have both Coretemp64 and Speedfan running on this system, have all the fans cranked up to 11(!), and am not very impressed with HP's extremely expensive (for what it appears you get) CPU coolers. Maybe the AS5 thermal paste needs to set up a little, but in core temp, they are running at 71-77* on both procs. This seems quite high to me, compared to other systems I have running, but to be honest, this is the only one in a case, the rest are just boards on trays out in the open. Oh, one other thing I should add, Speedfan and Coretemp don't agree on the CPU temps, Coretemp is giving the 70* readings on the cores, Speedfan only lists one temp, about 55* but doesn't break it out per CPU/core like Coretemp does, so I guess I don't know what to believe...

Almost all the systems/boards I've recently built have aftermarket cooling, mostly Megahelms with Noctua fans on all the quad cores (with a few that had OEM Intels on them when I bought them, and haven't bothered to remove them, they seem adequate though I find not nearly as effective as the aftermarket ones I use), water cooled H110 on the i7, and Noctua Xeon cooler and fans on the first dual X5690 I built. This Z600 system has the stock HP parts, the matching 2nd one for proc #2 was $60, and they don't cool within 15* of the other systems, and considering this is a bit lower clocked 6 core, I expected less heat, or at least no more than the fire breather X5690...

What are your temps in your Z600 running, are they comparable, or a bit more 'normal'? Thanks for any suggestions!

ID: 1784968 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1784992 - Posted: 5 May 2016, 16:43:40 UTC - in response to Message 1784968.  

Jim, I have both Coretemp64 and Speedfan running on this system, have all the fans cranked up to 11(!), and am not very impressed with HP's extremely expensive (for what it appears you get) CPU coolers. Maybe the AS5 thermal paste needs to set up a little, but in core temp, they are running at 71-77* on both procs. This seems quite high to me, compared to other systems I have running, but to be honest, this is the only one in a case, the rest are just boards on trays out in the open. Oh, one other thing I should add, Speedfan and Coretemp don't agree on the CPU temps, Coretemp is giving the 70* readings on the cores, Speedfan only lists one temp, about 55* but doesn't break it out per CPU/core like Coretemp does, so I guess I don't know what to believe...

Almost all the systems/boards I've recently built have aftermarket cooling, mostly Megahelms with Noctua fans on all the quad cores (with a few that had OEM Intels on them when I bought them, and haven't bothered to remove them, they seem adequate though I find not nearly as effective as the aftermarket ones I use), water cooled H110 on the i7, and Noctua Xeon cooler and fans on the first dual X5690 I built. This Z600 system has the stock HP parts, the matching 2nd one for proc #2 was $60, and they don't cool within 15* of the other systems, and considering this is a bit lower clocked 6 core, I expected less heat, or at least no more than the fire breather X5690...

What are your temps in your Z600 running, are they comparable, or a bit more 'normal'? Thanks for any suggestions!

Sounds like Speedfan is only reading the case/package temp. I haven't used Speedfan in many years as it just seems rather clunky to me.
Two other apps you might want to have a look at are:
HWMonitor from the people that make CPUz
http://www.cpuid.com/softwares/hwmonitor.html
HWInfo
http://www.hwinfo.com/
Both are available in portable versions so you don't have to mess around with installing them on each machines. You can just download, decompress, and copy to each system across the network.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1784992 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1785025 - Posted: 5 May 2016, 19:40:56 UTC - in response to Message 1784992.  

Hal, the only reason I'm actually using Speedfan is to control the speeds of the fans other than the GPU. If you have a suggestion of a better piece of software out there to do that, please let me know, as it certainly isn't the most user friendly, at least in terms of auto starting and coming back to the same settings you left it at, like Precision X 16 does.

ID: 1785025 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1785028 - Posted: 5 May 2016, 20:08:39 UTC - in response to Message 1785025.  

Hal, the only reason I'm actually using Speedfan is to control the speeds of the fans other than the GPU. If you have a suggestion of a better piece of software out there to do that, please let me know, as it certainly isn't the most user friendly, at least in terms of auto starting and coming back to the same settings you left it at, like Precision X 16 does.

I normally configure the fans in the BIOS. On some of my systems I am using Gigabytes EasyTune, but I suspect it may only work on their motherboards.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1785028 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1785068 - Posted: 5 May 2016, 23:03:57 UTC - in response to Message 1785028.  

Fortunately or unfortunately depending on how you look at it, they are mostly Intel boards (Sturdy/stable, but feature poor), and the extent of their BIOS adjustments is pretty much setting the fan to either normal or aggressive. Nothing in the way of fine tuning there, and of course I set them to aggressive, since they are running 2-3 of 4 cores full bore 24x7 so I'd think the cooler the better.

One thing you probably never thought about, and neither did I till I was sitting there with Speedfan open and sort of daydreaming for a minute while configuring it. I noticed that one of the fans was running 3900 RPMs and I was thinking about longevity.

3900rpm x 60min x 24hours x 365days = over 2 Billion revolutions a year. Pretty amazing they don't self destruct sooner than they do when you think about it.

Oh well, I will keep using it for now, until I find some better software that controls the CPU fan. I wonder if HP has some utility that I can try tuning the fans with? I'll have to put that on the list to look into.

ID: 1785068 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 12 · Next

Message boards : Number crunching : Building a 32 thread xeon system doesn't need to cost a lot


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.