Building a 32 thread xeon system doesn't need to cost a lot

Message boards : Number crunching : Building a 32 thread xeon system doesn't need to cost a lot
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next

AuthorMessage
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829140 - Posted: 8 Nov 2016, 7:10:52 UTC
Last modified: 8 Nov 2016, 7:17:43 UTC

Hi,

I noticed that some of you had bought the Natex S2600CP2J boards with Xeon E5-2670's, I wondered if I could ask a question about this as I have just built one of these?

(Aplogies about the size of the images; I am not sure how to force them to be smaller via this forum)



Everything has put together well and seems to be working okay except that the CPUs are getting real hot, 80c-90c; the Heat sinks and fans are spinning, and are plugged into CPU_fan headers.


I am using the: Noctua NH-D9DX i4 3U 92mm Intel Xeon CPU Cooler's.

I thought maybe this was because it was out of the box, so put the MB into the box but it has not made any difference. The case front fan is pulling, and back fan is pushing out air.



The problem seems to be that that CPU fans are not spinning fast enough to expel enough air. The case fan hub cable is plugged into Sys_fan_7, the only 4 pin fan header left (after cpu_fan1 and cpu_fan2).

The only thing I have Not done is update the BIOS.

Could anyone who has built one of these offer some advice on how I can get the CPUs to a normal temperature?

Is this related to not updating the BIOS?

Thank you for any help you can offer me..

Chris
ID: 1829140 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1829141 - Posted: 8 Nov 2016, 7:17:18 UTC - in response to Message 1829140.  

May I ask the obvious? Did you put the thermal paste on properly?
ID: 1829141 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829142 - Posted: 8 Nov 2016, 7:19:04 UTC - in response to Message 1829141.  
Last modified: 8 Nov 2016, 7:21:07 UTC

Hi Brent,

May I ask the obvious? Did you put the thermal paste on properly?


Yeah I believe so, about a pea sized amount on both heat-sinks.

The heat sinks are getting real hot so I guess there is heat transfer from the the CPU into the sinks, plus the fans are expelling hot air.

Chris
ID: 1829142 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1829143 - Posted: 8 Nov 2016, 7:28:19 UTC - in response to Message 1829140.  

Hi,

I noticed that some of you had bought the Natex S2600CP2J boards with Xeon E5-2670's, I wondered if I could ask a question about this as I have just built one of these?

(Aplogies about the size of the images; I am not sure how to force them to be smaller via this forum)
I usually just resize them before posting to the web.


The problem seems to be that that CPU fans are not spinning fast enough to expel enough air. The case fan hub cable is plugged into Sys_fan_7, the only 4 pin fan header left (after cpu_fan1 and cpu_fan2).

The only thing I have Not done is update the BIOS.

Chris

Am wondering if there are fan controls in the BIOS that need to be cranked up? I know I needed to on my HP and Gigabyte Tech mobos ...
ID: 1829143 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1829144 - Posted: 8 Nov 2016, 7:30:13 UTC
Last modified: 8 Nov 2016, 7:30:27 UTC

My obvious questions are

why aren't you connected into the CPU fan header? Can't you move whatever is plugged in there to the system fan header?

Since you are using a system fan header, can't you increase the fan speed via the BIOS page?
ID: 1829144 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1829145 - Posted: 8 Nov 2016, 7:34:57 UTC

TCASE for that CPU is 80.0°C at 80-90C you should be well into thermal throttling.

What is you idle temp?
ID: 1829145 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829146 - Posted: 8 Nov 2016, 7:36:37 UTC
Last modified: 8 Nov 2016, 7:39:43 UTC

Hi Zalster and Jimbocous,

At present I have no graphics software to resize the images.

Due the BIOS being the first version they does not seem to be any options on changing fan speeds as there is in other BIOS setups that I have seen.

Sorry perhaps I did not explain very well; Both Xeons CPUs are plugged into CPU_fan_1 and CPU_fan_2 headers, respectively.

I have plugged the Case fan hub into Sys_fan_7 header as the other sys_fan_N headers are all 3 pin but the case lead is a 4 pin. Sys_fan_7 takes a 4 pin lead.

Chris
ID: 1829146 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829147 - Posted: 8 Nov 2016, 7:37:20 UTC - in response to Message 1829145.  
Last modified: 8 Nov 2016, 8:01:35 UTC

Hi Brent,

TCASE for that CPU is 80.0°C at 80-90C you should be well into thermal throttling.

What is you idle temp?


Idle temp is around 55 which seems high.

Chris
ID: 1829147 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1829148 - Posted: 8 Nov 2016, 7:39:19 UTC - in response to Message 1829146.  

Hi Zalster and Jimbocous,

At present I have no graphics software to resize the images.

Due the BIOS being the first version they does not seem to be any options on changing fan speeds as there is in other BIOS setups that I have seen.

Both Xeons are using CPU_fan_1 and CPU_fan_2 headers. I have plugged the Case fan hub into Sys_fan_7 header as the other sys_fan_N headers are all 3 pin but the case lead is a 4 pin.

Chris


Could you use a 4 pin splitter to split 1 of the CPU between the Xeons? Leave the other free for the Case Fan hub?
ID: 1829148 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829150 - Posted: 8 Nov 2016, 7:44:26 UTC - in response to Message 1829148.  
Last modified: 8 Nov 2016, 7:45:14 UTC

Hi Zalster,

Could you use a 4 pin splitter to split 1 of the CPU between the Xeons? Leave the other free for the Case Fan hub?


Err, I am not sure. I guess so if I bought a splitter.

Surely the problem is not with the case fans, but with the CPU fan(s) not spinning quick enough?

CPU Fan 1 is going into CPU_Fan_1 header, and CPU Fan 2 is going into CPU_Fan_2 header which to me would seem logical?

Can I ask, why would you think this would make a difference, surely Sys_fan_7 is also reading temps from the MB and possibly CPU temps?

I am very new to all this as this is the first machine I have built!


Thanks,

Chris
ID: 1829150 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1829151 - Posted: 8 Nov 2016, 7:52:16 UTC - in response to Message 1829150.  

Hi Zalster,

Could you use a 4 pin splitter to split 1 of the CPU between the Xeons? Leave the other free for the Case Fan hub?


Err, I am not sure. I guess so if I bought a splitter.

Surely the problem is not with the case fans, but with the CPU fan(s) not spinning quick enough?

CPU Fan 1 is going into CPU_Fan_1 header, and CPU Fan 2 is going into CPU_Fan_2 header which to me would seem logical?

Can I ask, why would you think this would make a difference, surely Sys_fan_7 is also reading temps from the MB and possibly CPU temps?

I am very new to all this as this is the first machine I have built!


Thanks,

Chris


Sorry Chris, I was getting confused by this statement

The problem seems to be that that CPU fans are not spinning fast enough to expel enough air. The case fan hub cable is plugged into Sys_fan_7, the only 4 pin fan header left (after cpu_fan1 and cpu_fan2).


For some reason I thought you were saying the CPU fans were not speeding up enough and had it connected to the system fan header.

You have it connected correctly. Forget my suggestion.
ID: 1829151 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829152 - Posted: 8 Nov 2016, 8:04:41 UTC - in response to Message 1829151.  
Last modified: 8 Nov 2016, 8:33:50 UTC

You have it connected correctly. Forget my suggestion.


No worries.

This is why I was hoping to find someone that had built one of these boxes via this Motherboard S2600CP2J as then hopefully they could spot the problem.

I very much appreciate all the help you guys have offered so far though.

I think it all may be related to not updating the BIOS as I think that may fix and add options to the BIOS.

Thanks again,

Chris

In-case anyone else has this problem, and wants a quick fix to crunch..

Turning off Hyper-theading made no difference, and turning off Turbo made little difference. About the only way to keep temps low, espically when some projects that really stress the CPU (PG) start crunching WU's, is to drop % CPU usage down from 100%.
ID: 1829152 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1829158 - Posted: 8 Nov 2016, 8:39:42 UTC - in response to Message 1829146.  

At present I have no graphics software to resize the images.

You're running Windows. MSpaint can do the job.
ID: 1829158 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1829161 - Posted: 8 Nov 2016, 8:55:51 UTC - in response to Message 1829152.  

You have it connected correctly. Forget my suggestion.


No worries.

This is why I was hoping to find someone that had built one of these boxes via this Motherboard S2600CP2J as then hopefully they could spot the problem.

I very much appreciate all the help you guys have offered so far though.

I think it all may be related to not updating the BIOS as I think that may fix and add options to the BIOS.

For now I have just turned off Hyper-threading and if necessary, will drop % of CPU usage via Boinc.

Thanks again,

Chris

I just finished putting the same MB & CPU config together on Saturday.

Before the heatsinks arrived on Saturday I had fired it up a few times with some old sock604 heatsinks sitting on top of the CPUs with no thermal compound or fans. Just so I could see it fire up and install an OS. At which time the temps would hover near throttling.

The BIOS my MB arrived with was fine, but lacked the option for Turbo. Which is really the only reason I updated it to 02.06.0005.
Special note on updating the firmware. After following the update processes. Clear the BIOS using the reset jumper. Then let the system do its thing. I was sure for a few minutes that I nuked the BIOS. I had some beeping and a "no system memory" message before I cleared the BIOS. After the BIOS was cleared it did some beeps, took a long time to boot, but was fine.

Once I installed the Nocturas I've had no thermal issues. Even with turbo keeping all of the cores running 3.0GHz the heatsinks are not really even warm to the touch. The CPUs are between 50-55ºC running flat out with HT on and the fans right around 1000RPM.
If you are seeing those kind of temps at idle something seems wonky. Like perhaps the heatsinks are not secured correctly or something, but if the heatsinks are getting warm that would point to them making good contact.

As a side note I normally use MS Paint to resize my photos before I upload them.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1829161 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1829162 - Posted: 8 Nov 2016, 8:59:36 UTC - in response to Message 1829146.  
Last modified: 8 Nov 2016, 9:14:55 UTC

At present I have no graphics software to resize the images.

I've been using ACDSee v3.1 for years, as it came bundled with my first digital camera. Should be able to get it here.

Nice, basic, intuitive, well behaved, and works nicely on everything from Win2k up to and including Win10. Wonderful for doing quick fixes to photos; dunno how I would live without it.
ID: 1829162 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829165 - Posted: 8 Nov 2016, 9:07:37 UTC
Last modified: 8 Nov 2016, 9:19:11 UTC

To everyone posting suggestions on how to resize images.. Thanks.

However, the edit button is now no longer visible on that post, and hence I cannot change the picture size.

Next time I post I will make sure I resize first, I just assumed there would be a size property setup for the tag used in this forum.

Chris
ID: 1829165 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829166 - Posted: 8 Nov 2016, 9:18:44 UTC - in response to Message 1829161.  
Last modified: 8 Nov 2016, 9:21:07 UTC

Hi HAL9000,

Thanks for the reply.


The BIOS my MB arrived with was fine, but lacked the option for Turbo. Which is really the only reason I updated it to 02.06.0005.
Special note on updating the firmware. After following the update processes. Clear the BIOS using the reset jumper. Then let the system do its thing. I was sure for a few minutes that I nuked the BIOS. I had some beeping and a "no system memory" message before I cleared the BIOS. After the BIOS was cleared it did some beeps, took a long time to boot, but was fine.

Once I installed the Nocturas I've had no thermal issues. Even with turbo keeping all of the cores running 3.0GHz the heatsinks are not really even warm to the touch. The CPUs are between 50-55ºC running flat out with HT on and the fans right around 1000RPM.

If you are seeing those kind of temps at idle something seems wonky. Like perhaps the heatsinks are not secured correctly or something, but if the heatsinks are getting warm that would point to them making good contact.


I just checked and my BIOS and it is: v1.02.0003 so a much lesser version than the one you updated to: v2.006.0005

Can I ask..

1) Did you update to the next (slightly higher) BIOS update, or just update straight to the latest and highest version BIOS update?

2) Also, what software are using to monitor CPU fan speeds?

3) And lastly, did you monitor CPU temp after you installed the Nocturas, but before you updated the BIOS, or did you not do it in that order?

Thanks HAL, hopefully I can get this finally sorted.

Chris
ID: 1829166 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1829168 - Posted: 8 Nov 2016, 9:24:07 UTC - in response to Message 1829165.  

... I just assumed there would be a size property setup for the tag used in this forum.

If there is, I've never found it, and I've looked :)
l8r
ID: 1829168 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1829225 - Posted: 9 Nov 2016, 2:57:31 UTC - in response to Message 1829166.  

Hi HAL9000,

Thanks for the reply.


The BIOS my MB arrived with was fine, but lacked the option for Turbo. Which is really the only reason I updated it to 02.06.0005.
Special note on updating the firmware. After following the update processes. Clear the BIOS using the reset jumper. Then let the system do its thing. I was sure for a few minutes that I nuked the BIOS. I had some beeping and a "no system memory" message before I cleared the BIOS. After the BIOS was cleared it did some beeps, took a long time to boot, but was fine.

Once I installed the Nocturas I've had no thermal issues. Even with turbo keeping all of the cores running 3.0GHz the heatsinks are not really even warm to the touch. The CPUs are between 50-55ºC running flat out with HT on and the fans right around 1000RPM.

If you are seeing those kind of temps at idle something seems wonky. Like perhaps the heatsinks are not secured correctly or something, but if the heatsinks are getting warm that would point to them making good contact.


I just checked and my BIOS and it is: v1.02.0003 so a much lesser version than the one you updated to: v2.006.0005

Can I ask..

1) Did you update to the next (slightly higher) BIOS update, or just update straight to the latest and highest version BIOS update?

2) Also, what software are using to monitor CPU fan speeds?

3) And lastly, did you monitor CPU temp after you installed the Nocturas, but before you updated the BIOS, or did you not do it in that order?

Thanks HAL, hopefully I can get this finally sorted.

Chris


I forget which BIOS my MB came with, but it was a v1.something. I figured why not try the newest v2.x BIOS first. Since there were release notes indicating that a staggered update was needed.

Mostly I use HWMonitor for quick temp monitoring. It isn't great for long term. As the values tend to go wacky after a while. Unless I have fans that intermittently run at 50,000RPM.
If I want to monitor specific things and/or log the data. Then I'll use HWINFO. It seems to do a pretty job and is customize able. http://i.imgur.com/N24WAu1.png

My temps were probably only changed in that they likely went up a bit once I updated the BIOS. Gaining the use of Turbo bumped the CPUs from 2.6GHz to 3.0GHz. I was focused more on the core temps than the package originally.
Are your CPUs running at 2.6GHz, or a different speed? Perhaps your older BIOS is trying to constantly push them to their full 3.3GHz Turbo freq? Which would not make cooling easy in any way.

Today the system is running a few degrees warmer than normal. It was a warm sunny day and I had all of the windows shut. I found the thermostat displaying 76º when I got home from work. The system is sitting in a lot a floor above the thermostat. I'd expect it to be a few degrees warmer up there in general.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1829225 · Report as offensive
Profile OVERTAKE_ME Project Donor
Avatar

Send message
Joined: 20 May 16
Posts: 21
Credit: 458,341
RAC: 0
United Kingdom
Message 1829400 - Posted: 9 Nov 2016, 18:51:43 UTC - in response to Message 1829225.  
Last modified: 9 Nov 2016, 18:55:32 UTC

Hi HAL,

Thanks for your reply.

I was hoping you were going to say your CPU temps were running really high until you updated the BIOS :(

Currently my Xeon's are running at 2.9 GHz (3Ghz) with Turbo turned on, so not at the full 3.3 GHz.

If the Bios is the not the problem then perhaps it is down to the way I have fitted the heat sinks/fans to the CPU's.

With 100% load across all CPU's and one GTX 970, the CPU core temps run at 88-90c. (via CoreTemp software).

I have read that if your cores are at running at 90c temp, then the heat sink's should be Very Hot; mine are not that hot to the touch, hot but not Very Hot.

So I will probably removed the heat sink/fans, clean off the thermal paste, and re-apply; hopefully this will resolve the heat problems.

Thanks for your advice and information,

Chris
ID: 1829400 · Report as offensive
Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next

Message boards : Number crunching : Building a 32 thread xeon system doesn't need to cost a lot


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.