Building a 32-Thread Xeon Monster PC for Less Than the Price of a Flagship Core i7

Message boards : Number crunching : Building a 32-Thread Xeon Monster PC for Less Than the Price of a Flagship Core i7
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1875878 - Posted: 30 Jun 2017, 12:12:46 UTC - in response to Message 1875863.  

A though ... The BIOS may be looking at CPU fan RPM as a 'high performance' guideline, try something with the most RPMs. It could be any fan 120/140 (40mm server fan, those things howl up a storm) just high RPM to fool it for testing.
Does the CPU fan go to full speed when just entering the BIOS?


The fan has run up to full speed a couple different times but still didn't allow a boot due to "cpu wants to much power". As I remember, that was when I hit the power button to try to restart it. Its cranky, beeping for many seconds after I unplug the power cord.

Here cooler, HERE cooler....

Tom
A proud member of the OFA (Old Farts Association).
ID: 1875878 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1875890 - Posted: 30 Jun 2017, 13:34:18 UTC - in response to Message 1875878.  

Tom, back last year when I was playing around with some 400 boards, and looking at the 600's, I ran into the same cooler situation you are now. I took a look at HP's Hi Perf coolers, and found out that Man, they are expensive! Like double the regular versions, and there is I believe something to do with the 5 pin that identifies it to the MB as Hi Perf. I put that project up on the shelf for the the time being, but remember that being an issue if you want to run the top supported chips in that board. Also, depending on what cards you plan on using, you might be looking at their higher end, 1000w PSU as well, but that is a little fuzzy at this point, it's been a while.

ID: 1875890 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1875941 - Posted: 30 Jun 2017, 18:48:55 UTC - in response to Message 1875890.  

Tom, back last year when I was playing around with some 400 boards, and looking at the 600's, I ran into the same cooler situation you are now. I took a look at HP's Hi Perf coolers, and found out that Man, they are expensive! Like double the regular versions, and there is I believe something to do with the 5 pin that identifies it to the MB as Hi Perf. I put that project up on the shelf for the the time being, but remember that being an issue if you want to run the top supported chips in that board. Also, depending on what cards you plan on using, you might be looking at their higher end, 1000w PSU as well, but that is a little fuzzy at this point, it's been a while.


Thank you Al,
The only video card that I might upgrade to from the current GTX 750 Ti is the GTX 1060 from my Z400. Since the Z400 is a 435 watt PS and the Z600 is a 650 watt PS (I think) plus the fact that I actually did run the 1060 in the Z600 for a couple of hours (or a day?) so I don't think the PS will be an issue.

Right now, I MIGHT have a line on an official HP "high performance" cpu cooler for $50 but I have to recheck the part numbers. I have discovered that the part number that someone was claiming was a HiPer is identical to the 4 wire cpu cooler I already have. So either FoxConn is doing a cheaper clone or the part numbers are wrong?

I have an after market cooler due on Saturday so I delayed re-thermal pasting my Z600 just so I can "try one more time" to see if I can get past the "draws to much power error."

I really want that "extra" .27 GHz if the price is a lot less than buying another 1060. Otherwise (shrug) it makes more sense to buy another Gpu. Except the main projects I am interested in besides Seti don't (currently) do gpu processing. So the more cores (even if slower) the merrier.

Tom
A proud member of the OFA (Old Farts Association).
ID: 1875941 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1875942 - Posted: 30 Jun 2017, 18:55:40 UTC - in response to Message 1875941.  
Last modified: 30 Jun 2017, 18:57:07 UTC

I have an after market cooler due on Saturday so I delayed re-thermal pasting my Z600 just so I can "try one more time" to see if I can get past the "draws to much power error."
Just unplug your fan and plug in an unmounted one. The CPU won't get hot just entering the BIOS, and it still has the heatsink.
EDIT: Opps, that would mean the CPU was working now, silly me ....
ID: 1875942 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1875981 - Posted: 30 Jun 2017, 20:49:22 UTC - in response to Message 1875942.  

EDIT: Opps, that would mean the CPU was working now, silly me ....


<GRIN> If I get that far... I get the WHOLE ice cream cone.
A proud member of the OFA (Old Farts Association).
ID: 1875981 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1876102 - Posted: 1 Jul 2017, 2:09:33 UTC - in response to Message 1875941.  

I will see if I can dig up the correct part number for that cooler, I know that I had bid on one on Ebay, so I should be able to find it in my emails. I will post back if I can locate it.

ID: 1876102 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1876169 - Posted: 1 Jul 2017, 11:01:49 UTC - in response to Message 1876102.  

I will see if I can dig up the correct part number for that cooler, I know that I had bid on one on Ebay, so I should be able to find it in my emails. I will post back if I can locate it.


I think it is "HP Z600 Z800 Processor Heatsink & Fan Assembly 463991-001 for extreme CPUs" which for this listing is $195 plus s/h.

I may have found one as low as $150. The crucial issue appears to be the regular version is: 463990-001 And I got real excited at sub-$50 prices until I FINALLY noticed that tiny part# difference.

Given that it would take $300 for the coolers, assuming the trick works, and I can get an i7-3770@4.2Ghz via a Dell 7010 for about $250 (although it only has 4 cores w/HT) it becomes a stalling point. If I can't get an after market cooler that is successful (plus as of the moment, the theory that this works is un-proven) I probably have stalled and will go with that X5675 solution ($100 for two used cpus).

Just gotta wait for the cpu cooler on order to get delivered.

I did find 24GB of Z600 memory for $50 and decided that it was a reasonable price given I am uncertain how much memory the non-Seti tasks could take up with 24 threads.... Once its here I can sell 6GB of Z600 memory that came with it.
-------
I am getting mixed messages on the cpu cooler delivery. One says now deliver on Monday. The other says sending to USPS. So I guess I will check the regular mail on Saturday.

Tom
A proud member of the OFA (Old Farts Association).
ID: 1876169 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1876172 - Posted: 1 Jul 2017, 11:40:23 UTC
Last modified: 1 Jul 2017, 11:49:43 UTC

Another thought - getting scary huh?

Contact some of the AIO manufacturers and see what they do to make their coolers work it that situation.
Since AIOs appears as a normal cooler, there must be a detection method.

EDIT: Just don't hint that you are not planning on buying their product :)
EDIT2: And it their products work, they are definitely cheaper than $150.
ID: 1876172 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1876197 - Posted: 1 Jul 2017, 13:54:08 UTC - in response to Message 1876169.  

I did some searching before I went to bed last night, and I think you are right about that number, but I am a little confused. When I was looking at coolers for them, I think I remember that the 'bestest' cooler had a different look, I thought that the fins on it were angled downwards where it exits in back, but from all the digging I did, looking at all of them for the 400, 600 and 800, none that I could find had that appearance, though there is a different part number for the 6-800 hi perf cooler.

I thought someone in the HP forum actually suggested to get that model instead of the 400, because the 400 model I believe has heat pipe nipples that extend beyond the top of the cooler, so if down the road you wanted to move to a 600 or 800, it couldn't be used as the case wouldn't close. That is, if you care about closing the case, of course. Anywho, that's all I have for now, let me know if I can answer anything else, I'll do my best.

ID: 1876197 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1876206 - Posted: 1 Jul 2017, 14:10:58 UTC - in response to Message 1876169.  

I will see if I can dig up the correct part number for that cooler, I know that I had bid on one on Ebay, so I should be able to find it in my emails. I will post back if I can locate it.


I think it is "HP Z600 Z800 Processor Heatsink & Fan Assembly 463991-001 for extreme CPUs" which for this listing is $195 plus s/h.

I may have found one as low as $150. The crucial issue appears to be the regular version is: 463990-001 And I got real excited at sub-$50 prices until I FINALLY noticed that tiny part# difference.

Given that it would take $300 for the coolers, assuming the trick works, and I can get an i7-3770@4.2Ghz via a Dell 7010 for about $250 (although it only has 4 cores w/HT) it becomes a stalling point. If I can't get an after market cooler that is successful (plus as of the moment, the theory that this works is un-proven) I probably have stalled and will go with that X5675 solution ($100 for two used cpus).

Just gotta wait for the cpu cooler on order to get delivered.

I did find 24GB of Z600 memory for $50 and decided that it was a reasonable price given I am uncertain how much memory the non-Seti tasks could take up with 24 threads.... Once its here I can sell 6GB of Z600 memory that came with it.
-------
I am getting mixed messages on the cpu cooler delivery. One says now deliver on Monday. The other says sending to USPS. So I guess I will check the regular mail on Saturday.

Tom

I think the largest memory requirements I have seen were for the Lattice project. Task memory usage can be as high as ~7.5GB per task.
That meant that with only 128GB of ram I could run ~16 at once on my 16c/32t system without thrashing the swap file. So I restricted the project to only 8 tasks at once.
I think the next largest, at least for my 18 projects, is Climate Prediction. Tasks memory usage runs between 500-700MB per task.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1876206 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1876417 - Posted: 2 Jul 2017, 10:59:15 UTC - in response to Message 1876206.  
Last modified: 2 Jul 2017, 11:04:31 UTC

Well, the low cost "after market" cpu heatsink/fan showed up. It had a 4 wire plug. I will see if my local computer store has any ideas.

According to HP Support. The way HP Zxxx figures out if you have a "high performance" heat sink/fan is a jumper between pin 1 and pin 5. I may consult with my store and see if they can fit my stock cooler with that. If yes, maybe it will be worth it to try the great "will an X5680 install and boot on a Z600" test yet again.

Note to self: Remember, when transplanting from one system to another move ONE hardware system at a time! I wasted the bulk of Saturday "fixing" the wrong problem.

Eventually got Gtx 1060 and HDs moved over. The good news is the Raid 1 hard disk system successfully (eventually) migrated from the Z400 to the Z600.

I finally remembered the two things. 1) Other threads discussing running really fast gpu's on really slow cpus (still gets lots of gpu production). 2) The reason for my policy of only keeping 2 desktop computers running Seti (electric bill).

So probably next Friday the Z400 goes up on ebay for the price of the X5680 cpu and the LCS. $225USD as the minimum bid and a buy it now of $500. With some great Ad copy about being able to run 100% cpu 24/7 :)

Arg! Its Sunday morning. Where did Saturday go? Oh, yeah.....

Tom
A proud member of the OFA (Old Farts Association).
ID: 1876417 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1876420 - Posted: 2 Jul 2017, 11:12:18 UTC - in response to Message 1876206.  
Last modified: 2 Jul 2017, 11:13:18 UTC

I think the largest memory requirements I have seen were for the Lattice project. Task memory usage can be as high as ~7.5GB per task.
That meant that with only 128GB of ram I could run ~16 at once on my 16c/32t system without thrashing the swap file. So I restricted the project to only 8 tasks at once.
I think the next largest, at least for my 18 projects, is Climate Prediction. Tasks memory usage runs between 500-700MB per task.


Hal.
Thank you for the memory requirements information.

Where/How do you do that kind of limiting? (Restricting the number of tasks from the same project) World Community Grid sometimes likes to "hog" my entire cpu. Even with 6c/12t. I expect when I get up to 12c/24t it will be the same. So does Rosetta.

I need an example listing for the how to. I'm not good at translating abstracts....

Thank you.
Tom
A proud member of the OFA (Old Farts Association).
ID: 1876420 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1876438 - Posted: 2 Jul 2017, 14:02:49 UTC - in response to Message 1876420.  

I think the largest memory requirements I have seen were for the Lattice project. Task memory usage can be as high as ~7.5GB per task.
That meant that with only 128GB of ram I could run ~16 at once on my 16c/32t system without thrashing the swap file. So I restricted the project to only 8 tasks at once.
I think the next largest, at least for my 18 projects, is Climate Prediction. Tasks memory usage runs between 500-700MB per task.


Hal.
Thank you for the memory requirements information.

Where/How do you do that kind of limiting? (Restricting the number of tasks from the same project) World Community Grid sometimes likes to "hog" my entire cpu. Even with 6c/12t. I expect when I get up to 12c/24t it will be the same. So does Rosetta.

I need an example listing for the how to. I'm not good at translating abstracts....

Thank you.
Tom

The easiest way is to use an app_config.xml and set project_max_concurrent to the value you would like.
My my HTPC I set SETI to 2 and then other projects cycle through are also set to 2. That way each project gets equal run time. instead of relying on the BOINC long term average system to work. Which actually doesn't when you cycle though projects.
For my 16c/32t system I set no restrictions for SETI and then Lattice was limited to 8. Since Lattice work was intermittent and had a shorter deadline Lattice tasks would run as soon as they were downloaded.
When I set up my next 16c/32t system my plan is to limit each project to 8 tasks at once and run 4 projects. Then have one or two backup projects in case the 4 main projects run out of work.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1876438 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1876514 - Posted: 2 Jul 2017, 23:04:51 UTC - in response to Message 1876438.  
Last modified: 2 Jul 2017, 23:13:57 UTC

The easiest way is to use an app_config.xml and set project_max_concurrent to the value you would like.
My my HTPC I set SETI to 2 and then other projects cycle through are also set to 2. That way each project gets equal run time. instead of relying on the BOINC long term average system to work. Which actually doesn't when you cycle though projects.


I was going to ask where the names were from but found them in the client_state.xml file.

So to help tame the World Community Grid I can use the following app_config.xml file (I think).


<app_config>
<app>
<name>zika</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>
<app>
<name>mcm1</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>
<app>
<name>oet1</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>
<app>
<name>fahv</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>
<app>
<name>scc1</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>
<app>
<name>fahb</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>
<app>
<name>hst1</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
<cpu_usage>1.0</cpu_usage>
</gpu_versions>
</app>

</app_config>

A proud member of the OFA (Old Farts Association).
ID: 1876514 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1876517 - Posted: 3 Jul 2017, 0:00:57 UTC - in response to Message 1876514.  

That should work. If you get work for all 7 apps then you will have 14 tasks going at once.
I kind of would like it if there was a a way to use a single set of parameters for all apps on a projects in the app_config.

Perhaps something like:
<app_config>
	<app>
		<name>all_apps</name>
		<max_concurrent>4</max_concurrent>
		<gpu_versions>
			<gpu_usage>1.0</gpu_usage>
			<cpu_usage>1.0</cpu_usage>
		</gpu_versions>
	</app>
<app_config>

That way the specific apps names wouldn't matter.
Maybe I'll see what the BOINC devs think if something like that is viable.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1876517 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1876518 - Posted: 3 Jul 2017, 0:02:45 UTC - in response to Message 1876514.  


I was going to ask where the names were from but found them in the client_state.xml file.

So to help tame the World Community Grid I can use the following app_config.xml file (I think).

Looks sensible and valid.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1876518 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1876526 - Posted: 3 Jul 2017, 1:10:56 UTC - in response to Message 1876517.  

I kind of would like it if there was a a way to use a single set of parameters for all apps on a projects in the app_config.

Perhaps something like:
<app_config>
	<app>
		<name>all_apps</name>
		<max_concurrent>4</max_concurrent>
		<gpu_versions>
			<gpu_usage>1.0</gpu_usage>
			<cpu_usage>1.0</cpu_usage>
		</gpu_versions>
	</app>
<app_config>

That way the specific apps names wouldn't matter.
Maybe I'll see what the BOINC devs think if something like that is viable.


For anyone running more than 2-4 cores this would be really helpful in controlling what you wanted on a single machine.

Please let me know as soon as it is implemented ;) :D

Tom
A proud member of the OFA (Old Farts Association).
ID: 1876526 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1876527 - Posted: 3 Jul 2017, 1:13:44 UTC

I thought there was a <max_project_concurrent> or something similar.
ID: 1876527 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1876528 - Posted: 3 Jul 2017, 1:20:21 UTC - in response to Message 1876527.  

I don't know about the "all_projects" but there is a max_project_concurrent and max_concurrent. But I would think, in order to get all projects to accept such a command, it would have to be in cc_config.xml but I don't think it would accept it there. Someone with more experience with BOINC will have to address that.
ID: 1876528 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1876542 - Posted: 3 Jul 2017, 3:09:19 UTC - in response to Message 1876528.  

I don't know about the "all_projects" but there is a max_project_concurrent and max_concurrent. But I would think, in order to get all projects to accept such a command, it would have to be in cc_config.xml but I don't think it would accept it there. Someone with more experience with BOINC will have to address that.


What file are max_project_concurrent and max_concurrent located in? I assume I have to find a "project." Under World Community Grid each one of the different sub-projects has its own project name. I am not sure how/where to put the above.

I don't see any doc for them when I tried to google them.

Thank you,
Tom
A proud member of the OFA (Old Farts Association).
ID: 1876542 · Report as offensive
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Building a 32-Thread Xeon Monster PC for Less Than the Price of a Flagship Core i7


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.