Building a 32-Thread Xeon Monster PC for Less Than the Price of a Flagship Core i7

Message boards : Number crunching : Building a 32-Thread Xeon Monster PC for Less Than the Price of a Flagship Core i7
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1873133 - Posted: 15 Jun 2017, 13:44:58 UTC

http://www.techspot.com/review/1155-affordable-dual-xeon-pc/page8.html

I ran across this article. The price of the CPU chips is down. Maybe the cost of the ram is down. I think I saw a respectable motherboard for under $100.

What is not clear is if building this for use as a Boinc/Seti processor would it need water cooling like so many of our larger systems do.

So, if you ignore the cost of the electricity, does this make sense?

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1873133 · Report as offensive
Profile Mr. KevvyCrowdfunding Project Donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 1720
Credit: 376,251,157
RAC: 516,850
Canada
Message 1873144 - Posted: 15 Jun 2017, 14:16:24 UTC

Very good thread on this with lots of useful info. Would have merged this one into it but it expired due to inactivity recently.
“Never doubt that a small group of thoughtful, committed citizens can change the world; indeed, it's the only thing that ever has.”
--- Margaret Mead

ID: 1873144 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1873152 - Posted: 15 Jun 2017, 15:02:25 UTC - in response to Message 1873144.  

Very good thread on this with lots of useful info. Would have merged this one into it but it expired due to inactivity recently.


I picked this up off that thread ".....However It would also be REALLY easy to bypass the per core limits. By setting the number of CPUs in a cc_config.xml and then use an app_config.xml to limit the number of running tasks...."

So I wonder could you do the same with the GPU "limit"?

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1873152 · Report as offensive
Profile ZalsterProject Donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 3993
Credit: 208,964,894
RAC: 38,056
United States
Message 1873170 - Posted: 15 Jun 2017, 16:50:28 UTC - in response to Message 1873152.  

Very good thread on this with lots of useful info. Would have merged this one into it but it expired due to inactivity recently.


I picked this up off that thread ".....However It would also be REALLY easy to bypass the per core limits. By setting the number of CPUs in a cc_config.xml and then use an app_config.xml to limit the number of running tasks...."

So I wonder could you do the same with the GPU "limit"?

Tom


Are you trying to bypass the 100 work unit per GPU limit or are you trying to run multiple instances of work units per GPU? Kind of confused as to what you are specifically asking.
ID: 1873170 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1873179 - Posted: 15 Jun 2017, 17:20:03 UTC - in response to Message 1873170.  

Very good thread on this with lots of useful info. Would have merged this one into it but it expired due to inactivity recently.


I picked this up off that thread ".....However It would also be REALLY easy to bypass the per core limits. By setting the number of CPUs in a cc_config.xml and then use an app_config.xml to limit the number of running tasks...."

So I wonder could you do the same with the GPU "limit"?

Tom


Are you trying to bypass the 100 work unit per GPU limit or are you trying to run multiple instances of work units per GPU? Kind of confused as to what you are specifically asking.


Ah, it sounded like (and I could be confused here) there was something about the 100 work unit limit for cpus in the app_config.xml file. So I was wondering if there was something similar for gpus?

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1873179 · Report as offensive
Profile ZalsterProject Donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 3993
Credit: 208,964,894
RAC: 38,056
United States
Message 1873180 - Posted: 15 Jun 2017, 17:26:25 UTC - in response to Message 1873179.  
Last modified: 15 Jun 2017, 17:27:21 UTC

As far as I know, it's 100 for each CPU (regardless as to how many cores it has) and 100 for each GPU in the computer. I know it wasn't supposed to be that way, it was supposed to be even more restrictive but regardless, that is what we have today.

The app_config and cc_config don't have anything to do with that, it's something from the server end
ID: 1873180 · Report as offensive
Profile petri33Project Donor
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1465
Credit: 270,454,893
RAC: 304,546
Finland
Message 1873203 - Posted: 15 Jun 2017, 20:13:39 UTC
Last modified: 15 Jun 2017, 20:14:01 UTC

Hi,
for the top 50 GPU/GPU users I'd say it is fairly easy to overcome the limit.
I've done that for demonstration purpose only on my computer/ GPU (I do not have 16 GPUs). I have 4.

Those of you running over 40 cores/3xGPUs and with a little of coding experience I'd suggest looking at boinc client code.

Petri
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1873203 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1873207 - Posted: 15 Jun 2017, 20:34:34 UTC - in response to Message 1873203.  

Hi,
for the top 50 GPU/GPU users I'd say it is fairly easy to overcome the limit.
I've done that for demonstration purpose only on my computer/ GPU (I do not have 16 GPUs). I have 4.

Those of you running over 40 cores/3xGPUs and with a little of coding experience I'd suggest looking at boinc client code.

Petri


Sounds like its being enforced at the client level instead of the Seti server level.

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1873207 · Report as offensive
rob smithProject Donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 15221
Credit: 252,874,900
RAC: 322,928
United Kingdom
Message 1873208 - Posted: 15 Jun 2017, 20:36:27 UTC

It is being "enforced" at server level, using data provided from the client.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1873208 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6467
Credit: 176,039,961
RAC: 56,435
United States
Message 1873249 - Posted: 16 Jun 2017, 2:45:21 UTC - in response to Message 1873180.  

As far as I know, it's 100 for each CPU (regardless as to how many cores it has) and 100 for each GPU in the computer. I know it wasn't supposed to be that way, it was supposed to be even more restrictive but regardless, that is what we have today.

The app_config and cc_config don't have anything to do with that, it's something from the server end

It is a limit of 100 CPU tasks in total per host. BOINC doesn't know how many physical CPUs are in a system. It only knows the count of total processors. :/
It would be great to have more than a cache of 100 tasks on my 16c/32t system. Since that is only about 6-7.5 hours.



The prices for dual E5-2670's are going up. months ago the CPUs were going for ~$50 and today are around $100
In October I bought a bundle with Intel S2600CP2J MB, 2 E5-2670's, & 128GB PC3-12800R for $470. Then a 2nd one a few weeks later.
Now that vendor has the same config for $626.

I did considering a LCS setup but air cooling with a pair of Noctura NH-U12DXi4's is working fine for me
http://i.imgur.com/ZMqgcrB.jpg
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1873249 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1873320 - Posted: 16 Jun 2017, 11:22:29 UTC - in response to Message 1873249.  

The prices for dual E5-2670's are going up. months ago the CPUs were going for ~$50 and today are around $100
In October I bought a bundle with Intel S2600CP2J MB, 2 E5-2670's, & 128GB PC3-12800R for $470. Then a 2nd one a few weeks later.
Now that vendor has the same config for $626.

I did considering a LCS setup but air cooling with a pair of Noctura NH-U12DXi4's is working fine for me
http://i.imgur.com/ZMqgcrB.jpg


Are there any other lower cost alternative cpu's that produce similar results?

Thank you for helping me return this thread to the topic in the title!

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1873320 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6467
Credit: 176,039,961
RAC: 56,435
United States
Message 1873467 - Posted: 17 Jun 2017, 0:26:42 UTC - in response to Message 1873320.  

The prices for dual E5-2670's are going up. months ago the CPUs were going for ~$50 and today are around $100
In October I bought a bundle with Intel S2600CP2J MB, 2 E5-2670's, & 128GB PC3-12800R for $470. Then a 2nd one a few weeks later.
Now that vendor has the same config for $626.

I did considering a LCS setup but air cooling with a pair of Noctura NH-U12DXi4's is working fine for me
http://i.imgur.com/ZMqgcrB.jpg


Are there any other lower cost alternative cpu's that produce similar results?

Thank you for helping me return this thread to the topic in the title!

Tom

There was a flood of E5-2670's to the 2nd hand market. As many data centers upgraded to the latest versions. So supply and demand are really the factors for pricing.
Once the E5-2600 v2 CPUs drop to a reasonable amount I'll pick up some to upgrade my current boards with v1 CPUs, but I don't expect that to happen soon.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1873467 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1873553 - Posted: 17 Jun 2017, 11:41:18 UTC - in response to Message 1873467.  

Are there any other lower cost alternative cpu's that produce similar results?
----------
There was a flood of E5-2670's to the 2nd hand market. As many data centers upgraded to the latest versions. So supply and demand are really the factors for pricing.
Once the E5-2600 v2 CPUs drop to a reasonable amount I'll pick up some to upgrade my current boards with v1 CPUs, but I don't expect that to happen soon.


I just saw some E5-2670's for $46 and some E5-2600's for $130-$150. I could not tell at the level I was looking if they were V2 or not.

Hmmmm.....

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1873553 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6467
Credit: 176,039,961
RAC: 56,435
United States
Message 1873587 - Posted: 17 Jun 2017, 14:57:59 UTC - in response to Message 1873553.  

Are there any other lower cost alternative cpu's that produce similar results?
----------
There was a flood of E5-2670's to the 2nd hand market. As many data centers upgraded to the latest versions. So supply and demand are really the factors for pricing.
Once the E5-2600 v2 CPUs drop to a reasonable amount I'll pick up some to upgrade my current boards with v1 CPUs, but I don't expect that to happen soon.


I just saw some E5-2670's for $46 and some E5-2600's for $130-$150. I could not tell at the level I was looking if they were V2 or not.

Hmmmm.....

Tom

For E5-2670's around $50 is good. At the peak low point you could find pairs of them for $80 or ~$45 alone, but mostly you find single CPUs for ~$100 today.
The issue I originally had was finding a dual LGA2011 socket MB for under $200. Sometimes I would find one but then it would turn out to be a dual LGA2011-3 MB. Which is the socket for v3 and v4 Xeons.

The newer CPUs should be clearly labeled as V2. Given that it is part of the product name.
Often you find the L versions of CPUs much cheaper and not listed correctly. You can check the advertised clock speed against Intel's spec list. E5-2650L v2 (1.7GHZ 10c/20t) vs the E5-2650 v2 (2.6GHZ 8c/16t)
To replace my E5-2670's I'm looking for either E5-2650 v2 (2.6GHZ 8c/16t) or E5-2670 V2 (2.5GHz10c20t) CPUs.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1873587 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1874634 - Posted: 23 Jun 2017, 12:53:24 UTC

I am now looking to upgrade a Z600 with a boot block date of 1/7/2010 or later. It appears I can install a couple of X56xx cpus (6 cores w/HT) which would give only a "24 core" Boinc/Seti server.

I have a couple of X5680's which are currently the sweet spot for fastest Ghz vs. price. You can buy X5690's for twice the price of the X5680 and get something like 3% faster baseline Ghz processing speed.

Once I have the Z600 1/7/2010, have updated the bios and located cpu coolers that both fit (the box has less overhead room than a Z400) and can actually keep it cool, I should be ready to roll out a 24 core server for less than $500 USD. ;)

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1874634 · Report as offensive
Profile Wiggo "Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 12611
Credit: 169,598,224
RAC: 87,158
Australia
Message 1874639 - Posted: 23 Jun 2017, 13:00:25 UTC

Does it need to be in a box Tom?

Cheers.
ID: 1874639 · Report as offensive
AlProject Donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1387
Credit: 260,755,621
RAC: 470,956
United States
Message 1874652 - Posted: 23 Jun 2017, 14:41:35 UTC

Naa, boxes are overrated. ;-) Free and breezy is the way for me.

ID: 1874652 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1874699 - Posted: 23 Jun 2017, 19:05:12 UTC - in response to Message 1874639.  

Does it need to be in a box Tom?

Cheers.


Probably does, I have a noisy cat who likes to run against stuff. I can see it now, his whiskers or tail getting caught in the cpu fan(s).... His offended expression afterwards would be priceless.....

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1874699 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1875772 - Posted: 30 Jun 2017, 1:04:29 UTC - in response to Message 1874634.  
Last modified: 30 Jun 2017, 1:07:56 UTC

I am now looking to upgrade a Z600 with a boot block date of 1/7/2010 or later. It appears I can install a couple of X56xx cpus (6 cores w/HT) which would give only a "24 core" Boinc/Seti server.


It turns out putting X5680's into a Z600 is tricky. If the "theory" I am reading about is right, you have to match the wattage increase of the cpu chip with a "high performance" cpu cooler. Otherwise the bios wakes up, looks around and says "cpu chip installed wants to much power". The X5680's want 135 watts/TDP? Most of the cpu chips are 95 watt chips. But one of the cpu chips on the parts list is 130 watts.

So the current/unproven theory is using a "high performance" cpu cooler will allow the Z600 to accept the X5680.

I read something about a jumper wire from pin one to pin 5 was how the cooler was declared to be "high performance". My spare stock Z400 cpu cooler was wired up like that so I tried that out. No joy.

So I am waiting for this other cpu cooler I ordered. It is supposed to be here no later than Saturday. The "official" high performance coolers are more expensive than the last LCS ($150) I bought for my Z400 :) Even on Ebay. The cooler I am waiting on was an "open box" special for $24 :D

So if the last test crashes and burns I will sell all my extra cpu chips ( X5680, W5680, 2 W5635's) and buy a couple of X5675's that will work (but Ma they are only 3.06Ghz! Right Junior but look at all those lovely cores (6/cpu w/HT) giving me (gasp) 24 cores for the glory of BOINC (and Seti). And I think I will need to scare up another 6GB of ram. I just can't imagine trying to run 12 cores in 2GB (the ram splits asymmetrically, So 4GB on one cpu and 2GB on the other unless I buy more ram.

Sigh, It has been a learning experience. "More cores than you know what to do with out assembling it from scratch for less money than you could imagine" was the goal. Haven't managed it yet, not sure I'm going to.

<martyred grin>

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1875772 · Report as offensive
Profile Brent Norman
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1824
Credit: 108,053,034
RAC: 457,853
Canada
Message 1875863 - Posted: 30 Jun 2017, 10:44:34 UTC

A though ... The BIOS may be looking at CPU fan RPM as a 'high performance' guideline, try something with the most RPMs. It could be any fan 120/140 (40mm server fan, those things howl up a storm) just high RPM to fool it for testing.
Does the CPU fan go to full speed when just entering the BIOS?
ID: 1875863 · Report as offensive
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Building a 32-Thread Xeon Monster PC for Less Than the Price of a Flagship Core i7


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.