8 GPUs

Message boards : Number crunching : 8 GPUs
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1085095 - Posted: 8 Mar 2011, 17:13:02 UTC - in response to Message 1085070.  

What set of cards would you have in a computer that see's more than 8 gpus to begin with? Most motherboard only fit 4x dual slot cards of 6 single slots......4x dual slots with 295/590/6990 etc. will only be 8 gpu's and 6 single slot cards would only be 6 cpus. The only time you would probably run into that issue is if you have 6x 295's water cooled for single slot which would be 12 gpu's however the Nvidia drivers only support 4 cards to begin with, which would be 8 cores max. At least that's my understanding. (I may be talking about SLi max and not total max not really sure)

-BeNT-, if you read a bit further down the thread, you'll see that the 8-GPU remark applies to a custom, hand-built workstation-class enclosure priced between 30 and 42 thousand dollars, depending whether you choose C2070 Tesla cards (the cheap option) or Quadro 6000 GPUs (rather more expensive) - either card with 6GB video memory per card.

At that price, I expect them to send round an NVidia engineer to install the drivers personally!
ID: 1085095 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1085136 - Posted: 8 Mar 2011, 23:33:42 UTC - in response to Message 1085095.  

What set of cards would you have in a computer that see's more than 8 gpus to begin with? Most motherboard only fit 4x dual slot cards of 6 single slots......4x dual slots with 295/590/6990 etc. will only be 8 gpu's and 6 single slot cards would only be 6 cpus. The only time you would probably run into that issue is if you have 6x 295's water cooled for single slot which would be 12 gpu's however the Nvidia drivers only support 4 cards to begin with, which would be 8 cores max. At least that's my understanding. (I may be talking about SLi max and not total max not really sure)

-BeNT-, if you read a bit further down the thread, you'll see that the 8-GPU remark applies to a custom, hand-built workstation-class enclosure priced between 30 and 42 thousand dollars, depending whether you choose C2070 Tesla cards (the cheap option) or Quadro 6000 GPUs (rather more expensive) - either card with 6GB video memory per card.

At that price, I expect them to send round an NVidia engineer to install the drivers personally!

As I said before that system setup with decent PSU's and lower end dual hex core CPU's will come in just under $10,000 as I can't see any cruncher wanting either Telsa or Quadro cards and then I suggested fitting 8 GTX560's which will bring the price to about $12,000 (others may go for GTX570's or 580's) which should be a pretty good beast (plus I'm sure that Todd would have fun with it). ;)

Cheers.
ID: 1085136 · Report as offensive
J. Mileski
Volunteer tester
Avatar

Send message
Joined: 9 Jun 02
Posts: 632
Credit: 172,116,532
RAC: 572
United States
Message 1085145 - Posted: 8 Mar 2011, 23:59:50 UTC - in response to Message 1085136.  

What set of cards would you have in a computer that see's more than 8 gpus to begin with? Most motherboard only fit 4x dual slot cards of 6 single slots......4x dual slots with 295/590/6990 etc. will only be 8 gpu's and 6 single slot cards would only be 6 cpus. The only time you would probably run into that issue is if you have 6x 295's water cooled for single slot which would be 12 gpu's however the Nvidia drivers only support 4 cards to begin with, which would be 8 cores max. At least that's my understanding. (I may be talking about SLi max and not total max not really sure)

-BeNT-, if you read a bit further down the thread, you'll see that the 8-GPU remark applies to a custom, hand-built workstation-class enclosure priced between 30 and 42 thousand dollars, depending whether you choose C2070 Tesla cards (the cheap option) or Quadro 6000 GPUs (rather more expensive) - either card with 6GB video memory per card.

At that price, I expect them to send round an NVidia engineer to install the drivers personally!

As I said before that system setup with decent PSU's and lower end dual hex core CPU's will come in just under $10,000 as I can't see any cruncher wanting either Telsa or Quadro cards and then I suggested fitting 8 GTX560's which will bring the price to about $12,000 (others may go for GTX570's or 580's) which should be a pretty good beast (plus I'm sure that Todd would have fun with it). ;)

Cheers.


Here is a PDF of the manual for the 8 GPU boardhttp://www.tyan.com/manuals/FT72-B7015_Manual_v1.01.pdf
ID: 1085145 · Report as offensive
Profile speedbump

Send message
Joined: 19 May 01
Posts: 247
Credit: 192,906,380
RAC: 0
United States
Message 1085180 - Posted: 9 Mar 2011, 2:21:49 UTC

Try 3000 watts of power and 8 gtx590s ;-)

ID: 1085180 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1085366 - Posted: 9 Mar 2011, 19:04:57 UTC - in response to Message 1085038.  

OTOH, IIRC, Windows see max 8 GPUs.. so the max is (currently) 8 cards with 1 GPU each, or 4 cards with 2 GPUs each. ...


In that case, just move over and use Linux. No problem.

Happy super-fast crunchin',
Martin

Actually it's not a Windows feature, the BOINC Scheduler won't deliver work for more than 8 GPUs:

const int MAX_GPUS = 8;
// don't believe clients who claim they have more GPUs than this


However, I think setting the cache to twice what you really want would take care of that awkwardness. The project's config.xml <min_sendwork_interval> of 300 seconds might be a limitation on that 16 GPU system, though. I expect they'll be willing to reduce that minimum somewhat if so.
                                                                   Joe
ID: 1085366 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1085482 - Posted: 9 Mar 2011, 23:48:31 UTC - in response to Message 1085366.  

... the BOINC Scheduler won't deliver work for more than 8 GPUs:

const int MAX_GPUS = 8;
// don't believe clients who claim they have more GPUs than this


However, I think setting the cache to twice what you really want would take care of that awkwardness. The project's config.xml <min_sendwork_interval> of 300 seconds might be a limitation on that 16 GPU system, though. I expect they'll be willing to reduce that minimum somewhat if so.


Eeek, that's a bit of a short sighted killer.

Then again, to increase the max GPUs count then resurrects the question of how do you stop a bad host from trashing too many WUs...? Yet without starving a new good host starting up...?


Happy fast crunchin',
Martin





See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1085482 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1085489 - Posted: 10 Mar 2011, 0:09:57 UTC - in response to Message 1085366.  

Actually it's not a Windows feature, the BOINC Scheduler won't deliver work for more than 8 GPUs:

const int MAX_GPUS = 8;
// don't believe clients who claim they have more GPUs than this


However, I think setting the cache to twice what you really want would take care of that awkwardness. The project's config.xml <min_sendwork_interval> of 300 seconds might be a limitation on that 16 GPU system, though. I expect they'll be willing to reduce that minimum somewhat if so.
                                                                   Joe

With the feeder/scheduler combination supplying up to 97 tasks every five minutes - or perhaps 40 each time, I've seen that recently - I don't think we'll hit that limit any time soon. Unless they're all shorties, in which case we're all f****d already, because the download limit bites before the scheduler limit.

And, yet again, we're here to process the data that SETI (the search, not the project) requires to be processed: SETI (the project, not the search) doesn't exist to supply data for the fastest cruncher we can build. If the scheduler can't supply enough data, join another project. I'm sure GPUGrid's new 'long task' subproject would be grateful for the extra help in their "batch of experiments to study the behavior of a protein likely implied in Alzheimer's disease".
ID: 1085489 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1085931 - Posted: 11 Mar 2011, 2:32:17 UTC - in response to Message 1084304.  

There is a 8 GPU motherboard and case, made by Tyan and sold through http://www.velocitymicro.com/wizard.php?sr=0&iid=190. It is not cheap and I am guessing not cheap to run. It can be configured with either the Tesla cards or the GeForce cards. There are only two 2.5" bays for SATA laptop drives or SSDs. There are no external drive bays.

Yeah, just for giggles, I clicked the link and configured it to the hilt. Using the 8 Quadro cards, fully decked out, came out to the low, low price of $53383. But hey, this week, this system ships for Free! Count me in, all because of the Free Shipping! ;)

ID: 1085931 · Report as offensive
-BeNt-
Avatar

Send message
Joined: 17 Oct 99
Posts: 1234
Credit: 10,116,112
RAC: 0
United States
Message 1085968 - Posted: 11 Mar 2011, 3:48:24 UTC - in response to Message 1085931.  

There is a 8 GPU motherboard and case, made by Tyan and sold through http://www.velocitymicro.com/wizard.php?sr=0&iid=190. It is not cheap and I am guessing not cheap to run. It can be configured with either the Tesla cards or the GeForce cards. There are only two 2.5" bays for SATA laptop drives or SSDs. There are no external drive bays.

Yeah, just for giggles, I clicked the link and configured it to the hilt. Using the 8 Quadro cards, fully decked out, came out to the low, low price of $53383. But hey, this week, this system ships for Free! Count me in, all because of the Free Shipping! ;)


Hey not bad! Free shipping sounds great! /sarcasm

If I spent that much on a system it better cause ET to give up hiding and call my house asking for the president.
Traveling through space at ~67,000mph!
ID: 1085968 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65709
Credit: 55,293,173
RAC: 49
United States
Message 1087255 - Posted: 15 Mar 2011, 2:13:38 UTC - in response to Message 1085931.  

There is a 8 GPU motherboard and case, made by Tyan and sold through http://www.velocitymicro.com/wizard.php?sr=0&iid=190. It is not cheap and I am guessing not cheap to run. It can be configured with either the Tesla cards or the GeForce cards. There are only two 2.5" bays for SATA laptop drives or SSDs. There are no external drive bays.

Yeah, just for giggles, I clicked the link and configured it to the hilt. Using the 8 Quadro cards, fully decked out, came out to the low, low price of $53383. But hey, this week, this system ships for Free! Count me in, all because of the Free Shipping! ;)

Jaw drops to floor, Then eyeballs pop out somewhat. Dang that would be one heck of a system, As that's a lot of moo la.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1087255 · Report as offensive
Profile Viciente
Volunteer tester
Avatar

Send message
Joined: 21 Jan 11
Posts: 228
Credit: 326,384
RAC: 0
Austria
Message 1087442 - Posted: 16 Mar 2011, 17:01:25 UTC - in response to Message 1084945.  
Last modified: 16 Mar 2011, 17:01:47 UTC

....... Ohh.. if I would have money.. *sigh* *dreaming*
We're crazy, aren't we? ;-D .......

yes .. definitely! :-)
i love hubble! ..

ID: 1087442 · Report as offensive
Profile speedbump

Send message
Joined: 19 May 01
Posts: 247
Credit: 192,906,380
RAC: 0
United States
Message 1087452 - Posted: 16 Mar 2011, 17:32:57 UTC

You can use any of the Nvidia cards with this chasis, and then the cost is not nearly so high. Still a lot, just not as much. Mine is already ordered and will be built and running soon I hope. Quadros for our purpose just dont make sense in terms of performance and cost. The enthusiast cards perform much better and cost much less.
ID: 1087452 · Report as offensive
Profile Hephaiston
Volunteer tester
Avatar

Send message
Joined: 15 Jun 08
Posts: 8
Credit: 604,600
RAC: 1
Germany
Message 1092628 - Posted: 2 Apr 2011, 13:41:00 UTC
Last modified: 2 Apr 2011, 13:41:46 UTC

@ Dragokatzov

With that you are able to run 16 GPUs in one case using HD3850X2, HD4870X2 and HD6990 or equal NV-cards. You have just be sure having enough power on the line ... :-P

Regards

Hephaiston
-
ID: 1092628 · Report as offensive
Profile speedbump

Send message
Joined: 19 May 01
Posts: 247
Credit: 192,906,380
RAC: 0
United States
Message 1092708 - Posted: 2 Apr 2011, 17:25:27 UTC

Here's another idea. Cubix make a unit that you can put up to four video cards in, with its own power supply, and attach it to one pcie slot in the host computer.So take 8 slots in host computer x4 video cards in each Cubix = 32, and with gtx590s multiply that by 2 and you get 64. Not sure how well that would work with the thouroughput and the ability to feed all the cards, but it sure would be interesting. Maybe it could be the next stage of upgrade.
ID: 1092708 · Report as offensive
Dave

Send message
Joined: 29 Mar 02
Posts: 778
Credit: 25,001,396
RAC: 0
United Kingdom
Message 1092737 - Posted: 2 Apr 2011, 19:12:53 UTC

Think that just stunned us all into silence.
ID: 1092737 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1092761 - Posted: 2 Apr 2011, 20:21:15 UTC

Well if that isn't enough for you. Then you might want an 18 x16 slot board.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1092761 · Report as offensive
Profile Area 51
Avatar

Send message
Joined: 31 Jan 04
Posts: 965
Credit: 42,193,520
RAC: 0
United Kingdom
Message 1092769 - Posted: 2 Apr 2011, 21:23:00 UTC - in response to Message 1092737.  

Think that just stunned us all into silence.




...which in itself is quite something!!!!
ID: 1092769 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 1092800 - Posted: 2 Apr 2011, 22:44:43 UTC - in response to Message 1092761.  

Well if that isn't enough for you. Then you might want an 18 x16 slot board.


From that site:

Eighteen (18) PCI Express x16 card slots (mechanical)

PCI Express card slot electrical configuration:
Four (4) PCIe x8 and fourteen (14) PCIe x4


So that would fit 9 dual-slot cards, but would there be enough shared bandwidth between all the slots to handle 9 cards efficiently? It also looks like it's only PCI Express 1.0, so it has half the bandwidth shared between those 9 cards.
ID: 1092800 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 1092838 - Posted: 3 Apr 2011, 0:31:49 UTC - in response to Message 1092761.  
Last modified: 3 Apr 2011, 0:32:51 UTC

When this kind of Power Computer {Cluster} becomes mainsteam, SETI might
not
been able to keep up whith the amount of data comming from and to this

Mega-Cruncher and a solid [i]High Speed
(A)DSL of at least 20Mbit/sec, which
isn't really fast, would be necessary, too. But it would be nice, to have one,
well you can always Dream about it which is totally free of any charge ;-)
(20MByte(160Mbit) per second is faster.)
ID: 1092838 · Report as offensive
Previous · 1 · 2

Message boards : Number crunching : 8 GPUs


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.