8 GPUs


log in

Advanced search

Message boards : Number crunching : 8 GPUs

Previous · 1 · 2
Author Message
Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8813
Credit: 53,491,536
RAC: 45,393
United Kingdom
Message 1085095 - Posted: 8 Mar 2011, 17:13:02 UTC - in response to Message 1085070.

What set of cards would you have in a computer that see's more than 8 gpus to begin with? Most motherboard only fit 4x dual slot cards of 6 single slots......4x dual slots with 295/590/6990 etc. will only be 8 gpu's and 6 single slot cards would only be 6 cpus. The only time you would probably run into that issue is if you have 6x 295's water cooled for single slot which would be 12 gpu's however the Nvidia drivers only support 4 cards to begin with, which would be 8 cores max. At least that's my understanding. (I may be talking about SLi max and not total max not really sure)

-BeNT-, if you read a bit further down the thread, you'll see that the 8-GPU remark applies to a custom, hand-built workstation-class enclosure priced between 30 and 42 thousand dollars, depending whether you choose C2070 Tesla cards (the cheap option) or Quadro 6000 GPUs (rather more expensive) - either card with 6GB video memory per card.

At that price, I expect them to send round an NVidia engineer to install the drivers personally!

Profile Wiggo
Avatar
Send message
Joined: 24 Jan 00
Posts: 8599
Credit: 99,263,254
RAC: 53,054
Australia
Message 1085136 - Posted: 8 Mar 2011, 23:33:42 UTC - in response to Message 1085095.

What set of cards would you have in a computer that see's more than 8 gpus to begin with? Most motherboard only fit 4x dual slot cards of 6 single slots......4x dual slots with 295/590/6990 etc. will only be 8 gpu's and 6 single slot cards would only be 6 cpus. The only time you would probably run into that issue is if you have 6x 295's water cooled for single slot which would be 12 gpu's however the Nvidia drivers only support 4 cards to begin with, which would be 8 cores max. At least that's my understanding. (I may be talking about SLi max and not total max not really sure)

-BeNT-, if you read a bit further down the thread, you'll see that the 8-GPU remark applies to a custom, hand-built workstation-class enclosure priced between 30 and 42 thousand dollars, depending whether you choose C2070 Tesla cards (the cheap option) or Quadro 6000 GPUs (rather more expensive) - either card with 6GB video memory per card.

At that price, I expect them to send round an NVidia engineer to install the drivers personally!

As I said before that system setup with decent PSU's and lower end dual hex core CPU's will come in just under $10,000 as I can't see any cruncher wanting either Telsa or Quadro cards and then I suggested fitting 8 GTX560's which will bring the price to about $12,000 (others may go for GTX570's or 580's) which should be a pretty good beast (plus I'm sure that Todd would have fun with it). ;)

Cheers.
____________

J. Mileski
Volunteer tester
Avatar
Send message
Joined: 9 Jun 02
Posts: 132
Credit: 23,319,557
RAC: 48
United States
Message 1085145 - Posted: 8 Mar 2011, 23:59:50 UTC - in response to Message 1085136.

What set of cards would you have in a computer that see's more than 8 gpus to begin with? Most motherboard only fit 4x dual slot cards of 6 single slots......4x dual slots with 295/590/6990 etc. will only be 8 gpu's and 6 single slot cards would only be 6 cpus. The only time you would probably run into that issue is if you have 6x 295's water cooled for single slot which would be 12 gpu's however the Nvidia drivers only support 4 cards to begin with, which would be 8 cores max. At least that's my understanding. (I may be talking about SLi max and not total max not really sure)

-BeNT-, if you read a bit further down the thread, you'll see that the 8-GPU remark applies to a custom, hand-built workstation-class enclosure priced between 30 and 42 thousand dollars, depending whether you choose C2070 Tesla cards (the cheap option) or Quadro 6000 GPUs (rather more expensive) - either card with 6GB video memory per card.

At that price, I expect them to send round an NVidia engineer to install the drivers personally!

As I said before that system setup with decent PSU's and lower end dual hex core CPU's will come in just under $10,000 as I can't see any cruncher wanting either Telsa or Quadro cards and then I suggested fitting 8 GTX560's which will bring the price to about $12,000 (others may go for GTX570's or 580's) which should be a pretty good beast (plus I'm sure that Todd would have fun with it). ;)

Cheers.


Here is a PDF of the manual for the 8 GPU boardhttp://www.tyan.com/manuals/FT72-B7015_Manual_v1.01.pdf
____________

Profile speedbump
Send message
Joined: 19 May 01
Posts: 247
Credit: 192,906,380
RAC: 0
United States
Message 1085180 - Posted: 9 Mar 2011, 2:21:49 UTC

Try 3000 watts of power and 8 gtx590s ;-)

____________

Josef W. SegurProject donor
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4347
Credit: 1,124,982
RAC: 772
United States
Message 1085366 - Posted: 9 Mar 2011, 19:04:57 UTC - in response to Message 1085038.

OTOH, IIRC, Windows see max 8 GPUs.. so the max is (currently) 8 cards with 1 GPU each, or 4 cards with 2 GPUs each. ...


In that case, just move over and use Linux. No problem.

Happy super-fast crunchin',
Martin

Actually it's not a Windows feature, the BOINC Scheduler won't deliver work for more than 8 GPUs:

const int MAX_GPUS = 8;
// don't believe clients who claim they have more GPUs than this


However, I think setting the cache to twice what you really want would take care of that awkwardness. The project's config.xml <min_sendwork_interval> of 300 seconds might be a limitation on that 16 GPU system, though. I expect they'll be willing to reduce that minimum somewhat if so.
Joe

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8601
Credit: 4,258,318
RAC: 1,344
United Kingdom
Message 1085482 - Posted: 9 Mar 2011, 23:48:31 UTC - in response to Message 1085366.

... the BOINC Scheduler won't deliver work for more than 8 GPUs:

const int MAX_GPUS = 8;
// don't believe clients who claim they have more GPUs than this


However, I think setting the cache to twice what you really want would take care of that awkwardness. The project's config.xml <min_sendwork_interval> of 300 seconds might be a limitation on that 16 GPU system, though. I expect they'll be willing to reduce that minimum somewhat if so.


Eeek, that's a bit of a short sighted killer.

Then again, to increase the max GPUs count then resurrects the question of how do you stop a bad host from trashing too many WUs...? Yet without starving a new good host starting up...?


Happy fast crunchin',
Martin





____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8813
Credit: 53,491,536
RAC: 45,393
United Kingdom
Message 1085489 - Posted: 10 Mar 2011, 0:09:57 UTC - in response to Message 1085366.

Actually it's not a Windows feature, the BOINC Scheduler won't deliver work for more than 8 GPUs:

const int MAX_GPUS = 8;
// don't believe clients who claim they have more GPUs than this


However, I think setting the cache to twice what you really want would take care of that awkwardness. The project's config.xml <min_sendwork_interval> of 300 seconds might be a limitation on that 16 GPU system, though. I expect they'll be willing to reduce that minimum somewhat if so.
Joe

With the feeder/scheduler combination supplying up to 97 tasks every five minutes - or perhaps 40 each time, I've seen that recently - I don't think we'll hit that limit any time soon. Unless they're all shorties, in which case we're all f****d already, because the download limit bites before the scheduler limit.

And, yet again, we're here to process the data that SETI (the search, not the project) requires to be processed: SETI (the project, not the search) doesn't exist to supply data for the fastest cruncher we can build. If the scheduler can't supply enough data, join another project. I'm sure GPUGrid's new 'long task' subproject would be grateful for the extra help in their "batch of experiments to study the behavior of a protein likely implied in Alzheimer's disease".

AlProject donor
Send message
Joined: 3 Apr 99
Posts: 481
Credit: 54,600,876
RAC: 21,135
United States
Message 1085931 - Posted: 11 Mar 2011, 2:32:17 UTC - in response to Message 1084304.

There is a 8 GPU motherboard and case, made by Tyan and sold through http://www.velocitymicro.com/wizard.php?sr=0&iid=190. It is not cheap and I am guessing not cheap to run. It can be configured with either the Tesla cards or the GeForce cards. There are only two 2.5" bays for SATA laptop drives or SSDs. There are no external drive bays.

Yeah, just for giggles, I clicked the link and configured it to the hilt. Using the 8 Quadro cards, fully decked out, came out to the low, low price of $53383. But hey, this week, this system ships for Free! Count me in, all because of the Free Shipping! ;)
____________

-BeNt-
Avatar
Send message
Joined: 17 Oct 99
Posts: 1234
Credit: 10,116,112
RAC: 0
United States
Message 1085968 - Posted: 11 Mar 2011, 3:48:24 UTC - in response to Message 1085931.

There is a 8 GPU motherboard and case, made by Tyan and sold through http://www.velocitymicro.com/wizard.php?sr=0&iid=190. It is not cheap and I am guessing not cheap to run. It can be configured with either the Tesla cards or the GeForce cards. There are only two 2.5" bays for SATA laptop drives or SSDs. There are no external drive bays.

Yeah, just for giggles, I clicked the link and configured it to the hilt. Using the 8 Quadro cards, fully decked out, came out to the low, low price of $53383. But hey, this week, this system ships for Free! Count me in, all because of the Free Shipping! ;)


Hey not bad! Free shipping sounds great! /sarcasm

If I spent that much on a system it better cause ET to give up hiding and call my house asking for the president.
____________
Traveling through space at ~67,000mph!

zoom314Project donor
Volunteer tester
Avatar
Send message
Joined: 30 Nov 03
Posts: 47136
Credit: 37,076,278
RAC: 4,241
United States
Message 1087255 - Posted: 15 Mar 2011, 2:13:38 UTC - in response to Message 1085931.

There is a 8 GPU motherboard and case, made by Tyan and sold through http://www.velocitymicro.com/wizard.php?sr=0&iid=190. It is not cheap and I am guessing not cheap to run. It can be configured with either the Tesla cards or the GeForce cards. There are only two 2.5" bays for SATA laptop drives or SSDs. There are no external drive bays.

Yeah, just for giggles, I clicked the link and configured it to the hilt. Using the 8 Quadro cards, fully decked out, came out to the low, low price of $53383. But hey, this week, this system ships for Free! Count me in, all because of the Free Shipping! ;)

Jaw drops to floor, Then eyeballs pop out somewhat. Dang that would be one heck of a system, As that's a lot of moo la.
____________
My Facebook, War Commander, 2015

Profile Viciente
Volunteer tester
Avatar
Send message
Joined: 21 Jan 11
Posts: 228
Credit: 323,302
RAC: 0
Austria
Message 1087442 - Posted: 16 Mar 2011, 17:01:25 UTC - in response to Message 1084945.
Last modified: 16 Mar 2011, 17:01:47 UTC

....... Ohh.. if I would have money.. *sigh* *dreaming*
We're crazy, aren't we? ;-D .......

yes .. definitely! :-)
____________
i love hubble! ..

Profile speedbump
Send message
Joined: 19 May 01
Posts: 247
Credit: 192,906,380
RAC: 0
United States
Message 1087452 - Posted: 16 Mar 2011, 17:32:57 UTC

You can use any of the Nvidia cards with this chasis, and then the cost is not nearly so high. Still a lot, just not as much. Mine is already ordered and will be built and running soon I hope. Quadros for our purpose just dont make sense in terms of performance and cost. The enthusiast cards perform much better and cost much less.
____________

Profile Hephaiston
Volunteer tester
Avatar
Send message
Joined: 15 Jun 08
Posts: 8
Credit: 474,555
RAC: 882
Germany
Message 1092628 - Posted: 2 Apr 2011, 13:41:00 UTC
Last modified: 2 Apr 2011, 13:41:46 UTC

@ Dragokatzov

With that you are able to run 16 GPUs in one case using HD3850X2, HD4870X2 and HD6990 or equal NV-cards. You have just be sure having enough power on the line ... :-P

Regards

Hephaiston
____________
-

Profile speedbump
Send message
Joined: 19 May 01
Posts: 247
Credit: 192,906,380
RAC: 0
United States
Message 1092708 - Posted: 2 Apr 2011, 17:25:27 UTC

Here's another idea. Cubix make a unit that you can put up to four video cards in, with its own power supply, and attach it to one pcie slot in the host computer.So take 8 slots in host computer x4 video cards in each Cubix = 32, and with gtx590s multiply that by 2 and you get 64. Not sure how well that would work with the thouroughput and the ability to feed all the cards, but it sure would be interesting. Maybe it could be the next stage of upgrade.
____________

Dave
Avatar
Send message
Joined: 29 Mar 02
Posts: 774
Credit: 23,193,139
RAC: 0
United Kingdom
Message 1092737 - Posted: 2 Apr 2011, 19:12:53 UTC

Think that just stunned us all into silence.

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4664
Credit: 123,566,749
RAC: 98,564
United States
Message 1092761 - Posted: 2 Apr 2011, 20:21:15 UTC

Well if that isn't enough for you. Then you might want an 18 x16 slot board.
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

Profile Area 51
Avatar
Send message
Joined: 31 Jan 04
Posts: 965
Credit: 42,193,520
RAC: 0
United Kingdom
Message 1092769 - Posted: 2 Apr 2011, 21:23:00 UTC - in response to Message 1092737.

Think that just stunned us all into silence.




...which in itself is quite something!!!!
____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13704
Credit: 31,711,402
RAC: 12,549
United States
Message 1092800 - Posted: 2 Apr 2011, 22:44:43 UTC - in response to Message 1092761.

Well if that isn't enough for you. Then you might want an 18 x16 slot board.


From that site:

Eighteen (18) PCI Express x16 card slots (mechanical)

PCI Express card slot electrical configuration:
Four (4) PCIe x8 and fourteen (14) PCIe x4


So that would fit 9 dual-slot cards, but would there be enough shared bandwidth between all the slots to handle 9 cards efficiently? It also looks like it's only PCI Express 1.0, so it has half the bandwidth shared between those 9 cards.

Profile Fred J. Verster
Volunteer tester
Avatar
Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,520
RAC: 119
Netherlands
Message 1092838 - Posted: 3 Apr 2011, 0:31:49 UTC - in response to Message 1092761.
Last modified: 3 Apr 2011, 0:32:51 UTC

When this kind of Power Computer {Cluster} becomes mainsteam, SETI might
not
been able to keep up whith the amount of data comming from and to this

Mega-Cruncher and a solid [i]High Speed
(A)DSL of at least 20Mbit/sec, which
isn't really fast, would be necessary, too. But it would be nice, to have one,
well you can always Dream about it which is totally free of any charge ;-)
(20MByte(160Mbit) per second is faster.)
____________

Previous · 1 · 2

Message boards : Number crunching : 8 GPUs

Copyright © 2014 University of California