Can I select which WU/project goes to which GPU?

Message boards : Number crunching : Can I select which WU/project goes to which GPU?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1189977 - Posted: 30 Jan 2012, 3:58:58 UTC - in response to Message 1189973.  
Last modified: 30 Jan 2012, 4:05:46 UTC

That may be Jason, but I really can't afford to scrap a bunch of GTX295 cards plus water blocks and go for 3 GTX590 cards(water cooled), those 3 cards are just around $2900 or so, I might be able to raise enough for 2 EVGA GTX590 cards, but not 3.

So there's no way to raise the amount of shared system memory, as I've figured out that 5 gpus are using 639.8MB, Where as 6 gpus would use about 533.17MB, for 12 gpus It's half of that or about 266.58MB, One would think that's plenty, But It isn't, I have 4 dummy plugs and 2 monitors and I'm using 275.50 x64(Windows 7 Pro x64 sp2 w/16GB system ram) which at least doesn't do a BSOD on a reboot like the 280's do, which is a bug I've read.


Yeah, like I said, whether it's a backward step is a matter of your personal perspective. With older cards, there is always the (Vyper style) option to go to old drivers & applications on XP, which has it's own set of problems (older applications aren't as good, older XP drivers don't have the added reliability & flexibility afforded by the new features, etc...)

Under the new OS/models, A few minor things will help lift these limits a bit, such as:
- Tighten your system RAM latencies as much as you can, even if that means dropping the frequency a bit.
- Check your system DPC latencies (a newer modern term for 'interrupts'), with DPC latency checker (http://www.thesycon.de/deu/latency_check.shtml), which will help find any problematic motherboard chipset drivers or network card drivers etc, that can interfere with overall system multimedia performance. [Directly impacting GPU utilisation]
- do a 'Mark-style' analysis to find where the bottlenecks have really shifted to... you may even find some card VRAM speed tweaks can help...

It's not something nVidia can 'solve', through sheer hardware limitations against OS & driver change, but there are certainly things you can try to alleviate the bottlenecks as much as possible.

Jason
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1189977 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65689
Credit: 55,293,173
RAC: 49
United States
Message 1189981 - Posted: 30 Jan 2012, 4:13:27 UTC - in response to Message 1189977.  
Last modified: 30 Jan 2012, 4:14:55 UTC

That may be Jason, but I really can't afford to scrap a bunch of GTX295 cards plus water blocks and go for 3 GTX590 cards(water cooled), those 3 cards are just around $2900 or so, I might be able to raise enough for 2 EVGA GTX590 cards, but not 3.

So there's no way to raise the amount of shared system memory, as I've figured out that 5 gpus are using 639.8MB, Where as 6 gpus would use about 533.17MB, for 12 gpus It's half of that or about 266.58MB, One would think that's plenty, But It isn't, I have 4 dummy plugs and 2 monitors and I'm using 275.50 x64(Windows 7 Pro x64 sp2 w/16GB system ram) which at least doesn't do a BSOD on a reboot like the 280's do, which is a bug I've read.


Yeah, like I said, whether it's a backward step is a matter of your personal perspective. With older cards, there is always the (Vyper style) option to go to old drivers & applications on XP, which has it's own set of problems (older applications aren't as good, older XP drivers don't have the added reliability & flexibility afforded by the new features, etc...)

Under the new OS/models, A few minor things will help lift these limits a bit, such as:
- Tighten your system RAM latencies as much as you can, even if that means dropping the frequency a bit.
- Check your system DPC latencies (a newer modern term for 'interrupts'), with DPC latency checker (http://www.thesycon.de/deu/latency_check.shtml), which will help find any problematic motherboard chipset drivers or network card drivers etc, that can interfere with overall system multimedia performance. [Directly impacting GPU utilisation]
- do a 'Mark-style' analysis to find where the bottlenecks have really shifted to... you may even find some card VRAM speed tweaks can help...

It's not something nVidia can 'solve', through sheer hardware limitations against OS & driver change, but there are certainly things you can try to alleviate the bottlenecks as much as possible.

Jason

Well the program says My system can handle any real time video or audio data and the latency is low, like around 191us, max of 282us, sometimes as low as 60us, I could try to tighten the ram timing from 9-9-9-24 to 8-8-8-24 or so, It's 1333MHz ram that is running at 1500.4MHz, ram volts is at 1.55vdc of course.

One thing I do know is that of the ability to run older 64 bit drivers under Win 7, video isn't one of them, It's Vista/7 or nothing.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1189981 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1189985 - Posted: 30 Jan 2012, 4:44:18 UTC - in response to Message 1189981.  
Last modified: 30 Jan 2012, 4:45:02 UTC

Well the program says My system can handle any real time video or audio data and the latency is low, like around 191us, max of 282us, sometimes as low as 60us, I could try to tighten the ram timing from 9-9-9-24 to 8-8-8-24 or so, It's 1333MHz ram that is running at 1500.4MHz, ram volts is at 1.55vdc of course.

One thing I do know is that of the ability to run older 64 bit drivers under Win 7, video isn't one of them, It's Vista/7 or nothing.


Same here with sticking with Win7. As long as you don't get red peaks, it means your chipset & network drivers etc are likely OK & not part of the issues. That mostly leaves PCIe, VRAM & System RAM performance as the limiting factors on what you can do, so whatever you can tweak there to reduce contention will help.

One thing to try in BIOS, is a bit obscure. The default PCI latency timer is usually 32 cycles (of the 100MHz PCIe bus). You can wind that out to 64, 128, or sometimes 256 & see if there is any improvement. If PCI timeouts occur at the default setting due to more cards, what happens is transactions become fragmented, adding overhead, so a larger figure than typical BIOS defaults is worth trying on systems under heavy bus contention.

As far as PCIe bus frequency is concerned, many chipsets & cards won't cope well with anything but being locked at the default 100MHz, but sometimes you can squeeze an extra 5MHz or so out without ill effects. That can make a difference under the newer models as well. (Roll on PCIe V3!)

Jason
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1189985 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65689
Credit: 55,293,173
RAC: 49
United States
Message 1189986 - Posted: 30 Jan 2012, 5:08:19 UTC - in response to Message 1189985.  

Well the program says My system can handle any real time video or audio data and the latency is low, like around 191us, max of 282us, sometimes as low as 60us, I could try to tighten the ram timing from 9-9-9-24 to 8-8-8-24 or so, It's 1333MHz ram that is running at 1500.4MHz, ram volts is at 1.55vdc of course.

One thing I do know is that of the ability to run older 64 bit drivers under Win 7, video isn't one of them, It's Vista/7 or nothing.


Same here with sticking with Win7. As long as you don't get red peaks, it means your chipset & network drivers etc are likely OK & not part of the issues. That mostly leaves PCIe, VRAM & System RAM performance as the limiting factors on what you can do, so whatever you can tweak there to reduce contention will help.

One thing to try in BIOS, is a bit obscure. The default PCI latency timer is usually 32 cycles (of the 100MHz PCIe bus). You can wind that out to 64, 128, or sometimes 256 & see if there is any improvement. If PCI timeouts occur at the default setting due to more cards, what happens is transactions become fragmented, adding overhead, so a larger figure than typical BIOS defaults is worth trying on systems under heavy bus contention.

As far as PCIe bus frequency is concerned, many chipsets & cards won't cope well with anything but being locked at the default 100MHz, but sometimes you can squeeze an extra 5MHz or so out without ill effects. That can make a difference under the newer models as well. (Roll on PCIe V3!)

Jason

I did some research on the G.Skill 1333MHz Ripjaws ram that I have 4x4GB, It supposedly can do 7-7-7-21 @ 1333MHz and 1.50v, I'll leave the voltage alone as 1.55v isn't much on Monday, I don't see to well at night, so daytime is best, but change the rest and up the multi from 15 to 17 to get My current speed. The ram is doing 2T of course. I'm looking at the Asus P7P55D Pro manual(Bios 2003 is what I have installed and it's the latest too) for the PCI Latency setting, It may be fixed and/or hidden by Asus.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1189986 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65689
Credit: 55,293,173
RAC: 49
United States
Message 1189987 - Posted: 30 Jan 2012, 5:16:37 UTC - in response to Message 1189986.  

Well the program says My system can handle any real time video or audio data and the latency is low, like around 191us, max of 282us, sometimes as low as 60us, I could try to tighten the ram timing from 9-9-9-24 to 8-8-8-24 or so, It's 1333MHz ram that is running at 1500.4MHz, ram volts is at 1.55vdc of course.

One thing I do know is that of the ability to run older 64 bit drivers under Win 7, video isn't one of them, It's Vista/7 or nothing.


Same here with sticking with Win7. As long as you don't get red peaks, it means your chipset & network drivers etc are likely OK & not part of the issues. That mostly leaves PCIe, VRAM & System RAM performance as the limiting factors on what you can do, so whatever you can tweak there to reduce contention will help.

One thing to try in BIOS, is a bit obscure. The default PCI latency timer is usually 32 cycles (of the 100MHz PCIe bus). You can wind that out to 64, 128, or sometimes 256 & see if there is any improvement. If PCI timeouts occur at the default setting due to more cards, what happens is transactions become fragmented, adding overhead, so a larger figure than typical BIOS defaults is worth trying on systems under heavy bus contention.

As far as PCIe bus frequency is concerned, many chipsets & cards won't cope well with anything but being locked at the default 100MHz, but sometimes you can squeeze an extra 5MHz or so out without ill effects. That can make a difference under the newer models as well. (Roll on PCIe V3!)

Jason

I did some research on the G.Skill 1333MHz Ripjaws ram that I have 4x4GB, It supposedly can do 7-7-7-21 @ 1333MHz and 1.50v, I'll leave the voltage alone as 1.55v isn't much on Monday, I don't see to well at night, so daytime is best, but change the rest and up the multi from 15 to 17 to get My current speed. The ram is doing 2T of course. I'm looking at the Asus P7P55D Pro manual(Bios 2003 is what I have installed and it's the latest too) for the PCI Latency setting, It may be fixed and/or hidden by Asus.

Nope, that command doesn't seem to exist in the Bios of the Asus P7P55D Pro motherboard, best I can do is the timing, speed and if needed voltage and that's all folks.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1189987 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1189995 - Posted: 30 Jan 2012, 6:13:10 UTC - in response to Message 1189987.  

Nope, that command doesn't seem to exist in the Bios of the Asus P7P55D Pro motherboard, best I can do is the timing, speed and if needed voltage and that's all folks.


weird, oh well. Don't know about where that could be hiding on ASUS, if they use a different name hidden in advanced setting or whatever. Just one thing to try if you stumble across it, anyway.

Jason
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1189995 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65689
Credit: 55,293,173
RAC: 49
United States
Message 1189997 - Posted: 30 Jan 2012, 6:25:16 UTC - in response to Message 1189995.  

Nope, that command doesn't seem to exist in the Bios of the Asus P7P55D Pro motherboard, best I can do is the timing, speed and if needed voltage and that's all folks.


weird, oh well. Don't know about where that could be hiding on ASUS, if they use a different name hidden in advanced setting or whatever. Just one thing to try if you stumble across it, anyway.

Jason

No It's not a hidden that can be uncovered, It's just not there, Believe Me, I've been into every nook and cranny, If It existed I'd have run across
It by now, at least once.

My EVGA P55 Classified 200 has It of course, plus a few other tricks and goodies up its gargantuan sleeve, I did figure out how to maybe make a 4th GTX590 work on the EVGA, with an x16 extender cable, which might work...

But It would need 2-1500w psus(I have one now, but I'd need a 1250w to replace the 1st 1500w Silverstone with), a 10 slot case(9 slots minimum), and for Me 30 months time to acquire It all...

All for 8 gpus...


The Yellow wire of course supplies 12v to the slot as the cable doesn't transmit that, probably not a good idea putting 12v through a ribbon cable.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1189997 · Report as offensive
Profile -= Vyper =-
Volunteer tester
Avatar

Send message
Joined: 5 Sep 99
Posts: 1652
Credit: 1,065,191,981
RAC: 2,537
Sweden
Message 1190289 - Posted: 31 Jan 2012, 8:42:56 UTC - in response to Message 1189977.  
Last modified: 31 Jan 2012, 8:43:07 UTC

there is always the (Vyper style) option to go to old drivers


Lol! I'm refered to as Vyper style!
Wth! It's the only way i can fully load those beasts on that particular platform and god know why really.
As long as it works it works because i don't think that the speed benefits would be that much better on pre-fermi cards anyway.

It's simply as running the platform on what makes it works the best and for me it is to use the good old 190.38 driver with a quad cpu feeding octo gpu with no cpu tasks congesting the cpu pipes to free it for the GPU's..

Simply as that :)

Kind regards Vyper

_________________________________________________________________________
Addicted to SETI crunching!
Founder of GPU Users Group
ID: 1190289 · Report as offensive
Previous · 1 · 2

Message boards : Number crunching : Can I select which WU/project goes to which GPU?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.