Message boards :
Number crunching :
Perhaps some tweaks needed to newly installed GTX650ti cards...
Message board moderation
Author | Message |
---|---|
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
Hello Everyone, Firstly I want to thank everyone in my previous thread regarding graphics cards to purchase. Everyone was extremely helpful, as the forums always are, and if it wasn't for everyone helping out this project would not be as successful as it is today. I have recently installed two GTX650ti cards, and they have been running for about 20-36 hours or so, off and on. In that amount of time, I have gotten 25+ inconclusive results, which seems quite high, and 5 invalid work units. Granted my new cards are doing MORE work than my old cards, but with that said, my old cards did not produce this many inconclusives/invalid results. What I have done so far: 1) Upgraded to BOINC version 7.2.33 2) Upgraded my NVIDIA drivers to 332.21 3) Ran the lunatics installer, selecting cuda50. 4) Edited the app_info.xml to allow each GTX650ti card to run two work units at the same time. 5) Manually set the GPU fan speed to be 50%, instead of the 25% or so that was being done automatically. (likely inconsequential) Can anyone suggest what might be going wrong with my cards' settings? Is there anything wrong? Any discussion is much appreciated! |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
First Clue. What are the temps? 50% for the fans could not be enought when crunching. Could be other cause but high temps is one of the most common cause of problems. |
rob smith Send message Joined: 7 Mar 03 Posts: 22200 Credit: 416,307,556 RAC: 380 |
Check the temps Set the fans to 100% Check the temps Inconclusives are not a problem unless they end up as invalids, so keep an eye on the invalid count (which at 5 is a bit high given the time you've been running with two '650s Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
I am at work right now, so I am not sure. However, after running the cards overnight I woke up to them being at about 50C. I'll check to see if that is still the case when I get home. Thanks for the replies! |
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
Looks like all of your Invalids so far are coming from Device 2, so it may be just one card that's having a problem. All of them were either overflows or have Spike counts that are wildly out of line with your wingmen. |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
Looks like all of your Invalids so far are coming from Device 2, so it may be just one card that's having a problem. All of them were either overflows or have Spike counts that are wildly out of line with your wingmen. Hm. That is interesting. It has of course also provided valid results as well, so I am doubtful that it is defective...but that is a great observation Jeff. I didn't realize they were from the same card. When you guys tweak your GPU fan %, what do you put it to? Does it really just depend on what maintains a good temperature? Does everyone normally just put it to 100% and forget about it? |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
When you guys tweak your GPU fan %, what do you put it to? Does it really just depend on what maintains a good temperature? Does everyone normally just put it to 100% and forget about it? Use the fan at 100% all the time is not good for the fan life, the best thing to do is adjust the fan curve to match the target temp. In my case for example (others sure use diferent settings) i use 70C as the max temp on the GPU, so i set the fan to run at 85% if the temp < 70C, an 100% above that. 50% at 50C, etc. |
petri33 Send message Joined: 6 Jun 02 Posts: 1668 Credit: 623,086,772 RAC: 156 |
I keep my GPUs just under 70C (158F). -- cut -- forget -- off topic -- for those who are just bored -- Since my computer room is in the basement half underground i do not have to do any seasonal adjustments, 90% is sufficient for both AP and MB. The two GTX-780's that are now crunching 2x(4AP i.e 0.25 or 3MB i.e. 0.33) (( ASUS Direct Cu II) have a fixed 90% fan speed. Crunching 24/7/356,25. After drying the laundry the warm air flows upstairs helping keep the house warm 8 months out of 12. The remaining four - I'll just have to manage with the heat. -- bla bla ends -- this space is intentionally left blank -- To overcome Heisenbergs: "You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
Thanks everyone for the comments. I just got home and checked both cards. They have been running all day, and have not gone above 60C. The fan speed switched back to automatic for whatever reason, and seems to be hovering around 30%. With this information, it doesn't sound heat related. No more invalids have occured since I began this thread, however a few more inconclusives did. |
bill Send message Joined: 16 Jun 99 Posts: 861 Credit: 29,352,955 RAC: 0 |
First I'd set the gpu's to run one work unit at a time until you figure out the problem. Second run a program that allows you to run a graph of the heat like GPUZ or TTHROTTLE and get an idea os what's happening over time. Third set your fans back to automatic to see what they are doing in regards to heat. GPUZ will let you watch how the fans spin up as the heat increases. Post what happens. edit: You posted before I hit send. Still, leave the fans on automatic and watch to see if you get any spikes in heat over an hour or two. |
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
Thanks everyone for the comments. It seems odd, but the handful of your inconclusives that I looked all seem to have Spike counts that are just a tad higher than your wingman's. I wonder if there's some instability in your voltage. Perhaps your old PSU is having trouble keeping up with your new GPUs. (I don't claim to be an expert in this area, but it's just a thought.) |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
Thanks bill. I'll keep an eye on it, however it does seem quite stable over the last 24 hours or so.
Hm, that is an interesting thought. My power supply is 750W, and I was under the impression that this was sufficient for two of these cards. Granted, it is an old power supply. I don't know much about voltage, but the program I run to monitor heat also has a min/max statistic about voltage. One card has had a max of 1100, and a min of 1087. A difference of only 13. However, the second card has had a maximum of 1100, and a minimum of 1050. A difference of 50. Not sure if this matters, but it did catch my eye. Edit: I'll clarify a bit. The voltages aren't jumping around, and are currently stable and constant. This is a min/max over the past 10 hours. |
Cruncher-American Send message Joined: 25 Mar 02 Posts: 1513 Credit: 370,893,186 RAC: 340 |
750 watts should be OK, if the PSU is not so old that it has lost some of its power (a known symptom of PSU aging). I have two machines running 2 GTX 6xx cards each (all more powerful than yours), and they draw about 500 watts each. (Roughly speaking, 150 for each GPU, 100 for the CPU and about 100 for the MB, HD and fans). |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
750 watts should be OK, if the PSU is not so old that it has lost some of its power (a known symptom of PSU aging). I made a point to unplug all of the unnecessary things inside the case that aren't needed such as additional HD's and the CDROM. I hope that that means I am running a bit under the 750W max. Before I installed these 650ti's, I had these installed: NVIDIA GeForce 8800 GTS 512 (512MB) AMD Radeon HD 7700 series (Cape Verde) (1024MB). I'm not sure of the wattage on those cards, but they both ran issue free for about 6 months until I took them out this week. |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
The 750W good PSU is far enought to drive 2x650TI, i use one to drive 2x670 with no problem for years. A long shoot is to split the PSU conectors on the 2 GPU´s, if the problem changes that indicates the PSU not have a single rail 12V, some old PSU don´t use single 12V rail, if that your case, it could cause the problem you notice and that point to change the PSU. But be aware it´s a very very long shoot most of the newer PSU´s uses a single 12V rail. |
Jeff Buck Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0 |
750 watts should be OK, if the PSU is not so old that it has lost some of its power (a known symptom of PSU aging). Yeah, I would think 750w would be plenty. My most power-hungry machine, with a GTX670 and 2 GTX660s doesn't draw more than 590w on a 1000w PSU. Anyway, as I said, I'm not an expert but thought those odd and only occasional extra counts (as opposed to runaway overflows) seemed like there might be some kind of slight instability somewhere. |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
The 750W good PSU is far enought to drive 2x650TI, i use one to drive 2x670 with no problem for years. I believe that I have a separate rail going into each GPU. Are you suggesting that separate rails could be the issue?
I appreciate the input nonetheless. |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
As i said, this is a very very long shoot. Why this could happening? If you have separe rails (not usual), while one rail feed only one GPU, the other rail could feed the GPU + the entire MB and probabily the HDD, etc. So it could reach it maximum due de power spikes needed by the GPU to work in some random moments. Then the GPU feeded by this rail could start to produce some random wierd results like the ones you are talking. Switch the cables will easely check the theory if the problem disapears or changes from one GPU to the other. Remember at this time we are trying to catch some "ghosts on the machine" <edit> I just remember something, did your GPU has a separate power conector? (i don´t remember if the 650Ti has or not) if no and it uses the power direct from the PCI-e slot forget my ideia. If that is your case, switch the GPU´s itself, that could show you if you could have some trouble with the PCI-e Slot itself (another long shoot) |
_ Send message Joined: 15 Nov 12 Posts: 299 Credit: 9,037,618 RAC: 0 |
As i said, this is a very very long shoot. Why this could happening? If you have separe rails (not usual), while one rail feed only one GPU, the other rail could feed the GPU + the entire MB and probabily the HDD, etc. So it could reach it maximum due de power spikes needed by the GPU to work in some random moments. Then the GPU feeded by this rail could start to produce some random wierd results like the ones you are talking. Switch the cables will easely check the theory if the problem disapears or changes from one GPU to the other. Remember at this time we are trying to catch some "ghosts on the machine" The 650ti does indeed have a separate power connector, so I believe I will try your idea of switching the power rails momentarily! |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Switch and leave running for some time and see if something changes. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.