Problem with Geforce GTX 560 Ti and Cuda_fermi


log in

Advanced search

Questions and Answers : GPU applications : Problem with Geforce GTX 560 Ti and Cuda_fermi

1 · 2 · Next
Author Message
Ethelred
Send message
Joined: 8 Jan 11
Posts: 4
Credit: 224,172
RAC: 0
Sweden
Message 1085553 - Posted: 10 Mar 2011, 3:01:16 UTC

Yesterday i replaced my old Geforce GTS250 with the brand new (Gigabyte) Geforce GTX 560 Ti card and for some reason my system now choose to download "cuda_fermi" instead of "cuda23". As a result all my tasks are terminating after a few minutes and get the "Completed, validation inconclusive" message. I have already reinstalled the boinc manager.

Panic, i had to close down SETI@Home!
Does anyone have an idea how to fix this problem?

____________

The Weasel
Send message
Joined: 6 Jun 99
Posts: 127
Credit: 53,064,159
RAC: 4,772
United States
Message 1085559 - Posted: 10 Mar 2011, 3:16:50 UTC
Last modified: 10 Mar 2011, 3:19:38 UTC

Not sure if this is going to help, but what Boinc version are you using, also what drivers for your video card? Usually helpfull to post that, so the higher ups that know more can help you out. Also maybe take a screen shot of how the units are failing, so they can look at the individual workunit. Not sure, but thats where I would start.

Sorry about previous post, after looking at your profile. I am able to see most of the info I was asking about. Just was trying to help. You'll get help from someone that qualified, lol. I was just going by what I've seen in the past.
____________

Ethelred
Send message
Joined: 8 Jan 11
Posts: 4
Credit: 224,172
RAC: 0
Sweden
Message 1085569 - Posted: 10 Mar 2011, 3:46:02 UTC

Thanks "The Weasel", I have tried to reinstall the Boinc Manager and is now running version 6.10.58 (the latest version i guess) and the driver version is 8.17.12.6666 (also the latest).

The failure itself is not so dramaticaly. The progress of work is only 10 - 30 % done when it suddenly jumps to completed and the result is uploaded.
When someone has validated my work the validation is marked as inconclusive.

I am also running einstein@home and there everyting is fine, so i guess the problem is between me and the cuda_fermi application...

Example of WU: 708632842, 709030525, 709030564
Computer: 5834543 [/img]
____________

Profile Gundolf Jahn
Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 357,953
RAC: 37
Germany
Message 1085618 - Posted: 10 Mar 2011, 7:29:55 UTC - in response to Message 1085569.
Last modified: 10 Mar 2011, 7:42:27 UTC

The failure itself is not so dramaticaly. The progress of work is only 10 - 30 % done when it suddenly jumps to completed and the result is uploaded.

That's the usual behaviour of "normal" -9 result_overflow tasks: they run until
"the number of results detected exceeds the storage space allocated."

The reason why now "cuda_fermi" instead of "cuda23" tasks are downloaded is that you now have a fermi-type GPU.

When someone has validated my work the validation is marked as inconclusive.

Meanwhile, at least one task (1833957845) has been validated against a CPU and a cuda23 task, so it's probably just a bunch of tasks from the same tape with heavy radar blanking.

Gruß,
Gundolf
[edit]But nevertheless, check the temperature of your Fermi card under full load, as overheating might produce "false" -9 result_overflow results.[/edit]
____________
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours

Profile Joel
Send message
Joined: 31 Oct 08
Posts: 100
Credit: 4,576,268
RAC: 20
United States
Message 1085624 - Posted: 10 Mar 2011, 7:41:14 UTC
Last modified: 10 Mar 2011, 7:46:04 UTC

Cuda_fermi is the app that is designed for Nvidia Fermi GPUs (http://www.nvidia.com/object/fermi_architecture.html). They are different enough from previous Nvidia GPUs that they needed modifications to the CUDA app to process SETI@home work correctly. This is the app you should be using with your GTX 560.

As for your results, you're getting -9 errors, which means you are finding too many results in the WU to store. This can happen if there's unblanked radio interference, I believe. If that's the case, several clients will find the same thing and validate it. But your wingmen are not finding this, so something may actually be wrong here. How is the temperature of the card? Overheating can cause weird results. It's also possible that some of these will validate, because your wingmen happened to have a problem of their own and your -9 results are correct. See if a bunch of the inconclusive results were from the same wingman.

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 15,283,877
RAC: 11,734
United States
Message 1085777 - Posted: 10 Mar 2011, 18:31:25 UTC - in response to Message 1085624.
Last modified: 10 Mar 2011, 18:31:52 UTC

Nope, he only has two that validated and they were confirmed -9s. He's also starting to get invalid results coming back from his inconclusive results.

I would guess you tried getting the latest drivers from NVidia so I would suggest you check to make sure your card is seated completely and that your power supply is up to the task of running it. You might have gotten a bit of dust down in the connector when you seated the card so you might want to pull it and try reseating it. Also keep a close eye on your temps, they can be a real killer on these new cards.
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile Joel
Send message
Joined: 31 Oct 08
Posts: 100
Credit: 4,576,268
RAC: 20
United States
Message 1085822 - Posted: 10 Mar 2011, 20:59:40 UTC - in response to Message 1085777.

and that your power supply is up to the task of running it


Definitely check this. Insufficient power and overheating will often result in general instability, but they may also cause less noticeable problems if they are intermittent or just on the border of being acceptable.

Ethelred
Send message
Joined: 8 Jan 11
Posts: 4
Credit: 224,172
RAC: 0
Sweden
Message 1085896 - Posted: 11 Mar 2011, 1:25:57 UTC

Thank you for your suggestions, I will take a look at the power supply this weekend.

As for the temperature it looks OK. The GPU is stable at 60 degrees C running einstein@home at full trottle. By the way, does anyone know if there is any difference between seti and einstein when it comes to the Cuda apps? It is strange that i have no problem running einstein.
____________

Profile Joel
Send message
Joined: 31 Oct 08
Posts: 100
Credit: 4,576,268
RAC: 20
United States
Message 1085919 - Posted: 11 Mar 2011, 2:18:17 UTC - in response to Message 1085896.

By the way, does anyone know if there is any difference between seti and einstein when it comes to the Cuda apps? It is strange that i have no problem running einstein.


You can find out for yourself, to an extent, by using an app to monitor the GPU usage. Nvidia offers a monitor, and there are others like GPU-Z. See if the GPU is being utilized at 100% under Einstein. It generally will be under SETI, or at least we usually like it to. If it is not for the other project, Einstein is not pushing it as hard as SETI, so it is using less power and not getting as hot. To test temperature under SETI alone, suspend Einstein and let it only crunch SETI and see if it gets any hotter.

f_n_t
Volunteer tester
Send message
Joined: 30 Apr 02
Posts: 6
Credit: 32,071,236
RAC: 8,887
Japan
Message 1098575 - Posted: 18 Apr 2011, 2:13:26 UTC - in response to Message 1085553.

VDDC is not steady to Clock 900MHz of GIGABYTE Geforce GTX 560 Ti card because it is low with 1.000V.
The seti@CUDA stability operation limit frequency of each voltage of GIGABYTE Geforce GTX 560 Ti card (GV-N560OC-1GI) was measured with MSI Afterburner.
Let's increase this list in the standard and increase VDDC to 1.025V(@900MHz).
Or,
The stability operation is done to 850.5MHz(@1.000V) GPU Core Clock if it lowers.

VDDC , GPU Core Clock (GPU-Z 0.5.3)
1.100V 1012.5MHz
1.087V 996.9MHz
1.075V 981.3MHz
1.062V 967.5MHz
1.050V 950.2MHz
1.037V 934.6MHz
1.025V 918.0MHz
1.012V 904.5MHz
1.000V 885.9MHz
0.987V 872.3MHz
0.975V 850.5MHz
0.962V 837.0MHz
0.950V 810.0MHz

Do not you read easily because it translated Japanese in the machine?

Profile [AF>HFR]falcon
Send message
Joined: 16 May 99
Posts: 10
Credit: 139,937,967
RAC: 109,756
France
Message 1106478 - Posted: 15 May 2011, 15:54:38 UTC - in response to Message 1098575.

i also buy a GTX560TI and replace a GTX285
i checked the GTX560TI with somes games and a stability utily: VideoCardStabilityTestSetup.exe hardware is OK


after reinstall boinc

and i get also all my SETI@home WU finish in a couple of seconds without error
in 1 hour time the GTX560TI finished calculating more than 1000 WU !!

what the trouble ?
i run 2 GPU instances and re install BOINC with official binary:
setigraphics_6.03_windows_intelx86.exe
cudart32_30_14.dll
setiathome_6.03_windows_intelx86.exe
libfftw3f-3-1-1a_upx.dll
setiathome_6.10_windows_intelx86__cuda_fermi.exe
cufft32_30_14.dll

same result 1000wu in one hour on GTX560TI


i replace GTX560TI with the GTX285 everything is normal again (1 Wu in 20mn)
even with optimized binaries:
MB_6.08_CUDA_V12_VLARKill_FPLim2048.exe
AK_v8b_win_SSSE3x.exe
libfftw3f-3-1-1a_upx.dll
cufft.dll
AKv8SSSE3x.aistub

how can i solve this ?


my app_info.xml looks like that:

<app_info>
<app>
<name>setiathome_enhanced</name>
</app>
<file_info>
<name>AK_v8b_win_SSSE3x.exe</name>
<executable/>
</file_info>
<app_version>
<app_name>setiathome_enhanced</app_name>
<version_num>603</version_num>
<file_ref>
<file_name>AK_v8b_win_SSSE3x.exe</file_name>
<main_program/>
</file_ref>
</app_version>
<app>
<name>setiathome_enhanced</name>
</app>
<file_info>
<name>MB_6.08_CUDA_V12_VLARKill_FPLim2048.exe</name>
<executable/>
</file_info>
<file_info>
<name>cudart.dll</name>
<executable/>
</file_info>
<file_info>
<name>cufft.dll</name>
<executable/>
</file_info>
<file_info>
<name>libfftw3f-3-1-1a_upx.dll</name>
<executable/>
</file_info>
<app_version>
<app_name>setiathome_enhanced</app_name>
<version_num>608</version_num>
<plan_class>cuda</plan_class>
<avg_ncpus>0.040000</avg_ncpus>
<max_ncpus>0.040000</max_ncpus>
<coproc>
<type>CUDA</type>
<count>0.5</count>
</coproc>
<file_ref>
<file_name>MB_6.08_CUDA_V12_VLARKill_FPLim2048.exe</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>cudart.dll</file_name>
</file_ref>
<file_ref>
<file_name>cufft.dll</file_name>
</file_ref>
<file_ref>
<file_name>libfftw3f-3-1-1a_upx.dll</file_name>
</file_ref>
</app_version>
<app_version>
<app_name>setiathome_enhanced</app_name>
<version_num>608</version_num>
<plan_class>cuda23</plan_class>
<avg_ncpus>0.040000</avg_ncpus>
<max_ncpus>0.040000</max_ncpus>
<coproc>
<type>CUDA</type>
<count>0.5</count>
</coproc>
<file_ref>
<file_name>MB_6.08_CUDA_V12_VLARKill_FPLim2048.exe</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>cudart.dll</file_name>
</file_ref>
<file_ref>
<file_name>cufft.dll</file_name>
</file_ref>
<file_ref>
<file_name>libfftw3f-3-1-1a_upx.dll</file_name>
</file_ref>
</app_version>
</app_info>

____________

The Weasel
Send message
Joined: 6 Jun 99
Posts: 127
Credit: 53,064,159
RAC: 4,772
United States
Message 1106483 - Posted: 15 May 2011, 16:12:06 UTC

Try droping your drivers back down to 266.58, not sure if this is going to help you, but it is definately helping my systems out. The new 270.61 is not playing nice with lunatics optimised app in my situation. Cards are getting hot and unstable.
____________

Profile [AF>HFR]falcon
Send message
Joined: 16 May 99
Posts: 10
Credit: 139,937,967
RAC: 109,756
France
Message 1106576 - Posted: 15 May 2011, 20:46:43 UTC - in response to Message 1106483.

thank for the answer but i solve my problem with GTX560TI doing this:

1) restoring my windows system partition using clonezilla USB version
2) uninstalling boinc
3) removing all my boinc files
4) reinstalling boinc official version wu ok
5) reinstalling lunatics fermi application (Lunatics_Win32v0.37_AP505r409_AKv8b_Cudax32f.exe)

wu calculations complet well now even by using resheduler for VHAR VLAR wu (boinc_rescheduler_2_5.zip)

have a nice day
____________

Profile Lint trapProject donor
Send message
Joined: 30 May 03
Posts: 859
Credit: 26,250,153
RAC: 14,419
United States
Message 1106649 - Posted: 16 May 2011, 1:27:47 UTC - in response to Message 1106478.
Last modified: 16 May 2011, 1:35:06 UTC


i replace GTX560TI with the GTX285 everything is normal again (1 Wu in 20mn)
even with optimized binaries:
MB_6.08_CUDA_V12_VLARKill_FPLim2048.exe
AK_v8b_win_SSSE3x.exe
libfftw3f-3-1-1a_upx.dll
cufft.dll
AKv8SSSE3x.aistub

how can i solve this ?



Nvidia 400 and 500 series video cards use FERMI architecture.

You need to run a FERMI compatible CUDA application to get the most from the latest GPU's; use either stock S@H 6.10 or Lunatics x32f. A FERMI application will keep your card from falsely returning bad results (99.99% of the time, IME). The old Lunatics apps (V12 or earlier) should Not be used by newer Nvidia hardware. AFAIK, old video hardware will work fine with the new apps.

The latest Lunatics Windows installer packages are here: http://lunatics.kwsn.net/index.php?module=Downloads;catd=9

Martin


[Edit] Glad you got it solved with the new x32f app. Oh well, maybe this message will serve useful to the next guy who looks at this thread...

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 15,283,877
RAC: 11,734
United States
Message 1106814 - Posted: 16 May 2011, 17:26:07 UTC - in response to Message 1106576.

Just wanted to say thanks for stopping in Falcon. Many people have upgraded to the new Fermi type cards and haven't upgraded their opt_apps. This has been the topic of quite a few threads over in the number crunching forum. I and others have tried to contact these people as we notice them as our wing men but many don't have PM notification turned on and don't check the forums. It is really good to see someone taking the time to ask. :-)
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,985,137
RAC: 3,797
Bulgaria
Message 1107038 - Posted: 17 May 2011, 13:16:41 UTC - in response to Message 1106576.
Last modified: 17 May 2011, 13:30:23 UTC

thank for the answer but i solve my problem with GTX560TI doing this:

1) restoring my windows system partition using clonezilla USB version
2) uninstalling boinc
3) removing all my boinc files
4) reinstalling boinc official version wu ok
5) reinstalling lunatics fermi application (Lunatics_Win32v0.37_AP505r409_AKv8b_Cudax32f.exe)

wu calculations complet well now even by using rescheduler for VHAR VLAR wu (boinc_rescheduler_2_5.zip)

have a nice day

I think you overdid this - the only needed step is:

5) reinstalling lunatics fermi application (Lunatics_Win32v0.37_AP505r409_AKv8b_Cudax32f.exe)


(And maybe "Try dropping your drivers back down to 266.58" is good advice)

"have a nice day"? - I think you wanted to say "have a nice year" ;)
(Problem solved - bye till the next
(When they say SETI@home v7 will be "born"?)
)


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile 13G
Volunteer tester
Send message
Joined: 31 Jan 01
Posts: 10
Credit: 5,359,783
RAC: 288
Czech Republic
Message 1115926 - Posted: 11 Jun 2011, 15:53:14 UTC - in response to Message 1107038.
Last modified: 11 Jun 2011, 16:23:53 UTC

I have the same problem:

2x 460GTX using (nvidia 275.33)

setiathome_6.10_windows_intelx86__cuda_fermi.exe
or
Lunatics_Win32v0.37_AP505r409_AKv8b_Cudax32f.exe

everything OK.

I changed to 1x 560GTX Ti (gigabyte OC) HW-OK (nvidia 275.33)

setiathome_6.10_windows_intelx86__cuda_fermi.exe
or
Lunatics_Win32v0.37_AP505r409_AKv8b_Cudax32f.exe

are causing errors after 2-10% computing of WU, seems like finished and uploading...

nvidia 266.58 don't support GTX 560Ti, with 270.61 it is the same
____________

Profile 13G
Volunteer tester
Send message
Joined: 31 Jan 01
Posts: 10
Credit: 5,359,783
RAC: 288
Czech Republic
Message 1115941 - Posted: 11 Jun 2011, 16:39:03 UTC - in response to Message 1115926.

Now I have 2 cards in computer: 1x GTX 560Ti and 1x GTX 460.

nvidia 275.33

GTX460 computes correctly, but 560Ti not...
____________

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 15,283,877
RAC: 11,734
United States
Message 1115944 - Posted: 11 Jun 2011, 16:43:08 UTC - in response to Message 1115926.

13G,
Your computers are hidden so we can't check to see what the error might be. Please unhide them and someone will check it out.
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile 13G
Volunteer tester
Send message
Joined: 31 Jan 01
Posts: 10
Credit: 5,359,783
RAC: 288
Czech Republic
Message 1116168 - Posted: 12 Jun 2011, 7:50:25 UTC - in response to Message 1115944.
Last modified: 12 Jun 2011, 7:50:41 UTC

With

NVIDIA GPU 0: GeForce GTX 560 Ti (driver version 26666, CUDA version 3020, compute capability 2.1, 993MB, 922 GFLOPS peak)

with setiathome_6.10_windows_intelx86__cuda_fermi.exe

it is working correctly... maybe CUDA version problem?
____________

1 · 2 · Next

Questions and Answers : GPU applications : Problem with Geforce GTX 560 Ti and Cuda_fermi

Copyright © 2014 University of California