GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error


log in

Advanced search

Message boards : Number crunching : GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error

1 · 2 · Next
Author Message
Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252153 - Posted: 27 Jun 2012, 17:19:38 UTC

Hello everyone,

these drivers...

NVIDIA GPU 0: GeForce GTX 680 (driver version 304.48, CUDA version 5.0, compute capability 3.0, 2048MB, 1916MB available, 3252 GFLOPS peak)
... cause computation errors after approximately 8 seconds on every SETI@Home CUDA work unit. Had to deactivate GPU computing for now.

I do require these drivers due to other bug fixes contained in them, so unless I'm a complete idiot, I think I'm out of luck for now :)

Just thought to mention this because I've not seen another post regarding this incompatibility yet--probably because the drivers are about a week old.

Thanks,
Toby
____________

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4066
Credit: 32,863,132
RAC: 6,791
United Kingdom
Message 1252166 - Posted: 27 Jun 2012, 17:35:42 UTC - in response to Message 1252153.

I do know the 6.09 Cuda23 app works with 304.48 Beta on legacy Nvidia GPUs like my 9800GTX+, not tried the 6.10 Cuda_Fermi app yet on anything newer,

You could try resetting the project, and seeing if freshly downloaded executionals and dll's help,

Claggy

Profile arkaynProject donor
Volunteer tester
Avatar
Send message
Joined: 14 May 99
Posts: 3617
Credit: 48,491,437
RAC: 38,829
United States
Message 1252169 - Posted: 27 Jun 2012, 17:38:21 UTC - in response to Message 1252153.

I do know that the drivers work fine with the optimized apps as I am also running 304.48 on both my 460 and 560.
____________

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252173 - Posted: 27 Jun 2012, 17:43:46 UTC

I'll try getting all new executables by resetting the project. That is actually a brilliant and simple idea (that I didn't think of lol).

If it doesn't work it must be a Kepler incompatibility, but the previous drivers worked. Let's see...

Thank you for the input and I will post with news.
____________

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252188 - Posted: 27 Jun 2012, 18:03:03 UTC

Nope, resetted project, new DLLs and whatnot didn't do anything. Still computation errors on every task.

Must be a Kepler incompatibility.
____________

Profile jason_geeProject donor
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 24 Nov 06
Posts: 4959
Credit: 73,032,672
RAC: 14,293
Australia
Message 1252193 - Posted: 27 Jun 2012, 18:09:17 UTC - in response to Message 1252153.
Last modified: 27 Jun 2012, 18:10:50 UTC

Hello everyone,

these drivers...
NVIDIA GPU 0: GeForce GTX 680 (driver version 304.48, CUDA version 5.0, compute capability 3.0, 2048MB, 1916MB available, 3252 GFLOPS peak)
... cause computation errors after approximately 8 seconds on every SETI@Home CUDA work unit. Had to deactivate GPU computing for now.

I do require these drivers due to other bug fixes contained in them, so unless I'm a complete idiot, I think I'm out of luck for now :)

Just thought to mention this because I've not seen another post regarding this incompatibility yet--probably because the drivers are about a week old.

Thanks,
Toby



Looks like you're using stock 6.10 (which mislabels itself online in stderr as 6.09)

Just found myself, on 680, this stock 6.10 app crashes under bench as well on this r304 Beta driver (for whatever reason).

6.10 of course *normally* runs 'fine' (but not 'good'), on 680.

Apart from rolling back your driver to last WHQL, x41g released optimised then is your only/best option for the moment.

Jason
____________
"It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change."
Charles Darwin

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252196 - Posted: 27 Jun 2012, 18:12:01 UTC - in response to Message 1252193.

Looks like you're using stock 6.10 (which mislabels itself online in stderr as 6.09)

Just found myself, on 680, this stock 6.10 app crashes under bench as well on this r304 Beta driver (for whatever reason).

6.10 of course *normally* runs 'fine' (but not 'good'), on 680.

Apart from rolling back your driver to last WHQL, x41g released optimised then is your only/best option for the moment.

Jason


Thank you!

Is there a bug tracker where I should be reporting this? I have no other option than to wait for a fix as I need the beta drivers in other applications.

Just wondering how to find out once it's fixed :)

Toby

____________

Kamu
Send message
Joined: 19 Jan 02
Posts: 56
Credit: 9,810,425
RAC: 0
Finland
Message 1252197 - Posted: 27 Jun 2012, 18:13:02 UTC

Hi

Tobias do you have Lunatics optimized apps?

I had same error with 304.48 and GTX 680 and without Lunatics 0.40.

Example of error here

http://setiathome.berkeley.edu/result.php?resultid=249574613

-Kimmo

____________
Computers: obelix

Profile jason_geeProject donor
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 24 Nov 06
Posts: 4959
Credit: 73,032,672
RAC: 14,293
Australia
Message 1252198 - Posted: 27 Jun 2012, 18:14:13 UTC - in response to Message 1252196.
Last modified: 27 Jun 2012, 18:46:18 UTC

Thank you!

Is there a bug tracker where I should be reporting this? I have no other option than to wait for a fix as I need the beta drivers in other applications.

Just wondering how to find out once it's fixed :)

Toby


I'm a registered Dev there & can report & see other reports... will see if it's already reported (probably is)

[A Bit Later:] I've reported it:


synopsis: GTX 680 + r304.48 beta driver leads to old Cufft 3.0 enabled apps to fail, worked fine on prior driver.

To Reproduce: With r304.48 beta driver Installed, and GTX 680,

visit seti@home ( http://setiathome.berkeley.edu/index.php )
- download & Install BOINC,
- when first run, attach to the the seti@home Boinc project
- allowing to fetch GPU tasks, allow the stock Cuda Fermi Cuda3 application (6.10) to download & start processing tasks

actual result: with 304.48 driver, every 6.10 task errors with a cufft error during startup within 10-20 seconds (cufft plan initialisations most likely) ....
Under 302.59 dev driver & earlier GTX 680 drivers, this Fermi compliant 6.10 application ran 'fine' (not 'great' but 'fine')

*note: This old stock Cuda 3.0 app application (6.10) is going to be replaced in the near future with improved work supporting newer processing. It's regarded as critical at this time, because if this change makes it to non-beta drivers, mass failures of existing users are highly likely, trashing numerous tasks.

Jason

____________
"It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change."
Charles Darwin

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252270 - Posted: 27 Jun 2012, 20:57:56 UTC

@jason_gee, thank you very much for taking care of this and filing the bug. I currently set SETI@home to not compute over GPU so I don't generate more failed units. :)

@Kamu, SETI@Home Enhanced 6.10 cuda_fermi are the only GPU apps I'm running.
____________

Profile Alex Storey
Volunteer tester
Avatar
Send message
Joined: 14 Jun 04
Posts: 536
Credit: 1,641,748
RAC: 521
Greece
Message 1252333 - Posted: 27 Jun 2012, 23:03:38 UTC - in response to Message 1252270.

Welcome to the boards Toby!

Jason is being too humble and everybody else is being too polite:D You should really, really, REALLY install the Lunatics apps. Even without the errors, it's just the right thing to do. Ask here for help on installing. You can always uninstall them if you want, but I've never, EVER heard anyone that wasn't greatful for their existence.

The only BUT that I can think of is that you have to check in to the forums once or twice a week to see if there have been any changes made. But since you have a 680 and are talking about bug trackers, I'm sure to you this is just common sense:) If you can't come to the forums that often, maybe you could get a Seti friend to promise to email/call/SMS (whatever) whenever changes are made to the project.

Profile Fred J. Verster
Volunteer tester
Send message
Joined: 21 Apr 04
Posts: 3236
Credit: 31,663,364
RAC: 5,739
Netherlands
Message 1252358 - Posted: 27 Jun 2012, 23:33:32 UTC - in response to Message 1252333.

Welcome to the boards Toby!

Jason is being too humble and everybody else is being too polite:D You should really, really, REALLY install the Lunatics apps. Even without the errors, it's just the right thing to do. Ask here for help on installing. You can always uninstall them if you want, but I've never, EVER heard anyone that wasn't greatful for their existence.

The only BUT that I can think of is that you have to check in to the forums once or twice a week to see if there have been any changes made. But since you have a 680 and are talking about bug trackers, I'm sure to you this is just common sense:) If you can't come to the forums that often, maybe you could get a Seti friend to promise to email/call/SMS (whatever) whenever changes are made to the project.


Alex, I can only agree on what you're saying ;-)


____________

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252641 - Posted: 28 Jun 2012, 14:06:33 UTC

Thank you for the welcome.

I must admit, I had no idea the project even existed. To me, all things SETI@Home started way back in 1999 but other than having it run a little on the side, I never looked into it that much. This doesn't mean it doesn't intrigue me, so I'll set it up soon :)

I got into GPU crunching with my new rig :D

Again thanks for the welcome, I'll look into Lunatics.
____________

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252694 - Posted: 28 Jun 2012, 16:37:23 UTC

I am confused and full of joy at the same time.
I'm assuming correctly that Lunatics provides optimized applications for setiathome_enchanced 6.10 cuda_fermi processing? Because I'm happy to announce that these don't crash. :D
____________

Profile MikeProject donor
Volunteer tester
Avatar
Send message
Joined: 17 Feb 01
Posts: 23762
Credit: 32,532,048
RAC: 24,336
Germany
Message 1252699 - Posted: 28 Jun 2012, 16:48:23 UTC

Congratz.
And it will process much faster.

____________

Profile Yanivicious
Avatar
Send message
Joined: 29 Mar 12
Posts: 157
Credit: 12,205,920
RAC: 9,606
United States
Message 1252722 - Posted: 28 Jun 2012, 17:18:57 UTC - in response to Message 1252694.

that is a pretty sweet rig you've got toby, the gtx 680 with lunatics app is probably the most punch packed by any crunching device, other than a gtx 690 (which is essentially 2 of your 680's)- i hope you get it fixed soon!

Profile Yanivicious
Avatar
Send message
Joined: 29 Mar 12
Posts: 157
Credit: 12,205,920
RAC: 9,606
United States
Message 1252729 - Posted: 28 Jun 2012, 17:22:45 UTC - in response to Message 1252694.

I am confused and full of joy at the same time.
I'm assuming correctly that Lunatics provides optimized applications for setiathome_enchanced 6.10 cuda_fermi processing? Because I'm happy to announce that these don't crash. :D


yes the lunatics installer .41 has all the programs built into it, and it considerably speeds up the rate at which WU's are completed. you cannot use the seti screensaver though if you are running the lunatics app, just FYI.

there is a thread on this number crunching board that is titled "lunatics installer release notes" or something like that, you should find it helpful.

Tobias Horvath
Avatar
Send message
Joined: 31 May 99
Posts: 8
Credit: 231,363
RAC: 0
Germany
Message 1252789 - Posted: 28 Jun 2012, 18:30:06 UTC

Thanks for all your help, it led to three things:

1) I made my first post on the forums since May 1999.
2) I now run Lunatics which seems great.
3) I no longer crash.

I'm using RSS to keep up to date with any Lunatics changes :)

Toby
____________

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8458
Credit: 48,672,085
RAC: 81,476
United Kingdom
Message 1253812 - Posted: 30 Jun 2012, 18:04:26 UTC - in response to Message 1252198.

[A Bit Later:] I've reported it:

synopsis: GTX 680 + r304.48 beta driver leads to old Cufft 3.0 enabled apps to fail, worked fine on prior driver.

To Reproduce: With r304.48 beta driver Installed, and GTX 680,

visit seti@home ( http://setiathome.berkeley.edu/index.php )
- download & Install BOINC,
- when first run, attach to the the seti@home Boinc project
- allowing to fetch GPU tasks, allow the stock Cuda Fermi Cuda3 application (6.10) to download & start processing tasks

actual result: with 304.48 driver, every 6.10 task errors with a cufft error during startup within 10-20 seconds (cufft plan initialisations most likely) ....
Under 302.59 dev driver & earlier GTX 680 drivers, this Fermi compliant 6.10 application ran 'fine' (not 'great' but 'fine')

*note: This old stock Cuda 3.0 app application (6.10) is going to be replaced in the near future with improved work supporting newer processing. It's regarded as critical at this time, because if this change makes it to non-beta drivers, mass failures of existing users are highly likely, trashing numerous tasks.

Jason

There's a very similar report on the Einstein message boards, relating to Linux driver 302.17 on a GT 640.

Amauri
Volunteer tester
Send message
Joined: 18 May 08
Posts: 26
Credit: 467,486
RAC: 284
Brazil
Message 1265736 - Posted: 30 Jul 2012, 2:32:36 UTC - in response to Message 1252153.

The recent NVidia drivers don't report the OpenCL capabilities to Boinc, like these lines in Boinc's "Event Log":

Dom 29 Jul 2012 17:24:08 BRT | | NVIDIA GPU 0: GeForce GT 640 (driver version unknown, CUDA version 4.20, compute capability 3.0, 2048MB, 134215651MB available, 692 GFLOPS peak)
Dom 29 Jul 2012 17:24:08 BRT | | OpenCL: NVIDIA GPU 0: GeForce GT 640 (driver version 295.49, device version OpenCL 1.1 CUDA, 2048MB, 134215651MB available)


Installed 302.17 (Linux), and that 2nd. line (OpenCL) is missing. Without OpenCL support, the GPU app can't run (??). Reverted to 295.49, and everything is ok.

Can anyone confirm this?

1 · 2 · Next

Message boards : Number crunching : GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error

Copyright © 2014 University of California