GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error

Message boards : Number crunching : GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252153 - Posted: 27 Jun 2012, 17:19:38 UTC

Hello everyone,

these drivers...
NVIDIA GPU 0: GeForce GTX 680 (driver version 304.48, CUDA version 5.0, compute capability 3.0, 2048MB, 1916MB available, 3252 GFLOPS peak)
... cause computation errors after approximately 8 seconds on every SETI@Home CUDA work unit. Had to deactivate GPU computing for now.

I do require these drivers due to other bug fixes contained in them, so unless I'm a complete idiot, I think I'm out of luck for now :)

Just thought to mention this because I've not seen another post regarding this incompatibility yet--probably because the drivers are about a week old.

Thanks,
Toby
ID: 1252153 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1252166 - Posted: 27 Jun 2012, 17:35:42 UTC - in response to Message 1252153.  

I do know the 6.09 Cuda23 app works with 304.48 Beta on legacy Nvidia GPUs like my 9800GTX+, not tried the 6.10 Cuda_Fermi app yet on anything newer,

You could try resetting the project, and seeing if freshly downloaded executionals and dll's help,

Claggy
ID: 1252166 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1252169 - Posted: 27 Jun 2012, 17:38:21 UTC - in response to Message 1252153.  

I do know that the drivers work fine with the optimized apps as I am also running 304.48 on both my 460 and 560.

ID: 1252169 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252173 - Posted: 27 Jun 2012, 17:43:46 UTC

I'll try getting all new executables by resetting the project. That is actually a brilliant and simple idea (that I didn't think of lol).

If it doesn't work it must be a Kepler incompatibility, but the previous drivers worked. Let's see...

Thank you for the input and I will post with news.
ID: 1252173 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252188 - Posted: 27 Jun 2012, 18:03:03 UTC

Nope, resetted project, new DLLs and whatnot didn't do anything. Still computation errors on every task.

Must be a Kepler incompatibility.
ID: 1252188 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1252193 - Posted: 27 Jun 2012, 18:09:17 UTC - in response to Message 1252153.  
Last modified: 27 Jun 2012, 18:10:50 UTC

Hello everyone,

these drivers...
NVIDIA GPU 0: GeForce GTX 680 (driver version 304.48, CUDA version 5.0, compute capability 3.0, 2048MB, 1916MB available, 3252 GFLOPS peak)
... cause computation errors after approximately 8 seconds on every SETI@Home CUDA work unit. Had to deactivate GPU computing for now.

I do require these drivers due to other bug fixes contained in them, so unless I'm a complete idiot, I think I'm out of luck for now :)

Just thought to mention this because I've not seen another post regarding this incompatibility yet--probably because the drivers are about a week old.

Thanks,
Toby



Looks like you're using stock 6.10 (which mislabels itself online in stderr as 6.09)

Just found myself, on 680, this stock 6.10 app crashes under bench as well on this r304 Beta driver (for whatever reason).

6.10 of course *normally* runs 'fine' (but not 'good'), on 680.

Apart from rolling back your driver to last WHQL, x41g released optimised then is your only/best option for the moment.

Jason
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1252193 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252196 - Posted: 27 Jun 2012, 18:12:01 UTC - in response to Message 1252193.  

Looks like you're using stock 6.10 (which mislabels itself online in stderr as 6.09)

Just found myself, on 680, this stock 6.10 app crashes under bench as well on this r304 Beta driver (for whatever reason).

6.10 of course *normally* runs 'fine' (but not 'good'), on 680.

Apart from rolling back your driver to last WHQL, x41g released optimised then is your only/best option for the moment.

Jason


Thank you!

Is there a bug tracker where I should be reporting this? I have no other option than to wait for a fix as I need the beta drivers in other applications.

Just wondering how to find out once it's fixed :)

Toby

ID: 1252196 · Report as offensive
Kamu

Send message
Joined: 19 Jan 02
Posts: 56
Credit: 11,009,499
RAC: 0
Finland
Message 1252197 - Posted: 27 Jun 2012, 18:13:02 UTC

Hi

Tobias do you have Lunatics optimized apps?

I had same error with 304.48 and GTX 680 and without Lunatics 0.40.

Example of error here

http://setiathome.berkeley.edu/result.php?resultid=249574613

-Kimmo

Computers: obelix
ID: 1252197 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1252198 - Posted: 27 Jun 2012, 18:14:13 UTC - in response to Message 1252196.  
Last modified: 27 Jun 2012, 18:46:18 UTC

Thank you!

Is there a bug tracker where I should be reporting this? I have no other option than to wait for a fix as I need the beta drivers in other applications.

Just wondering how to find out once it's fixed :)

Toby


I'm a registered Dev there & can report & see other reports... will see if it's already reported (probably is)

[A Bit Later:] I've reported it:


synopsis: GTX 680 + r304.48 beta driver leads to old Cufft 3.0 enabled apps to fail, worked fine on prior driver.

To Reproduce: With r304.48 beta driver Installed, and GTX 680,

visit seti@home ( http://setiathome.berkeley.edu/index.php )
- download & Install BOINC,
- when first run, attach to the the seti@home Boinc project
- allowing to fetch GPU tasks, allow the stock Cuda Fermi Cuda3 application (6.10) to download & start processing tasks

actual result: with 304.48 driver, every 6.10 task errors with a cufft error during startup within 10-20 seconds (cufft plan initialisations most likely) ....
Under 302.59 dev driver & earlier GTX 680 drivers, this Fermi compliant 6.10 application ran 'fine' (not 'great' but 'fine')

*note: This old stock Cuda 3.0 app application (6.10) is going to be replaced in the near future with improved work supporting newer processing. It's regarded as critical at this time, because if this change makes it to non-beta drivers, mass failures of existing users are highly likely, trashing numerous tasks.

Jason

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1252198 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252270 - Posted: 27 Jun 2012, 20:57:56 UTC

@jason_gee, thank you very much for taking care of this and filing the bug. I currently set SETI@home to not compute over GPU so I don't generate more failed units. :)

@Kamu, SETI@Home Enhanced 6.10 cuda_fermi are the only GPU apps I'm running.
ID: 1252270 · Report as offensive
Profile shizaru
Volunteer tester
Avatar

Send message
Joined: 14 Jun 04
Posts: 1130
Credit: 1,967,904
RAC: 0
Greece
Message 1252333 - Posted: 27 Jun 2012, 23:03:38 UTC - in response to Message 1252270.  

Welcome to the boards Toby!

Jason is being too humble and everybody else is being too polite:D You should really, really, REALLY install the Lunatics apps. Even without the errors, it's just the right thing to do. Ask here for help on installing. You can always uninstall them if you want, but I've never, EVER heard anyone that wasn't greatful for their existence.

The only BUT that I can think of is that you have to check in to the forums once or twice a week to see if there have been any changes made. But since you have a 680 and are talking about bug trackers, I'm sure to you this is just common sense:) If you can't come to the forums that often, maybe you could get a Seti friend to promise to email/call/SMS (whatever) whenever changes are made to the project.
ID: 1252333 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 1252358 - Posted: 27 Jun 2012, 23:33:32 UTC - in response to Message 1252333.  

Welcome to the boards Toby!

Jason is being too humble and everybody else is being too polite:D You should really, really, REALLY install the Lunatics apps. Even without the errors, it's just the right thing to do. Ask here for help on installing. You can always uninstall them if you want, but I've never, EVER heard anyone that wasn't greatful for their existence.

The only BUT that I can think of is that you have to check in to the forums once or twice a week to see if there have been any changes made. But since you have a 680 and are talking about bug trackers, I'm sure to you this is just common sense:) If you can't come to the forums that often, maybe you could get a Seti friend to promise to email/call/SMS (whatever) whenever changes are made to the project.


Alex, I can only agree on what you're saying ;-)


ID: 1252358 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252641 - Posted: 28 Jun 2012, 14:06:33 UTC

Thank you for the welcome.

I must admit, I had no idea the project even existed. To me, all things SETI@Home started way back in 1999 but other than having it run a little on the side, I never looked into it that much. This doesn't mean it doesn't intrigue me, so I'll set it up soon :)

I got into GPU crunching with my new rig :D

Again thanks for the welcome, I'll look into Lunatics.
ID: 1252641 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252694 - Posted: 28 Jun 2012, 16:37:23 UTC

I am confused and full of joy at the same time.
I'm assuming correctly that Lunatics provides optimized applications for setiathome_enchanced 6.10 cuda_fermi processing? Because I'm happy to announce that these don't crash. :D
ID: 1252694 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34249
Credit: 79,922,639
RAC: 80
Germany
Message 1252699 - Posted: 28 Jun 2012, 16:48:23 UTC

Congratz.
And it will process much faster.



With each crime and every kindness we birth our future.
ID: 1252699 · Report as offensive
Profile Yanivicious
Avatar

Send message
Joined: 29 Mar 12
Posts: 157
Credit: 15,529,301
RAC: 0
United States
Message 1252722 - Posted: 28 Jun 2012, 17:18:57 UTC - in response to Message 1252694.  

that is a pretty sweet rig you've got toby, the gtx 680 with lunatics app is probably the most punch packed by any crunching device, other than a gtx 690 (which is essentially 2 of your 680's)- i hope you get it fixed soon!
ID: 1252722 · Report as offensive
Profile Yanivicious
Avatar

Send message
Joined: 29 Mar 12
Posts: 157
Credit: 15,529,301
RAC: 0
United States
Message 1252729 - Posted: 28 Jun 2012, 17:22:45 UTC - in response to Message 1252694.  

I am confused and full of joy at the same time.
I'm assuming correctly that Lunatics provides optimized applications for setiathome_enchanced 6.10 cuda_fermi processing? Because I'm happy to announce that these don't crash. :D


yes the lunatics installer .41 has all the programs built into it, and it considerably speeds up the rate at which WU's are completed. you cannot use the seti screensaver though if you are running the lunatics app, just FYI.

there is a thread on this number crunching board that is titled "lunatics installer release notes" or something like that, you should find it helpful.
ID: 1252729 · Report as offensive
Tobias Horvath
Avatar

Send message
Joined: 31 May 99
Posts: 8
Credit: 236,113
RAC: 0
Germany
Message 1252789 - Posted: 28 Jun 2012, 18:30:06 UTC

Thanks for all your help, it led to three things:

1) I made my first post on the forums since May 1999.
2) I now run Lunatics which seems great.
3) I no longer crash.

I'm using RSS to keep up to date with any Lunatics changes :)

Toby
ID: 1252789 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1253812 - Posted: 30 Jun 2012, 18:04:26 UTC - in response to Message 1252198.  

[A Bit Later:] I've reported it:

synopsis: GTX 680 + r304.48 beta driver leads to old Cufft 3.0 enabled apps to fail, worked fine on prior driver.

To Reproduce: With r304.48 beta driver Installed, and GTX 680,

visit seti@home ( http://setiathome.berkeley.edu/index.php )
- download & Install BOINC,
- when first run, attach to the the seti@home Boinc project
- allowing to fetch GPU tasks, allow the stock Cuda Fermi Cuda3 application (6.10) to download & start processing tasks

actual result: with 304.48 driver, every 6.10 task errors with a cufft error during startup within 10-20 seconds (cufft plan initialisations most likely) ....
Under 302.59 dev driver & earlier GTX 680 drivers, this Fermi compliant 6.10 application ran 'fine' (not 'great' but 'fine')

*note: This old stock Cuda 3.0 app application (6.10) is going to be replaced in the near future with improved work supporting newer processing. It's regarded as critical at this time, because if this change makes it to non-beta drivers, mass failures of existing users are highly likely, trashing numerous tasks.

Jason

There's a very similar report on the Einstein message boards, relating to Linux driver 302.17 on a GT 640.
ID: 1253812 · Report as offensive
Amauri
Volunteer tester

Send message
Joined: 18 May 08
Posts: 26
Credit: 1,107,140
RAC: 0
Brazil
Message 1265736 - Posted: 30 Jul 2012, 2:32:36 UTC - in response to Message 1252153.  

The recent NVidia drivers don't report the OpenCL capabilities to Boinc, like these lines in Boinc's "Event Log":

Dom 29 Jul 2012 17:24:08 BRT | | NVIDIA GPU 0: GeForce GT 640 (driver version unknown, CUDA version 4.20, compute capability 3.0, 2048MB, 134215651MB available, 692 GFLOPS peak)
Dom 29 Jul 2012 17:24:08 BRT | | OpenCL: NVIDIA GPU 0: GeForce GT 640 (driver version 295.49, device version OpenCL 1.1 CUDA, 2048MB, 134215651MB available)


Installed 302.17 (Linux), and that 2nd. line (OpenCL) is missing. Without OpenCL support, the GPU app can't run (??). Reverted to 295.49, and everything is ok.

Can anyone confirm this?
ID: 1265736 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.