Computation error in all Seti GPU work

Message boards : Number crunching : Computation error in all Seti GPU work

1 · 2 · 3 · 4 · Next

Author	Message
Charlie Send message Joined: 4 Jul 00 Posts: 25 Credit: 1,426,700 RAC: 2	Message 1281467 - Posted: 8 Sep 2012, 15:02:23 UTC
	I need help sorting this problem out. Since installing a new graphics card GeForce GTX 660Ti all my Seti GPU work has failed with the message "Computation Error", usually after 20 seconds. The card is working fine, there is plenty of memory and it is running cool. Looking at the work unit, it gives the following details. Stderr output <core_client_version>7.0.28</core_client_version> <![CDATA[ <message> Incorrect function. (0x1) - exit code 1 (0x1) </message> <stderr_txt> setiathome_CUDA: Found 1 CUDA device(s): Device 1 : GeForce GTX 660 Ti totalGlobalMem = 2147155968 sharedMemPerBlock = 49152 regsPerBlock = 65536 warpSize = 32 memPitch = 2147483647 maxThreadsPerBlock = 1024 clockRate = 1045500 totalConstMem = 65536 major = 3 minor = 0 textureAlignment = 512 deviceOverlap = 1 multiProcessorCount = 7 setiathome_CUDA: CUDA Device 1 specified, checking... Device 1: GeForce GTX 660 Ti is okay SETI@home using CUDA accelerated device GeForce GTX 660 Ti setiathome_enhanced 6.09 Visual Studio/Microsoft C++ libboinc: 6.3.22 Work Unit Info: ............... WU true angle range is : 0.417776 Optimal function choices: ----------------------------------------------------- name ----------------------------------------------------- v_BaseLineSmooth (no other) v_GetPowerSpectrum 0.00027 0.00000 v_ChirpData 0.01776 0.00000 v_Transpose4 0.00431 0.00000 FPU opt folding 0.00515 0.00000 CUFFT error in file 'd:/Projects/SETI/seti_boinc/client/cuda/cudaAcc_fft.cu' in line 62. </stderr_txt> Any suggestions? Thanks
	ID: 1281467 ·

Charlie Send message Joined: 4 Jul 00 Posts: 25 Credit: 1,426,700 RAC: 2	Message 1281469 - Posted: 8 Sep 2012, 15:05:19 UTC - in response to Message 1281467.
	Just to add I have BOINC 7.0.28 and I've installed the latest drivers for the graphics card.
	ID: 1281469 ·

msattler Volunteer tester Send message Joined: 9 Jul 00 Posts: 34718 Credit: 355,317,411 RAC: 379,278	Message 1281470 - Posted: 8 Sep 2012, 15:08:44 UTC - in response to Message 1281467. Last modified: 8 Sep 2012, 15:09:23 UTC
	You are using a very new beta NV driver, might try backing down to the latest WHQL version, which should be 301.42 and see if that clears the errors. Choose custom installation, then clean install. ____________ ****** "Ask not, what your kitty can do for you. Ask what you can do for your kitty." As it is kitten, so shall it be done.
	ID: 1281470 ·

Charlie Send message Joined: 4 Jul 00 Posts: 25 Credit: 1,426,700 RAC: 2	Message 1281502 - Posted: 8 Sep 2012, 16:12:33 UTC - in response to Message 1281470.
	Version 301.42 wont install, I'm guessing that's because the 660Ti is very new. 305.68 will install, I'm trying to find if there are any earlier WHQL drivers that might install.
	ID: 1281502 ·

Richard Haselgrove Volunteer tester Send message Joined: 4 Jul 99 Posts: 7670 Credit: 40,279,114 RAC: 21,513	Message 1281532 - Posted: 8 Sep 2012, 17:09:13 UTC - in response to Message 1281470.
	You are using a very new beta NV driver, might try backing down to the latest WHQL version, which should be 301.42 and see if that clears the errors. Choose custom installation, then clean install. Oh, SH1T. (sorry - no, not you, Mark - nVidia) 305.68 IS the WHQL driver for a 660Ti (I checked under WindowsXP, as Charlie is using, but the same applies to Windows 7) We had a thread, a couple of months ago: GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error It looks as if the GTX 660Ti, driver 305.68 (WHQL) has exactly the same problem - it won't run the stock v6.10 'fermi' application which nVidia themselves supplied to the project. Deja vu, all over again. Charlie, let Mark guide you through the 'optimised apps' installation procedure while I go and knock Jason out of bed.
	ID: 1281532 ·

Charlie Send message Joined: 4 Jul 00 Posts: 25 Credit: 1,426,700 RAC: 2	Message 1281547 - Posted: 8 Sep 2012, 17:27:38 UTC - in response to Message 1281532.
	Sounds like this is going to be err... educational.
	ID: 1281547 ·

David Kirkby Send message Joined: 3 Apr 99 Posts: 6 Credit: 5,293,903 RAC: 7,899	Message 1281559 - Posted: 8 Sep 2012, 17:42:47 UTC - in response to Message 1281547.
	Hi, You just beat me to this question, may I add that mine crunches GPUGRID ok. ____________
	ID: 1281559 ·

Richard Haselgrove Volunteer tester Send message Joined: 4 Jul 99 Posts: 7670 Credit: 40,279,114 RAC: 21,513	Message 1281562 - Posted: 8 Sep 2012, 17:46:20 UTC - in response to Message 1281547. Last modified: 8 Sep 2012, 17:52:10 UTC
	Sounds like this is going to be err... educational. LOL. It's not too bad - just download and run the Lunatics installer. For a quick-and-dirty fix for your situation (and current workload), I suggest you: Untick the Astropulse section. Choose the SSSE3x option in the middle (Multibeam) section - note that's triple 'S'. Ensure the 'Multibeam (cuda)' section is ticked - it should be by default. Apart from that, just let it run like any other installer - please come back and let us know how it runs (always a tricky moment with these new cards...) Edit - my post was tailored for Charlie's Q9550 CPU. We'd normally advise the double-'S' SSE3 program for David's i7. Either will do for either of you - the differences are marginal. But if any AMD CPU users find this thread - ignore the CPU advice we gave to Charlie and David, make sure you choose an application marked as suitable for AMD.
	ID: 1281562 ·

Claggy Volunteer tester Send message Joined: 5 Jul 99 Posts: 3514 Credit: 26,388,900 RAC: 18,063	Message 1281563 - Posted: 8 Sep 2012, 17:46:59 UTC - in response to Message 1281547. Last modified: 8 Sep 2012, 17:48:03 UTC
	Sounds like this is going to be err... educational. Grab the Lunatics 0.40 Installer, the 32bit version in your case, Run it and install the SSSE3x MB app (that's fastest on a Core 2 processor), one of the AP apps (if you want to run AP), and the x41g Cuda32 app, http://lunatics.kwsn.net/index.php?module=Downloads;catd=9 Claggy
	ID: 1281563 ·

tbret Send message Joined: 28 May 99 Posts: 1915 Credit: 127,668,281 RAC: 87,269	Message 1281566 - Posted: 8 Sep 2012, 17:53:40 UTC - in response to Message 1281562.
	please come back and let us know how it runs (always a tricky moment with these new cards...) Richard, Just for reference, my dual 660Ti machine is running fine and has been since day one. I never tried the standard apps, so was unaware there was a problem. I'm sorry I didn't guinea pig that combination.
	ID: 1281566 ·

Richard Haselgrove Volunteer tester Send message Joined: 4 Jul 99 Posts: 7670 Credit: 40,279,114 RAC: 21,513	Message 1281570 - Posted: 8 Sep 2012, 17:58:26 UTC - in response to Message 1281566.
	please come back and let us know how it runs (always a tricky moment with these new cards...) Richard, Just for reference, my dual 660Ti machine is running fine and has been since day one. I never tried the standard apps, so was unaware there was a problem. I'm sorry I didn't guinea pig that combination. But that's with x41z, Cuda 4.20. You don't know of anyone on your team still running x41g, do you? Jason has gone off to test a potential environment variable fix, suggested by nVidia.
	ID: 1281570 ·

rob smith Volunteer moderator Send message Joined: 7 Mar 03 Posts: 5572 Credit: 22,639,187 RAC: 40,513	Message 1281580 - Posted: 8 Sep 2012, 18:24:58 UTC
	Charlie - Nvidia have just launched new drivers specifically for your 660ti - the 306.02 looks to be the one you need as you are running XP, presumably 32bit. Claim this version sorts all sorts of problems of incompatability beween your card and the 301 whql drivers. (Also a hint that Nvidia are getting close to launching the 7xx series of cards - I might hold off my mega machine for a bit and see what happens on that front) (checked to see what the current driver for the 690 is - its still the 301.42 - so you are at the bleeding edge) ____________ Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe?
	ID: 1281580 ·

Claggy Volunteer tester Send message Joined: 5 Jul 99 Posts: 3514 Credit: 26,388,900 RAC: 18,063	Message 1281581 - Posted: 8 Sep 2012, 18:26:28 UTC - in response to Message 1281570. Last modified: 8 Sep 2012, 18:28:19 UTC
	please come back and let us know how it runs (always a tricky moment with these new cards...) Richard, Just for reference, my dual 660Ti machine is running fine and has been since day one. I never tried the standard apps, so was unaware there was a problem. I'm sorry I didn't guinea pig that combination. But that's with x41z, Cuda 4.20. You don't know of anyone on your team still running x41g, do you? alephnull is running x41g_Cuda32 on his 690, but is only running 301.42 drivers, Claggy
	ID: 1281581 ·

David Kirkby Send message Joined: 3 Apr 99 Posts: 6 Credit: 5,293,903 RAC: 7,899	Message 1281583 - Posted: 8 Sep 2012, 18:30:53 UTC - in response to Message 1281581.
	Hi, I've just tried the Lunatics installer and it's worked. Cheers!! 2wu completed. ____________
	ID: 1281583 ·

tbret Send message Joined: 28 May 99 Posts: 1915 Credit: 127,668,281 RAC: 87,269	Message 1281587 - Posted: 8 Sep 2012, 18:35:18 UTC - in response to Message 1281581.
	alephnull is running x41g_Cuda32 on his 690, but is only running 301.42 drivers, Claggy The 660Ti won't run on those drivers. I tried. It has to have 305.x or later. I did experience that. It made me unhappy, temporarily.
	ID: 1281587 ·

tbret Send message Joined: 28 May 99 Posts: 1915 Credit: 127,668,281 RAC: 87,269	Message 1281588 - Posted: 8 Sep 2012, 18:37:30 UTC - in response to Message 1281570.
	But that's with x41z, Cuda 4.20. You don't know of anyone on your team still running x41g, do you? As always, your keen eye caught what I didn't think-of. No, I don't know anyone using a 660Ti and x41g.
	ID: 1281588 ·

Charlie Send message Joined: 4 Jul 00 Posts: 25 Credit: 1,426,700 RAC: 2	Message 1281593 - Posted: 8 Sep 2012, 18:45:13 UTC - in response to Message 1281580. Last modified: 8 Sep 2012, 18:46:43 UTC
	Alas, I've tried 306.02 and it still resulted in "Computation Error".
	ID: 1281593 ·

msattler Volunteer tester Send message Joined: 9 Jul 00 Posts: 34718 Credit: 355,317,411 RAC: 379,278	Message 1281598 - Posted: 8 Sep 2012, 18:52:44 UTC - in response to Message 1281593. Last modified: 8 Sep 2012, 18:53:06 UTC
	Alas, I've tried 306.02 and it still resulted in "Computation Error". Hey, Charlie.... Do the Lunatics opti install thingy.... It should fix the errors and speed up your processing to boot. It isn't hard, Richard linked in an earlier post. Just a few tick boxes to properly select, and you should be off and running. Any questions, post here and I or somebody else shall get you sorted. ____________ ****** "Ask not, what your kitty can do for you. Ask what you can do for your kitty." As it is kitten, so shall it be done.
	ID: 1281598 ·

Richard Haselgrove Volunteer tester Send message Joined: 4 Jul 99 Posts: 7670 Credit: 40,279,114 RAC: 21,513	Message 1281600 - Posted: 8 Sep 2012, 19:04:31 UTC Last modified: 8 Sep 2012, 19:30:38 UTC
	OK, been doing some bench testing. Using the 306.02 driver on my GTX 670 - closest I can get to reproducing the problem, without spending £250 tomorrow. The stock v6.10 application fails - as expected. The Lunatics x41g application - as included in the Lunatics v0.40 installer - works. The stock application can be persuaded to work by setting an environment variable, suggested by nVidia. This shows how to do it in Windows 7 - you have to be running as an administrator. I haven't tested yet whether it survives a reboot. (Edit - yes, it does) Make sure you copy the variable name exactly: CUDA_GRID_SIZE_COMPAT and give it the value 1 (one) And for XP, it's very similar:
	ID: 1281600 ·

Charlie Send message Joined: 4 Jul 00 Posts: 25 Credit: 1,426,700 RAC: 2	Message 1281604 - Posted: 8 Sep 2012, 19:15:17 UTC - in response to Message 1281562.
	/For a quick-and-dirty fix for your situation (and current workload), I suggest you: Untick the Astropulse section. Choose the SSSE3x option in the middle (Multibeam) section - note that's triple 'S'. Ensure the 'Multibeam (cuda)' section is ticked - it should be by default. Apart from that, just let it run like any other installer - please come back and let us know how it runs (always a tricky moment with these new cards...)/ Downloaded and installed. Now it's a matter of waiting for work units to test it on - seems to go a bit quiet at this time of day. Two questions: i) When installing I ticked the boxes as directed but was then given a second lot of boxes on the next "page" which were unticked by default. Not sure what they were for and I left them unticked. Was this the correct thing to do? ii) There's an application for Astropulse but it doesn't run as an installer and I can't see any guidance on how to install. Should I install this application and if so, how?
	ID: 1281604 ·

1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Computation error in all Seti GPU work