Computation error in all Seti GPU work

Message boards : Number crunching : Computation error in all Seti GPU work
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Profile Charlie

Send message
Joined: 4 Jul 00
Posts: 25
Credit: 1,977,620
RAC: 0
United Kingdom
Message 1281467 - Posted: 8 Sep 2012, 15:02:23 UTC

I need help sorting this problem out.
Since installing a new graphics card GeForce GTX 660Ti all my Seti GPU work has failed with the message "Computation Error", usually after 20 seconds. The card is working fine, there is plenty of memory and it is running cool.

Looking at the work unit, it gives the following details.
Stderr output

<core_client_version>7.0.28</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
setiathome_CUDA: Found 1 CUDA device(s):
Device 1 : GeForce GTX 660 Ti
totalGlobalMem = 2147155968
sharedMemPerBlock = 49152
regsPerBlock = 65536
warpSize = 32
memPitch = 2147483647
maxThreadsPerBlock = 1024
clockRate = 1045500
totalConstMem = 65536
major = 3
minor = 0
textureAlignment = 512
deviceOverlap = 1
multiProcessorCount = 7
setiathome_CUDA: CUDA Device 1 specified, checking...
Device 1: GeForce GTX 660 Ti is okay
SETI@home using CUDA accelerated device GeForce GTX 660 Ti
setiathome_enhanced 6.09 Visual Studio/Microsoft C++
libboinc: 6.3.22

Work Unit Info:
...............
WU true angle range is : 0.417776
Optimal function choices:
-----------------------------------------------------
name
-----------------------------------------------------
v_BaseLineSmooth (no other)
v_GetPowerSpectrum 0.00027 0.00000
v_ChirpData 0.01776 0.00000
v_Transpose4 0.00431 0.00000
FPU opt folding 0.00515 0.00000
CUFFT error in file 'd:/Projects/SETI/seti_boinc/client/cuda/cudaAcc_fft.cu' in line 62.

</stderr_txt>

Any suggestions?
Thanks
ID: 1281467 · Report as offensive
Profile Charlie

Send message
Joined: 4 Jul 00
Posts: 25
Credit: 1,977,620
RAC: 0
United Kingdom
Message 1281469 - Posted: 8 Sep 2012, 15:05:19 UTC - in response to Message 1281467.  

Just to add I have BOINC 7.0.28 and I've installed the latest drivers for the graphics card.
ID: 1281469 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1281470 - Posted: 8 Sep 2012, 15:08:44 UTC - in response to Message 1281467.  
Last modified: 8 Sep 2012, 15:09:23 UTC

You are using a very new beta NV driver, might try backing down to the latest WHQL version, which should be 301.42 and see if that clears the errors.
Choose custom installation, then clean install.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1281470 · Report as offensive
Profile Charlie

Send message
Joined: 4 Jul 00
Posts: 25
Credit: 1,977,620
RAC: 0
United Kingdom
Message 1281502 - Posted: 8 Sep 2012, 16:12:33 UTC - in response to Message 1281470.  

Version 301.42 wont install, I'm guessing that's because the 660Ti is very new. 305.68 will install, I'm trying to find if there are any earlier WHQL drivers that might install.
ID: 1281502 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1281532 - Posted: 8 Sep 2012, 17:09:13 UTC - in response to Message 1281470.  

You are using a very new beta NV driver, might try backing down to the latest WHQL version, which should be 301.42 and see if that clears the errors.
Choose custom installation, then clean install.

Oh, SH1T.

(sorry - no, not you, Mark - nVidia)

305.68 IS the WHQL driver for a 660Ti (I checked under WindowsXP, as Charlie is using, but the same applies to Windows 7)

We had a thread, a couple of months ago:
GTX 680, NVIDIA drivers 304.48 (beta), CUDA 5.0 lead to Computation Error

It looks as if the GTX 660Ti, driver 305.68 (WHQL) has exactly the same problem - it won't run the stock v6.10 'fermi' application which nVidia themselves supplied to the project. Deja vu, all over again.

Charlie, let Mark guide you through the 'optimised apps' installation procedure while I go and knock Jason out of bed.
ID: 1281532 · Report as offensive
Profile Charlie

Send message
Joined: 4 Jul 00
Posts: 25
Credit: 1,977,620
RAC: 0
United Kingdom
Message 1281547 - Posted: 8 Sep 2012, 17:27:38 UTC - in response to Message 1281532.  

Sounds like this is going to be err... educational.
ID: 1281547 · Report as offensive
David Kirkby

Send message
Joined: 3 Apr 99
Posts: 6
Credit: 13,332,091
RAC: 36
United Kingdom
Message 1281559 - Posted: 8 Sep 2012, 17:42:47 UTC - in response to Message 1281547.  

Hi,
You just beat me to this question, may I add that mine crunches GPUGRID ok.
ID: 1281559 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1281562 - Posted: 8 Sep 2012, 17:46:20 UTC - in response to Message 1281547.  
Last modified: 8 Sep 2012, 17:52:10 UTC

Sounds like this is going to be err... educational.

LOL. It's not too bad - just download and run the Lunatics installer.

For a quick-and-dirty fix for your situation (and current workload), I suggest you:

Untick the Astropulse section.
Choose the SSSE3x option in the middle (Multibeam) section - note that's triple 'S'.
Ensure the 'Multibeam (cuda)' section is ticked - it should be by default.

Apart from that, just let it run like any other installer - please come back and let us know how it runs (always a tricky moment with these new cards...)

Edit - my post was tailored for Charlie's Q9550 CPU. We'd normally advise the double-'S' SSE3 program for David's i7. Either will do for either of you - the differences are marginal.

But if any AMD CPU users find this thread - ignore the CPU advice we gave to Charlie and David, make sure you choose an application marked as suitable for AMD.
ID: 1281562 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1281563 - Posted: 8 Sep 2012, 17:46:59 UTC - in response to Message 1281547.  
Last modified: 8 Sep 2012, 17:48:03 UTC

Sounds like this is going to be err... educational.

Grab the Lunatics 0.40 Installer, the 32bit version in your case, Run it and install the SSSE3x MB app (that's fastest on a Core 2 processor), one of the AP apps (if you want to run AP), and the x41g Cuda32 app,

http://lunatics.kwsn.net/index.php?module=Downloads;catd=9

Claggy
ID: 1281563 · Report as offensive
tbret
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 3380
Credit: 296,162,071
RAC: 40
United States
Message 1281566 - Posted: 8 Sep 2012, 17:53:40 UTC - in response to Message 1281562.  

please come back and let us know how it runs (always a tricky moment with these new cards...)


Richard,

Just for reference, my dual 660Ti machine is running fine and has been since day one. I never tried the standard apps, so was unaware there was a problem.

I'm sorry I didn't guinea pig that combination.
ID: 1281566 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1281570 - Posted: 8 Sep 2012, 17:58:26 UTC - in response to Message 1281566.  

please come back and let us know how it runs (always a tricky moment with these new cards...)

Richard,

Just for reference, my dual 660Ti machine is running fine and has been since day one. I never tried the standard apps, so was unaware there was a problem.

I'm sorry I didn't guinea pig that combination.

But that's with x41z, Cuda 4.20. You don't know of anyone on your team still running x41g, do you?

Jason has gone off to test a potential environment variable fix, suggested by nVidia.
ID: 1281570 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22158
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1281580 - Posted: 8 Sep 2012, 18:24:58 UTC

Charlie - Nvidia have just launched new drivers specifically for your 660ti - the 306.02 looks to be the one you need as you are running XP, presumably 32bit.
Claim this version sorts all sorts of problems of incompatability beween your card and the 301 whql drivers.


(Also a hint that Nvidia are getting close to launching the 7xx series of cards - I might hold off my mega machine for a bit and see what happens on that front)
(checked to see what the current driver for the 690 is - its still the 301.42 - so you are at the bleeding edge)
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1281580 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1281581 - Posted: 8 Sep 2012, 18:26:28 UTC - in response to Message 1281570.  
Last modified: 8 Sep 2012, 18:28:19 UTC

please come back and let us know how it runs (always a tricky moment with these new cards...)

Richard,

Just for reference, my dual 660Ti machine is running fine and has been since day one. I never tried the standard apps, so was unaware there was a problem.

I'm sorry I didn't guinea pig that combination.

But that's with x41z, Cuda 4.20. You don't know of anyone on your team still running x41g, do you?


alephnull is running x41g_Cuda32 on his 690, but is only running 301.42 drivers,

Claggy
ID: 1281581 · Report as offensive
David Kirkby

Send message
Joined: 3 Apr 99
Posts: 6
Credit: 13,332,091
RAC: 36
United Kingdom
Message 1281583 - Posted: 8 Sep 2012, 18:30:53 UTC - in response to Message 1281581.  

Hi,
I've just tried the Lunatics installer and it's worked. Cheers!! 2wu completed.
ID: 1281583 · Report as offensive
tbret
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 3380
Credit: 296,162,071
RAC: 40
United States
Message 1281587 - Posted: 8 Sep 2012, 18:35:18 UTC - in response to Message 1281581.  



alephnull is running x41g_Cuda32 on his 690, but is only running 301.42 drivers,

Claggy


The 660Ti won't run on those drivers. I tried. It has to have 305.x or later.

I did experience that. It made me unhappy, temporarily.
ID: 1281587 · Report as offensive
tbret
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 3380
Credit: 296,162,071
RAC: 40
United States
Message 1281588 - Posted: 8 Sep 2012, 18:37:30 UTC - in response to Message 1281570.  




But that's with x41z, Cuda 4.20. You don't know of anyone on your team still running x41g, do you?



As always, your keen eye caught what I didn't think-of.

No, I don't know anyone using a 660Ti and x41g.
ID: 1281588 · Report as offensive
Profile Charlie

Send message
Joined: 4 Jul 00
Posts: 25
Credit: 1,977,620
RAC: 0
United Kingdom
Message 1281593 - Posted: 8 Sep 2012, 18:45:13 UTC - in response to Message 1281580.  
Last modified: 8 Sep 2012, 18:46:43 UTC

Alas, I've tried 306.02 and it still resulted in "Computation Error".
ID: 1281593 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1281598 - Posted: 8 Sep 2012, 18:52:44 UTC - in response to Message 1281593.  
Last modified: 8 Sep 2012, 18:53:06 UTC

Alas, I've tried 306.02 and it still resulted in "Computation Error".

Hey, Charlie....
Do the Lunatics opti install thingy....
It should fix the errors and speed up your processing to boot.
It isn't hard, Richard linked in an earlier post.
Just a few tick boxes to properly select, and you should be off and running.

Any questions, post here and I or somebody else shall get you sorted.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1281598 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1281600 - Posted: 8 Sep 2012, 19:04:31 UTC
Last modified: 8 Sep 2012, 19:30:38 UTC

OK, been doing some bench testing. Using the 306.02 driver on my GTX 670 - closest I can get to reproducing the problem, without spending £250 tomorrow.

The stock v6.10 application fails - as expected.
The Lunatics x41g application - as included in the Lunatics v0.40 installer - works.

The stock application can be persuaded to work by setting an environment variable, suggested by nVidia.

This shows how to do it in Windows 7 - you have to be running as an administrator. I haven't tested yet whether it survives a reboot. (Edit - yes, it does)

Make sure you copy the variable name exactly:

CUDA_GRID_SIZE_COMPAT

and give it the value 1 (one)



And for XP, it's very similar:

ID: 1281600 · Report as offensive
Profile Charlie

Send message
Joined: 4 Jul 00
Posts: 25
Credit: 1,977,620
RAC: 0
United Kingdom
Message 1281604 - Posted: 8 Sep 2012, 19:15:17 UTC - in response to Message 1281562.  

/*For a quick-and-dirty fix for your situation (and current workload), I suggest you:

Untick the Astropulse section.
Choose the SSSE3x option in the middle (Multibeam) section - note that's triple 'S'.
Ensure the 'Multibeam (cuda)' section is ticked - it should be by default.

Apart from that, just let it run like any other installer - please come back and let us know how it runs (always a tricky moment with these new cards...)*/

Downloaded and installed. Now it's a matter of waiting for work units to test it on - seems to go a bit quiet at this time of day.

Two questions:
i) When installing I ticked the boxes as directed but was then given a second lot of boxes on the next "page" which were unticked by default. Not sure what they were for and I left them unticked. Was this the correct thing to do?

ii) There's an application for Astropulse but it doesn't run as an installer and I can't see any guidance on how to install. Should I install this application and if so, how?
ID: 1281604 · Report as offensive
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Computation error in all Seti GPU work


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.