Ubuntu 14.04 updates break cuda

Message boards : Number crunching : Ubuntu 14.04 updates break cuda
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile David Anderson (not *that* DA) Project Donor
Avatar

Send message
Joined: 5 Dec 09
Posts: 215
Credit: 74,008,558
RAC: 74
United States
Message 1613440 - Posted: 13 Dec 2014, 19:01:12 UTC

Updated kernel etc today. Minor update
supposedly. Ubuntu 14.04. x86_64
nvidia gtx 760SC (2 of them)
Suddenly CUDA ceased working, boinc
cannot find GPUs. Ubuntu bug 1401350.
Non-working driver: 331.113.
ID: 1613440 · Report as offensive
Wedge009
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 451
Credit: 431,396,357
RAC: 553
Australia
Message 1613743 - Posted: 14 Dec 2014, 6:57:24 UTC
Last modified: 14 Dec 2014, 6:57:36 UTC

Whenever there's a kernel update I uninstall any proprietary display drivers, update the kernel and then re-install the display drivers. It's a hassle, but I've had enough troubles with kernel updates breaking GPU processing that it's the procedure I follow every time nowadays.
Soli Deo Gloria
ID: 1613743 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1613859 - Posted: 14 Dec 2014, 14:11:12 UTC - in response to Message 1613743.  
Last modified: 14 Dec 2014, 14:13:52 UTC

Whenever there's a kernel update I uninstall any proprietary display drivers, update the kernel and then re-install the display drivers. It's a hassle, but I've had enough troubles with kernel updates breaking GPU processing that it's the procedure I follow every time nowadays.

That's a 'side effect'/'problem' of the nVidia driver being a (proprietary) "binary blob" that is tied to a particular kernel. Will need a reboot to pick up a new kernel and pull in the nVidia driver. (Your system must rerun the nVidia install for any kernel update... And both kernel and nVidia versions must be supported...)

Usually, that is all taken care of by the normal distro updates. I believe the nVidia licensing requires their proprietary bits be downloaded from the nVidia site and so you can never be sure of the nVidia process... NVidia can change things...


So far from what I've seen, that works well enough. However, the FLOSS nouveau driver is much to be preferred unless you're into ultimate performance gaming! ... Or cuda for crunching...


Happy fast crunch in
Martin
ID: 1613859 · Report as offensive
Profile David Anderson (not *that* DA) Project Donor
Avatar

Send message
Joined: 5 Dec 09
Posts: 215
Credit: 74,008,558
RAC: 74
United States
Message 1613899 - Posted: 14 Dec 2014, 16:47:30 UTC

Switched to Noveau driver.
reboot
(Aside: the boot screen is much prettier with Noveau driver
than with nvidia driver, noveau giving blue screen with cute
xubuntu image and fancy progress indicator, nvidia drivers
give boring black/white screen with dots progress indicator)
Removed all nvidia (with synaptic).
reboot
installed nvidia-331
reboot

lsmod:

nvidia-uvm did not get loaded. Did modprobe which
loaded it, but its use count is zero...

No GPU seen by boinc. So this is not a case of
the upgrade mechanism getting mixed up by my specific
history on the the machine.

While Seti has had no CUDA work for me for a while
Einstein has a steady CUDA supply (now those are stuck).
Reports on Ubuntu suggest this bug is maybe fixed(?) for
next release but I see no sign it's been backported to 14.04. Yet.
ID: 1613899 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1613903 - Posted: 14 Dec 2014, 17:11:18 UTC - in response to Message 1613899.  
Last modified: 14 Dec 2014, 17:11:56 UTC

Do you also need the "nv" module loading for nVidia? Or has that been renamed for the systemd way of working?

Also, do you have the libcuda files installed for your kernel version?

Or drop back to your previous kernel that worked?


Good luck,

Happy fast crunch in,
Martin
ID: 1613903 · Report as offensive
Profile David Anderson (not *that* DA) Project Donor
Avatar

Send message
Joined: 5 Dec 09
Posts: 215
Credit: 74,008,558
RAC: 74
United States
Message 1614000 - Posted: 14 Dec 2014, 21:20:47 UTC
Last modified: 14 Dec 2014, 21:21:22 UTC

For now just living with it. No time
to deal with this.
Plus I could make things much worse... :-)

Anyway, thanks for the tips, ML1.
ID: 1614000 · Report as offensive
Wedge009
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 451
Credit: 431,396,357
RAC: 553
Australia
Message 1614104 - Posted: 15 Dec 2014, 4:58:24 UTC

For Trusty Tahr, I have a recollection of manually linking the libOpenCL.so and libcuda.so in /usr/lib/x86_64/ to their respective .so.1 files. (This was with standard Ubuntu repositories, nvidia-331.)

For Utopic Unicorn I am using the xorg-edgers PPA and installed nvidia-340, nvidia-340-dev and nvidia-340-uvm. When I only had nvidia-340 installed previously, BOINC wouldn't recognise the CUDA/OpenCL libraries.
Soli Deo Gloria
ID: 1614104 · Report as offensive
Profile David Anderson (not *that* DA) Project Donor
Avatar

Send message
Joined: 5 Dec 09
Posts: 215
Credit: 74,008,558
RAC: 74
United States
Message 1617632 - Posted: 23 Dec 2014, 1:15:25 UTC
Last modified: 23 Dec 2014, 1:16:24 UTC

(was out of town a week, just now back).
Enabled Trusty-Backports so I could install
nvidia-modprobe. Installed it and ran it (executable
is the same name).
Made no visible difference in the output of
lsmod|grep nv
but after
sudo /etc/init.d/boinc-client restart
now boinc sees my GPUs and nvidia_uvm suddenly
shows up in the output of
lsmod|grep nv
!.

GPUs at work again!
See Ubuntu bug 1361207 for additional details if you care.
ID: 1617632 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1617959 - Posted: 23 Dec 2014, 23:23:25 UTC - in response to Message 1617632.  

Good stuff and good you're back on cuda.

Thanks for letting us know and for noting the bug.


Happy fast crunchin!
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1617959 · Report as offensive

Message boards : Number crunching : Ubuntu 14.04 updates break cuda


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.