Linux CUDA 'Special' App finally available, featuring Low CPU use

Message boards : Number crunching : Linux CUDA 'Special' App finally available, featuring Low CPU use
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 30 · 31 · 32 · 33 · 34 · 35 · 36 . . . 83 · Next

AuthorMessage
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1873504 - Posted: 17 Jun 2017, 3:33:36 UTC

The zi3v scans the Wu and if it finds any suspects it runs the part pf the wu again with unroll 1. That idea came from jason_gee. I tried and coded it and That is what I'm running now. It may be more accurate and a bit slower. Just keep testing.
It seems to be better on the Pulses, but now I'm seeing a few Bad Best gaussian results. Can you also have it check the Best gaussians?
https://setiathome.berkeley.edu/workunit.php?wuid=2573994560
Best gaussian: peak=3.208247, mean=0.5376316, ChiSq=1.405401, time=37.75, d_freq=1419524825.06, score=-1.281584, null_hyp=2.177316, chirp=-74.201, fft_len=16k
Best gaussian: peak=3.839457, mean=0.570847, ChiSq=1.367193, time=88.08, d_freq=1419521976.84, score=-1.280418, null_hyp=2.15331, chirp=91.898, fft_len=16k

I'm waiting on these;
https://setiathome.berkeley.edu/workunit.php?wuid=2572586193
https://setiathome.berkeley.edu/workunit.php?wuid=2548202405
https://setiathome.berkeley.edu/workunit.php?wuid=2537036980

The new build is being run at Beta here, https://setiweb.ssl.berkeley.edu/beta/results.php?hostid=76256
ID: 1873504 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1873651 - Posted: 17 Jun 2017, 20:22:32 UTC
Last modified: 17 Jun 2017, 20:33:45 UTC

After 300 tasks at Beta, I see an example of the Bad Best gaussian; https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9814735
setiathome v8 enhanced x41p_zi3v
Best gaussian: peak=4.179945, mean=0.5265673, ChiSq=1.137641, time=89.76, d_freq=1418819127.86, score=1.12354, null_hyp=2.163572, chirp=48.412, fft_len=16k
SSE2x Linux64 Build 3584
Best gaussian: peak=4.148942, mean=0.5508794, ChiSq=1.387269, time=26, d_freq=1418820668.26, score=1.099337, null_hyp=2.292666, chirp=75.539, fft_len=16k

Otherwise the only 3 Inconclusives out of 300 tasks were against two Intel iGPUs and a Laptop running CUDA 32. Not bad.
ID: 1873651 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873674 - Posted: 17 Jun 2017, 21:30:03 UTC - in response to Message 1873651.  

That's great to hear.
ID: 1873674 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13732
Credit: 208,696,464
RAC: 304
Australia
Message 1873690 - Posted: 17 Jun 2017, 22:53:46 UTC - in response to Message 1873651.  

Otherwise the only 3 Inconclusives out of 300 tasks were against two Intel iGPUs and a Laptop running CUDA 32. Not bad.

1% i'd consider Excellent.
Grant
Darwin NT
ID: 1873690 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1873754 - Posted: 18 Jun 2017, 6:27:37 UTC - in response to Message 1873690.  
Last modified: 18 Jun 2017, 6:40:13 UTC

Yep, looking at a few tasks it looks as though the idea worked. +1 for 3am lightbulb moments. I guess how costly it is will come out over time. Will see if I can spot a Gaussian issue. Usually those issues come from minor precision issues in the normalisation sums (easily checked and remedied). @Petri33 please email latest and I'll update the alpha folder ASAP.

Earlier than was planned, due to Petri's pushing the technology so far, it opens multiple doors for 'next level' pre-scans. Namely Wavelet &/or Convolutional Neural Network based feature recognition, followed up with sparser traditional Fourier processing so as to get the same numbers.

[Sidenote:]
At present, still wrestling with home & work issues myself, though We get semi decent internet here in a month or so. Sick of working for the benefit of other bosses, I'll probably end up streaming during development at some point, and work in the Open Source stuff somehow. Probably priority would be on survival and having a laugh, we'll see.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1873754 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873757 - Posted: 18 Jun 2017, 7:07:16 UTC - in response to Message 1873754.  

Yep, looking at a few tasks it looks as though the idea worked. +1 for 3am lightbulb moments. I guess how costly it is will come out over time. Will see if I can spot a Gaussian issue. Usually those issues come from minor precision issues in the normalisation sums (easily checked and remedied). @Petri33 please email latest and I'll update the alpha folder ASAP.


. . Does any of this portend a possible Windows version anytime in the forseeable future ??

Stephen

??
ID: 1873757 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1873761 - Posted: 18 Jun 2017, 7:17:25 UTC - in response to Message 1873757.  
Last modified: 18 Jun 2017, 7:19:06 UTC

Yep, looking at a few tasks it looks as though the idea worked. +1 for 3am lightbulb moments. I guess how costly it is will come out over time. Will see if I can spot a Gaussian issue. Usually those issues come from minor precision issues in the normalisation sums (easily checked and remedied). @Petri33 please email latest and I'll update the alpha folder ASAP.


. . Does any of this portend a possible Windows version anytime in the forseeable future ??

Stephen

??


Sure. I've built Windows ones on multiple occasions, which is how I stumbled on the unroll validation problem and possible fix (lots of help from TBar and Jeff Buck in spotting the patterns solved that). I only sat on them because the scope of damage saturating results with 1000 Windows hosts would potentially compromise the project too far.

Probably there will still be niggles to address, and the high-end device requirements will need tailoring to solve, Though advanced user 'Use at you own risk' Windows builds become viable once again.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1873761 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873767 - Posted: 18 Jun 2017, 7:46:51 UTC - in response to Message 1873761.  


. . Does any of this portend a possible Windows version anytime in the forseeable future ??
Stephen


Sure. I've built Windows ones on multiple occasions, which is how I stumbled on the unroll validation problem and possible fix (lots of help from TBar and Jeff Buck in spotting the patterns solved that). I only sat on them because the scope of damage saturating results with 1000 Windows hosts would potentially compromise the project too far.

Probably there will still be niggles to address, and the high-end device requirements will need tailoring to solve, Though advanced user 'Use at you own risk' Windows builds become viable once again.


. . Hmm, I would be up for that.

Stephen

:)
ID: 1873767 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1873782 - Posted: 18 Jun 2017, 10:54:19 UTC - in response to Message 1873651.  
Last modified: 18 Jun 2017, 10:56:02 UTC

After 300 tasks at Beta, I see an example of the Bad Best gaussian; https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9814735
setiathome v8 enhanced x41p_zi3v
Best gaussian: peak=4.179945, mean=0.5265673, ChiSq=1.137641, time=89.76, d_freq=1418819127.86, score=1.12354, null_hyp=2.163572, chirp=48.412, fft_len=16k
SSE2x Linux64 Build 3584
Best gaussian: peak=4.148942, mean=0.5508794, ChiSq=1.387269, time=26, d_freq=1418820668.26, score=1.099337, null_hyp=2.292666, chirp=75.539, fft_len=16k
Hmmm, maybe it's the 'other' Apps that's having gaussian problems. Besides the above task, I have two others that were decided in favor of CUDA.
The Above with SETI@home v8 v8.05: Best gaussian: peak=4.179937, mean=0.5265682, ChiSq=1.13764, time=89.76, d_freq=1418819127.86, score=1.123425, null_hyp=2.163565, chirp=48.412, fft_len=16k

Then these; https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9816865 & https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9815080

Anyway, Beta started sending out Arecibo VLARs, so, I stopped that. Back to Main for now.
ID: 1873782 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1873783 - Posted: 18 Jun 2017, 11:15:11 UTC - in response to Message 1873782.  

It's possible. We need to always reference to Win32 stock CPU, just because Eric Made it. It's not that 1-5% is problematic, just that every goal needs a reference.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1873783 · Report as offensive
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1873882 - Posted: 18 Jun 2017, 21:19:52 UTC - in response to Message 1873754.  

@Petri33 please email latest and I'll update the alpha folder ASAP.

Emailed.
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1873882 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1873904 - Posted: 18 Jun 2017, 23:26:32 UTC - in response to Message 1873767.  

. . Hmm, I would be up for that.

Stephen

:)


If I understood the conversation right, I would be willing to alpha/beta test any windows-based Seti program too.

Tom
A proud member of the OFA (Old Farts Association).
ID: 1873904 · Report as offensive
Jim1348

Send message
Joined: 13 Dec 01
Posts: 212
Credit: 520,150
RAC: 0
United States
Message 1873926 - Posted: 19 Jun 2017, 1:45:57 UTC - in response to Message 1871570.  
Last modified: 19 Jun 2017, 2:00:06 UTC

I am still at a loss to get BOINC installed.

. . One thing that has cause me to lose even more hair is that when I attempted to copy the download files I have on one of the working Linux rigs onto the new rig via a Flashdrive I ran head on into permission/ownership issues. One of the files I was able to copy was the BOINC install file, but could that be why I cannot install it ??

This works for me on Ubuntu 16 (and 17). I don't know why they make it such a hassle, or why no one has a standard set of installation instructions. I had to gather them up in bits and pieces. Note that it includes installing the files (“gui_rpc_auth.cfg” and “remote_hosts.cfg”) that are used for monitoring BOINC over a LAN using BOINCTasks on a Windows machine, so you may not need them.
Note that "USER" is of course your user name.


BOINC: Search for BOINC in Ubuntu Software and install
• Join the root group: sudo adduser "USER" root
• Join the BOINC group: sudo adduser "USER" boinc
• Allow group to read, write and execute in /etc/boinc-client folder:
sudo chmod -R g+rwx /etc/boinc-client
• Allow group to read, write and execute in /var/lib/boinc-client:
sudo chmod -R g+rwx /var/lib/boinc-client
• Reboot
• Copy “gui_rpc_auth.cfg” to /etc/boinc-client folder
• Copy “remote_hosts.cfg” to /etc/boinc-client folder
• Copy “cc_config.xml” (set half-life to 1 day) to /etc/boinc-client folder (if needed)
• Copy "gui_rpc_auth.cfg" to the home directory and reboot
• Copy the "app_config.xml" files to the project folders in /var/lib/boinc/ (if needed)


ID: 1873926 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1873928 - Posted: 19 Jun 2017, 1:59:23 UTC - in response to Message 1873926.  
Last modified: 19 Jun 2017, 2:16:55 UTC

Thanks Jim for the insight on how to change permissions for USER access to the 'simple' install of BOINC.
Notes have been added to my tickle trunk :D

Brent

EDIT:
Jim, did you get BOINC v7.6.33 from the Ubuntu Software in v16/17?
Ubuntu 14.04 distributes BOINC v7.2.42
Just curious ...
ID: 1873928 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873937 - Posted: 19 Jun 2017, 4:12:30 UTC - in response to Message 1873928.  

Thanks Jim for the insight on how to change permissions for USER access to the 'simple' install of BOINC.
Notes have been added to my tickle trunk :D

Brent

EDIT:
Jim, did you get BOINC v7.6.33 from the Ubuntu Software in v16/17?
Ubuntu 14.04 distributes BOINC v7.2.42
Just curious ...


. . If you are running the later versions of Ubuntu you can run the repository version of BOINC and you get 7.6.33, but then you have your BOINC files scattered over several folders/directories. If you want BOINC to be installed under your 'Home' directory for ease of access and backing up then you need the Seti version of BOINC and you get 7.2.42 but are restricted to the earlier version of Ubuntu 14.04. Horses for courses.

Stephen

..
ID: 1873937 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1873938 - Posted: 19 Jun 2017, 4:13:42 UTC - in response to Message 1873926.  

I am still at a loss to get BOINC installed.

. . One thing that has cause me to lose even more hair is that when I attempted to copy the download files I have on one of the working Linux rigs onto the new rig via a Flashdrive I ran head on into permission/ownership issues. One of the files I was able to copy was the BOINC install file, but could that be why I cannot install it ??

This works for me on Ubuntu 16 (and 17). I don't know why they make it such a hassle, or why no one has a standard set of installation instructions. I had to gather them up in bits and pieces. Note that it includes installing the files (“gui_rpc_auth.cfg” and “remote_hosts.cfg”) that are used for monitoring BOINC over a LAN using BOINCTasks on a Windows machine, so you may not need them.
Note that "USER" is of course your user name.


BOINC: Search for BOINC in Ubuntu Software and install
• Join the root group: sudo adduser "USER" root
• Join the BOINC group: sudo adduser "USER" boinc
• Allow group to read, write and execute in /etc/boinc-client folder:
sudo chmod -R g+rwx /etc/boinc-client
• Allow group to read, write and execute in /var/lib/boinc-client:
sudo chmod -R g+rwx /var/lib/boinc-client
• Reboot
• Copy “gui_rpc_auth.cfg” to /etc/boinc-client folder
• Copy “remote_hosts.cfg” to /etc/boinc-client folder
• Copy “cc_config.xml” (set half-life to 1 day) to /etc/boinc-client folder (if needed)
• Copy "gui_rpc_auth.cfg" to the home directory and reboot
• Copy the "app_config.xml" files to the project folders in /var/lib/boinc/ (if needed)



. . Those pointers could solve several issues I am having. Even though I am running 14.04 and BOINC 7.2.42, like Brent they are going to be saved here.

Stephen

:)
ID: 1873938 · Report as offensive
Jim1348

Send message
Joined: 13 Dec 01
Posts: 212
Credit: 520,150
RAC: 0
United States
Message 1873997 - Posted: 19 Jun 2017, 14:15:23 UTC - in response to Message 1873928.  

EDIT:
Jim, did you get BOINC v7.6.33 from the Ubuntu Software in v16/17?
Ubuntu 14.04 distributes BOINC v7.2.42
Just curious ...

Ubuntu 16.04 gives you BOINC 7.6.31, whereas Ubuntu 16.10 (and 17) gives you BOINC 7.6.33.
There is really no difference between them for Linux. I have had no problems with them.
ID: 1873997 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1874019 - Posted: 19 Jun 2017, 15:57:44 UTC - in response to Message 1873783.  

It's possible. We need to always reference to Win32 stock CPU, just because Eric Made it. It's not that 1-5% is problematic, just that every goal needs a reference.
The Plot thickens...
Ran a few hundred more tasks and ran across this one, https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9821403
setiathome v8 enhanced x41p_zi3v
Best gaussian: peak=4.149051, mean=0.5146711, ChiSq=1.19371, time=79.69, d_freq=1420106366.11, score=2.326251, null_hyp=2.25355, chirp=-71.774, fft_len=16k

SSE3xj Win32 Build 3584
Best gaussian: peak=3.718165, mean=0.5199579, ChiSq=1.359161, time=81.37, d_freq=1420106245.69, score=2.270708, null_hyp=2.337314, chirp=-71.774, fft_len=16k

I ran it with My CPU App, SSE4.1xjf OS X 64bit Build 3344
Best gaussian: peak=4.14904, mean=0.5146721, ChiSq=1.193707, time=79.69, d_freq=1420106366.11, score=2.326065, null_hyp=2.253538, chirp=-71.774, fft_len=16k

You can run it yourself if you wish, you might want to hurry though; http://boinc2.ssl.berkeley.edu/beta/download/62/04oc08ab.31484.890.13.47.11
The third Host is also running SETI@home v8 v8.22 (opencl_nvidia_SoG) windows_intelx86, so, I suspect My results will be overruled.
That adds the Windows version to the Linux version of SoG that appear to be giving the wrong Best Gaussian results on occasion.
I believe I remember a discussion about this very problem at Beta some time ago, I think it was agreed the Best Gaussian didn't necessarily have to be listed with the signals to be correct.
It's probably still there if someone wants to track it down.
ID: 1874019 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1874029 - Posted: 19 Jun 2017, 17:06:11 UTC - in response to Message 1874019.  
Last modified: 19 Jun 2017, 17:09:24 UTC

Saved for analysis when I can. Will probably see what win32 CPU stock and ye-olde Cuda turn up. There are some differences between the way stock cpu and 3rd party cpu sum, with the former having been refined to greater portability & less cumulative error. Wouldn't have thought that alone could explain that great of a difference in numbers though, so other explanations could be in the compilation used etc. The lack of 80-bit FPU availability in the later apps can be problematic there if not handled carefully. That's just some possibilities assuming no lingering bugs in either case though, which could easily have been buried in the noise of prior other issues amongst different applications.

Overall I'd suspect the SSE3xj Win32 gaussian values are 'more correct', as since the powerspectrum values are positive any increase in cumulative error tends to push peaks higher rather than lower. [Edit: hmmm, a different time value too]
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1874029 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1874034 - Posted: 19 Jun 2017, 17:37:10 UTC - in response to Message 1874029.  
Last modified: 19 Jun 2017, 18:35:06 UTC

Overall I'd suspect the SSE3xj Win32 gaussian values are 'more correct', as since the powerspectrum values are positive any increase in cumulative error tends to push peaks higher rather than lower. [Edit: hmmm, a different time value too]
It's basically a replay of yesterday's task where the Windows CPU App agreed with CUDA zi3v over Build 3584 SoG;
After 300 tasks at Beta, I see an example of the Bad Best gaussian; https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9814735

setiathome v8 enhanced x41p_zi3v
Best gaussian: peak=4.179945, mean=0.5265673, ChiSq=1.137641, time=89.76, d_freq=1418819127.86, score=1.12354, null_hyp=2.163572, chirp=48.412, fft_len=16k

SSE2x Linux64 Build 3584
Best gaussian: peak=4.148942, mean=0.5508794, ChiSq=1.387269, time=26, d_freq=1418820668.26, score=1.099337, null_hyp=2.292666, chirp=75.539, fft_len=16k
Hmmm, maybe it's the 'other' Apps that's having gaussian problems. Besides the above task, I have two others that were decided in favor of CUDA.
The Above with SETI@home v8 v8.05: Best gaussian: peak=4.179937, mean=0.5265682, ChiSq=1.13764, time=89.76, d_freq=1418819127.86, score=1.123425, null_hyp=2.163565, chirp=48.412, fft_len=16k
After some thought I remember looking at this some weeks ago. The CUDA Special App matched the CPU on two different Platforms for Best Gaussian. Now that a Widows CPU also matches, that makes it Three platforms.

It shouldn't be that difficult, just look at some Hosts at Beta running CPU only, here's one;
https://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=9810628
I suspect there will be others.
ID: 1874034 · Report as offensive
Previous · 1 . . . 30 · 31 · 32 · 33 · 34 · 35 · 36 . . . 83 · Next

Message boards : Number crunching : Linux CUDA 'Special' App finally available, featuring Low CPU use


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.