Upgraded to a GTX 1060

Message boards : Number crunching : Upgraded to a GTX 1060
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1806587 - Posted: 2 Aug 2016, 8:11:18 UTC

There's a huge thread round here somewhere where we explored (and largely fixed) the 'abandoned tasks' issue. I've got to go out for a while, but I'll look for it when I get back. There are source links, even line numbers, in there which relate to HostID generation.
ID: 1806587 · Report as offensive
Profile Stubbles
Volunteer tester
Avatar

Send message
Joined: 29 Nov 99
Posts: 358
Credit: 5,909,255
RAC: 0
Canada
Message 1806601 - Posted: 2 Aug 2016, 9:56:18 UTC - in response to Message 1806483.  

If anybody has any ideas I'm all ears. I think an "application reset" should be added to BOINC so when a rig is upgraded it can determine the best application again.
Cheers
Mark

Since no one else has jumped in, I'll share my limited perspective.

The selection of the best application by the server will greatly depend on the task-types (Guppi, nonVLAR, etc) that are processed by each GPU application.
Any GPU app that processes many Guppis will be at a huge disadvantage.

And if I remember correctly (what others have stated a few times during the last 3 months), each GPU app has to process 11 tasks before the server pics the "best".

For the process to be ~fair~, you would need to only allow it to process only 1 task-type by aborting the other task-types.
(There might be another without aborting tasks, but it eludes me right now)
ID: 1806601 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1806642 - Posted: 2 Aug 2016, 14:06:53 UTC

OK, I've found that old thread, and the line numbers it contains.

message 1695726

If you want to find out when a new HostID is generated (and from what), start reading at

https://github.com/BOINC/boinc/blob/master/sched/handle_request.cpp#L237
ID: 1806642 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1806653 - Posted: 2 Aug 2016, 14:53:26 UTC - in response to Message 1806601.  

The selection of the best application by the server will greatly depend on the task-types (Guppi, nonVLAR, etc) that are processed by each GPU application.

Good point. On my laptop gupppi WUs take twice as long as Arecibo ones. Sometimes I get blocks of one type and then the other.

Anyway, I have a new system to implement at work on Thursday so I need to concentrate on that for the next few days. I hate it when work gets in the way of my hobby ;-)

Cheers
Mark
ID: 1806653 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1806654 - Posted: 2 Aug 2016, 14:55:35 UTC - in response to Message 1806642.  

Thanks Richard,

Nice find. I will take a look in a couple of days.

Cheers
Mark
ID: 1806654 · Report as offensive
Profile Stubbles
Volunteer tester
Avatar

Send message
Joined: 29 Nov 99
Posts: 358
Credit: 5,909,255
RAC: 0
Canada
Message 1806693 - Posted: 2 Aug 2016, 23:44:08 UTC - in response to Message 1806653.  

The selection of the best application by the server will greatly depend on the task-types (Guppi, nonVLAR, etc) that are processed by each GPU application.

Good point. On my laptop gupppi WUs take twice as long as Arecibo ones. Sometimes I get blocks of one type and then the other.

Mr Kevvy confirmed that his GUPPIrescheduler works on stock. https://setiathome.berkeley.edu/forum_thread.php?id=79954&postid=1806688
Yes, and for most people it should work with it.

But my guess is that it would likely interfere with the server's assessment of the best app for your GPU since it ignores feedback from the client as to tasks crunched on a diff device type.

Good luck with the work work! ;-)
ID: 1806693 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1806944 - Posted: 4 Aug 2016, 3:21:18 UTC

I will try and make a long story as short as I can...

After searching the BOINC code I was not able to get a new HostID for my GTX 1060 rig. Ok, I’m big enough to admit temporary defeat! For now ;-)

All I want to do is compare my M3000M and 1060 using the same application. So, on to plan B:

I took a deep breath and installed Lunatics with SoG_r3472. I wasn’t sure what I was doing but in the end I learned a lot about how BOINC works. It ran very nicely and cut my run times in half verses stock CUDA50! Very impressive.

I was also going to install SoG_r3472 via Lunatics on my laptop (NVidia M3000M) but then I found SoG_r3500. I installed it by hand (no Lunatics) on my laptop and it worked perfectly the first time.

I then let the cache on the 1060 rig clear and I installed SoG_r3500 just like I did on my laptop. I'm getting two errors.

App version needs OpenCL but GPU doesn't support it

and then 15 minutes later (15 min? That's kind of weird)

Message from server: Your app_info.xml file doesn't have a usable version of SETI@home v8.

Has anybody seen this before? I did reboot and reinstall BOINC that that didn’t help.

8/3/2016 7:27:14 PM |  | cc_config.xml not found - using defaults
8/3/2016 7:27:14 PM |  | Starting BOINC client version 7.6.22 for windows_x86_64
8/3/2016 7:27:14 PM |  | log flags: file_xfer, sched_ops, task
8/3/2016 7:27:14 PM |  | Libraries: libcurl/7.45.0 OpenSSL/1.0.2d zlib/1.2.8
8/3/2016 7:27:14 PM |  | Data directory: C:\ProgramData\BOINC
8/3/2016 7:27:14 PM |  | Running under account Mark
8/3/2016 7:27:15 PM |  | CUDA: NVIDIA GPU 0: GeForce GTX 1060 6GB (driver version 368.81, CUDA version 8.0, compute capability 6.1, 4096MB, 3566MB available, 4374 GFLOPS peak)
8/3/2016 7:27:15 PM | SETI@home | Found app_info.xml; using anonymous platform
8/3/2016 7:27:15 PM |  | App version needs OpenCL but GPU doesn't support it
8/3/2016 7:27:15 PM |  | App version needs OpenCL but GPU doesn't support it
8/3/2016 7:27:15 PM |  | Host name: SETI-01
8/3/2016 7:27:15 PM |  | Processor: 2 GenuineIntel Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz [Family 6 Model 15 Stepping 6]
8/3/2016 7:27:15 PM |  | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 syscall nx lm vmx tm2 pbe
8/3/2016 7:27:15 PM |  | OS: Microsoft Windows 10: Professional x64 Edition, (10.00.10586.00)
8/3/2016 7:27:15 PM |  | Memory: 2.00 GB physical, 3.81 GB virtual
8/3/2016 7:27:15 PM |  | Disk: 92.67 GB total, 74.74 GB free
8/3/2016 7:27:15 PM |  | Local time is UTC -7 hours
8/3/2016 7:27:15 PM | SETI@home | Found app_config.xml
8/3/2016 7:27:15 PM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 7992114; resource share 100
8/3/2016 7:27:20 PM | SETI@home | General prefs: from SETI@home (last modified 03-Aug-2016 13:19:41)
8/3/2016 7:27:20 PM | SETI@home | Computer location: home
8/3/2016 7:27:20 PM | SETI@home | General prefs: no separate prefs for home; using your defaults
8/3/2016 7:27:20 PM |  | Preferences:
8/3/2016 7:27:20 PM |  | max memory usage when active: 1022.97MB
8/3/2016 7:27:20 PM |  | max memory usage when idle: 1841.34MB
8/3/2016 7:27:20 PM |  | max disk usage: 1.00GB
8/3/2016 7:27:20 PM |  | (to change preferences, visit a project web site or select Preferences in the Manager)
8/3/2016 7:27:20 PM | SETI@home | Sending scheduler request: To fetch work.
8/3/2016 7:27:20 PM | SETI@home | Requesting new tasks for NVIDIA GPU
8/3/2016 7:27:21 PM | SETI@home | Scheduler request completed: got 0 new tasks
8/3/2016 7:27:21 PM | SETI@home | Not sending work - last request too recent: 32 sec
8/3/2016 7:42:27 PM | SETI@home | Sending scheduler request: To fetch work.
8/3/2016 7:42:27 PM | SETI@home | Requesting new tasks for NVIDIA GPU
8/3/2016 7:42:33 PM | SETI@home | Scheduler request completed: got 0 new tasks
8/3/2016 7:42:33 PM | SETI@home | No tasks sent
8/3/2016 7:42:33 PM | SETI@home | No tasks are available for SETI@home v7
8/3/2016 7:42:33 PM | SETI@home | No tasks are available for SETI@home v8
8/3/2016 7:42:33 PM | SETI@home | Message from server: Your app_info.xml file doesn't have a usable version of SETI@home v8.
8/3/2016 7:42:33 PM | SETI@home | Tasks for CPU are available, but your preferences are set to not accept them
8/3/2016 7:42:33 PM | SETI@home | Tasks for Intel GPU are available, but your preferences are set to not accept them
8/3/2016 7:42:33 PM | SETI@home | This computer has finished a daily quota of 1 tasks

ID: 1806944 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1806957 - Posted: 4 Aug 2016, 4:58:13 UTC

It is correct, you do not have OpenCL available for that computer.

You should probably reinstall the drivers as a first attack.

ID: 1806957 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1806959 - Posted: 4 Aug 2016, 5:21:36 UTC

It looks like you're using the M$ supplied driver instead of 1 from Nvidia itself and that's always a no no to do. ;-)

Cheers.
ID: 1806959 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1807095 - Posted: 4 Aug 2016, 18:42:08 UTC

Thanks for the help...

I installed the NVIDIA drivers when I first got the 1060.
I reinstalled using the same drivers but it said "No supported hardware was found" OK... very weird. I went the NVIDIA web site and got a slightly newer driver. OpenCL works now. I tested it using a simple app from the NVIDIA SDK.

Now when I run BOINC I still get: Your app_info.xml file doesn't have a usable version of SETI@home v8.

I think I did something rather stupid... When I installed SoG_r3500 I forgot to let my WU cache run dry. Could this be the reason I'm seeing this error?

Cheers
Mark
ID: 1807095 · Report as offensive
AMDave
Volunteer tester

Send message
Joined: 9 Mar 01
Posts: 234
Credit: 11,671,730
RAC: 0
United States
Message 1807097 - Posted: 4 Aug 2016, 18:51:27 UTC - in response to Message 1807095.  

Now when I run BOINC I still get: Your app_info.xml file doesn't have a usable version of SETI@home v8.

I think I did something rather stupid... When I installed SoG_r3500 I forgot to let my WU cache run dry. Could this be the reason I'm seeing this error?

Mark
Check every tag in your app_info.xml file.  I had that same error msg earlier this week, and the cause was a single missing "<".
ID: 1807097 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1807103 - Posted: 4 Aug 2016, 19:08:05 UTC - in response to Message 1807097.  

I double checked all the tags. I also copied app_info.xml from my laptop where it's running fine.

I searched the BOINC source code and found the function that returns the error to the client. Unfortunately this code runs on the server so I can't see the debug messages. I think!

// scan through client's anonymous apps and pick the best one
//
CLIENT_APP_VERSION* get_app_version_anonymous(
    APP& app, bool need_64b, bool reliable_only
) {
    unsigned int i;
    CLIENT_APP_VERSION* best = NULL;
    bool found = false;
    char message[256];

    if (config.debug_version_select) {
        log_messages.printf(MSG_NORMAL,
            "[version] get_app_version_anonymous: app %s%s\n",
            app.name, reliable_only?" (reliable only)":""
        );
    }
    for (i=0; i<g_request->client_app_versions.size(); i++) {
        CLIENT_APP_VERSION& cav = g_request->client_app_versions[i];
        if (!cav.app) continue;
        if (cav.app->id != app.id) {
            continue;
        }
        if (need_64b && !is_64b_platform(cav.platform)) {
            continue;
        }
        int gavid = host_usage_to_gavid(cav.host_usage, app);
        if (reliable_only && !app_version_is_reliable(gavid)) {
            if (config.debug_version_select) {
                log_messages.printf(MSG_NORMAL,
                    "[version] %d %s not reliable\n",
                    cav.version_num, cav.plan_class
                );
            }
            continue;
        }
        if (daily_quota_exceeded(gavid, cav.host_usage)) {
            if (config.debug_version_select) {
                log_messages.printf(MSG_NORMAL,
                    "[version] %d %s daily quota exceeded\n",
                    cav.version_num, cav.plan_class
                );
            }
            continue;
        }
        if (cav.version_num < app.min_version) {
            if (config.debug_version_select) {
                log_messages.printf(MSG_NORMAL,
                    "[version] %d %s version < min version\n",
                    cav.version_num, cav.plan_class
                );
            }
            continue;
        }
        found = true;
        if (!need_this_resource(cav.host_usage, NULL, &cav)) {
            if (config.debug_version_select) {
                log_messages.printf(MSG_NORMAL,
                    "[version] %d %s don't need resource\n",
                    cav.version_num, cav.plan_class
                );
            }
            continue;
        }
        if (best) {
            if (cav.host_usage.projected_flops > best->host_usage.projected_flops) {
                best = &cav;
            }
        } else {
            best = &cav;
        }
    }
    if (!best) {
        if (config.debug_version_select) {
            log_messages.printf(MSG_NORMAL,
                "[version] Didn't find anonymous platform app for %s\n",
                app.name
            );
        }
    }
    if (!found) {
        sprintf(message,
            "%s %s.",
            _("Your app_info.xml file doesn't have a usable version of"),
            app.user_friendly_name
        );
        add_no_work_message(message);
    }
    return best;
}
[/code]
ID: 1807103 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1807108 - Posted: 4 Aug 2016, 19:48:26 UTC

All of a sudden the error went away and I got 14 new work units running SoG_r3500. I didn't change anything. I think the cause was my stupidity aborting a bunch of work units.

Anyway, I can finally compare apples to apples.

Cheers
Mark
ID: 1807108 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1807353 - Posted: 5 Aug 2016, 20:41:09 UTC

From 20,000 feet I was curious to see how my new GTX 1060 would perform compared to my Quadro M3000M.
Note: I did not use a huge sample size. I was only interested in ballpark numbers for now.

I ran MB8_win_x86_SSE3_OpenCL_NV_SoG_r3500 on my NVidia 1060 and M3000M rigs for the past 24 hours. Both units were only crunching one WU at a time.

The 1060 is paired with a 10 year old CPU. I’m not sure how much of an increase I would see if I stuffed it into a newer PC.

Average Run Times:
                      M3000M       1060
                     -------     ------
        Arecibo WU:    9 min      6 min
Green Bank / guppi:   21 min     12 min 

Just for reference: below are the run times running the Cuda50 stock app on the 1060. (Guppies where horrible at around 35 minutes each. I didn’t bother running them in parallel)
Parallel    Run       Average    
Work Units  Time      Time
----------  -------   --------
1           14.0 min  14.0 min
2           18.5 min   9.2 min
3           24.0 min   8.0 min
4           36.0 min   9.0 min

Observations:
- Even a 10 year old CPU paired with a new GPU can produce pretty darn good results!
- NV_SoG_r3500 seems to keep the GPU around 95% busy running one WU.
- The GTX rig averages about 150 watts total (I have a meter hooked up to it). With no CPU or GPU load the rig hovers around 70 watts.
- I was a bit surprised to see OpenCL beat Cuda50. I would have expected Cuda to outperform OpenCL since it’s NVidia specific and closer to the silicon. Maybe Cuda50 needs some updating?

Here are print screens of the two systems running
M3000M
GTX 1060

Cheers
Mark
ID: 1807353 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1807365 - Posted: 5 Aug 2016, 21:15:50 UTC - in response to Message 1807353.  
Last modified: 5 Aug 2016, 21:31:39 UTC

...
- I was a bit surprised to see OpenCL beat Cuda50. I would have expected Cuda to outperform OpenCL since it’s NVidia specific and closer to the silicon. Maybe Cuda50 needs some updating?


Highly likely with a card so new (on top of other updates being needed). With Cuda versions the version released closest to the architecture tends to be the best choice (due to new instructions etc). Cuda 8.0 (which the 1060 and its drivers would be) is still in prerelease, while the OpenCL compiler is in the drivers already.

Several major contributions toward the next Cuda update have been made (by Petri33), though there are some major things to overcome before any widespread testing/release could happen there, so for the time being there are some rolling there own with alpha code, and the rest relying on the recently greatly improved OpenCL apps. [My own direction switched from maintaining the stock Cuda apps to building new infrastructure some time back, which became more pressing with the switch to v8/Guppis and the slough of new GPUs. That's likely to take some time]
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1807365 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 1807383 - Posted: 5 Aug 2016, 22:04:34 UTC - in response to Message 1807353.  

The 1060 is paired with a 10 year old CPU. I’m not sure how much of an increase I would see if I stuffed it into a newer PC.

Both the CUDA50 & SoG applications can give significantly better throughput when they are set to use 1 CPU core per WU being crunched & then given that CPU core. And with the latest hardware 3-4WUs appears to give the most WUs processed per hour.
Grant
Darwin NT
ID: 1807383 · Report as offensive
Mark Loukko

Send message
Joined: 7 Jun 99
Posts: 52
Credit: 40,406,567
RAC: 108
Canada
Message 1807394 - Posted: 5 Aug 2016, 22:27:25 UTC - in response to Message 1807383.  


Both the CUDA50 & SoG applications can give significantly better throughput when they are set to use 1 CPU core per WU being crunched & then given that CPU core.

Hi Grant,

Interesting... I noticed my M3000M was setup with the following but my 1060 was not.

<app_config>
	<app>	
		<name>setiathome_v8</name>
		<gpu_versions>
			<gpu_usage>1</gpu_usage>
			<cpu_usage>1</cpu_usage>
		</gpu_versions>
	</app>
</app_config>


Is this all I need to do? Or are you also referring about cpu affinity? If so, how do I set that up in BOINC?

Cheers
Mark
ID: 1807394 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 1807399 - Posted: 5 Aug 2016, 22:43:09 UTC - in response to Message 1807394.  

<gpu_usage>1</gpu_usage>
<cpu_usage>1</cpu_usage>

The gpu_usage determines how many WUs run per GPU.
1 = 1
0.5= 2
0.33 = 3
0.25 = 4

And the cpu_usage determines how many CPU cores per WU to reserve- it doesn't mean the GPU application will actually use them, it just stops them from being used for CPU crunching.
1 = 1 CPU core per WU
0.5 = 1 CPU core per 2 WUs
0.25 = 1 CPU core per 4 WUs etc.


Is this all I need to do?

Afraid not.
All that does is reserve one CPU core per WU, the applications need to be set to actually make use of the CPU core.

For CUDA you need to use the Anonymous platform as the settings have to go in the app_info.xml file. (if anyone knows how to get -poll to work without using it i'd be very interested).

For SoG, Raistmer has developed the application considerably so it no longer uses a core by default- however using the command line settings you can greatly boost it's performance, and giving it 1 CPU core per WU can make a big difference.
Mike is the one who can best help with suggested command line settings to try.


Or are you also referring about cpu affinity?

Nope, that's a whole different issue.
Grant
Darwin NT
ID: 1807399 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19044
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1807408 - Posted: 5 Aug 2016, 23:46:50 UTC - in response to Message 1807383.  

The 1060 is paired with a 10 year old CPU. I’m not sure how much of an increase I would see if I stuffed it into a newer PC.

Both the CUDA50 & SoG applications can give significantly better throughput when they are set to use 1 CPU core per WU being crunched & then given that CPU core. And with the latest hardware 3-4WUs appears to give the most WUs processed per hour.

What happens if you have less CPU cores than the number of tasks you would like to run on the GPU concurrently?

I ask because this cpu/mobo (it is a desktop computer) still work and I could fit my current GPU in there when I upgrade. Don't ask me what to I haven't decided yet or when, other factors might decide that decision.
ID: 1807408 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1807409 - Posted: 5 Aug 2016, 23:49:12 UTC - in response to Message 1807408.  

Depending on the work units. You could starve the gpu. Is the gpu is waiting for a free CPU cycle to run
ID: 1807409 · Report as offensive
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Upgraded to a GTX 1060


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.