From app's readme:

Advanced level options (some app code reading and understanding of algorithms used is recommended before use, not fool-proof even in same degree as
options above):
-tune N Mx My Mz : to make app more tunable this param allows user to fine tune kernel launch sizes of most important kernels.
N - kernel ID (see below)
Mxyz - workgroup size of kernel. For 1D workgroups Mx will be size of first dimension and My=Mz=1 should be 2 other ones.
N should be one of values from this list:
FFA_FETCH_WG=1,
FFA_COMPARE_WG=2
For best tuning results its recommended to launch app under profiler to see how particular WG size choice affects particular kernel.
This option mostly for developers and hardcore optimization enthusiasts wanting absolute max from their setups.
No big changes in speed expected but if you see big positive change over default please report.
Usage example: -tune 2 32 1 1 (set workgroup size of 32 for 1D FFA comparison kernel).

The tune param will define the kernel size of the GPU into chunks.
Your GPU has work group size of 256 for example.

Mike, please explain HOW you know his GPU has a work group size of 256. How can someone determine their own GPU parameters?

The tune param will define the kernel size of the GPU into chunks.
Your GPU has work group size of 256 for example.

Mike, please explain HOW you know his GPU has a work group size of 256. How can someone determine their own GPU parameters?

Look into stderr output of GPU task.
Our apps list WG sizes for all GPUs in system and WG for used GPU too.

EDIT: example from one of your own hosts:

Number of OpenCL platforms: 2


OpenCL Platform Name: Intel(R) OpenCL
Number of devices: 1
Max compute units: 20
Max work group size: 512
Max clock frequency: 400Mhz
Max memory allocation: 460587008
Cache type: Read/Write
Cache line size: 64
Cache size: 2097152
Global memory size: 1842348032
Constant buffer size: 65536
Max number of constant args: 8
Local memory type: Scratchpad
Local memory size: 65536
Queue properties:
Out-of-Order: No
Name: Intel(R) HD Graphics 4600
Vendor: Intel(R) Corporation
Driver version: 9.18.10.3089
Version: OpenCL 1.2
Extensions: cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_gl_sharing cl_khr_d3d10_sharing cl_intel_dx9_media_sharing cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_event cl_khr_gl_msaa_sharing cl_khr_depth_images cl_khr_gl_depth_images cl_khr_dx9_media_sharing cl_khr_d3d11_sharing cl_khr_image2d_from_buffer


OpenCL Platform Name: NVIDIA CUDA
Number of devices: 1
Max compute units: 4
Max work group size: 1024
Max clock frequency: 719Mhz
Max memory allocation: 536870912
Cache type: Read/Write
Cache line size: 128
Cache size: 65536
Global memory size: 2147483648
Constant buffer size: 65536
Max number of constant args: 9
Local memory type: Scratchpad
Local memory size: 49152
Queue properties:
Out-of-Order: Yes
Name: GeForce GTX 760M
Vendor: NVIDIA Corporation
Driver version: 311.30
Version: OpenCL 1.1 CUDA
Extensions: cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_d3d9_sharing cl_nv_d3d10_sharing cl_khr_d3d10_sharing cl_nv_d3d11_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64

Thanks for the quick reply and the information!

I inserted the tune option in the command line file -use_sleep -unroll 10 -ffa_block 12288 -ffa_block_fetch 6144 -tune 1 64 4 and restarted BOINC and immediately lost all of my 138 AP GPU tasks due to computional error.

[edit] I just noticed this in the CUDA Version thread -tune 1 64 4 1. Did the above errors occur because I was missing the last sub parameter?[/edit] If so, I'll reinsert the option and wait patiently for the next AP feeding frenzy.

I dont think you lost your cache because of the param edit.
Something else must have happened.

---

With each crime and every kindness we birth our future.

I inserted the tune option in the command line file -use_sleep -unroll 10 -ffa_block 12288 -ffa_block_fetch 6144 -tune 1 64 4 and restarted BOINC and immediately lost all of my 138 AP GPU tasks due to computional error.

[edit] I just noticed this in the CUDA Version thread -tune 1 64 4 1. Did the above errors occur because I was missing the last sub parameter?[/edit] If so, I'll reinsert the option and wait patiently for the next AP feeding frenzy.

I dont think you lost your cache because of the param edit.
Something else must have happened.

stderr_txt makes it look very much as if he did.

<message>
One or more arguments are invalid
(0x80000003) - exit code -2147483645 (0x80000003)
</message>
<stderr_txt>
Running on device number: 0
Sleep() & wait for event loops will be used in some places
DATA_CHUNK_UNROLL set to:10
FFA thread block override value:12288
FFA thread fetchblock override value:6144
ERROR: Invalid parameter detected in function (null). File: (null) Line: 0
ERROR: Expression: (null)


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x76663226

Processed four parameters correctly without complaint, and barfed on the fifth.

-tune N Mx My Mz
FFA_FETCH_WG=1,
FFA_COMPARE_WG=2

It is not 100% clear to me but is work group size calculated by Mx * My * Mz or is Mz for another use? It would make sense to me if -tune 1 8 8 8 = WG size 512.
Setting value for N sets the function to use fetch vs compare? This would be more "Try each and see if one is better for your hardware"?

---

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[

Question regarding workgroup size:

From the stdout of my GPU apps the maximum workgroup size of my GT 640 and GT 430 is 1024, but Mike always suggests 256 for my setup, for example 1,128,2,1 1,64,4,1 1,32,8,1 or 1,16,16,1

Would i gain performance when using workgroup size 1024 instead? For example 1,512,2,1 1,256,4,1 1,128,8,1 1,64,16,1 or 1,32,32,1

Regards, Uli

---

Aloha, Uli

I inserted the tune option in the command line file -use_sleep -unroll 10 -ffa_block 12288 -ffa_block_fetch 6144 -tune 1 64 4 and restarted BOINC and immediately lost all of my 138 AP GPU tasks due to computional error.

[edit] I just noticed this in the CUDA Version thread -tune 1 64 4 1. Did the above errors occur because I was missing the last sub parameter?[/edit] If so, I'll reinsert the option and wait patiently for the next AP feeding frenzy.

I dont think you lost your cache because of the param edit.
Something else must have happened.

stderr_txt makes it look very much as if he did.

<message>
One or more arguments are invalid
(0x80000003) - exit code -2147483645 (0x80000003)
</message>
<stderr_txt>
Running on device number: 0
Sleep() & wait for event loops will be used in some places
DATA_CHUNK_UNROLL set to:10
FFA thread block override value:12288
FFA thread fetchblock override value:6144
ERROR: Invalid parameter detected in function (null). File: (null) Line: 0
ERROR: Expression: (null)


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x76663226

Processed four parameters correctly without complaint, and barfed on the fifth.

Thats weird indeed.

---

With each crime and every kindness we birth our future.

Question regarding workgroup size:

From the stdout of my GPU apps the maximum workgroup size of my GT 640 and GT 430 is 1024, but Mike always suggests 256 for my setup, for example 1,128,2,1 1,64,4,1 1,32,8,1 or 1,16,16,1

Would i gain performance when using workgroup size 1024 instead? For example 1,512,2,1 1,256,4,1 1,128,8,1 1,64,16,1 or 1,32,32,1

Regards, Uli

I cant test it because my card only has WG 256.

Like the readme says testing required.

I would run some off line benches to make sure.

---

With each crime and every kindness we birth our future.

I want to thank everyone for assisting me in understanding and applying the -tune option to the AP cmdline file. I did correct the argument for the -tune option and I've learned a very important lesson when copy & paste is involved for suggestions - copy & paste, but verify before applying. I do have 5 AP tasks in the queue, but were not affected by the error because of their deadline, so we will wait and see what happens.

Note to all developers - Attempt to ensure that parameters, options, arguments, etc. are described in more laymen terms so that those of us who are not proficient is C++ or any other PC language, who still want to attempt more advanced options, can understand them. Does not have to be a tutorial, but should be enough to comprehend.

For example -

Advanced level options (some app code reading and understanding of algorithms used is recommended before use, not fool-proof even in same degree as
options above):
-tune N Mx My Mz : to make app more tunable this param allows user to fine tune kernel launch sizes of most important kernels.
N - kernel ID (see below)
Mxyz - workgroup size of kernel. For 1D workgroups Mx will be size of first dimension and My=Mz=1 should be 2 other ones.
N should be one of values from this list:
FFA_FETCH_WG=1,
FFA_COMPARE_WG=2
For best tuning results its recommended to launch app under profiler to see how particular WG size choice affects particular kernel.
This option mostly for developers and hardcore optimization enthusiasts wanting absolute max from their setups.
No big changes in speed expected but if you see big positive change over default please report.
Usage example: -tune 2 32 1 1 (set workgroup size of 32 for 1D FFA comparison kernel).

I would probably win a bet that most of us have not heard of the profiler mentioned above, but would understand better if the following were inserted in the text -

The tune param will define the kernel size of the GPU into junks.
Your GPU has work group size of 256 for example.

So tune 1 64 4 means it will load 64 4 times until 256 is reached.

Possible values are 128 2 or 32 8 or 16 16.
Using unequal number results in lower speed because value of 256 can`t be reached in this case.

Without tune params the kernel will be loaded without any structure.

Or even mentioning something from the following would help -

The tune param will define the kernel size of the GPU into chunks.
Your GPU has work group size of 256 for example.

Mike, please explain HOW you know his GPU has a work group size of 256. How can someone determine their own GPU parameters?

Look into stderr output of GPU task.
Our apps list WG sizes for all GPUs in system and WG for used GPU too.

I still don't know what the profiler is or how to apply it.

---

I don't buy computers, I build them!!

Well, a profiler will "profile" an application while running, logging the times spent in different parts of the program, so allowing to identify the parts, in which most of the runtime is spent. These are the spots, where optimization is most effective. Also you can make different profiles for different sets of command line parameters, identifying the set, gaining the most performance boost.

BTW:
I am using now the following parameter set, which seems to be the best for my setup running 2 instances of r2399 per card:

-use_sleep -unroll 4 -ffa_block 2048 -ffa_block_fetch 1024 -tune 1 128 8 1

So the GT430 is maxed out and the GT640 has a little headroom for other tasks.

---

Aloha, Uli

Well, a profiler will "profile" an application while running, logging the times spent in different parts of the program, so allowing to identify the parts, in which most of the runtime is spent. These are the spots, where optimization is most effective. Also you can make different profiles for different sets of command line parameters, identifying the set, gaining the most performance boost.

Thanks Ulrich, and where can I find such an animal? I would like to see what else it can do.

---

I don't buy computers, I build them!!

Thanks Ulrich, and where can I find such an animal? I would like to see what else it can do.

A profiler is normally included into development environments, for example Microsoft Visual Studio. Unfortunately it is not included in the "free" versions, aka "Express Edition", it is only included in the "professional" or "ultimate" editions. Also the usage is not trivial and most effective on so called debug builds. For real use you need developer skills and the source code of the application under test.

A simple form of profiling would be to run the same AP test wu under the same conditions with only changing parameter sets and recording the total time.

Sorry, but i don't know of any profiling application, which runs standalone and has an user interface suitable for laymens. :/

---

Aloha, Uli

-tune N Mx My Mz
FFA_FETCH_WG=1,
FFA_COMPARE_WG=2

It is not 100% clear to me but is work group size calculated by Mx * My * Mz or is Mz for another use? It would make sense to me if -tune 1 8 8 8 = WG size 512.
Setting value for N sets the function to use fetch vs compare? This would be more "Try each and see if one is better for your hardware"?

As stated in ReadMe I classified this param as "advanced". That is, better to consult with app code before usage. Yes, 8x8x8 would give total WG size of 512 (not supported by ATi GPUs) but even 8x8x4 has no meaning for 2D kernel. Not all combos are possible for supported kernels. PErhas more kernels will be supported in future, but again, one need to understand where most load goes.. For now it's fetch kernel then FFT ones. Hence there is no much sense to "optimize" smth that takes only fraction of % of total execution time.
Version included in installer has 1D fetch kernel hance Mz should be 1 and My should be 1 for this type of kernel. Prev builds that support this key and APv7 include 2D fetch kernel, there Mz should be 1, MxxMy give workgroup size with that kernel will be called.

I want to thank everyone for assisting me in understanding and applying the -tune option to the AP cmdline file. I did correct the argument for the -tune option and I've learned a very important lesson when copy & paste is involved for suggestions - copy & paste, but verify before applying. I do have 5 AP tasks in the queue, but were not affected by the error because of their deadline, so we will wait and see what happens.

Note to all developers - Attempt to ensure that parameters, options, arguments, etc. are described in more laymen terms so that those of us who are not proficient is C++ or any other PC language, who still want to attempt more advanced options, can understand them. Does not have to be a tutorial, but should be enough to comprehend.

For example -

Advanced level options (some app code reading and understanding of algorithms used is recommended before use, not fool-proof even in same degree as
options above):
-tune N Mx My Mz : to make app more tunable this param allows user to fine tune kernel launch sizes of most important kernels.
N - kernel ID (see below)
Mxyz - workgroup size of kernel. For 1D workgroups Mx will be size of first dimension and My=Mz=1 should be 2 other ones.
N should be one of values from this list:
FFA_FETCH_WG=1,
FFA_COMPARE_WG=2
For best tuning results its recommended to launch app under profiler to see how particular WG size choice affects particular kernel.
This option mostly for developers and hardcore optimization enthusiasts wanting absolute max from their setups.
No big changes in speed expected but if you see big positive change over default please report.
Usage example: -tune 2 32 1 1 (set workgroup size of 32 for 1D FFA comparison kernel).

I would probably win a bet that most of us have not heard of the profiler mentioned above, but would understand better if the following were inserted in the text -

The tune param will define the kernel size of the GPU into junks.
Your GPU has work group size of 256 for example.

So tune 1 64 4 means it will load 64 4 times until 256 is reached.

Possible values are 128 2 or 32 8 or 16 16.
Using unequal number results in lower speed because value of 256 can`t be reached in this case.

Without tune params the kernel will be loaded without any structure.

Or even mentioning something from the following would help -

The tune param will define the kernel size of the GPU into chunks.
Your GPU has work group size of 256 for example.

Mike, please explain HOW you know his GPU has a work group size of 256. How can someone determine their own GPU parameters?

Look into stderr output of GPU task.
Our apps list WG sizes for all GPUs in system and WG for used GPU too.

I still don't know what the profiler is or how to apply it.

Sorry, I would better try to code, I'm not a writer ;)
And to have smth new to learn - what better could be for Homo Sapiens specie? ;) (joke).
Actually, if feel uncomfortable with description and ESPECIALLY if see advanced warning given - better to do as manual describe... or just leave it at default. And report bug if default fails.

General note regarding advanced options: I think better to have some tool for tweak than to have hardwired values and no way to change besides to write own code, right? But with tool comes responsibility for usage that tool. Yes, better fool-proof can and perhaps will be added. And this will cost time for coding new options/optimizations. Make your choice wisely.

Sorry, but i don't know of any profiling application, which runs standalone and has an user interface suitable for laymens. :/

CodeXL for ATi, Nsight for NV.
Still have to find free good profiler for iGPU though.

EDIT: not for laymans perhaps though. But then продвинутые опции comes in play. Perhaps I need to write in own native language instead of english next time, to make it more clear if english "advanced" word not comprehendable :-/

Sorry, but i don't know of any profiling application, which runs standalone and has an user interface suitable for laymens. :/

CodeXL for ATi, Nsight for NV.
Still have to find free good profiler for iGPU though.

EDIT: not for laymans perhaps though. But then продвинутые опции comes in play. Perhaps I need to write in own native language instead of english next time, to make it more clear if english "advanced" word not comprehendable :-/

I think instead of "advanced" it is more "developer" option. Unless you are developer, or reading the code like one, you probably do not understand what the option is actually doing.

---

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[

Sorry, but i don't know of any profiling application, which runs standalone and has an user interface suitable for laymens. :/

CodeXL for ATi, Nsight for NV.
Still have to find free good profiler for iGPU though.

EDIT: not for laymans perhaps though. But then продвинутые опции comes in play. Perhaps I need to write in own native language instead of english next time, to make it more clear if english "advanced" word not comprehendable :-/

I think instead of "advanced" it is more "developer" option. Unless you are developer, or reading the code like one, you probably do not understand what the option is actually doing.

Нет проблем, исправлю на "для разработчиков"

Sorry, but i don't know of any profiling application, which runs standalone and has an user interface suitable for laymens. :/

CodeXL for ATi, Nsight for NV.
Still have to find free good profiler for iGPU though.

EDIT: not for laymans perhaps though. But then продвинутые опции comes in play. Perhaps I need to write in own native language instead of english next time, to make it more clear if english "advanced" word not comprehendable :-/

I think instead of "advanced" it is more "developer" option. Unless you are developer, or reading the code like one, you probably do not understand what the option is actually doing.

Нет проблем, исправлю на "для разработчиков"

Yes, exactly!

---

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[