Tutorials and/or samples of: cc_config.xml and app_info.xml

Questions and Answers : GPU applications : Tutorials and/or samples of: cc_config.xml and app_info.xml
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1536275 - Posted: 5 Jul 2014, 2:38:18 UTC

All I have found so far are reference articles. http://boinc.berkeley.edu/wiki/client_configuration covers both.

It lacks well filled out examples. And it doesn't answer the question of:

app_info.xml -> in the Seti at home project directory

<app_config>
<app>
<name>astropulse_v6</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.5</cpu_usage>
</gpu_versions>
</app>
<app>
<name>setiathome_v7</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.5</cpu_usage>
</gpu_versions>
</app>
</app_config>

How does gpu_usage interact with cpu_usage? Why does it cause (usually) 2 tasks on each of two gpus? if you also have:
cc_config.xml -> in the main bionnic directory.

<cc_config>
<options>
<use_all_gpus>1</use_all_gpus>
</options>
</cc_config>

Thanks,
Tom
A proud member of the OFA (Old Farts Association).
ID: 1536275 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1536285 - Posted: 5 Jul 2014, 3:09:38 UTC - in response to Message 1536275.  

<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.5</cpu_usage>

<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.5</cpu_usage>

The above tells program what ratio of GPU to CPU. GPU 0.5 tells the program to use 1/2 of a gpu for each work unit. So 1/2 of a GPU then means 2 work units per GPU. If the number was 0.33 then it would use 1/3 of the GPU for each work unit, so 0.33 means 3 work units per GPU.

Now the CPU usage tells the program how much of each core to use. Again 0.5 is 1/2 so use each core supports 2 work units. If the number was 0.33 then it's 3 work units per Core. It would be easy to say that these portions of the cores are reserved but in truth they are "floating" meaning that they rotate between all the cores. Gets a little confusing about that.



<cc_config>
<options>
<use_all_gpus>1</use_all_gpus>
</options>
</cc_config>



What this is telling the program is to use all available graphic cards. I don't know the specifics of it but I know if you don't use it and have different GPUs of non-similiar speed or class, Bonic will only use the higher rated class.

Hope this helps..

Zalster
ID: 1536285 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1536329 - Posted: 5 Jul 2014, 6:04:32 UTC - in response to Message 1536275.  
Last modified: 5 Jul 2014, 6:43:43 UTC

It lacks well filled out examples. And it doesn't answer the question of:

app_info.xml (you mean app_config.xml - Jord) -> in the Seti at home project directory

The Application configuration section starts with:
This mechanism allows you to specify scheduling parameters for specific applications or app versions. It is available with 7.0.40+ client versions.

To do this, create a file app_config.xml in the project's directory, e.g. when the project is SETI@home: {BOINC Data directory}\projects\setiathome.berkeley.edu\


How does gpu_usage interact with cpu_usage? Why does it cause (usually) 2 tasks on each of two gpus?

One has to keep in mind that the main developer (and boss, owner etc.) of BOINC (not Bionic, certainly not Bionnic unless you know what that acronym stands for?) goes through the User Wiki and weeds out what he finds superfluous and useless information on a regular basis. So even when I today put in the information into that Wiki, I cannot guarantee that it will stay there. It may be gone by day-break tomorrow.

app_info.xml is a whole different kettle of fish. It is used by anonymous platform applications - which are non-default by project delivered, non-downloaded automatically by BOINC applications - and is described in http://boinc.berkeley.edu/wiki/Anonymous_platform. Please do not confuse the one with the other. Although both, since they work on a per-project basis, run from the project directory.
ID: 1536329 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1536418 - Posted: 5 Jul 2014, 13:47:56 UTC - in response to Message 1536285.  

After I got my other hickups straightened out I started trying smaller and smaller cpu numbers and smaller and smaller gpu numbers. Since I am running two gpus I thought I would see "how many" tasks I could run in parallel so to speak. I keep getting larger and larger Cuda# so I don't think my actual time to work unit is directly comparable.

With gpu at 0.25 and cpu at: 0.0157 or so, I was running 7 cpu tasks on a non-seti project while running 8 Cuda tasks on the 2 gpu's I have.

For a while I even got 8 cpu tasks until the "memory used" line on the task manager got higher. Then it moved down to 6 cpu tasks.

The fans on the Xeon got louder and louder. :)

And the video got pretty flaky. Since I have a couple of un-used PCI slots on that box I have ordered a PCI video card to see if I can use it for the video while running 8 gpu tasks.

Thank you for the discussion and the all the other help you have offered on various threads.

Tom Miller
A proud member of the OFA (Old Farts Association).
ID: 1536418 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1536419 - Posted: 5 Jul 2014, 13:53:01 UTC - in response to Message 1536329.  

For "gpu type" I think you are referring to cc_config.xml and in app_config.xml is the "coproc"?

Tom
A proud member of the OFA (Old Farts Association).
ID: 1536419 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1536427 - Posted: 5 Jul 2014, 14:32:15 UTC - in response to Message 1536419.  
Last modified: 5 Jul 2014, 14:46:26 UTC

For "gpu type" I think you are referring to cc_config.xml and in app_config.xml is the "coproc"?

Perhaps that the question setting makes sense for you, but it doesn't for me. So could you please elaborate? As far as I know, I didn't refer to gpu_type anywhere.

As I have written elsewhere in answer to you as well, since you continue to say app_info.xml where you mean app_config.xml:

The file ..is the application configuration file, its name shortened to app_config.xml
With this file you can configure how many tasks BOINC should schedule to run on an appointed application and GPU. This file can be used with the default project provided science applications, and the third party science applications used through the anonymous platform module.

-- -- -- -- --

The application information file (app_info.xml) is used by anonymous platform applications, and is used so the user can dictate to BOINC what application it should run instead of the default science application provided by the project. This way you can use apps that are optimized specifically for your CPU/GPU, or that you compiled yourself.

-- -- -- -- --

The two aren't synonyms of each other. You can use an app_info.xml file at all times, with or without an additional app_config.xml file. Many of the things you can do in the app_config.xml file, you can do in the app_info.xml file. But for fine tuning the scheduling process, it's best to use the app_config.xml file.
ID: 1536427 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1536838 - Posted: 6 Jul 2014, 16:27:30 UTC - in response to Message 1536427.  

Your right. I am getting the app_info.xml file and the app_config.xml file confused. I apologize. I have been tinkering with the cc_config.xml file and the app_config.xml files. Sorry!

I do now understand how to set the total number of tasks (#gpus X (number of tasks per card based on (1/x tasks = decimal number to set gpu_usage).
I am currently running the following on my Quadro only.
----------------------------
<app_config>
<app>
<name>astropulse_v6</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.04</cpu_usage>
</gpu_versions>
</app>
<app>
<name>setiathome_v7</name>
<gpu_versions>
<gpu_usage>0.125</gpu_usage>
<cpu_usage>0.04</cpu_usage>
</gpu_versions>
</app>
</app_config>
----------------------------------------
This now munching along with all 8 cpu's (4 core cpu running hyper-thread) running basic project tasks and the Quadro is running 8 cuda-type tasks.

I have been poking around in the .xml files and now have another question related to tuning/maximizing throughput of my gpu's.

I have two non-identical video cards. Both will process cuda-type files.

------------------Extract of latest Event log--------------------------
7/6/2014 10:40:44 AM | | CUDA: NVIDIA GPU 0: Quadro FX 1800 (driver version 337.88, CUDA version 6.0, compute capability 1.1, 768MB, 720MB available, 264 GFLOPS peak)
7/6/2014 10:40:44 AM | | CUDA: NVIDIA GPU 1 (not used): GeForce 8500 GT (driver version 337.88, CUDA version 6.0, compute capability 1.1, 512MB, 471MB available, 44 GFLOPS peak)
7/6/2014 10:40:44 AM | | OpenCL: NVIDIA GPU 0: Quadro FX 1800 (driver version 337.88, device version OpenCL 1.0 CUDA, 768MB, 720MB available, 264 GFLOPS peak)
7/6/2014 10:40:44 AM | | OpenCL: NVIDIA GPU 1 (ignored by config): GeForce 8500 GT (driver version 337.88, device version OpenCL 1.0 CUDA, 512MB, 471MB available, 44 GFLOPS peak)
-------------------------end of Event Log--------------------------

As you can see the GeForce card is pretty dinky compared to the Quadro. But I would like to take full advantage of both....

While poking around in the .XML files I found a "computation maximum" listed for each one. The Quardro says 8 processes. The Geforce says 2 processes.

Is there a way to tell each card to run a different X number of tasks? I see the logic and understand how to tell both cards to run 2 tasks, 4 tasks etc. I do have what appear to be the [bus1slot0] numbers for each one.

The related question is, is there any "standard" not to difficult to use "benchmark" that will offer advice on what number of tasks will maximize my through put? This could be on a card-by-card basis or both cards in combination.

Thanks for your thoughtful and low key error corrections!

Tom
A proud member of the OFA (Old Farts Association).
ID: 1536838 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22190
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1536913 - Posted: 6 Jul 2014, 19:23:22 UTC

Your Quadro is pretty ancient technology, while it may be able to run 8 tasks it is very unlikely that it is going to do them with any degree of alacrity - drop it back to 2 and you will find it will start to perform much better - see my earlier comments on tuning for maximum throughput.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1536913 · Report as offensive
Profile BilBg
Volunteer tester
Avatar

Send message
Joined: 27 May 07
Posts: 3720
Credit: 9,385,827
RAC: 0
Bulgaria
Message 1537334 - Posted: 7 Jul 2014, 16:27:18 UTC - in response to Message 1536838.  

The related question is, is there any "standard" not to difficult to use "benchmark" that will offer advice on what number of tasks will maximize my through put?

Yes, there is
Already posted in answer to your post (10 hours before you asked this):
http://setiathome.berkeley.edu/forum_thread.php?id=75108&postid=1536503#1536503
 


- ALF - "Find out what you don't do well ..... then don't do it!" :)
 
ID: 1537334 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1541769 - Posted: 14 Jul 2014, 22:46:54 UTC - in response to Message 1537334.  

I just noticed the following behaviors.

1) On an Xeon running 4 cores plus hyper thread when I set the

cpu to gpu ratio to 1 for the gpu and 1 for the cpu.... and restarted seti, it came up with 8 cores still running basic seti and the other one running apparently as a "virtual" cpu.

So I tried replicating that on my smaller machines which run two real cpus or 1 real cpu. All of them did what I remembered. They all paused one cpu and ran the other 2 (one regular seti wu plus 1 gpu) or in the single cpu's case just ran the gpu.

It gets better. Prior to this I thought I had experienced that if the cpu was set above .24 it tied up a whole cpu. Then I tried .5 and my 2 regular cpus kept chugging along. So I raised the bid to .75 and they still are just chugging along.

Can't really tell if it makes a difference on the processing speed of the gpu (which are all set to 1.0) but it makes me feel like I have a spare 3/4 of a cpu laying around here someplace. Yes, the trick even works on my 1 cpu box.

Thanks,
Tom
A proud member of the OFA (Old Farts Association).
ID: 1541769 · Report as offensive
Darrell Wilcox Project Donor
Volunteer tester

Send message
Joined: 11 Nov 99
Posts: 303
Credit: 180,954,940
RAC: 118
Vietnam
Message 1542831 - Posted: 17 Jul 2014, 0:16:46 UTC - in response to Message 1541769.  
Last modified: 17 Jul 2014, 0:22:24 UTC

The app_config.xml is telling the BOINC scheduler how much of the resources to EXPECT the WUs to use so that it doesn't over commit them. It does not "know" what is actually being used and it does NOT actually "reserve" them from being used by tasks. It is not a "ratio" of cpu/gpu. It tells the BOINC scheduler how much of "a" CPU and "a" GPU each WU of a type is expected to use.

Your O/S schedules the actual work getting done. Over commitment means more overhead and less throughput after 95% utilization (some think higher, some lower).

Some people dedicate the entire machine, others only a portion. You, as the tweaker, must "tweak and try" to find what is "best" for your needs/wants. Just watch your RAC and see what works for you and your configuration.

Good luck with your quest for your best.

[EDIT: added words about 'ratio']
ID: 1542831 · Report as offensive

Questions and Answers : GPU applications : Tutorials and/or samples of: cc_config.xml and app_info.xml


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.