Best way to free cores when running multiple projects

Message boards : Number crunching : Best way to free cores when running multiple projects

To post messages, you must log in.

AuthorMessage
mikeej42

Send message
Joined: 26 Oct 00
Posts: 109
Credit: 791,863,759
RAC: 0
United States
Message 1378662 - Posted: 8 Jun 2013, 20:15:57 UTC
Last modified: 8 Jun 2013, 20:20:46 UTC

I am running multiple BOINC projects (currently SETI and Eintein). If you are running multiple projects what is the best way to free CPU core(s) to help GPU based computation?
What are the lines that need to be added to app_info.xml or cc_config.xml?


ID: 1378662 · Report as offensive
Mike Davis
Volunteer tester

Send message
Joined: 17 May 99
Posts: 238
Credit: 5,371,812
RAC: 21
Isle of Man
Message 1378667 - Posted: 8 Jun 2013, 20:29:14 UTC - in response to Message 1378662.
Last modified: 8 Jun 2013, 20:29:26 UTC

<app>
<name>setiathome_v7</name>
<gpu_versions>
<gpu_usage>0.33</gpu_usage>
<cpu_usage>.2</cpu_usage>
</gpu_versions>
</app>

The option in bold is what you need to change for reserving cores. Or you can just change the option in your boinc preferences to 75%, 87.5% etc as this is only for cpu apps, gpu apps can use any spare


ID: 1378667 · Report as offensive
mikeej42

Send message
Joined: 26 Oct 00
Posts: 109
Credit: 791,863,759
RAC: 0
United States
Message 1378676 - Posted: 8 Jun 2013, 20:46:49 UTC - in response to Message 1378667.

What is the equivalent line(s) in cc_config.xml to changing boinc preferences to 75% cpu utilization?


ID: 1378676 · Report as offensive
Mike Davis
Volunteer tester

Send message
Joined: 17 May 99
Posts: 238
Credit: 5,371,812
RAC: 21
Isle of Man
Message 1378687 - Posted: 8 Jun 2013, 21:02:50 UTC

just go into your computing preferences in boinc manager. It should be labelled as 'On multiprocessor systems use at most ... % of the processors' or something very similar. Just change that to 75%


ID: 1378687 · Report as offensive
spitfire_mk_2
Avatar

Send message
Joined: 14 Apr 00
Posts: 542
Credit: 19,898,302
RAC: 12,855
United States
Message 1378707 - Posted: 8 Jun 2013, 21:56:32 UTC

BOINC Manager>Tools>Computing preferences>processor usage tab>On multiprocessor systems, use at most xx.xx % of the processors

I have quad core, I set it 75% so that I use 3 out of 4 cores.


ID: 1378707 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1378711 - Posted: 8 Jun 2013, 22:15:29 UTC
Last modified: 8 Jun 2013, 22:15:55 UTC

What is the equivalent line(s) in cc_config.xml to changing boinc preferences to 75% cpu utilization?
There's not a good one. Here's a comment about the only one that is close from Jord's FAQ on cc_config.xml:

<ncpus>
Act as if there were N CPUs: run N tasks at once. This is for debugging, i.e. to simulate 2 CPUs on a machine that has only 1. Don't use it to limit the number of CPUs used by BOINC; use general preferences instead.

Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.

ID: 1378711 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45861
Credit: 814,598,610
RAC: 121,845
United States
Message 1378735 - Posted: 9 Jun 2013, 0:16:14 UTC - in response to Message 1378687.

just go into your computing preferences in boinc manager. It should be labelled as 'On multiprocessor systems use at most ... % of the processors' or something very similar. Just change that to 75%

That would not work too well for him, as he has some 171 computers or so he is administrating. It would have to be the global setting in preferences to update them all. Unless he only wants to change a few select computers.
Kitties make wonderful traveling companions on your journey through life.

Have made a few friends in this life.
Most were cats.

ID: 1378735 · Report as offensive
mikeej42

Send message
Joined: 26 Oct 00
Posts: 109
Credit: 791,863,759
RAC: 0
United States
Message 1378744 - Posted: 9 Jun 2013, 0:46:12 UTC - in response to Message 1378735.

Yes, pushing out to all nodes is the issue. Hopefully I can figure out how to recover my account on BOINCSTATS-BAM and use the groups to push out a new global_prefs.xml.

setting ncpus did free a core but I will head the warning that it should not be used.


ID: 1378744 · Report as offensive
juan BFP
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 5847
Credit: 330,512,893
RAC: 7,687
Panama
Message 1378745 - Posted: 9 Jun 2013, 0:46:41 UTC
Last modified: 9 Jun 2013, 0:50:20 UTC

Not the right thread but anyone knows who to make the boinc to run for example:

2 GPUGrid WU + 4 Cuda WU + 6 CPU task fixed on the same host

Explain: I have 2x690 and want to run GPUGrid on one GPU (2 WU) and SETI on the second (4 Cuda WU) the rest is to be crunched by the CPU, ask because i don´t want to leave the scheduler choose what to run, i want to decide how many task of each king will run at any given time.

I know there is a set for fix the max number of concurrent task (max_concurrent) on each hosts, but the user manual does not expain if the setting is for each project or the setting is global.


ID: 1378745 · Report as offensive
Profile MarkJProject Donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 08
Posts: 978
Credit: 33,082,131
RAC: 10,475
Australia
Message 1378748 - Posted: 9 Jun 2013, 0:57:56 UTC - in response to Message 1378745.

Not the right thread but anyone knows who to make the boinc to run for example:

2 GPUGrid WU + 4 Cuda WU + 6 CPU task fixed on the same host

Explain: I have 2x690 and want to run GPUGrid on one GPU (2 WU) and SETI on the second (4 Cuda WU) the rest is to be crunched by the CPU, ask because i don´t want to leave the scheduler choose what to run, i want to decide how many task of each king will run at any given time.

I know there is a set for fix the max number of concurrent task (max_concurrent) on each hosts, but the user manual does not expain if the setting is for each project or the setting is global.


About all you can do is exclude GPUgrid from one and vice-versa on the other one. Use the exclude_gpu tags in cc_config to do this. It causes scheduling issues though.

BOINC blog

ID: 1378748 · Report as offensive
mikeej42

Send message
Joined: 26 Oct 00
Posts: 109
Credit: 791,863,759
RAC: 0
United States
Message 1378751 - Posted: 9 Jun 2013, 1:17:50 UTC - in response to Message 1378744.

So when using Intel CPUs that have hyper-threading (4 cores and 8 threads)and you are trying to "free a core for updating a GPU" should MAX CPU percentage be at 87.5 or 75?


ID: 1378751 · Report as offensive
juan BFP
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 5847
Credit: 330,512,893
RAC: 7,687
Panama
Message 1378752 - Posted: 9 Jun 2013, 1:20:15 UTC - in response to Message 1378748.
Last modified: 9 Jun 2013, 1:21:05 UTC

Not the right thread but anyone knows who to make the boinc to run for example:

2 GPUGrid WU + 4 Cuda WU + 6 CPU task fixed on the same host

Explain: I have 2x690 and want to run GPUGrid on one GPU (2 WU) and SETI on the second (4 Cuda WU) the rest is to be crunched by the CPU, ask because i don´t want to leave the scheduler choose what to run, i want to decide how many task of each king will run at any given time.

I know there is a set for fix the max number of concurrent task (max_concurrent) on each hosts, but the user manual does not expain if the setting is for each project or the setting is global.


About all you can do is exclude GPUgrid from one and vice-versa on the other one. Use the exclude_gpu tags in cc_config to do this. It causes scheduling issues though.

With 7.0.64 there are a new variable <max_concurrent> that allow you to do that, more easely, afeter my last post i made some test and apears the configuration works per project allowing exactly what i want to do. You choose the exact number of concurrent task per aplication. I ´m just not sureif that is the best way to do that.

ID: 1378752 · Report as offensive
juan BFP
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 5847
Credit: 330,512,893
RAC: 7,687
Panama
Message 1378753 - Posted: 9 Jun 2013, 1:22:48 UTC - in response to Message 1378751.
Last modified: 9 Jun 2013, 1:23:39 UTC

So when using Intel CPUs that have hyper-threading (4 cores and 8 threads)and you are trying to "free a core for updating a GPU" should MAX CPU percentage be at 87.5 or 75?

The number depens if you have HT enabled or no in your BIOS, if enabled use 87,5 if no 75. Normaly the defoult of the BIOS is HT ON.

ID: 1378753 · Report as offensive
Richard HaselgroveProject Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 11136
Credit: 83,501,584
RAC: 41,140
United Kingdom
Message 1378890 - Posted: 9 Jun 2013, 11:04:35 UTC - in response to Message 1378745.

Not the right thread but anyone knows who to make the boinc to run for example:

2 GPUGrid WU + 4 Cuda WU + 6 CPU task fixed on the same host

Explain: I have 2x690 and want to run GPUGrid on one GPU (2 WU) and SETI on the second (4 Cuda WU) the rest is to be crunched by the CPU, ask because i don´t want to leave the scheduler choose what to run, i want to decide how many task of each king will run at any given time.

I know there is a set for fix the max number of concurrent task (max_concurrent) on each hosts, but the user manual does not expain if the setting is for each project or the setting is global.

The app_config.xml file is per project, because it lives in the project directory.

Within the file, max_concurrent applies to the <app><name> tag - so at GPUGrid you can set different values for acemdlong and acemdshort, for example. What you can't do is to set a value for the project as a whole, so it would be best to restrict yourself to a single queue and set the value to suit that one.

ID: 1378890 · Report as offensive
juan BFP
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 5847
Credit: 330,512,893
RAC: 7,687
Panama
Message 1378904 - Posted: 9 Jun 2013, 12:27:02 UTC - in response to Message 1378890.
Last modified: 9 Jun 2013, 12:27:29 UTC

Not the right thread but anyone knows who to make the boinc to run for example:

2 GPUGrid WU + 4 Cuda WU + 6 CPU task fixed on the same host

Explain: I have 2x690 and want to run GPUGrid on one GPU (2 WU) and SETI on the second (4 Cuda WU) the rest is to be crunched by the CPU, ask because i don´t want to leave the scheduler choose what to run, i want to decide how many task of each king will run at any given time.

I know there is a set for fix the max number of concurrent task (max_concurrent) on each hosts, but the user manual does not expain if the setting is for each project or the setting is global.

The app_config.xml file is per project, because it lives in the project directory.

Within the file, max_concurrent applies to the <app><name> tag - so at GPUGrid you can set different values for acemdlong and acemdshort, for example. What you can't do is to set a value for the project as a whole, so it would be best to restrict yourself to a single queue and set the value to suit that one.


I made that and works perfect!, thanks for the info.

ID: 1378904 · Report as offensive

Message boards : Number crunching : Best way to free cores when running multiple projects


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.