Running One CPU Unit with Multiple GPU Units on a Multi-Processor CPU

Message boards : Number crunching : Running One CPU Unit with Multiple GPU Units on a Multi-Processor CPU
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982399 - Posted: 27 Feb 2019, 4:43:20 UTC - in response to Message 1982396.  

Sorry, I'm not following you.
Are you saying the formula doesn't work?
100 divided by 16 = 6.25 x 11 = 68.75
To run 11 CPU tasks you would set it to 69%
100 divided by 24 = 4.17 x 16 = 66.67
To run 16 CPU tasks set it to 67%
The CPU setting has nothing to do with the GPUs....at least in SETI.
I run SETI, nothing else. This was the SETI board the last I checked.
ID: 1982399 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1982401 - Posted: 27 Feb 2019, 4:49:32 UTC - in response to Message 1982399.  

No I want to run only 11 tasks total on the Ryzen's.

I only want to run 16 total tasks on the TR.

I keep cpu tasks only on the physical cores. The gpu tasks only supported by the virtual cores.

Ryzen 2700X only has 8 physical cores. TR 2920X only has 12 physical cores. So my settings allow exactly that.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1982401 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982403 - Posted: 27 Feb 2019, 5:06:47 UTC - in response to Message 1982401.  

I suppose it's too late, still don't understand your problem.
If you want to run 8 CPUs then set the CPU settings as such.
In SETI the CPU settings are for the CPU, nothing else.
ID: 1982403 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1982404 - Posted: 27 Feb 2019, 5:13:00 UTC - in response to Message 1982403.  

Setting cpu % the way you describe does not keep me limited to only 8 cpu or 12 cpu tasks when I run other project gpu tasks. I would be running more than 8 or 12 cpu tasks once Einstein, MilkyWay or GPUGrid gpu tasks move onboard. I've tried your way, it does not work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1982404 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982405 - Posted: 27 Feb 2019, 5:29:07 UTC - in response to Message 1982404.  

I suppose it's a good thing I don't run Einstein, MilkyWay or GPUGrid then if they use some other method.
Sounds like you'll have to ask them why they are different, cause I've never been there. The method I described is how SETI has worked for years.
It works very well for me and any one else here.
ID: 1982405 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1982406 - Posted: 27 Feb 2019, 5:44:57 UTC

this has already been argued several times ad nauseam. when everyone is in consensus with a lone party that is convinced it's everyone else who's wrong, sometimes it's better to just move on.

it all comes down to resource accounting.

TBar is using settings that tell BOINC he is using 0.1 of a CPU (thread) per GPU job.
x12 GPU jobs = BOINC accounts for 1.2 CPU threads
rounds to 1 thread, integer

in that case, you can run it all the way down to 1% CPU setting and BOINC will keep all his jobs running because at even at 1% it will try to keep 1 thread running, since that's the minimum it can control.
but he's clearly using more than 0.1 thread per GPU job since his total CPU use with only GPU jobs is 62%, BOINC just isnt accounting for it. BOINC doesnt know how much CPU is actually being used, it's not looking at the CPU utilization, it's looking at how many jobs are running, how many CPUs (threads) you have, and the config files with whatever you told it about CPU use from GPU jobs.
if TBar bumped that to say 0.4 so BOINC accounted for actual CPU use, it would probably act accordingly. but that suggestion is also an act in futility since he doesn't want to change it. with his current settings he wont see any affect on his GPU jobs by manipulating the CPU use setting until he had over 15 GPUs, which also wont happen since his hardware wont support it.

Keith (along with several other, probably Zal also), if i recall correctly, tells BOINC to allocate (or account) 1 full CPU thread per GPU job.
so GPU jobs and CPU jobs act equally in this case.

if you use -nobs, you should tell boinc you're using the full thread so it properly accounts for it when/if you decide to try to limit your CPU use for whatever reason. makes things more predictable and intuitive in my opinion.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1982406 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982409 - Posted: 27 Feb 2019, 5:55:08 UTC - in response to Message 1982406.  

Actually 0.1 is More than the CUDA App is using. I suggest you look at what SETI uses for their settings. SETI uses a Lower setting than I use.
The App doesn't use anywhere near 1 full CPU without User settings. I believe the problem is with a little Group that wants to change the way SETI is working.
Good Luck with that.
ID: 1982409 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1982410 - Posted: 27 Feb 2019, 6:05:00 UTC - in response to Message 1982409.  
Last modified: 27 Feb 2019, 6:06:27 UTC

we can see from your screen shot that your CPU use is 62%
you have stated that you are not running CPU jobs, GPU only.
you have 12 GPUs, 12 GPU jobs
62%/12 GPUs = roughly 5% CPU use per GPU job.
5% of your CPU (8 threads) = 0.4 of a thread being used per GPU

but BOINC only being told to account for 0.1 from your settings. so it's acting how i described. you are correct that the config file settings do not control or restrict how much CPU is used by the app, but it's pretty obvious that it's being used for BOINCs own accounting. this is independent of whatever project you're running.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1982410 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1982412 - Posted: 27 Feb 2019, 6:20:13 UTC - in response to Message 1982406.  

Keith (along with several other, probably Zal also), if i recall correctly, tells BOINC to allocate (or account) 1 full CPU thread per GPU job.
so GPU jobs and CPU jobs act equally in this case.

if you use -nobs, you should tell boinc you're using the full thread so it properly accounts for it when/if you decide to try to limit your CPU use for whatever reason. makes things more predictable and intuitive in my opinion.

Yes, exactly. I use a full cpu core to support every gpu task, irregardless of project origin.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1982412 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22250
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1982415 - Posted: 27 Feb 2019, 6:23:56 UTC

Doing so makes life so much simpler
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1982415 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982416 - Posted: 27 Feb 2019, 6:25:00 UTC - in response to Message 1982410.  
Last modified: 27 Feb 2019, 6:44:34 UTC

Try opening Top and see what you get for setiathome_x41p without any of your kludge settings.
Then try one task on a test machine. There is a Lot of overhead running 12 tasks.
Then run a setiathome Stock CUDA App and see what settings they use. It's lower than 0.1.
You still think the <max_ncpus> has some effect on actual CPU use, it Doesn't. It's only used for scheduling, see what SETI uses.

Here, go to Mike's, Download the CUDA App and checkout the setting. It's the same that SETI uses;
http://mikesworld.eu/download.html
<avg_ncpus>0.040000</avg_ncpus>
<max_ncpus>0.040000</max_ncpus>
ID: 1982416 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1982420 - Posted: 27 Feb 2019, 6:39:18 UTC - in response to Message 1982416.  

I never said the App was using it. I said your system was.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1982420 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1982422 - Posted: 27 Feb 2019, 6:42:16 UTC - in response to Message 1982416.  
Last modified: 27 Feb 2019, 6:43:19 UTC


You still think the <max_ncpus> has some effect on actual CPU use, it Doesn't.


I never said anything of the sort. I explicitly said I agreed with you on that point.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1982422 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1982424 - Posted: 27 Feb 2019, 6:51:28 UTC - in response to Message 1982416.  

Try opening Top and see what you get for setiathome_x41p without any of your kludge settings.
Then try one task on a test machine. There is a Lot of overhead running 12 tasks.
Then run a setiathome Stock CUDA App and see what settings they use. It's lower than 0.1.
You still think the <max_ncpus> has some effect on actual CPU use, it Doesn't. It's only used for scheduling, see what SETI uses.

My cpu utilization monitor chart in GKrellm matches almost exactly my % of cpu in BOINC preferences. 68% when just BOINC running and background processes. 73% when I open Chrome to browse. Chrome is a memory hog with ridiculous number of threads to run a simple web page like Seti@home.

Don't need to. The gpus use 100% of a cpu core as I intend. My settings are doing exactly what I ask of them. No need to experiment with or change what is working perfectly.
You can use your preferences and settings as you intend and are happy with.
Won't change anything in what I use for preferences and settings and achieve what I want.

YMMV.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1982424 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982426 - Posted: 27 Feb 2019, 7:13:15 UTC - in response to Message 1982424.  

The settings I'm using is what SETI will use If they ever load the App.
Get use to seeing them.
ID: 1982426 · Report as offensive
mmonnin
Volunteer tester

Send message
Joined: 8 Jun 17
Posts: 58
Credit: 10,176,849
RAC: 0
United States
Message 1982447 - Posted: 27 Feb 2019, 11:02:16 UTC - in response to Message 1982394.  
Last modified: 27 Feb 2019, 11:02:32 UTC

[quote]The 'Use X % of CPUs' only affects how many tasks are running. GPU tasks included. As a test, I just set my E@H app_config to use 3.0 CPUs per task and I am running two tasks in Parallel. Setting it to anything less than 50% of CPUs stops one of the GPU tasks. I'm not sure why it's 50% though. As 3 CPU threads per task is 6 required out of 8 threads on a 3770k. Anything less than 75% should have stopped the 2nd task. The only thing else running on that client is WUProp (NCI and 0.025 CPUs).

How much actual CPU usage is used by the GPU app's exe is totally up the the app whether it's CUDA vs OpenCL or if it has other params like -nobs or SWAN_SYNC.

This isn't the E@H app Board, please show that in SETI.
Apparently different Project use different Settings, test it in SETI.
Note the CPU use;

This has nothing to do with SETI but how BOINC treats tasks in general. I chose E@H since I actually run 2x tasks in parallel all the time and I do so on the PC I am typing. East of access.

I said PER APP_CONFIG that I set it to 3.0 CPUs. You can do the same with SETI if you wish but to get one task to stop running and for easier testing I had to make it more than the 0.1 CPU cores I typically run every GPU project.
ID: 1982447 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982451 - Posted: 27 Feb 2019, 11:22:55 UTC - in response to Message 1982447.  

Oh, I see. You screwed around with the default settings to get it to do something it wouldn't do with the proper settings.
Why on Earth would I want to do that? The default settings are there for a reason.... they work.

Notice they shut up when I told them to look at Top? Top shows the CUDA App using 0.1 CPU, just as the default settings are set.
That's why there are default settings, set by people that know what they are doing.
I don't care what a handful of fanatics want, I want what will work for the most people using SETI. Maybe that's why I have a handful of Apps on the SETI server, and,
you can count the number of people that have Apps on the SETI server with...well...one hand.
ID: 1982451 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24881
Credit: 3,081,182
RAC: 7
Ireland
Message 1982456 - Posted: 27 Feb 2019, 12:31:14 UTC - in response to Message 1982451.  

I don't care what a handful of fanatics want, I want what will work for the most people using SETI.
+1
I can recall a time when the Number Crunching board was extremely helpful. However since GPU crunching became the norm, that helpfulness has diminished.
It seems that there are those who have completely forgot the purpose of the project and want the best from their system/s, commendable, but the project did not demand that of anyone.

The worst has been seen over the past couple of years. For close to 50 years I have held the same particular view.
We all have 3 lives - Personal, Social & Working.
Allow the project staff that courtesy.

If the servers fall over, they'll be recovered in time, no sense in thumping one's chest in annoyance.
If one's system/s fall over, find out why & recover it.
ID: 1982456 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1982470 - Posted: 27 Feb 2019, 15:55:55 UTC - in response to Message 1982451.  

I don't know how you get top to show 0.1% cpu usage for a gpu task if using -nobs. My choice is to use -nobs and one full cpu thread to support a gpu task. That is all I have to say. It runs exactly as configured. The app gives me that choice and that is what I choose.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1982470 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1982475 - Posted: 27 Feb 2019, 16:42:44 UTC - in response to Message 1982470.  

This -> Try opening Top and see what you get for setiathome_x41p without any of your kludge settings.
Forgot that part did you?
Listen, I don't care what you do with your machine. What I do care about is this habit a certain 'person' has were he makes it a goal to follow me around and tell people to do exactly the opposite of what I tell them.
I'm sick and tied of it. Now he's trying to convince people there is something wrong with using the same default setting SETI has been using for years.
This is EXACTLY WHY SETI has so few Developers, No One will put up with the type of crap certain users level on the people trying to make SETI work.
If it doesn't stop, there is very dim future ahead.
ID: 1982475 · Report as offensive
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Running One CPU Unit with Multiple GPU Units on a Multi-Processor CPU


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.