Advice on system optimization needed.

Message boards : Number crunching : Advice on system optimization needed.
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

AuthorMessage
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11648
Credit: 173,434,505
RAC: 116,154
Australia
Message 2008269 - Posted: 19 Aug 2019, 9:35:24 UTC - in response to Message 2008268.  
Last modified: 19 Aug 2019, 9:35:34 UTC

Not to mention if you ever wanted to run say, 5 GPUs on that 4 core CPU, your method wouldn't work.
True, so reserving .5 CPUs for each GPU WU, or .33, or .25 etc would produce the same result where a full core isn't necessary- making use of all available CPU resources, without over committing it.
Grant
Darwin NT
ID: 2008269 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 4859
Credit: 582,768,508
RAC: 1,344,176
United States
Message 2008271 - Posted: 19 Aug 2019, 9:49:11 UTC - in response to Message 2008269.  
Last modified: 19 Aug 2019, 9:49:41 UTC

Why not just change the CPU setting in the BOINC Manager? I mean, the BOINC Developers Made that setting for a Reason, don't you think? Do you really think the Developers made that setting so you could keep it at 100% all the time? I don't think so. Oh, the Developers from nVidia came up with a setting of 0.04 CPU when they Developed the CUDA App for SETI. You think they didn't know what they were doing either? Dang, according to You, none of the Developers know what they are doing. Interesting.
ID: 2008271 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11648
Credit: 173,434,505
RAC: 116,154
Australia
Message 2008274 - Posted: 19 Aug 2019, 10:07:32 UTC - in response to Message 2008271.  

Why not just change the CPU setting in the BOINC Manager? I mean, the BOINC Developers Made that setting for a Reason, don't you think? Do you really think the Developers made that setting so you could keep it at 100% all the time? I don't think so. Oh, the Developers from nVidia came up with a setting of 0.04 CPU when they Developed the CUDA App for SETI. You think they didn't know what they were doing either? Dang, according to You, none of the Developers know what they are doing. Interesting.

What's with the hostility and sarcastic comments? Helping develop the Special Application is appreciated, but it doesn't give you the right to behave like a complete tool & an angry petulant child when someone expresses an opinion that is different to yours.

So instead of tantrums and snarky comments, how about actually addressing the issue?
Your suggestion allows the CPU not to be over-committed, by not making full use of it for crunching. My suggestion allows you to make full use of the CPU for crunching, without over committing the CPU.
So people can try them both (and any other suggestions other people might have) and see what works best for them.
Grant
Darwin NT
ID: 2008274 · Report as offensive     Reply Quote
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 17783
Credit: 406,492,682
RAC: 155,283
United Kingdom
Message 2008277 - Posted: 19 Aug 2019, 10:22:16 UTC

One advantage of TBar's approach is that is "instant" - reducing to 75% from 100% and you will immediately see the reduction in CPU usage (assuming a 4-core or greater CPU), whereas with Glenn's approach one has to do at least one other action (read local prefs, or set use according to prefs), when talking to those that don't understand the how and where of BOINC (and how to edit a .xml file without wrecking it) it is far safer to use the built-in feature than have to talk them though the convolutions of editing the control files.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2008277 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 1791
Credit: 757,871,530
RAC: 2,544,073
United States
Message 2008281 - Posted: 19 Aug 2019, 11:13:53 UTC - in response to Message 2008274.  

lol...
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2008281 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 1791
Credit: 757,871,530
RAC: 2,544,073
United States
Message 2008282 - Posted: 19 Aug 2019, 11:27:15 UTC - in response to Message 2008268.  

Not to mention if you ever wanted to run say, 5 GPUs on that 4 core CPU, your method wouldn't work.


you can count on 1 finger the number of people trying to do things like that. Most people would upgrade the CPU to be more capable in that situation. Also look how few people are even running more than 4 GPUs.

There used to be a guy here that would lament and complain of people giving advice only useful to a niche group. I wonder where he went.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2008282 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 4859
Credit: 582,768,508
RAC: 1,344,176
United States
Message 2008284 - Posted: 19 Aug 2019, 11:46:20 UTC - in response to Message 2008282.  

^^^^^^^
This is why I refuse to have any dealings on this forum.
EVERY Time I post this A-H shows up to harass me. I refuse to have any dealings with him. Period.
I have already posted the reasons for using the CPU control, and believe it or not, it deals with the Average SETI user, not just a handful of Team-mates.
If you think it through, you will see why the Developers went the way they did... for the average user.
Bye now.
ID: 2008284 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 1791
Credit: 757,871,530
RAC: 2,544,073
United States
Message 2008286 - Posted: 19 Aug 2019, 11:57:33 UTC - in response to Message 2008284.  

Pretty sure the BOINC developers didn’t intend to have to set some arbitrary number in the CPU% field in order to get your desired CPU use.

There’s no one setting that fits all people and use cases. If you use nobs, use 1:1 so BOINC accounts for the CPU use properly. If you don’t use nobs, then you’re fine leaving it at 0.1 because that more accurately reflects the level of CPU use in that case.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2008286 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 3540
Credit: 209,037,369
RAC: 534,314
United States
Message 2008293 - Posted: 19 Aug 2019, 12:24:34 UTC - in response to Message 2008259.  

Running nobs helps, but, you should lower your CPU use below 100% in the Task Manager. Keep lowering CPU use until the Task Manager shows around 80-90%.
Or just reserve a CPU core to support each GPU WU being processed.

In app_confi.xml
<app_config>
 <app>
  <name>setiathome_v8</name>
  <gpu_versions>
  <gpu_usage>1.0</gpu_usage>
  <cpu_usage>1.0</cpu_usage>
  </gpu_versions>
 </app>
 </app_config>
It's that simple.

No more overcommitted CPU.


I did. It did.

Tom
Oh NO.... I lost my tagline....
ID: 2008293 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 3540
Credit: 209,037,369
RAC: 534,314
United States
Message 2008295 - Posted: 19 Aug 2019, 12:26:43 UTC - in response to Message 2008277.  

One advantage of TBar's approach is that is "instant" - reducing to 75% from 100% and you will immediately see the reduction in CPU usage (assuming a 4-core or greater CPU), whereas with Glenn's approach one has to do at least one other action (read local prefs, or set use according to prefs), when talking to those that don't understand the how and where of BOINC (and how to edit a .xml file without wrecking it) it is far safer to use the built-in feature than have to talk them though the convolutions of editing the control files.


+1
Oh NO.... I lost my tagline....
ID: 2008295 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 3540
Credit: 209,037,369
RAC: 534,314
United States
Message 2008296 - Posted: 19 Aug 2019, 12:36:24 UTC - in response to Message 2008282.  

Not to mention if you ever wanted to run say, 5 GPUs on that 4 core CPU, your method wouldn't work.


you can count on 1 finger the number of people trying to do things like that. Most people would upgrade the CPU to be more capable in that situation. Also look how few people are even running more than 4 GPUs.

There used to be a guy here that would lament and complain of people giving advice only useful to a niche group. I wonder where he went.


Yes, but I am one of the minority of users who has a larger than 3-4 gpu count on a few of my machines. So I want to optimize the work load on the cpus while not slowing the gpus down.

It looks like if there was a way to (easily, under Linux) aggregate and keep all the gpu threads onto a small number of cpu threads then the rest of the cpu threads could crunch at full tilt without being interrupted by gpu threads borrowing some cpu time from them.... that might allow some more cpu threads to be run for cpu tasks...
If I could get 0.10:1 that would mean only 2 cpu threads were driving the gpus.... which would, in my case, give me 12 threads to run cpu apps rather than 8 like now.

Right now, with the 1:1 ratio, the cpu tasks are all running faster. Where is my "shoehorn" :)

Tom
Oh NO.... I lost my tagline....
ID: 2008296 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 1791
Credit: 757,871,530
RAC: 2,544,073
United States
Message 2008298 - Posted: 19 Aug 2019, 12:45:07 UTC - in response to Message 2008296.  
Last modified: 19 Aug 2019, 12:50:38 UTC

More than 4. But less than your total thread count, which was my point. In your case it’s much cleaner just set 1:1 and just use the BOINC settings to set exactly how much CPU use you want.

If you really want to run more CPU WUs, you can remove nobs and return the ncpu setting to 0.1. But GPU speeds will take a hit.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2008298 · Report as offensive     Reply Quote
Profile Eric Claussen

Send message
Joined: 31 Jan 00
Posts: 22
Credit: 2,314,798
RAC: 1,749
United States
Message 2008560 - Posted: 21 Aug 2019, 1:41:48 UTC
Last modified: 21 Aug 2019, 1:43:38 UTC

Considering additional cards. Single RTX 2080 super or dual RTX 2070? If I get a single 2080 there is a decent chance a 2nd one would follow at some point. I can add 2 additional 2 slot cards or 4 single slot cards to my machine without using riser cards or anything like that. Plenty of power for either option. Cards have to be "Turbo" style. Exhaust exit out the back.

Eric
ID: 2008560 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 9839
Credit: 923,149,877
RAC: 1,518,928
United States
Message 2008577 - Posted: 21 Aug 2019, 3:08:50 UTC - in response to Message 2008560.  

Considering additional cards. Single RTX 2080 super or dual RTX 2070? If I get a single 2080 there is a decent chance a 2nd one would follow at some point. I can add 2 additional 2 slot cards or 4 single slot cards to my machine without using riser cards or anything like that. Plenty of power for either option. Cards have to be "Turbo" style. Exhaust exit out the back.

Eric

As far as I know there are no single slot cards of any performance value to Seti. Plenty of dual slot blower style cards. If you have the moolah now to purchase either choice, I would get the two RTX 2070 Supers. If you upgrade one in the future to a RTX 2080, that gives you a leg up on another cruncher build with the card ready to go.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 2008577 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 9839
Credit: 923,149,877
RAC: 1,518,928
United States
Message 2008580 - Posted: 21 Aug 2019, 3:18:13 UTC

I have been playing around with both methods of providing cpu support to gpus today on the 7.16.1 client. All I can say is that if you have a Intel processor either method works and everything runs fine. If on the other hand you have a AMD processor, you will still be cussing the brain-dead Linux AMD cpu thread scheduler and looking for compromises.

Neither way works the way it should. It would be best to set cpu usage to 100% but you will end up with overcommitted cpu threads. And trying to use a max concurrent breaks things entirely. So the only option is to use cpu% to reduce the number of cpu cores used. But the thread scheduler can't keep the task on the same thread and constantly moves it around. And you end up with both an overcommitted cpu and poor cpu_time/run_time tracking to boot.

Maybe in five years of maturity, they will have figured out the scheduler for AMD cpu and have the stability, performance and reliability of the the Intel thread scheduler.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 2008580 · Report as offensive     Reply Quote
Profile Eric Claussen

Send message
Joined: 31 Jan 00
Posts: 22
Credit: 2,314,798
RAC: 1,749
United States
Message 2008583 - Posted: 21 Aug 2019, 3:27:49 UTC - in response to Message 2008577.  

I did find a gtx 1070 single slot called a Katana. There are a couple on ebay but not cheap enough.


Considering additional cards. Single RTX 2080 super or dual RTX 2070? If I get a single 2080 there is a decent chance a 2nd one would follow at some point. I can add 2 additional 2 slot cards or 4 single slot cards to my machine without using riser cards or anything like that. Plenty of power for either option. Cards have to be "Turbo" style. Exhaust exit out the back.

Eric

As far as I know there are no single slot cards of any performance value to Seti. Plenty of dual slot blower style cards. If you have the moolah now to purchase either choice, I would get the two RTX 2070 Supers. If you upgrade one in the future to a RTX 2080, that gives you a leg up on another cruncher build with the card ready to go.

ID: 2008583 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 4859
Credit: 582,768,508
RAC: 1,344,176
United States
Message 2008589 - Posted: 21 Aug 2019, 4:25:58 UTC - in response to Message 2008580.  
Last modified: 21 Aug 2019, 4:30:00 UTC

I have been playing around with both methods of providing cpu support to gpus today on the 7.16.1 client. All I can say is that if you have a Intel processor either method works and everything runs fine. If on the other hand you have a AMD processor, you will still be cussing the brain-dead Linux AMD cpu thread scheduler and looking for compromises.

Neither way works the way it should. It would be best to set cpu usage to 100% but you will end up with overcommitted cpu threads. And trying to use a max concurrent breaks things entirely. So the only option is to use cpu% to reduce the number of cpu cores used. But the thread scheduler can't keep the task on the same thread and constantly moves it around. And you end up with both an overcommitted cpu and poor cpu_time/run_time tracking to boot.

Maybe in five years of maturity, they will have figured out the scheduler for AMD cpu and have the stability, performance and reliability of the the Intel thread scheduler.
Is this using the Standard BOINC Build or one of the Hacked (Spoofed) Builds? As far as I know, the only people having those types of problems are using a non-standard BOINC. What happens if you try the same version of BOINC as 99% of the SETI world? I don't use complicated hacks on My BOINC builds, just the one change of a single integer, and then maybe the driver hack on the one BOINC.xml file.
ID: 2008589 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 1791
Credit: 757,871,530
RAC: 2,544,073
United States
Message 2008630 - Posted: 21 Aug 2019, 14:48:10 UTC - in response to Message 2008577.  

Considering additional cards. Single RTX 2080 super or dual RTX 2070? If I get a single 2080 there is a decent chance a 2nd one would follow at some point. I can add 2 additional 2 slot cards or 4 single slot cards to my machine without using riser cards or anything like that. Plenty of power for either option. Cards have to be "Turbo" style. Exhaust exit out the back.

Eric

As far as I know there are no single slot cards of any performance value to Seti. Plenty of dual slot blower style cards. If you have the moolah now to purchase either choice, I would get the two RTX 2070 Supers. If you upgrade one in the future to a RTX 2080, that gives you a leg up on another cruncher build with the card ready to go.


The Quadro RTX 4000 is single slot, and based on the specs should perform similarly to an RTX 2070. but it also costs about $900-$1000.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2008630 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 1791
Credit: 757,871,530
RAC: 2,544,073
United States
Message 2008633 - Posted: 21 Aug 2019, 15:06:39 UTC - in response to Message 2008589.  

I have been playing around with both methods of providing cpu support to gpus today on the 7.16.1 client. All I can say is that if you have a Intel processor either method works and everything runs fine. If on the other hand you have a AMD processor, you will still be cussing the brain-dead Linux AMD cpu thread scheduler and looking for compromises.

Neither way works the way it should. It would be best to set cpu usage to 100% but you will end up with overcommitted cpu threads. And trying to use a max concurrent breaks things entirely. So the only option is to use cpu% to reduce the number of cpu cores used. But the thread scheduler can't keep the task on the same thread and constantly moves it around. And you end up with both an overcommitted cpu and poor cpu_time/run_time tracking to boot.

Maybe in five years of maturity, they will have figured out the scheduler for AMD cpu and have the stability, performance and reliability of the the Intel thread scheduler.
Is this using the Standard BOINC Build or one of the Hacked (Spoofed) Builds? As far as I know, the only people having those types of problems are using a non-standard BOINC. What happens if you try the same version of BOINC as 99% of the SETI world? I don't use complicated hacks on My BOINC builds, just the one change of a single integer, and then maybe the driver hack on the one BOINC.xml file.


You're also not using AMD CPUs or even running any CPU work at all, so you're really comparing apples to oranges.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2008633 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 4859
Credit: 582,768,508
RAC: 1,344,176
United States
Message 2008641 - Posted: 21 Aug 2019, 16:09:00 UTC
Last modified: 21 Aug 2019, 16:15:29 UTC

^^^^^^
See what I mean.... He's back.
All I did was state No One Else has reported Keith's problem, and reminded him My builds don't have those Hacks if he wants to test with one of them.
I would suggest My build for 19.04, it has the Finish File Fix and seems to work with 18.04. Just remember to Lock the coproc file before starting the Manager with the different version of boinc. Or, build Your Own 7.16.1 without the Hacks, and lock the mentioned file.
ID: 2008641 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

Message boards : Number crunching : Advice on system optimization needed.


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.