contoling cuda tasks

Message boards : Number crunching : contoling cuda tasks
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile AlienDancer
Volunteer tester
Avatar

Send message
Joined: 8 Sep 99
Posts: 68
Credit: 12,473,416
RAC: 0
Message 1554863 - Posted: 11 Aug 2014, 2:51:27 UTC

I want to limit how many cuda tasks I have at any given time. I don't see anyway to do that and still get tasks for my CPU. If I set cuda to no in seti preferences will the tasks I have continue to run?
ID: 1554863 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1554864 - Posted: 11 Aug 2014, 3:06:02 UTC - in response to Message 1554863.  

I want to limit how many cuda tasks I have at any given time. I don't see anyway to do that and still get tasks for my CPU. If I set cuda to no in seti preferences will the tasks I have continue to run?


yes, the settings on the site only control what you get not what you already have.

ID: 1554864 · Report as offensive
Darth Beaver Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 20 Aug 99
Posts: 6728
Credit: 21,443,075
RAC: 3
Australia
Message 1554892 - Posted: 11 Aug 2014, 4:11:24 UTC - in response to Message 1554863.  
Last modified: 11 Aug 2014, 4:13:22 UTC

Allen can i ask why you wish to do this ?

you should only recieve what you have set in the buffer ?
both cuda and CPU the servers work out how many you need but i don't think you can limit just 1 type if your doing both types

Automatically , but you mite be able to do it manually by changing setting and keeping a eye open to what and how many you have

pain in the bum if you ask me to do it that way
ID: 1554892 · Report as offensive
Profile AlienDancer
Volunteer tester
Avatar

Send message
Joined: 8 Sep 99
Posts: 68
Credit: 12,473,416
RAC: 0
Message 1555431 - Posted: 12 Aug 2014, 3:32:50 UTC - in response to Message 1554892.  
Last modified: 12 Aug 2014, 3:35:47 UTC

I have burned up a couple of GPU's by pushing them too hard. I haven't run CUDA for awhile but just got two new machines. I won't run CUDA on the laptop at all even though it's probably capable so I set up a special profile for it. Also stopped running CUDA on my AMD machine about a year ago because it was getting too hot. I don't have central air and the window unit is in another room so I thought I would try CUDA on my ASUS but only want to let CUDA run on nights that are fairly cool at least until October or so when it cools off. I thought to do that it would be best to let the machine finish crunching what it already had before getting new tasks.

I set it up last night and its working OK. Shouldn't be too much trouble just suspend the GPU during the day when I am using the machine then right before I quit let it run again. Whenever I want new GPU tasks I just need to set the preference so it will grab some more. I could just let it get tasks but I noticed that on my slower machine and my laptop Einstein was sending too many tasks and I ended up aborting the ones that weren't going to be finished by the deadline. I didn't want that to start happening with the CUDA since I will limit when they run. I probably need to check my profile over on Einstein and maybe see if I can cut back how many tasks I get.

I was actually doing this when CUDA first came out but could not remember exactly how I had set it up, so I asked justg to be sure I got it right.
ID: 1555431 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1555438 - Posted: 12 Aug 2014, 3:42:01 UTC - in response to Message 1555431.  

I have burned up a couple of GPU's by pushing them too hard. I haven't run CUDA for awhile but just got two new machines. I won't run CUDA on the laptop at all even though it's probably capable so I set up a special profile for it. Also stopped running CUDA on my AMD machine about a year ago because it was getting too hot. I don't have central air and the window unit is in another room so I thought I would try CUDA on my ASUS but only want to let CUDA run on nights that are fairly cool at least until October or so when it cools off. I thought to do that it would be best to let the machine finish crunching what it already had before getting new tasks.

I set it up last night and its working OK. Shouldn't be too much trouble just suspend the GPU during the day when I am using the machine then right before I quit let it run again. Whenever I want new GPU tasks I just need to set the preference so it will grab some more. I could just let it get tasks but I noticed that on my slower machine and my laptop Einstein was sending too many tasks and I ended up aborting the ones that weren't going to be finished by the deadline. I didn't want that to start happening with the CUDA since I will limit when they run. I probably need to check my profile over on Einstein and maybe see if I can cut back how many tasks I get.

I was actually doing this when CUDA first came out but could not remember exactly how I had set it up, so I asked justg to be sure I got it right.

There is a value "While BOINC running, % of time GPU work is allowed" that BOINC keeps track of. Once that feature is enabled here then BOINC should only request as much as you would actually finish. Much like it does for the system usage percentages.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1555438 · Report as offensive
Darth Beaver Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 20 Aug 99
Posts: 6728
Credit: 21,443,075
RAC: 3
Australia
Message 1555511 - Posted: 12 Aug 2014, 6:14:00 UTC - in response to Message 1555431.  

AlienDancer

if you are going to crunch then it does not matter weather i'm talking about the CPU or GPU you must CLEAN THE WHOLE MACHINE AT LEAST EVERY 6 Months or dust will build up and burn out both CPU and GPU . if your place is hot leave the case open and open a window . I too have lost at least 1 GPU but that was years ago and i learnt then CLEAN MACHINE REGULARLY i do mine every 3 months the whole thing i pull apart clean all the boards with a elctronic solvent cleaner and then put it back togeather and i don't have problems and i even get out the vacume cleaner and clean the lappy vents with it. (Lappy's to hard to pull apart to many dan screws and i allway have 1 or 2 left over ) so now i find the vac does a good enough job

do this and you wont lose any more GPU's
ID: 1555511 · Report as offensive
HighTech67

Send message
Joined: 22 Jun 03
Posts: 20
Credit: 5,936,475
RAC: 126
United States
Message 1555519 - Posted: 12 Aug 2014, 6:42:15 UTC - in response to Message 1555438.  


There is a value "While BOINC running, % of time GPU work is allowed" that BOINC keeps track of. Once that feature is enabled here then BOINC should only request as much as you would actually finish. Much like it does for the system usage percentages.


HAL9000,

When you say "Once that feature is enabled here", where do you mean? At Seti@home? Or on our clients? I cannot find that anywhere on my client and I am running 7.2.42 on Winbloze.

AlienDancer,

When I do crunch on my GPUs, I control heat on them using app_config.xml, telling the tasks how much of a percentage of the GPU they can use. It works wonders. Doing this, I can also run multiple GPU tasks if I want and increase my RAC, but as you can imagine, this really increases heat output. I also use Precision X on my EVGA cards to slow them down when needed. BUT, I also have central A/C and a room A/C where the computers are located. I seldom use the room A/C anymore due to cost and cover the vent in the winter in that room as I do not need heat in there.

Glenn savill,

You are correct, dust will kill a machine. I live in a very dusty location and I use positive case ventilation or whatever it is called. I have fans blowing into my cases except for my PS and video card. All the fans blowing in have filters all them but I still have to take the cases apart and blow them out every few months anyway.
ID: 1555519 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1555637 - Posted: 12 Aug 2014, 12:18:46 UTC

HighTech67 If you look at the detail for one of your machines you will see
While BOINC running, % of time GPU work is allowed 0.00%
The features is implemented in BOINC, but the project has not flipped the switch to use it as of yet.

It could be that the feature is not fully ready yet or they have just chosen not to use it yet.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1555637 · Report as offensive
HighTech67

Send message
Joined: 22 Jun 03
Posts: 20
Credit: 5,936,475
RAC: 126
United States
Message 1555699 - Posted: 12 Aug 2014, 15:05:19 UTC - in response to Message 1555637.  

HighTech67 If you look at the detail for one of your machines you will see
While BOINC running, % of time GPU work is allowed 0.00%
The features is implemented in BOINC, but the project has not flipped the switch to use it as of yet.

It could be that the feature is not fully ready yet or they have just chosen not to use it yet.


HAL9000, thank you. I had never noticed that.

The only reason I look that far down on that page is to set the location for a new computer and I have not done that for a while. Even then, I admit I don't pay attention to most of what is there.

Could that be a not yet fully implemented feature in the latest release of the BOINC server software? I have no idea when that was released as I don't have anything to do with any project except crunch and I don't do that very well. I think I need to thin my projects down.
ID: 1555699 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1555710 - Posted: 12 Aug 2014, 15:25:18 UTC - in response to Message 1555699.  

HighTech67 If you look at the detail for one of your machines you will see
While BOINC running, % of time GPU work is allowed 0.00%
The features is implemented in BOINC, but the project has not flipped the switch to use it as of yet.

It could be that the feature is not fully ready yet or they have just chosen not to use it yet.


HAL9000, thank you. I had never noticed that.

The only reason I look that far down on that page is to set the location for a new computer and I have not done that for a while. Even then, I admit I don't pay attention to most of what is there.

Could that be a not yet fully implemented feature in the latest release of the BOINC server software? I have no idea when that was released as I don't have anything to do with any project except crunch and I don't do that very well. I think I need to thin my projects down.

The client has been tracking the information for a while. I think the 7.0.64 version I am suing is doing it as well. There might be something in the BOINC white papers as to why it is not being used yet, but I haven't felt like reading them.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1555710 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1555717 - Posted: 12 Aug 2014, 15:39:41 UTC - in response to Message 1555710.  

HighTech67 If you look at the detail for one of your machines you will see
While BOINC running, % of time GPU work is allowed 0.00%
The features is implemented in BOINC, but the project has not flipped the switch to use it as of yet.

It could be that the feature is not fully ready yet or they have just chosen not to use it yet.


HAL9000, thank you. I had never noticed that.

The only reason I look that far down on that page is to set the location for a new computer and I have not done that for a while. Even then, I admit I don't pay attention to most of what is there.

Could that be a not yet fully implemented feature in the latest release of the BOINC server software? I have no idea when that was released as I don't have anything to do with any project except crunch and I don't do that very well. I think I need to thin my projects down.

The client has been tracking the information for a while. I think the 7.0.64 version I am suing is doing it as well. There might be something in the BOINC white papers as to why it is not being used yet, but I haven't felt like reading them.

The client has been tracking the information since August 2010, and tracking it accurately since November 2010.

But the server wasn't given the ability to store/use/display the data until March 2014.

http://boinc.berkeley.edu/gitweb/?p=boinc-v2.git&a=search&h=HEAD&st=commit&s=gpu_active_frac
ID: 1555717 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1555727 - Posted: 12 Aug 2014, 15:51:56 UTC - in response to Message 1555710.  

HighTech67 If you look at the detail for one of your machines you will see
While BOINC running, % of time GPU work is allowed 0.00%
The features is implemented in BOINC, but the project has not flipped the switch to use it as of yet.

It could be that the feature is not fully ready yet or they have just chosen not to use it yet.


HAL9000, thank you. I had never noticed that.

The only reason I look that far down on that page is to set the location for a new computer and I have not done that for a while. Even then, I admit I don't pay attention to most of what is there.

Could that be a not yet fully implemented feature in the latest release of the BOINC server software? I have no idea when that was released as I don't have anything to do with any project except crunch and I don't do that very well. I think I need to thin my projects down.

The client has been tracking the information for a while. I think the 7.0.64 version I am suing is doing it as well. There might be something in the BOINC white papers as to why it is not being used yet, but I haven't felt like reading them.


Yeah, some of us have been trawling the scheduler (server) design for a while now, and the only figures really wired into work fetch at the sendWork point relate to total host-usage. A very general CPU/GPU distinction is made there, converted to Flops, as opposed to detailed per device resource usage by applications. At the moment that's a design issue coming forward as new devices like FPGAs and specialised ASICs are used in some projects (like bitcoin mining) already, and don't really use general purpose floating point operations.

WHere that's relevant here, is that GPUs were kindof tacked onto the pre-existing CPU mechanism, so there are holes in the logic there when supporting a wide range of devices in a heterogeneous arrangement.

This, self-made incomplete UML notation use-case, documents more of the scheduler work fetch structure and operation than is documented in the white papers.


"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1555727 · Report as offensive
Profile AlienDancer
Volunteer tester
Avatar

Send message
Joined: 8 Sep 99
Posts: 68
Credit: 12,473,416
RAC: 0
Message 1556028 - Posted: 13 Aug 2014, 2:44:56 UTC - in response to Message 1554892.  

Thanks Glenn, I actually am cleaning the machines about that often but not quite as thoroughly as you described. I usually take a can of air and sprayed away obvious dust. I'll try your method. Can't keep the case open though, I used to do that but since I acquired two cats I keep the cases sealed, not because the cats can get in they generally stay away but because the first time I cleaned them after I got the cats I was pulling cat hair out of the CPU fan. I was not a happy camper about that. I do have an industrial strength fan that I have aimed at the case.
ID: 1556028 · Report as offensive
Darrell Wilcox Project Donor
Volunteer tester

Send message
Joined: 11 Nov 99
Posts: 303
Credit: 180,954,940
RAC: 118
Vietnam
Message 1556378 - Posted: 13 Aug 2014, 20:12:37 UTC - in response to Message 1555431.  

I have burned up a couple of GPU's ...
You have my sympathy. I live in Vietnam just south of Ho Chi Minh, where daytime temps reach nearly 100F in my computer room. I run both CPU and GPU full time (24/7) but protect them from overtemps with Tthrottle by Efmer. No one has mentioned it here but you might want to take a look to see if it fits your needs.

I have to blow the dust out of my machines each 3 months or so also.
ID: 1556378 · Report as offensive
Profile AlienDancer
Volunteer tester
Avatar

Send message
Joined: 8 Sep 99
Posts: 68
Credit: 12,473,416
RAC: 0
Message 1556468 - Posted: 14 Aug 2014, 0:28:55 UTC - in response to Message 1556378.  

Thanks, will defiantly check out Tthrottle.
ID: 1556468 · Report as offensive

Message boards : Number crunching : contoling cuda tasks


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.