More than one WU per GPU at the same time ?


log in

Advanced search

Questions and Answers : GPU applications : More than one WU per GPU at the same time ?

1 · 2 · 3 · 4 . . . 5 · Next
Author Message
Profile TOM
Volunteer tester
Avatar
Send message
Joined: 5 Apr 01
Posts: 59
Credit: 54,930,063
RAC: 0
Germany
Message 1078159 - Posted: 17 Feb 2011, 0:46:34 UTC

Is it possible to get more than one WU per GPU running at the same time ?
____________

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1078171 - Posted: 17 Feb 2011, 1:17:28 UTC - in response to Message 1078159.


Only if you use app_info.xml and know how to edit it.

(your computers are hidden so you have to help yourself, we can't see what hardware you have and it's pain to ask for every detail)


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile TOM
Volunteer tester
Avatar
Send message
Joined: 5 Apr 01
Posts: 59
Credit: 54,930,063
RAC: 0
Germany
Message 1078189 - Posted: 17 Feb 2011, 2:43:29 UTC - in response to Message 1078171.

I've made the computers visible, so you can take a look if you like ;-)
____________

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1078217 - Posted: 17 Feb 2011, 6:27:28 UTC - in response to Message 1078189.
Last modified: 17 Feb 2011, 6:43:22 UTC



Your GeForce GTX 480 (1503MB) can run 2 or 3 tasks at the same time (don't try this with GeForce GT 220 - it can't)

You are using app_info.xml created by Lunatics' Unified Installer v0.37
so you can edit it with Notepad to change every occurrence (probably 6 places) of:

<coproc> <type>CUDA</type> <count>1</count> </coproc>


to:
<coproc> <type>CUDA</type> <count>0.5</count> </coproc>

for 2 tasks

or to:
<coproc> <type>CUDA</type> <count>0.33</count> </coproc>

for 3 tasks

For easy change you can make 3 copies of app_info.xml (for 1,2,3 tasks per GPU) and just rename (Exit BOINC first) e.g.:
app_info.xml -> 1app_info.xml
3app_info.xml -> app_info.xml


Do you know where the app_info.xml file is?


Use GPU-Z
http://www.techpowerup.com/gpuz/

to check the GPU load and temperatures.


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile TOM
Volunteer tester
Avatar
Send message
Joined: 5 Apr 01
Posts: 59
Credit: 54,930,063
RAC: 0
Germany
Message 1078229 - Posted: 17 Feb 2011, 7:53:57 UTC - in response to Message 1078217.

Thank you for help, that looks realy nice ;-).
3 GPU WU's :
GPU Load : 100 %
Mem Used : 998 MB
GPU Temp : up from 42°C to 44°C

Do i have to change anything if there will be a second GTX 480 ?
____________

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1078230 - Posted: 17 Feb 2011, 8:06:55 UTC - in response to Message 1078229.
Last modified: 17 Feb 2011, 8:28:37 UTC

GPU Temp : up from 42°C to 44°C

That is really cool running GPU at 100% load, do you use liquid cooling?

Do i have to change anything if there will be a second GTX 480 ?

No, this setting applies to all the CUDA GPUs in the computer (3 tasks per GPU * 2 GPUs = 6 CUDA tasks total will be running)


P.S.
I wonder why do you have VLAR tasks sent to the GPU, the server is supposed to send them only to CPU as they run slow (ineffectively) on GPU
http://setiathome.berkeley.edu/results.php?hostid=5817418&offset=0&show_names=1&state=2&appid=

You can use Fred's Rescheduler to move them to CPU
http://setiathome.berkeley.edu/forum_thread.php?id=60712
http://www.efmer.eu/forum_tt/index.php?topic=428.0

Direct link to BoincRescheduler V 2.4
http://www.efmer.eu/download/boinc/scheduler/boinc_rescheduler_2_4.zip


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile TOM
Volunteer tester
Avatar
Send message
Joined: 5 Apr 01
Posts: 59
Credit: 54,930,063
RAC: 0
Germany
Message 1078264 - Posted: 17 Feb 2011, 10:58:38 UTC - in response to Message 1078230.

GPU Temp : up from 42°C to 44°C

That is really cool running GPU at 100% load, do you use liquid cooling?


Yes liquid cooling, one triple radiator with 3 120mm fans and one single radiator with 1 120mm fan to make it fit into a big tower case. I can't even hear them running, and i thing there's enough cooling capacity to run a second GPU.

I'll check Fred's Rescheduler to move them to CPU.

Thank You


____________

Profile ReiAyanami
Avatar
Send message
Joined: 6 Dec 05
Posts: 31
Credit: 63,583,246
RAC: 0
Japan
Message 1081120 - Posted: 25 Feb 2011, 3:43:01 UTC

Is the use of Lunatics' Unified Installer only way to run multiple WU per GPU?
____________

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1081149 - Posted: 25 Feb 2011, 5:44:18 UTC - in response to Message 1081120.


AFAIK the only way is to use app_info.xml no matter is it created by Lunatics' Unified Installer or manually.

Maybe the future versions of BOINC will have some option to choose the number of simultaneous tasks/GPU on per-project basis.


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile Sunny129
Avatar
Send message
Joined: 7 Nov 00
Posts: 190
Credit: 3,163,755
RAC: 0
United States
Message 1088132 - Posted: 18 Mar 2011, 17:31:58 UTC - in response to Message 1078229.


3 GPU WU's :
GPU Load : 100 %
Mem Used : 998 MB

so do S@H GPU tasks typically take up ~330MB of GPU memory (and i'm assuming we're specifically talking about MB tasks, since they far outnumber AP tasks)? or is this only with regard to nVidia GPUs (i.e. will the different architecture of my 5870 2GB GPU change the amount of GPU memory used per task)? also OP, what was your GPU load while crunching 1 task only and 2 tasks only respectively? and what are you using to monitor GPU memory consumption? i use either GPU-Z or MSI Afterburner to monitor my GPU statistics, but i have yet to find a utility that monitors GPU memory usage.


Your GeForce GTX 480 (1503MB) can run 2 or 3 tasks at the same time (don't try this with GeForce GT 220 - it can't)

Bill, is this simultaneous task assessment based on the memory capacity of the GPU? and if so, why can't the OP's GTX 480 run a 4th simultaneous task if 3 tasks are only using up ~1GB of GPU memory and he has ~500MB still available?


...and on a completely different note, i'm familiar with Multibeam/Enhanced tasks and Astropulse tasks, but what are VLAR tasks? and are there other types of tasks in addition to these 3?

TIA,
Eric
____________

Profile Gundolf Jahn
Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 357,953
RAC: 37
Germany
Message 1088141 - Posted: 18 Mar 2011, 18:03:00 UTC - in response to Message 1088132.
Last modified: 18 Mar 2011, 18:03:51 UTC

...and on a completely different note, i'm familiar with Multibeam/Enhanced tasks and Astropulse tasks, but what are VLAR tasks? and are there other types of tasks in addition to these 3?

VLAR stands for Very Low Angle Range, as opposed to Very High Angle Range (VHAR), of MultiBeam tasks. So, there are only two types of tasks (with various subtypes).

Gruß,
Gundolf

Profile Sunny129
Avatar
Send message
Joined: 7 Nov 00
Posts: 190
Credit: 3,163,755
RAC: 0
United States
Message 1088146 - Posted: 18 Mar 2011, 18:12:37 UTC - in response to Message 1088141.

VLAR stands for Very Low Angle Range, as opposed to Very High Angle Range (VHAR), of MultiBeam tasks. So, there are only two types of tasks (with various subtypes).

Gruß,
Gundolf

thanks for clearing that up Gundolf. so do AP tasks have sub-types as well, or is it just the MB tasks?
____________

Profile ignorance is no excuse
Avatar
Send message
Joined: 4 Oct 00
Posts: 9529
Credit: 44,433,274
RAC: 0
Korea, North
Message 1088148 - Posted: 18 Mar 2011, 18:18:50 UTC - in response to Message 1088146.

For ATI you do. Nvidia GPU's aren't able to run AP WU's at present
____________
In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope

End terrorism by building a school

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1088265 - Posted: 18 Mar 2011, 23:35:49 UTC - in response to Message 1088132.
Last modified: 19 Mar 2011, 0:23:47 UTC

Your GeForce GTX 480 (1503MB) can run 2 or 3 tasks at the same time (don't try this with GeForce GT 220 - it can't)

Bill, is this simultaneous task assessment based on the memory capacity of the GPU? and if so, why can't the OP's GTX 480 run a 4th simultaneous task if 3 tasks are only using up ~1GB of GPU memory and he has ~500MB still available?


The number of tasks per GPU is determined by try and success/error.

People try first 2 simultaneous tasks - if they both finish in less total time compared to 2 consecutive tasks you have success (and may try 3).
At one point (e.g. 4 tasks) you will notice that they fight for GPU/RAM resources and run too slow (e.g. 3 tasks finish in 2 hours (40 min/task) and 4 in 4 hours (60 min/task))

GPU load is good indicator - if you see 60% with 2 tasks you may try 3 to get 90-95% GPU load (but if you see 90% with 2 tasks the GPU is unlikely to handle more tasks)

So by experimenting people found that Fermi can run 2-3 tasks (GF 460-480) but e.g. GF 275 have enough load with only 1 task and is less effective with 2 tasks.

Do your own experiments monitoring the GPU load, temperature and result production (tasks/day) and find the limits (optimum) of your GPU/VideoRAM


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1088283 - Posted: 19 Mar 2011, 0:11:20 UTC - in response to Message 1088146.
Last modified: 19 Mar 2011, 0:59:56 UTC

VLAR stands for Very Low Angle Range, as opposed to Very High Angle Range (VHAR), of MultiBeam tasks. So, there are only two types of tasks (with various subtypes).

Gruß,
Gundolf

thanks for clearing that up Gundolf. so do AP tasks have sub-types as well, or is it just the MB tasks?


If you open (look inside) of any MB task file at line ~20 you will see something like this:
<true_angle_range>0.41546625839622</true_angle_range> ("normal")
or
<true_angle_range>0.011820249584185</true_angle_range> (VLAR)

If the number is "too" small (< 0.04 ?) the task is considered VLAR (during the recording of the 107 sec (any task duration) the telescope was focused at one "point" of the sky)

You can only consider that VLAR tasks are some "sub-type" of MB tasks
if you also consider the half-inflated car tyre as a different "sub-type" of tyre ;)
The car will be harder to drive but in fact the tyres are the same, only with different air pressure.


How to compute what is "normal" angle_range (the telescope was NOT focused at "something" in the sky, the telescope "stays still" i.e. just rotates with Earth)
One revolution of Earth = 360 degrees
24 hours * 3600 = 86400 seconds
One task duration is ~107 seconds so in one day there are 86400 / 107 = 807 "chunks" of 107 seconds each

360 degrees / 807 = 0.446 degrees (is "normal" angle_range)
(the "parked" telescope travels 0.446 degrees through the sky during the recording of 107 seconds length)

_________

AP tasks "sub-type" is called "percent blanked":
<fraction_blanked>0.289200</fraction_blanked> (I found it in ...\slots\0\pulse.out1)

(in this case 28.92%)

More % blanked means slower computation.


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile Sunny129
Avatar
Send message
Joined: 7 Nov 00
Posts: 190
Credit: 3,163,755
RAC: 0
United States
Message 1088326 - Posted: 19 Mar 2011, 3:46:27 UTC - in response to Message 1088265.

The number of tasks per GPU is determined by try and success/error.

People try first 2 simultaneous tasks - if they both finish in less total time compared to 2 consecutive tasks you have success (and may try 3).
At one point (e.g. 4 tasks) you will notice that they fight for GPU/RAM resources and run too slow (e.g. 3 tasks finish in 2 hours (40 min/task) and 4 in 4 hours (60 min/task))

GPU load is good indicator - if you see 60% with 2 tasks you may try 3 to get 90-95% GPU load (but if you see 90% with 2 tasks the GPU is unlikely to handle more tasks)

So by experimenting people found that Fermi can run 2-3 tasks (GF 460-480) but e.g. GF 275 have enough load with only 1 task and is less effective with 2 tasks.

Do your own experiments monitoring the GPU load, temperature and result production (tasks/day) and find the limits (optimum) of your GPU/VideoRAM

thanks for hashing it out. i'll have to see what my GPU can do once i get some time to experiment. perhaps this weekend...well, whenever i get around to it i'll post up the results, as i have yet to find such numbers for an HD 5870 2GB GPU here on the message boards.

i'm hoping TOM will chime in, b/c i'm not sure how he was able to tell that his GPU memory consumption was 998MB. was it through his nVidia driver utilities? could it have been some other OCing/hardware monitoring utility? or was he just taking notice of the matching decrease in available system memory (as monitored via the task manager) caused by the GPU tasks, and measuring that difference?

also, thanks for explaining VLARs. i see now that VLARs and VHARs cover the gamut of a range of angles of surveyed sky for the MB/Enhanced SETI app. and i see simply by your example of a 28.92% blanked WU that there must be quite a range of percentages blanked. but is blanking literally to Astropulse what angle width is to Multibeam? in other words, does percent blanking have to do with the angle width of a survey that generates AP WUs?
____________

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1088530 - Posted: 19 Mar 2011, 19:28:03 UTC - in response to Message 1088326.
Last modified: 19 Mar 2011, 19:47:24 UTC

i'm hoping TOM will chime in, b/c i'm not sure how he was able to tell that his GPU memory consumption was 998MB.


GPU-Z shows "Memory Used" but maybe not on all video cards:





Also:
http://www.overclock.net/attachments/software-news/156434d1274481904-tpu-gpu-z-v0-4-3-gpuz.jpg


Try older version if GPU-Z v0.5.1 do not show it:
TechPowerUp GPU-Z v0.4.9
http://www.techpowerup.com/downloads/1907/TechPowerUp%20GPU-Z%20v0.4.9.html

TechPowerUp GPU-Z v0.4.2
http://www.techpowerup.com/downloads/1788/TechPowerUp%20GPU-Z%20v0.4.2.html


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1088536 - Posted: 19 Mar 2011, 19:51:29 UTC - in response to Message 1088326.

but is blanking literally to Astropulse what angle width is to Multibeam? in other words, does percent blanking have to do with the angle width of a survey that generates AP WUs?


No, I think blanking is removing the contamination by (military?) radar signals near Arecibo.


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Profile Sunny129
Avatar
Send message
Joined: 7 Nov 00
Posts: 190
Credit: 3,163,755
RAC: 0
United States
Message 1088567 - Posted: 19 Mar 2011, 21:34:15 UTC
Last modified: 19 Mar 2011, 21:56:16 UTC

...had no luck with the older versions GPU-Z. so it must just be a feature GPU-Z only makes available for nVidia GPUs and not ATI GPUs. i wonder if there is any other software out there that does allow ATI users to monitor their GPU memory usage?


also, i tried running 2 S@H GPU tasks simultaneously on the 5870, and while BOINC allowed it without errors, only 1 task was actually making progress even though both task's run times were ticking away. i wasn't expecting it, but after changing the app_info.xml file back to "run 1 task at a time" and restarting BOINC, the other task that made 0% progress no longer had 4:25 of run time on the clock and was being shown as "ready to start," which is nice to say the least b/c i didn't get an error. but this does seem odd. on the other hand, my GPU usage hovers at 90+% while crunching only 1 S@H GPU task, so perhaps that's why it isn't taking on more simultaneous work, even when BOINC and/or the application itself is instructed to allow it. is this normal for an ATI GPU? are nVidia GPUs known to be more efficient than ATI GPUs when it comes to crunching S@H MB and AP?
____________

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2640
Credit: 5,988,660
RAC: 3,812
Bulgaria
Message 1088657 - Posted: 20 Mar 2011, 1:31:59 UTC - in response to Message 1088567.
Last modified: 20 Mar 2011, 2:23:07 UTC

...had no luck with the older versions GPU-Z. so it must just be a feature GPU-Z only makes available for nVidia GPUs and not ATI GPUs. i wonder if there is any other software out there that does allow ATI users to monitor their GPU memory usage?


also, i tried running 2 S@H GPU tasks simultaneously on the 5870, and while BOINC allowed it without errors, only 1 task was actually making progress even though both task's run times were ticking away. i wasn't expecting it, but after changing the app_info.xml file back to "run 1 task at a time" and restarting BOINC, the other task that made 0% progress no longer had 4:25 of run time on the clock and was being shown as "ready to start," which is nice to say the least b/c i didn't get an error. but this does seem odd. on the other hand, my GPU usage hovers at 90+% while crunching only 1 S@H GPU task, so perhaps that's why it isn't taking on more simultaneous work, even when BOINC and/or the application itself is instructed to allow it. is this normal for an ATI GPU? are nVidia GPUs known to be more efficient than ATI GPUs when it comes to crunching S@H MB and AP?


Do you remember what I (pardon - Yoda) told you here?:
http://setiathome.berkeley.edu/forum_thread.php?id=63294&nowrap=true#1086213

<count> is one of the things to change but "there is another"

So did you change both:
<coproc> <type>ATI</type> <count>0.5</count> </coproc>

and:
<cmdline>-instances_per_device 2</cmdline>

_______

For GPU monitoring try some of the programs here:
http://setiathome.berkeley.edu/forum_thread.php?id=62044
http://setiathome.berkeley.edu/forum_thread.php?id=59292

(e.g. GPU Caps Viewer, EVEREST (AIDA64))


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

1 · 2 · 3 · 4 . . . 5 · Next

Questions and Answers : GPU applications : More than one WU per GPU at the same time ?

Copyright © 2014 University of California