AK V8 + CUDA MB team work mod

Message boards : Number crunching : AK V8 + CUDA MB team work mod
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 3 · 4 · 5 · 6 · 7 · 8 · 9 . . . 12 · Next

AuthorMessage
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 861570 - Posted: 3 Feb 2009, 11:00:13 UTC - in response to Message 861424.  

@ Raistmer

Your V7 mod thread is closed, so I ask here..

In the app_info.xml is this entry:

<avg_ncpus>0.040000</avg_ncpus>
<max_ncpus>0.040000</max_ncpus>

This is for the CPU-support for the GPU?
That the CPU [one Core] support max. 0.04 % the GPU?

AFAIK it's just hint to BOINC how much CPU this app will use that should allow correct scheduling, but apparently only in theory.


If I would like to crunch only on the GPU/s, I could insert there 1.000000 ?
Or I could delete the 6 [all] entries?

Sorry, I try to increase performance, if you wanna decrease it you should figure out how by himself ;)


Which BOINC Version you recommend for your mod?

No recommendations on BOINC part. I use 6.4.5 still.
In general - I recommend most stable version that support correct CUDA/CPU scheduling and project pairing... but not sure it exist ;)


And which version for stock CUDA?

All needed libs included. If you find that it will be faster with another CUDA DLL set, please, report here.


<ncpus>NUMBER_OF_CORES+2</ncpus> entry is needed for only GPU-crunching?
[Quad-CPU + 2 GPUs]


No, it needed to fully utilize both CPU and GPU, look head posts of both threads...
ID: 861570 · Report as offensive
Morten Ross
Volunteer tester
Avatar

Send message
Joined: 30 Apr 01
Posts: 183
Credit: 385,664,915
RAC: 0
Norway
Message 861596 - Posted: 3 Feb 2009, 12:59:31 UTC - in response to Message 858922.  

Ok, my little C2D is chugging along nicely except for one little annoyance. During our last little outage I ran out of MBs. When it didn't have anything to keep the cuda busy it picked up a third AP to crunch. I let it go for awhile and noticed it caused both APs to slow down. Now that I have more MBs, it seems to think it was ok to run three APs so it keeps picking up on the AP instead of starting an MB on the Cuda. I have to watch the little bugger to make sure it picks up an MB each time it finishes one.

LOL, BOINC is quite capable of doing that itself without Raistmer's help.

Since adding CUDA to a quad (BOINC v6.4.5, ncpus=5), I'm often seeing five CPU cores in use, plus a sixth task on the GPU. Doesn't matter if you're running fast tasks on the GPU (sorts itself out with the reschedule when the GPU task finishes), but when you pick up the occasional VLAR it can stay in 'n+1+1' mode for quite a while.


Raistmer:
Now that there are no more AP WUs (I'm only running S@H), I've noticed the following using V8a:

After some time I hear that my GPU fan is silent - confirmed in task manager - 5 CPU-tasks are running on Quad system, and Cuda-executable is 100% idle - no GPU-feeding. I can also confirm by looking at the CPU-time for each completed WU being "CPU-time" - much longer than for the GPU-crunched WUs.
I exit and restart boincmgr and the GPU is once again being fed and being fed for some time until this repeats itself.

Is this "working as designed"?

Morten Ross
Morten Ross
ID: 861596 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 861601 - Posted: 3 Feb 2009, 13:28:36 UTC - in response to Message 861596.  
Last modified: 3 Feb 2009, 13:29:26 UTC


Is this "working as designed"?

Morten Ross


No, it definitely works not as designed :)
Could you next time you noticed such thing do next:
1) try to open gpu_file_lock in some editor, notepad for example - could you open it?
2) try to wait till one of CPU tasks (one of 5 CPU tasks in that condition) finishes. Will next one be started on GPU or on CPU again ?

I will look closely on my host if this behavior repeats here (although I have some einstein tasks to do too)
ID: 861601 · Report as offensive
Morten Ross
Volunteer tester
Avatar

Send message
Joined: 30 Apr 01
Posts: 183
Credit: 385,664,915
RAC: 0
Norway
Message 861618 - Posted: 3 Feb 2009, 14:23:08 UTC - in response to Message 861601.  
Last modified: 3 Feb 2009, 14:39:34 UTC

Hi,

The gpu_file_lock is Zero bytes and timestamp is 01.02.2009 12:02 (?).... I cannot copy this file nor open it, so it's really locked :-)

I notice that the GPU WU is in the current scenario set to "Waiting to run"..... It's got a deadline of 13.02.200913.40:20, whilst the running ones have 13.02.2009 13:39:42 - they run on hight priority.

It seems the problem might lie in prioritization mechanism...?

When I restart boincmgr, the "Waiting to run" task is still waiting and now the GPU is fed one of the perviously CPU-fed WUs.

Morten
Morten Ross
ID: 861618 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 861633 - Posted: 3 Feb 2009, 14:58:08 UTC - in response to Message 861570.  

@ Raistmer

Your V7 mod thread is closed, so I ask here..

In the app_info.xml is this entry:

<avg_ncpus>0.040000</avg_ncpus>
<max_ncpus>0.040000</max_ncpus>

This is for the CPU-support for the GPU?
That the CPU [one Core] support max. 0.04 % the GPU?

AFAIK it's just hint to BOINC how much CPU this app will use that should allow correct scheduling, but apparently only in theory.



If I would like to crunch only on the GPU/s, I could insert there 1.000000 ?
Or I could delete the 6 [all] entries?

Sorry, I try to increase performance, if you wanna decrease it you should figure out how by himself ;)



Thanks for reply!

Sorry for my ignorance.. I have no knowledge to mod the app_info.xml..

Only to 'increase' my knowledge.. ;-)
I could delete this entries or mod it to 1.000000, or it will crash something?

I would like to test, how much the performance of my two GPUs with highest possible CPU support is..

This app_info.xml would be well for only MB - CUDA crunching on GPUs?

<app_info>
<app>
<name>setiathome_enhanced</name>
</app>
<file_info>
<name>MB_6.08_mod_VLAR_kill_CUDA.exe</name>
<executable/>
</file_info>
<file_info>
<name>cudart.dll</name>
<executable/>
</file_info>
<file_info>
<name>cufft.dll</name>
<executable/>
</file_info>
<file_info>
<name>libfftw3f-3-1-1a_upx.dll</name>
<executable/>
</file_info>
<app_version>
<app_name>setiathome_enhanced</app_name>
<version_num>608</version_num>
<plan_class>cuda</plan_class>
<coproc>
<type>CUDA</type>
<count>1</count>
</coproc>
<file_ref>
<file_name>MB_6.08_mod_VLAR_kill_CUDA.exe</file_name>
<main_program/>
</file_ref>
<file_ref>
	<file_name>cudart.dll</file_name>
</file_ref>
<file_ref>
	<file_name>cufft.dll</file_name>
</file_ref>
<file_ref>
	<file_name>libfftw3f-3-1-1a_upx.dll</file_name>
</file_ref>
</app_version>
</app_info>


ID: 861633 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 861634 - Posted: 3 Feb 2009, 15:00:12 UTC - in response to Message 861618.  

Hi,

The gpu_file_lock is Zero bytes and timestamp is 01.02.2009 12:02 (?).... I cannot copy this file nor open it, so it's really locked :-)

I notice that the GPU WU is in the current scenario set to "Waiting to run"..... It's got a deadline of 13.02.200913.40:20, whilst the running ones have 13.02.2009 13:39:42 - they run on hight priority.

It seems the problem might lie in prioritization mechanism...?

When I restart boincmgr, the "Waiting to run" task is still waiting and now the GPU is fed one of the perviously CPU-fed WUs.

Morten


Ah... Sorry, only BOINC restart, indeed, in this situation...
As I already said, maybe in another thread, BOINC goes crazy in High priority mode... It can start dozen apps even if "leave app in memory" enabled... They all will hang in memory and it will start another and another ones...
If someone knows how to forbid BOINC from switching from not completed task to another one, please, post here.
BOINC devs made BOINC too clever, it thinks it know better what app should work and sometimes it leads to undesired effects...
ID: 861634 · Report as offensive
Profile Paul D Harris
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1122
Credit: 33,600,005
RAC: 0
United States
Message 861767 - Posted: 4 Feb 2009, 4:56:08 UTC - in response to Message 860191.  


How do you set up the cc_config.xml file for a core i7 chip?
I have one coming next week.


Interesting question indeed. It depends of how good AK v8 and AP will utilize HT.
In HT P4 days pair of opt app could perform worse than single opt app with HT disabled.

If app is highly optimized it tends to use all available hardware resources of core - there is too low amount of idle core parts that can benefit from HyperThreading.

So I consider it as open field for experimentation on this stage.

So do I set it for 4 cores+1 or 8 cors+1

I got my i7 up and crunching with 8+1 seems to be crunching right along.
ID: 861767 · Report as offensive
Profile Westsail and *Pyxey*
Volunteer tester
Avatar

Send message
Joined: 26 Jul 99
Posts: 338
Credit: 20,544,999
RAC: 0
United States
Message 861785 - Posted: 4 Feb 2009, 6:22:05 UTC

next one be started on GPU
"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, 'hmm... that's funny...'" -- Isaac Asimov
ID: 861785 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 861913 - Posted: 4 Feb 2009, 15:44:32 UTC - in response to Message 861634.  

If someone knows how to forbid BOINC from switching from not completed task to another one, please, post here.


The only way I have found to ease this situation is to run with a long switch interval, which can give rise to some oddities and does not cure all the issues as there are still internal events that can cause BOINC to ignore everything you have told it ... at the moment I run with a 12 hour (720 min) interval and for the most part tasks are run to completion before another is started.

One more point, this works better and better the more cores you have running. So, it is so-so on duals, fair on 4 core and good on 8 ... can't wait for a system that has more than 8 CPUs ...
ID: 861913 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 861962 - Posted: 4 Feb 2009, 18:45:34 UTC - in response to Message 861913.  

Version for SSE2 only CPUs added.
ID: 861962 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 862050 - Posted: 4 Feb 2009, 22:40:42 UTC - in response to Message 861962.  

SSSE3 variant updated in attempt to reschedule CPU-assigned SETI MB task if GPU sits idle.
Should decrease GPU idle time and increase system performance.
http://lunatics.kwsn.net/gpu-crunching/ak-v8-cuda-mb-team-work-mod.msg13778.html#msg13778
ID: 862050 · Report as offensive
Profile Ghery S. Pettit
Avatar

Send message
Joined: 7 Nov 99
Posts: 325
Credit: 28,109,066
RAC: 82
United States
Message 862164 - Posted: 5 Feb 2009, 3:07:35 UTC
Last modified: 5 Feb 2009, 3:15:21 UTC

After following this thread, I've now got 5 WU crunching at the same time. They all look the same on the task list. Any way to tell which one is running on the GPU? Tough to tell when one is running high priority and the other 4 are not. The one running on 6.08, rather than 5.28, perhaps?

Q6700 processor, 8800 GTS video card.

Even if the GPU goes the same speed at the Q6700, this is going to be a nice RAC bump for the machine.
ID: 862164 · Report as offensive
Profile Paul D Harris
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1122
Credit: 33,600,005
RAC: 0
United States
Message 862224 - Posted: 5 Feb 2009, 6:06:09 UTC
Last modified: 5 Feb 2009, 6:10:40 UTC

I have my i7 920 and 9500gt up and running with 6.4.5 boinc app and the cpus set to 9 in the cc_config.xml file and I got errors and the Wus started to get less and less until it was only doing 1 WU at a time instead of 9 WUs. So I reloaded BOINC 6.4.5 app and removed the Raistmer app and set the cpus to 9 in the cc_config.xml file and I started to do 9 WUs again. Bonic says I have 1 processor and not 4 or 8. If I set it to 9 cpus n the cc_config.xml file will it slow down my WUs or do I set it for 5 cpus in the cc_config.xml file or to 2 cpus in the cc_config.xml file?
ID: 862224 · Report as offensive
Grey Shadow
Volunteer tester
Avatar

Send message
Joined: 26 Nov 08
Posts: 41
Credit: 139,654
RAC: 0
Russia
Message 862228 - Posted: 5 Feb 2009, 6:44:47 UTC - in response to Message 862164.  
Last modified: 5 Feb 2009, 6:48:18 UTC

After following this thread, I've now got 5 WU crunching at the same time. They all look the same on the task list. Any way to tell which one is running on the GPU? Tough to tell when one is running high priority and the other 4 are not. The one running on 6.08, rather than 5.28, perhaps?

Q6700 processor, 8800 GTS video card.

Even if the GPU goes the same speed at the Q6700, this is going to be a nice RAC bump for the machine.

Look at Progress bar and CPU time in "Tasks" section of BOINC. You'll see that for 4 tasks CPU time increases each second, but for fifth one it grows very slowly despite good progress, and final CPU time is several times smaller than for other WUs. It means that this fifth WU is being crunched by GPU.

For example my PC needs about 50 minutes of CPU time to crunch standard 60-credit MB WU at CPU and only 4 minutes of CPU time to crunch it using GPU (but actually GPU-crunching consumes about 15 minutes).

Also when GPU-crunching starts you'll see that CPU time increases but progress bar is idle. It happens because GPU initiation requires about 30 seconds when CPU is active but WU crunching wasn't started yet (so no moves in progress bar).
ID: 862228 · Report as offensive
Profile Paul D Harris
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1122
Credit: 33,600,005
RAC: 0
United States
Message 862250 - Posted: 5 Feb 2009, 8:30:30 UTC - in response to Message 862228.  

After following this thread, I've now got 5 WU crunching at the same time. They all look the same on the task list. Any way to tell which one is running on the GPU? Tough to tell when one is running high priority and the other 4 are not. The one running on 6.08, rather than 5.28, perhaps?

Q6700 processor, 8800 GTS video card.

Even if the GPU goes the same speed at the Q6700, this is going to be a nice RAC bump for the machine.

Look at Progress bar and CPU time in "Tasks" section of BOINC. You'll see that for 4 tasks CPU time increases each second, but for fifth one it grows very slowly despite good progress, and final CPU time is several times smaller than for other WUs. It means that this fifth WU is being crunched by GPU.

For example my PC needs about 50 minutes of CPU time to crunch standard 60-credit MB WU at CPU and only 4 minutes of CPU time to crunch it using GPU (but actually GPU-crunching consumes about 15 minutes).

Also when GPU-crunching starts you'll see that CPU time increases but progress bar is idle. It happens because GPU initiation requires about 30 seconds when CPU is active but WU crunching wasn't started yet (so no moves in progress bar).


Thanks
I found out that to enable hyper threading on W2kAS I have to install the OS while the i7 chip is installed. I installed the OS when I had a celeron installed. So I have to re-install Window 2000 Advance Server in order for it to enable hyper threading with the i7 chip installed.
ID: 862250 · Report as offensive
Profile mr.kjellen
Volunteer tester
Avatar

Send message
Joined: 4 Jan 01
Posts: 195
Credit: 71,324,196
RAC: 0
Sweden
Message 862257 - Posted: 5 Feb 2009, 9:40:48 UTC

Is there still no way to enable multi GPU processing?
ID: 862257 · Report as offensive
Fred W
Volunteer tester

Send message
Joined: 13 Jun 99
Posts: 2524
Credit: 11,954,210
RAC: 0
United Kingdom
Message 862259 - Posted: 5 Feb 2009, 9:53:45 UTC - in response to Message 862257.  
Last modified: 5 Feb 2009, 9:55:15 UTC

Is there still no way to enable multi GPU processing?

Well, I finally got both halves of my GTX295 going, if that counts, so don't look over your shoulder in a couple of weeks :)

F.
Vista x64
Boinc 6.6.3
NVidia 185.20 (Beta)
ID: 862259 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 862260 - Posted: 5 Feb 2009, 9:54:54 UTC - in response to Message 862257.  
Last modified: 5 Feb 2009, 9:57:55 UTC

Is there still no way to enable multi GPU processing?

All is possible (in virtual world ;) ) but will require some amount of work.
If my hopes on BOINC itself will fail again maybe I will try to do multi-GPU support too (it's weird indeed - it's all BOINC functions, science app should not care about it at all. As soon as BOINC will support separate CPU and GPU apps for the same task set this "team" mod will be unneeded completely).

For now - either stock 6.08 or V7 of my mod.
ID: 862260 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 862261 - Posted: 5 Feb 2009, 9:56:20 UTC - in response to Message 862260.  
Last modified: 5 Feb 2009, 10:05:26 UTC

There is some situation when VLAR autokill will waste resources:

If VLAR task is already near completion on CPU but re-scheduling to GPU occurs, almost complete task will be killed by CUDA app autokill mod.

To avoid such situation VLAR task will be not killed but processed on GPU if it was started before (if it has non-zero progress already). If it's fresh VLAR task it will be aborted as before.
That way CPU "investments" in task will be saved.

Update to CUDA app attached: http://lunatics.kwsn.net/gpu-crunching/ak-v8-cuda-mb-team-work-mod.msg13789.html#msg13789
This update appropriate to any SSE level of CPU.


ADDON: you can see if it works by this line in stderr:
"VLAR WU (AR: xxxxx )detected, but task partially done already, continuing computations"
ID: 862261 · Report as offensive
Profile mr.kjellen
Volunteer tester
Avatar

Send message
Joined: 4 Jan 01
Posts: 195
Credit: 71,324,196
RAC: 0
Sweden
Message 862265 - Posted: 5 Feb 2009, 10:32:49 UTC - in response to Message 862259.  

Is there still no way to enable multi GPU processing?

Well, I finally got both halves of my GTX295 going, if that counts, so don't look over your shoulder in a couple of weeks :)

F.
Vista x64
Boinc 6.6.3
NVidia 185.20 (Beta)


OT

Just got the first GTX295 (out of two) installed...my neck should be just fine
*evil grin*

Please PM How-to if you feel like it...
/Anton
ID: 862265 · Report as offensive
Previous · 1 . . . 3 · 4 · 5 · 6 · 7 · 8 · 9 . . . 12 · Next

Message boards : Number crunching : AK V8 + CUDA MB team work mod


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.