GTX780 Ti system

Message boards : Number crunching : GTX780 Ti system
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1848491 - Posted: 13 Feb 2017, 9:35:08 UTC - in response to Message 1848368.  

I would suggest that you not do that at this time. I've burned out 2 780Tis a while back by increasing the number of instances of work on them. They held out for a while but no one likes that smell of burnt electronics when they come home.

As you get more into Seti, you will start to learn more of the things we do to increase our productivity. Feel free to ask questions.


Thank you ! I was just thinking a GTX 780Ti Dual Classified should be able to do more...
A question : Is the file mb_cmdline-8.22_intel_opencl_nvidia_SoG.txt (in projects dir.) generated by the application ? Should it be empty ?
Humans may rule the world...but bacteria run it...
ID: 1848491 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 1848493 - Posted: 13 Feb 2017, 10:38:53 UTC
Last modified: 13 Feb 2017, 10:45:59 UTC

A question : Is the file mb_cmdline-8.22_intel_opencl_nvidia_SoG.txt (in projects dir.) generated by the application ? Should it be empty ?


It is part of the OpenCL_SoG package and yes it should be empty by default.
For optimisation users can add app args to fine tune the application.

I was just thinking a GTX 780Ti Dual Classified should be able to do more...


It certainy is.
If your temps are O.K. i would suggest to consider installing Lunatics optimised apps and running 2 instances on the GPU.


With each crime and every kindness we birth our future.
ID: 1848493 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1848495 - Posted: 13 Feb 2017, 10:45:30 UTC - in response to Message 1848491.  
Last modified: 13 Feb 2017, 10:47:46 UTC

It is not auto generated, but sent to you by the server to place customized command line parameters that you want to use. You may or not also have an app_config.xml file as well with you would use to run more than 1 task at a time.

For my 750Ti I use:
-sbs 256 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64 -hp
for the command line, 1 task at a time.

EDIT: I don't feel that running 2 tasks on my card is an advantage since it ties up 2 cores for very little GPU increase.
ID: 1848495 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1848500 - Posted: 13 Feb 2017, 11:19:27 UTC - in response to Message 1848493.  

It is part of the OpenCL_SoG package and yes it should be empty by default.
For optimisation users can add app args to fine tune the application.

Understood...

Right after the time I was able to take the SoG application off of my Firewall blacklist, a stderr.txt file was created.
It looks like this :
09:24:17 (1964): Can't open init data file - running in standalone mode
09:24:17 (1964): Can't open init data file - running in standalone mode
Not using mb_cmdline.txt-file, using commandline options.
Priority of worker thread raised successfully
Priority of process adjusted successfully, below normal priority class used
09:24:17 (1964): Can't open init data file - running in standalone mode
WARNING: init_data.xml missing
OpenCL platform detected: Advanced Micro Devices, Inc.
OpenCL platform detected: NVIDIA Corporation
WARNING: BOINC supplied wrong platform!
BOINC assigns device 0
WARNING: BOINC failed to provide OpenCL device, using own enumeration abilities
SETI@home error -5 Can't open file
(work_unit.sah) in read_wu_state() errno=2

File: ..\worker.cpp
Line: 136

Would this have something to do with the SoG application not running ? I don't have the knowledge to understand what this actually means...
Is a project reset in order ?
Humans may rule the world...but bacteria run it...
ID: 1848500 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 1848501 - Posted: 13 Feb 2017, 11:24:51 UTC

SETI@home error -5 Can't open file


It seems its still blocked somewhere.


With each crime and every kindness we birth our future.
ID: 1848501 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1848502 - Posted: 13 Feb 2017, 11:36:55 UTC - in response to Message 1848501.  

For SoG to run, you need, an EXE file with SoG in it's name, and a MultiBeam CL file with the same r#### as the EXE file.

You might want to restart BOINIC and watch for errors in ~ the first 30 lines and post them here so that we can see it. BOINC 'should' complain if needed files are missing and try to get them from the server if/when needed (unless you manually installed the SETI apps).

It sure seems like you are stuck on CUDA42 at the moment.
ID: 1848502 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1848503 - Posted: 13 Feb 2017, 11:43:59 UTC

Wait a minute,
Did you just whitelist the EXE file in your anti virus?

The EXE will create WISDOM and BIN files that it creates specific to your card before it runs. If those are getting blocked by your AV it will not run. Whitelist the entire directory I.E. *.*
ID: 1848503 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1848511 - Posted: 13 Feb 2017, 12:57:07 UTC - in response to Message 1848503.  

Wait a minute,
Did you just whitelist the EXE file in your anti virus?

The EXE will create WISDOM and BIN files that it creates specific to your card before it runs. If those are getting blocked by your AV it will not run. Whitelist the entire directory I.E. *.*

I whitelisted the whole project directory on Feb.11. The downloaded SoG units errored out long before that, of course. The quoted stderr.txt appeared AFTER I whitelisted the project diretory. HTH...
Humans may rule the world...but bacteria run it...
ID: 1848511 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1848512 - Posted: 13 Feb 2017, 13:13:57 UTC - in response to Message 1848502.  

You might want to restart BOINIC and watch for errors in ~ the first 30 lines and post them here so that we can see it.

Will do...any flags to enable apart from the default ones ?
Humans may rule the world...but bacteria run it...
ID: 1848512 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1848875 - Posted: 15 Feb 2017, 19:00:32 UTC

Just a small update :
I did a project reset when the servers came online after yesterday's outage. No WUs for the SoG application so far, so the files for it are absent.
For some reason, crunching times have gone up significantly...dunno if this is coincidence or not. GPU load is down and CPU load is up, can't explain why.
Humans may rule the world...but bacteria run it...
ID: 1848875 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1851019 - Posted: 24 Feb 2017, 17:42:22 UTC

Chose to post here instead of making a new thread, hope that's ok.

In app_config, I initially set my GPU to run 2 tasks on cuda42, by setting :
<app_config>
<app_version>
<app_name>setiathome_v8</app_name>
<plan_class>cuda42</plan_class>
<avg_ncpus>0.2</avg_ncpus>
<ngpus>0.5</ngpus>
</app_version>
</app_config>

That worked just fine. But then the SoG-app did not run any tasks until I changed it back to 1.0 GPUs and chewed through the cuda42's. So I added a section in the app_config for the SoG-app, so it looks like this :
<app_config>
<app>
<name>setiathome_v8</name>
<gpu_versions>
<gpu_usage>1.0</gpu_usage>
</gpu_versions>
</app>
<app_version>
<app_name>setiathome_v8</app_name>
<plan_class>opencl_nvidia_SoG</plan_class>
<ngpus>1.0</ngpus>
</app_version>
<app>
<name>setiathome_v8</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
</gpu_versions>
</app>
<app_version>
<app_name>setiathome_v8</app_name>
<plan_class>cuda42</plan_class>
<ngpus>0.5</ngpus>
</app_version>
</app_config>

Will Boinc now be able to run both apps in normal order (mostly FIFO, I guess), without me having to interfere...or have I made a mess of it ? Apps are stock, and I'm running 75% of CPU and 75% of CPU time.
Humans may rule the world...but bacteria run it...
ID: 1851019 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1851023 - Posted: 24 Feb 2017, 17:51:30 UTC - in response to Message 1851019.  

This might work better for you
<app_config>
	<app>
	<name>setiathome_v8</name>
	<gpu_versions>
		<gpu_usage>0.5</gpu_usage>
		<cpu_usage>1.0</cpu_usage>
	</gpu_versions>
	</app>

	<app>
	<name>astropulse_v7</name>
	<gpu_versions>
		<gpu_usage>1</gpu_usage>
		<cpu_usage>1.0</cpu_usage>
	</gpu_versions>
	</app>
</app_config>

Nice and simple and will run them both FIFO
ID: 1851023 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1851029 - Posted: 24 Feb 2017, 18:11:38 UTC - in response to Message 1851023.  

This might work better for you
---
Nice and simple and will run them both FIFO


Thank you :) Could you explain a little on how this would work ? I'm mostly blank when it comes to AP....only ever had 1 of those. Does it mean the GPU will run 2 cuda42-tasks OR 1 SoG-task ?
Humans may rule the world...but bacteria run it...
ID: 1851029 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1851031 - Posted: 24 Feb 2017, 18:29:27 UTC - in response to Message 1851029.  

It doesn't specify cuda/SoG so it will run either
0.5 GPU = Use 1/2 the GPU per task, so 2 tasks per card.
1.0 CPU = reserve (shutdown) 1 core per GPU task running, so 2 cores which you will need if running SoG

BTW, did you ever get SOG tasks to start downloading again, or still stuck on cuda 42?
ID: 1851031 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1851039 - Posted: 24 Feb 2017, 19:05:36 UTC - in response to Message 1851031.  

It doesn't specify cuda/SoG so it will run either
0.5 GPU = Use 1/2 the GPU per task, so 2 tasks per card.
1.0 CPU = reserve (shutdown) 1 core per GPU task running, so 2 cores which you will need if running SoG

I understand. What I wanted, was to run either 2 cuda42-tasks OR 1 SoG-task. That way, I can keep the 6 CPU tasks running concurrently. At least, that was the idea behind the config I posted earlier. Now Boinc hasn't touched any of the available SoG tasks yet...it chugs away at 2 cuda tasks at a time besides the 6 CPU tasks. Maybe this is the wrong way to use my resources ?

BTW, did you ever get SOG tasks to start downloading again, or still stuck on cuda 42?

Yes, got some SoG tasks very early this morning (and again some this afternoon), and they ran fine as described above (as a byproduct of changing the setting for cuda42 to 1GPU).
Humans may rule the world...but bacteria run it...
ID: 1851039 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1851068 - Posted: 24 Feb 2017, 21:38:28 UTC

The suggested configuration cut back to 4 CPU tasks, as expected. In addition, it will run 2 cuda tasks, OR 1 cuda task and 1 SoG task. The GPU load gets a bit high when using 1.5 GPU (?) as in 1 cuda and 1 SoG task concurrently.
To me, that looks like a setback, when compared to 6 CPU tasks and 2 cuda tasks (OR 6 CPU tasks and 1 SoG task). Running times seem pretty much the same as before, but of course it is hard to say with any certainty without lots more data.
I feel I should be able to have my cake and it eat it too, I guess (i.e. 6 CPU tasks, plus 2 cuda tasks OR 1 SoG task).
Humans may rule the world...but bacteria run it...
ID: 1851068 · Report as offensive
Profile Michel Makhlouta
Volunteer tester
Avatar

Send message
Joined: 21 Dec 03
Posts: 169
Credit: 41,799,743
RAC: 0
Lebanon
Message 1851185 - Posted: 25 Feb 2017, 9:13:52 UTC

I was running 3 cuda 42/50 on my 780's back in the day when I still had them. I used gpu-z to check the gpu load and they were nowhere near 100% on 1 WU.
ID: 1851185 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22205
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1851187 - Posted: 25 Feb 2017, 9:32:35 UTC

Running with 1.5 GPU per task will not result in more than 1 GPU being used for a task - the applications is not built to handle the required inter-GPU data and process transfers.
SoG presents a very "lumpy" load on both the CPU and GPU, thus it is best to be a little conservative when deciding how many tasks to run at a time. Your assumption of 2 per GPU may be OK, given your CPU with 1 core (thread) free for each GPU tasks actually being run.

The current applications are much more demanding on the GPU than the v7 applications were, thus the days of 3 or more tasks per GPU have passed. Even my GTX1080 struggled with v8 tasks running, but it was very comfortable running 3 tasks with the v7 applications.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1851187 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1851188 - Posted: 25 Feb 2017, 9:40:29 UTC - in response to Message 1851185.  

I was running 3 cuda 42/50 on my 780's back in the day when I still had them. I used gpu-z to check the gpu load and they were nowhere near 100% on 1 WU.


Yes, using GPU-Z to keep an eye on the numbers. Running 2 cuda tasks doesn't stress GPU load, temperature or memory controller. The SoG app is much more taxing. My card hasn't worked this hard since I bought it over 3 years ago.. ;-)
Humans may rule the world...but bacteria run it...
ID: 1851188 · Report as offensive
Ghia
Avatar

Send message
Joined: 7 Feb 17
Posts: 238
Credit: 28,911,438
RAC: 50
Norway
Message 1851192 - Posted: 25 Feb 2017, 9:52:07 UTC - in response to Message 1851187.  

Running with 1.5 GPU per task will not result in more than 1 GPU being used for a task - the applications is not built to handle the required inter-GPU data and process transfers.
SoG presents a very "lumpy" load on both the CPU and GPU, thus it is best to be a little conservative when deciding how many tasks to run at a time. Your assumption of 2 per GPU may be OK, given your CPU with 1 core (thread) free for each GPU tasks actually being run.

That was mye guess too...the 1.5 GPU being a "false" number. Observations here indicate that both CPU and GPU are okay with 2 cuda tasks. I haven't tried 2 SoG tasks, and probably won't anytime soon.
The current applications are much more demanding on the GPU than the v7 applications were, thus the days of 3 or more tasks per GPU have passed. Even my GTX1080 struggled with v8 tasks running, but it was very comfortable running 3 tasks with the v7 applications.

That's interesting...does it mean that the v8 applications have higher demands on accuracy ? This Seti interest has seen the upgrade monster raise its ugly head here....maybe it is time to build a new rig ;-)
Humans may rule the world...but bacteria run it...
ID: 1851192 · Report as offensive
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : GTX780 Ti system


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.