SETI@home v8.22 Windows GPU applications support thread

Message boards : Number crunching : SETI@home v8.22 Windows GPU applications support thread
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 16 · Next

AuthorMessage
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918113 - Posted: 10 Feb 2018, 20:56:44 UTC

I noticed the thread here about the latest NVidia drivers changing some OpenCL stuff so I went back to 390.65 and let it try again - locked up again.
ID: 1918113 · Report as offensive     Reply Quote
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6005
Credit: 83,627,969
RAC: 24,310
Russia
Message 1918128 - Posted: 10 Feb 2018, 21:31:55 UTC - in response to Message 1918061.  

Put that task into offline bench and re-run - will it lock?
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1918128 · Report as offensive     Reply Quote
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 4650
Credit: 293,003,425
RAC: 479,084
United States
Message 1918140 - Posted: 10 Feb 2018, 22:12:12 UTC - in response to Message 1918113.  

Shaggie, do you run Process Lasso as well on your machines?
ID: 1918140 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918147 - Posted: 10 Feb 2018, 22:31:46 UTC - in response to Message 1918140.  

Shaggie, do you run Process Lasso as well on your machines?


I do not.
ID: 1918147 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918149 - Posted: 10 Feb 2018, 22:36:57 UTC - in response to Message 1918128.  

Put that task into offline bench and re-run - will it lock?


I set it up and ran it through once -- no lockup -- but in doing so I noticed the checkpoint file and wondered if that might be to blame. Can I bench 'from the checkpoint somehow? I'm not sure how to do this in the bench but I'll fiddle around a bit.
ID: 1918149 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 10189
Credit: 136,241,585
RAC: 87,985
Australia
Message 1918157 - Posted: 10 Feb 2018, 23:41:11 UTC - in response to Message 1918061.  

Full disclosure: this is a 1080 Ti running in an eGPU enclosure over Thunderbolt 3

So there's a whole other kettle of fish thrown in to the mix.
Does it lock if the Video card is physically in that machine? (ie take the Thunderbolt connection out of the mix. I've seen several reviews where certain benchmarks won't complete on certain systems when running GPUs in external enclosures over Thunderbolt, yet will run on a different motherboard. Various hardware/BIOS/driver incompatibilities between different motherboards & external units).
Grant
Darwin NT
ID: 1918157 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918171 - Posted: 11 Feb 2018, 0:46:18 UTC - in response to Message 1918157.  

Full disclosure: this is a 1080 Ti running in an eGPU enclosure over Thunderbolt 3

So there's a whole other kettle of fish thrown in to the mix.
Does it lock if the Video card is physically in that machine? (ie take the Thunderbolt connection out of the mix. I've seen several reviews where certain benchmarks won't complete on certain systems when running GPUs in external enclosures over Thunderbolt, yet will run on a different motherboard. Various hardware/BIOS/driver incompatibilities between different motherboards & external units).


It's a laptop; I won't bother with eGPU otherwise.

-g
ID: 1918171 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 10189
Credit: 136,241,585
RAC: 87,985
Australia
Message 1918172 - Posted: 11 Feb 2018, 0:50:44 UTC - in response to Message 1918171.  
Last modified: 11 Feb 2018, 0:51:24 UTC

It's a laptop; I won't bother with eGPU otherwise.

Any other system you can run the eGPU on?
If it runs OK on all other systems & GPUs, then that just leaves Thunderbolt as the issue.
Be it drivers or hardware.
Any updated Thunderbolt drivers/BIOS for the motherboard or the eGPU unit about?
Grant
Darwin NT
ID: 1918172 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918206 - Posted: 11 Feb 2018, 3:59:33 UTC - in response to Message 1918172.  

It's a laptop; I won't bother with eGPU otherwise.

Any other system you can run the eGPU on?
If it runs OK on all other systems & GPUs, then that just leaves Thunderbolt as the issue.
Be it drivers or hardware.
Any updated Thunderbolt drivers/BIOS for the motherboard or the eGPU unit about?


Well I have 3 matching cards in another machine -- I'm not really interested in moving hardware around because, like I said, the card runs 3D apps just fine -- it's been running our game for 4 hours now, it's cleared over a hundred SETI work units, and the same workunit runs start to finish in the benchmark.

And yes, I checked drivers.

I hacked the XML to clear the checkpoint but still locked up after-- I must not have done it right. I'm not sure if there's a proper way to do that.
ID: 1918206 · Report as offensive     Reply Quote
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6005
Credit: 83,627,969
RAC: 24,310
Russia
Message 1918224 - Posted: 11 Feb 2018, 6:30:33 UTC - in response to Message 1918206.  

simple deletion of state.sah in slot should do.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1918224 · Report as offensive     Reply Quote
Stephen "Heretic" Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 3638
Credit: 80,173,334
RAC: 164,927
Australia
Message 1918236 - Posted: 11 Feb 2018, 10:44:56 UTC - in response to Message 1918206.  


I hacked the XML to clear the checkpoint but still locked up after-- I must not have done it right. I'm not sure if there's a proper way to do that.


. . Try just setting the minimum checkpoint time longer than the task run time. That should avoid checkpointing.

Stephen

:)
ID: 1918236 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918278 - Posted: 11 Feb 2018, 16:35:14 UTC - in response to Message 1918224.  

simple deletion of state.sah in slot should do.


I couldn't find that but I removed
<active_task_set>
<active_task>
...

from client_state
I also deleted the partial result file from the app directory. I will try again.

If this doesn't work the other thing I can try is replicating the use_sleep option to the offline benchmark.
ID: 1918278 · Report as offensive     Reply Quote
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6005
Credit: 83,627,969
RAC: 24,310
Russia
Message 1918293 - Posted: 11 Feb 2018, 17:47:03 UTC - in response to Message 1918278.  
Last modified: 11 Feb 2018, 17:47:47 UTC

Look harder

https://clip2net.com/s/3ROBaAH
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1918293 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918364 - Posted: 11 Feb 2018, 22:30:02 UTC - in response to Message 1918293.  

Ok, It wasn't in the BOINC or the project directory but ProgramData\BOINC\slots\0\state.sah

I will kill that and try again in a minute. I should also note that I've tried running the task 3x back to back in the bench by duplicating the WU and no problems there.
ID: 1918364 · Report as offensive     Reply Quote
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 264
Credit: 150,723,871
RAC: 223,800
Canada
Message 1918372 - Posted: 11 Feb 2018, 22:48:21 UTC

Seemed like we had a winner but then lockup at 95.38%. After restart it resumed from a CP from around 91% and locked up again shortly afterward.

I'm growing uncomfortable with so many hard shutdowns on this system -- I'm over 10 now and I'm worried about FS corruption on a machine I need to work on. I've run it 5 or 6 times in the benchmark but in BOINC it always brings down my machine.

If you aren't really interested in anything diagnostic I'll abort this task and see if any others cause problems.
ID: 1918372 · Report as offensive     Reply Quote
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6005
Credit: 83,627,969
RAC: 24,310
Russia
Message 1918507 - Posted: 12 Feb 2018, 15:01:46 UTC - in response to Message 1918372.  


If you aren't really interested in anything diagnostic I'll abort this task and see if any others cause problems.


Well, app behavior should be the same being run under BOINC or not, provided you supplied same command line switches.
Absence of lockups in offline run tells that there could be some interference fro boinc per se. Worth to consult with BOINC devs/boards if BOINC itself communicates with GPU while scientific app running.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1918507 · Report as offensive     Reply Quote
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 369
Credit: 1,413,694
RAC: 1,523
Finland
Message 1918530 - Posted: 12 Feb 2018, 17:10:43 UTC - in response to Message 1918507.  

Worth to consult with BOINC devs/boards if BOINC itself communicates with GPU while scientific app running.


It doesn't.
ID: 1918530 · Report as offensive     Reply Quote
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6005
Credit: 83,627,969
RAC: 24,310
Russia
Message 1918539 - Posted: 12 Feb 2018, 18:36:08 UTC - in response to Message 1918530.  

Worth to consult with BOINC devs/boards if BOINC itself communicates with GPU while scientific app running.


It doesn't.

How it allocated OpenCL ID then? Exclusively via linked to app lib part?
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1918539 · Report as offensive     Reply Quote
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 369
Credit: 1,413,694
RAC: 1,523
Finland
Message 1920279 - Posted: 21 Feb 2018, 19:17:15 UTC - in response to Message 1918539.  

I'm not sure if I understand your question.

When the client starts up it queries drivers for how many devices there are. After that it doesn't touch drivers again. It knows which GPUs are in use by simply keeping track of which GPUs it has assigned tasks to.

If you mean boinc_get_opencl_ids() it queries drivers to get available platform and device ids and then returns ones matching requested device. Apart from driver queries its code runs entirely in science app.
ID: 1920279 · Report as offensive     Reply Quote
rudabega Project Donor

Send message
Joined: 14 May 02
Posts: 17
Credit: 5,612,163
RAC: 0
United States
Message 1921466 - Posted: 26 Feb 2018, 23:52:26 UTC

Hello,

I am having an issue with an AMD/ATI Radeon V5900 GPU in one of my machines. I recently updated the driver as I had noticed that BOINC was not recognizing the GPU. When I checked the driver, it had never been updated. Therefore, I checked the AMD website for this card and found the most recent driver for Windows 7, date 02/15/2017. I updated the drive with this one: https://support.amd.com/en-us/download/workstation/previous/graphics?os=Windows%207%20-%2064&rev=15.201.2401.1009. Once installed, I restarted BOINC and it recognized the GPU and was able to download WUs. However, it appears that any WUs that it tries to process end in errors. I have attached it to two separate projects (Einstein and Seti) and both end with errors. In most cases, both process a WU for about 25 seconds and then say it is done and move on to the next one. I detached Einstein as I had heard about a minimum RAM requirement and I may not have enough to process these, but I thought Seti would work as the identical computer I have processes Seti WUs just fine.

Here is a WU from the computer (C1) that errors out: https://setiathome.berkeley.edu/result.php?resultid=6438007814
Here is a WU from the computer (C2) that processes just fine: https://setiathome.berkeley.edu/result.php?resultid=6415146737

Following is first few lines from the BOINC Event Log for information:

2/26/2018 12:25:44 PM | | cc_config.xml not found - using defaults
2/26/2018 12:25:44 PM | | Starting BOINC client version 7.8.3 for windows_x86_64
2/26/2018 12:25:44 PM | | log flags: file_xfer, sched_ops, task
2/26/2018 12:25:44 PM | | Libraries: libcurl/7.47.1 OpenSSL/1.0.2g zlib/1.2.8
2/26/2018 12:25:44 PM | | Data directory: C:\ProgramData\BOINC
2/26/2018 12:25:44 PM | | Running under account hclsysadmin
2/26/2018 12:25:44 PM | | CAL: ATI GPU 0: AMD Radeon HD 6900 series (Cayman) (CAL version 1.4.1848, 2048MB, 2016MB available, 1536 GFLOPS peak)
2/26/2018 12:25:44 PM | | OpenCL: AMD/ATI GPU 0: AMD Radeon HD 6900 series (Cayman) (driver version 1800.12 (VM), device version OpenCL 1.2 AMD-APP (1800.12), 2048MB, 2016MB available, 1536 GFLOPS peak)
2/26/2018 12:25:44 PM | | Processor: 4 GenuineIntel Intel(R) Xeon(R) CPU W3530 @ 2.80GHz [Family 6 Model 26 Stepping 5]
2/26/2018 12:25:44 PM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt syscall nx lm vmx tm2 dca pbe
2/26/2018 12:25:44 PM | | OS: Microsoft Windows 7: Professional x64 Edition, Service Pack 1, (06.01.7601.00)
2/26/2018 12:25:44 PM | | Memory: 12.00 GB physical, 23.99 GB virtual
2/26/2018 12:25:44 PM | | Disk: 446.36 GB total, 162.37 GB free
2/26/2018 12:25:44 PM | | Local time is UTC -7 hours
2/26/2018 12:25:44 PM | | VirtualBox version: 5.1.26
2/26/2018 12:25:44 PM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 8456249; resource share 200
2/26/2018 12:25:44 PM | | General prefs: from https://www.grcpool.com/ (last modified ---)
2/26/2018 12:25:44 PM | | Host location: none
2/26/2018 12:25:44 PM | | General prefs: using your defaults
2/26/2018 12:25:44 PM | | Reading preferences override file
2/26/2018 12:25:44 PM | | Preferences:
2/26/2018 12:25:44 PM | | max memory usage when active: 6142.78 MB
2/26/2018 12:25:44 PM | | max memory usage when idle: 11057.00 MB
2/26/2018 12:25:44 PM | | max disk usage: 162.56 GB
2/26/2018 12:25:44 PM | | max CPUs used: 3
2/26/2018 12:25:44 PM | | don't compute while active
2/26/2018 12:25:44 PM | | don't use GPU while active
2/26/2018 12:25:44 PM | | suspend work if non-BOINC CPU load exceeds 25%
2/26/2018 12:25:44 PM | | (to change preferences, visit a project web site or select Preferences in the Manager)
2/26/2018 12:25:44 PM | | Using account manager grcpool.com
2/26/2018 12:25:44 PM | | Suspending computation - computer is in use
2/26/2018 12:26:53 PM | | Suspending GPU computation - computer is in use
2/26/2018 12:26:54 PM | | Resuming GPU computation
2/26/2018 12:27:30 PM | SETI@home | task postponed 600.000000 sec: Waiting to acquire lock
2/26/2018 12:27:30 PM | SETI@home | Starting task blc13_2bit_guppi_58137_40062_HIP64048_0039.28299.818.22.45.39.vlar_0
2/26/2018 12:27:59 PM | SETI@home | Computation for task blc13_2bit_guppi_58137_40062_HIP64048_0039.28299.818.22.45.39.vlar_0 finished
2/26/2018 12:27:59 PM | SETI@home | Sending scheduler request: To fetch work.
2/26/2018 12:27:59 PM | SETI@home | Requesting new tasks for AMD/ATI GPU
2/26/2018 12:28:01 PM | SETI@home | Started upload of blc13_2bit_guppi_58137_40062_HIP64048_0039.28299.818.22.45.39.vlar_0_r1422422241_0
2/26/2018 12:28:03 PM | SETI@home | Scheduler request completed: got 0 new tasks
2/26/2018 12:28:03 PM | SETI@home | Not sending work - last request too recent: 241 sec
2/26/2018 12:28:06 PM | SETI@home | Finished upload of blc13_2bit_guppi_58137_40062_HIP64048_0039.28299.818.22.45.39.vlar_0_r1422422241_0


You can see that on the eight line from the bottom, a task was started and 29 seconds later it was done. This is not the task I noted above, but I am sure will deliver similar results.


Following is the output from the CLInfo program:

Number of platforms: 1
Platform Profile: FULL_PROFILE
Platform Version: OpenCL 2.0 AMD-APP (1800.12)
Platform Name: AMD Accelerated Parallel Processing
Platform Vendor: Advanced Micro Devices, Inc.
Platform Extensions: cl_khr_icd cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_event_callback cl_amd_offline_devices


Platform Name: AMD Accelerated Parallel Processing
Number of devices: 2
Device Type: CL_DEVICE_TYPE_GPU
Device ID: 4098
Max compute units: 8
Max work items dimensions: 3
Max work items[0]: 256
Max work items[1]: 256
Max work items[2]: 256
Max work group size: 256
Preferred vector width char: 16
Preferred vector width short: 8
Preferred vector width int: 4
Preferred vector width long: 2
Preferred vector width float: 4
Preferred vector width double: 2
Max clock frequency: 600Mhz
Address bits: 32
Max memory allocation: 536870912
Image support: Yes
Max number of images read arguments: 128
Max number of images write arguments: 8
Max image 2D width: 16384
Max image 2D height: 16384
Max image 3D width: 2048
Max image 3D height: 2048
Max image 3D depth: 2048
Max samplers within kernel: 16
Max size of kernel argument: 1024
Alignment (bits) of base address: 2048
Minimum alignment (bytes) for any datatype: 128
Single precision floating point capability
Denorms: No
Quiet NaNs: Yes
Round to nearest even: Yes
Round to zero: Yes
Round to +ve and infinity: Yes
IEEE754-2008 fused multiply-add: Yes
Cache type: None
Cache line size: 0
Cache size: 0
Global memory size: 2147483648
Constant buffer size: 65536
Max number of constant args: 8
Local memory type: Scratchpad
Local memory size: 32768
Error correction support: 0
Profiling timer resolution: 1
Device endianess: Little
Available: Yes
Compiler available: Yes
Execution capabilities:
Execute OpenCL kernels: Yes
Execute native function: No
Queue properties:
Out-of-Order: No
Profiling : Yes
Platform ID: 5607E8C0
Name: Cayman
Vendor: Advanced Micro Devices, Inc.
Driver version: 1800.12 (VM)
Profile: FULL_PROFILE
Version: OpenCL 1.2 AMD-APP (1800.12)
Extensions: cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_atomic_counters_32 cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_image2d_from_buffer_read_only cl_khr_spir cl_khr_gl_event


Device Type: CL_DEVICE_TYPE_CPU
Device ID: 4098
Max compute units: 4
Max work items dimensions: 3
Max work items[0]: 1024
Max work items[1]: 1024
Max work items[2]: 1024
Max work group size: 1024
Preferred vector width char: 16
Preferred vector width short: 8
Preferred vector width int: 4
Preferred vector width long: 2
Preferred vector width float: 4
Preferred vector width double: 2
Max clock frequency: 2800Mhz
Address bits: 32
Max memory allocation: 1073741824
Image support: Yes
Max number of images read arguments: 128
Max number of images write arguments: 64
Max image 2D width: 8192
Max image 2D height: 8192
Max image 3D width: 2048
Max image 3D height: 2048
Max image 3D depth: 2048
Max samplers within kernel: 16
Max size of kernel argument: 4096
Alignment (bits) of base address: 1024
Minimum alignment (bytes) for any datatype: 128
Single precision floating point capability
Denorms: Yes
Quiet NaNs: Yes
Round to nearest even: Yes
Round to zero: Yes
Round to +ve and infinity: Yes
IEEE754-2008 fused multiply-add: Yes
Cache type: Read/Write
Cache line size: 64
Cache size: 32768
Global memory size: 2147483648
Constant buffer size: 65536
Max number of constant args: 8
Local memory type: Global
Local memory size: 32768
Error correction support: 0
Profiling timer resolution: 365
Device endianess: Little
Available: Yes
Compiler available: Yes
Execution capabilities:
Execute OpenCL kernels: Yes
Execute native function: Yes
Queue properties:
Out-of-Order: No
Profiling : Yes
Platform ID: 5607E8C0
Name: Intel(R) Xeon(R) CPU W3530 @ 2.80GHz
Vendor: GenuineIntel
Driver version: 1800.12 (sse2)
Profile: FULL_PROFILE
Version: OpenCL 1.2 AMD-APP (1800.12)
Extensions: cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_device_fission cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_d3d10_sharing cl_khr_spir cl_khr_gl_event

Computers C1 and C2 are nearly identical in that they were both purchased at the same time and contained the exact same hardware. The only difference now is that the hard drives were updated to SSDs and the SSD manufacturer's are different. I performed the same upgrade on C2's GPU driver as C1. It too had issues with Einstein GPU WUs, but I switched to Seti GPU WUs and no issues since.

If anyone has any ideas as to what the issue might be I would be very interested to know. If additional information is required, just ask an d I will get it.

Sincerely,

Rudabega
ID: 1921466 · Report as offensive     Reply Quote
Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 16 · Next

Message boards : Number crunching : SETI@home v8.22 Windows GPU applications support thread


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.