Einstein@Home Locks PC

Message boards : Number crunching : Einstein@Home Locks PC
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Bob Bainbridge

Send message
Joined: 21 Mar 01
Posts: 63
Credit: 128,919,333
RAC: 82
United States
Message 1604376 - Posted: 23 Nov 2014, 3:50:43 UTC

I decided to start crunching Einstein@Home WU's again during this draught of Seti WU's. I have no problem on 3 machines but my Intel 970 machine running Win 7 Ultimate keeps hanging. The clock and mouse freeze and I have to do a Hardware Reset. With Hyperthreading I'm running 12 cpu's and a NVDIA GTX 770 GPU. I tried reinstalling BOINC and restarting the project with no change.

Bob B.
ID: 1604376 · Report as offensive
zombie67 [MM]
Volunteer tester
Avatar

Send message
Joined: 22 Apr 04
Posts: 758
Credit: 27,771,894
RAC: 0
United States
Message 1604385 - Posted: 23 Nov 2014, 4:13:53 UTC

How many tasks at a time on your GPU? FWIW, each GPU task requires 300-350mb of GPU RAM. So for example, if you have 1gb of GPU RAM, You should limit it to just 2 or maybe 3 tasks at a time on your GPU.

Aside from that, The things I would look into:

1) Perhaps Einstein runs your CPU hotter, and your cooling is not up to the task. Or your thermal connection to the heatsink is poor.

2) Perhaps Einstein uses more RAM, and one of your DIMMs is marginal. With the extra usage, it is exposing the issue.
Dublin, California
Team: SETI.USA
ID: 1604385 · Report as offensive
Profile JakeTheDog
Avatar

Send message
Joined: 3 Nov 13
Posts: 153
Credit: 2,585,912
RAC: 0
United States
Message 1604485 - Posted: 23 Nov 2014, 10:26:54 UTC

Don't know if it's related to your problem, but I've started adding other projects to my BOINC manager these past couple weeks and have had performance issues. I've read that a lot of BOINC projects have tasks that don't run well with nVidia cards with the newest drivers. I've had to disable GPU for these projects, because I'd get like 5 FPS on the screen when using basic apps like browsers and word processors. The solution was to revert to an older driver version or to disable GPU.
ID: 1604485 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1604491 - Posted: 23 Nov 2014, 11:16:52 UTC - in response to Message 1604485.  

Don't know if it's related to your problem, but I've started adding other projects to my BOINC manager these past couple weeks and have had performance issues. I've read that a lot of BOINC projects have tasks that don't run well with nVidia cards with the newest drivers. I've had to disable GPU for these projects, because I'd get like 5 FPS on the screen when using basic apps like browsers and word processors. The solution was to revert to an older driver version or to disable GPU.

Unless those "lot of BOINC projects" provide some technical details of how they program their nVidia cards, and in what respect they "don't run well" - whether that applies to all cards, or just the older ones - you can't really make a proper decision on how to mitigate any possible problem in each user's particular case.

It seems to depend a great deal on whether the project programs using the 'CUDA' platform or the 'OpenCL' platform - that much information, at least, should be available. Depending on the programming decisions made by the project (and the nature of the work being computed), some OpenCL applications run badly if the CPU is heavily loaded as well, others don't. And here at SETI, we're having OpenCL accuracy problems when the very newest drivers are paired with the very oldest nVidia cards.

If you hunt down all that information, you should be able to make a rational decision on what course of action to take.
ID: 1604491 · Report as offensive
WB8ILI

Send message
Joined: 27 May 03
Posts: 11
Credit: 12,942,299
RAC: 0
United States
Message 1604509 - Posted: 23 Nov 2014, 13:03:16 UTC

I have had problems with Einstein GPU tasks on certain computers too. On one XP system it locked up the computer similar to what was described below. On the other XP system, the OS spent so much time in the kernel state it didn't do much useful work. I just gave up and run other projects on those systems.
ID: 1604509 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 1604513 - Posted: 23 Nov 2014, 13:20:34 UTC - in response to Message 1604509.  

I am running Einstein@Qhome and Albert@home on two Linux boxes without problems, no GPU. On a Windows 8.1 PC with a Radeon 8470 graphic board I am running both CPU and GPU tasks, also without problems.
Tullio
ID: 1604513 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1604540 - Posted: 23 Nov 2014, 16:34:38 UTC - in response to Message 1604376.  

So why haven't you asked at the Einstein forums? Since it's their applications that run and give you so much trouble, wouldn't their forums be the first place to ask for help on this, instead of the Seti forums?
ID: 1604540 · Report as offensive
Profile Bob Bainbridge

Send message
Joined: 21 Mar 01
Posts: 63
Credit: 128,919,333
RAC: 82
United States
Message 1604715 - Posted: 24 Nov 2014, 3:48:00 UTC - in response to Message 1604540.  

So why haven't you asked at the Einstein forums? Since it's their applications that run and give you so much trouble, wouldn't their forums be the first place to ask for help on this, instead of the Seti forums?

Good Point. Probably because I haven't been very active with the Einstein project and have never visited their forums.

Bob B.
ID: 1604715 · Report as offensive
Profile Bob Bainbridge

Send message
Joined: 21 Mar 01
Posts: 63
Credit: 128,919,333
RAC: 82
United States
Message 1604716 - Posted: 24 Nov 2014, 3:57:10 UTC - in response to Message 1604385.  

How many tasks at a time on your GPU? FWIW, each GPU task requires 300-350mb of GPU RAM. So for example, if you have 1gb of GPU RAM, You should limit it to just 2 or maybe 3 tasks at a time on your GPU.

Aside from that, The things I would look into:

1) Perhaps Einstein runs your CPU hotter, and your cooling is not up to the task. Or your thermal connection to the heatsink is poor.

2) Perhaps Einstein uses more RAM, and one of your DIMMs is marginal. With the extra usage, it is exposing the issue.


I'm running 3 tasks on a GTX 770 with 2gb ram, same as I run under Seti. I gave up all overclocking and all 6 cores of the 970 are running around 70C at 100% load with a water cooler. I have 4gb ram which is the max for 32bit Win7 OS.

Bob B.
ID: 1604716 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1604732 - Posted: 24 Nov 2014, 5:13:09 UTC - in response to Message 1604716.  
Last modified: 24 Nov 2014, 5:14:25 UTC



I'm running 3 tasks on a GTX 770 with 2gb ram, same as I run under Seti. I gave up all overclocking and all 6 cores of the 970 are running around 70C at 100% load with a water cooler. I have 4gb ram which is the max for 32bit Win7 OS.

Bob B.

Running 3 Einstein tasks on that GPU and only using a 32-bit OS with your limited system memory likely is a bad combination and a possible cause of your problem, turn the GPU down to 2 tasks and reserve at least 1 CPU core if you already havn't (two maybe needed though).

Why didn't you go with a 64-bit OS and at least twice the memory?

Cheers.
ID: 1604732 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1604888 - Posted: 24 Nov 2014, 13:36:15 UTC - in response to Message 1604732.  
Last modified: 24 Nov 2014, 13:38:37 UTC



I'm running 3 tasks on a GTX 770 with 2gb ram, same as I run under Seti. I gave up all overclocking and all 6 cores of the 970 are running around 70C at 100% load with a water cooler. I have 4gb ram which is the max for 32bit Win7 OS.

Bob B.

Running 3 Einstein tasks on that GPU and only using a 32-bit OS with your limited system memory likely is a bad combination and a possible cause of your problem, turn the GPU down to 2 tasks and reserve at least 1 CPU core if you already havn't (two maybe needed though).

Why didn't you go with a 64-bit OS and at least twice the memory?

Cheers.


hmm, yes, this mix of 32 Bit Windows, and this Hardware is problematic for a host of accumulated reasons. These point toward the described problems/symptoms, before necessarily considering specific application(s).

Some bullet points (probably not comprehensive)
- Windoiws Vista/7 onwards moved to the WDDM ( Windows Display Driver Model) that 'virtualises' video memory.
- the described system will not be able to utilise the full 4GiB host RAM, and 2GiB VRAM, since the 32 bit address space won't address that much combined (physically) [ note no physical address extension here... ]
- something (probably both host memory and the GPU VRAM) will be being substantially capped.
- The Kernel portion will be reserving around half of whatever's left, including for various driver crash recovery & caching purposes.
- When whatever VRAM is actually allowed becomes close to filled, this will rely on the virtual memory and page to disk
- This in turn will induce delays prone to expose system resources limits/timeouts and produce various kinds of failures.

All that considered, I would venture to guess that even running 2 instances of a demanding application may become unreliable. Most likely just switching to a 64 Bit version of Windows would resolve the problem. Even still, 2GiB VRAM is a lot to mirror (As WDDM does) with only ~2GiB kernel space (leaving 2GiB for user applications, minus overheads).

Most likely, IMO, these mounting limitations go a long way toward the GPU Vendors' gradual moves toward mostly 64 bit support for the newer cards. There just simply aren't enough system resources available under 32 bit to support all the extra Driver+OS heavy lifting in the background.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1604888 · Report as offensive
Profile Bob Bainbridge

Send message
Joined: 21 Mar 01
Posts: 63
Credit: 128,919,333
RAC: 82
United States
Message 1605133 - Posted: 25 Nov 2014, 4:20:46 UTC - in response to Message 1604888.  



I'm running 3 tasks on a GTX 770 with 2gb ram, same as I run under Seti. I gave up all overclocking and all 6 cores of the 970 are running around 70C at 100% load with a water cooler. I have 4gb ram which is the max for 32bit Win7 OS.

Bob B.

Running 3 Einstein tasks on that GPU and only using a 32-bit OS with your limited system memory likely is a bad combination and a possible cause of your problem, turn the GPU down to 2 tasks and reserve at least 1 CPU core if you already havn't (two maybe needed though).

Why didn't you go with a 64-bit OS and at least twice the memory?

Cheers.


hmm, yes, this mix of 32 Bit Windows, and this Hardware is problematic for a host of accumulated reasons. These point toward the described problems/symptoms, before necessarily considering specific application(s).

Some bullet points (probably not comprehensive)
- Windoiws Vista/7 onwards moved to the WDDM ( Windows Display Driver Model) that 'virtualises' video memory.
- the described system will not be able to utilise the full 4GiB host RAM, and 2GiB VRAM, since the 32 bit address space won't address that much combined (physically) [ note no physical address extension here... ]
- something (probably both host memory and the GPU VRAM) will be being substantially capped.
- The Kernel portion will be reserving around half of whatever's left, including for various driver crash recovery & caching purposes.
- When whatever VRAM is actually allowed becomes close to filled, this will rely on the virtual memory and page to disk
- This in turn will induce delays prone to expose system resources limits/timeouts and produce various kinds of failures.

All that considered, I would venture to guess that even running 2 instances of a demanding application may become unreliable. Most likely just switching to a 64 Bit version of Windows would resolve the problem. Even still, 2GiB VRAM is a lot to mirror (As WDDM does) with only ~2GiB kernel space (leaving 2GiB for user applications, minus overheads).

Most likely, IMO, these mounting limitations go a long way toward the GPU Vendors' gradual moves toward mostly 64 bit support for the newer cards. There just simply aren't enough system resources available under 32 bit to support all the extra Driver+OS heavy lifting in the background.

That's probably the source of the problem. I'm running Einstein & Seti on a 64bit AMD 4-core Win 8.1 machine with 8gb ram and on a 64bit AMD 2-core with only 2 gb ram with no problem. The 32bit Intel 6-core is my main computer and has been upgraded without doing a clean install since Win 2000, to retain the desktop and programs. I still run many older programs that won't run under 64 bit. Some are actually old Dos programs. Does PC-File and Wordperfect 6.0 for Dos ring a bell? I'll just omit the Einstein project on this machine for the time being. Curiously this machine is not getting ANY WU's during the shortage but the AMD 2-core with GTX-660 and a Thinkpad with no GPU are still getting some. Thanks for the info.

Bob B.
ID: 1605133 · Report as offensive
WB8ILI

Send message
Joined: 27 May 03
Posts: 11
Credit: 12,942,299
RAC: 0
United States
Message 1612789 - Posted: 12 Dec 2014, 13:19:14 UTC

Bob Bainbridge - Did you ever get your issue resolved? I posted to this thread previously that I had a similar issue and just gave up. However, I spent some more time and resolved my issue.

I was just trying to run too much stuff at the same time. I was trying to run 4 Climate Prediction tasks and two Einstein tasks simultaneously. I have a four-core computer. My thinking was the Einstein units use practically no CPU time and the graphics card can handle two Einstein tasks.

However, it turns out Einstein actually does use a fair amount of CPU time.

I set my Einstein tasks to use 1 CPU and 1 GPU. This way 3 cores are used for CP tasks and one core for Einstein and all the other system stuff.

I didn't try setting Einstein to 0.75 CPU and 0.5 GPU but I think I could run 2 CP and 2 Einstein together.
ID: 1612789 · Report as offensive

Message boards : Number crunching : Einstein@Home Locks PC


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.