Panic Mode On (101) Server Problems?

Message boards : Number crunching : Panic Mode On (101) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 24 · 25 · 26 · 27

AuthorMessage
Rasputin42
Volunteer tester

Send message
Joined: 25 Jul 08
Posts: 412
Credit: 5,834,661
RAC: 0
United States
Message 1758322 - Posted: 22 Jan 2016, 21:35:49 UTC
Last modified: 22 Jan 2016, 21:39:37 UTC

I noticed that as well.

Well done to whoever did that!

( just remove the combined bit, please)
ID: 1758322 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758325 - Posted: 22 Jan 2016, 21:38:55 UTC - in response to Message 1758316.  
Last modified: 22 Jan 2016, 21:42:48 UTC

v7 vs v8 stats for creation rate, received in the last hour, and turnaround time are now showing independent figures.

Good to see.
Grant
Darwin NT
ID: 1758325 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1758334 - Posted: 22 Jan 2016, 21:49:06 UTC - in response to Message 1758316.  

Lunatic downloaded and installed fine and I am now running 1 V8 on the CPU and 1 V8 on the GT430. GPU-Z is showing very poor GPU loading and the task is showing 3% completion after 40m. The task is being run as cuda42. This is not good.


Would like to take a quick look for obvious issues, but you'll need to report at least one first.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1758334 · Report as offensive
Profile betreger
Avatar

Send message
Joined: 29 Jun 99
Posts: 9538
Credit: 25,964,150
RAC: 22,669
United States
Message 1758368 - Posted: 22 Jan 2016, 23:00:06 UTC - in response to Message 1758334.  

Lunatic downloaded and installed fine and I am now running 1 V8 on the CPU and 1 V8 on the GT430. GPU-Z is showing very poor GPU loading and the task is showing 3% completion after 40m. The task is being run as cuda42. This is not good.


Would like to take a quick look for obvious issues, but you'll need to report at least one first.

At 2 hr 14 m it is now showing 14.6% done. At this rate I'm projecting over 15 hr run time.
ID: 1758368 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 519,256,974
RAC: 300,300
Australia
Message 1758369 - Posted: 22 Jan 2016, 23:03:32 UTC - in response to Message 1758334.  
Last modified: 22 Jan 2016, 23:04:23 UTC

Have loaded v8 GPU app onto this machine: 7114519.

So far completed 3 x GPU v8 WUs in around 400 seconds or less (run time).

Don't forget Installer sets <count> to 1 so you need to change it to suit.

cheers
ID: 1758369 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758372 - Posted: 22 Jan 2016, 23:05:59 UTC - in response to Message 1758369.  

Don't forget Installer sets <count> to 1 so you need to change it to suit.
cheers

Or just use app_config.xml if your manager version supports it.
Grant
Darwin NT
ID: 1758372 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 50378
Credit: 983,779,992
RAC: 2,114
United States
Message 1758387 - Posted: 22 Jan 2016, 23:28:53 UTC - in response to Message 1758372.  

Don't forget Installer sets <count> to 1 so you need to change it to suit.
cheers

Or just use app_config.xml if your manager version supports it.

Us old farts still running old Boincs have to do that in app_info.

Been there, done that, kitties are running 3/per.

Meow.
"Learn from yesterday. Live for today. Hope for tomorrow." Albert Einstein
"With cats." kittyman

ID: 1758387 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758396 - Posted: 22 Jan 2016, 23:44:03 UTC - in response to Message 1758387.  

Been there, done that, kitties are running 3/per.

Meow.

I'm seeing how 3 per card do as well.
Due to not being as optimised as the v7 application it looks like 3 might give slightly more output per hour than 2 on my GTX 750Tis with the current application.
Might have to wait for the estimated times to settle a bit, and hopefully get a batch of WUs from the same file to make it easier to see if 3 really is better than 2.
So far, it looks like it might give me about .5WUs more per hour. Not much, but over a year it adds up.
Grant
Darwin NT
ID: 1758396 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 50378
Credit: 983,779,992
RAC: 2,114
United States
Message 1758398 - Posted: 22 Jan 2016, 23:47:10 UTC - in response to Message 1758396.  
Last modified: 22 Jan 2016, 23:48:14 UTC

Been there, done that, kitties are running 3/per.

Meow.

I'm seeing how 3 per card do as well.
Due to not being as optimised as the v7 application it looks like 3 might give slightly more output per hour than 2 on my GTX 750Tis with the current application.
Might have to wait for the estimated times to settle a bit, and hopefully get a batch of WUs from the same file to make it easier to see if 3 really is better than 2.
So far, it looks like it might give me about .5WUs more per hour. Not much, but over a year it adds up.

I never analyze it that closely..........just, in my experience, and given the GPU loading I see in the EVGA Precision app, 3 seems to be the best choice. I don't do spreadsheets on it or anything like that. Sometimes, it's just what the kitties tell me to do.

Meow...sometimes you just have to listen to the meows.
"Learn from yesterday. Live for today. Hope for tomorrow." Albert Einstein
"With cats." kittyman

ID: 1758398 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2766
Credit: 573,659,846
RAC: 910,370
Canada
Message 1758406 - Posted: 22 Jan 2016, 23:56:04 UTC - in response to Message 1758396.  
Last modified: 22 Jan 2016, 23:57:55 UTC

My AMD 4200 750Ti is running 5 per hour running 2 tasks. EDIT: 17:00 to 25:00 runtimes.

I haven't converted my Intel box yet, trying to run off the CPU tasks.
ID: 1758406 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1758421 - Posted: 23 Jan 2016, 0:23:43 UTC - in response to Message 1758368.  
Last modified: 23 Jan 2016, 0:24:08 UTC

Lunatic downloaded and installed fine and I am now running 1 V8 on the CPU and 1 V8 on the GT430. GPU-Z is showing very poor GPU loading and the task is showing 3% completion after 40m. The task is being run as cuda42. This is not good.


Would like to take a quick look for obvious issues, but you'll need to report at least one first.

At 2 hr 14 m it is now showing 14.6% done. At this rate I'm projecting over 15 hr run time.


If something is indeed wrong (which seems likely as you do OpenCL APs in ~3 hours), then Boinc may abort due to time exceeded at some point. I dont know what performance to expect from that model', and the tasks are longer, but app behaviour should not be that different to v7 at all. The tasks do speed up after some percentage, so that's something. I'd jack up the process priority to abovenormal as per the readme, and see if things kick up a notch [starved for CPU or something].

Probably best to start a specific thread if we need to figure out what's going on there. Fermi class (however small) should be running fine on that build.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1758421 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758445 - Posted: 23 Jan 2016, 1:04:31 UTC - in response to Message 1758398.  

I never analyze it that closely..........just, in my experience, and given the GPU loading I see in the EVGA Precision app, 3 seems to be the best choice. I don't do spreadsheets on it or anything like that. Sometimes, it's just what the kitties tell me to do.


I've found GPU loading not to be a good indicator.
Often running at 75% GPU load does more work per hour than when running at 90% (or more). Then there are times where running at 95% GPU load does more work than running at only 85%.
Then you get those people that run 5 at a time to get the GPU & GPU memory load up to 100%, and do less than half the work per hour than they would with only 2WUs and 75% GPU load.



Generally I just like to get a bunch of similar WUs with all the same run time estimates, and see how long they take.
Then try them at 2 at a time, then 3.

I don't bother with a spread sheet, just pen and paper.
eg

1 WU in 15min = 4.0 per hour.
2 WU in 28min = 4.3 per hour.
3 WU in 40min = 4.5 per hour.

The more you run, the longer it takes- but what should happen is you do more WUs per hour. If not, then there's no point in running that extra WU; especially if it results in less work actually being done.
Grant
Darwin NT
ID: 1758445 · Report as offensive
Profile betreger
Avatar

Send message
Joined: 29 Jun 99
Posts: 9538
Credit: 25,964,150
RAC: 22,669
United States
Message 1758461 - Posted: 23 Jan 2016, 1:57:18 UTC - in response to Message 1758421.  

Lunatic downloaded and installed fine and I am now running 1 V8 on the CPU and 1 V8 on the GT430. GPU-Z is showing very poor GPU loading and the task is showing 3% completion after 40m. The task is being run as cuda42. This is not good.


Would like to take a quick look for obvious issues, but you'll need to report at least one first.

At 2 hr 14 m it is now showing 14.6% done. At this rate I'm projecting over 15 hr run time.


If something is indeed wrong (which seems likely as you do OpenCL APs in ~3 hours), then Boinc may abort due to time exceeded at some point. I dont know what performance to expect from that model', and the tasks are longer, but app behaviour should not be that different to v7 at all. The tasks do speed up after some percentage, so that's something. I'd jack up the process priority to abovenormal as per the readme, and see if things kick up a notch [starved for CPU or something].

Probably best to start a specific thread if we need to figure out what's going on there. Fermi class (however small) should be running fine on that build.

Probably not necessary, it seems that the GPU V8 tasks are allover the chart as far as run time. I have got a few with quite OK times. As an aside when it was running "slow" the task manager showed 100% CPU usage and 50% of that was system idle process. Now both GPU-Z and task manager look "normal".
ID: 1758461 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 4879
Credit: 30,743,375
RAC: 227
United States
Message 1758468 - Posted: 23 Jan 2016, 2:53:45 UTC

Isn't the Avg daily turnaround broken down by app
(https://setiathome.berkeley.edu/host_app_versions.php?hostid=7246450) a good indicator of the efficiency? By checking that parameter periodically you can measure and adjust the number of jobs running concurrently on each app.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1758468 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758481 - Posted: 23 Jan 2016, 3:41:48 UTC - in response to Message 1758468.  

Isn't the Avg daily turnaround broken down by app
(https://setiathome.berkeley.edu/host_app_versions.php?hostid=7246450) a good indicator of the efficiency? By checking that parameter periodically you can measure and adjust the number of jobs running concurrently on each app.

Problem with that method is you'd need to run things for several days, then change them, then run them for several more days, then change them.
It'd take close to a week to figure out reasonably accurately just what was best.
And that's if there were no system oopsies in that period.
Grant
Darwin NT
ID: 1758481 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 4879
Credit: 30,743,375
RAC: 227
United States
Message 1758482 - Posted: 23 Jan 2016, 3:44:34 UTC - in response to Message 1758481.  

Isn't the Avg daily turnaround broken down by app
(https://setiathome.berkeley.edu/host_app_versions.php?hostid=7246450) a good indicator of the efficiency? By checking that parameter periodically you can measure and adjust the number of jobs running concurrently on each app.

Problem with that method is you'd need to run things for several days, then change them, then run them for several more days, then change them.
It'd take close to a week to figure out reasonably accurately just what was best.
And that's if there were no system oopsies in that period.

I'm in for the long haul so time's not a problem. And any precision instrument always needs tuning.....

"Sour Grapes make a bitter Whine." <(0)>
ID: 1758482 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2766
Credit: 573,659,846
RAC: 910,370
Canada
Message 1758505 - Posted: 23 Jan 2016, 5:23:58 UTC

TThrottle is nice for monitoring progress, Just click on HISTORY and count the number of files you have done in 3 hours (or 12) and it gives you a good idea how you are doing.
ID: 1758505 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758823 - Posted: 23 Jan 2016, 22:22:15 UTC - in response to Message 1758396.  

I'm seeing how 3 per card do as well.
Due to not being as optimised as the v7 application it looks like 3 might give slightly more output per hour than 2 on my GTX 750Tis with the current application.
Might have to wait for the estimated times to settle a bit, and hopefully get a batch of WUs from the same file to make it easier to see if 3 really is better than 2.
So far, it looks like it might give me about .5WUs more per hour. Not much, but over a year it adds up.


MBv8 is similar to MBv7. Running 3 at a time on my GTX 750Tis gives a slight improvement in throughput on longer running WUs. But on shorties, the crunching time is at least double, sometimes more.
So 2 at a time is still the best for performance for my hardware.
Grant
Darwin NT
ID: 1758823 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 4879
Credit: 30,743,375
RAC: 227
United States
Message 1758829 - Posted: 23 Jan 2016, 22:38:24 UTC

So far running 3 jobs on my 750ti has produced:
Average turnaround time 0.47 days
on 62 completed jobs.
While running 3 on my 660 has produced:
Average turnaround time 0.30 days
on 49 completed jobs.
Running 4 jobs on an older Xeon machine with a GT 740 has produced:
Average turnaround time 0.32 days
on 29 completed tasks.

Both the GTX cards are running on Haswell rigs with the Intel GPU also running 3 jobs and the CPU running 7 jobs. The Xeon machine is running 3 CPU jobs. The Intel Haswell GPU rigs are averageing 2.5 to 2.79 turnaround days.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1758829 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11658
Credit: 174,111,649
RAC: 119,468
Australia
Message 1758837 - Posted: 23 Jan 2016, 22:58:36 UTC - in response to Message 1758829.  

Both the GTX cards are running on Haswell rigs with the Intel GPU also running 3 jobs and the CPU running 7 jobs. The Xeon machine is running 3 CPU jobs. The Intel Haswell GPU rigs are averageing 2.5 to 2.79 turnaround days.

Turn around days are like Credits, they are a lagging indicator & it will take several days for the numbers to reflect reality.
That's why I prefer to use actual run times.
Grant
Darwin NT
ID: 1758837 · Report as offensive
Previous · 1 . . . 24 · 25 · 26 · 27

Message boards : Number crunching : Panic Mode On (101) Server Problems?


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.