New PC not performing as well as expected

Message boards : Number crunching : New PC not performing as well as expected
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profile Eric B

Send message
Joined: 9 Mar 00
Posts: 88
Credit: 168,875,085
RAC: 762
United States
Message 2016353 - Posted: 23 Oct 2019, 8:51:53 UTC

I want to compare 2 pc's i have. I did some rudimentary data gathering form each one and was surprised to see the newer machine being outperformed by the older one. So... my idea was to do the same WU on both machines and see how long it takes. Is there a way to copy a WU from PC-A to PC-B and have it run on both A and B? Alternatively, is there a special downloadable WU that i could use to do the same thing? Actually i need 16WU's to do the test correctly, one PC is a i9-9900k and the other is the i7-3960x The i7 is beating the i9 30,000 to 21,000 'host average' and that doesn't seem correct to me. The i9 is running at 4.1Ghz and the i7 at 3.8Ghz . I have the exact same GPU (gtx 1660 ti) in both and have plenty of memory i9- 16G and i7 - 32G.
oh and this is all under openSuse Linux, tyhe i9 is using Leap 15.1 and the i7 is running Tumbleweed
ID: 2016353 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016354 - Posted: 23 Oct 2019, 9:02:10 UTC

The 2 systems cannot be compared as one is running the Stock application (with no command line options so it's output Is much less than it could be), and the other is running the Special Application- which will produce much more work from the GPU than the stock application can, even with the most aggressive command line settings.

You need to run the same applications on both systems in order to compare their outputs.
Grant
Darwin NT
ID: 2016354 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2016436 - Posted: 24 Oct 2019, 0:42:24 UTC

As Grant mentioned, you can't compare the two systems when each is using a different gpu application. The tasks don't matter. It is the applications that are making the difference. If you want to compare applications running the same tasks then you can run offline using benchmark tools. Rick's BenchMT is my preference:
https://github.com/Ricks-Lab/benchMT/releases/tag/v1.6.0
but there is the older bench tools at Lunatics also.
http://lunatics.kwsn.info/index.php?action=downloads;cat=5
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2016436 · Report as offensive
Profile Eric B

Send message
Joined: 9 Mar 00
Posts: 88
Credit: 168,875,085
RAC: 762
United States
Message 2016554 - Posted: 25 Oct 2019, 8:18:41 UTC

Ok, I just updated the second system to use the same optimized apps. I'll let things run for a day or two and then see how they compare. Once I have the info I'll post back my findings.
ID: 2016554 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016556 - Posted: 25 Oct 2019, 8:51:21 UTC - in response to Message 2016554.  
Last modified: 25 Oct 2019, 9:01:26 UTC

Ok, I just updated the second system to use the same optimized apps. I'll let things run for a day or two and then see how they compare. Once I have the info I'll post back my findings.
Significant changes to a system (hardware or software) take about 6-8 weeks for RAC to stabilise (if there are no other Seti server hiccups in that period, or power outages or other fun and games at your end).
But looking at actual WU processing times will give a good indication of what to expect at the end of that period- and a quick look shows the GPU of the newer system now putting out about 3-4 times as much work s it was, and if the same application is now being used for the CPU on both systems, the system in question should be putting out a lot more CPU work than it was previously as well.


Edit- you might want to check that the updated system is OK, as it hasn't contacted the Seti servers for over 30min. Something pumping out that amount of work will usually make contact every 5-6 min (Although that system has tonne of Ghosts, so that could muck things up a bit).
Grant
Darwin NT
ID: 2016556 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016557 - Posted: 25 Oct 2019, 9:53:39 UTC - in response to Message 2016556.  
Last modified: 25 Oct 2019, 9:54:27 UTC

Edit- you might want to check that the updated system is OK, as it hasn't contacted the Seti servers for over 30min. Something pumping out that amount of work will usually make contact every 5-6 min (Although that system has tonne of Ghosts, so that could muck things up a bit).
It's now making contact again.
There is a huge backlog with the Validators at the moment (it was improving for a while, but now it's reached a new high & continues to climb), so it could take a while for Valid results to start coming through (and for RAC to go up instead of down).
Grant
Darwin NT
ID: 2016557 · Report as offensive
Profile Eric B

Send message
Joined: 9 Mar 00
Posts: 88
Credit: 168,875,085
RAC: 762
United States
Message 2016704 - Posted: 26 Oct 2019, 12:11:33 UTC

I now have both systems running identical SW and OS. In one system i took the opportunity to install an NVME (Samsung 970 EVO plus) drive and do a fresh install of openSuse Tumbleweed. So it should be an apples to apples comparison. I'll check back in a day or so and post my results.
One sort of related question I have tho is - can I have 2 cuda apps listed in my app_info.xml? I've got setiathome_x41p_V0.98b1_x86_64-pc-linux-gnu_cuda90 and setiathome_x41p_V0.98b1_x86_64-pc-linux-gnu_cuda101 software but I am using the cuda 90 at the moment. Is the 101 better? faster?
ID: 2016704 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 2016708 - Posted: 26 Oct 2019, 12:26:02 UTC - in response to Message 2016704.  
Last modified: 26 Oct 2019, 12:28:38 UTC

You appear to be getting slower performance for the gpu than I would expect. A Gtx 1060 3Gb's under Linux with identical apps are running faster on average than your gpu's appear to be running.

Are you running your cpu's at 90% of available threads? You can set that globally on the website or locally with each Boinc manager.

Do you have 1 cpu core / gpu? If the Boinc Manager shows less than 1 full core for your gpu you need to setup an app_config.xml file to fix that.

Do you have "-nobs" in the command line of the app_info.xml file for your cuda app?

Tom
A proud member of the OFA (Old Farts Association).
ID: 2016708 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 2016754 - Posted: 26 Oct 2019, 20:17:16 UTC

I have to agree with Tom M, there is something not quite right there, as my old 3GB 1060's are completing work 40-60 seconds faster than either of your 1660Ti's are.

Cheers.
ID: 2016754 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 2016765 - Posted: 26 Oct 2019, 21:43:17 UTC - in response to Message 2016754.  

I have to agree with Tom M, there is something not quite right there, as my old 3GB 1060's are completing work 40-60 seconds faster than either of your 1660Ti's are.

Cheers.


I am still running 5 gtx 1060 3gb's under Linux/Tbar's All-in-One so I have current comparisons. And that is why I spoke up. Surely a 1660Ti should be faster at Seti GPU crunching than a 1060?

Tom
A proud member of the OFA (Old Farts Association).
ID: 2016765 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 2016774 - Posted: 26 Oct 2019, 22:39:18 UTC

Now these are the times that I'd expect from those 1660Ti's, https://setiathome.berkeley.edu/results.php?hostid=8747061&offset=0&show_names=0&state=4&appid= (using -nobs), and https://setiathome.berkeley.edu/results.php?hostid=8794661&offset=0&show_names=0&state=4&appid= (without -nobs), so as you can see yours are well out of the ballpark for some reason.

Cheers.
ID: 2016774 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016775 - Posted: 26 Oct 2019, 22:50:10 UTC
Last modified: 26 Oct 2019, 22:50:56 UTC

Their CPU runtimes are longer than i'd expect, and there is some difference between Run time & CPU time on one system, and a very large difference between them on the other.
I'm wondering if the systems are actually running at their rated CPU clock speeds. Thermal throttling? Systems busy doing other work, impacting on both CPU & GPU crunching performance?
Grant
Darwin NT
ID: 2016775 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2016785 - Posted: 27 Oct 2019, 0:01:29 UTC
Last modified: 27 Oct 2019, 0:04:45 UTC

I'm surprised no one has stated the obvious. He's probably running 100% CPU use and starving the GPU of CPU cycles needed to feed the GPU. He should back the CPU setting down to 80-90%.

and it's easy to see what frequency his CPU is running, it's recorded in the stderr.txt file.

Build features: SETI8 Non-graphics FFTW FFTOUT JSPF AVX 64bit 
 System: Linux  x86_64  Kernel: 5.3.6-1-default
 CPU   : Intel(R) Core(TM) i7-3960X CPU @ 3.30GHz
 12 core(s), Speed :  3700.211 MHz
 L1 : 64 KB, Cache : 15360 KB
 Features : FPU TSC PAE APIC MTRR MMX SSE  SSE2 HT PNI SSSE3 SSE4_1 SSE4_2 AVX 

Build features: SETI8 Non-graphics FFTW FFTOUT JSPF AVX 64bit 
 System: Linux  x86_64  Kernel: 5.3.6-1-default
 CPU   : Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
 16 core(s), Speed :  4699.998 MHz
 L1 : 64 KB, Cache : 16384 KB
 Features : FPU TSC PAE APIC MTRR MMX SSE  SSE2 HT PNI SSSE3 SSE4_1 SSE4_2 AVX  AVX2  


that i9-9900k, even though it's running 4.7 GHz seems to be processing WUs really slow though.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2016785 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016789 - Posted: 27 Oct 2019, 0:22:25 UTC - in response to Message 2016785.  
Last modified: 27 Oct 2019, 0:25:13 UTC

and it's easy to see what frequency his CPU is running, it's recorded in the stderr.txt file.
Well, that's the clock speed at the time the CPU was queried for that log, but is it the clock speed while the CPU is processing the WU?


that i9-9900k, even though it's running 4.7 GHz seems to be processing WUs really slow though.
Exactly. An hour longer than a slower 6 year older CPU for the same type of WU.
Both systems are showing slow computation times for their given clock speed & the application being used.

Wasn't there a LINUX distro that had issues with power saving kicking in when it shouldn't have some time back? Could this be something like that?
Grant
Darwin NT
ID: 2016789 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2016792 - Posted: 27 Oct 2019, 0:32:50 UTC - in response to Message 2016789.  

but is it the clock speed while the CPU is processing the WU?

yes. the other threads are under load from other WUs running. 4.7 GHz is the all-core turbo speed for that processor so this makes sense.


Wasn't there a LINUX distro that had issues with power saving kicking in when it shouldn't have some time back? Could this be something like that?


I'm not familiar with that issue, will have to defer to someone else on that.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2016792 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 2016795 - Posted: 27 Oct 2019, 0:39:05 UTC

Wasn't there a LINUX distro that had issues with power saving kicking in when it shouldn't have some time back? Could this be something like that?
Yes I remember that, though I can't remember which distro was involved (it wasn't Ubuntu or Mint though) or even who it was, but they did have to go through several OS settings to get the OS to use the hardware to its max.

Cheers.
ID: 2016795 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016796 - Posted: 27 Oct 2019, 0:41:36 UTC - in response to Message 2016792.  

but is it the clock speed while the CPU is processing the WU?
yes. the other threads are under load from other WUs running. 4.7 GHz is the all-core turbo speed for that processor so this makes sense.
So if the CPU is running at it's rated clock speed, how can it take so long to process a WU? (there is a discrepancy in the CPU & Runtimes, but the CPU time itself is excessive for such a chip that is meant to be running at it's rated clock speed).
Other than running at half speed, how would a WU take twice as long to process than it should for a given CPU & application?
Grant
Darwin NT
ID: 2016796 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2016798 - Posted: 27 Oct 2019, 0:53:07 UTC - in response to Message 2016796.  

but is it the clock speed while the CPU is processing the WU?
yes. the other threads are under load from other WUs running. 4.7 GHz is the all-core turbo speed for that processor so this makes sense.
So if the CPU is running at it's rated clock speed, how can it take so long to process a WU? (there is a discrepancy in the CPU & Runtimes, but the CPU time itself is excessive for such a chip that is meant to be running at it's rated clock speed).
Other than running at half speed, how would a WU take twice as long to process than it should for a given CPU & application?


you're assuming there's nothing else running. it's not always black and white.

there very well could be some other application outside of BOINC trying to compete for CPU resources.

or something else, hardware problems, OS problems, etc. hard to know for sure without more info from Eric.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2016798 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 2016800 - Posted: 27 Oct 2019, 1:14:03 UTC - in response to Message 2016798.  
Last modified: 27 Oct 2019, 1:24:24 UTC

you're assuming there's nothing else running. it's not always black and white.

there very well could be some other application outside of BOINC trying to compete for CPU resources.
Other things competing for CPU resources show up as a discrepancy between CPU time and Runtime.
Long Runtimes are long Runtimes- things aren't running as quickly as they should. And that's set by the CPU architecture, it's clock speed, and the application.
If the architecture & the application aren't in question, that leaves clock speed. Unless someone can come up with a 4th option? (such as memory bandwidth restrictions which has casued massive impacts on multi-socket quad channel systems in the past with not enough modules populated/and or the wrong ones).
Hmmm.


One system has 16GM of RAM, the other 32GB. Even 8GB is more than enough.
However- how many modules make up that amount of RAM? Are they in the correct memory slots?
Grant
Darwin NT
ID: 2016800 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2016801 - Posted: 27 Oct 2019, 1:21:52 UTC - in response to Message 2016789.  

Wasn't there a LINUX distro that had issues with power saving kicking in when it shouldn't have some time back? Could this be something like that?

Shouldn't be an issue. He is running a very recent kernel with all the latest cpu scheduler improvements.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2016801 · Report as offensive
1 · 2 · 3 · Next

Message boards : Number crunching : New PC not performing as well as expected


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.