The Saga Begins (LotsaCores 2.0)

Message boards : Number crunching : The Saga Begins (LotsaCores 2.0)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 8 · Next

AuthorMessage
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1811840 - Posted: 23 Aug 2016, 13:11:52 UTC - in response to Message 1811807.  

Btw Al, I'm not pleased that you're starting a new saga when mine's not finished! :-p
But I like yours much better than mine! lol :-D

"Keep Calm and carry Crunch on!"

LOL, thanks, Rob! And sorry.. ;-)

ID: 1811840 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1811847 - Posted: 23 Aug 2016, 13:27:54 UTC - in response to Message 1811836.  

Thanks for the input guys. So at this point, there isn't a known downside to going with the latest version drivers for the 750? Older isn't necessarily faster? I plan on loading them up this morning sometime, and see how it goes, then load up BOINC w/SoG, and then will be back for questions about command lines, as I hear that they can make a significant difference.

Slightly risky on a Tuesday - the tools you'll need, such as the Lunatics installer, are designed to minimise the risk of losing a cache, but things can and have gone wrong in the past. And the same applies to driver updates. Usually regarded as wise to take a break from major testing on Maintenance Day.
ID: 1811847 · Report as offensive
Profile Cliff Harding
Volunteer tester
Avatar

Send message
Joined: 18 Aug 99
Posts: 1432
Credit: 110,967,840
RAC: 67
United States
Message 1811856 - Posted: 23 Aug 2016, 13:59:56 UTC

Hey al,

I'm running 368.39 on my 4770K with 3 tasks per 2 x GTX750Ti FTW using Lunatics 0.45 beta -4. My config.xml is set at .33 GPU & .5 CPU. The cmdline.txt is

-use_sleep -sbs 256 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64 -hp

Compared to the -3 beta my run times have been cut almost in half, but that was my fault. Basically my est. run times run in 2 groups approx. 35 min & 1hr 35 min.

BTW, I'm running at 90% CPU, which comes down to 6 GPU % 4 CPU. Depending on what's running, sometimes I can run at 100% adding another CPU task. The Intel 4600 iGPU takes care of all of my monitor needs.


I don't buy computers, I build them!!
ID: 1811856 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1811886 - Posted: 23 Aug 2016, 15:36:10 UTC

I recently had some frequent "memory related failure" issues (on Cuda50 tasks) resulting in task postponement on my two 4-GPU machines with driver 361.91. At that time, one machine (a new build) included a single 750Ti in the mix and the other had (and still has) two. (I had just upgraded the driver on that one because I was attempting to replace a GTX 660 with a 960, a move which I've had to back off from for other reasons.)

No actual memory, CPU or other issues were detected in many hours of diagnostic testing, but since both machines had just had driver 361.91 installed, I decided to try backing up to earlier drivers. Going back to 361.75 on the new build didn't eliminate the problem, but backing up to 359.00 seemed to make it go away. Since the other machine had been crunching just fine for a long time on 353.30, and since the GTX 960 had been removed, I simply reverted to that version and the problem also went away. Both machines have been error free for about 10 days now.

Not definitive evidence of a problem with the newer drivers, perhaps, but good enough for me to avoid them for now.
ID: 1811886 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1811909 - Posted: 23 Aug 2016, 21:53:43 UTC - in response to Message 1811836.  

Thanks for the input guys. So at this point, there isn't a known downside to going with the latest version drivers for the 750? Older isn't necessarily faster? I plan on loading them up this morning sometime, and see how it goes, then load up BOINC w/SoG, and then will be back for questions about command lines, as I hear that they can make a significant difference.

I dreaded having to change from my stable 353.62 driver that I have been using for over a year on my 970's to one of the later drivers that I would have to update for the 1070 changeover. As said elsewhere, there has been no reason to update drivers for GPGPU performance reasons for several years now. The only reason that drivers are updated by Nvidia is to support new games or fix game troubles. The only reason one needs to update drivers for GPGPU work is to support new card generations.

I did bump the Win 10 machine from the stock Microsoft installed 358.91 driver to the 362.00 driver because the forum traffic consensus was that it was one of the most stable releases in that timeframe.

I have had no issues so far with the 372.54 drivers I installed for the 1070's. I would have to try that driver on the 970's and compare the task times to see if there is any benefit from moving to a later driver. The structural performance gains of the 1070 can't be separated from driver change improvements and compared to the 970 and its current driver level.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1811909 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1811919 - Posted: 23 Aug 2016, 22:21:40 UTC

Got everything installed last nite and this morning, it's up and running, but not allowing it any tasks at this moment. GPU-Z sees 7 cards though the readings are 0 on all of them but 2, and after tossing the latest Nvidia driver at it, it only sees those 2 in Precision X16. I am still getting those beeps and resource error messages during bootup, so I'm sure that has everything to do with it. Won't have time today to contact Supermicro support for assistance, hopefully tomorrow, because I still need to install SoG before I turn on the taps.

Man, looking at Coretemp, it's kinda crazy, takes up the whole taskbar, or top to bottom of the screen if I pull up the control panel. 56 cores total, fun stuff, but will be much funner when it's all settled down and crunching furiously! :-) Plus, if anyone has any suggestions about the command line and if I should use it, that would be great too.

ID: 1811919 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1811940 - Posted: 24 Aug 2016, 0:00:53 UTC - in response to Message 1811919.  

Well you know that you can run the CPU apps without troubles. Its just trying to get the 750Tis' seen and running tasks. Cliff's command line for SoG would be a good place to start. I would however experiment with taking off the -use_sleep and -hp parameters and see whether you get unacceptable system lags. I would hope that with 56 CPU cores to manage whatever desktop maintenance is needed, you wouldn't get system lags. Only way to find out is try it. I would also try bumping -sbs to 384 for more aggressive running.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1811940 · Report as offensive
Profile Cliff Harding
Volunteer tester
Avatar

Send message
Joined: 18 Aug 99
Posts: 1432
Credit: 110,967,840
RAC: 67
United States
Message 1811973 - Posted: 24 Aug 2016, 1:34:03 UTC - in response to Message 1811940.  

Well you know that you can run the CPU apps without troubles. Its just trying to get the 750Tis' seen and running tasks. Cliff's command line for SoG would be a good place to start. I would however experiment with taking off the -use_sleep and -hp parameters and see whether you get unacceptable system lags. I would hope that with 56 CPU cores to manage whatever desktop maintenance is needed, you wouldn't get system lags. Only way to find out is try it. I would also try bumping -sbs to 384 for more aggressive running.


The 256 that I stated from the -sbs should be 512. Big thumb keying - my bad. The -sleep parm is cut down on kernel thrashing and the -hp is a leftover from earlier and haven't tried to take it out.


I don't buy computers, I build them!!
ID: 1811973 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1811994 - Posted: 24 Aug 2016, 2:39:52 UTC - in response to Message 1811856.  

Hey al,

I'm running 368.39 on my 4770K with 3 tasks per 2 x GTX750Ti FTW using Lunatics 0.45 beta -4. My config.xml is set at .33 GPU & .5 CPU. The cmdline.txt is

-use_sleep -sbs 256 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64 -hp

Compared to the -3 beta my run times have been cut almost in half, but that was my fault. Basically my est. run times run in 2 groups approx. 35 min & 1hr 35 min.

BTW, I'm running at 90% CPU, which comes down to 6 GPU % 4 CPU. Depending on what's running, sometimes I can run at 100% adding another CPU task. The Intel 4600 iGPU takes care of all of my monitor needs.
Got it, and will modify it with your last post. But, where do I put it? Downloaded the Lunatics Win64 0.45 Beta 4 and just installed it, didn't choose the GPU options yet, as I am going to hold off with that till I get the full compliment of cards working in the system. I presume once I get them working properly, I can just re-run the installer and choose them, and it won't mess anything else up? Or do I have to clear the cache and start over again?

ID: 1811994 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1812002 - Posted: 24 Aug 2016, 3:03:20 UTC - in response to Message 1811994.  

I would try -sbs 512 too. I run with that setting on my 970 and 1070. The tuning parameter would either go into mb_cmdline_win_x86_SSE3_OpenCL_NV_SoG.txt for the SoG app or into app_config.xml in a <cmdline>-sbs 512 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64</cmdline> entry in the <app_version> section of the MB Seti_V8 section. Either way works. This is with the Lunatics 0.45 Beta 4 installer.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1812002 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1812004 - Posted: 24 Aug 2016, 3:05:13 UTC - in response to Message 1812002.  

A good thing about the Lunatics installer is that it preserves work downloaded under previous plan classes and doesn't force you to empty your cache. You shouldn't dump work if it works as it normally does.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1812004 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1812147 - Posted: 24 Aug 2016, 12:02:31 UTC - in response to Message 1812002.  

Keith, I did a search for the mb_cmdline_win_x86_SSE3_OpenCL_NV_SoG.txt file to add that line into, because it seemed easier with less chance of me mucking it up, but after doing a search of the entire drive, it says the file isn't found. Is this something I need to create, and if so, where should I put it?

ID: 1812147 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 1812155 - Posted: 24 Aug 2016, 12:07:55 UTC - in response to Message 1812147.  
Last modified: 24 Aug 2016, 12:08:22 UTC

Keith, I did a search for the mb_cmdline_win_x86_SSE3_OpenCL_NV_SoG.txt file to add that line into, because it seemed easier with less chance of me mucking it up, but after doing a search of the entire drive, it says the file isn't found. Is this something I need to create, and if so, where should I put it?


This file is already present in you projects folder.
File size should be zero.


With each crime and every kindness we birth our future.
ID: 1812155 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1812173 - Posted: 24 Aug 2016, 12:38:33 UTC - in response to Message 1812155.  

Hmm, in the C:\ProgramData\BOINC\projects folder I have a file called placeholder.txt, which is zero bytes, dated 5/11/15, but that doesn't seem correct. In the C:\ProgramData\BOINC\projects\setiathome.berkeley.edu folder I have a mb_cmdline_win_x64_AVX_VS2010.txt file that is 0 bytes, but nothing else mb_cmdline*.txt in there either. The only other zero byte file in there is ap_cmdline_win_x64_AVX_CPU.txt. Is it something I need to create, and any idea why it wasn't when I installed Lunatics beta 4?

Oh, and here is the Coretemp of it running:



ID: 1812173 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1812177 - Posted: 24 Aug 2016, 13:16:31 UTC - in response to Message 1812173.  

Hmm, in the C:\ProgramData\BOINC\projects folder I have a file called placeholder.txt, which is zero bytes, dated 5/11/15, but that doesn't seem correct. In the C:\ProgramData\BOINC\projects\setiathome.berkeley.edu folder I have a mb_cmdline_win_x64_AVX_VS2010.txt file that is 0 bytes, but nothing else mb_cmdline*.txt in there either. The only other zero byte file in there is ap_cmdline_win_x64_AVX_CPU.txt. Is it something I need to create, and any idea why it wasn't when I installed Lunatics beta 4?

It's a file which should have been copied if (and only if) you selected the SoG application on the NVidia page when you ran the Beta4 installer (fourth item in lower selection group).

That file only has any effect if you also install the application it's associated with.
ID: 1812177 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1812182 - Posted: 24 Aug 2016, 13:30:06 UTC - in response to Message 1812177.  
Last modified: 24 Aug 2016, 13:33:22 UTC

So if I re-run the installer, making sure I choose the correct option this time, will it mess anything up?

*Edit* Duh! I am only running CPU tasks right now, didn't install the video portion of it, so _of course_ it isn't there yet. I need some caffeine, Badly! Sorry to bug you guys about it, I will re-install the app with the proper selections once I get my PCI-E issues resolved, and all the video cards seen by the OS and the programs.

ID: 1812182 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1812185 - Posted: 24 Aug 2016, 13:35:51 UTC

Oh, and if anyone was curious, per Kill-a-Watt, it is pulling 414 watts out of the wall with 7 750Ti video cards plugged in, doing nothing but spinning their fans, and running 56 tasks at once.

ID: 1812185 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1812215 - Posted: 24 Aug 2016, 16:20:41 UTC - in response to Message 1812185.  

I'll hazard a guesstimate that with all 7 750 crunching and cores processing, the power from the wall is going to hit 800 or more watts. Hope you have a 1000 watt power supply in the beast.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1812215 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1812334 - Posted: 25 Aug 2016, 1:00:07 UTC - in response to Message 1812215.  

Yep, I sure do, a PC Power & Cooling Turbo-Cool 1200W.



I initially tried their Silencer 1200, but it didn't have enough Molex/SATA power connectors for this setup, plus I was wary about using an Molex to 4 pin adapter for the last 4 pin connection I needed to power the motherboard. This one has both 8's plus the 4 pin, so it was actually perfect for this application, like it was made for it! :-) I now have all those extra 6 and 8 pins that I don't even use, usually it's the other way around. Lucky I had it up on the shelf unused.

I have both the new one and CrunchMonster upstairs in the same room, and since I've fired LotzaCores2dot0 up, temps have certainly climbed up there. The A/C isn't keeping up in that room, so I think I will be moving it downstairs where it's cooler. Those 750's definitely put out heat even if they aren't doing anything, kind of surprising. Wish I had time today to contact Supermicro, but hopefully tomorrow I can make some time to do that.

ID: 1812334 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1812349 - Posted: 25 Aug 2016, 2:26:19 UTC - in response to Message 1812334.  

When I needed to replace the evaporator in the house A/C, I upgraded from a 3 ton to a 4 ton unit and had a dedicated conduit routed into the bedroom that has housed my main two crunchers. Certainly has helped the temps in the summer. In winter I never need to run the gas furnace, I have 1.5KW of space heaters (computers) :-) keeping me warm and comfortable.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1812349 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 . . . 8 · Next

Message boards : Number crunching : The Saga Begins (LotsaCores 2.0)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.