that seems to be the max, but i'd say judging by the many issues others have had in the mining community with that board, some might just have "issues".

there was one person running that board on seti who had 13x 1080tis. they were briefly on the top of the leaderboard for Hosts, but they kept having issues. they dont post here so we can only speculate that they just couldnt resolve them, or didnt care to resolve them and just gave up to do something else, like go back to mining or something. we'll never know for sure.

but looking at their task run times and stderr.txt files, the system clearly had problems. and even when it wasnt having problems the per-card run times were about 25% slower than those with lower car counts. my theory on that one is CPU bottleneck. the board can only support CPUs up to 8 threads. and it becomes a struggle to drive 13 cards. just my speculation of course.

you could look into the ASUS H370 Mining Master if you truly cared to get as many GPUs in one system as possible. I've seen it work up to 14 GPUs (after that fails, just like the B250 does). The plus with that one, is that it will support up to 16 threads as it supports the i9-9900k. but you're in unknown and expensive territory at that point. and there might be even more consequences to cramming that many cards through the DMI 3.0 interface. it's a cool thought though.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

I see, so the magic # is 13...

Gotcha!

maybe you missed my post previously.

you can run a MAX of 13 GPUs on that board. 1 in the main x16 slot with lanes from the CPU and the 12 x1 lanes provided on the B250 chipset.

the only way to get 19 cards on that board is is you use 6 mining specific GPUs, these are P106-100 and P104-100 cards. they have no display outputs, and are designed really only for mining. it still has more or less normal GPU hardware (the P104 card for example is a GTX 1070 core, with 4GB of DDR5X memory), but I'm unsure if they could be used by SETI, i doubt anyone has tried. Your bigger problem with that many cards is CPU resources.

ASUS also for the most part makes no official support for linux on a lot of their motherboards. I've seen more than a few times where someone had a linux problem, asked ASUS about it and they said they dont support linux and to run Windows instead. It doesnt mean it wont work of course, many people including myself have ASUS boards with linux here on SETI, just that ASUS wont bother to fix anything regarding linux.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Maybe right...

It will be interesting to see if the published LIT from ASUS turns out to be correct WRT how many of one type of GPU will work on this platform.
I have yet to see anyone with all 19 slots working the same model card.
Although they refer to this limitation in the mining mode, it remains to be seen if BOINC/Linux/Seti can overcome this reported limitation.

The last time I tried, I was not very successful at getting more than one or two more GTX 1060's working in the second bank.
I dont recall if I was ever able to get more than that working simultaneously.

Even if it is possible to get all 19 slots populated and crunching for BOINC/Seti, I wonder why then ASUS tells B250 mobo owners it cant be done?
How much more different is mining mode than what I'm trying to max out??

Hummmm....

I think the problem is related to ASUS not having such a good reputation with using PCIe switches in Linux. If you look at the Specs, the B250 has 12 PCIe lanes in the Chipset, Max # of PCI Express Lanes = 12. That means, to have more than 12 x1 slots the board must use a PCIe switch. nVidia has a thread about PCIe switches in Linux, and specifically about some ASUS products, ...when you start splitting PCI-E busses with bridge/switch chips you can encounter issues not present in more straightforward bus to device implementations.
The board I'm using has 11 PCIe3 x1 slots, and claims, 12 PCIe native lanes directly from chipset The board you are using doesn't make that claim. I think it must be using a switch, and ASUS calls it a Mining Board, NOT a Linux board. The Gigabyte board I'm using seems to be running Linux quite well with it's Native PCIe slots.

Look at what BIOSTAR says about PCIe switches and the B250 board,
Native PCI-e for Best Compatibility

NO Switches

Have you checked the Price on that 'Expert' board lately,
$149.99
$99.99
Save: $50.00 (33%)!!!
It's cheaper than some of the "Non-Expert" boards now.

Right now, if my math is right, I have 14 machines allocated providing 73 cores or 146 threads for a total of ~244 GFLOPs, 6 STD stock laptop GPU's running windows for a total of ~267 GFLOPs and 17 GTX 1060 GPU's under Linux on 5 PC's with the BOINC special app providing ~66 TFLOPs for a total of 66.8 TFLOPs.

Whew!

It will be interesting to see if the published LIT from ASUS turns out to be correct WRT how many of one type of GPU will work on this platform.
I have yet to see anyone with all 19 slots working the same model card.
Although they refer to this limitation in the mining mode, it remains to be seen if BOINC/Linux/Seti can overcome this reported limitation.

The last time I tried, I was not very successful at getting more than one or two more GTX 1060's working in the second bank.
I dont recall if I was ever able to get more than that working simultaneously.

Even if it is possible to get all 19 slots populated and crunching for BOINC/Seti, I wonder why then ASUS tells B250 mobo owners it cant be done?
How much more different is mining mode than what I'm trying to max out??

Hummmm....

Well, there ya have it!

PCI set to GEN1, 4G disabled = 5 GPU's MAX.
4G enabled = 7 GPU's (So far)
4G on AND PCI to GEN3, mobo has boot issues AND NIVIDA only sees four GPU's!!
4G on and PCI set to GEN2, Linux/NVIDIA see all 7 GPU's!!

Could still be a power supply issue. Higher data rates across most any digital bus architecture can get odd results if starved for power.
Oddly enough, with 4G on and PCI set to GEN1, NVIDIA reports all GPU's at performance level 2!
With 4G on and PCI set to GEN2, they report performance level 3!

Go figure!

Looks good at the moment! As long as the MINER can continue to run ~2 minute miles, I'm pleased.
When I get those better power supplies, I will try to populate the second section with as many as I can get working.
Right now, the platform is as stable as I have ever had it so its in a good posture to try and tinker up the # of GPU's until I reach max saturation.

what does nvidia-smi say? does it show that it's there?

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Stranger than that! Not only does the mobo boot but the OS works fine! The sixth GPU is just ignored!
Weird. Even that NVIDIA driver tool that shows you all the GPU's does not see the sixth card...
I didn't see a "4G" setting in that BIOS but I will go back and look.
Mining mode if off but when i turn it on for one boot the extra card is good and working.
I have tried a number of different cards for #6 so I am sure its not the card...

I opted to remove those GPU's from the Windows box as it doesn't make sense to run them there while I have so many working Linux systems to run good GPU's in.
I got the MINER back up and running BUT, for some unknown reason to me, it will not let me attach more then five GPU's.
It may be a limited power supply issue. I have new ones on the way.
If that allows me to get the last two cards working that I have left, I will be "full on".
If that doesn't fix the issue, then I have one more box that will let me install the last two GPU's when I get those new power supplies.

when you say not more than 5 GPUs, what happens? it doesnt POST? or doesnt boot the OS?

go into your BIOS settings and make sure the setting for "Above 4G Decoding" is set to Enabled if it is not already. you need that for high GPU counts.

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

I opted to remove those GPU's from the Windows box as it doesn't make sense to run them there while I have so many working Linux systems to run good GPU's in.
I got the MINER back up and running BUT, for some unknown reason to me, it will not let me attach more then five GPU's.
It may be a limited power supply issue. I have new ones on the way.
If that allows me to get the last two cards working that I have left, I will be "full on".
If that doesn't fix the issue, then I have one more box that will let me install the last two GPU's when I get those new power supplies.

Eng4hire,
I was looking through your "farm" and stopped to look at this system https://setiathome.berkeley.edu/results.php?hostid=8575791&offset=0&show_names=0&state=2&appid=

In general, you Windows SOG tasks should be running near 7-8 minutes.

So I am wondering if they are being CPU starved in some way.

Ideas to try include: Try renaming the "app_config.xml" and then restarting BOINC Manager. This will drop the CPU -> gpu ratio to the default.
Make sure you are using this command line in the Seti directory for the "MB*SOG.txt" command line file.

-sbs 1024 -period_iterations_num 10 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64

Don't run more than one gpu task on a gtx 1060 3GB (or 6GB) for that matter. :)

Are you running the MB in Mining mode or standard in the Bios?
Can you push the PCIex to at least Gen2?

HTH,
Tom

---

"I owe", "I owe", "Its off to work I go" (from a bumper sticker on a smallish Mercedes Benz)
(on the back of a Semi Tractor) "If you can read this bumper sticker, I've LOST MY TRAILER!"

Almost forgot. If you are running different models of video cards and you want Seti to use all the cards you need this your "cc_config.xml" file. In the BOINC folder.

<use_all_gpus>1</use_all_gpus>

It appears to need to be inside <options></options> and those needs to be inside of <cc_config></cc_config>

There is a good chance that file already exists and you just need to search for "use_all_gpus" and switch it from 0 to 1.

Tom

---

"I owe", "I owe", "Its off to work I go" (from a bumper sticker on a smallish Mercedes Benz)
(on the back of a Semi Tractor) "If you can read this bumper sticker, I've LOST MY TRAILER!"

I just added a 3rd gtx 1060 3GB to my e5-2690v2 box. No fuss, just slid it into the half PCi slot I had open at the "bottom" of the MB. Added the 6 pin power plugin and screwed down the usual thumb screws. This keeps each of my gpu's the furthest apart that I can manage for maximum cooling.

And booted. Didn't change anything (I have just reset it to the bios defaults except for, turning off virtual services function, internal VGA primary and a custom power plan which keeps the CPU(s) running in turbo mode).

Getting turbo mode to run constantly on the CPU of an AM (American Megatrends) bios.
So far I have been able to get the turbo boost to enable only if I have SpeedStep enabled. I also have a "Cx" parameter then I have set to the lowest available # (which is 0 in my case). I have played with the report C3 and C6 states to the OS and am not sure I have seen any difference as long as the Cx parameter is set to 0. If it offers a "power plan" set it to performance. :)

HTH,
Tom

---

"I owe", "I owe", "Its off to work I go" (from a bumper sticker on a smallish Mercedes Benz)
(on the back of a Semi Tractor) "If you can read this bumper sticker, I've LOST MY TRAILER!"

Also, I think I asked this once already, is there any way to mix AMD cards and Nvidia cards in this hyper Linus app environment?

Yes. Basically, the command line files are specific to each application. And it is possible to set different amounts of CPU to gpu for each type of app/card in the "app_config.xml".

The mixing of video cards is not common but does work in both windows and in Linux. The OS install would be with both types of cards on the MB during the install.

To add a 396 or 410 Nvidia driver you can do this:

Adding this PPA to your system
You can update your system with unsupported packages from this untrusted PPA by adding ppa:graphics-drivers/ppa to your system's Software Sources. (Read about installing)

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update

Then run the "add drivers" in the preference menu. Select 396 or 410. Reboot and Linux/CUDA91 is good to go.

I don't have information about adding AMD drivers though :( Probably need to "ask Mike" :)

I just saw a thread on running a mixed card environment here: https://setiathome.berkeley.edu/forum_thread.php?id=83523#1964125

HTH,
Tom

---

"I owe", "I owe", "Its off to work I go" (from a bumper sticker on a smallish Mercedes Benz)
(on the back of a Semi Tractor) "If you can read this bumper sticker, I've LOST MY TRAILER!"

Quite a few folks hanve reported much the same sort of thing. Things to consider are the psu, some with multiple 12v outputs are ok, but most are not, and do not share the 12v input to one card from two 12v lines as that can cause "intersting" problems; the quality of headers is very variable and price is not a good guide; always make sure you use a header which supplies 12v as most gpu need a few watts coming in from the mobo connector.
There probably a load more considerations.
One thing to try is to make sure that a gpu wil run in the header before going for all three gpus in place - chech one mobo gpu plus one on heeader, and try the header in different mobo slots....

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Also, I think I asked this once already, is there any way to mix AMD cards and Nvidia cards in this hyper Linus app environment?

Hey Tom,

Ya know, several of the PC's I am running have more than two PCI slots.
Even though I can only fit two double wide GPU's in each case, I can get to the other PCI connectors using the riser card method to add another GPU but none of the systems I have "like that". When I try to do that, it drives the stable working Linux OS and or BOINC MGR bananas!

Any idea why or how to solve that? I dont mind having GPU's hanging around outside a case or super glued inside...

Dave

Okay Tom but there's no way to force what kind of units it gets right?

The anonymous techniques will allow you to "pick" the apps you want to run but as rob smith said "its tricky". I would advise patience. Probably in less than a week the Nvidia gpu tasks will be sorted out and be running the fastest choice for your Nvidia setup.

If you are getting different Amd gpu tasks, they will also settle (probably) on a app that is running the fasted for your video card(s).

In about that same time, the fastest processing cpu tasks should be decided.

After that you can just sit and admire them as they zoom by and your RAC goes up ;)

But, You can probably speed up the total production of the system by running at 90% of the available cores.

You can probably speed up the GTX 1060 some by using a command line like: "-sbs 1024 -period_iterations_num 10 -spike_fft_thresh 4096 -tune 1 64 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 64 -oclfft_tune_cw 64" This is from a fellow GTX 1060er named "Wiggo".

There is a guy named Mike (of Mikes World) over in messages -> "number crunching" who can offer advice the best command lines for Amd gpu's. He is responding on the "Gpu Gflops Theory and Reality"(sp?) at the moment.

Tom
ps. Once you get settled into the Windows Seti world you can explore the Lunatics Distro for a ~30% improvement in processing.

---

"I owe", "I owe", "Its off to work I go" (from a bumper sticker on a smallish Mercedes Benz)
(on the back of a Semi Tractor) "If you can read this bumper sticker, I've LOST MY TRAILER!"