In lieu of this discussion on ArsTechnica's forum's, and my own frustration with my newly purchased gigabit ethernet router, I decided to use IPerf as suggested in that thread. To my dismay, I am only getting roughly 350MB/s throughput (out of what should be 900MB/s total throughput [90% saturation with Cat5e and gigabit protocols]). I also noticed that real world file transfers are slower than what I'd expect for GigE, at around 3MB/s. I tend to do a lot of large file copying and archiving (100MB-9GB file sizes with total datasets of up to 20GB or more) over my home network, so I was hoping the performance of GigE would cut my times and increase my productivity more than they have.

I used IPerf from my GF's machine and a third node on my network to see if it was just my connection from my machine, but hers is getting about 350MB/s throughput too, while the third node, being connected to a 100Mb switch linked to the GigE router is getting a respectable 98MB/s throughput (respectable for 100Mb, that is).

In the aforementioned thread on ArsTechnica's forum, they suggested server configuration and CPU speed are very limiting factors. I figure my old dual Pentium III 700MHz machine with 1GB PC100 ECC SDRAM might just barely be there in terms of CPU power. However, my Supermicro P6DBE motherboard only has standard 32bit, 33MHz PCI slots with a shared throughput of only 133MB/s between all five slots. I have my Intel GigE card in one slot which should be able to easily use up about 100MB/s by itself, and my Promise Ultra100TX2 HD controller card in another slot, which can easily chew up another 100MB/s too. My conclusion is that my server's I/O are overloaded and I need more throughput.

So I had the Tech Department talk to the Sales Department (yes, I talked to myself) to have them pitch a new file server to the Purchasing Department (my GF). It must have been one helluva sales job because she agreed!

My new system will be based upon Intel's STL2 server board with 4 standard 32bit 33MHz PCI and 2 64bit 66MHz PCI slots, 2 Intel Pentium III 1.4GHz 512K L2 133MHz FSB CPUs and 4GB PC133 ECC Reg. SDRAM (system will be maxed out; total purchase price of about $340 with parts on eBay).

Both the Intel GigE adapter and the Promise Ultra100TX2 cards will be re-used in the new server (along with all three storage hard drives). Both cards state that they can be plugged into 64bit slots, but do not state if they are specifically 64bit and/or 66MHz compatible.

What I am looking for from anyone reading are experiences or suggestions. Does anyone know if my two PCI cards are 64bit or 66MHz compatible? 64bit 33MHz will still give me a total of 266MB/s throughput, as will 32bit 66MHz. 64bit 66MHz will give me 533MB/s total throughput, which will be the most helpful, but I must be using compatible cards to get those speed improvements. A new Intel 64bit 66MHz GigE adapter will be another $160 that I'm hoping to avoid, but will purchase if I have to so I can cut my times down.

Also, I just might give Linux a try on the new server, though honestly I'm not too concerned with security (the server will be secure enough on my network). I am, however, concerned about how easy it would be to A) integrate into my Windows-only network and B) if I can still use my NTFS formatted storage disks without having to format them using EXT or other file systems. What distro would you suggest for a Linux beginner that would be suitable as a file server yet still be easy to configure?

Thanks in advance to anyone interested, and sorry for the long post! 8-)

---

Addendum: I finally found the manufacturer specs online for both the Promise Ultra100 TX2 and the Intel PRO/1000GT NIC, and both state they fully support the 32bit 66MHz PCI bus specification, which will give me roughly 266MB/s shared between the two cards. Should this be enough to support GigE speeds and high HDD transfer rates?

---

Addendum: I finally found the manufacturer specs online for both the Promise Ultra100 TX2 and the Intel PRO/1000GT NIC, and both state they fully support the 32bit 66MHz PCI bus specification, which will give me roughly 266MB/s shared between the two cards. Should this be enough to support GigE speeds and high HDD transfer rates?

In reply to your total posts, you shouldn't use Linux on an NTFS partition. It was just a few months ago, well maybe a year that if Linux even WROTE to an NTFS partition it would be instantly trashed. It's because Linux couldnt write the journal information properly and Windows wouldn't know how to correct it. Write support for NTFS was introduced and AFAIK it is still experimental. I wouldn't use NTFS for a primary Linux partition anyway. Reiserfs is a much safer alternative, as it is a journaling file system like NTFS but 100% Linux native.

Also do you have any 100mb/s cards or equipment in your system? I have found that even though 99% of your system may be high speed, just plugging in one slower speed item will drop the entire network speed. I haven't reread your post, but I got the impression that you have a mixed network of 100mb/s and gigabyte. If this is so you will only get 100mb/s out of it unless you replace all 100mb/s stuff with gig.

---

Jim

Some people plan their life out and look back at the wealth they've had.
Others live life day by day and look back at the wealth of experiences and enjoyment they've had.

In reply to your total posts, you shouldn't use Linux on an NTFS partition. It was just a few months ago, well maybe a year that if Linux even WROTE to an NTFS partition it would be instantly trashed. It's because Linux couldnt write the journal information properly and Windows wouldn't know how to correct it. Write support for NTFS was introduced and AFAIK it is still experimental. I wouldn't use NTFS for a primary Linux partition anyway. Reiserfs is a much safer alternative, as it is a journaling file system like NTFS but 100% Linux native.

Hi Jim,

Yeah, I wouldn't have used it as my primary partition anyway. I would have set Linux on its own drive (40GB) attached to the motherboard, while the (correction: 2) storage drives would be attached to the Promise Ultra100 TX2 card. Both drives (a 750GB and a 300GB) have data on them and would really take more time than I'm willing to spend to back up and format for Linux then restore the data. That is why I was hoping that I could install Linux on the primary HDD and let it access the other drives natively.

But I understand that this isn't fully supported (I kinda figured it wouldn't be, MS would probably get very angry if Linux even touched any of it's own proprietary stuff), so I guess I should stick with Windows 2000 Server.

Also do you have any 100mb/s cards or equipment in your system? I have found that even though 99% of your system may be high speed, just plugging in one slower speed item will drop the entire network speed. I haven't reread your post, but I got the impression that you have a mixed network of 100mb/s and gigabyte. If this is so you will only get 100mb/s out of it unless you replace all 100mb/s stuff with gig.

Sorry for the confusion. The server only has a single GigE adapter made by Intel, which is connected to a gigabit Netgear router. The Netgear router has 4 ethernet ports. Port 1 is connected to my system (Xeon 5130), port 2 is connected to my GF's system, port 3 is auto-linked to a Netgear 24 port 100Mb switch and port 4 is connected to the file server. The indicator lights show that Ports 1, 2 and 4 are all operating at GigE speeds while port 3 (the switch) is operating at 100Mb, as expected.

It would seem that indeed, my network is operating at two different speeds. I would imagine that if the 100Mb is slowing down the network, that IPerf would show the other systems as getting the 100Mb speed (98MB/s), but instead they are getting triple (350MB/s) the 100Mb speed yet they are not getting the full GigE boost (up to 900MB/s).

According to the manuals, both the Netgear gigabit router and the Netgear 24 port switch support mixed networks of 10Mb/100Mb (switch) and 1Gigabit (router) speeds.


Thanks for your input!
Also, I've done some more testing and I noticed that only the write speeds are abysmally slow (3MB/s). Read speeds are as high as 10MB/s.

---

But I understand that this isn't fully supported (I kinda figured it wouldn't be, MS would probably get very angry if Linux even touched any of it's own proprietary stuff), so I guess I should stick with Windows 2000 Server.

If the drives you're talking about don't contain the windows os itself you should be alright mounting them and using them to store or retrieve data, but with the knowledge that the write to NTFS IS experimental so it may or may not crash the partition. I use two partitions for my windows (I only use Windows to play games) one small one ~5g for the os itself on an NTFS, and a second large partition for my other programs and data in FAT32. I mount the windows partition read only, so I can copy info off of it if need be, then I mount the data partition read/write to transfer data from Linux to Windows. (I even have some programs that both access the same data such as my media player, etc. both storing their data on the FAT32 partition so I only have to keep one copy of the data and it's accessible from both oses.

PS, I run Slackware 11.0 on my main computer and it was listed as experimental on that. Slackware 12.0 is out now and they may have upgraded write support on NTFS to mainstream stable now. Don't know but I'm going to check it out, since I'm going to upgrade very soon. (I like keeping this machine on the "bleeding edge" even though it means a few crashes now and then! The stable improvements are worth it in my opinion.)

The indicator lights show that Ports 1, 2 and 4 are all operating at GigE speeds while port 3 (the switch) is operating at 100Mb, as expected.

It would seem that indeed, my network is operating at two different speeds. I would imagine that if the 100Mb is slowing down the network, that IPerf would show the other systems as getting the 100Mb speed (98MB/s), but instead they are getting triple (350MB/s) the 100Mb speed yet they are not getting the full GigE boost (up to 900MB/s).

Good enough on the network speeds. My friend had a Linksys router that evidently support mixed networks. I took a 10mb/s 16 port hub down there one day so we could network a couple computers in his grandmother's basement and still connect to his main network at his house beside hers. As soon as we plugged the 10mb/s hub into the network, the entire network slowed down to 10mb/s!
I think he's still got it and uses it when he has to plug more computers in, then unplugs it when he's not needing it.

Have you checked all your cables for good connections? Interference from outside noise sources? Extremely sharp bends in your cable? I think the minimum bend radius is like 3 inches for cat5. hmm that's about all I could think of that would cut down the throughput unless it was something hardware related.

I wish I could afford to play around with stuff like this more, but I am on disability so all I can afford is to get stuff at flea markets, and stuff where companies upgrade equipment, etc. The 1.3ghz comp I'm on now I traded a cousin a 1ghz AMD computer for it. He got it out of a dumpster where this company upgraded their computer and just threw this one in the dump!
I got the 16 port hubs at a "hamfest" (an amateur radio flea market) for 5 bucks apiece, and a friend has the above mentioned linksys router he said I could get any time I needed it, but I've set up a Linux box for a router feeding into the 16 port hub so except for slowing down my comp to comp speeds it works fine. I'm on DSL which is capped off at 6mb/s so it doesn't slow down anything really important!

Good luck finding your bottleneck! Wish I knew more about networking to be able to help.

---

Jim

Some people plan their life out and look back at the wealth they've had.
Others live life day by day and look back at the wealth of experiences and enjoyment they've had.

For networks, we're still stuck in the days of the 1960's and ARPAnet with meager mtus set at just 1500 bytes...

For Gbit transmission rates, this puts a very high interrupts loading on the NIC and CPU to move the very high rate of packets to get Gbit well utilized...

Take a look at Increasing network performance for various parts of the story.

In summary, if you have full control of your network, then you may see a big gain by moving to an mtu of 9000 ('jumbo frames'). Also, look at opening up multiple parallel connections across the LAN.

Your next bottleneck is then what write speed you can get through your host CPU and then onto the target HDD.


Aside: I consider ReiserFS 3.6 to be very much superior to NTFS, especially so if you work with lots of small files or extremes of directory sizes.

Good luck,

Regards,
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

The problem you have is harddisk related. Considering you are using ATA100 connector, your drives are most likely topping out at 20-30mb/s write 30-45mb/s read. That translates to 160-240mbps writes and 240-360mbps reads.

The only way you gonna make full use of your GBE is if you go for a hardware raid controller running 4 drives in raid 0. And a much faster cpu & newer chipset to handle the load.

---

PS, I run Slackware 11.0 on my main computer and it was listed as experimental on that. Slackware 12.0 is out now and they may have upgraded write support on NTFS to mainstream stable now. Don't know but I'm going to check it out, since I'm going to upgrade very soon. (I like keeping this machine on the "bleeding edge" even though it means a few crashes now and then! The stable improvements are worth it in my opinion.)

Have you taken look at linux-ntfs and ntfs-3g projects? ntfs-3g claims to be stable with read/write access to NTFS. If I'm reading the site right you can get it to Slackware 11.0 too.

-Juha

---

The problem you have is harddisk related. Considering you are using ATA100 connector, your drives are most likely topping out at 20-30mb/s write 30-45mb/s read. That translates to 160-240mbps writes and 240-360mbps reads.

The only way you gonna make full use of your GBE is if you go for a hardware raid controller running 4 drives in raid 0. And a much faster cpu & newer chipset to handle the load.

Thanks for the input. I figured my HDD throughput might be a limiting factor, which is why I was hoping that moving the controller to a faster bus might help out a little bit.

Unfortunately, I have no intention of moving to RAID 0. I do not like the idea of halving my potential error rate. The potential loss to my data is too great to bear. I have, however, thought about RAID 5, but I don't have the capital to purchase more drives at this point, so I was hoping I could get away with this cheaply by moving things to a faster system. We'll see if I'm right. Otherwise, I might have to take a different route.

---

Take a look at Increasing network performance for various parts of the story.

Thanks Martin. At this point, I'm not convinced that my transmission size is causing my problems, but it may be worth looking into if I get stumped and can't find any other options.

---

... Unfortunately, I have no intention of moving to RAID 0. I do not like the idea of halving my potential error rate. The potential loss to my data is too great to bear...

Then use a fully mirrored system. That can be very easily done by using a (slow, cheap,) slave machine that just rsyncs all the data from the master/main machine.

I would also include versioning to avoid having a corrupt file on the master then overwriting the good file on the mirror.

And I consider RAID to be greatly overrated. If you are going to use it to overcome the expected failures from using a large array of inexpensive disks, then fine. However, you cannot rely on RAID to robustly protect your data from loss.

I've seen and heard too many cases where the raid controller has scrambled the lot, or a PSU failure scrambled multiple disks that then caused the raid to scramble the lot.

Much better than RAID for securing your data is to have a completely separate automatically updated copy independently somewhere else.

Another downside for RAID is that although you can get higher data throughput, you usually suffer greater latency...

Good luck,
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

And I consider RAID to be greatly overrated.

That makes two of us that feel that way.

Much better than RAID for securing your data is to have a completely separate automatically updated copy independently somewhere else.

Unfortunately, I don't have that kind of money. I'm very short on cash as is (which is why I'm surprised she let me buy my upgrade; price probably had a lot to do with it, but double for another system she'd never go for).

---

Just a thought, to throw in another option what about a small NAS unit?

This one here as example does gigabit speeds (sorry Australian site because that's where I am :D)

I am pretty sure there are better ones that have ftp servers etc.. built in but that's more money of course.

---

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.

Just a thought, to throw in another option what about a small NAS unit?

This one here as example does gigabit speeds (sorry Australian site because that's where I am :D)

I am pretty sure there are better ones that have ftp servers etc.. built in but that's more money of course.

A fine suggestion (one that I had considered, and am still considering in a distant future purchase), but I already have an investment of PATA drives in my current file server. With that unit, I'd have to buy two new SATA drives to put in there.

---

A fine suggestion (one that I had considered, and am still considering in a distant future purchase), but I already have an investment of PATA drives in my current file server. With that unit, I'd have to buy two new SATA drives to put in there.

Oh! no PATA version is cheaper. here But not gigabit :(

[EDIT: I wonder if the even cheaper ones that attach to USB2.0 drives would work fast enough (some of them say gigabit), USB enclosures for those PATA drives and duct tape the whole lot together :P]

---

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.

Oh! no PATA version is cheaper. here But not gigabit :(

[EDIT: I wonder if the even cheaper ones that attach to USB2.0 drives would work fast enough (some of them say gigabit), USB enclosures for those PATA drives and duct tape the whole lot together :P]

Then I lose the gigabit speeds. Not really worth it if I'm going to lose bandwidth. :(

A USB enclosure would be fine for a single system, but I need my files to be available to everyone on the network without bogging my system down as a server to the other nodes. Hence, that is why I have a separate file server. I just need to improve the performance of the existing file server.

That's why a NAS or SAN would be a great idea, but I've already invested in this server upgrade. Perhaps my next upgrade will be to a NAS or SAN.

---

Then I lose the gigabit speeds. Not really worth it if I'm going to lose bandwidth. :(

If you are using a switch, and you mix gigabit and 100 megabit (or even 10 megabit) devices, the only time you'd slow down is when a faster device is talking to a slower device.

Two gigabit devices would still talk at gigabit speeds.

There are some incredibly inexpensive SAN devices. I've got a couple of these and they work great for the money (even if they're kinda slow).

---

A fine suggestion (one that I had considered, and am still considering in a distant future purchase), but I already have an investment of PATA drives in my current file server. With that unit, I'd have to buy two new SATA drives to put in there.

Any old (reliable) PC + Linux will give you that. Monitor and keyboard not required even.

The other alternative is a USB2 attached drive or drive enclosure.

Good luck,
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

Then I lose the gigabit speeds. Not really worth it if I'm going to lose bandwidth. :(

If you are using a switch, and you mix gigabit and 100 megabit (or even 10 megabit) devices, the only time you'd slow down is when a faster device is talking to a slower device.

Two gigabit devices would still talk at gigabit speeds.

Precisely. The PATA SAN device Jason suggested is only 10/100Mbit. If I access it from my GigE system, I would be slowed to 100Mbit accesses, which I upgraded from after I bought my new GigE router. I would have to buy a GigE SAN, which I would love to do, but I only had a small amount of funds available for this purchase, which I already did in the form a new server parts.

My hope was to increase network performance for as cheap as possible based upon my very limited funds. My theory is that the PCI bus is over-saturated on my existing server, so I am hoping that by upgrading to 32bit 66MHz PCI would offer enough of a performance boost to alleviate my problem. The only way to get the faster PCI bus is through a newer server motherboard, which required new CPUs and new RAM (I didn't want to reuse my old PC100 with these FSB133 CPUs and create another performance issue - besides, the PC133 RAM was fairly cheap on eBay).

I was essentially wondering if anyone had any experience with a similar configuration and if they'd be able to tell me what to expect from my upgrade.

Or perhaps people are telling me I chose the wrong upgrade path? Perhaps I should have gone a different route? If so, then I guess I'm stuck until more funds are available (probably next year) and I'll have to make do with the new hardware for now.

---

Any old (reliable) PC + Linux will give you that. Monitor and keyboard not required even.

Will any old PC + Linux be able to withstand high GigE throughputs? It is my understanding that GigE requires a decent CPU in order to sustain such speeds and large file transfers.

And as suggested, I'm not sure an old PC would be able to do the trick. I think it's an over-saturated PCI bus that's the issue. An old PC would also have standard PCI slots which would still pose a problem. I would have to have PCI-X (64bit PCI, 66MHz) in order to attain the I/O throughputs I require while also using a Promise controller card, which is also saturating the PCI bus.

Unless the performance bottleneck is in the controller-to-disk area, then faster disks would be my only option. I'll wait to see if faster PCI I/O helps first before looking elsewhere.

The other alternative is a USB2 attached drive or drive enclosure.

I would imagine that a USB2 drive would only be ideal for a single machine. The other nodes on my network would not be able to access the data unless I shared the USB2 drive, which I hear is not a good idea for removable drives to be shared. Even if I were to share a USB2 drive, I would still be limited via USB2's increased latency. Wouldn't a 32bit 66MHz PCI controller give faster performance?

Remember, not only am I running GigE through the PCI bus, but I'm also running a Promise Ultra100 controller card through the PCI bus. USB2 is attached through the PCI bus as well (in the south bridge), which would only be moving the problem around. I don't think it would solve the issue in network performance.

---