I was wondering if anybody could kindly comment on the read/write speeds in real-life. In theory, I understood that write/read speed is dependent on the type of the SATA controller (mobo SATA controller > PCIe SATA > PCI SATA) as well as HDD and the network bandwidth. I have the parity and cache disk on the motherboard SATA controllers and data disks on the PCI SATA controllers. The HDD on the mobo side - seagate (7200RPM) is in fact slower than the WD green (5400RPM) on the PCI bus which can be seen clearly in the following speed test:  

_________

 

PARITY: WD 2T, on MoBo SATA controller, Timing buffered Disk Read = 86.40 MB/S

CACHE: Samsung 500G (used 0%), on MoBo SATA controller, Timing buffered Disk Read = 78.98 MB/S

DISK1: WD 2T (used 13%), on PCI SATA controller (2 ports, 2 devices), Timing buffered Disk Read = 107.35 MB/S

DISK2: Seagate 1.5T (used 38%), on PCI SATA controller (2 ports, 2 devices), Timing buffered Disk Read = 89.96 MB/S

DISK3: Samsung 300G (used 18%), on MoBo IDE controller, Timing buffered Disk Read = 58.41 MB/S

DISK4: WD 320G (used 0%), on MoBO IDE controller, Timing buffered Disk Read = 62.78 MB/S

 

MoBo: Intel D945GCLF2 (Intel i945GC/ICH7)

Processor: Atom 330 @1.60GHz

Memory (1 x DDR2 667/533): 2GB

NIC: onboard 1000Mb/s RTL 81111/8168B (r8169 driver)

PCI SATA-II Controller (66MHz): Initio Co, INI-1623

unRAID 4.5.3

 

_________

 

I need two additional SATA drives for which I was even planning to change the motherboard since I knew that conventional PCI bus is crab and should be avoided if possible. The test results do not support this and PCI manages even better than my SATA controller on the motherboard.

 

Any suggestions with regard to the following questions:

 

1) How come that a 5400RPM on a conventional PCI bus wins a 7200RPM drive on the mother board side?

2) if I rely on this and swap the two port PCI SATA controller for a 4-port controller, will I manage to keep these read-speed results?

3) Since writing speed to cache drive seems to be identical to the drive's write capacity, will I be able to have a speed boost of 30-35% (from 79MB/s to 108MB/s) if replace the Seagate cache drive with WD green and put it on the PCI side?

 

PS: Otherwise, I am very happy with the current write/read speeds since I can get a steady 67-75MB/s during writing and 80MB/s during reading to the cache drive using NFS and Mac Os X clients (without cache but with parity, it is only on average 23MB/s). Using samba and windows clients the related figures stay around 45-55MB/s. if I use SMB with Mac Os X, these drop to 35-40MB/s.

 

with many thanks

 

The PCI bus is not slower - it just has a low max bandwidth. For single drive accesses it will not saturate, but when accessing several at the same time (like during a parity chech) the PCI bus will slow things down.

For single drive accesses it will not saturate, but when accessing several at the same time (like during a parity chech) the PCI bus will slow things down.

 

1. Can I assume from this, that I can swap the cache drive to the PCI controller without sacrificing the write speed?

 

2. Since I need space for 2 additional drives, can I fetch a 4-port PCI SATA controller or should I swap the motherboard and get one with more onboard SATA ports or PCIe port?

 

The fact that given drive spins faster does not indicate it can be read faster.  It is one factor, but not the whole picture.

You were quoting buffered reads.  The buffering in the on-disk cache and and linux buffer cache would have a lot to do with perceived performance.   When playing a movie, or performing a parity check the cache is not as helpful, since you almost never find the block of data you are seeking in the cache, it almost always has to be read from the physical disk.

 

When writing to a drive in the protected array, since both the parity drive and the data drive being written must spin approx 2.8 times per set of sectors read, two faster spinning disks (parity&data) will always beat any combination with a slower spinning disk, regardless of PCI bus bandwidth limitations, since the bus is not saturated.

 

Joe L.

Thx Joe. Within this scenario and giving attention to the reading speed (most of the time it will be the streaming of media content SD or HD movies from one disk at a time) what would you recommend for my setting? Should I go and fetch another mobo (which I don't want since I just bought this one), or put a 4-port PCI SATA controller, put the cache drive on the PCI port (and less accessed disks along with it), keep the parity and the most accessed disk on the motherboard's SATA controllers?

Thx Joe. Within this scenario and giving attention to the reading speed (most of the time it will be the streaming of media content SD or HD movies from one disk at a time) what would you recommend for my setting? Should I go and fetch another mobo (which I don't want since I just bought this one), or put a 4-port PCI SATA controller, put the cache drive on the PCI port (and less accessed disks along with it), keep the parity and the most assessed disk on the motherboard's SATA controllers?

In your situation is makes little difference.  (since most of the time you are viewing stored media and not writing)

 

I would put the parity disk and the 2TB data disk on the motherboard's controller.

I've never bothered with a cache drive.  The writes to my older PCI based server are fast enough for me.

 

Joe L.

 

 

I transferred the cache drive now (which ought to be the fastest according to the drive manufacturer) to the PCI controller and the 2T WD green to the motherboard's SATA in light of your suggestions. While WD's read speed remained at around 108MB/S (exactly as good as the figure when it was on the PCI bus), now my cache drive reads at 107.57MB/s (at PCI SATA). On mobo controller it was about 79MB/S (vs. 35% increase when switched to conventional PCI bus). However, this did not effect the real life write/read speed. I cannot get over >70-75MB/s write and 82MB/s read speed during copying a 1.8GB single test file to the cache drive for testing purposes.

A good while back I wrote up a small article for the wiki that seems pertinent to your situation:

 

Improving unRAID Performance - Move Largest and Fastest Data Drives Off PCI Bus

 

Sounds like you've already figured it out, but I thought I would throw it out there anyway.

 

On my end, my current server sports 6 onboard SATA slots and 4 PCI SATA slots (via Promise TX4).  I have only 8 drives, so I chose to place as many drives as possible on my mobo's onboard slots.  My parity and cache drives got first priority, then my largest data drives.  My smallest/least used data drives use up only 2 of the 4 PCI-SATA slots I've got.  Given my configuration, I see no noticeable slowdowns due to the PCI bottleneck.  The reason for this is twofold: first, I use a cache drive to defer writes, and second, I use only 2 drives on the PCI bus, which means that the bus is rarely saturated.  My parity check speeds likely still suffer a bit, but a full parity check still completes within 6-8 hours, which is plenty fast for me.  If I ever feel the need to go past the current 8-10 drives that my server currently supports (which seems unlikely, given the ever-increasing capacity of hard drives), then I'll consider getting a PCI-e SATA card to replace my PCI Promise TX4.

The difference is largely due to areal density.

 

Lets say the 7200 RPM drive has 28 sectors per track at 512 bytes per sector.

 

So that is 103Mb /head/min

 

The 5400 RPM drive has more density... often 1.5x the density (500GB platters versus 333GB platters).

 

So the 5400 RPM drive can have 42 sectors in the same track, which is 116 MB/head/min.

 

So it is slower, but more data is read per revolution due to the areal density.

The difference is largely due to areal density.

 

Lets say the 7200 RPM drive has 28 sectors per track at 512 bytes per sector.

 

So that is 103Mb /head/min

 

The 5400 RPM drive has more density... often 1.5x the density (500GB platters versus 333GB platters).

 

So the 5400 RPM drive can have 42 sectors in the same track, which is 116 MB/head/min.

 

So it is slower, but more data is read per revolution due to the areal density.

 

This is true for sequential access.  But for random access, the faster rotational speed is still an advantage.

Your pci card looks like it might be running at 66Mhz, or 266MB/s. That would mean two drives would happily supported. We'd need to see a syslog to see what drivers and modes the drives are using. The onboard sata might be in ide mode rather than raid or ahci. The areal density might be it but I think that 2TB parity disk would be a 500gb per platter model. All I can think off now.

Thanks to all for the comments. I did some changes, swapped the disks, took the x2 port PCI SATA controller off and put instead even a cheaper no-name x4 port PCI controller (23€). Removed the IDE drives from the array, put the largest data drive and kept the parity on the mobo's SATA.

 

My write throughput without a cache drive (using user shares) went from 23MB/s up to 30-33MB/s (avarage, peaks up to 50MB occasionally).

Write to cache drive: 50-60 MB/s

Read from cache drive: around 70-80 MB/s avarage

Parity check: 66 MB/s (from syslog)

 

And all these figures are from a MAC Os X client. On Win 7, I get 10% increase when writing to SMB shares.

 

Do these figure sound bad/reasonable/good? I ask this question since I am running very modest and cheap hardware and still achieving rather good write/read speeds (in my opinion, I guess) and was wondering if I upgraded the hardware could I get even better results or will the gain be only marginal? The benchmarking page did not help, since even twith this modest/cheap hardware my figures manage well compared to those listed on the benchmark list.

 

Nevertheless, elsewhere and here at these forums quite often I get the impression that unRAID is painfully slow and I realize that some of the users are using rather high-end motherboards and processors to achieve better write/read speeds. I have used various combinations and got various results. My best read/write speeds quoted above, I got when I removed the IDEs from the array, kept the parity on mobo's SATA controller, put the cache disk on the PCI controller. 

 

Long story in short:

 

What would be the theoretical upper limit for max "write" throughput of unRAID using parity w/o cache drive (assuming 6 disks -2TB each- in the array and all are SATA II/III and max 7200rpm, and using a large file less than the memory available, i.e. 2GB here)? I understand from the discussions at the forums that it may not be far away from 40 MB/s. If this is true, why should one invest fortunes to the hardware, processors and controllers using unRAID if this could be "almost" achieved with minimal investment and 30-50W of energy consumption?

 

 

 

your cache drive speeds look normal, i dont understand your argument for the speeds without a cache drive. what do you mean expensive hardware?

 

also, good luck getting 6 hdds below 50W.

I am sorry if I made myself misunderstood. With expensive hardware I meant the dedicated RAID controllers which are rather expensive in Finland as well as some mobos like supermicro's or server mobos. I did not intend to criticise anybody but just wanted to gain some more insight. Without cache drive, I meant the data disks in the array don't use cache drive but the data is written directly with parity.

 

For example, can I increase my write speed (without using cache drive) above the current 30-33MB/s if I bought a better mobo or processor (vs. mine 1,6G atom processor and the old Intel board with only 2 SATA and one PCI controller on board)? Could 2 channel DDR3 memory make a difference instead of the current 1 channel DDR2? If I ran all SATA disks using the mobo controllers (which means I shall replace the mobo and get one with e.g 6 onboard controllers) could this make a difference? If all this upgrade took place, how much would I gain in write speeds?

 

In other words would an upgrade effect just the "expandability" (more hard drives) or the "performance"?

 

ps: 50W is only assumed not measured and when idle - I meant the situation when all HDDs kept spinned down or with cache drive used for immediate writing. The processor consumes about 10w, the chip takes 25W. And memory probably 5W? One green WD HDD 5-10W when idle? This argument is for myself against using S3 sleep for 24/7 use. When I use S3, I get occasional sync errors and WOL is a painful task for the other family members. Instead when I use it 24/7, the consumption will be around 50-60W. When all HDDs powered of course it will be at least twice as much.

 

 

All of the following applies to the stock unRAID scenario (no add-ons):

 

Upgrading your CPU won't affect your transfer speeds.  I've used a single core 1.6 GHz, a dual core core 2.0 GHz, and I currently use a single core 2.7 GHz CPU.  My transfer speeds have been consistent throughout.

 

Upgrading your motherboard can affect your transfer speeds, depending on other factors.  For example, simply upgrading the number of onboard SATA ports (going from 2 to 6, in your case) will allow you to have more drives on fast ports, and off the PCI bus.  Replacing your PCI card with a PCIe card (assuming your current motherboard supports it) will have the same effect.

 

Upgrading your RAM won't affect performance in any noticeable way.  DDR2 vs DDR3 doesn't matter.

 

WD Green drives actually use only 1 W when idle, and I think about 5-8 W when spun up.  Getting an idle server below 50 W is definitely doable.

 

I'm not too interested in theory, but in practice the max write speed of an unRAID server without a cache drive is about 40 mb/s.  With an SSD cache drive, its about 70 mb/s.  With a HDD cache drive, its about 60 mb/s.