Array Write Speed - Is mine OK at approx 50MB/s

[Marshalleq]

I was talking to a guy on another thread today and he reckons this is not normal.  Before I go down the rabbit hole of looking for improvements I thought I'd check in again on if other people think this is normal.

 

I have 8 Seagate Enterprise Capacity SATA drives, all flashed to latest firmware and a Seagate Ironwolf 8TB.  Their speed is all roughly identical - I've re-done disk speed tests on them today (screenshot below) to ensure there is not an individual drive issue nor an obvious cable issue and it all looks good.  They're attached to a Dell Perc H310 flashed to IT mode which has PCIe 2.0 x 8 lanes which has more than enough bandwidth.  

 

I had assumed this low speed was due to parity, but another member says this is not normal and not what he gets.

 

I have previously done some optimisations using the script to find fastest disk / raid settings, but have reverted to default to keep system as vanilla as possible while testing something.

 

Is anyone with a similar configuration able to show me that they get more speed?  Note that the kind of copy where I get these slow speeds include a large file transfer (not over network) using rsync --whole-file, from the cache drive to spinning disks.  The large file transfer is a folder of multi-gig iso files (not small files).

 

 

Many thanks for your help.

 

Edited August 19, 2019 by Marshalleq
It is polite to say thank you...

[itimpi]

That sounds like a good speed for writing to the parity protected array   you might get better speeds if you have Turbo write enabled but that is at the expense of needing all drives to be spinning which is not the case for the default mode.

[Marshalleq]

Thanks, that's what I thought.  Although I am often in disbelief that it is that much slower than a single drive, but so be it I guess.

[itimpi]

2 hours ago, Marshalleq said:

 Although I am often in disbelief that it is that much slower than a single drive, but so be it I guess

That is because each 'write' operation is not simply a write.  it involves:

1. Reading the appropriate sector from both the target drive and the parity drive(s) in parallel.
2. Calculating the new contents of the parity sector(s) based in the changes to the target drive data and the current parity drive(s) data.
3. Waiting for the parity drive(s) and target drive to complete a disk revolution (as this is slower than the previous step)
4. Writing the updated sector to the parity drive(s) and the target array drive. 

in this mode there is always at least one revolution of both the parity drive(s) and the target drive (whichever is slowest) before the 'write' operation completes and this is what puts an upper limit on the speed that is achievable. 

 

The Turbo write mode tries to speed things up by eliminating the initial read of the parity drive is) and target drive by:

1. Reading the target sector from ALL data drives except the target drive (in parallel).  The parity drive(s) are not read at this stage.
2. Calculating the new contents of the parity sector(s) based on the contents of the target drive and the equivalent sectors on all the data drive (this is the same calculation to that done when initially building parity)
3. Writing the updated sector(s) to the parity drive. 

Whether this actually speeds things up is going to vary between systems as it depends on the rotational state of many drives, but in tends to by eliminating the need to wait for a full disk revolution to occur on both the parity drive(s) and the target drive and this tends to be the slowest step.

 

in both cases the effective speed will be lower than raw disk performance might suggest.  

 

The potential attraction of SSD only arrays (that some users have been discussing) is that delays due to disk rotation are eliminated thus speeding up the above processes.

[Marshalleq]

Nice description.  Thanks for that.  I'll live with the slower speed - I do like that the drives spin down.  Another guy on here was quite convincing telling me I had a problem at that speed was all.  Thanks again for the nicely written description - that should be a sticky!

[detz]

How did you do these benchmarks, I need to go down this hole.