Faster parity drive does make a difference - almost 10MB/s more for me!

[thejtiggidy]

I have read numerous times in the forums that having a faster parity drive makes no difference and that writes are only as fast as the drive your writing to, yada yada.  It never really made absolute sense to me because without parity you can obviously write much faster.  Why would you be limited to 25-27MB/s with drives that are easily capable of much more... there is at least a chance for a gain here!  I emailed Tom for his insight and he basically said the jury is still out and give it a shot; so I did.

 

I replaced my WD15EADS parity drive with a WD Black 2002FAEX 2GB and it has made a huge difference.  I was getting about 25-28 MB/s before the switch; immediately after I get 33-35 MB/s and sometimes even up to 40.  No other hardware was changed!

 

I now run 1 x WD2002FAEX parity and 5 x WD15EAD/RS' it was worth the extra dough! 

 

Anyway, I was on the fence... if you are too I say go for it!

[Rajahal]

We need more information for your tests to be conclusive.  What type of files were you transferring?  How large?  How much RAM does your server have?  How many drives were you writing to at once?

 

Generally speaking a faster parity drive will only be helpful if you are writing to more than one data drive at one time or in some cases if you are transferring a lot of small files (smaller than your drive's cache).  I expect that if you used a large file that was far larger than your RAM (say a 25 GB BluRay ISO) and transferred it to only one disk while there were no other IO operations going, you would see that the faster parity drive makes no difference.

 

For comparison, I routinely see between 25 and 35 MB/s writes to my server (w/o a cache drive).  I'm using a Seagate LP 5900 RPM drive as my parity drive, and my data drives range between WD Greens and Seagate/Samsung 7200 rpm drives.  My speeds will change depending on what drives are in play, how many drives are in play, what type of files I'm transferring (large movies versus folders full of smaller pictures), and how much activity there is on my network at the time (there's usually a lot as I'm constantly seeding torrents).

 

Remember that there's four IO operations every time you write to the server.  First the data disk is read, then the parity disk is read, then the data disk is written to, and finally the parity disk is written to.  This is why we don't see the full 60+ MB/s writes of which most modern drives are capable.  There's also other factors, such as network latency, potential PCI bus bottlenecks, etc.

[WeeboTech]

it makes a difference, but only if you need to squeeze out every ounce of performance.

 

The common consensus is that it does not matter much.

 

In my experience the faster drive helps depending on architecture and tunings.

If you look at my historical posts, I have empirical results that state I've seen improvements.

It also depends on how your drive is being used.

 

Switching from a 5400 RPM parity to 7200 RPM parity improved my rtorrent by a great deal.

Latency of operation with multiple drives improved also.

 

In fact, I know the faster parity provides a measured increase so I purchased an areca hardware controller and run 2 1.5TB 7200RPM drives in RAID0 mode for parity.

 

It would be good if you define your system specs and how you are using/benchmarking your transfer speed.

[thejtiggidy]

Ok, well here it is. 

 

ASUS P7H55M-PRO / I3 550 / 4 GB

1 x WD2002FAEX 2GB (Black)

5 x WD15EAD/RS 1.5GB (Green)

 

Gigabit Ethernet - Jumbo frames - No utilization

I was transferring a 7GB MKV.

I was writing to a single drive that was empty. 

After the new parity drive was installed.. I deleted that single file and recopied the very same file to the same drive.

 

My benchmark was copying this one file in both situations repeatedly using teracopy and logging my throuput and timing.

 

Again I dont understand where the logic came from that a faster parity drive doesnt make a difference;  it seems flawed to me.  What I think is more accurate is every drive that is involved in a write impacts performance and they are NOT directly linked to eachother.  The writes are not congruent; they are utlimately a series of events, any increase in performance in one of the steps along that series nets some gain in the end.  Just as you have stated in your response, you get different results with different sets of drives.  If the parity and data write were in lock step with eachother you would not see any performance gain amongs any set of drives as your parity drive would dictate your performance for all of them equally.  So basically your experience confirms that it fact makes a difference.

 

I am not saying that my results are the holy grail and backed by intesive testing and documentation. But, I wish this "limitation" had not been so overstated and protrayed repeatedly as a forgone conclusion as I would have gone with a faster parity drive in the first place; saving me some time and frustration.  I am new to the forums so I am not familiar with the discussions around this topic but I fee like this is kind of like global warming... once enough people believe, it becomes fact; regardless of the actual results.  I am sure there are situations and methods where the benefit is greater or less significant but again... it is not black and white and certainly not a fact that it makes no difference.

[BRiT]

If the parity and data write were in lock step with eachother

 

They are closely locked together. It's the essence of Parity Protection. If they weren't then you could have write holes where your parity is invalid and offers no protection at all.

[WeeboTech]

Just for the record,

It's been shown in the past that going from a faster parity drive to a slower one on some systems showed no difference in write speed.

 

I think that became the standard way of thinking.

Perhaps with that release, it was true.

 

With every release, kernel change, tweaks, there are changes which can prove or disprove this scenario.

[Heretic]

interesting stuff. if this turns out to be true on multiple configurations its an easy way to improve speeds

 

 

In fact, I know the faster parity provides a measured increase so I purchased an areca hardware controller and run 2 1.5TB 7200RPM drives in RAID0 mode for parity.

does the risk of losing your parity when one drive fails not scare you?

[WeeboTech]

In fact, I know the faster parity provides a measured increase so I purchased an areca hardware controller and run 2 1.5TB 7200RPM drives in RAID0 mode for parity.

does the risk of losing your parity when one drive fails not scare you?

 

I don't use drives I should be throwing away in my array. They are fairly recent drives and I have spares on site.

Since I use the areca's raid 0 volume set in a manner that that does not exceed a 2TB maximum spindle size, replacement is easy.

If there were an issue I could re-assign parity to a single drive's spare slot and fire up a parity generate while I am replacing the drive in the areca controller.

 

Even with 20 drives, parity generate is still within 10 hours.

 

 

[Heretic]

fair point.

 

what speed improvements did it give you overall?

[WeeboTech]

fair point.

 

what speed improvements did it give you overall?

 

Mine were not as much as thejtiggidy, probably because I had a fast and tuned environment already.

 

I saw about 3-7MB/s improvement in a DD bulk drop benchmark just from the arecea alone.

 

But response time on the machine doing allot of other work was vastly improved.

With some other kernel tunings I burst at 60MB/s for the first 1GB and slowly drop to 35MB/s after about 4-8GB.

 

The Areca is a caching controller, parity generate speeds are pretty fast as the writes are cached.

Since it's a read other drive write cached parity drive, it's fast.

Parity check speeds are different as it requires reading the cache drive and reading each individual drive.

Still adequate for what I need.

Plus the areca allows dividing the two drives in a SAFE30 environment.

So with 2 1.5TB 7200RPM drives I have A 2TB virtual RAID0 drive and A 350GB RAID1 Cache.

 

 

[Heretic]

thanks for all the info, well thought out. good stuff.

[lionelhutz]

It never really made absolute sense to me because without parity you can obviously write much faster.

 

Do you understand that a single write to the array with parity protection involves 4 operations, 2 per drive.

Parity drive - read the sectors being modified, calculate new parity, write the sectors back.

Data drive - read the sectors being modified, use the new vs old data to calculate the new parity, write the new data.

 

The reading and writing is done in parallel to both drives. Now, this is where your system comes into play. The ultimate speed depending on a number of factors including at least the following;

- Size of the file or files.

- Rotating speed of each drive.

- Cache memory on each drive.

- Drive efficiency (how quickly it can do this read/write/read/write/read/write operation).

- Memory in the server.

- Tuning of the kernel.

 

One thing I haven't read much about is the low level operation of the disk system. The parity calculation produces a number. But does that new parity data and the user data get sent in parallel to both disks or is the data sent sequencially to the disks? If it's sequencially then a faster cache drive (more likely is more cache memory on that disk) might allow the cache data to get transferred to the disk system quicker so the user data can start to be transferred quicker.

 

The slower disk arguement comes from the fact that the both disks must more or less both read and write the same amount of data. So, in theory, the shortest length of time to get the data from the network to the disks is the time the slower disk will take to do both the reading and writing of the amount of data being transferred.

 

Now, a write to the server is performed VERY differently when you do not have a parity. Basically, the data is taken from the network, buffered and dumped to the data disk as fast as possible. There is no need to perform the 4 read/write operations listed above. Much less complex.

 

Also, know that the 1.5T EADS drives are just plain slow. I realized a speed increase by replacing my 1.5T EADS drive with a 2T EARS drive. Funny, since both rotate the same speed. Guess it goes back to the "drive efficiency" factor.

 

Finally, I understood the lower write speed limitations when I purchased unRAID. So, I'm really not concerned about it. Good post but I'm not spending a bunch more for a little extra write speed. I'd set up a RAID5 if I really wanted the best write speeds.

 

Peter