2 Drive System: Mirror or Parity?


dev_guy

Recommended Posts

With only 2 identical drives can unRAID run a simple mirror for better performance vs performing needless parity calculations? This would be on a tiny NAS box so I'm not worried about adding more drives in the future. In my testing a simple mirror using other NAS operating systems far outperforms unRAID on writes with a low power CPU. I realize this isn't the typical use case for unRAID but I at least wanted to ask?

Link to comment
1 hour ago, dev_guy said:

With only 2 identical drives can unRAID run a simple mirror for better performance vs performing needless parity calculations?

Yes, with only 2 drives Unraid uses them like a mirror automatically, same with 2 parity drives and a single data drive, it will work as a three way mirror.

Link to comment
On 10/17/2018 at 9:43 AM, johnnie.black said:

Yes, with only 2 drives Unraid uses them like a mirror automatically, same with 2 parity drives and a single data drive, it will work as a three way mirror.

I think this answer may need clarification. It sounds like the original question is "Does unRAID have a Mirror mode for disks that offers more performance than the normal unRAID data array parity scheme." The answer to that question is no, the data array has only 2 modes single and dual parity. What @johnnie.black is pointing out above is that in the case of an array with only 2 drives parity is equivalent to drive mirroring. This is not a special mode, and as far as i know does not do anything to increase the speed of the array. If you have already tested a system with a 2 drive mirror vs an unRAID system with only two array drives then the speed difference is what it is.

Edited by primeval_god
My information was incorrect.
Link to comment
1 hour ago, primeval_god said:

This is not a special mode, and as far as i know does not do anything to increase the speed of the array.

I am not so sure about that. I can't quickly find a post that limetech directly says it, but I'm pretty sure I remember Tom saying that it is in fact treated differently in the code. There has been more than one post by someone complaining about a speed drop when they added the 2nd data drive.

Link to comment

Thanks all for the input. The answers tell me it's at least worth testing. The short version is I get about 90+ MB/sec writes on the small NAS box (Intel N3150 low power embedded CPU) running FreeNAS even with compression and encryption enabled (both are native to ZFS). My unRAID server, however, using the same drives and a much faster Zeon CPU, can only manage about 60 MB/sec sustained writes without encryption or compression (and no cache drive) on larger file transfers. So I'm worried the sustained writes will be even slower on the much lower power CPU if unRAID is trying to calculate parity information when it doesn't need to. As to why I don't just run FreeNAS, it's another OS to learn, maintain, prefers ECC RAM, and has its own issues--especially with stability. I'd rather use unRAID across everything if it can deliver similar performance.

Link to comment
18 hours ago, johnnie.black said:

Sorry for shifting us slightly OT here, but this has managed to pique my interest somewhat. My reading of the comment you linked leads me to believe that I was correct in asserting that there is no special handling (or special case in the code) for mirroring. Rather it is the nature of the parity calculation that in the case of a single data disk the  P parity bits end up being the same as the data bits. And in the case of the Q bits the same is true when the disk is placed in a specific slot. This is worth noting, as i have to assume there is a performance impact to doing a parity calculation that results in mirroring vs just sending bytes to both disks, although maybe that impact is negligible. I am not saying that they couldn't have added an optimization for this particular array arrangement, the comments @limetech made on the other side of the link just don't really sound like it to me.

Link to comment
26 minutes ago, primeval_god said:

This is worth noting, as i have to assume there is a performance impact to doing a parity calculation that results in mirroring vs just sending bytes to both disks, although maybe that impact is negligible.

I don't remember if Tom ever clarified that, but unless you're using a >10 year old single core CPU there won't be any performance impact in the parity calculation, even with larger arrays and a modest dual core CPU you can enable turbo write and parity calculations doen't cause any performance penalty, i.e. you can write as fast as your disks/controllers/bus can handle, so for writing performance it will act as a mirror, on the other hand reads are only still done from the data disk, unlike other true raid1 scenarios that read from both mirrors to improve performance.

Link to comment

To clarify: in the case of a single "disk1" and either one or two parity devices, the md/unraid driver will write the same content to disk1 and to either or both parity devices, without invoking XOR to generate P and without using matrix arithmetic to calculate Q.  Hence in terms of writes, single disk1 with one parity device functions identically to a 2-way mirror, and with a second parity device, as a 3-way mirror.  The difference comes in reads.  In a typical N-way mirror, the s/w keeps track of the HDD head position of each device and when a read comes along, chooses the device which might result in the least seek time.  This particular optimization has not been coded into md/unraid - all reads will directly read disk1.  Also, N-way mirrors might spread reads to all the devices, md/unraid doesn't have this optimization either.

 

Note things are different if, instead of a single disk1, you have single disk2 (or disk3, etc), and also a parity2 device (Q).  In this case parity2 will be calculated, so not a true N-way mirror in this case.

  • Like 4
Link to comment
4 hours ago, limetech said:

In a typical N-way mirror, the s/w keeps track of the HDD head position of each device and when a read comes along, chooses the device which might result in the least seek time.  This particular optimization has not been coded into md/unraid - all reads will directly read disk1.  Also, N-way mirrors might spread reads to all the devices, md/unraid doesn't have this optimization either.

What do you think, worthwhile improvement?

Link to comment
10 minutes ago, limetech said:

What do you think, worthwhile improvement?

My first instinct was to dismiss it because of a super small target audience that would use only 2 or 3 array drives, but then I remembered all the new customers coming on board to only run VM's, and the availability of 10+ TB drives, and suddenly it doesn't sound like such a bad idea. Three 10TB data store drives with combined read speeds in the 500MB/s range, and four 1 or 2TB NVME SSD's for VM vdisks would make a potent little machine.

Link to comment
6 hours ago, jonathanm said:

My first instinct was to dismiss it because of a super small target audience that would use only 2 or 3 array drives, but then I remembered all the new customers coming on board to only run VM's, and the availability of 10+ TB drives, and suddenly it doesn't sound like such a bad idea. Three 10TB data store drives with combined read speeds in the 500MB/s range, and four 1 or 2TB NVME SSD's for VM vdisks would make a potent little machine.

Agree, not something I would likely use but there are potentially enough users, especially those running Unraid mainly as a hypervisor, to make it worthwhile.

Link to comment
  • 2 weeks later...

Thanks again to everyone who contributed. I now have 6.6.3 running on my low power (Intel N3160 SoC Qnap) NAS with 2 drives (1 XFS data and 1 parity) and I'm getting 90 MB/sec sustained writes which is something like 50% faster than with 3+ drives. So that confirms there's not much of a performance hit in a "mirrored" 2 drive configuration. The downside is it still took over 12 hours to do the parity sync on EMPTY 6TB drives which seems a bit ridiculous if it really is operating as a mirror. I realize this isn't a common use case but it does make me wonder what unRaid spent 12 hours doing besides copying 6TB of empty sectors to 6TB of empty sectors? It seems that bit of code could use some optimization?

 

As an aside, Qnap's QTS operating system has devolved into form over function which was the whole point of installing an alternative OS on their hardware. QTS is loaded with bugs, there are multiple major security holes uncovered every week, out of the box the drives won't stay in standby and there's no way to make then stay in standby as the OS is running off the main drive array, and Qnap is constantly trying to upsell you with paid extras. Qnap literally seems more interested in supporting Karaoke (yes my NAS has dual microphone jacks) and a zillion other buggy apps nobody likes than being a solid and secure file server. It's sad. Anyway, life should be much better with unRaid running on the Qnap hardware. So thanks again!

  • Like 1
Link to comment

When building parity Unraid makes no assumption about the existing contents of the parity and simply overwrites it without looking at the sector contents.   Since parity has no concept of file systems it cannot make assumptions about the meaning of any sectors contents.

 

Unless your drives had just been pre-cleared then UnRAID would not have been copying empty sectors.     A formatted drive has the file system control information written to the relevant sectors.   That is actually a non-trivial amount of information as it can run around 1GB per TB of useable space.

Link to comment
8 minutes ago, dev_guy said:

The downside is it still took over 12 hours to do the parity sync on EMPTY 6TB drives which seems a bit ridiculous if it really is operating as a mirror.

That would be another set of 'mirrored' optimizations.  Even with Parity+Data1 the 'sync' process is copying the contents of Disk1 to Parity in order to ensure both devices have identical block contents.  This would be unnecessary in a true Raid-1 where the array is established first, then file system built on the data disk (because all writes from that point on will go to both devices).  Same is true for 3-way mirror of Parity1+Parity2+Data1.  However, if you ever added another device to the array, such as Disk2, s/w would have to 'know' a full parity sync is necessary.  This can be done, but as mentioned before, part of 'optimization'.

Link to comment

Thanks. Both drives were identically freshly formatted with XFS. There was a tick box before starting the array that said something to the effect of "parity already correct" and I wondered about ticking it as the drives should have already been identical? But I figured I'd be safe and let it create the parity hoping it wouldn't take forever but it did. Hopefully the array won't have any unclean shutdowns or other reasons to have to do it again anytime soon.

 

Our fearless @Administrator is correct. On a normal Raid-1 mirror with new drives there is effectively no initialization. That's how FreeNAS, Qnap, Open Media Vault, XigmaNAS (aka Nas4free), Synology, etc. all work. You're up and running in just a few minutes with a protected 2 drive mirrored array. Anyway, the important part is there's no significant ongoing performance hit with a 2 drive unRaid setup unlike with 3 or more drives. So thanks for that optimization. :)

 

And now I have a new project... The Qnap NAS uses an eUSB DOM (Disk On Module) for the OS. It's basically just a USB flash drive but with a 10 pin connector to plug into a motherboard. I'm going to see if I can install and license unRaid onto the DOM. Hopefully I can copy the current config from the trial on the external thumb drive, minus the key, and hopefully get a paid license for it.

 

 

Edited by dev_guy
Added eUSB DOM
Link to comment
51 minutes ago, dev_guy said:

unlike with 3 or more drives

Parity1 + Parity2 + Data1 should not have that much of a performance hit, it's just a 3-way mirror.  But it's important to use Data1 and not any other Data device such as Parity1 + Parity2 + Data3 - this is because the Parity2 calculation is no longer a mirror for any column other than the first.

Link to comment
7 hours ago, limetech said:

Parity1 + Parity2 + Data1 should not have that much of a performance hit, it's just a 3-way mirror.  But it's important to use Data1 and not any other Data device such as Parity1 + Parity2 + Data3 - this is because the Parity2 calculation is no longer a mirror for any column other than the first.

Sorry, by 3 drives I meant 2 or more data drives and 1 parity drive (a more common scenario than 2 parity drives and 1 data drive). In that situation unRaid has to do more work to calculate the parity data on writes and it shows in the write performance especially with a low power CPU/system. In my testing it's 90+ MB/sec for a single data drive vs ~50 MB/sec on sustained large writes for 2+ data drives. Even on my Xeon server I still don't get anywhere close to 90+ MB/sec on sustained large writes without using a cache drive.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.