Mirror Share type

[karateo]

I would like to have a share type which would mirror it's files silently in two or more different disks (excuding parity drive of course).

Something like btrfs subvolumes raid1.

 

It would also provide checksumming and self-healing.

 

This would provide higher security level for some files without using an extra drive for 2nd parity and the checksumming.

 

I believe there are lots of other people who use a small portion of the array for sensitive files (personal photos/videos) and the rest is movies/music.

 

What do you think about it?

[RobJ]

Mirroring often sounds like the perfect solution at first, until you think through the situations you want it to handle properly.  Unfortunately, instant or relatively quick mirroring just propagates the bad operations onto the mirror.  A mirroring process can't know which user operations are valid and which are not.  Some examples -

 

* You locate some pictures that are out of focus and delete them.  The mirror deletes them from the second drive.  Then your kid accidentally deletes a special folder of baby pictures, and the mirror dutifully deletes them too from the backup.  The mirror can't know which deletions are valid.

* Ransomware strikes and your files are encrypted, and the originals removed.  The mirror then repeats the process on the backup.

* Bitrot occurs (extremely unlikely but possible).  A mirror can help here *if* a process is run soon enough to locate the corruption.  One point for mirroring!

* The drive fails or the sectors under the file go bad.  Yes, mirroring could help here, but this is what unRAID already protects you from.  Rebuilding the drive restores the files.

* You (or your editing program) malfunction and corrupt an important file.  If the mirror is too quick, the corrupted file is then copied to the backup, possibly overwriting your only remaining copy.  In some situations, the mirror can become your enemy, not your friend.

 

Instant or quick mirroring solves very little in an unRAID system.  Delayed mirroring or versioning can really help, if it delays the loss of the backup copy sufficiently long for the problem to be discovered.  I believe what would be better is a versioning process, with the backup versions inaccessible outside the machine (or as inaccessible as possible).  The Recycle Bin plugin can be very useful too, for recovering accidental deletions.

 

There's a lot of related discussion in the Ransomware Resistance thread, and I recommend reading through it for ideas.  It's clearly still a work in progress though.

[JonathanM]

* Bitrot occurs (extremely unlikely but possible).  A mirror can help here *if* a process is run soon enough to locate the corruption.  One point for mirroring!

But, unless you have checksumming or other mechanism to spot corruption in place, it could be difficult to tell which copy is correct.

 

1/2 point deduction, because mirroring by itself just makes for 2 copies, one of which is bad.

 

Bottom line, as RobJ said, a backup of your files in a different physical location is a much better usage of the drive space. An automatic mirror inside unraid would just be a waste of space.

[karateo]

Thank you both for your feedback.

 

I have already read the ransomware thread and I have thought a lot about it.

Currenlty I use an unassigned drive which I mount manually and sync using beyond compare (from a win VM) after comparing with checksumming.

Also, I have an old synology 30 kilometeres away, that I manually power up and update once per month. I use beyond compare to find the files and copy them to an external usb drive which I plug to my synology and copy.

 

I think I have done enought to protect agains Ransomware.

 

Now about the on topic feature request.

 

If there could be a raid1 subvolume the bitrod protection is provided.

Also, the files would be protected for if (number of parity drives)+1 fail simultaneously.

In addition the the above, the snapshot feature of btrfs+raid1 could provide protection against ransomware. The snapshots could be programmed to be taken in an interval that the user feels comfortable with.

 

Unless all these are not compatible with how parity/ies work, it shouldn't be difficult to implement (A noob says ).

 

A year ago when I purchased Unraid, I tried and created btrfs snapshots which the seem to work, but didn't do any more testing. I was just playing.