SnapRAID + Unraid for best of both worlds to prevent and recover from "bit-rot"?


unraidfan

Recommended Posts

I've recently learned that SnapRAID is a solution that can recover from bit-rot in addition to any other scenario. It has been interesting to see people running it hourly with OpenMediaVault + MergerFS to accomplish very similar goals to Unraid. I have to say, I really like Unraid. The polish is great and the other functionality for VMs + containers like Docker are really icing on the cake...

 

Can we run SnapRAID on top of Unraid to accomplish an anti-bit-rot regiment? I'm not going to get into a debate of whether or not bit-rot happens or if it's critical or anything like that. I have enough evidence from engineers in the industry to convince me that if I had unlimited money and resources I'd just go with with raidz3 on TrueNAS SCALE and be done with this entirely, but I don't and I'm here and I'd really like to see if I can make this work.

 

Has anyone successfully done this?

 

Here's a reference document by the way doing the OMV + SnapRAID + MergerFS: https://www.michaelxander.com/diy-nas/ and another from archive.org on a down site: https://web.archive.org/web/20210308170014/https://www.networkshinobi.com/snapraid-and-mergerfs-on-openmediavault/

  • Like 2
Link to comment

I don't really think it's fair inferring laugh-ability by using the term bogeyman (although your use is cute). Furthermore using ZFS in conjunction with Unraid is a bit of overkill in light of SnapRAID. SnapRAID is compatible with the "light" expandability philosophy of Unraid and therefore much more generally compatible with the system such that bit-rot becomes a non issue while also facilitating recovery (even if bit-rot doesn't occur) at any given time based on frequency of the execution of SnapRAID. This seems to fit in perfectly with Unraid and seems as though it would and should be something Unraid would have already adopted, so I'm very surprised it's not yet integrated, let alone no plugin for it.

Edited by unraidfan
Link to comment

unRaid has a dedicated following, but there are some areas of general data integrity and security that unRaid hasn't developed as far as it has with Docker and VM support.  I would like Open ZFS baked in at some point, and I have seen some interest from the developers, but they have to get around the Oracle legal bogeyman.

 

I have seen no discussion around snapraid.

 

Check out ZFS here. 

 

Link to comment

I do agree that ZFS is a fantastic filesystem. I have used it for smaller client arrays for roughly 11 years and it has never failed to deliver rock solid filesystem integrity even with various crashes. So I can agree with that.

 

But I would like a lighter option than raidz*. SnapRAID appears to deliver at a happy medium ground where you can maintain integrity similar to raidz[1,2,3], but at a reduced frequency. This is great for archival situations where you can have hourly or daily snapshots instead of "always up-to-date" protection.

 

I can tolerate losing an hour of changes or additions to my storage array, but I can't tolerate invisible and unrecoverable loss.

Edited by unraidfan
  • Like 1
Link to comment

Yeah, it would be so nice. I was heartbroken when I discovered this was missing. Had already installed it on two servers and really was falling in love with the overall UX and of course UI. I'm actually rather sick over this right now.

 

TrueNAS SCALE is pretty sexy and I like their Chia functionality and integration setup, but man, it's just overkill for my budget in terms of the expandability of drives. 

 

I think the only reasonable solution for the time being is going to be Proxmox + OpenMediaVault (maybe LXC or just VM depending on if I can get bindmounts to work for LXC) + SnapRAID + MergerFS. I'm scared just typing all of these strung together since I really wanted a cohesive UI to minimize config and headaches, but that's where we're at... 😭

  • Like 2
Link to comment
  • 1 month later...

I was thinking about this same Idea. Unraid PLUS snapraid. Could a person use snapraid in an OMV VM (or natively on the baremetal) to backup to a third "parity" disk to allow for file level corruption restoration and provide a delayed snapshot backup of the server? I'm already running the file verification plugin. Is there anything I'm not thinking of that is already baked into the OS? I'm running two parity right now, but nothing except my backups will stop file corruption. And it really sucks if you unknowingly overwrite a backup file with a corrupt file. If I start getting parity errors, I presently have no way to recover files, only check them. Unless I recover from a backup. My backups are powered off and disconnected except for once a month. My cloud backup syncs the important stuff daily. I'm looking for a live snapshot with maybe a 24 hour delay type of backup and snapraid was the first thing I could think of, but it requires another disk (which I already have and  an empty slot in the server).

  • Like 1
Link to comment
  • 1 year later...

I know that this is an UNRAID forum but for this particular user I have found that Ubuntu server and snapraid, merger fs works great. If you need a GUI you can install the desktop version and it is rock solid. Also supports natively ZFS, docker, anything that you could dream off. Not sure why this is not a more popular solution?

Link to comment
  • 2 weeks later...

A couple thoughts... maybe I'm missing something. Isn't SnapRAID essentially the same concept as Unraid - parity calculated "externally" - Unraid does it at runtime, SnapRAID does it on request, but neither provides bitrot protection via detection. Simply by restoration (if I detect a file has become corrupted, I can restore from parity)

 

Also would be very curious how ZFS gets introduced, instead of XFS or BTRFS for a disk's filesystem, it would be simply a single disk zvol? I would expect performance could be more affected then. Also, seems like the only benefit there would be help detecting bitrot happening (which is a good thing)

Link to comment
21 hours ago, miketew said:

Also would be very curious how ZFS gets introduced, instead of XFS or BTRFS for a disk's filesystem, it would be simply a single disk zvol?

I would be somewhat surprised if ZFS were introduced as an FS option for the main array. More likely zfs will be an alternative to BTRFS for multidisk pools.

Link to comment

I too am curious if SnapeRid can be used on Unraid.

 

I am an avid photographer as well as a 2d and 3D digital artist. 

 

I have many photo files both in the RAW format and edited PSDs and JPGs. Along with 3D models I've created or bought and rendered illustrations.

 

Lightroom informs me I have just over 171,000 photos and illustrations on my Windows 10 hard drive. Which are backed up to Unraid and a second backup up to external USB drives.

 

I had file on my hard drive corrupted due to bit-rot. I went to my backups (Unraid and USB drives) only to find out that they were overwritten with the corrupt file.

 

So I am interested in Snap Raid or QPAR/Multipar as a possible solution.

 

 

  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.