SnapRAID on UNRAID

A plugin for advanced users installing SnapRAID onto UNRAID systems.

 

Possible Use-Cases:

• Parity protection and a degree of corruption detection/repair for custom-mounted (e.g. using UD), possibly mergerFS-fused/pooled, unassigned disks outside of the primary Unraid array - either due to not always being online (cold backups) or exceeding the maximum possible array size of 30 disks, as commonly seen with large mixed disk JBOD/DAS solutions not wanting or being able to go full ZFS (primary use-case).
• A degree of corruption detection/repair for rarely changing large file libraries (e.g. on media or backup servers) in conjunction with Unraid parity, sacrificing an Unraid array data disk for SnapRAID parity to leverage the advantages of both Unraid and SnapRAID (experimental use-case).
• You tell me - post here if you have another exciting use case that worked well for your specific storage needs.

General Usage Warning:

As with any software interacting with your data, only ever use it on backed up data and in combination with a solid backup strategy.

Read the manual and help-texts before acting and if you do not understand a 100% what you are doing, please just do not do it... 🙂 

 

This thread is not really for teaching how to use SnapRAID. It is primarily intended for reporting and solving problems around the plugin itself. If the detailed SnapRAID manual and amply provided hover/F1 help-texts are not enough to bring clarity on using the software, this plugin will most likely have no real benefit for you (no offense). It was made for advanced users with very specific storage needs, not to just play around with without purpose (if you value your data).

 

Installable via Community Applications

 

Edited May 5, 20242 yr by Rysz

Just wanted to say thank you @Rysz for putting this together, it's been on my wish list for awhile now as an alternative setup!  Time to put my concept to the test and though I'm horrible at it, hopefully I'll report back.

Will watch this with interest as i love the idea of being able to expand my array past the 30 unraid drives for non changing data.

This is great!  I just set this up running 1 parity disk and 6 data disks.  On the SnapRaid settings page, I can see the parity disk with filesystem of XFS.  I can see the other 6 data disks but filesystem is tmpfs?  Is this supposed to say XFS as well since all the data disks are formatted as XFS.  Total Space for the data disks is also wrong, showing only 1.05MB instead of 12TB.  When I do a sync, diff, list, etc, it does show the correct number of files and size.  Other than that, everything is checking out correctly. 

13 minutes ago, timc6896 said:

This is great!  I just set this up running 1 parity disk and 6 data disks.  On the SnapRaid settings page, I can see the parity disk with filesystem of XFS.  I can see the other 6 data disks but filesystem is tmpfs?  Is this supposed to say XFS as well since all the data disks are formatted as XFS.  Total Space for the data disks is also wrong, showing only 1.05MB instead of 12TB.  When I do a sync, diff, list, etc, it does show the correct number of files and size.  Other than that, everything is checking out correctly. 

Please ignore this.  This is supposed to be this way.  Everything seems to be running   

4 hours ago, timc6896 said:

Please ignore this.  This is supposed to be this way.  Everything seems to be running   

 

No problem, thanks for using the plugin - your parity and data disks should all show as XFS (see below for my primary array). 🙂 

If your data disks do not show up, please make sure you used a trailing slash / at the end of the mount points ( e.g. /mnt/disk1/ ).

 

 

The TMPFS disk shown at the end is the plugin's RAM disk, this is where all operational output will be written to (logfiles, etc.). It is a safeguard, so that a large amount of logfiles and/or exceptionally large logfiles cannot use up all your RAM and destabilize your operating system. The amount of total disk space shown for that disk is 30% of your total available RAM and is just the maximum possible RAM usage for the plugin, but the space is not in fact blocked when not in use.

 

Edited July 9, 20241 yr by Rysz

8 hours ago, Rysz said:

 

No problem, thanks for using the plugin - your parity and data disks should all show as XFS (see below for my primary array). 🙂 

If your data disks do not show up, please make sure you used a trailing slash / at the end of the mount points ( e.g. /mnt/disk1/ ).

 

 

The TMPFS disk shown at the end is the plugin's RAM disk, this is where all operational output will be written to (logfiles, etc.). It is a safeguard, so that a large amount of logfiles and/or exceptionally large logfiles cannot use up all your RAM and destabilize your operating system. The amount of total disk space shown for that disk is 30% of your total available RAM and is just the maximum possible RAM usage for the plugin, but the space is not in fact blocked when not in use.

 

 Wow!  Added the trailing slash "/" and not it shows correctly!  Stupid mistake on my part.  I ran the maintenance cronjob and it worked as expected.  No issues to report there.  Also ran a rebuild by pulling out a drive and replacing with a new drive, worked 100%.  This is an awesome plug-in that helps extend unRaid's limitation of a single array.

20 hours ago, timc6896 said:

 Wow!  Added the trailing slash "/" and not it shows correctly!  Stupid mistake on my part.  I ran the maintenance cronjob and it worked as expected.  No issues to report there.  Also ran a rebuild by pulling out a drive and replacing with a new drive, worked 100%.  This is an awesome plug-in that helps extend unRaid's limitation of a single array.

 

Thanks for reporting back, I thought that might have been the case for you and I appreciate the feedback. To be fair, that previous limitation or rather requirement was not all that visible among the other information back then. I did address this in the latest update with some additional checks against malformed configurations, so it should be better visible in the future if something is misconfigured. That limitation itself has since also been resolved. I'm glad that everything works for you now, I've also used SnapRAID intensively on my backup server for these past few months this plugin has existed and although it does have a bit of a learning curve and some limitations, it has also worked exceptionally well for me throughout various recovery operations. 🙂 

 

Edited July 10, 20241 yr by Rysz

Hi @Rysz,

thank you for all your effort in this case and that you made it possible to use SnapRaid on UnRaid as a plugin.

I am a BitRot-Paranoid 😇 and big fan of SnapRaid for my WORM-files (Write Once, Read Many) like media files on OpenMediaVault since years and allways wanted to get rid of OMV (different reasons) and was lurking around UnRaid over a year now and only did not gave it a try cause there was no proper Anti-BitRot-solution from my point of view here.

 

The only existing possible solution on UnRaid seems to be this (snippet from a german post from @mgutt (thank him for all his great effort, videos and work as well)):

Source Link

 

On OpenMediaVault i am using a SnapRaid-AiO-Script from auanasgheps on github (GitHub Link) which is really improving the whole thing.

I will give it a try soon on UnRaid and hopefully i just have to adjust a few things here.

I will report when its done ... or maybe your or others already did this? 😉
 

Thank you again !

10 minutes ago, BeckenrandschwimmerTim said:

On OpenMediaVault i am using a SnapRaid-AiO-Script from auanasgheps on github (GitHub Link) which is really improving the whole thing.

I will give it a try soon on UnRaid and hopefully i just have to adjust a few things here.

I will report when its done ... or maybe your or others already did this? 😉

 

I think the AiO script doesn't work on Unraid because of the dependencies, but you can try it. 🙂 

But my plugin has "SnapRAID maintenance" settings integrated, which cover most functions of the AiO script.

These settings were inspired and based on the AiO script and take care of periodic syncs, scrubs and other things.

I've been using SnapRAID on my backup server for about half a year and it's been rock solid and a good solution for me. 🙂 

Currently i did not gain any progress transforming the AiO-script from auanasgheps to UnRaid, but i agree: Your SnapRaid maintenance covers most of the functions i love from the original script 👍 (especially the Healthchecks.io-part and short Dashboard overview ❤️)

 

Feature Request: Hook script or implementation of Telegram notifications. Is this feasible?

 

Fun fact: An old backup disk covered by SnapRaid died, lets see what the Plugin can do here, when my PrimeDay-Disk arrives ...

21 hours ago, BeckenrandschwimmerTim said:

Fun fact: An old backup disk covered by SnapRaid died, lets see what the Plugin can do here, when my PrimeDay-Disk arrives ...

 

Let me know how the recovery goes, I've done 2 or 3 disk replacements without problems so far.

 

21 hours ago, BeckenrandschwimmerTim said:

Feature Request: Hook script or implementation of Telegram notifications. Is this feasible?

 

Notifications are handled through Unraid's OS-wide notification settings (Settings => Notification Settings).

SnapRAID and SnapRAID maintenance will send Telegram notifications, if configured so in the OS-wide settings:

 

 

On 10/10/2024 at 8:03 AM, Rysz said:

Let me know how the recovery goes, I've done 2 or 3 disk replacements without problems so far.

Hi there, sorry for not replying here for some weeks (family stuff, you know :-D). I will try the replacement on the next weekend.

What is you preferred way to replace a snapraid-covered disk under unraid?

54 minutes ago, BeckenrandschwimmerTim said:

Hi there, sorry for not replying here for some weeks (family stuff, you know :-D). I will try the replacement on the next weekend.

What is you preferred way to replace a snapraid-covered disk under unraid?

 

If you are using SnapRAID on top of Unraid parity (for integrity; meaning on Unraid parity-protected array disks):
I'd rebuild the disk using Unraid parity first and then do a check operation on the replaced disk using SnapRAID.

 

If you are using SnapRAID only, meaning without Unraid parity:

I'd pull the old disk, install the new disk, and then run a fix operation on the replaced disk using SnapRAID.

On 11/6/2024 at 1:41 PM, Rysz said:

 

If you are using SnapRAID on top of Unraid parity (for integrity; meaning on Unraid parity-protected array disks):
I'd rebuild the disk using Unraid parity first and then do a check operation on the replaced disk using SnapRAID.

 

If you are using SnapRAID only, meaning without Unraid parity:

I'd pull the old disk, install the new disk, and then run a fix operation on the replaced disk using SnapRAID.

It worked like a charm!

I have your first mentioned option running (Snapraid-parity on top of Unraid-parity).

Only 2 files where not confirm with the checksum after restore with Unraid-parity.

I fixed it with the Snapraid-parity. Checksum seems to be fine after this step.

Thank you for your help!

Thanks a lot for this plugin!

 

I’ll try it as soon as I have some free time.

 

Does it support writing parity to an external usb drive? I think it would be great to maximise internal hd slots and only spin up the external drive when needed to sync or check the parity periodically…

18 hours ago, kinoapparatom said:

Does it support writing parity to an external usb drive?

Yes, it does. I have no experience how reliable a usb connection is in this case, but i assume it should be fine.

It even don't have to be connected all the time, since SnapRaid is more a snapshot backup, completely different from UnRaid parity.

Just be sure that its connected and mounted when syncing etc.

It is worth pointing out that Unraid parity is realtime so writes to any drive array will also read and write to the parity drive.  There is also the issue that USB has proven less reliable so more likely to error in the first place.

35 minutes ago, BeckenrandschwimmerTim said:

It even don't have to be connected all the time, since SnapRaid is more a snapshot backup, completely different from UnRaid parity.

Just be sure that its connected and mounted when syncing etc.

Yeah, that's exactly the plan, I'm planning to have an unraid array with no real time built in parity but with a big zfs mirrored cache like 2TB or 4TB ssd and then use the mover to the array only weekly or biweekly and at the same time execute the snapraid sync so I'll have the parity usb drive only mounted a few hours every couple of weeks or so 

1 hour ago, kinoapparatom said:

Yeah, that's exactly the plan, I'm planning to have an unraid array with no real time built in parity but with a big zfs mirrored cache like 2TB or 4TB ssd and then use the mover to the array only weekly or biweekly and at the same time execute the snapraid sync so I'll have the parity usb drive only mounted a few hours every couple of weeks or so 

 

Yes, that should be possible without problem. 🙂 Since it's not real-time parity, connection type doesn't really matter with SnapRAID.

Wow, just came across this plugin randomly!   Great work!  I've used snapraid for years on other servers and its been an indispensable tool, and it's great I can add this to unraid.  Now I'll need to upgrade my servers lol.   Thanks again! 

Hello, sorry I am new to SnapRaid. I got few questions...

 

1. What will be the best way to encrypt the disks?

 

2. If on my UnRaid Array I already reach my 30 disks, but still got few spares hdds that I would like to add in. How will be the way to do it? is it possible to add it in the Array or only by creating a pool with the disks left?

 

Thank You

Best Regards

Edited January 2, 20251 yr by FTW

On 1/2/2025 at 3:25 PM, FTW said:

Hello, sorry I am new to SnapRaid. I got few questions...

 

1. What will be the best way to encrypt the disks?

 

2. If on my UnRaid Array I already reach my 30 disks, but still got few spares hdds that I would like to add in. How will be the way to do it? is it possible to add it in the Array or only by creating a pool with the disks left?

 

Thank You

Best Regards

 

SnapRAID doesn't care about encryption as long as your drives are unlocked when it's running.

It only operates on the files, it doesn't care about filesystem or where/how your data is stored. 🙂

 

You could mount the extra disks direct or using e.g. Unassigned Devices and use SnapRAID to parity protect them.

mergerFS could pool those SnapRAID-protected extra drives and then you'd set up a custom network share for that pool.

22 hours ago, Rysz said:

 

SnapRAID doesn't care about encryption as long as your drives are unlocked when it's running.

It only operates on the files, it doesn't care about filesystem or where/how your data is stored. 🙂

 

You could mount the extra disks direct or using e.g. Unassigned Devices and use SnapRAID to parity protect them.

mergerFS could pool those SnapRAID-protected extra drives and then you'd set up a custom network share for that pool.

okay thanks a lot

I'm looking into moving my small media server from OVM to Unraid. Is it feasible to run Unraid just with a snapRAID setup?

My current OMV setup looks like this.

SSD1 (ext4): Docker appdata

SSD2 (ext4): data drive 1
SSD3 (ext4): data drive 2

HDD1 (ext4): parity drive

I guess all drives, except SSD1 with the system and appdata, just need to be mounted as unassigned devices. Do I have to start with a clean parity drive or can I work of the existing parity file on HDD1?