Is ZFS the future of Unraid? Do you want it to be?

[MrCrispy]

After watching the recent interview as well as the general direction, it seems the intention is to have ZFS be the new filesystem for arrays in Unraid?

 

If so, I am rather hesitant, because even though I'm sure it will not be mandatory, I still don't think ZFS is a good fit. There are a lot of ZFS fans used to enterprise level hw and sw and obviously it has many great things, but IMO it fundamentally doesn't really follow the basis ethos and tenets of a home server fs -

 

- data disks should remain in native format and independently readable

- only the disk with data needed spins up

- you can mix and match disks of any size

 

The basic issue is that a striping implementation such as traditional Raid, ZFS or btrfs can have none of these.

 

And ZFS is also notoriously 'heavy', it will eat up ram, it needs bigger caches etc. Honestly I think for its intended audience, btrfs is a much better solution, except it doesn't have the mind share or history.

 

The other big feature of course is checksumming, snapshots etc, totally lacking in Unraid. I think the best way to add this would be to be pragmatic - doing these in a realtime system is impossible without block level striping.

 

What Snapraid does is IMO perfect - periodic checksums, snapshots, backups, bitrot protection, undelete, dedup and so many other features become possible with very little overhead, with the only caveat being its not realtime - which if you think about it is actually better for an array since most media is RO anyway. 

 

I know this has been discussed before but there was never any real conclusion and it would be nice to get a fresh discussion and official thoughts?

 

[Squid]

13 minutes ago, MrCrispy said:

What Snapraid does is IMO perfect

Safe assumption that it effectively being unmaintained is less than perfect?

 

SnapRAID HISTORY
================

12.3 2024/01
============
 * Fix potential integer overflow when computing the completion percentage.
   No effect on the functionality.
 * Documentation improvements.

12.2 2022/08
============
 * Fix build issue with GLIBC 2.36

12.1 2022/01
============
 * Reduce stack usage to work in environments with limited stack size, like
   MUSL.
 * Increase the default disk cache from 8 MiB to 16 MiB.

12.0 2021/12
============
 * Parallel disk scanning. It's always enabled but it doesn't cover the -m
   option that still process disks sequentially.

11.6 2021/10

 

Or that the GUI for it is completely unofficial and also suffers from effectively zero updates. The last commit was April 27, 2023

 

FWIW, for something that has effectively been vaporware since day 1 I did try it a number of years ago.   The lack of real-time recovery / emulation was a major problem.  Not to mention that since it's snapshots are period based upon the schedule you set Murphy says that any random file that you need to recover due to a failure is not going to be covered by the snapshot.  I ran it for about a week and then decided to run for the hills.  Trusting your data to something like it don't give me a warm and fuzzy feeling.  

 

But to each their own.

 

 

[MrCrispy]

The discussion is about using features similar to Snapraid, not to discuss that product. Calling it vaporware is not called for, it seems to be a very well designed and regarded product from a technical perspective.

 

I presented several benefits over Unraid and shfs. as well as the compromises made. The discussion is about those, as well as thoughts on ZFS.

 

4 minutes ago, Squid said:

Not to mention that since it's snapshots are period based upon the schedule you set Murphy says that any random file that you need to recover due to a failure is not going to be covered by the snapshot

such can be said for other systems like unRaid too. or you could say that Snapraid gives you additional features like undepete, snapshots for specific folders, bitrot protection, and offline parity.

 

there are many many ways to slice and dice and improve. Just dismissing other approaches isn't good.  I know this is an Unraid forum but I was hoping for a discussion on technical merits. FWIW I own a Pro key, its not like I'm not aware of the benefits.

[itimpi]

I see no reason why ZFS becomes the future - just an option within Unraid for users who gave appropriate Use Cases.

 

I see Unraid continuing to support a variety of file systems to suit different Use Cases.   The main change I see coming is the current Unraid array becoming another pool type probably called something new like a ‘hybrid array’.   It could simply be called a Unraid pool but I see this being confusing with Unraid also being the OS name.   Within that hybrid pool it would continue to have each drive as a free-standing file system in any of the currently supported formats.

[primeval_god]

Well ZFS is not the future for my unRAID servers. I have no interest in it and no intention of using it. As for something like Snapraid I also have no interest, realtime parity protection is a must for me.  The unRAID array type is what brought me here and my filesystem of choice is BTRFS. For the purposes of single disk filesystems with unRAID driver parity i see no advantages of ZFS over BTRFS. Really my only wish for improvement to the unRAID style array (that isnt already in the works) would be leveraging filesystem checksums along with parity for bitrot protection (which i have accepted really isnt as big a threat to data integrity as its made out to be). 

[MAM59]

I agree, ZFS is contra-productive and not needed.

 

UNRAID should focus the main feature (the "UNRAID array") by adding improvements like read caches or so. Now with ZFS people ask themselfs, why they have (or should) bought UNRAID when it moves on to a "just another os with zfs" .

 

BTW: if I WANTED ZFS, I could have it natively for years already (FreeBSD) instead of using a halfly done Linux copy.

 

[JorgeB]

Zfs is just another option, it can be used together with all other filesystems already available. 

 

Personally I use it mostly with flash based raidz pools, good performance and robust, unlike btrfs with raid5/6, for the array devices I was using btrfs and will keep using btrfs, I see no benefit in converting for single device filesystems.

[Jaybau]

On 3/2/2024 at 1:41 PM, MrCrispy said:

checksumming, snapshots etc, totally lacking in Unraid

 

I think these are being accomplished by extending Unraid with File Integrity plugin and backup software.  This basically becomes scrubs, snapshots, backup. 

 

I think what is really being requested is for an integrated, seamless, easy solution.  And to do this without the need of doubling the drives with backups/mirror/replication; accomplish via parity algorithm.  I prefer the storage/redundancy/restoration/protection efficiency of parity versus 1:1 backup/mirror (50% efficiency).  What I don't understand is why Unraid doesn't think this concept would be an evolutionary step.  It doesn't seem like the solution is that far away or costly, so I don't understand why its not being developed.

 

I am looking for a seamless suite of tools to EASILY manage scrubs and restoration.  If SnapRAID+Unraid was offered, I would probably go that route (all the benefits of Unraid+what I want from ZFS/BTRFS without the disadvantages).  Since SnapRAID is not offered, I am preparing for ZFS.  My hope is that once I get ZFS configured, scrubs and repairs are just a matter of clicking a button.  The price I pay for ZFS is worth not having the cost of a headache with data integrity problems, panics, time, restoring from a backup.

 

I don't think ZFS is ideal, but it is offering the next step towards what I want, until something else better is offered.  Maybe someone will create a SnapRAID plugin.  Maybe Unraid will develop the next generation of Unraid parity. 

 

In the meantime, I'm going to spend more money on buying more drives to get the solution I want.

[MrCrispy]

2 hours ago, Jaybau said:

 

I think these are being accomplished by extending Unraid with File Integrity plugin and backup software.  This basically becomes scrubs, snapshots, backup. 

 

I think what is really being requested is for an integrated, seamless, easy solution.  And to do this without the need of doubling the drives with backups/mirror/replication; accomplish via parity algorithm.  I prefer the storage/redundancy/restoration/protection efficiency of parity versus 1:1 backup/mirror (50% efficiency).  What I don't understand is why Unraid doesn't think this concept would be an evolutionary step.  It doesn't seem like the solution is that far away or costly, so I don't understand why its not being developed.

 

I am looking for a seamless suite of tools to EASILY manage scrubs and restoration.  If SnapRAID+Unraid was offered, I would probably go that route (all the benefits of Unraid+what I want from ZFS/BTRFS without the disadvantages).  Since SnapRAID is not offered, I am preparing for ZFS.  My hope is that once I get ZFS configured, scrubs and repairs are just a matter of clicking a button.  The price I pay for ZFS is worth not having the cost of a headache with data integrity problems, panics, time, restoring from a backup.

 

I don't think ZFS is ideal, but it is offering the next step towards what I want, until something else better is offered.  Maybe someone will create a SnapRAID plugin.  Maybe Unraid will develop the next generation of Unraid parity. 

 

In the meantime, I'm going to spend more money on buying more drives to get the solution I want.

I agree with you. Alas it seems ZFS has a very large mind share and is considered some kind of uber-solution, which I don't agree with at all. Its resource hungry, enterprise focused, and brings nothing to the table as far as I can tell vs btrfs, except a better name.

 

And of course, ZFS/btrfs and any other form of striping is a complete non starter for a home solution, which is why unRaid exists in the first place.

 

Is it really too much to ask to integrate snapRaid style checksumming/integrity checks into shfs? It would bring a huge host of benefits to everyone. Those who want to run enterprise style pools with ZFS and spend $$$$ on memory, drives etc, are welcome. But honestly, why not run TrueNas or plain old Linux if thats what you want, since the only thing lacking would be the docker integration and the CA app store, which honestly can be added manually quite easily. 

[Jaybau]

The problem with BTRFS is the RAID 5/6 bug where you can lose the entire pool.  This isn't something I'm willing to risk.

 

I am curious why SnapRAID isn't added to NerdTools and plugged into Unraid.  It can be very simple to do. But, I also could be completely wrong, and would like to know why.  There might be scenarios I haven't considered.  It might be a lot more difficult to manage than I think.

 

I'll be experimenting with ZFS to find out how resource hungry it is.  So now I am very curious.  It's not like I'm running Unraid on a Raspberry Pi or minipc.  I have 3 spare 1TB drives to learn with.  I assume my home network will be the speed bottleneck.

 

 

[MrCrispy]

1 hour ago, Jaybau said:

The problem with BTRFS is the RAID 5/6 bug where you can lose the entire pool.  This isn't something I'm willing to risk.

 

I am curious why SnapRAID isn't added to NerdTools and plugged into Unraid.  It can be very simple to do. But, I also could be completely wrong, and would like to know why.  There might be scenarios I haven't considered.  It might be a lot more difficult to manage than I think.

 

I'll be experimenting with ZFS to find out how resource hungry it is.  So now I am very curious.  It's not like I'm running Unraid on a Raspberry Pi or minipc.  I have 3 spare 1TB drives to learn with.  I assume my home network will be the speed bottleneck.

 

 

I believe ZFS ideally needs a lot of RAM. As well as higher resource requirements in general.

 

ZFS will use half of RAM as cache and upto 90% if needed.  A quote from another forum - "ZFS is memory hungry, more than other file systems. Given the way it writes data to disk (non-linear, log-structured), it tends to have a large amount of metadata in memory, and make heavy use of that metadata. If that metadata gets corrupted in memory, bad things will happen. An even more important argument is the following: ZFS always reads and writes CRCs, so the data on disk and on the way to/from the disk is safe against corruption. That means that data in memory is now the largest target for corruption. While this doesn't mean that ECC is more necessary than for other filesystems (which also fill all unused memory with cache data), for ZFS the relative importance of memory errors is higher"

 

SnapRaid as an external tool would not make any use of unRaid. The shfs code already has every write/read to the array go thru it. It doesn't make sense to have to run an external tool like snapRaid sync to read data and check parity, when that is already being done inside unraid. What could be added is checksums, as well as other functionality, in the same driver in Unraid that computes and checks parity. Then its a simple matter of using snapraid's other tools and utils. 

 

Unless we hear from the official team, this isn't going to go anywhere. Sadly I don't think anyone else cares about this, all the hype is about ZFS, and its far easier to sell ZFS as a new feature. There is a big inertia in the Unraid community, the few threads I've read on this all basically amount to 'i'm happy with what I have and we shouldn't demand more'. The file integrity plugin is already deprecated I believe and in any case would not prevent against bitrot.

 

 

[Jaybau]

12 hours ago, MrCrispy said:

SnapRaid as an external tool would not make any use of unRaid. The shfs code already has every write/read to the array go thru it. It doesn't make sense to have to run an external tool like snapRaid sync to read data and check parity, when that is already being done inside unraid. What could be added is checksums, as well as other functionality, in the same driver in Unraid that computes and checks parity. Then its a simple matter of using snapraid's other tools and utils.

 

My proposal would be:

• Optional Unraid parity array.
• Use SnapRaid on any pool. (it is inherent to Snapraid, so nothing needs to be developed.)

Future Unraid:

• Full independent multi-pool support.
• SnapRaid like functionality, or better.  Or at least provide a Snapraid plugin.  Effort is minimal.
• Union filesystems (pooling) (not just btrfs/zfs/array).  There's already several open source projects that already do this.  Effort is minimal.

I would use a mirror Unraid cache for write operations in between periodic snapshots/parity.  This will keep my data safe until parity operation.

 

I would experiment with more optional frequent snapshots on the cache, since I am expecting the write operations to be small.  With Snapraid this can be limited to folders, file extensions, etc.  With snapshots, I think it would give me versioning too.

 

Limetech / Unraid Mission Creep?

 

I do not think btrfs/zfs is the inline with Unraid's mission.  Too much effort that isn't aligned with mission = shaking head...I just don't get it.  Unraid is incorporating RAID (zfs) in their next major release !?!?!?  I just don't get it at all.  But Limetech is spending money on doing it, and I've read Limetech was even reluctant to doing it.  So I'm not sure even Limetech thinks they are doing the right thing.

 

I do think something similar to SnapRaid is inline with Unraid's mission.  I just don't get why Limetech doesn't go full throttle in this direction instead of integrating ZFS.  Unraid would still be UNraid.

 

BTRFS/ZFS multi-disk pools just aren't un-raid.

SnapRAID is.  But we aren't getting it.

 

I believe the only reason why Unraid users are going ZFS is because Unraid won't give them what they really want (scrubs, repair, snapshots).  So I'm curious, if Unraid provided SnaprRAID/unionfs (or similar), would ZFS demand drop?

[primeval_god]

1 hour ago, Jaybau said:

 

I believe the only reason why Unraid users are going ZFS is because Unraid won't give them what they really want (scrubs, repair, snapshots).  

I dont think, generally speaking, that unRAID users are going for ZFS. Its inclusion seems like mainly an attempt to bring in people from truenas who see ZFS as an essential (mainly due to a perceived need for speed and bitrot protection).

[Jaybau]

Great points...

 

1 hour ago, primeval_god said:

I dont think, generally speaking, that unRAID users are going for ZFS.

 

 

If I was in charge and had the budget, I would have enhanced the unRAID product to include the important features that ZFS offers (integrity, scrub, repair, snapshot), but done with the unRAID brand/mission.

 

And if I didn't have the budget to do it in the unRAID way, the next best thing is SnapRAID.

 

I just read unRAID's about "mission":

 

Quote

Unraid Today

What started out as a personal quest to improve media storage has evolved into the most popular at-home storage and computing solution: Unraid OS.

Our powerful operating system allows sophisticated media aficionados, gamers, and other intensive data-users to have ultimate control over their data, media, applications, and desktops. Best of all, Unraid allows you to use just about any combination of hardware that you wish, giving you the freedom to customize and maximize your systems to precisely fit your needs.

 

Besides wanting to centrally store my data, I also want to keep my data for a many decades.  Hence why data integrity verification and some way of restoring data in the case of hardware failure or some bit rot event caused by whatever (I have experienced bit loss, but can't explain how/why, and don't really care, I just want mitigation), and done efficiently (money/time).  It becomes hard to justify the cost when it exceeds the purchase cost of my media collection, except the cost of time accumulating the media.  Which is why the data integrity/recovery has become more important to me.  Backups mean doubling my storage costs.  Parity allows me to just buy one drive for parity, so the storage ratio is excellent.  Parity gives me the risk vs. cost ratio that is desirable for me at the moment.

 

But something else was mentioned...gamers.  I'm not a gamer, but I have heard their performance requirements are very demanding.  So maybe this is the real reason ZFS is coming...it's for the high performance demanding gamers, with the high performing hardware investment.  So ZFS's hardware demands and costs aren't a big deal to the gamers who already have expensive rigs.

 

 

 

 

 

[ross232]

I don't think ZFS is displacing any of the existing file system options. It's just another option you can optionally choose to use. In my case I'm using individual ZFS formatted drives in a normal parity protected array.

 

The advantage here is that you can use snapshots, compression etc. But I retain the advantage of being able to use disks of different size and I can mount them individually. I also use ZFS on my cache drives because I don't love the way BTRFS reports free-space and have had issues with this in the past.

 

I can see why there are use cases for people wanting a full ZFS array too. If speed is a priority then this will always win out over a parity + cache combo. People doing media creation (e.g. YouTubers etc) may want to start with a parity array, then migrate to a full ZFS backed one later. Add in the paid support options UNRAID now has and you have something you can rely on as your needs grow/change.

[MrCrispy]

If speed is a priority and you want data striping, IMHO Unraid is not the product for you. There are any number of free and commercial RAID based solutions that can accomplish this.

 

As to why Limetech is spending money on ZFS while seemingly completely ignoring Snapraid like functionality, I believe the answer is what I said earlier - ZFS is a very big name and can bring in popularity and $$$, and many of the big Unraid adopters and popularizers are people with $$$ and giant servers, youtube channels, who can afford it. ZFS makes almost no sense for the typical home Unraid user, vs the ones with giant server racks and multiple servers and enterprise gear.