ZFS filesystem support


jphipps

Recommended Posts

7 minutes ago, sdamaged said:

So is there no way at all for unRAID to be able to silently fix file corruption? (even in the future?)

I can't say for sure but doubt it, one of the big Unraid advantages is that each disk is a separate filesystem, but this also bring some limitations.

 

12 minutes ago, sdamaged said:

it becomes almost impossible to have backups (50+TB of movies on mine) that i can't back up without another array.

Yes, it gets expensive, I have a full backup of all my servers, each server has a clone synced at least once a month with btrfs send/receive, largest one is at 102TB currently, and I'm far from wealthy, for me it's a question of priorities, though I'm now at a point where the main expenses are done, just need to upgrade a few disks each year as more capacity is needed.

 

8 minutes ago, sdamaged said:

Incidentally, what benefits could BTFRS give me over XFS, as i may be building a new server early next year

Main btrfs advantages for me are snapshots, send/receive and checksums, I've been using btrfs exclusively on all my Unraid servers for years now without issues, but won't deny that ZFS is more mature and stable, I also have one FreeNAS server, though it's not perfect, it's not flexible at all, and if there's a problem, there are no tools to fix it or recover, just nuke pool and restore from backup.

Link to comment
53 minutes ago, sdamaged said:

it becomes almost impossible to have backups (50+TB of movies on mine) that i can't back up without another array.

 

This entire plugin thread says otherwise:

 

 

Link to comment
55 minutes ago, BRiT said:

 

This entire plugin thread says otherwise:

 

 

Oooh thanks for this!  I did try an older video where you could backup to Google Drive, and then it stopped working as Google apparently wouldn't allow it?

 

If this works then great, as i currently have a 2TB volume added to a VM (which is basically duplicating data) with the google drive client installed

Link to comment

Ok I know  that ZFS is at the moment not possible to implement in unRAID due to legal issues.
But... what if someone (unfortunately I lack the programming skills) was to take some of the features and make a rudimentary construct? Would that still be ok legal-wise?

 

Let us pretend it is, and let me theorize how it could benefit us unRAIDers...

Today we have individual disks for parity and data, only cache can consist of multiple disks.
To my knowledge (and i do not claim to burdened with in-dept-knowledge) of ZFS uses vdev(s) that consists of on or more disks that can vary in sizes. With multiple vdevs you are able to expand the ZFS. vdev(s) uses ZFS own data protection single, mirror, RaidZ1, RaidZ2, RaidZ3 etc. where the number indicates how many parity disks in the pool(s).


Now if we use a somewhat similar approach in unRAID where we bundle individual disk to a pool-of-disks (Abbreviated as POD from this point on) running in stripped-mode via "mdadm" and assigning PODs different roles like individual disks are today.

In theory the benefits should be increased speed. Also we should be able to make a POD that consist of SSD's for parity-POD and a data-POD, thus get a very high transfer speed outside the cache and still have data protected in parity.

For those wanting multiple arrays, this could be seen as each POD could be seen as one array.

I have tried to create a visual of how disk assignment is today, and with the POD design:

 

Today:
8tb Disk0-parity0
6tb Disk1-parity1
4tb Disk2-data0
4tb Disk4-data1
4tb Disk5-data2
4tb Disk6-data3
4tb Disk7-data4
4tb Disk8-data5
6tb Disk9-data6
6tb Disk10-data7
6tb Disk11-data8
6tb Disk12-data9
4tb Disk13-data10
4tb Disk14-data11
4tb Disk15-data12


POD-design:
4tb Disk2    POD0-parity0
4tb Disk4    POD0-parity0
4tb Disk5    POD0-parity0

4tb Disk6    POD1-parity1
4tb Disk7    POD1-parity1
4tb Disk8    POD1-parity1

6tb Disk9    POD2-data0
6tb Disk10    POD2-data0

6tb Disk11    POD3-data1
6tb Disk12    POD3-data1

4tb Disk13    POD4-data2
4tb Disk14    POD4-data2
4tb Disk15    POD4-data2

8tb Disk0    POD5-data3

6tb Disk1    POD5-data4

 

In the example above overall performance in the array should in theory be faster, even with unRAID's speed penalty. Further POD4 should have superior read/write speed (given that no other PODs are active at that moment). And one could argue that you now have 4 smaller arrays.

This is will no doubt make the "master"-array more vulnerable to disk fails, but by using mdadm you are not limited to RAID0, but can in theory use RAID6 in one POD and RAID0 in another should there be a need for that. Also if memory serves me correctly, ZFS has no way of expanding a vdev. It is possible to expand a POD using mdadm. (with limitations in RAID10)


When using mdadm it is also possible for RAID6 (most likely also RAID5) to add a hot spare disk for minimal "rebuild-time" if needed.

So data residing in a POD with RAID6 will have "internal POD protection" in form of RAID6 but also be protected by the parity POD(s).

Data residing in a POD with RAID0 will only have the protection of the parity POD(s), and thus be more vulnerable than a setup where you have 1 disk for parity and 1 disk for data. The reason is if ONE disk in a RAID0 array breaks down ALL data is lost. So with the POD design using RAID0 you are more vulnerable than if you have 1 disk for parity and 1 disk for data.

So I think a 1 disk for parity and 1 disk for data solution should still be possible, at least as having a POD with only 1 disk in it.

 

Now I know some may think this is a bad idea, but I am sure some think it can be useful. I think that if Limetech chooses to implement this it should be for PLUS & PRO licenses only as 6 disks and below will get little benefit from this.

I have tried to list pros and cons for this POD design:

 

PRO:
1a) if theory meets practice we should see a read speed bump for each disk added to a POD

1b) if theory meets practice we should see a write speed bump for each(* 2 for RAID10) disk added to a RAID0/RAID10*/RAID5/RAID6 POD

2) possibility to "kind of" having multiple arrays

3) possibility to have hot spares for selected PODs

4) selective elevated data protection for each POD

 

CON:
1) more vulnerable to data loss when RAID0 is used in POD

2) less available space from disks where anything else but RAID0 is used in POD

3) CPU usage may be higher due to the extra "layer" of RAID

 

 

This suggestion is to give inspiration I have only used what I know/heard of. I know it is a bit far out. But at least no one got hurt. ;)

 

/Alphahelix

Link to comment
  • 2 months later...

Personaly i am happy with the main array as it is. But have been bitten now about 3 times with corrupt btrfs cache pool.

I use a large multi TB ssd (btrfs raid1) cache pool for speed and direct fast (via 10GB ether) access to video data . I keep projects that i am working on in the pool and once done trigger mover to move them to the (still fast 7200rmp) spindels (xfs). Back and forth with unbalance keeps it very workable. For my vm's i enjoy the snapshots feature of btrfs

But 2 of these times i was hit it resulted in corrupt VM images that btrfs could not fix and the last one into complete pool corruption, that was not fixable by any of the btrfs methods i researched. Of course have good backups but it sucks having to rebuild everything and drops my sense of btrfs reliabilty to near zero (while i was bragging about btrfs to most of my friends untill then)

At that time i did not know much if any about zfs. Started reading up on it, installed the plugin, moved most data i kept on the btrfs cache to ssd only zfs raid1 pools and they have been stellar since. 

Also zfs literay (in plain english rather then useless syslog messages) told me the main reason that i had issues with btrfs already in the first weeks , which was a wobbly connection to 2 of 8 ssds in a hotswap cage, but in contrast to btrfs it only reported it but kept fixing any issue continously and kept the pool healthy, where btrfs broke in same setup / same disks / same cables / same problems.

I unplugged and replugged live ssd's to see which drive it was and all the time zfs kept smiling like nothing happend.

There is just "zero" comparison or competition with btrfs . Specs or having a system runnig for years without issues are nice but real world behavior when there "are" issues and how you recover from them is what counts. The argument that btrfs has and zfs does not have recovery tools was mute in my case as none of the tools helped me at all other then to save some part of the data but not it fixing the pool, while zfs just fixed it for me while helping me also identify the problem.

So i have become a complete convert from a btrfs advocate to a zfs fanboy.

So a great option would be to be able to have zfs for cache only so get the best of both worlds. I can live without it for the main array as i see less benefits and more complications on implementing it there as already discussed.

Nice pool features but still a fast and super reliable cache pool (with raidzx options as i did not even mention btrfs lack of anything (reliable) more then raid1/10)

 

Edited by glennv
Link to comment
  • 4 weeks later...

I've just installed the ZFS plugin on unraid and love it. 

I decided to take two drives out of my unraid array and assign them to a ZFS mirror to store important photos and documents on.  This makes unraid nearly the perfect system.

 

I agree it would be great to have ZFS cache pool.  I've been bitten several times with btrfs cache and I note depsite some contrary views, the latest Unraid 6.8 rc1 changelog which confirms some of these bugs.  Would be great if ZFS was included - even if only hidden under an advanced option, then among other things we'd probably find e.g. proper integration with plugins to help e.g. the scheduling of scrubs and such.

 

Licensing doesn't seem to be a concern any more either:

 

E.g. ubuntu and software freedom legal council

Actual releases in Ubuntu and proxmox

 

Others if you google it.

Link to comment
  • 1 month later...

I have just discovered this function, and I am doing a lot of reading.

I feel the same as @marshalleq that I also like to have a area that I can place my important data, but I don't like the ideal of relaying on a plugin to do the work, as I feel that this should be a core element.

 

To to ZFS users can you please tell me how you have found the ZFS environment and setting up.

and also did you find any good reference "Step by step Guide"

 

Link to comment

I wouldn't worry about the plugin aspect of it - yes it would be better to be built into unraid, but it works at native speeds and is very robust and reliable.  I've not had a single issue in regards to performance or stability since I installed it - in that regard I do consider it native.  The only thing it means is that after an unraid upgrade, the developer has to recompile it for the latest kernel.  Not a big deal - he's very responsive.  Hopefully one day, limetech will find a way to make this integrated more as others such as ubuntu are now doing (legacy licencing concerns that no longer apply - but some are still cautious).  I think there's a step by step guide a few posts up.  You gotta be comfortable with the command line is the only thing.

Link to comment
  • 8 months later...
  • 3 months later...
  • 4 weeks later...

+1

I would like to see a UI for ZFS in unraid and a one-click installation for the module (so we avoid the license issues)!


PS. No,  ECC not requiered it's only recommanded!
Unraid isn't protecting us at the moment from bit-rot neither in ram nor disk.

ZFS without ECC would at least protect the data that is already on the disk from bit-rot!

Link to comment
  • 10 months later...
On 12/28/2020 at 10:57 AM, PyCoder said:

PS. No,  ECC not requiered it's only recommanded!
Unraid isn't protecting us at the moment from bit-rot neither in ram nor disk.

ZFS without ECC would at least protect the data that is already on the disk from bit-rot!

 

Seems like someone has been having a few trying discussions!

 

Two more:

No ZFS doesn't eat your RAM, some distro's just haven't configured the memory limits properly

Yes dedup is useful and no it doesn't eat your RAM if applied properly, especially with the special metadata vdev.

 

We should make a list of the FUD ;)

 

BTW, my entire system is now ZFS and has been for quite some time.  I won't be going back, the benefits are just too huge.  The speedup of the system was quite noticeable also.

 

Edited by Marshalleq
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.