Why use ZFS and can you convert existing array from XFS?

[questionbot]

There is a lot of hype around ZFS and I recently watched SpaceInvaders video on the rc5 and how to make ZFS pools... but I'm confused as to what ZFS is and why I should be excited. I know it is another file system... but is it worth changing my array from XFS... and if so.. can you convert an existing array to ZFS?

[MAM59]

I've asked for the same, no answer so far. Its just a hype, it will fade away as usual...

[itimpi]

The main reason you might want ZFS (or BTRFS) as s file format for disks in the array is that those file systems will detect bitrot. 

[JorgeB]

https://forums.unraid.net/topic/131857-soon™️-612-series/?do=findComment&comment=1262978

 

[MAM59]

yeah I already know all that stuff, but where is the "WOW!" effect???

Liking the original UNRAID idea its a great step back into the wrong direction I think.

 

[itimpi]

22 minutes ago, MAM59 said:

yeah I already know all that stuff, but where is the "WOW!" effect???

I would say the WOW effect is when it is used in a Pool as it gives you the performance that is often one of the frequent criticisms of Unraid.

 

23 minutes ago, MAM59 said:

Liking the original UNRAID idea its a great step back into the wrong direction I think.

I disagree here.   The Unraid type array is not going to go away - it is just going to become another pool type (probably in type 6.13 timescale).  The Unraid array has advantages in terms of flexibility around using mixed drives so that will not disappear. 

 

At that point the need for the current main array being mandatory will be disappear and the primary/secondary storage types introduced with the 6.12 release will be able to use any combination of pool types (including the current Unraid array type).   You will be able to pick the combination of pool types that best suits your particular work mix.

[MAM59]

4 minutes ago, itimpi said:

I would say the WOW effect is when it is used in a Pool as it gives you the performance that is often one of the frequent criticisms of Unraid.

I don't see any pool problems (but then, all my pools are on nvme drives, fast as hell, no caching needed)

I only miss the read (cache) speed in the main array. But ZFS won't help here too.

 

If I wanted to have a ZFS based main storage, I would use "the real thing" like FBSD, no need for Unraid.

 

[JorgeB]

No one is forcing you to use ZFS, if you don't need or care for it don't use it.

[MAM59]

1 hour ago, JorgeB said:

No one is forcing you to use ZFS

yeah I know, but I have had higher ("WOW!") expectations besides "yet-another-file-system". So I just asked if I had missed some important aspect.

 

[itimpi]

7 minutes ago, MAM59 said:

yeah I know, but I have had higher ("WOW!") expectations besides "yet-another-file-system". So I just asked if I had missed some important aspect.

 

There are facilities like snapshot support, built-in trim support, snapshot send that only  a few users will use, but for those they can be very useful options.

[TimTheSettler]

I wouldn't bother if you're just a typical home user which many people here are.  However, there are some nice benefits that @itimpi has mentioned.  Bit rot detection is nice but a rare occurrence and hard drives are designed to catch this so is it worth it just for that?  Maybe.  ZFS also has snapshot but would you use that?  No.  The one thing that I like is that ZFS supports inline compression such that when you save a file it's compressed and then saved to the disk in the compressed state.  This can save some space for you.  However, you need to keep in mind that reading and writing files requires the extra step of compressing and decompressing it.  Will unRAID support that feature?  I'm not sure.  Another cool thing is deduplication but this requires massive amounts of memory and some planning of file block sizes since the file block determines how much memory is used to keep track of all the blocks.  There are also ways to improve performance like ARC/L2ARC and SLOG/ZIL.  Are you confused yet?  Join the club.  It's not an easy file system to just plug in and use.  So when you take all these "configurable" parts away you're left with "just another file system".

 

So for now I don't see a benefit but as people use the file system and other smart people make it easier to use and/or set up then it will become "the next best thing".

 

19 hours ago, itimpi said:

it is just going to become another pool type

 

This is what I'm hoping for.  I see the array or caches as being synonymous with a ZFS vdev.  The drawback is that you need to think about your setup before you build your server whereas right now it's all pretty simple with unRAID which is what I love about unRAID.

 

Forgive me but I think the following is a great explanation of ZFS.  The link is to TrueNAS.  Someone here needs to write up something similar but with reference to unRAID once ZFS officially shows up for unRAID.

 

https://www.truenas.com/docs/references/zfsprimer/

[tjb_altf4]

22 hours ago, questionbot said:

There is a lot of hype around ZFS and I recently watched SpaceInvaders video on the rc5 and how to make ZFS pools... but I'm confused as to what ZFS is and why I should be excited. I know it is another file system... but is it worth changing my array from XFS... and if so.. can you convert an existing array to ZFS?

Most users already on Unraid will probably be happy with the status quo, but ZFS on Unraid will appeal to a lot of new users that liked all the other aspects of Unraid, but were put off by lack of ZFS.

 

The ZFS file system itself is the gold standard and is extremely robust with great performance, at the cost of being less flexible and more $/TB compared to the classic Unraid array.

[questionbot]

can you convert an existing unraid array / pool to zfs if they all have the same size hdrives without loosing the existing data?

[itimpi]

40 minutes ago, questionbot said:

can you convert an existing unraid array / pool to zfs if they all have the same size hdrives without loosing the existing data?

No - Converting to a new file system requires reformatting the drive which wipes any existing data.  

[Stixbnr32]

Well not true - Spaceinvader has released a video on how to do this and "scatter" the data over the array then format the disk without losing anything or breaking parity.

 

I have completed 3 disks so far for no other reason than to future proof for more zfs related features been used in my array (also compression) as my server is a 13th gen i7 with decent memory ZFS extra overhead is not a problem.

 

 

[JonathanM]

1 hour ago, Stixbnr32 said:

Well not true - Spaceinvader has released a video on how to do this and "scatter" the data over the array then format the disk without losing anything or breaking parity.

Formatting a disk erases it. No way around that. IF you have enough free space on other drives in the array, yes you can move data off of one of the disks, emptying it so formatting doesn't lose data, it's empty after all.

 

On 5/20/2023 at 5:00 PM, itimpi said:

Converting to a new file system requires reformatting the drive which wipes any existing data.  

This statement is still VERY true.

[gramio]

 

On 8/17/2023 at 10:29 PM, JonathanM said:

Formatting a disk erases it. No way around that. IF you have enough free space on other drives in the array, yes you can move data off of one of the disks, emptying it so formatting doesn't lose data, it's empty after all.

Not to split hairs here but, most NAS is based on the principle of data duplication so you always have the space that you need to move from one FS to another. It just takes time to move the data around.

 

That said, if questionbot doesn't see the point of ZFS then clearly you are not the usecase that Unraid is solving for. Keep using what you have and don't worry about ZFS. It's quite clear that this is another option and it's not replacing any of the existing choices for FS.

 

[JonathanM]

43 minutes ago, gramio said:

Not to split hairs here but, most NAS is based on the principle of data duplication so you always have the space that you need to move from one FS to another. It just takes time to move the data around.

Unraid isn't "most NAS". You very definitely can fill each data drive completely full, leaving no room to move data around.

[gramio]

Back to the og question in the thread, before you start dismissing zfs as useless and not being the Unraid way consider this. I recently upgraded a config that is a simple NAS media server ( 3 identical hdds, one setup as parity) and cache with redundancy i.e. 2 nvms setup to mirror each other. The only slightly unusual feature of the system is 64gb of ram which is more than most but definitely just a nice to have and not a necessary feature. Moved the cache drives from btrfs to zfs which took some time but definitely not a hard process. The new setup does use RAM ( 1/8 of the total available so 8gb in this case) but what you gain from it is hard to believe. Nothing has changed in the hardware config of the system but the whole system is faster to respond and some workflows such as moving data from cache to the array are substantially faster (some cases are 6-7x faster). Keep in mind that none of the Unraid principles have changed and from a user pov this is the same system. You also gain a much better snapshot management but forget about that for a sec because that's changing user/admin behaviour. So if you really want to know why you should care about ZFS, the answer is speed. If you are stuck on 16gb of ram, it's definitely not for you. But if you can spare some RAM, you might as well consider moving at least your cache to zfs.

[MrDice]

On 5/19/2023 at 6:10 AM, JorgeB said:

No one is forcing you to use ZFS, if you don't need or care for it don't use it.

wow, if anyone would have said this to you or another mod they would be ban or something like that wow FAF

[MrDice]

On 5/19/2023 at 2:30 AM, questionbot said:

There is a lot of hype around ZFS and I recently watched SpaceInvaders video on the rc5 and how to make ZFS pools... but I'm confused as to what ZFS is and why I should be excited. I know it is another file system... but is it worth changing my array from XFS... and if so.. can you convert an existing array to ZFS?

Its like everyone did read what you said I would like to know to if you can't change over to ZFS from XFS and not one person said anything about it more on how it was either better or not lol and the mods my god me and you would be ban for the things they said lol did you find out if you could switch over or not? thanks

[JorgeB]

9 hours ago, MrDice said:

did you find out if you could switch over or not?

You can switch to zfs.

 

9 hours ago, MrDice said:

wow, if anyone would have said this to you

Read the entire thread, I was not replying to the OP, but to the user who was saying that zfs was the wrong direction for Unraid, probably could have word it better, but was just pointing out he doesn't have to use it.