Would it be possible for unRAID to someday use ZFS or BTRFS on the array?


RockDawg

Recommended Posts

I don't know squat about the inner workings of software, drivers or file systems.  Hell, most things Linux are over my head.  But I was wondering if it is technically possible for unRAID to someday use ZFS or BTRFS on the array?  As it stands now, each data drive is individually formatted using the ReiserFS.  Is it technically possible to have each data drive individually formatted with, say, ZFS instead and still have unRAID calculate parity and do it's thing?  Or is there something that makes that technically not possible?

 

I love ZFS and some of it's features (and I've read that BTRFS has some of the same features).  So much so that I would probably switch from unRAID to a ZFS set up if it were able to use drives of various sizes.  Unfortunately, the drive size issue is a pretty big one for me so I'll stay (happily mind you) with unRAID.  I'm just wondering if maybe someday unRAID might be able to do something similar.

Link to comment

I don't know squat about the inner workings of software, drivers or file systems.  Hell, most things Linux are over my head.  But I was wondering if it is technically possible for unRAID to someday use ZFS or BTRFS on the array?  As it stands now, each data drive is individually formatted using the ReiserFS.  Is it technically possible to have each data drive individually formatted with, say, ZFS instead and still have unRAID calculate parity and do it's thing?  Or is their something that makes that technically not possible?

 

I love ZFS and some of it's features (and I've read that BTRFS has some of the same features).  So much so that I would probably switch from unRAID to a ZFS set up if it were able to use drives of various sizes.  Unfortunately, the drive size issue is a pretty big one for me so I'll stay (happily mind you) with unRAID.  I'm just wondering if maybe someday unRAID might be able to do something similar.

 

 

What exactly would be the point of switching to ZFS if you're not going to use it how ZFS is intended to be used?

 

As for Btrfs, it is coming with 5.0 final. It's intended to be used for a cache pool but Tom has said you could essentially make your entire array a Btrfs pool and forego a traditional unraid array. Again though, if you're looking to keep things working like they do at the present where only the disk that is being read or written to (plus the parity during writes) then I don't see why the desire to switch to either of those file systems.

Link to comment

Well again, I'm no expert on all of this, but how would I not be using it how it is intended?  My thinking is to let ZFS be the file system on each disk so you gain the benefit of it's data corruption protection/self-healing along with snapshots and let unRAID handle the RAID portion of things like usual.

 

I know about BTRFS for the cache, but that doesn't seem like a big deal to me (maybe I'm missing something), especially since I don't use cache.  My understanding is it will allow for better speed and larger cache which is great if you use a cache drive, but I don't see the benefit of it's ZFS-like data protection on a cache drive/pool.  In most cases, the data is only on the cache drive for a relatively short period of time until the mover kicks in.  I'd much rather have the additional data protection on the array where my data stays.

 

Didn't read about using a large BTRFS pool as your array, but again, will it have parity protection and allow for various drive sizes?

Link to comment

Well again, I'm no expert on all of this, but how would I not be using it how it is intended?  My thinking is to let ZFS be the file system on each disk so you gain the benefit of it's data corruption protection/self-healing along with snapshots and let unRAID handle the RAID portion of things like usual.

 

 

Self healing and corruption recovery require multiple disks in a mirror or pool. These are not features of a single disk formatted with ZFS. As for snapshots, I'm not sure I see much benefit with what unraid servers are typically used for. Snapshots are also not "free", they take up additional space on your disk. And because snapshots in ZFS work at the block level I'm not sure that the way unraid handles parity if this could actually even work.

 

 

I know about BTRFS for the cache, but that doesn't seem like a big deal to me (maybe I'm missing something), especially since I don't use cache.  My understanding is it will allow for better speed and larger cache which is great if you use a cache drive, but I don't see the benefit of it's ZFS-like data protection on a cache drive/pool.  In most cases, the data is only on the cache drive for a relatively short period of time until the mover kicks in.  I'd much rather have the additional data protection on the array where my data stays.

 

Didn't read about using a large BTRFS pool as your array, but again, will it have parity protection and allow for various drive sizes?

 

 

Some people don't want to trust their data to sit on the cache drive and having the scheduled mover script move it before that disk fails. It's not something I worry about but it is an issue for some people that data is not parity protected right away. Cache pooling will get you high speed writes and immediate parity protection. You can use drives of varying size but you can only use as much space on each disk as is available on your smallest disk in the pool. i.e. one 250GB disk and two 500GB disks will lead to all disks acting as 250GB disks.

Link to comment

Well again, I'm no expert on all of this, but how would I not be using it how it is intended?  My thinking is to let ZFS be the file system on each disk so you gain the benefit of it's data corruption protection/self-healing along with snapshots and let unRAID handle the RAID portion of things like usual.

 

 

Self healing and corruption recovery require multiple disks in a mirror or pool. These are not features of a single disk formatted with ZFS. As for snapshots, I'm not sure I see much benefit with what unraid servers are typically used for. Snapshots are also not "free", they take up additional space on your disk. And because snapshots in ZFS work at the block level I'm not sure that the way unraid handles parity if this could actually even work.

 

I didn't realize you needed multiple disks for the corruption protection/self-healing.  That's the kind of info I was looking for.  Thanks.  Granted snapshots are pretty much irrelevant for most audio/video, but some of us store other files on unRAID too that could definitely benefit from snapshots.  Plus, while I have yet to test it, I am told that snapshots can recover a deleted file.  Even media files are subject to accidental deletion.

 

 

I know about BTRFS for the cache, but that doesn't seem like a big deal to me (maybe I'm missing something), especially since I don't use cache.  My understanding is it will allow for better speed and larger cache which is great if you use a cache drive, but I don't see the benefit of it's ZFS-like data protection on a cache drive/pool.  In most cases, the data is only on the cache drive for a relatively short period of time until the mover kicks in.  I'd much rather have the additional data protection on the array where my data stays.

 

Didn't read about using a large BTRFS pool as your array, but again, will it have parity protection and allow for various drive sizes?

 

 

Some people don't want to trust their data to sit on the cache drive and having the scheduled mover script move it before that disk fails. It's not something I worry about but it is an issue for some people that data is not parity protected right away. Cache pooling will get you high speed writes and immediate parity protection. You can use drives of varying size but you can only use as much space on each disk as is available on your smallest disk in the pool. i.e. one 250GB disk and two 500GB disks will lead to all disks acting as 250GB disks.

 

I get the high speed writes, but like you said above, considering what unRAID servers are typically used for, I wouldn't see this as a major concern either.  Most audio/video media is either ripped or downloaded which are both usually speed limited by things other than the write speed of the destination drive.  I know about some users worrying about the protection of the data until it's moved to the array, and it's not for me to judge what's important to others, but I don't get the paranoia.  I also know that you can use various size disks in ZFS but like you said, you are limited to the size of the smallest drive (as is the case in most RAID arrays if I'm not mistaken) and that's just a waste of space.  That is the single most important feature of unRAID to me.

 

Thanks for the feedback.  Maybe ZFS (or BTRFS) won't make a huge difference on the array, but so far I still see enough benefit to consider it.

Link to comment

I'd say it is likely at some point that btrfs will make it to unRAID.

 

http://lime-technology.com/forum/index.php?topic=27226.msg239203#msg239203

 

Interesting.  Maybe I'm just not quite getting it, but I'm not grasping the wide reaching benefit of a really fast, better protected "temporary" staging place for data.  Granted, pretty much all improvements are a bonus, but once the mover moves your data, all those benefits are lost.  And if you are ripping or downloading your media, there's only so fast you can go anyhow.  Maybe there is something I am missing or people are using their arrays differently?

Link to comment

Rockdawg>> I am with you on this. I have been running unRaid since 2011.  5.0betta13.

Free setup, no cache. ..

I do not see any issues.  I have about 2t worth of data on it.

Writes are fast enough.

I had ripped multitude of cd and dvds right on the array share. Transcoded 100s of movies from one share onto enother with no issues. Stream to htpc just fine.

 

So, I dont get all the obsession with transfer speeds and cache use.

 

Sent from my SGH-T889 using Tapatalk 4 Beta

 

 

Link to comment

People use cache drives for more than just temporary storage.  They are app drives as well so the apps would get the protection from a cache pool setup.  If you use the MySQL setup you could put the database on the cache to improve access speeds for access to the database while maintaining parity protection.

Link to comment

People use cache drives for more than just temporary storage.  They are app drives as well so the apps would get the protection from a cache pool setup.  If you use the MySQL setup you could put the database on the cache to improve access speeds for access to the database while maintaining parity protection.

 

As Bob says, Many of us use parts of the cache drive as an Apps / Program Files drive.

I have for years.

I used RAID1 protection of the Areca controller to give me a high level of confidence. It paid off big time for me.

If BTRFS is going to provide another highly reliable array without the performance penalty of unraid's parity protection, It's a good thing!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.