Feature Request Poll for 6.11

[jonp]

Hey everyone!  As you may have noticed, today we put out a release for Unraid 6.10-rc1 and with that release in the wild, we wanted to get feedback from you, our loyal community, on what feature you'd like to see MOST in Unraid 6.11.  To better explain the options in the attached poll, here's a breakdown:

 

ZFS File System

Ever since the release of Unraid 6, we have supported the use of btrfs for the cache pool, enabling users to create fault-tolerant cache storage that could be expanded as easily as the Unraid array itself (one disk at a time).  Adding ZFS support to Unraid would provide users with another option for pooled storage, and one for which RAID 5/6 support is considered incredibly stable (btrfs today is most reliable when configured in RAID 1 or RAID 10).  ZFS also has many similar features like snapshot support that make it ideal for inclusion.

 

Multiple Arrays

As many of you already know, the Unraid array is limited to 30 total devices (28 data and 2 parity).  This limit is set to prevent users from configuring too wide of an array and ending up in a situation where the likelihood of multi-device failure during a rebuild operation is too high.  This only is exacerbated by the ever-increasing size of HDDs which further elongates the rebuild process.  So how do users with a full 30 disk array further expand?  The answer is with multiple array support.  This feature would be similar to "multiple pools" which were introduced in Unraid 6.9, but would apply to the Unraid array.  Users with multiple arrays could have those arrays still participate in the same shares, allowing the same management but with more storage devices.

 

QEMU-ARM for VMs

I know a few people in our community who have personally requested this of us in the past.  Adding this to Unraid would allow users to create ARM-based VMs which is ideal for testing out mobile OSes and other platforms.  While you won't likely be passing GPUs through here, this is still a very interesting use-case for mobile developers who could use this as a way to test their applications in a variety of scenarios (as well as to gain the benefits of running mobile applications from your server).

 

So make sure you vote in here and let your voice be heard!  I know I'm rooting for a very specific feature in this list.  What about you? 

[CrashnBrn]

Has the plan for VM snapshots gone away? 

[AgentXXL]

While I would really like to see ZFS implemented natively, I also want it to be available as a second pool/array. With 6.9.x allowing up to 35 pools in the Pool Devices section, that's likely the place I would be happy with configuring a ZFS pool. Of course if it was available as a second array beside the main parity protected array, that would work too.

 

Use the slower unRAID pool for storage but use the ZFS pool for disk I/O intensive tasks like video editing/scrubbing.

 

[Thorsten]

48 minutes ago, CrashnBrn said:

Has the plan for VM snapshots gone away? 

That would be very nice if Unraid would support snapshots for VMs. I would prefer this feature above all others.

[ljm42]

1 hour ago, AgentXXL said:

While I would really like to see ZFS implemented natively, I also want it to be available as a second pool/array. With 6.9.x allowing up to 35 pools in the Pool Devices section, that's likely the place I would be happy with configuring a ZFS pool. Of course if it was available as a second array beside the main parity protected array, that would work too.

 

Use the slower unRAID pool for storage but use the ZFS pool for disk I/O intensive tasks like video editing/scrubbing.

 

 

Yes this is the intent. 

 

You would go to Main -> Pool Devices and create a ZFS pool similar to how you can add a btrfs pool today. In general, ZFS and btrfs pools are for speed whereas the Unraid array would be used when you value flexibility.

[isaac.olsen94]

Dang I meant to click on multiple pools but got to hasty 🤦 All three are features i'd love to see in unraid. Especially ARM support as I do a lot of mobile stuff. But multiple pools is a big feature I see asked for a lot! 

[jonp]

55 minutes ago, Thorsten said:

That would be very nice if Unraid would support snapshots for VMs. I would prefer this feature above all others.

The reason it isn't on this list for this poll is for reasons that might not be so obvious. As it stands today, there are really 3 ways to do snapshots on Unraid today (maybe more ;-). One is using btrfs snapshots at the filesystem layer. Another is using simple reflink copies which still relies upon btrfs. Another still is using the tools built into QEMU to do this. Each method has pros and cons. 

 

The qemu method is universal as it works on every filesystem we support because it isn't filesystem dependent. Unfortunately it also performs incredibly slow.

 

Btrfs snapshots are really great, but you have to first define subvolumes to use them. It also relies on the fact that the underlying storage is formatted with btrfs. 

 

Reflink copies are really easy because they are essentially a smart copy command (just add --reflink to the end of any cp command). Still requires the source/destination to be on btrfs, but it's super fast, storage efficient, and doesn't even require you to have subvolumes defined to make use of it.

 

And with the potential for ZFS, we have yet another option as it too supports snapshots!

 

There are other challenges with snapshots as well, so it's a tougher nut to crack than some other features. Doesn't mean it's not on the roadmap

[Aer]

100% ZFS support, snapshots for any data area(vms/shares) would be second.

[BeardedNoir]

Hi,

 

Been using unRaid for around two years now and love it, especially the community.

 

The continued flow of updates is first class. The only thing I would reiterate is what has been mentioned above, ZFS and VM Snapshots are the two things that I would really like most in unRaid.

 

Thanks as always and keep up the great work!

[PilaScat]

A page with all the scheduled things, from plugins to mover, trim ecc would be very appreciated

[Steace]

4 hours ago, PilaScat said:

A page with all the scheduled things, from plugins to mover, trim ecc would be very appreciated

There isn't one already?

/Settings/Scheduler  <-- In the URL bar

Plugins add it there when you install them. At least for the trim, I think that parity and mover are there by design.

[Aer]

You mention ZFS on pooled devices, but does that mean array devices would also be able to utilize the ZFS format?

[dada051]

Multiple arrays will be great! ZFS support is already (partially?) done by the ZFS plugin

[PilaScat]

On 8/8/2021 at 4:58 PM, Steace said:

There isn't one already?

/Settings/Scheduler  <-- In the URL bar

Plugins add it there when you install them. At least for the trim, I think that parity and mover are there by design.

Not all are there, but probably all of them are using cron

[svh1985]

Snapshots and  vTPM (for Windows 11)!

Edited September 18, 2021 by svh1985

[KoNeko]

if you going to make a ZFS pool does that still enable adding hdds like we currently can in that pool. In normal ZFS that isnt that easy/impossible ?

 

 

[jonp]

On 8/8/2021 at 7:56 PM, Aer said:

You mention ZFS on pooled devices, but does that mean array devices would also be able to utilize the ZFS format?

 

Unless I'm missing something, yes, you would be able to format individual devices in the array with ZFS.  They would not be part of a ZFS "pool" in that scenario, however.

[jonp]

5 hours ago, KoNeko said:

if you going to make a ZFS pool does that still enable adding hdds like we currently can in that pool. In normal ZFS that isnt that easy/impossible ?

 

 

 

No.  Any requirements to expanding a ZFS pool will still hold weight if we implement it, which means a lot more UI programming work for us to make sure the UI respects the rules of ZFS.  That being said, there is an active project to make this possible with ZFS on GitHub:  https://github.com/openzfs/zfs/pull/8853.  Its still in Alpha stage so no idea when it would make it up the stack to be a native part of the project.

[KoNeko]

2 hours ago, jonp said:

 

No.  Any requirements to expanding a ZFS pool will still hold weight if we implement it, which means a lot more UI programming work for us to make sure the UI respects the rules of ZFS.  That being said, there is an active project to make this possible with ZFS on GitHub:  https://github.com/openzfs/zfs/pull/8853.  Its still in Alpha stage so no idea when it would make it up the stack to be a native part of the project.

It would be really great when that is possible with ZFS yes. Best of both worlds. The freedom of how Unraid does it  currently and the extra taht ZFS offers.

 

Would be great if we can convert our current array to the new zfs one with this feature. Ofcouse with the hope that you can add any size upto the parity one etc.

[jonp]

1 hour ago, KoNeko said:

It would be really great when that is possible with ZFS yes. Best of both worlds. The freedom of how Unraid does it  currently and the extra taht ZFS offers.

 

Would be great if we can convert our current array to the new zfs one with this feature. Ofcouse with the hope that you can add any size upto the parity one etc.

 

TBH, file system conversions literally scare me to death.  You are putting yourself in a position where if anything goes awry during the operation, you lose it all.  I think the better approach for folks will be to empty their devices one by one by shifting data to other disks, then reformatting the emptied disks and moving the data back.  Do that one by one for each disk in the array.  Heck, might even be able to script that operation ;-).

[KoNeko]

3 hours ago, jonp said:

 

TBH, file system conversions literally scare me to death.  You are putting yourself in a position where if anything goes awry during the operation, you lose it all.  I think the better approach for folks will be to empty their devices one by one by shifting data to other disks, then reformatting the emptied disks and moving the data back.  Do that one by one for each disk in the array.  Heck, might even be able to script that operation ;-).

would need atleast 1 extra empty disk in the system then to make a begin and copy the data. But we'll see when its that far. I think it will take some time before it will get a final version that can do this in unraid.

[itimpi]

3 hours ago, KoNeko said:

would need atleast 1 extra empty disk in the system then to make a begin and copy the data. But we'll see when its that far. I think it will take some time before it will get a final version that can do this in unraid.

This is the same basic approach as is used to convert array drives from reiserfs to XFS or BTRFS.

[dada051]

I really hope "multiple arrays" will win, as array is a core feature of Unraid, and a plugin already allow ZFS on Unraid. In addition, ZFS almost recommand lot of RAM and ECC one whereas Unraid philosophy is to run on whatever hardware you have.

[KoNeko]

4 hours ago, dada051 said:

I really hope "multiple arrays" will win, as array is a core feature of Unraid, and a plugin already allow ZFS on Unraid. In addition, ZFS almost recommand lot of RAM and ECC one whereas Unraid philosophy is to run on whatever hardware you have.

Well ZFS or not ECC RAM is always a nice to have in the Unraid server. It helps extra against data corruption etc.

For a normal PC it isnt really needed. But for a Nas system or a server with important stuff its a very nice thing to have.

 

ECC is an additional feature which adds a new and directly applicable protection, something that does not exist in any capacity without ECC. ECC is always used on hardware RAID cards and by the same measure should be used when implementing software RAID or any other software storage technology which uses RAM as a cache or integrity calculation buffer. 

Edited August 11, 2021 by KoNeko
added extra info

[dada051]

Sure. But it has not the same inpact on ZFS that on another filesystem (starting with XFS for example).

 

I don't understand why for a normal PC, it's not really needed. My normal PC use and manipulate data from and to my Unraid, so if my Unraid needs ECC, my PC should too, no?