[PLUGIN] ZFS Master

[Iker]

Hi @sjerisman, Thanks for the proposal. The first one is already on my tasks list; I have to evaluate how it can be implemented and keep everything compatible with the cache and lazy load features; please wait a little bit longer (Next year); I will come up with something.

 

As for the second one, sure!, it's straightforward to add a new property; I'll be working on a new version with a little fix and include the new property on the snapshots table.

 

Best,

[dlandon]

3 hours ago, Iker said:

Good catch; the first one is shown because you haven't made any config changes to the plugin config, and it doesn't check if the file exits.

What I do is have the .plg create a default config file:

<FILE Name="/boot/config/plugins/&name;/&name;.cfg">
<INLINE>
<![CDATA[
[general]
refresh_interval="30"
lazy_load="0"
destructive_mode="0"
exclusion=""
directory_listing=""
snap_max_days_alert="30"
snap_pattern="Y-m-d-His"
snap_prefix=""
]]>
</INLINE>
</FILE>

The brackets around "general" may be an issue.

[Revan335]

@IkerCan you remove the .48 in the Version that fix the Sorting in Plugins Overview.

[Iker]

Yes, I can, but, can you please explain why or how that is something helpful for the plugin? (I use it in to keep track of multiple things, and, to be able to release multiple versions on the same day if necessary).

[Revan335]

2 hours ago, Iker said:

Yes, I can, but, can you please explain why or how that is something helpful for the plugin? (I use it in to keep track of multiple things, and, to be able to release multiple versions on the same day if necessary).

 

 

I mean this. The 2023.12.08.x is above the 2023.12.11

Maybe its a Bug in the Plugins Sorting and you can used the actually Schema too.

[Iker]

That seems to me an error on the sorting algorithm. However, I can change it to letters after the day as other plugins do.

[skler]

I'm experiencing some random read/writes on disk, I did a test on a disk not added in any share.
With the plugin installed I have all this disk write access, without it it stops 

 

Disk12 (sdn) doesn't have anything inside, but there are some data somewhere: 

 

# zpool iostat disk12
              capacity     operations     bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
disk12      12.4M  2.72T      0      1    485  9.29K

 

is someone else having the same errors? 

[dyno]

I noticed that this plugin displays space in a manner inconsistent with the rest of the unraid Main page.  Specifically, it appears to show data in GiB/TiB, but the plugin uses the GB/TB nomenclature.  It's a bit confusing.  If you're going to continue using base 2 units, would you consider changing the nomenclature to be accurate?   

 

Or perhaps change the units to base 10 instead?

[Iker]

Hi @skler, does this situation happen with the main tab open?, if so, very likely the plugin is the culprit. Still, you can go to the plugin config section and set how often the plugin refreshes the datasets and snapshot information.

 

Hi @dyno that is very accurate; I was aware of the plugin technical debt in that regard when I started the development and forgot about it for a while. Regarding the units that I'm using, those are base 2 because ZFS uses the same units; the output of the plugin and, for example, the zpool list, and zfs list commands are consistent; however, I would take a look at other metrics and make sure that are also in line and use the appropriate nomenclature.

Edited December 20, 2023 by Iker

[skler]

On 12/20/2023 at 3:37 PM, Iker said:

Hi @skler, does this situation happen with the main tab open?, if so, very likely the plugin is the culprit. Still, you can go to the plugin config section and set how often the plugin refreshes the datasets and snapshot information.

 

yes I fixed removing the refresh (in tabbed view btw the link to manually refresh doesn't appear). btw, you know how is used this disk space? what is written, and where?  

[Iker]

Yeah, I'm planning to fix the tabbed view on the next release. Generally speaking read and writes at that stage is mostly associated with metadata.

[isvein]

Any chance it will be possible to create zvols with this plugin too?

[Iker]

No, or at least, not yet; however, the plugin is able to list and perform multiple operations over them .

[unham]

Hello, great plugin, thank you for it.

 

Currently the default view only allows to show all root datasets and child datasets expanded.

 

Is it possible to add two default view options that allow for the plugin to be viewed with all datasets collapsed.

 

1. Toggle to choose between "Show/Hide Datasets" as default view option

2. Option to only show only root datasets expanded (not child datasets) if "Show Datasets" is set as default view option

[Iker]

I'm not sure if I follow your idea correctly, but currently, the plugin remembers which pools and datasets are collapsed, and the next time you visit the page, you get the same view; I believe that the functionality you mention can be solved that way.

[Renegade605]

I notice the directory listing feature doesn't list top level directories for entire pools, if enabled for them. This is where I'd personally find that most useful if it could. In fact I would suggest it do so by default, as there's basically no reason to have a top level folder in a zpool.

 

[Iker]

Hi @Renegade605, thanks for the report; I'll take a look and come up witth something to include the top level dataset, it's a very nice idea to do it by default just for visibility.

[Renegade605]

I've got a fair number of ideas for more advanced zfs functionality I wish unraid (or a plugin like this) would do. They may be far too advanced for average Unraid user; in fact I'm leaning towards abandoning Unraid at this point because I feel like it's holding me back. But, if you'd want to chat about them anyway, just let me know. 

[isvein]

42 minutes ago, Renegade605 said:

I've got a fair number of ideas for more advanced zfs functionality I wish unraid (or a plugin like this) would do. They may be far too advanced for average Unraid user; in fact I'm leaning towards abandoning Unraid at this point because I feel like it's holding me back. But, if you'd want to chat about them anyway, just let me know. 

Why not just say what you want to be there and the devs can tell if that is something they have planed or not?

[Niklas]

Wait for the next release of Unraid (6.13) for more zfs functionality. The zfs support is very new in Unraid.

[Renegade605]

Just now, Niklas said:

Wait for the next release of Unraid (6.13) for more zfs functionality. The zfs support is very new in Unraid.

I know. That's why I haven't made any formal feature request at this time nor gone full send on switching away. 

[Iker]

@Renegade605 feel free to make any suggestions on this thread, I've been working on this plugin for 2 years now, long before unraid 6.12, and my idea is to provide all the functionality not cover or that may take to long to be covered by Unraid, so, any suggestions or ideas are more than welcome.

[Renegade605]

One big thing would be znapzend integration with a nice GUI. I have znapzend but it's CLI only. Which is, fine, but it can be a complex enough task that GUI would be nice.

 

Configuring your Samba to use shadow copy for snapshots as well, especially if you know of a way to make that work with nested datasets and recursive snapshotting, cause I have yet to figure that one out. I don't know if that can work with datasets on multiple pools/disks after they've been through the FUSE system, but also would be great.

 

Configuring ZFS properties in the Share page and then auto-creating datasets with those props on all applicable disks/pools and keeping them updated. Maybe with a tunables for array vs pool to optimize properties for (eg.) an all SSD cache pool and array disks separately.

 

Some way to alias ZFS pools so you could say "this disk is called 'backup'" so it can be in the array but if you change the disk number later you (or automated tools) won't be running commands on the wrong disk. I assumed that would exist natively in ZFS but hell if I can find it.

 

Auto (and/or offer to) create datasets and zvols in the Docker container and VM setup pages, with property tuning. (I keep a separate child dataset for every container in appdata to allow rolling back or cloning one appdata folder from a snapshot without affecting the rest.) Perhaps a one-click way to exclude certain subfolders from snapshots on their parent datasets. I've done this manually and it isn't the worst but it's kind of a drag especially if you forget to do it before there's data in there.

 

I'm sure there's more I'm forgetting, but that comes to mind at the moment.

[Renegade605]

One thing not related to new features is that I'd suggest is that sending notifications through Unraid's notification handler is largely unnecessary. Especially since some are sent as warning importance, I get an email for "cannot unmount x; dataset busy" (for example), even though I already knew that from the popup as soon as I attempted it. Notifications should be saved for things that the system does while unattended that I'd otherwise miss.

[Iker]

Going through the suggestions:

1. znapzend integration with a nice GUI: I also use znapzend for dataset snapshot management; however, the project looks abandoned, so I don't know if it is a good idea to integrate it directly. I've been thinking about how to provide this functionality, even up to the point of coding it by myself, but the path could be clearer. If you folks think that Zanpzend is a good solution, I am on board with that, but there are other options that I'm also open to.
2. Samba to use shadow copy: Another good old functionality that worked well when ZFS was a plugin. But, this is more for unraid to implement, not this plugin.
3. ZFS properties in the Share: Also a suggestion for Unraid to implement, not this plugin.
4. Alias ZFS pools: I am not sure if I follow how this should work or what benefits it brings. On the other hand, I don't use ZFS disks on the array; I use exclusively pools with a a dumb USB stick on the array.
5. Autocreate datasets and zvols in the Docker container and VM setup pages: Sorry, this is also for Unraid to implement.
6. Sending notifications through Unraid's notification handler: I'm planning to ditch those notifications and leave the code within the plugin for future functionality.