Iker

New update, now you can take snapshots; in the config section, it's possible to define the pattern and prefix for snapshots name; all the dataset buttons (Snapshot and Destroy) have been merged to "Actions". @gyto6 The bulk deletion of snapshots it's in the near future plans as a button in the "Admin Dataset Snapshots".

@ich777 Github link added!, for the support link, I'll wiat for the next update (A couple of days). @muddro Thanks for the feedback; 1. It will be great; I'll give a thought to the UI design and add on future versions; 2. I'm not so sure if it's convenient; having dangling snapshots mounted here and there could be troublesome; the current way of using .zfs directory to access the snapshots seems more convenient for me.

Most of the answers are "yes"; the snapshot administration is per Dataset; otherwise, it could be incredibly complicated; my main Pool has 1400 Snapshots, for example. If there are no Snapshots for the Dataset, the button is disabled; The functionality for taking Snapshots over the GUI is on the way; the next update probably. BTW, Support Topic is ready

What is ZFS Master? The ZFS Master plugin provides information and control over the ZFS Pools in your Unraid Server. Available ZFS Pools are listed under the "Main/ZFSMaster" tab. This plugin requires Unraid 6.12 or higher; for Unraid 6.11 or lower ZFS for Unraid 6 Plugin needs to be installed. Features (All related to ZFS) Main tab GUI integration Report Health for the Pool Scrub Pools and Export Pool Lazy load for snapshot information Cache last data when using "no refresh" option Create new Datasets, including options for name, mount point, access time, case sensitive, compression, quota, permissions, extended attributes, record size, primary cache, read-only, encryption, and size. Edit Datasets properties like compression, access time, quota, etc. List children directories on a given dataset. Convert directories to datasets. Destroy Datasets and an option for forcing and recursively destroy all children and dependents Filter the Datasets listed in the GUI using Lua Patterns. Clone snapshots and Promote datasets Dataset Snapshots Administration (Take a snapshot, Hold a snapshot, Released it, Rollback to it, Batch Delete and Destroy them) Destructive Mode, you could restrict buttons that appear in the GUI by turning on or off destructive mode. GUI Indicator of last snapshot date, through an icon on the right. Note About Disks not Spinning Down ZFS Master wakes up the disks only when you open the dashboard in the main tab; this is because some snapshot information needs to be read from the disks, but again, this only happens when you open the dashboard in the main tab. If you want your disks not to spin up, you have to change the "Refresh Interval" option on the settings to "No refresh"; this will cause the information not to be loaded, even on page refresh. Configuration Unraid Settings Menu-> ZFS Master. It's possible to specify the refresh interval for the GUI, if destructive mode is on or off; Lua pattern for dataset exclusions; these are convenient if you are using a docker folder in a Dataset; and the amount of days used for the Snapshot Icon alert and snapshot options for naming snapshots taken from the GUI. Exclusion Patterns (Or Why I'm seeing a lot of docker daemon related datasets)? When you configure docker to use a directory on a ZFS Pool, the docker daemon detects the underlying filesystem and uses some of its capabilities; in this case, it creates multiple datasets and snapshots; this slows quite a bit the user interface, that is why you have to exclude such directory; this can be configured using the "Datasets Exclusion Patterns" option. To learn how to create your exclusion pattern, please check http://lua-users.org/wiki/PatternsTutorial. It's important to note that, if you are using Unraid 7 you can change the docker filesystem driver to overlay2 and get rid of this issue. What is Lazy Load? Lazy load is a feature (You should enable it on the settings page) that loads the data in two stages: Load datasets: This stage loads all the datasets and the associated information for the pools (size, attributes, etc.) except the Snapshots data. This little change improves initial loading times by up to 90% (Less than a second in most cases). However, be aware that all Snapshot-related information and options will be unavailable until the second stage finish. Load Snapshots: In this stage, the snapshot information and options are loaded and updated dynamically in the GUI; the time this takes depends on how many datasets and snapshots you have in your pool. This change increments the total load time up to 15%; however, the interface feels more responsive. In summary, Lazy Load provides a very good improvement on initial load times, but it increments the total load time; the following is a comparison of what you can expect: Classic Load: Load time - 1.4s Lazy Load: Load Datasets time - 196ms, Load Snapshots: 1.65s (This includes the initial 196 ms). What is Directory Listing? Directory Listing is a feature (You should enable it per Dataset or in the plugin configuration) that lists the top-level folders for a given dataset. This functionality should give you better visibility over your pools, allowing you to spot possible duplicates and directories that may be associated with leftovers of a migration. The folders are listed as children elements, after the datasets and with a different icon (a folder); the plugin doesn't gather any information about the directory besides its name. Given that a Dataset Snapshot covers his subfolders, the Snapshots count is associated with all the subfolders, even if the folders are brand new and not present in any snapshot; this is by design. This new feature needs to be enabled per Dataset, using the Actions menu or the plugin configuration; it's important to note that it may impact loading times up to 5 or 10% depending on the number of folders under the Dataset. How Convert to Dataset Works? The process is divided into three steps: Rename Directory: Source directory is renamed to <folder_name>_tmp_<datetime> Create Dataset: A dataset with the directory's original name is created in the same pool (and path); the dataset options are the default ones. Copy the data: Data is copied using the command "rsync rsync -ra --stats --info=progress2 <source_directory> <dataset_mountpoint>"; the GUI displays a dialog with a progress bar and some relevant information about the process. If anything fails on steps 1 or 2, the plugin returns an error, and the folder is renamed back to its original name. If something fails in step 3, an error is returned, but the dataset and directory remain intact. Why is the reported free/used space is different from what Unraid shows? The plugin shows the very same information that you get from the regular "zpool" and "zfs" commands; for example, here is a pool reported via zpool list vs Unraid: But why is that? There are two factors: Unraid shows the units using SI (TeraBytes, Gigabytes, etc.) instead of IEC (Tibibytes, Gibibytes, etc.); zfs natively uses IEC units, so there is a discrepancy. The larger the pool, the larger the difference. Unraid shows you the usable space on the GUI interface reported from the filesystem, but that's not the same information reported by ZFS; for example for a RAIDZ-1 config with three disks, you get 2TB * (3-p) with p = 1 disk for redundancy; in practical terms, that means 4 TB of usable space; but at the pool level that's not what gets reported, instead ZFS reports 6 TB (Convert it to IEC TBi and you get 5.5 TB) of total space because that's the pool topology, not the one showed at the filesystem level. Now, at the dataset level (zfs list) ZFS reports the actual numbers for the filesystem because that config applies to the filesystem level, how much space you can use, excluding the parity, slop space, snapshots, etc. ChangeLog 2024.12.08 Add - Config for pulling ZnapZend plans Fix - Refresh and settings icon Fix - Corner case if no information is loaded 2024.11.17 Add - ZnapZend plans information 2024.11.10 Fix - Style for Unraid 7 Add - Properties extraction for ZFS Volumes Fix - for folder listing corner cases 2024.05.05 Fix - Malicious content in SweetAlert2 package. Thanks @Ubsefor Add - Initial support for ZFS Vols (Just detection, more is coming) Fix - Exclussion patterns keywords 2024.02.15 Fix - Directory Listing not Working 2024.02.10 Add - "-X" option for Connvert Dataset rsync command Fix - Directory Listing detecting datasets as folders 2024.02.9 Add - Convert directory to dataset functionality Add - Written property for snapshots Add - Directory listing for root datasets Fix - Tabbed view support Fix - Configuration file associated errors Fix - Units nomenclature Fix - Pool information parsing errors Remove - Unraid Notifications 2023.12.8 Add - Directory Listing functionality Fix - Optimize multiple operations 2023.12.4 Fix - Used and Free % bars/texts are now consistent with unraid theme and config Fix - Set time format for the last refresh to short date and time Fix - Detect Pools with used % under 0% Fix - ZPool regex not caching some pools with dots or Underscore in the name 2023.10.07 Add - Cache last data in Local Storage when using "no refresh" Fix - Dataset admin Dialog - Error on select all datasets Fix - Multiple typos Fix - Special condition crashing the backend Fix - Status refresh on Snapshots admin dialog Change - Date format across multiple dialogs Change - Local Storage for datasets and pools view options 2023.09.27 Change - "No refresh" option now doesn't load information on page refresh Fix - Dynamic Config reload 2023.09.25.72 Fix - Config load Fix - Exclusion patterns for datasets with spaces Fix - Destroy dataset functionality 2023.09.25 Add - Lazy load functionality Add - Nchan for updates Add - Refresh options (Including on demand) Add - Last refresh timestamp Change - Quota Unit setting on Create Dataset Dialog Change - Notifications and messages improvement Change - Edit datasets UI as a dropdown menu Fix - Default permissions for datasets (u:nobody, g:users) Fix - Dataset passphrase input not masked Fix - ZPool regex not caching some pools Fix - Dataset passphrase size difference Fix - Multiple typos Fix - PHP 8 Compatibility 2022.07.04 Fix - Dataset names with spaces not being properly handled 2023.04.03 Add - Rename datasets UI Add - Edit datasets UI Add - unRaid 6.12 compatiblity Add - Lazy load for snapshots admin UI Fix - Improve PHP 8 Compatibility 2023.02.28 Fix - PHP 8 Upgrades Fix - Export pool command Fix - Error on parsing dataset origin property 2022.12.04 Fix - Error on counting childs 2022.11.12 Fix - Error on dialogs and input controls Add - Clone capabilities for snapshots Add - Promote capabilities for datasets 2022.11.05 Fix - Error on pools with snapshots but without datasets Fix - Dialogs not sizing properly Add - Snapshot Batch Deletion 2022.08.21 Change - UI into "folder" structure Add - Support for ZFS Encryption Add - Unlock and Lock actions for encrypted datasets Fix - Error on unRaid 6.9.2 associated with session management 2022.08.02 Warning - Please Update your exclusion pattern! Add - Browse Button for Datasets Add - Support for listing volumes!! Add - Lua script backend for loading dataset information (50% faster loading times) Change - Exclusion pattern for datasets (Please check http://lua-users.org/wiki/PatternsTutorial) Change - UI columns re-organized to the unraid way (sort of) 2022.04.13 Add - Dataset Snapshot Creation Option Add - Settings for Snapshot Creation (pattern and prefix) Change - "Destroy" and "Snapshots" buttons merged to "Actions" 2022.04.10 Add - Dataset Snapshot management (rollback, hold, release, destroy) Fix - Installation script bug 2022.04.08 Add - Set permissions for new Datasets 2021.11.09a Add - List of current Datasets at Dataset Creation Add - Option for export a Pool (In construction) Fix - Compatibility with unRAID RC versions 2021.10.08e Add - SweetAlert2 for notifications Add - Refresh and Settings Buttons Add - Mountpoint information for Pools Add - Configurable Settings for Refresh Time, Destructive Mode, Dataset Exclussions, Alert Max Days Snapshot Icon Fix - Compatibility with Other Themes (Dark, Grey, etc.) Fix - Improper dataset parsing Fix - Regex warnings Fix - UI freeze error on some system at destroying a Dataset Remove - Unassigned Devices Plugin dependency 2021.10.04 Initial Release. Official GitHub Repo https://github.com/IkerSaint/ZFS-Master-Unraid

Hey folks, a minor update about ZFS Master: Today I have released version "2022.04.10.42"; this includes a couple of new things: In the Dataset creation Dialog, there is a new field for setting the permissions of the new dataset. A new button called "Snaps" in the main UI (Requires Destructive mode On). The "Snaps" button allows some primary administration of the Dataset Snapshots (Hold a snapshot, Released it, Rollback to it, and Destroy it) Any feedback regarding this new functionality or even an old one, including UI design, will be appreciated. I expect to continue working and improving plugin features, like creating snapshots and adding support for volume; i'll request specific thread for the Plugin support; so we could stop flooding this thread :P.

Well, there is no ZFS Master support topic yet... so I answer that question right here :P; the color of the camera icon is about the last snapshot date; blue means everything is fine, grey means that there is no snapshots for the dataset; and yellow/orange means that the last snapshot is older than the amount of days that you configured in the settings. BTW, ZVols are not supported yet; I hope to get some time off in the coming months, so I can get hands on all the functionality in the plugin.

@ich777 That was my first intention, but my local branches are completely mess up so, it was quicker to just updated the current. @subivoodoo That's one of the features that I'm working to be release in a couple of versions. @diannao Now, I'm not so sure about your error, mainly because for the fix, I replicated your config and the plugin was working just fine; and on the other hand, you posted that at least the dataset info was working fine a couple of days back; I'm going to send you a PM with a set of commands to be executed, so I can debug in deep what is wrong with the plugin in your system.

@JJJonesJr33 @diannao The fixed version is already live, as I'm planning a new version with additional features, the fix was applied to the current version; what that means is that you have to uninstall and install the plugin again in order to get the version with the fix. Thanks for your help!.

I'm completely sure that is a parse error, however with the specifics I could modify the regular expressions to catch this case and the plugin to work properly; thanks for the info I get back to you when the update is live.

Sorry for the delay, @diannao please post an image with the result of the commands "zpool status -v" and "zpool list -v" ; this is the first step that the plugin performs to identify all the pools and then, list the datasets that belong to the pool; I will deploy a clone of the configuration that you post, so I can debug and update the plugin according to the results.

That's weird, I already updated zfs to the latest version and never saw that problem; could you please share your ZFS & ZFS Master plugin version?

Most probably it's a parse error; I'm planning a new version with a couple of new features, let met check that bug and get back when it's fixed. Thanks for using the plugin, by the way.

You could use the Global Section, but I'm not really an expert in SMB, so YMMV; however, another option is editing the smb-extra.conf file directly from the flash/flash-share so you don't have the 2048 characters limit from the GUI.

ZFS Master 2021.11.09a is live with a few changes, check it out: 2021.11.09a - Add - List of current Datasets at Dataset Creation - Add - Option for export a Pool - Fix - Compatibility with RC version of unRAID

Yeah, it's mainly a guess, based on the info in the thread and what users are asking for. I agree with @jortan that the magic of unRaid is having a bunch of different size disks and just made them work as one with redundancy without so much trouble, and it's okay, that always should be an option, however, being the only option it's not sustainable for the times to come, unRaid is a business, they have a market, getting behind in characteristics that the competition offers is not a great move for them. Every month it's announced a new SATA SSD cheaper than ever, a board with 2.5Gb/5Gb network card included; think about the new 2022 offers from intel & amd with ddr5, pcie5, etc; all of that outperform by a lot unRaid current storage strategy; check Linus videos, there is a long time since he used the array for anything. If unRaid keeps the current "array" for a couple of years, it's not going to be competitive; and don't get me wrong, I have been using unRaid for the last 5 years, I Love it!, but it has started to feel a little old in terms of storage capabilities (snapshots, backups without having to copy 300 GB vms disk every night, wait for hours for a copy to complete, 10Gb support, etc...). For me this ZFS Plugins had been a game changer, make unRaid just the perfect modern system, I really hope that unRaid give us options and not just stick to the old good times.

It's mentioned later in the same page… and a lot on next pages; however, as I mentioned, it doesn't matter, the point is the same is just an import/export operation, the important part and what unRaid brings into the game it's the GUI.

Probably, in the features for 6.11 pool is mentioned that array devices could be formatted as ZFS, but, at least for me, it doesn't change anything, is just an import/export operation.

It's a fair question, however, unRaid it's not going to personalize ZFS, just offer it as an option for the array file system; the most likely (and Easy) scenario is that you only have to export & import your pool, the information there will be intact. Take for example SpaceInvaderOne videos about exporting the pool from unRaid and importing it in TrueNas.

You could check in the repository the available ZFS versions for you unRaid. Repository: https://github.com/Steini1984/unRAID6-ZFS/tree/master/packages

Hope the rc2 is more stable for you, zfs 2.1 introduced draid and even is you are not using it, it's a nice to have. In the monitoring aspect there are more options with zpool_influxdb and some already present are different, when my dashboard is more mature I will write another post about it. This is a little sneak peek:

Maybe it's a better idea to set the docker folder from the GUI to another point (reinstalling every docker is mandatory), with zfs as the underlying system, it's not just "move the files". Once that is done this should be enough: zfs list -o name,mountpoint | grep legacy | awk '{printf "zfs destroy -frR %s\n", $1}' The output it's all you need to destroy all of those remaining datasets, be extremely careful review every single line including the path, the command has no filters whatsoever.

Hi, thanks; initially I thought filtering by attributes, however, the filter implemented its by name. I'm under the impression that your docker folder is the root of your RaidZ2, it's correct? If that the case you should find a regular expression that suits the name of the folders, maybe something like "/^RaidZ2\/([0-9A-Fa-f]{2})+$/", but really, you should move docker folder to a dataset of his own.

As an update, I was checking out ZFS version 2.1, but it was marked as "unstable" on unRAID 6.9.2, so simply YOLO it and upgrade to 6.10rc1, the result, zfs_exporter works just fine; but telegraf is not polling any metrics from the pools.... but, I find zpool_influxdb is working without so much trouble, at least for pools (not datasets), I'm going to start working with just zfs_exporter & zpool_influxdb for the next guide.

It's not really a big deal, I suppose that it's because the folder doesn't exist in unraid, being the whole file system ephemeral and the pools by default does not have the property setted for a cache file, maybe zfs init before the path exist, so it could be very problematic in unraid having the cache file configured for the pools. More info: https://openzfs.github.io/openzfs-docs/Project and Community/FAQ.html#the-etc-zfs-zpool-cache-file https://github.com/openzfs/zfs/issues/1035 Why could be problematic: https://github.com/openzfs/zfs/issues/2433

ZDB as long as I have checked works, however, requires a cache file that doesn't exist in the unRAID config; try with this: UNRAID:~# mkdir /etc/zfs UNRAID:~# zpool set cachefile=/etc/zfs/zpool.cache hddmain UNRAID:~# zdb -C hddmain