Iker
-
Posts
270 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by Iker
-
-
I'l take a look; but most likely it's going to take a while any new major features. Currently I'm working on rewriting some of the backend; the load/refresh speed it's not really that good so I'm exploring ZFS API/Programs that should help the plugin to be free of parsing errors and improve overall speed.
-
1
-
-
58 minutes ago, RedTechie said:
One feature request I would have is the ability browse to the dataset using the built in Unraid web based file browser....
In the early days of the plugin that was a feature; but... it's not possible for all the pools; for example, my main pool is "hddmain" mounted in "/hddmain"; the file browser doesn't work there, why? i don't have any idea :S, it works for root directories like boot or mnt, but not for others like tmp or home; so it's not a problem on my end, but probably a filebrowser restriction on which directories it could navigate.
-
New update, now you can take snapshots; in the config section, it's possible to define the pattern and prefix for snapshots name; all the dataset buttons (Snapshot and Destroy) have been merged to "Actions".
@gyto6 The bulk deletion of snapshots it's in the near future plans as a button in the "Admin Dataset Snapshots".
-
1
-
1
-
1
-
-
@ich777 Github link added!, for the support link, I'll wiat for the next update (A couple of days).
@muddro Thanks for the feedback; 1. It will be great; I'll give a thought to the UI design and add on future versions; 2. I'm not so sure if it's convenient; having dangling snapshots mounted here and there could be troublesome; the current way of using .zfs directory to access the snapshots seems more convenient for me.
-
1
-
-
6 hours ago, gyto6 said:
Sounds like a good idea indeed. 😁
Most of the answers are "yes"; the snapshot administration is per Dataset; otherwise, it could be incredibly complicated; my main Pool has 1400 Snapshots, for example. If there are no Snapshots for the Dataset, the button is disabled; The functionality for taking Snapshots over the GUI is on the way; the next update probably.
BTW, Support Topic is ready
-
1
-
-
What is ZFS Master?
The ZFS Master plugin provides information and control over the ZFS Pools in your Unraid Server. Available ZFS Pools are listed under the "Main/ZFSMaster" tab. This plugin requires Unraid 6.12 or higher; for Unraid 6.11 or lower ZFS for Unraid 6 Plugin needs to be installed.
Features (All related to ZFS)
- Main tab GUI integration
- Report Health for the Pool
- Scrub Pools and Export Pool
- Lazy load for snapshot information
- Cache last data when using "no refresh" option
- Create new Datasets, including options for name, mount point, access time, case sensitive, compression, quota, permissions, extended attributes, record size, primary cache, read-only, encryption, and size.
- Edit Datasets properties like compression, access time, quota, etc.
- List children directories on a given dataset.
- Convert directories to datasets.
- Destroy Datasets and an option for forcing and recursively destroy all children and dependents
- Filter the Datasets listed in the GUI using Lua Patterns.
- Clone snapshots and Promote datasets
- Dataset Snapshots Administration (Take a snapshot, Hold a snapshot, Released it, Rollback to it, Batch Delete and Destroy them)
- Destructive Mode, you could restrict buttons that appear in the GUI by turning on or off destructive mode.
- GUI Indicator of last snapshot date, through an icon on the right.
Note About Disks not Spinning Down
ZFS Master wakes up the disks only when you open the dashboard in the main tab; this is because some snapshot information needs to be read from the disks, but again, this only happens when you open the dashboard in the main tab. If you want your disks not to spin up, you have to change the "Refresh Interval" option on the settings to "No refresh"; this will cause the information not to be loaded, even on page refresh.
Configuration
Unraid Settings Menu-> ZFS Master. It's possible to specify the refresh interval for the GUI, if destructive mode is on or off; Lua pattern for dataset exclusions; these are convenient if you are using a docker folder in a Dataset; and the amount of days used for the Snapshot Icon alert and snapshot options for naming snapshots taken from the GUI.
Exclusion Patterns (Or Why I'm seeing a lot of docker daemon related datasets)?
When you configure docker to use a directory on a ZFS Pool, the docker daemon detects the underlying filesystem and uses some of its capabilities; in this case, it creates multiple datasets and snapshots; this slows quite a bit the user interface, that is why you have to exclude such directory; this can be configured using the "Datasets Exclusion Patterns" option. To learn how to create your exclusion pattern, please check http://lua-users.org/wiki/PatternsTutorial.
What is Lazy Load?
Lazy load is a feature (You should enable it on the settings page) that loads the data in two stages:
- Load datasets: This stage loads all the datasets and the associated information for the pools (size, attributes, etc.) except the Snapshots data. This little change improves initial loading times by up to 90% (Less than a second in most cases). However, be aware that all Snapshot-related information and options will be unavailable until the second stage finish.
- Load Snapshots: In this stage, the snapshot information and options are loaded and updated dynamically in the GUI; the time this takes depends on how many datasets and snapshots you have in your pool. This change increments the total load time up to 15%; however, the interface feels more responsive.
In summary, Lazy Load provides a very good improvement on initial load times, but it increments the total load time; the following is a comparison of what you can expect:
- Classic Load: Load time - 1.4s
- Lazy Load: Load Datasets time - 196ms, Load Snapshots: 1.65s (This includes the initial 196 ms).
What is Directory Listing?
Directory Listing is a feature (You should enable it per Dataset or in the plugin configuration) that lists the top-level folders for a given dataset. This functionality should give you better visibility over your pools, allowing you to spot possible duplicates and directories that may be associated with leftovers of a migration.
The folders are listed as children elements, after the datasets and with a different icon (a folder); the plugin doesn't gather any information about the directory besides its name. Given that a Dataset Snapshot covers his subfolders, the Snapshots count is associated with all the subfolders, even if the folders are brand new and not present in any snapshot; this is by design.
This new feature needs to be enabled per Dataset, using the Actions menu or the plugin configuration; it's important to note that it may impact loading times up to 5 or 10% depending on the number of folders under the Dataset.
How Convert to Dataset Works?
The process is divided into three steps:
- Rename Directory: Source directory is renamed to <folder_name>_tmp_<datetime>
- Create Dataset: A dataset with the directory's original name is created in the same pool (and path); the dataset options are the default ones.
- Copy the data: Data is copied using the command "rsync rsync -ra --stats --info=progress2 <source_directory> <dataset_mountpoint>"; the GUI displays a dialog with a progress bar and some relevant information about the process.
If anything fails on steps 1 or 2, the plugin returns an error, and the folder is renamed back to its original name. If something fails in step 3, an error is returned, but the dataset and directory remain intact.
ChangeLog
2024.05.05
- Fix - Malicious content in SweetAlert2 package. Thanks @Ubsefor
- Add - Initial support for ZFS Vols (Just detection, more is coming)
- Fix - Exclussion patterns keywords
2024.02.15
- Fix - Directory Listing not Working
2024.02.10
- Add - "-X" option for Connvert Dataset rsync command
- Fix - Directory Listing detecting datasets as folders
2024.02.9
- Add - Convert directory to dataset functionality
- Add - Written property for snapshots
- Add - Directory listing for root datasets
- Fix - Tabbed view support
- Fix - Configuration file associated errors
- Fix - Units nomenclature
- Fix - Pool information parsing errors
- Remove - Unraid Notifications
2023.12.8
- Add - Directory Listing functionality
- Fix - Optimize multiple operations
2023.12.4
- Fix - Used and Free % bars/texts are now consistent with unraid theme and config
- Fix - Set time format for the last refresh to short date and time
- Fix - Detect Pools with used % under 0%
- Fix - ZPool regex not caching some pools with dots or Underscore in the name
2023.10.07
- Add - Cache last data in Local Storage when using "no refresh"
- Fix - Dataset admin Dialog - Error on select all datasets
- Fix - Multiple typos
- Fix - Special condition crashing the backend
- Fix - Status refresh on Snapshots admin dialog
- Change - Date format across multiple dialogs
- Change - Local Storage for datasets and pools view options
2023.09.27
- Change - "No refresh" option now doesn't load information on page refresh
- Fix - Dynamic Config reload
2023.09.25.72
- Fix - Config load
- Fix - Exclusion patterns for datasets with spaces
- Fix - Destroy dataset functionality
2023.09.25
- Add - Lazy load functionality
- Add - Nchan for updates
- Add - Refresh options (Including on demand)
- Add - Last refresh timestamp
- Change - Quota Unit setting on Create Dataset Dialog
- Change - Notifications and messages improvement
- Change - Edit datasets UI as a dropdown menu
- Fix - Default permissions for datasets (u:nobody, g:users)
- Fix - Dataset passphrase input not masked
- Fix - ZPool regex not caching some pools
- Fix - Dataset passphrase size difference
- Fix - Multiple typos
- Fix - PHP 8 Compatibility
2022.07.04
- Fix - Dataset names with spaces not being properly handled
2023.04.03
- Add - Rename datasets UI
- Add - Edit datasets UI
- Add - unRaid 6.12 compatiblity
- Add - Lazy load for snapshots admin UI
- Fix - Improve PHP 8 Compatibility
2023.02.28
- Fix - PHP 8 Upgrades
- Fix - Export pool command
- Fix - Error on parsing dataset origin property
2022.12.04
- Fix - Error on counting childs
2022.11.12
- Fix - Error on dialogs and input controls
- Add - Clone capabilities for snapshots
- Add - Promote capabilities for datasets
2022.11.05
- Fix - Error on pools with snapshots but without datasets
- Fix - Dialogs not sizing properly
- Add - Snapshot Batch Deletion
2022.08.21
- Change - UI into "folder" structure
- Add - Support for ZFS Encryption
- Add - Unlock and Lock actions for encrypted datasets
- Fix - Error on unRaid 6.9.2 associated with session management
2022.08.02
- Warning - Please Update your exclusion pattern!
- Add - Browse Button for Datasets
- Add - Support for listing volumes!!
- Add - Lua script backend for loading dataset information (50% faster loading times)
- Change - Exclusion pattern for datasets (Please check http://lua-users.org/wiki/PatternsTutorial)
- Change - UI columns re-organized to the unraid way (sort of)
2022.04.13
- Add - Dataset Snapshot Creation Option
- Add - Settings for Snapshot Creation (pattern and prefix)
- Change - "Destroy" and "Snapshots" buttons merged to "Actions"
2022.04.10
- Add - Dataset Snapshot management (rollback, hold, release, destroy)
- Fix - Installation script bug
2022.04.08
- Add - Set permissions for new Datasets
2021.11.09a
- Add - List of current Datasets at Dataset Creation
- Add - Option for export a Pool (In construction)
- Fix - Compatibility with unRAID RC versions
2021.10.08e
- Add - SweetAlert2 for notifications
- Add - Refresh and Settings Buttons
- Add - Mountpoint information for Pools
- Add - Configurable Settings for Refresh Time, Destructive Mode, Dataset Exclussions, Alert Max Days Snapshot Icon
- Fix - Compatibility with Other Themes (Dark, Grey, etc.)
- Fix - Improper dataset parsing
- Fix - Regex warnings
- Fix - UI freeze error on some system at destroying a Dataset
- Remove - Unassigned Devices Plugin dependency
2021.10.04
- Initial Release.
Official GitHub Repo
https://github.com/IkerSaint/ZFS-Master-Unraid-
3
-
Hey folks, a minor update about ZFS Master:
Today I have released version "2022.04.10.42"; this includes a couple of new things:
- In the Dataset creation Dialog, there is a new field for setting the permissions of the new dataset.
- A new button called "Snaps" in the main UI (Requires Destructive mode On).
- The "Snaps" button allows some primary administration of the Dataset Snapshots (Hold a snapshot, Released it, Rollback to it, and Destroy it)
Any feedback regarding this new functionality or even an old one, including UI design, will be appreciated. I expect to continue working and improving plugin features, like creating snapshots and adding support for volume; i'll request specific thread for the Plugin support; so we could stop flooding this thread :P.
-
2
-
12 hours ago, gyto6 said:
Hi everyone,
It's been a while I noticed it, but some of the dockers datasets display a yellow icon, even the snapshot as we can see in the screenshot. Those managed by docker itself, not mines.
I imagine that it means that the dataset or snapshot is damaged if the colors rules are the same as for the pools.
Everything is working whatever. I'd simply ask to you how you manage this on your own servers or if you let it sink?
Well, there is no ZFS Master support topic yet... so I answer that question right here :P; the color of the camera icon is about the last snapshot date; blue means everything is fine, grey means that there is no snapshots for the dataset; and yellow/orange means that the last snapshot is older than the amount of days that you configured in the settings. BTW, ZVols are not supported yet; I hope to get some time off in the coming months, so I can get hands on all the functionality in the plugin.
-
@ich777 That was my first intention, but my local branches are completely mess up so, it was quicker to just updated the current.
@subivoodoo That's one of the features that I'm working to be release in a couple of versions.
@diannao Now, I'm not so sure about your error, mainly because for the fix, I replicated your config and the plugin was working just fine; and on the other hand, you posted that at least the dataset info was working fine a couple of days back; I'm going to send you a PM with a set of commands to be executed, so I can debug in deep what is wrong with the plugin in your system.
-
@JJJonesJr33 @diannao The fixed version is already live, as I'm planning a new version with additional features, the fix was applied to the current version; what that means is that you have to uninstall and install the plugin again in order to get the version with the fix.
Thanks for your help!.
-
1
-
-
I'm completely sure that is a parse error, however with the specifics I could modify the regular expressions to catch this case and the plugin to work properly; thanks for the info I get back to you when the update is live.
-
7 hours ago, diannao said:
That could be a coincidence that the problem is caused by other factors?
How do I check the logs of the ZFS Master plugin?
Sorry for the delay, @diannao please post an image with the result of the commands "zpool status -v" and "zpool list -v" ; this is the first step that the plugin performs to identify all the pools and then, list the datasets that belong to the pool; I will deploy a clone of the configuration that you post, so I can debug and update the plugin according to the results.
-
2 hours ago, diannao said:
Why can't I display the dataset information in the webui after I updated this version.
No response when i clicking the button. and no useful information in the webui logs.
But the zpool actually contains not only the dataset but also the snapshot.
That's weird, I already updated zfs to the latest version and never saw that problem; could you please share your ZFS & ZFS Master plugin version?
-
On 1/7/2022 at 9:08 AM, zxhaxdr said:
I found a strange behaviour for ZFS Master. I have three pools that are using lz4 as the compression algorithm. I changed one of them to gzip-9, and ZFS Master stopped showing dataset info. Only one row which has SHOW DATASETS, SCRUB POOL, EXPORT POOL and CREATE DATASET is showing. The rest pools are displayed correctly. Also, I checked with the command line, and the pool with gzip is normal.
Does anyone else have this issue?
Most probably it's a parse error; I'm planning a new version with a couple of new features, let met check that bug and get back when it's fixed. Thanks for using the plugin, by the way.
-
1
-
-
51 minutes ago, asopala said:
Would the recycle bin vfs object setting need to be applied to every SMB share in smbextra.config? I'm running into a 2048 character limit and have been dealing with that.
You could use the Global Section, but I'm not really an expert in SMB, so YMMV; however, another option is editing the smb-extra.conf file directly from the flash/flash-share so you don't have the 2048 characters limit from the GUI.
-
ZFS Master 2021.11.09a is live with a few changes, check it out:
2021.11.09a
- Add - List of current Datasets at Dataset Creation
- Add - Option for export a Pool
- Fix - Compatibility with RC version of unRAID
-
3
-
-
Yeah, it's mainly a guess, based on the info in the thread and what users are asking for. I agree with @jortan that the magic of unRaid is having a bunch of different size disks and just made them work as one with redundancy without so much trouble, and it's okay, that always should be an option, however, being the only option it's not sustainable for the times to come, unRaid is a business, they have a market, getting behind in characteristics that the competition offers is not a great move for them.
Every month it's announced a new SATA SSD cheaper than ever, a board with 2.5Gb/5Gb network card included; think about the new 2022 offers from intel & amd with ddr5, pcie5, etc; all of that outperform by a lot unRaid current storage strategy; check Linus videos, there is a long time since he used the array for anything. If unRaid keeps the current "array" for a couple of years, it's not going to be competitive; and don't get me wrong, I have been using unRaid for the last 5 years, I Love it!, but it has started to feel a little old in terms of storage capabilities (snapshots, backups without having to copy 300 GB vms disk every night, wait for hours for a copy to complete, 10Gb support, etc...). For me this ZFS Plugins had been a game changer, make unRaid just the perfect modern system, I really hope that unRaid give us options and not just stick to the old good times.
-
1
-
-
It's mentioned later in the same page… and a lot on next pages; however, as I mentioned, it doesn't matter, the point is the same is just an import/export operation, the important part and what unRaid brings into the game it's the GUI.
-
Probably, in the features for 6.11 pool is mentioned that array devices could be formatted as ZFS, but, at least for me, it doesn't change anything, is just an import/export operation.
-
3 hours ago, asopala said:
I ask cause currently the plugin works through Unassigned Devices, and if it eventually switches so that the array can be on ZFS, I'm wondering if that would cause an issue there where I'd need to wipe my data.
It's a fair question, however, unRaid it's not going to personalize ZFS, just offer it as an option for the array file system; the most likely (and Easy) scenario is that you only have to export & import your pool, the information there will be intact. Take for example SpaceInvaderOne videos about exporting the pool from unRaid and importing it in TrueNas.
-
1
-
-
2 hours ago, ensnare said:
When I upgrade to 6.10.0-rc2g ZFS downgrades to 2.0.6 (from 2.1.1).
You could check in the repository the available ZFS versions for you unRaid.
Repository:
https://github.com/Steini1984/unRAID6-ZFS/tree/master/packages
-
Hope the rc2 is more stable for you, zfs 2.1 introduced draid and even is you are not using it, it's a nice to have. In the monitoring aspect there are more options with zpool_influxdb and some already present are different, when my dashboard is more mature I will write another post about it. This is a little sneak peek:
-
16 minutes ago, Joly0 said:
Alright, i moved all the files to a dataset, but the legacy datasets remained. Even after completly moving it away from the zfs pool. Any idea or command on how to quickly delete/destroy all those legacy datasets?
Maybe it's a better idea to set the docker folder from the GUI to another point (reinstalling every docker is mandatory), with zfs as the underlying system, it's not just "move the files". Once that is done this should be enough:
zfs list -o name,mountpoint | grep legacy | awk '{printf "zfs destroy -frR %s\n", $1}'The output it's all you need to destroy all of those remaining datasets, be extremely careful review every single line including the path, the command has no filters whatsoever.
-
Hi, thanks; initially I thought filtering by attributes, however, the filter implemented its by name. I'm under the impression that your docker folder is the root of your RaidZ2, it's correct? If that the case you should find a regular expression that suits the name of the folders, maybe something like "/^RaidZ2\/([0-9A-Fa-f]{2})+$/", but really, you should move docker folder to a dataset of his own.
[PLUGIN] ZFS Master
in Plugin Support
Posted
Could you please post the output (In Text) of this command:
zpool list -v