[PLUGIN] ZFS Master

[Tullaris]

I wasnt sure what you mean by remove the exclusion because it was empty.

There only was this kind of example text by default i have never wrote anything in those textboxes since i used this plugin.

So i entered /eee/ and enabled the interval and this nothing shows up.

[Iker]

Sorry I confuse the placeholder with an actual value, I'll send you a couple DMs over the weekend for additional info from your polls.

[jbartlett]

Running ZFS Master on 6.13.0-beta.2

 

When creating a new Dataset, the directory does not have write permission granted on it. Requires manually adding the permission to be usable in a SMB share.

 

[Iker]

Yes, it's by design; a couple of versions ago, the permissions scheme was changed to 755, with nobody as the owner and users as the group by default, which I believe is pretty much the standard in Unraid and also the most common use case. I can add an option on the dataset actions menu to change permissions if that is convenient for you.

Edited May 3 by Iker

[Revan335]

The Exclusion Hint is not working. /dockerfiles/.*

 

/docker/.* not working

/docker.* working but include the docker and docker_old Share/Folder

 

Can you fixed this?

[Iker]

Hi, a new version has dropped; please report any new or not resolved problems that you may face. Changelog:

 

2024.05.05

• Fix - Malicious content in SweetAlert2 package. Thanks @Ubsefor
• Add - Initial support for ZFS Vols (Just detection, more is coming)
• Fix - Exclussion patterns keywords

[Revan335]

10 hours ago, Iker said:

Fix - Exclussion patterns keywords

Mean this my Issue?

 

Its not working. > /docker/.*

 

With /docker.* two folders are hint/exclude.

 

This is without filter:

Edited May 6 by Revan335

[Iker]

It's working, or at least as it has been working for a while. Datasets are not entirely hidden if you use ".*" termination, but his children are not listed, as it's helpful to know how much space they take and what their properties are. If you want to get rid entirely of the dataset on the main interface, use "/docker", but be aware that it can match multiple datasets outside of the ones you mentioned.

[HumanTechDesign]

Hi there, since the latest upgrade I have the following issue: I have a zpool (backuppool) on an external USB HDD. Since the update, the refresh of the dataset list on the main tab does not work anymore. Also (and this is the bigger issue), there is a constant read load of around ~400K/s for the USB HDD. Based on iotop, this seems to be based on the following command - originating from the plugin:

 

15728 be/4 root      393.29 K/s    0.00 B/s  ?unavailable?  zfs program -jn -m 20971520 backuppool /usr/local/emhttp/plugins/zfs.master/scripts/zfs_pool_get_datasets_snapshots.lua backuppool

 

I checked your commit history on github and this seems to have been modified based on the new zvol functionality.

 

Is anyone else seeing this issue?

 

EDIT: I reinstalled the plugin and now the dataset list is back. However, the snapshot lazy load takes extremely long now (a few minutes instead of a few seconds before the update). Is there something that can be done?

Edited May 6 by HumanTechDesign

[Revan335]

3 hours ago, Iker said:

It's working, or at least as it has been working for a while. Datasets are not entirely hidden if you use ".*" termination, but his children are not listed, as it's helpful to know how much space they take and what their properties are. If you want to get rid entirely of the dataset on the main interface, use "/docker", but be aware that it can match multiple datasets outside of the ones you mentioned.

OK, how can I include only the /docker/ folder and his children's? Others like docker_old are visible.

[Iker]

22 hours ago, Revan335 said:

OK, how can I include only the /docker/ folder and his children's? Others like docker_old are visible.

"/docker/.*" works for me, but the "docker" dataset is still showing, I think that's the best you can achieve with the current limitations. (Give it 30 seconds to pickup the config change)

 

On 5/6/2024 at 9:30 AM, HumanTechDesign said:

EDIT: I reinstalled the plugin and now the dataset list is back. However, the snapshot lazy load takes extremely long now (a few minutes instead of a few seconds before the update). Is there something that can be done?

I don't know if that is related directly to the plugin or if the pools over USB HDD are extremely slow when dealing with random reads. You can try the following command and check if it's slow: 

 

zfs list -r -t snapshot -o name,creation backuppool

 

If that's the case, there is nothing to do on my side, as it's something inherent to the pool.

[MiguelNdeCarvalho]

Hey,

 

So I have been using the ZFS Master plugin and it's really cool and help so much managing my ZFS pools, but I saw that my pools weren't able to spin down, and I could track that it was related to ZFS Master plugin refreshing the information every 1 minute, so even if I spun them down they should spin back up again within a minute. I tried changing the "refresh interval" to "no refresh", but it was still updating every minute.

 

Thanks,

MiguelNdeCarvalho

[Iker]

Hi, there is an indicator for the last successful refresh, it includes the date and hour up to the second, if that doesn't change, it means that it's not refreshing, also, please give it 30 seconds to pickup the new config and check again if it's working as expected.

[MiguelNdeCarvalho]

16 hours ago, Iker said:

Hi, there is an indicator for the last successful refresh, it includes the date and hour up to the second, if that doesn't change, it means that it's not refreshing, also, please give it 30 seconds to pickup the new config and check again if it's working as expected.

Hey again, so yeah, I gave it more then 30 seconds, I even closed my browser and waited 5 minutes then came back and it was still updating every minute, I was taking a look into that indicator in the "Main" tab. Thanks!

 

Edited: So I have uninstalled the plugin, installed again, then quickly changed the Refresh Interval to "No Refresh" and it now is working as it suppose and don't spin up my disks up anymore.

Edited May 9 by MiguelNdeCarvalho
Update

[Drew6274]

Hey, I just saw the update notes. Do you have more details on "Malicious content in SweetAlert2 package" ?

[Iker]

Hi, you can find more details in the following PR https://github.com/IkerSaint/ZFS-Master-Unraid/pull/15.

[HumanTechDesign]

On 5/7/2024 at 5:45 PM, Iker said:

I don't know if that is related directly to the plugin or if the pools over USB HDD are extremely slow when dealing with random reads. You can try the following command and check if it's slow: 

 

zfs list -r -t snapshot -o name,creation backuppool

 

If that's the case, there is nothing to do on my side, as it's something inherent to the pool.

Thanks for the suggestion. I have been using the plugin for quite a long time now (already under 6.11.x) and I specifically requested the lazy load option as a feature because of this exact USB HDD.

After you implemented it, I never had problems again up until this current version. I am not sure if the added "type" detection is the culprit. I will see if I can manually downgrade again and see if the problem appeared due to other changes to Unraid (which I am not aware of at the moment).

What I don't understand: when this long loading time happens, the ZFS Master area on the Main tab just stays blank (not even showing the "Last refresh at XXX" notification at the top. I can't even trigger a rescan via the button as usual (nothing happens). Once the ZFS master list is populated (and the snapshots are loaded via lazy load), the system returns to normal. I have a feeling that this only happens once after a fresh install or reboot. Is there any kind of caching involved (apart from the time in the refresh delay)?

[Iker]

As you have pointed out, the only change since the last time you were "problems-free" was the addition of a new property to the list pulled from the pools. That's a really simple change, and I don't believe it could be the cause of the problems that you are facing. It's tricky to replicate on my side (I already try, and it works just fine).

 

From my perspective, it might be something related to the USB pool, or even worse; it might signal a dying disk with lots of command timeouts, slow spin-up times, or other errors; please try to execute the zfs list command that I mentioned to you and report back, if the error persists, I can send you a link for installing the plugin without the last commit and test a little bit further, so we can check if that new property is the culprit, if that turns out to be the case I will revert the changes on the main branch and everything returns to normal.

Edited May 16 by Iker

[andyd]

I'm seeing the below which I'm not sure how to handle. I'll get 1 data error any time something gets added to the disk now that related to zfs backup since I use the disk in the array as well

 

Best way to handle? I am thinking of disabling backups (possible?) and reformating the drive to see if that helps. For now, I don't care about backups

 

 

  pool: disk3
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 02:06:06 with 0 errors on Tue May 14 18:00:32 2024
config:

	NAME          STATE     READ WRITE CKSUM
	disk3         ONLINE       0     0     0
	  /dev/md3p1  ONLINE       0     0   164

errors: 1 data errors, use '-v' for a list

 

Edited May 18 by andyd

[JorgeB]

19 hours ago, andyd said:

I'm seeing the below which I'm not sure how to handle.

I would recommend starting by running memtest.

[hamun8]

is it some how possible using a script to run the refresh instead of the default values. I have a drive that only runs 1 a day for ZFS snapshot and replication so with like it to refresh during that time instread of between 0,5 min to 30 min