[PLUGIN] ZFS Master

[TheThingIs]

Versions

Unraid 6.12.2

ZFS Master 2023.04.03.64

 

ZFS Master list:

 

Actual datasets:

 

Hope that helps

[Iker]

6 hours ago, TheThingIs said:

Versions

Unraid 6.12.2

ZFS Master 2023.04.03.64

...

Hope that helps

 

Please go to "Settings->ZFS Master", and check that the "Datasets Exclussion Patterns (Just One!):" is not an empty space, if it is, please change it to something that doesn't match with any dataset name in your pools.

[TheThingIs]

it wasn't an empty space, just empty as I'd never touched it. However, changing the pattern to something like "eee" does make the missing datasets with a space in appear

 

thanks

 

oh and a quick suggestion if I may. When you manipulate the datasets, create or destroy etc, do an immediate refresh. If you have the refresh quite high then your changes take a while to appear and it makes you wonder if the action worked.

Edited July 4, 2023 by TheThingIs

[Iker]

Hi guys, a new minor-update is live with the following changelog:

 

2022.07.04

• Fix - Dataset names with spaces not being properly handled

This applies both for creation and listing, so you should be able to create and list datasets that contain spaces in the name. Now the exclussion pattern supports true "empty" values, so you can  leave empty and it should work without problems.

 

4 hours ago, TheThingIs said:

oh and a quick suggestion if I may. When you manipulate the datasets, create or destroy etc, do an immediate refresh. If you have the refresh quite high then your changes take a while to appear and it makes you wonder if the action worked.

 

That is planned for the next major version that I'm working on, 

Edited July 5, 2023 by Iker

[boomam]

On 7/3/2023 at 12:51 PM, Iker said:

Quota shows B because is the smaller unit, but you can specify it in K, M, G, T at dataset creation or when you edit a dataset. (Probably worth to include the unit as a list in the Dataset creation dialog for clarity)

Thanks.

Attempted to put in 1T as a test on a dataset, but it doesnt want to apply.

 

Unable to update dataset
Output: 'cannot set property for '\''storage/dataset-test'\'': '\''compression'\'' must be one of '\''on | off | lzjb | gzip | gzip-[1-9] | zle | lz4 | zstd | zstd-[1-19] | zstd-fast | zstd-fast-[1-10,20,30,40,50,60,70,80,90,100,500,1000]'\'''

 

Tested with various settings, and not touching them beyond what shows up in the popup. Always some variant of the above.

As an example with the following settings - 

• Compression - Inherit
• Quota - 1 T
• Record Size - Inherit
• Access Time - Off
• Extended Attributes - On
• Primary Cache - Inherit
• Read Only - Off
• Sync - Standard (Default)

Unable to update dataset
Output: 'cannot unmount '\''/mnt/storage/dataset-test'\'': pool or dataset is busy'

 

[Iker]

I just tried with multiple combinations as you suggest and you are correct, there seems to be a bug in one of the commands. My best shoot will be that it's in the inherit part, that's particularly tricky. Let me do a couple more checks and I will include the fix in the next update. In the meantime, this is the cmd equivalent for seting the quota for a Dataset:

 

zfs set quota=1T storage/dataset-test

 

Thanks for the report!.

[boomam]

1 hour ago, Iker said:

I just tried with multiple combinations as you suggest and you are correct, there seems to be a bug in one of the commands. My best shoot will be that it's in the inherit part, that's particularly tricky. Let me do a couple more checks and I will include the fix in the next update. In the meantime, this is the cmd equivalent for seting the quota for a Dataset:

 

zfs set quota=1T storage/dataset-test

 

Thanks for the report!.

No worries, I can wait on the final fix and then test that if you want? I'm in no rush to apply the quota. 🙂

[dopeytree]

Investigating issue of array drives never spinning down or not staying spun down.

A small amount of data i.e a few Kb is written everyone & then causing the to stay alive.

Nothing is show in file activity or open files. 

 

Array is 

Parity

Disk 1 - XFS

Disk 2 - XFS

Disk 4 - ZFS

 

Cache - ZFS

Cache mirror - ZFS

 

Sata-cache - ZFS

Sata-cache mirror - ZFS

 

Jorge mentioned this plugin can cause drives to stay spun up. 

Having a look through this thread, it looked like those with HBA LSI SAS cards were having the issue? Did anyone get anywhere with this?

 

I have a 9207-8i - I did notice the firmware today was on v13 so updated to the latest v20.07

 

My unraid is on 6.12.2 & didn't notice issue on 6.12 or 6.12.1

 

May be a general bug with spin down in 6.12.2 as I notice the default spin down is not saved to other drives. And manual setting does not persist.

 

 

moulin-rouge-diagnostics-20230704-2348.zip

Edited July 5, 2023 by dopeytree

[Iker]

3 hours ago, dopeytree said:

Jorge mentioned this plugin can cause drives to stay spun up. 

Having a look through this thread, it looked like those with HBA LSI SAS cards were having the issue? Did anyone get anywhere with this?

moulin-rouge-diagnostics-20230704-2348.zip 138.29 kB · 0 downloads

 

That's correct; my latest investigation about this issue is that "querying" snapshot names reads the data from the disk, and by that, it spins up those disks; unfortunately, there is nothing to do about it; however, I want to be very clear about this, because it seems that there is some misinformation about the matter; ZFS Master wakes up the disks only when you open the dashboard in the main tab, there are no background processes or any other activity performed by the plugin outside of when you open your dashboard; if you keep it closed or in another tab, there is no way that the plugin is the culprit for preventing the disk from spinning down.

[Revan335]

An Option for Exclude the Docker Images thats are automated displayed as Datasets and there Snapshots.

 

Thats have no Function. The Snapshots can't removed and the Datasets are Docker/ZFS Function. I have the Information from this: https://docs.docker.com/storage/storagedriver/zfs-driver/#image-layering-and-sharing

 

And an Option for the system Share. That's converted to Datasets the Docker ... stop Correct Working.

 

A Example:

Edited July 9, 2023 by Revan335

[Iker]

3 hours ago, Revan335 said:

An Option for Exclude the Docker Images thats are automated displayed as Datasets and there Snapshots.

 

Check the Configuration section in the first post of this thread. You can define a pattern for excluding datasets from showing in the UI, for you specific case "/system/.*" should do the trick.

Edited July 9, 2023 by Iker

[Revan335]

13 hours ago, Iker said:

 

Check the Configuration section in the first post of this thread. You can define a pattern for excluding datasets from showing in the UI, for you specific case "/system/.*" should do the trick.

Is this override the existing entry dockerfiles?

[Revan335]

Can you add a Tooltip for this?

For example, when you are using Docker on ZFS its recommended to excluded this Path with /system/docker/*

 

Or you can add the Tooltip for the Information about the Exclusion works and the Default Docker Path as a Exclusion.

[Iker]

I'll see if it is possible to detect if you are using docker directory instead of docker image, and suggest the exclussion. Thanks for your feedback.

[Swarles]

I'm seeing an issue where I believe this plugin is waking up two disks (disk1 and parity).
I can see from earlier comments that it is expected if it is querying snapshots. However what I find strange is that my parity drive is being spun up as well, shouldn't this only happen if a write is occurring? 

Also in my case I only have a snapshot of my 'appdata' share folder which is located on my cache.

For context my setup is as follows (unraid 6.12.2):

Array:

Parity

Disk 1 - ZFS

Pool:

Cache - ZFS

Cache mirror - ZFS

 

I expect that the HDD might spin up if I select the "SHOW DATASETS" in the "main" tab but I usually have these collapsed. I have had the "File Activity" plugin running and it has displayed no files being modified or accessed so I'm a bit stuck. Additionally, I have experienced both disk1 and parity spinning up when entering the "shares" tab as well which was unexpected.

 

Any thoughts/suggestions would be great. I'll try uninstalling the plugin to confirm the issue  

 

On 7/6/2023 at 2:56 AM, dopeytree said:

My unraid is on 6.12.2 & didn't notice issue on 6.12 or 6.12.1

I installed this plugin right after installing 6.12.2 so I too am wondering if my issues are related to this plugin or 6.12.2 haha

 

EDIT: Upon removing the plugin I no longer get HDD start-up when entering the "Main" tab. This includes both disk1 and parity.

Edited July 14, 2023 by Swarles

[Iker]

This is a known issue: 

 

 

The "Show Datasets" Button is a mere graphic thing, not really something that performs an operation on the plugin. There are no plans to change that behavior in the short term. I could include it as part of the versions after the one with the lazy load functionality.

 

Thanks for the report.

[rpmalouin]

If I create a zfs dataset using the plug in on a standalone zsf formatted drive (not part of the array) why does it show as mounted on the array and the stand-alone disk?  When using LuckyBackups to backup array files to a dataset on the stand-alone drive all the data is put on the array disk not the stand alone. 

[rpmalouin]

Disk 1 is xsf (array disk) and Backup is zfs (stand alone)

 

even creating sub-directories data is stored on the array.

Edited July 17, 2023 by rpmalouin
clarify

[Iker]

Seem like a bug given that your Pool it's not really "part" of the standard system pools, however, please check release notes for Unraid 6.12 and the new change in Shares. https://docs.unraid.net/unraid-os/release-notes/6.12.0/. Long story short... set the Share to Exclusive shares only present on your ZFS Pool.

[rpmalouin]

Figured it out, the plugin creates a dataset on zfs just fine, it also creates a share but creates the share as array primary and zfs pool secondary. The fix is to go into the share and change it to zsf pool primary no secondary (in my case). In Lucky Backups I needed to change the share from /mnt/user to just /mnt this will show all my disks. 

Edited July 17, 2023 by rpmalouin

[purplechris]

Apologies if this has already been answered.

 

but with this plugin installed my array zfs disk does not spin down 

[Niklas]

4 hours ago, purplechris said:

Apologies if this has already been answered.

 

but with this plugin installed my array zfs disk does not spin down 

 

Please read back just a couple of posts before yours in this thread. 

 

Like this one 

 

Edited July 18, 2023 by Niklas

[Alyred]

Regarding array disks not spinning down: I've set the re-scan time to 5 minutes, but is there any way we can have ZFS master exclude disks so that we can prevent them from spinning up?  Or as a feature request, only scan a drive if it shows as spun up, or only spin it up if it is making a change in ZFS snapshots/datasets?

 

Having my drives that aren't being used spinning up frequently eliminates one of the main reasons for using UnRAID (power saving/heat reduction/noise reduction on low-use drives).

[Iker]

Hi, I have added to the plugin description on the first page a little message about the subject,

 

Note About Disks not Spinning Down

ZFS Master wakes up the disks only when you open the dashboard in the main tab; this is because some snapshot information needs to be read from the disks, but again, this only happens when you open the dashboard in the main tab.

 

However, I will clarify it again, this only happens when you access the dashboard main tab, given that the plugin doesn't have any background processes. If you close your browser or switch to a different dashboard section, let's say "shares", it does not query any information and will not mess with the disk's sleep.

 

3 hours ago, Alyred said:

Regarding array disks not spinning down: I've set the re-scan time to 5 minutes, but is there any way we can have ZFS master exclude disks so that we can prevent them from spinning up?  Or as a feature request, only scan a drive if it shows as spun up, or only spin it up if it is making a change in ZFS snapshots/datasets?

 

The plugin doesn't query the disks directly but the ZFS system, so, as far as I know, there is no way to "exclude" disks. As stated before, if you are not in the Dashboard Main Tab, the plugin is not running, so if your monitoring solutions tell you that the disks are spinning up even with the browser window closed or in a different section,  there is something else beyond the plugin messing with your disks.

Edited July 19, 2023 by Iker

[Alyred]

39 minutes ago, Iker said:

The plugin doesn't query the disks directly but the ZFS system, so, as far as I know, there is no way to "exclude" disks. As stated before, if you are not in the Dashboard Main Tab, the plugin is not running, so if your monitoring solutions tell you that the disks are spinning up even with the browser window close or in a different section,  there is something else beyond the plugin messign with your disks.

Thanks for the clarification.  I keep a tab open fairly regularly, especially while I'm getting a new server going and monitoring a few other factors before I deploy it in prod. So yeah, it looks like it was just powering on as soon as it was spinning down.

 

Bummer that there's no way to query or skip specific drives in the ZFS system, but at least we know.

 

Can you extend the maximum time option between automatic refreshes, and whenever the plugin is used/a button clicked to create/edit/destroy something? Seems like it wouldn't need to be updated too often in some situations for anyone that doesn't make too many changes.

 

Otherwise, I could just use the dashboard more often than the "main" page...

Edited July 19, 2023 by Alyred