[PLUGIN] ZFS Master

[Iker]

ZFS shows the very same information that you get from the regular "zpool" and "zfs" commands; for example, here is my hddmain pool reported via zpool list vs unraid:

 

But why is that? There are two factors:

1. Unraid shows the units using SI (TeraBytes, Gigabytes, etc.) instead of IEC (Tibibytes, Gibibytes, etc.); zfs natively uses IEC units, so there is a discrepancy. The larger the pool, the larger the difference.
2. Unraid shows you the usable space on the GUI interface reported from the filesystem, but that's not the same information reported by ZFS; in your case, I'm guessing a RAIDZ-1 config, so you get 2TB * (3-p) with p = 1 disk for redundancy; in practical terms, that means 4 TB of usable space; but at the pool level that's not what gets reported, instead ZFS reports 6 TB (Convert it to IEC TBi and you get your 5.5 TB) of total space because that's the pool topology, not the one showed at the filesystem level. Now, at the dataset level (zfs list) ZFS reports the actual numbers you want to know because that config applies to the filesystem level, how much space you can use, excluding the parity, slop space, snapshots, etc.

I hope this sheds some light on why the numbers are different and why I keep them as they are (Because I think they are useful!).

 

PS: BTW, I noticed the plugin is not correctly appending the Pool stats IEC (TiB); I'll fix that in an upcoming release.

 

 

 

Edited May 26 by Iker

[Yock]

Many thanks for the answer, that does indeed explain it!

 

Keep up the great work.

[RichardU]

I just noticed maybe a hundred of these legacy datasets. What are they? How can I get rid of them, or prevent them from happening? Or should I not?

 

[Niklas]

5 minutes ago, RichardU said:

I just noticed maybe a hundred of these legacy datasets. What are they? How can I get rid of them, or prevent them from happening? Or should I not?

 

That's docker storage (set to folder stored on zfs).

Look back like one page. 

This needs an FAQ. 😉 

Edited May 28 by Niklas

[Ademar]

Originally posted this as a separate thread, posting here as it's more relevant to the plugin.
https://forums.unraid.net/topic/165486-no-permission-to-write-to-dataset/

I created a new share called "files", which is on my new ZFS pool. That created a dataset on the pool with the same name. With the ZFS Master plugin, I created a new dataset files/something. I can access both folders over SMB from Windows, but I don't have permission to write to files/something, only to "files". Is there a fix for this? or should I just avoid nesting datasets like this, and create "something" as it's own separate share instead?

[Iker]

Hi, it's a matter of permissions, change them to 777 on the roof folder and add the -R flag for recursive operation and all should be good to go.

[sjerisman]

@Iker - Not sure if you've tried out the v7.0.0 beta yet?  One (extremely minor) issue I noticed with this plugin is the usage information is offset vertically now.  Probably some style(s) changed with v7.

 

[Iker]

I tried the 6.13 beta that got changed to 7.0 beta; there were some changes in styles that are not yet definitive, I'm waiting for another RC version or beta to update all the styles as other changes may be introduced.

[sjerisman]

10 hours ago, Iker said:

I tried the 6.13 beta that got changed to 7.0 beta; there were some changes in styles that are not yet definitive, I'm waiting for another RC version or beta to update all the styles as other changes may be introduced.

Ok makes sense.  beta2 did come out yesterday and has the exact same style issues.  But again, no rush, just wanted to make sure you knew.

[zer0ish]

Hello,

 

I'm wondering why sometimes new docker container folders show in the Main tab for the ZFS appdata, but sometime they don't.

I often have to use the Spaceinvader user script to convert the new folders to a dataset, then they show up in the list on the Main tab where I can then do a snapshot.

 

Is there a way to show these new docker folders so I can change them to datasets from the GUI to then make the snapshots?

 

Is there a way to update Directory listings from the Action menu?

 

Thanks.

[Iker]

Hi, you have to enable "Destructive Mode" on the settings page; then, you can enable the directory listing functionality for any dataset and convert the child folders to a dataset with a single click.

[zer0ish]

On 7/19/2024 at 1:03 AM, Iker said:

Hi, you have to enable "Destructive Mode" on the settings page; then, you can enable the directory listing functionality for any dataset and convert the child folders to a dataset with a single click.

 

Hello,

Yes I do have "Destructive Mode" on. When I click on the appdata "ACTIONS" menu and click "add directory listings" I get the bellow screenshot.

 

If I do a recursive snapshot, it doesn't list the other new docker folders. So I can't turn them into datasets unless I use the user scripts by Spaceinverder.

 

I can click on another dataset ACTIONS menu and when I take a snapshot recursively, and then add directory listings button, the folders inside show up. 

 

It seems to just not want to detect new folders inside the main appdata when I add new docker containers.

 

Maybe it has something to do with the Directory listings in the ZFS Master settings?

I never added those lines to the Directory Listing section. I assume they were added when I ran the "Add directory listings" button.

[Iker]

Thanks for the detailed report, let me try to replicate the issue so I can push an update with the fix.

[capduncan]

Hi sorry if this is not the correct place to post. It is a issue i have with zfs master

 

I did all this useing zfs-master

 

I setup a new data set with in my main dataset data\

data \marks-stuff

When i went over the set up page i selected encrypt drive.

 

I am sure i left the box empty but i did set encryption yes,

 

I carried on setting it up then clicked apply . I then moves over 1tb of files data to the new dataset

 

data\marks-stuff

 

Upon reboot its locked and i cant get into it. I have tried the main data passphrase but no luck.

 

Is it possible the dataset data\marks-stuff

 

inherited the key or paraphrase form the root dataset. \data

 

any ideas welcome

 

Thank you

 

 

 

 

 

[Iker]

Hi @capduncan, indeed, the child dataset inherits the encryption key; a side effect is that when you unlock the father, you also unlock the child dataset and vice versa.

[capduncan]

thanks for the reply

 

when i reboot i enter the key for \data the main data set.

but this child dataset remains locked.

 

\data\Marks-Stuff

 

the other datasets all open up OK with their passphrase

 

I will reboot tomorrow and see if i can get it to unlock.

 

Thanks for the Tips

[daTroll]

In the heading band, I see the following:

   Last refresh at 8/30/2024, 20:32:22

This date format can be confusing to many of us. Would it be possible for the ZFS Master plugin to use the Unraid Display settings for the date & time format, please? 

[Iker]

That's a great idea, and I'll add it to my ever-growing set of features for the next major release. Please bear with me for a little while; I'm finishing other things, but I plan to implement all the pending fixes and new features by the end of the year, so stay tuned!

[Entxawp]

If anyone ever want to clone a snapshot from one pool to another pool here is how you can do so on Unraid 6.12.x with ZFS Master.

 

zfs send pool1/dataset@snapshot_name | zfs receive pool2/dataset

 

It will look something like this in practice:

zfs send bxtyr/appdata@2024-09-02-190650 | zfs receive ark/test

 

If the dataset "test" already existed you'll have to add the -F

zfs send bxtyr/appdata@2024-09-02-190650 | zfs receive -F ark/test

 

 

You'll have to mount the dataset if it's not available
 

root@Example:~# cd /mnt/ark/test
root@Example:~# bash: cd: /mnt/ark/test: No such file or directory
root@Example:~# zfs mount ark/test
root@Example:~# cd /mnt/ark/test
root@GExample:/mnt/ark/test# 

 

 

Edited September 5 by Entxawp
Added Context

[balog]

Just wondering if it's possible to use the clone snapshot option in the snapshot Admin interface, to clone a snapshot to a different pool/dataset

 

I have one pool, cache, and within it a music dataset. I have manually created a snapshot.

On the other pool, disk3, I have a dataset set up, zfs_backups.

 

I've gone into the snapshot admin for cache/music, and clicked clone on the snapshot, and entered "disk3/zfs_backups" in the popup.

 

I get back "cache/music@2024-09-11-162017: Operation not permitted"

 

Cheers,

 

Brock

Edited September 11 by balog

[Iker]

You can only create a clone in the same pool as the parent snapshot because a clone initially shares all its disk space with the original snapshot; the previous message from @Entxawp has the instructions for sending a snapshot to another pool; it can be within the same system or even remote.

Edited September 11 by Iker

[Dudi]

The plugin no longer offers me the function to convert a folder to a dataset when I hit the "actions" button. The only option I got is renaming. What could that be?

 

I have recently changed my array pool from zfs to xfs but kept the cache pool (where I cannot convert anymore) at zfs

Edited Thursday at 05:29 AM by Dudi

[daTroll]

On 9/18/2024 at 5:44 PM, Dudi said:

The plugin no longer offers me the function to convert a folder to a dataset when I hit the "actions" button. The only option I got is renaming. What could that be?

 

I have recently changed my array pool from zfs to xfs but kept the cache pool (where I cannot convert anymore) at zfs

If I understand this correctly, you would like to convert a folder on your cache drive to a dataset, but all of the drives that have the shares which are being cached on your cache drive are formatted using XFS. Do you really want to convert a cached folder to a dataset?

Edited Friday at 10:26 AM by daTroll
Correction & amplification

[Iker]

On 9/18/2024 at 11:44 AM, Dudi said:

The plugin no longer offers me the function to convert a folder to a dataset when I hit the "actions" button. The only option I got is renaming. What could that be?

You need to enable "Destructive Mode" on the settings.

[Dudi]

8 hours ago, daTroll said:

If I understand this correctly, you would like to convert a folder on your cache drive to a dataset, but all of the drives that have the shares which are being cached on your cache drive are formatted using XFS. Do you really want to convert a cached folder to a dataset?

Actually it is a subfolder of appdata. I have appdata only on cache drive with daily backups of it to the array. the cache is zfs in order to make snapshots of my appdata. I created a subfolder in appdata and could not convert it. But now it seems to work again. To be honest, I am not sure if I realy forgot to re-anable destructive mode after I have re-installed zfs master.