[PLUGIN] ZFS Master

[erfanghasemy]

For the last few months I've had an imported Truenas pool working perfectly. Two days ago I had to reboot my server for the first time and now there's this weird issue with my pool. My docker containers can only see a few folders in the pool, even though none of the configuration has changed. Same thing with my SMB setup. I can only access certain folders through those shares.

Any insights as to what to try?

[Laov]

15 hours ago, erfanghasemy said:

For the last few months I've had an imported Truenas pool working perfectly. Two days ago I had to reboot my server for the first time and now there's this weird issue with my pool. My docker containers can only see a few folders in the pool, even though none of the configuration has changed. Same thing with my SMB setup. I can only access certain folders through those shares.

Any insights as to what to try?

I doubt this is related to ZFS master plugin so you might want to make a general support thread for this.

 

On a side note from personal experience I've had issues with UNRAID resetting user passwords and permissions. Though this happened when rebuilding unraid USB from a backup. It is quite simple to reset user password and permissions so you may as well try. 

Edited November 5, 2023 by Laov

[sfef]

I can not get the exclusions to work. Reading through these posts I am pretty confident that I should set it to /docker-sys/.*

I have tried variations on that and also including the cache drive name /nvme_1_nvme/ 

When I first installed zfs master I had a parent folder above my docker-sys folder. It was /docker/docker-sys/ when this was my folder scheme I remember them being excluded. However I have now removed that parent folder and the docker-sys folder is its own dataset.

I have dozens of these docker images listed.

 

[Revan335]

49 minutes ago, sfef said:

I can not get the exclusions to work. Reading through these posts I am pretty confident that I should set it to /docker-sys/.*

I have tried variations on that and also including the cache drive name /nvme_1_nvme/ 

When I first installed zfs master I had a parent folder above my docker-sys folder. It was /docker/docker-sys/ when this was my folder scheme I remember them being excluded. However I have now removed that parent folder and the docker-sys folder is its own dataset.

I have dozens of these docker images listed.

 

I must delete my Folder but he was not created from ZFS self and a old btrfs folder before ZFS:

https://docs.unraid.net/unraid-os/release-notes/6.12.0/#zfs-pools

https://docs.unraid.net/unraid-os/release-notes/6.12.0/#docker

 

Than my exclude for the /mnt/cache/docker/ Path working.

 

With Exclusive Share.

Edited November 7, 2023 by Revan335

[KyleK29]

3 hours ago, sfef said:

I can not get the exclusions to work. Reading through these posts I am pretty confident that I should set it to /docker-sys/.*

I have tried variations on that and also including the cache drive name /nvme_1_nvme/ 

When I first installed zfs master I had a parent folder above my docker-sys folder. It was /docker/docker-sys/ when this was my folder scheme I remember them being excluded. However I have now removed that parent folder and the docker-sys folder is its own dataset.

I have dozens of these docker images listed.

 

 

I had to tinker with mine to get it to work as well.

 

My setup, Docker-Directory is set to: 

/mnt/zfspool/.docker_system/docker_directory/

 

ZFS Master Exclusion is set to:

.docker_system

 

 

 

[Iker]

Please keep in mind, that ZFS master doesn't use Regex, but Lua patterns for matching the exclusion folder, that comes with some downsides, in your particular case @sfef, at first sight your pattern seems fine, but it actually contains a reserved symbol "-", combined with "r" it means a completely different thing; you can check your patterns here:

 

https://gitspartv.github.io/lua-patterns/

 

Long story short, this should do the trick "/docker%-sys/.*"

 

Additional doc on Lua patterns:

 

https://www.fhug.org.uk/kb/kb-article/understanding-lua-patterns/

 

 

Edited November 7, 2023 by Iker

[sfef]

Thanks for the help, @Iker that was what I needed was the % in there.

This works:  /docker%-sys/.* 

Edited November 7, 2023 by sfef
After I posted orig, my page refreshed with the response from Iker which has the same answer

[dlandon]

You should add the code to show the % used and free consistent with the display type selected in 'Used / Free Columns' and the color theme:

Your % Used and Free is gray in my setup:

This is the UD display for this device:

When doing GUI work, we want to keep the UI consistent.  Feel free to PM me and I can show you the code to accomplish this.

[wacko37]

@Iker Firstly as always I would like to thankyou for this great plugin and for all that you do for this great community.

I have encountered an issue with a ZFS disk mounted via UD not showing up in ZFS Master.

After a discussion with the UD developer @dlandon in the UD thread, it appears this is to do with ZFS compatibility in the upcoming Unraid 6.13 release, UD now accommodates for 6.13 when formatting a disk to ZFS rendering it unmountable in ZFS Master...... see below

 

 

(p.s I hope i have relayed this information correctly, I apologise in advance if this isn't the case and would be due to a lack of understanding on my part)

 

[wacko37]

On 11/13/2023 at 9:45 AM, wacko37 said:

@Iker Firstly as always I would like to thankyou for this great plugin and for all that you do for this great community.

I have encountered an issue with a ZFS disk mounted via UD not showing up in ZFS Master.

After a discussion with the UD developer @dlandon in the UD thread, it appears this is to do with ZFS compatibility in the upcoming Unraid 6.13 release, UD now accommodates for 6.13 when formatting a disk to ZFS rendering it unmountable in ZFS Master...... see below

 

 

(p.s I hope i have relayed this information correctly, I apologise in advance if this isn't the case and would be due to a lack of understanding on my part)

 

If anyone runs into this issue,  1 must run this command in Unraid console to make the zpool mountable in ZFS Master

 

UNRAID:~# zpool upgrade (zpool name)

 

The command will upgrade (downgrade really) the ZFS disk to the 6.12 ZFS features.

Edited November 14, 2023 by wacko37
layout - extra info

[Michel Amberg]

I am sure this is not a problem with this plugin I just wanted some clearification from someone here if they had ran into the same problem. Basicaly I used @SpaceInvaderOne script to create datasets of all my appdata folders. Today I did a big messup in plex and I basically deleted a library and it wanted to rescan the whole thing which takes foreever for me. So instead I restored my snapshot from earlier today without thinking. This caused all my shares on my unraid server to disappear and not matter what I did I could not get them back with the server up. So I rebooted it and they all got readded. I am guessing since ZFS messes blocklevel this freaked fuse up and caused all shares to dissapear. Is there some guideline on how to restore snapshots without this happening? or is this the only workaround?

[Iker]

On 11/12/2023 at 6:45 PM, wacko37 said:

I have encountered an issue with a ZFS disk mounted via UD not showing up in ZFS Master.

After a discussion with the UD developer @dlandon in the UD thread, it appears this is to do with ZFS compatibility in the upcoming Unraid 6.13 release, UD now accommodates for 6.13 when formatting a disk to ZFS rendering it unmountable in ZFS Master...... see below

Thanks for the info, currently I don't have access to the 6.13 beta version, as soon as it is released to the general public, I will try to reproduce your issue and check why the plugin is not picking up the pools.

 

7 hours ago, Michel Amberg said:

Is there some guideline on how to restore snapshots without this happening? or is this the only workaround?

 

Seems that is a something that should go on the General Support thread, I'm not sure that I follow exactly what is going on with your datasets.

Edited November 15, 2023 by Iker

[dlandon]

46 minutes ago, Iker said:

Thanks for the info, currently I don't have access to the 6.13 beta version, as soon as it is released to the general public, I will try to reproduce your issue and check why the plugin is not picking up the pools.

Just format a disk zfs with the latest version of UD in 6.12 and the issue will show.

[andyd]

Thanks for the plugin!

 

I set this up today - it picks up one of the pool drives but I have two formatted with zfs. Any reason the one would be ignored

[Iker]

8 minutes ago, andyd said:

Thanks for the plugin!

 

I set this up today - it picks up one of the pool drives but I have two formatted with zfs. Any reason the one would be ignored

 

Can you please share the result of the following command (As text):

 

zpool list -v

 

Best

[andyd]

22 hours ago, Iker said:

 

Can you please share the result of the following command (As text):

 

zpool list -v

 

Best

 

Looks like it's there. "apps" which is not showing up under the plugin

 

[andyd]

Could it be how I moved over the data? This drive has been sitting around for a while. Yesterday I decided to set up zfs snapshotting. 
 

I moved a folder I had another drive (nextcloud) to this drive using unbalance. Not sure if that is the reason?

 

is there some way data should be moved to the drive? Do I have to manually create datasets for it to appear in zfs master 

[Iker]

Please post it as text, that would help me to check if there are any parsing errors in the plugin and implement the neccesary fixes for the upcoming version.

[andyd]

Ah sorry at this point I got beyond that issue. For some reason, reformatting the drive to zfs again had it show up in the plugin. I ended up reformatting because I was getting weird errors that there was insufficient space. I'm all good now

[Masterwishx]

On 11/12/2023 at 3:08 PM, dlandon said:

You should add the code to show the % used and free consistent with the display type selected in 'Used / Free Columns' and the color theme:

 

 

@Iker Thanks again for the Plugin

if you will have time for it ,will be cool to see unraid setup like Used/Free in plugin ...

in my case : 

 

 

[Iker]

Sure thing; I'm planning to release a new version the next week, with all the bugs and little improvements implemented; stay tuned.

[boyish-stair3681]

I do not use Unraid but TrueNAS with a ZFS mirror. But I can also imagine using Unraid. But ZFS must be stable and all relevant features including native ZFS encyption must be supported. Is OpenZFS 2.2 already supported? Where are there problems? What can we expect in the future?

[Iker]

This thread over here seems like a better place for your questions: https://forums.unraid.net/forum/37-pre-sales-support/

[boyish-stair3681]

16 hours ago, Iker said:

This thread over here seems like a better place for your questions: https://forums.unraid.net/forum/37-pre-sales-support/

OK.

 

Due to problems in OpenZFS 2.2, it is good that Unraid (6.12.5-rc1) still uses the older ZFS version (2.1.13-1).

But Open 2.1.13-1 is also still affected by a problem.

"So are we tracking three different disk corruption issues?

With strict hole reporting (i.e. zfs_dmu_offset_next_sync set to 1). This has been a silent disk corruption issue since 2.1.4 and a fix is not in any current release.

With block cloning which results in silent disk corruption. This is only an issue in 2.2.0 and has been resolved in 2.2.1 by disabling it, but no long term fix exists yet.

When using LUKS which results in write errors with ZFS on 2.2.1. A fix is not in any current release.

So the safest thing to do for now to avoid further data loss is just use 2.1.13 and set zfs_dmu_offset_next_sync=0? Is that the blessed thing to do by the OpenZFS team until an official release is made for 2.1.14 and 2.2.2?"

https://github.com/openzfs/zfs/issues/15526#issuecomment-1825113314

 

So if zfs_dmu_offset_next_sync is set to 1, then it is best to set the value to 0 until the problem has been solved by an update: 

echo 0 >> /sys/module/zfs/parameters/zfs_dmu_offset_next_sync

Edited November 25, 2023 by boyish-stair3681

[bugsysiegals]

I've several folders on my ZFS cache which I moved from the array using MC instead of the Mover so the folder and all sub-directories/files are not datasets.  I believe I can move these back to the array and then move them again with mover to make them datasets but is there an easier way with some command/script?