ZFS plugin for unRAID


steini84

Recommended Posts

7 hours ago, ich777 said:

This is really strange, since already said above, nothing was changed that would prevent this, the Kernel module and the applications completely the same, there where only a few lines added and ZED runs now in background but that should not harm the functionality from @Iker's plugin.

 

6 hours ago, SimonF said:

Works ok my test system and shows datasets and correct snapshot count.

That could be a coincidence that the problem is caused by other factors?

How do I check the logs of the ZFS Master plugin?

Link to comment
7 hours ago, diannao said:

 

That could be a coincidence that the problem is caused by other factors?

How do I check the logs of the ZFS Master plugin?

 

Sorry for the delay, @diannao please post an image with the result of the commands "zpool status -v" and "zpool list -v" ; this is the first step that the plugin performs to identify all the pools and then, list the datasets that belong to the pool; I will deploy a clone of the configuration that you post, so I can debug and update the plugin according to the results.

 

 

Edited by Iker
Link to comment
8 hours ago, Iker said:

 

Sorry for the delay, @diannao please post an image with the result of the commands "zpool status -v" and "zpool list -v" ; this is the first step that the plugin performs to identify all the pools and then, list the datasets that belong to the pool; I will deploy a clone of the configuration that you post, so I can debug and update the plugin according to the results.

 

 

 

here is

image.thumb.png.f608f25664122285e464fc06cceedf2e.png

Earlier in the day, the info of dataset in webui was working fine, and I didn't change my zpool configuration during this this time.

 

Do you think the bug might be related to the name of the snapshot?

image.png.42fed2a4339259dc02b270cbf9caa18f.png

Edited by diannao
Link to comment

I'm completely sure that is a parse error, however with the specifics I could modify the regular expressions to catch this case and the plugin to work properly; thanks for the info I get back to you when the update is live.

Link to comment
6 hours ago, Iker said:

I'm completely sure that is a parse error, however with the specifics I could modify the regular expressions to catch this case and the plugin to work properly; thanks for the info I get back to you when the update is live.


I am seeing the exact same issue that diannao is.

image.thumb.png.065935a2d2e4d8af86a5f81ec91acda0.png

Link to comment
14 hours ago, Iker said:

@JJJonesJr33 @diannao The fixed version is already live, as I'm planning a new version with additional features, the fix was applied to the current version; what that means is that you have to uninstall and install the plugin again in order to get the version with the fix.

 

Thanks for your help!.

I reinstalled version 2021.11.09a. But the problem is still not solved, am I getting the wrong version?

Link to comment

@ich777 That was my first intention, but my local branches are completely mess up so, it was quicker to just updated the current.

 

@subivoodoo That's one of the features that I'm working to be release in a couple of versions.

 

@diannao Now, I'm not so sure about your error, mainly because for the fix, I replicated your config and the plugin was working just fine; and on the other hand, you posted that at least the dataset info was working fine a couple of days back; I'm going to send you a PM with a set of commands to be executed, so I can debug in deep what is wrong with the plugin in your system.

 

 

Link to comment
12 minutes ago, toastyuce said:

Would it be possible to include zpool_influxdb command? Or is something wrong on my end?

I'm using Unraid 6.9.2, ZFS 2.1.2-1

BR

No, nothing wrong, you can find it there:

/usr/libexec/zfs/zpool_influxdb

 

  • Like 1
Link to comment
On 2/2/2022 at 5:56 PM, ich777 said:

... and ZED runs now in background ...

 

I've created my own zed.rc to receive notifications through the notify script and I am starting zed via the go file, will this change interfere with that? I assume i dont have to start zed via the go file anymore, right?

Link to comment
11 minutes ago, 0xcd0e said:

I assume i dont have to start zed via the go file anymore, right?

No, you don't have to start it though the go file anymore.

 

13 minutes ago, 0xcd0e said:

I've created my own zed.rc to receive notifications through the notify script and I am starting zed via the go file, will this change interfere with that?

Depending on what notifications you've set or that you want you don't have to create your own "scrub_finish-notify.sh" anymore.

 

I've modified it to notify you on every finished scrub if the pool is healthy or not, of course you could also add other notifications if you want to.

  • Thanks 1
Link to comment

Thank you to all the people working on ZFS in Unraid. I am currently planning to switch my latest Unraid server which i am currently building to use zfs.

 

One question is how to enable l2arc prefetch to enable cache for sequential heavy workloads aka read "streaming". I read this can be done by using l2arc_noprefetch=1 but i am not sure if that goes in the go file or if it is set like a pool option? It currently does not work as zpool set or zfs set. Is this not included in openzfs? This was mentioned in a presentation by Ryan McKenzie that is worth a read, since it is now considered best practice for those with fast l2arc nvme drives (aka two 980 pro 2tb nvme on pcie gen4).

Thank you all for your hard work.

Edited by platt85
Link to comment
On 2/9/2022 at 5:47 PM, ich777 said:

Depending on what notifications you've set or that you want you don't have to create your own "scrub_finish-notify.sh" anymore.

 

I've modified it to notify you on every finished scrub if the pool is healthy or not, of course you could also add other notifications if you want to.

I havent really enabled any specific events, I basically just enabled verbose notifications and set ZED_EMAIL_PROG="/usr/local/emhttp/plugins/dynamix/scripts/notify" so i get notified about all events

Link to comment
16 hours ago, 0xcd0e said:

I havent really enabled any specific events, I basically just enabled verbose notifications and set ZED_EMAIL_PROG="/usr/local/emhttp/plugins/dynamix/scripts/notify" so i get notified about all events

You will get a notification on every finished scrub like in the screenshot above regardless if you've set it on a schedule or if it is triggered by the plugin.

 

If that's enough for you, you can remove all custom things that you've done to make it work. :)

  • Like 1
Link to comment

Could we get a script to send a notification on the ZFS event for state change? Specifically I have been testing this by creating pools and then removing a hard drive to force a degraded state and there are no notification sent. 

 

Don't get me wrong, notifications on scrub finishing are good, but if the pool goes into a degraded state because a drive failed, it should notify you pretty consistently since you're at risk for data loss. I haven't been able to trigger any notifications on my simulations of getting a test pool into a degraded state.

Link to comment

I use check_mk for monitoring and that works perfectly for everything from low space to high load and even failed pools. It takes some time to set up but is really good!

A simple bash script can be rigged to check zpool status -x

I can set one up when I'm in front of a computer


Sent from my iPhone using Tapatalk

Link to comment
9 minutes ago, steini84 said:

I use check_mk for monitoring and that works perfectly for everything from low space to high load and even failed pools. It takes some time to set up but is really good!

A simple bash script can be rigged to check zpool status -x

I can set one up when I'm in front of a computer


Sent from my iPhone using Tapatalk

 

Everything is pretty much already there, seems like it would make the most sense to hook into ZED. The plugin includes the notification for scrubs, I'm just suggesting we add notifications for state change as well and make it part of the package. 

Link to comment
4 hours ago, Stokkes said:

ould we get a script to send a notification on the ZFS event for state change?

Sure thing, but this needs to be tested and a custom script needs to be made.

The last thing that I want is that you get two or more messages when one operation finished for example...

 

I only built the notification through ZED for a finished scrub into the plugin because it is actually trivial to do a scrub after a unclean shutdown and it is much nicer that you get a notification when it's finished.

 

If now users want more notifications/features maybe a WebUI for the ZFS plugin is necessary to turn on/off notifications/features, but this means serious work and I'm also not to sure if this will actually make sense because ZFS is sometime in the near future built into unRAID itself.

 

Also keep in mind you can add your own script(s) for notifications you want to ZED, I also don't know what makes more sense, what's your opinion about that @steini84?

Link to comment
Sure thing, but this needs to be tested and a custom script needs to be made.
The last thing that I want is that you get two or more messages when one operation finished for example...
 
I only built the notification through ZED for a finished scrub into the plugin because it is actually trivial to do a scrub after a unclean shutdown and it is much nicer that you get a notification when it's finished.
 
If now users want more notifications/features maybe a WebUI for the ZFS plugin is necessary to turn on/off notifications/features, but this means serious work and I'm also not to sure if this will actually make sense because ZFS is sometime in the near future built into unRAID itself.
 
Also keep in mind you can add your own script(s) for notifications you want to ZED, I also don't know what makes more sense, what's your opinion about that @steini84?

What I think would make most sense is to have the zfs plug-in as vanilla as possible and extra functionality to a companion plugin. Then the companion plug-in would continue to work when zfs becomes native


Sent from my iPhone using Tapatalk
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.