ZFS plugin for unRAID


steini84

Recommended Posts

From what I've read, this xattr=sa setting, extensively reduces the array from doing writes to files on disk, instead writing to inodes.  If this is always the way it has worked (and I believe it is), changing this is only like to be a workaround of some other thing, impacted by lots of writes such as when creating a new VM image.  I had thought this was probably kernel related, but it doesn't happen on 6.8.3 and 6.9 if I recall.  Either way, I'm not convinced this is the end of it. :(

Link to comment

Upgrade to beta 4 has completely borked my system.  That's my current technical assessment anyway.  It took me ages to figure that out too.

 

All sorts of errors about missing shared libraries, and also doing odd things like removing the docker and virtual machine menus from the Unraid GUI, also noticed it's telling me docker can't start because VT-d is missing - well it's a dual Xeon I don't think so.

 

So far I don't recommend any of these betas and regret using them at all.

 

I've tried to downgrade to beta 3 using the above method which is saying ZFS modules are not loaded so something is not right.

 

This is all happening at a time when I have no time.  But I need it to run.  If I had the time I would reformat and downgrade to stable ZFS.

 

ZFS beta HIGHLY NOT RECOMMENDED on UNRAID is my current opinion.

 

The funny thing is that ZFS itself is running in the background - even when the menus are missing docker and vm's are running too (or were at least).  It's also causing other issues like randomly not being able to boot from USB stick.

 

My guess is that like the other assumption it's more of an issue to do with the implementation of ZFS as per how it's attached to unraid.  But I don't know.

 

Even the nerd pack is throwing up errors with this installed.  It's possible it's something else I haven't thought of, but when I disable ZFS everything else starts to run normally again.

 

Painful!

Link to comment

@steini84 Are you sure something in the new build of ZFS beta4 isn't overwriting parts of the system it shouldn't?  I've even got it complaining I've got less that 2G of memory in the GUI, yet in the console it's saying I have 96.  Check out my main screen. I've done a memory test too just in case.  Check out how docker and VM tabs are missing, but actually they're running.  Weird scripting / coding language showing up in the GUI.

338324810_ScreenShot2020-10-23at3_00_30PM.thumb.png.341e9f42eaf153c01a2f0f1100f25ffb.png1007523295_ScreenShot2020-10-23at3_00_40PM.thumb.png.7b4bededd0c4be30e12c61b57e3cbba7.png

Link to comment

I use /mnt/x y z

 

I've just reformatted the key and this time not copied the config.  It was all working fine until I added ZFS.  I note that only the beta of ZFS is now being offered not stable, which I think is a huge mistake given it's the stable 6.8.3 I have installed.  @steini84 perhaps you could implement something so that stable / next can be chosen for ZFS.

 

Anyway, I'm back to believing it's a ZFS issue.  I don't know why it is.  I'm going to try the community kernel and see if that sheds any light.

 

Community Kernel (Unraid Kernel Helper) same result (using my normal config though, but I did prove it did this with none of my config also). This is a nightmare.  The only thing I can say for sure is it only appears when ZFS is installed.  But earlier known good versions of ZFS also exhibit the behaviour.

Edited by Marshalleq
Link to comment
4 hours ago, Marshalleq said:

I use /mnt/x y z

 

I've just reformatted the key and this time not copied the config.  It was all working fine until I added ZFS.  I note that only the beta of ZFS is now being offered not stable, which I think is a huge mistake given it's the stable 6.8.3 I have installed.  @steini84 perhaps you could implement something so that stable / next can be chosen for ZFS.

 

Anyway, I'm back to believing it's a ZFS issue.  I don't know why it is.  I'm going to try the community kernel and see if that sheds any light.

 

Community Kernel (Unraid Kernel Helper) same result (using my normal config though, but I did prove it did this with none of my config also). This is a nightmare.  The only thing I can say for sure is it only appears when ZFS is installed.  But earlier known good versions of ZFS also exhibit the behaviour.

Good idea, I completely forgot to separate the stable builds from the RC version. I upgraded the plugin so that by default you install the cached version if available. If not the plugin check for a stable version by default.

However you can touch a file on disk and enable unstable builds (like the 2.0.0 RC series) 

 

#Enable unstable builds
touch /boot/config/plugins/unRAID6-ZFS/USE_UNSTABLE_BUILDS
rm /boot/config/plugins/unRAID6-ZFS/packages/*
#Then reboot

#Disable unstable builds
rm /boot/config/plugins/unRAID6-ZFS/USE_UNSTABLE_BUILDS
rm /boot/config/plugins/unRAID6-ZFS/packages/*
#Then reboot

The builds are pretty much vanilla ZFS and you can check the build scripts on Github:

https://github.com/Steini1984/unRAID6-ZFS/blob/master/build.sh (to build latest ZFS)

https://github.com/Steini1984/unRAID6-ZFS/blob/master/build_github.sh (to build custom versions like RC)

 

I feel your pain and it´s incredibly frustrating when the server has issues! I remember having incredible problems with my server a few years back. Random disks failing, processes crashing and all around pain. Took me probably a week of debugging and the solution was a new PSU. Hope you can fix your problems and I really hope that at least ZFS in not the culprit

  • Like 1
Link to comment

Thanks - I actually installed a new version of 6.8.3 on my USB using the Mac install tool, which in the process wipes the usb stick.  I copied over no config and installed ZFS from the App Store and somehow that ended up with the beta4 version of zfs.  So I don't know.  Anyway a bit of a learning for me!  I have to wonder what is on that disk that's so problematic - I only created it recently.  I have a suspicion it may have a duplicate mount point or something.

Link to comment
  • 2 weeks later...

Using zfs-2.0.0-rc4-unRAID-6.9.0-beta30.x86_64.tgz with unRaid 6.9-beta30, the /dev/zvol/<zvols-by-name> are missing (even the /dev/zvol/ folder is missing) while they appear as /dev/zd* and are fully usable, but it is exhausting to check all the /dev/zd* with fdisk to find out wich one belongs to the corresponding zvol (and for volumes with same size and disk format this problem is only solvable by try and error).

 

Is this only a problem for me or a problem with the package?

 

Thanks and regards.

 

I have written a lil bash script that creats the links, you can use/modify it if you need this too:

#!/bin/bash
mkdir /dev/zvol/
for i in $(ls /dev/zd*)
do
        TMP=$(/lib/udev/zvol_id $i)
        TMP0=$(cut -d '/' -f1 <<<$TMP)
        TMP1=$(cut -d '/' -f2 <<<$TMP)
        mkdir /dev/zvol/$TMP0
        ln -s $i /dev/zvol/$TMP0/$TMP1
done

 

Edited by efschu
  • Like 1
Link to comment
27 minutes ago, segator said:

question, how do you guys monitor the disk smart when using ZFS?
unassgined devices plugin is not monitoring disks that are not mounted right?

 

Not using ZFS, but this is a nice solution as it collects smart data so you can get historic data.

Works for all drive devices, not just array/cache

 

Link to comment
question, how do you guys monitor the disk smart when using ZFS?

unassgined devices plugin is not monitoring disks that are not mounted right?

 

I use check_mk for overall monitoring of my servers and everything that has a network. It seems that the unassigned drives are being monitored by unraid :b9c441f6e8c8b8da80c32544c880fce7.jpg&key=2ef2c864ae165ac407b212a8775ce26d642a357267c109c8054e4d0424645afe

 

 

Sent from my iPhone using Tapatalk

Link to comment

oh.. sorry what a newbie I am...
yes says smart ok, so I supose if in the future some issues are detected I will be notified as any disk in the array, right¿?

BTW someone tried ZFS over USB disks? is that a crazy idea? i am out of space on my rack server :( and i'm considering buying a 4slots usb enclosure. any other idea if not?

Link to comment
On 11/8/2020 at 5:24 AM, segator said:

Well the idea is to have it there 24/7 running to expand my array, some will be disks on unraid array (so i supose no problem as disks works independently, but i also plan to create another ZFS raid there(3disks).

 

 

I've recently been wondering about the speed of rebuild's / checks on USB.  I have a 5 bay USB 3 box that came from a QNAP.  It's seen in unraid.  But I think it will be too slow.

Link to comment
8 hours ago, steini84 said:

Building right now smile.png&key=d7bc0bbbfa2056a2a63af437c1f4a1188d2d7eb0deb5aa11e8f34aa88765e668

 

... Running into some build issues. Hopefully I'll have time to figure it out over the weekend.

 

Sent from my iPhone using Tapatalk

Did you read the beta 35 notes?  It seems that there's some new methods for deploying drivers and modules.  They actually include in tree AMD GPU too.  At first I was thinking this was going to save the day - then I realised we still have to compile it against the new kernel right?  I've found these particular new Unraid beta's to be the worst betas I've ever used for some reason, so I think I'll stay away until they at least put the new linux kernel in.

Link to comment
Did you read the beta 35 notes?  It seems that there's some new methods for deploying drivers and modules.  They actually include in tree AMD GPU too.  At first I was thinking this was going to save the day - then I realised we still have to compile it against the new kernel right?  I've found these particular new Unraid beta's to be the worst betas I've ever used for some reason, so I think I'll stay away until they at least put the new linux kernel in.

Yeah I just put it all together and pushing it out now


Sent from my iPhone using Tapatalk
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.