ZFS plugin for unRAID

[torch2k]

I'm a bit of a tinkerer and I'm interested in giving FreeNAS a go right now.

 

Is it reasonable for me to think that I could fire up my server with a FreeNAS boot disk and have it import my zfs pool I currently have on Unraid without any hitches?

 

What if I wanted to come back to Unraid later on? 

[steini84]

I'm a bit of a tinkerer and I'm interested in giving FreeNAS a go right now.
 
Is it reasonable for me to think that I could fire up my server with a FreeNAS boot disk and have it import my zfs pool I currently have on Unraid without any hitches?
 
What if I wanted to come back to Unraid later on? 

Yeah ZFS is really portable, but you have to be sure that you don't have feature flags turned on on the pool that is not supported on Freebsd/Freenas

See this link: https://openzfs.org/wiki/Feature_Flags


Sent from my iPhone using Tapatalk

[steini84]

Built for unRAID-6.9.0-beta24


Sent from my iPhone using Tapatalk

[steini84]

Hey, thanks for the report.  ZFS is very cool, but @steini84maybe we should update post1 with "Where not to use ZFS"
 
It would appear that using ZFS to mount USB devices is not a good use case, (or should only be done in cases where  you are aware that ZFS is not plug and play with USB).  For normal disk maintenance, unRaid leads us to believe that we can safely do this with the array stopped.  ZFS is a totally different animal. 
 
Would best practice be to put a  "zpool export" command into Squid's awesome user scripts plugin and set it to happen  on array stop, and "zpool import" on array start?  On first startup of array, you should not need this as ZFS will automatically do the zpool import.  It would seem that user scripts supports all this and could make ZFS behave like the unRaid array.  Would this make sense?
 

I want to provide a vanilla experience of ZFS on unRAID until it’s natively supported

I don’t want to add some some functions that might or might not benefit user use-cases.

If you want the pool to automatically import export it’s a great idea to add zpool export -a and zpool import -a on stopping/starting the array via the awesome user scripts plugin.

Considering this is a plug-in for advanced users I don’t think the target audience for this will have a problem adding these commands if preferred.

Now we just wait and see if native zfs support comes in the next betas

Ps I have built Sanoid/Syncoid for myself and was wondering if there was any demand for a plug-in?

[tr0910]

On 7/8/2020 at 4:12 PM, steini84 said:

Ps I have built Sanoid/Syncoid for myself and was wondering if there was any demand for a plug-in?

What is your use case for Sanoid that you cannot already do with above ZFS tools??  Yes, we are always interested in new shiny things..  Is there any issues with it?

Edited July 10, 2020 by tr0910

[steini84]

What is your use case for Sanoid??  Yes, we are always interested in new shiny things..  Is there any issues with it?

I had some problems with ZnapZend and remote replication, but Sanoid/Syncoid has been rock solid.

The plugin is pretty much complete, but I’m just wondering how much of the setup I should build into the plugin or leave to the user. I prefer flexibility and don’t want to limit the user, but I don’t want it to be difficult either.

I’ll grab some coffee this weekend and throw something together


Sent from my iPhone using Tapatalk

[tr0910]

1 hour ago, steini84 said:

I had some problems with ZnapZend and remote replication, but Sanoid/Syncoid has been rock solid.

Remote snapshot replication using SSH??  I wrote up a rsync ssh guide that has been very solid, but ZFS and snapshot replication was never part of that.

 

 

[Dtrain]

1 hour ago, steini84 said:

I had some problems with ZnapZend and remote replication, but Sanoid/Syncoid has been rock solid.

The plugin is pretty much complete, but I’m just wondering how much of the setup I should build into the plugin or leave to the user. I prefer flexibility and don’t want to limit the user, but I don’t want it to be difficult either.

I’ll grab some coffee this weekend and throw something together


Sent from my iPhone using Tapatalk

Steini thats awesome !

- Vielen Dank -

 

[steini84]

Bitte

 

The wife went out and the kids fell a sleep early so...

I probably have to make some rewrites to the "tutorial" but it will be fun to see if someone gets it up and running 

[testdasi]

Stupid question alert:

Does "errors: No known data errors" mean there is no error i.e. all is well?

Edited July 12, 2020 by testdasi

[steini84]

Stupid question alert:
Does "errors: No known data errors" mean there is no error i.e. all is well?

Yes exactly

But to be clear ZFS only finds errors when you try to read files so too be sure that your hard drives are not plotting against you (spoiler alert they are it’s good to scrub regularly

https://docs.oracle.com/cd/E23823_01/html/819-5461/gbbwa.html


Sent from my iPhone using Tapatalk

[testdasi]

Does anyone have spam of this warning on the syslog?

Jul 12 23:21:58 tower smbd[994]: [2020/07/12 23:21:58.445544,  0] ../../source3/lib/sysquotas.c:565(sys_get_quota)
Jul 12 23:21:58 tower smbd[994]:   sys_path_to_bdev() failed for path [.]!

[.] is replaced with various folders that is accessed.

 

After some more googling, I found this samba bug report. It specifically mentions btrfs but I reckon it's the same reason for the zfs.

Basically because the zfs mounts are automounted by zfs and thus not in /etc/fstab, smb fails to read the quota because there's nothing in /etc/fstab for it to read.

 

Does anyone know how to mount the zfs pools through /etc/fstab and NOT automount?

 

Source 1: https://bugzilla.samba.org/show_bug.cgi?id=10541

Source 2: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1735953

 

 

Edited July 13, 2020 by testdasi

[testdasi]

Figured it out. No need to mount through /etc/fstab.

 

What's missing are entries in /etc/mtab,  which are created if mounted from fstab.

So a few echo into /etc/mtab is the solution. Just need to do this at boot.

Each filesystem that is accessible by smb (even through symlinks) needs a line in mtab to stop the spurious warning spam.

echo "[pool]/[filesystem] /mnt/[pool]/[filesystem] zfs rw,default 0 0" >> /etc/mtab

 

[steini84]

Updated for 6.9.0-beta25

[hackersarchangel]

Quick question about the plugin. It is my understanding that unraid has a cache disk, and it asks for a Parity disk and such. I want to use ZFS as much as I can, and wasn't planning on having a cache drive.

Disk arrangement as planned: NVMe drive as a VM hosting drive for the 2-3 VM's I'll have, preferably using ZFS. Will accept doing the default UnRaid setup for this if necessary.
Two hard drives already in a mirrored pool setup, moving from FreeNAS to UnRaid using zpool import force
Have UnRaid store all data on the pool, not including VM files which will be backed up to the Zpool.

 

Does this sound doable? I'm going to build a server at some point in the future once I get all the parts together.

[testdasi]

On 7/14/2020 at 6:00 PM, hackersarchangel said:

Quick question about the plugin. It is my understanding that unraid has a cache disk, and it asks for a Parity disk and such. I want to use ZFS as much as I can, and wasn't planning on having a cache drive.

Disk arrangement as planned: NVMe drive as a VM hosting drive for the 2-3 VM's I'll have, preferably using ZFS. Will accept doing the default UnRaid setup for this if necessary.
Two hard drives already in a mirrored pool setup, moving from FreeNAS to UnRaid using zpool import force
Have UnRaid store all data on the pool, not including VM files which will be backed up to the Zpool.

 

Does this sound doable? I'm going to build a server at some point in the future once I get all the parts together.

Doable with a caveat. You need at least 1 device in the array. That's a requirement.

Some were even able to make do with just an additional USB stick (in addition to the Unraid OS stick).

 

I was able to move everything (docker, libvirt, appdata, vdisk, daily network driver) to my zfs pool of 2 NVMe - half in RAID-0 and half in RAID-1.

 

My array currently has a single 10TB spinner that is spun down most of the time. I only use it as emergency temp space (e.g. when upgrading my SSD backup btrfs raid-5 pool).

 

[hackersarchangel]

5 hours ago, testdasi said:

Doable with a caveat. You need at least 1 device in the array. That's a requirement.

Some were even able to make do with just an additional USB stick (in addition to the Unraid OS stick).

 

I was able to move everything (docker, libvirt, appdata, vdisk, daily network driver) to my zfs pool of 2 NVMe - half in RAID-0 and half in RAID-1.

 

My array currently has a single 10TB spinner that is spun down most of the time. I only use it as emergency temp space (e.g. when upgrading my SSD backup btrfs raid-5 pool).

 

Terrific. I have some old thumb drives I'm about to retire thanks to Grub2 File Manager so I can co-opt one or two of those to satisfy that, assuming it is OK with them.

 

I'll have to look into what it will take to do that kind of setup, if you have a general direction (RTFM pointers) I'd appreciate it. I'm a terminal junkie so that will work for me just as well as a GUI.

[etsjessey]

I tried to look as much as I could but, hadn't seen anyone with my problem just yet. Let me know if best to post on another topic..

I was able to create a pool, make a share using the SMB extras, and mount the share on another computer with the intended credentials....
However, when I try to drag a file to drop it in it has a circle with a line through it. I am sure it's some kind of permissions things but my username shows reads and write when I right click and press get info. I attached the smb extras I made hoping I am just missing something small. Thanks in advance!

 

Server: UnRaid Newest Stable 

Client: MacOS High Sierra

 

Edited July 16, 2020 by etsjessey

[hackersarchangel]

Your config looks alright, what version of SMB are you using? macOS won't use anything that's too old like SMBv1. It needs to be at least. SMBv2.

[etsjessey]

1 hour ago, hackersarchangel said:

Your config looks alright, what version of SMB are you using? macOS won't use anything that's too old like SMBv1. It needs to be at least. SMBv2.

It looks like the version negotiated is 3.02 when I looked in MacOS

[hackersarchangel]

20 minutes ago, etsjessey said:

It looks like the version negotiated is 3.02 when I looked in MacOS

Ok he other thing it can be are folder permissions on the server side, meaning you gave that user access in the config but the actual folder and files are still restricted. SSH into the server as a user that can gain root/sudo privileges, and run the following on the folders in question:

ls -la

This will tell you what permissions are on the folders. If they don't match the user OR a group the user is in then you would see what you are seeing without giving everyone permission to access.
 

** All commands below may need sudo or root access, proceed with caution! ***
If you will be the only user accessing these (or that specific user credential will be the only one) do the following:

chown -Rv username "/path/to/folder"

If other users will be using the share with different credentials, do this instead:

chmod -Rv 770 "/path/to/parentfolder"
chown -Rv username:sharedgroup "/path/to/folder"

Explanation of the commands (I'm not assuming you don't know what you are doing, this is for those that come after us and have all the questions!)

 

ls -la will show the permissions and the allowed level of permissions. It will show who the owner and group attached to the folder, and it will show what level of access each "ring" has.   rwx means Read Write eXecute, and is necessary to change anything in the folders. The permissions are in 3 sets, first being user, second group, third is everyone else.

 

chmod is CHange MODe and that allows you to change the access levels of each category explained above. 7 = RWX, 6 is RW. The eXecute bit (a numerical of 1) must be set on all directories or you will be unable to use those folders. Example: chmod 100 "/home/user" means you have set the execute bit on your home folder.

chown is CHange OWNer, -R is recursive, v is verbose. Owner is in the first half of the : group is second.

 

Hope this helps anyone else that comes along. If you have made it this far, welcome!

Edited July 16, 2020 by hackersarchangel
Added a note/warning

[etsjessey]

20 minutes ago, hackersarchangel said:

Ok he other thing it can be are folder permissions on the server side, meaning you gave that user access in the config but the actual folder and files are still restricted. SSH into the server as a user that can gain root/sudo privileges, and run the following on the folders in question:

ls -la

This will tell you what permissions are on the folders. If they don't match the user OR a group the user is in then you would see what you are seeing without giving everyone permission to access.
 

** All commands below may need sudo or root access, proceed with caution! ***
If you will be the only user accessing these (or that specific user credential will be the only one) do the following:

chown -Rv username "/path/to/folder"

If other users will be using the share with different credentials, do this instead:

chmod -Rv 770 "/path/to/parentfolder"
chown -Rv username:sharedgroup "/path/to/folder"

Explanation of the commands (I'm not assuming you don't know what you are doing, this is for those that come after us and have all the questions!)

 

ls -la will show the permissions and the allowed level of permissions. It will show who the owner and group attached to the folder, and it will show what level of access each "ring" has.   rwx means Read Write eXecute, and is necessary to change anything in the folders. The permissions are in 3 sets, first being user, second group, third is everyone else.

 

chmod is CHange MODe and that allows you to change the access levels of each category explained above. 7 = RWX, 6 is RW. The eXecute bit (a numerical of 1) must be set on all directories or you will be unable to use those folders. Example: chmod 100 "/home/user" means you have set the execute bit on your home folder.

chown is CHange OWNer, -R is recursive, v is verbose. Owner is in the first half of the : group is second.

 

Hope this helps anyone else that comes along. If you have made it this far, welcome!

 

 

That was exactly the problem! I was able to change everything to one user but, it takes away root access.
If I only have one other user besides root what would I do to accomplish changing permissions back to root but, also giving that one user access? Thank you for the detailed explanation by the way. For someone like me who didn't particularly know what each part meant but actually wants to learn, it's nice to finally know!

[hackersarchangel]

5 minutes ago, etsjessey said:

 

 

That was exactly the problem! I was able to change everything to one user but, it takes away root access.
If I only have one other user besides root what would I do to accomplish changing permissions back to root but, also giving that one user access? Thank you for the detailed explanation by the way. For someone like me who didn't particularly know what each part meant but actually wants to learn, it's nice to finally know!

That's interesting, root should always have access for root = God. You can't login to the share as root, that's a big security no no. *wags finger*

The changes I described are done to the folder via ssh, so I'm not sure I'm following what you did in that regard. As for the config on the share itself via SMB that looks fine.

Also, it's late for me so I may not be grokking this well. I'll check again tomorrow and see if I read your post differently LOL

[testdasi]

6 hours ago, hackersarchangel said:

I'll have to look into what it will take to do that kind of setup, if you have a general direction (RTFM pointers) I'd appreciate it. I'm a terminal junkie so that will work for me just as well as a GUI.

A few pointers:

• If you don't have any (fast) device as cache then managing shares will have to be done through the smb-extra config file. With a cache disk, you can create shares and then use symlinks to your zfs mounts / folders etc. and it would be transparent. Note that any zfs filesystem shared in smb will need a line in /etc/mtab to prevent spurious quota warning spam in syslog. Some echo into /etc/mtab at array start will do.

  echo "[pool]/[filesystem] /mnt/[pool]/[filesystem] zfs rw,default 0 0" >> /etc/mtab

   

• I had VM's using qcow2 image that had a rather strange issue. It would boot up originally fine but if shutdown and start back up, it would hang libvirt completely. So I now switched to using zvol instead (use qemu-img convert to convert from qcow2 to raw). Note that there's a bug that causes the zvol mounts to not show up which may or may not be solved by zvol export and then import after the initial import at boot. Without the zvol mounts, you have to use /dev/zd# which can change after 1st reboot (but won't change again without additional changes to the volumes) so I recommend doing a reboot after making changes to volume to solidify the zd# devices + you might want to use slightly different volume sizes between VM so it's easier to identify which is which.

 

 

 

 

 

[testdasi]

What do you guys use for a quick GUI-based check on pool information e.g. available space, snapshot sizes etc?

 

I can see stuff in the command line but wondering if there's something like the Main page of Unraid but for ZFS.