Unraid OS version 6.12.0-rc6 available

5 hours ago, JorgeB said:

AFAIK there were no changes that would explain that, try downgrading back to rc5 to confirm it's really update related.

What i have noticed  in rd5 it took 4 reboot to get it recognize me 1gb nic as 1gb  speed was always at 10mb

 

could it be something similar

 

Moving a share folder under rc6 leads to 'Share 'appdata' has been deleted'
Which seems like odd behavior, though I test this on a fresh install, and found the same behavior.

11 hours ago, TexasUnraid said:

Lastly, will BTRFS compression be able to be applied to existing shares/drives?

Yes, but it will only compress newly written (or re-written) data.

RC6 Note: "Use symlinks instead of bind-mounts for exclusive shares" creates issues inside my containers.

 

If I add a path variable and set the container path to for instance /net, and the host path to /mnt/user, exclusive shares are not accessible with symlinks from within the container, worked fine before with bind-mounts.

Edited May 19, 2023 by jbear

Hi,

 

I know it is a bit off topic, but I know not any better place to ask my question.  I am in process of reorganizing my server ,now running 6.11.5.  As I am following the development of the RC series of 6.12 I know ZFS will be available. I also watched Spaceinvader one's youtube video which explains all the new features (see below). 
 

After watching this video I was left with some questions on the capabilities of unraid array and zfs.  
I tried to learn the basics of zfs, and learned that an RAIDZ1 vdev pool requires all the drives are the same size. However this is not the case in my server.  Creating multiple RAIDz1 vdevs in which drives of the same size are clustered is certainly possible but then I have less storage capacity remaining. ZFS seems compelling for its capability to create snapshots on datasets.

 

The question: is it possible to have the best of both worlds with Unraid?

for example:  I create single disk vdevs and combine these in the Unraid array, with a parity drive. 

then in ZFS create a pool combining all (or some) of the VDEVS and create the datasets, and use the ZFS snapshot capabilities.

 

Downside would be that if the rebuild (if a drive fails) also fails, all data is lost, whilst with a xfs or btrfs filesystem one would lose only the data that was on the failed drive, right?

 

Thanks, Richard

 

 

nv

35 minutes ago, Richard Aarnink said:

The question: is it possible to have the best of both worlds with Unraid?

for example:  I create single disk vdevs and combine these in the Unraid array, with a parity drive. 

then in ZFS create a pool combining all (or some) of the VDEVS and create the datasets, and use the ZFS snapshot capabilities.

No, that's not possible, you can use ZFS in the array and/or pools, each has its advantages and disadvantages, main ones are posted here.

On 5/18/2023 at 7:32 AM, isvein said:

Seems to still be problems with Docker running off folder on zfs
If I look in say  NginxproxyManager:
 

[app         ] [5/18/2023] [7:28:47 AM] [Global   ] › ℹ  info      Manual db configuration already exists, skipping config creation from environment variables
[app         ] [5/18/2023] [7:28:49 AM] [Migrate  ] › ℹ  info      Current database version: none
[app         ] [5/18/2023] [7:28:51 AM] [Setup    ] › ℹ  info      Added Certbot plugins certbot-dns-domeneshop~=0.2.8 
[app         ] [5/18/2023] [7:28:51 AM] [Setup    ] › ℹ  info      Logrotate Timer initialized
[app         ] [5/18/2023] [7:28:51 AM] [Setup    ] › ℹ  info      Logrotate completed.
[app         ] [5/18/2023] [7:28:51 AM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...
[app         ] [5/18/2023] [7:28:51 AM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
[app         ] [5/18/2023] [7:28:56 AM] [IP Ranges] › ✖  error     getaddrinfo EAI_AGAIN ip-ranges.amazonaws.com
[app         ] [5/18/2023] [7:28:56 AM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[app         ] [5/18/2023] [7:28:56 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...
[app         ] [5/18/2023] [7:28:56 AM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[app         ] [5/18/2023] [7:28:56 AM] [Global   ] › ℹ  info      Backend PID 437 listening on port 3000 ..

I get what looks like an network error.
This error does not happen if set to default bridge

Tried again, following the procedure in this post:
 

Quote

First, create a docker user share configured as follows:

Share name: docker

Use cache pool: Only

Select cache pool: name of your ZFS pool

Next, on Docker settings page:

Enable docker: Yes

Docker data-root: directory

Docker directory: /mnt/user/docker

Even deleted my old "Docker" share and created an new with the name "docker"
Same happens, same errors containers wont work on custom networks created in terminal with "docker create network "name" --subnet "custom-subnet" and then setting fixed ip for the docker(s)

Not setting a static ip on container with custom subnet does also NOT work.

What does work, is custom docker network created WITHUOUT the --subnet flag so you get an 172.18.0.0/16 subnnet and with no static IP to the container.

------------------

So I checked the settings for both this networks using "docker network inspect "name"" and I see that the 172 subnet gets an gateway in its config but my custom subnet netwrok does NOT get an gateway.

But, even creating a network with for example  "docker create network "name" --subnet "192.168.2.0/24" --gateway "192.168.2.1"" does not work, the config gets the gateway but the container cant connect

So something in the back-end seems to not route things right and/or setup custom subnets correct when docker is on zfs folder

-----------
When using the same command for example  "docker create network "name" --subnet "192.168.2.0/24" --gateway "192.168.2.1"" when docker is on an btrfs image (still on the zfs-pool) it works

Also found out that if you dont use the --gateway flag, the docker network does not list an gateway, but it still works on the brtfs image

Just using "docker network create "name" so you get an 172 subnet, does create an gateway in the config tho.
--------------

So I cant understand it any different that when the docker is in a folder on zfs, something strange happens with the routing when creating a custom network with custom subnet or that the docker implementation on Unraid is not meant to work with custom networks with custom subnets

 

-----
Edit again:
I tried to make an custom subnet of 172.50.0.0/16 and that DOES work 😮 (just to see what would happen if I used the same network class as the auto generated ones)
even with custom ip address for the container.
So for some reason, docker on my side does NOT like 192.168.X.0/24 subnets when on folder.

if this is what it takes, I guess I can change my custom networks scopes
 

Edited May 19, 2023 by isvein

Quick question. I have scheduled monthly scrubs on my zfs cache pool. Will Unraid send me a notification if it finds/correct broken data? 

Edited May 19, 2023 by Niklas

10 minutes ago, Niklas said:

Will Unraid send me a notification if it finds/correct broken data?

Nope, but you can see here for a script for better pool monitoring, for btrfs and zfs, so you get a notification if errors are found, and not just after a scrub.

What's the reason for changing exclusive mounts from bind mount to symlink? Seems a number of services aren't happy with the symlink...

3 hours ago, jbear said:

RC6 Note: "Use symlinks instead of bind-mounts for exclusive shares" creates issues inside my containers.

 

If I add a path variable and set the container path to for instance /net, and the host path to /mnt/user, exclusive shares are not accessible with symlinks from within the container, worked fine before with bind-mounts.

You need to put a trailing "/" on your paths.

21 minutes ago, Kilrah said:

What's the reason for changing exclusive mounts from bind mount to symlink?

Bind mounts caused issues with zfs datasets resulting in confusion with the kernel mount table.

1 hour ago, JorgeB said:

Nope, but you can see here for a script for better pool monitoring, for btrfs and zfs, so you get a notification if errors are found, and not just after a scrub.

 

Thanks. Script added. Can't understand why Unraid doesn't do this out of the box. 

 

 

 

Edit 

I also had to adapt to the symlinks but no big problem.

Edited May 19, 2023 by Niklas

47 minutes ago, dlandon said:

You need to put a trailing "/" on your paths.

In that situation, using /mnt/usr/ still wouldn't fix their issue. They would need to individually map every single share, being sure to include include "/" at the end of every host path.

 

This seems Ike it will break a lot of use cases that simply map /mnt/usr/ . 

I have a question re exclusive shares:

Are exclusive shares only for shares that reference pools? Or are shares on the array that don't use a (old terminology) cache, and are restricted to a single disk also gain the benefit of transparently bypassing FUSE?

I have all my music on a share restricted to a single disk in the array, and also have my backup share restricted to a single disk in the array.

I currently have disk shares enabled to bypass FUSE in situations where I want a little more speed. Eg running a sync from my Windows machine to a share (scan time drops from 4 minutes to 20 seconds when using a disk share while transfer speed is unaffected); or running a media library scan in Logitech Media Server (twice as fast) or Plex (time drops from a few minutes to a few seconds).

It would be great if shares restricted to a single disk would automatically bypass FUSE and get the speed benefits.

This way I can just reference the "user" share directories (rather than direct disk) and in the future if either share fills up the single disk and I allow it to use another disk I don't have to worry about making sure that I've updated all my scripts and shortcuts to reference "user" shares instead of "disk" shares - the share will just start using FUSE transparently.

Thanks for your input!

2 hours ago, BRiT said:

In that situation, using /mnt/usr/ still wouldn't fix their issue. They would need to individually map every single share, being sure to include include "/" at the end of every host path.

 

This seems Ike it will break a lot of use cases that simply map /mnt/usr/ . 

 

I used /mnt/user for my duplicati backup container but I have now added the full path to shares/folders I want to backup instead (like /mnt/user/appdata - /source/appdata) 

 

I could have used /mnt/cache but I want no problems if I ever move stuff to array or something. 

Edited May 19, 2023 by Niklas

25 minutes ago, jademonkee said:

Are exclusive shares only for shares that reference pools? Or are shares on the array that don't use a (old terminology) cache, and are restricted to a single disk also gain the benefit of transparently bypassing FUSE?

At the moment only for pools, you can still bypass FUSE for an array disk by using the disk path, e.g. /mnt/disk1.

8 hours ago, JohnBee said:

Moving a share folder under rc6 leads to 'Share 'appdata' has been deleted'
Which seems like odd behavior, though I test this on a fresh install, and found the same behavior.

 

Thanks, at one point I messed with manual compression and found that you can run a "defragment" operation to force the compression of existing data, I suppose unraid is just not using that option? Maybe it could be added before 6.12 official?

If not I suppose I could run the defrag manually and it would not hurt anything?

I guess I can just move all the data off and then move it back, just seems like a lot of wasted effort lol.

44 minutes ago, TexasUnraid said:

If not I suppose I could run the defrag manually and it would not hurt anything?

You can do it manually, don't see LT adding that automatically because it's not always a good idea, e.g. don't run it for any btrfs fs with snapshots or they will start using a lot more space.

1 minute ago, JorgeB said:

You can do it manually, don't see LT adding that automatically because it's not always a good idea, e.g. don't run it for any btrfs fs with snapshots or they will start using a lot more space.

Interesting, was not aware of that catch. I will just remove the snapshots before I suppose.

2 hours ago, BRiT said:

In that situation, using /mnt/usr/ still wouldn't fix their issue. They would need to individually map every single share, being sure to include include "/" at the end of every host path.

 

This seems Ike it will break a lot of use cases that simply map /mnt/usr/ . 

The only difference will be that the /mnt/user/share for an exclusive share will be a symlink.  If you don't include the trailing "/", the mapping of /mnt/user/share will appear to be a file and not a directory.  If you use /mnt/user/share/ it will be treated as a directory.

25 minutes ago, dlandon said:

The only difference will be that the /mnt/user/share for an exclusive share will be a symlink.  If you don't include the trailing "/", the mapping of /mnt/user/share will appear to be a file and not a directory.  If you use /mnt/user/share/ it will be treated as a directory.

No. You're not understanding.

 

They are using 1 single "/mnt/usr/" to pass everything through. The symlinks underneath are thus treated as just file and thus the dockers fail.

2 minutes ago, BRiT said:

No. You're not understanding.

 

They are using 1 single "/mnt/usr/" to pass everything through. The symlinks underneath are thus treated as just file and thus the dockers fail.

The dockers are assuming that the symlinks are files and not directories.

 

The option is to not to set up exculsive shares and map the share directly in cases where it is needed.  For example, I have my appdata mapped directly to /mnt/cache/appdata/ and not /mnt/user/appdata/.  This in effect provides the same operation as exclusive share access at /mnt/user/appdata/ symlinked to /mnt/cache/appdata/.

 

The whole idea behind exclusive shares is to get around the /mnt/user/ file handling so access is faster and the user doesn't have to be concerned with the direct share mapping.  You don't have to use it.

7 hours ago, isvein said:

... the docker implementation on Unraid is not meant to work with custom networks with custom subnets

 

Unraid is not involved in custom networks created by the user, it is however managing macvlan / ipvlan custom networks.

 

When you create your own custom networks using docker CLI, it is your own responsiblity to do the correct creation of such a network.

 

4 hours ago, Niklas said:

 

Thanks. Script added. Can't understand why Unraid doesn't do this out of the box. 

 

 

 

Edit 

I also had to adapt to the symlinks but no big problem.

I actually already had a trailing / and did not state that in my post.   I have a few dockers that use this, Krusader being one ...  A few others as well ...  For now I just added a folder on one of my drives with the same name as the share to disable exclusive access.

 

Thank you for replying.

Edited May 19, 2023 by jbear