Marshalleq

Ah yeah, that annoying app data folder that sometimes insists on creating itself in a location not set in the defaults!

That doesn't seem to make a lot of sense does it. I assume you know about zpool upgrade and that shouldn't be impacting this. Assuming you're using the plugin, perhaps just uninstall and reinstall the zfs plugin and reboot. Also make sure this is the latest unraid version as recent versions were problematic. Also, if your docker is in a folder rather than in an image I would suggest trying it as an image. Docker folders on ZFS seem to be hit and miss depending on your luck. Can't think of anything else to try right now.

I get that you want to make the best of your disks, it is in my experience generally recommended to give ZFS whole disks due to the way it works. That said I think you can give it partitions. What I'm not sure about is if you could give unraid partitions. This configuration is probably going to be terrible performance though. I'd suggest replace the 10TB disks with what you buy, then create a ZFS mirror with the 10TB disks (assuming you want the bigger ones in unraid. You can still use both disks to store things, you can't get around losing two disks for parity though unless you go all ZFS or all unraid array.

Good answer, this question gets asked so often, I wish someone would just include it in dev tools or something. Clearly there's demand.

It didn’t work for me either but haven’t tried lately. While I prefer folders it seems risky if there’s no confirmation that something has been fixed, changed it added. It might break at next upgrade. So will stick to images for now.

Gotcha. I wish I had developer mates around so I could learn from watching - I reckon there would be a ton of basic dev stuff I'd be good at. I'd be a bit scared of losing my life to IT though - it's bad enough already!

It's pretty hard to beat ZFS when it had over a billion dollars spent on it and has been around for 30 years. Compare that to BTRFS being open source and a little over 10 years. But it's the denial of any issue on BTRFS that concerns me, you're never going to get a good file system if the project is going to deny there's anything wrong all the time or worse ask why you would even want these things fixed! Oh and thanks for the podcast @ConnectivIT good to have actual examples explained for everyone. It's been a year since then, so hopefully one of the devs heard it and did something about it. Personally I'd like to see BTRFS removed from Unraid and replaced with ZFS. Or at least the option!

So no redundancy for your dockers then?

I heard a rumour, last time I tried it didn't work because Unraid needed some kind of ZFS drive compiled into docker or something - I'm not really sure why everything has started working - so I'm nervous to change more. Perhaps Unraid guys included basic ZFS libraries in the new version of unraid or something.

Nice Podcast! Signed up - they seem like good fun.

All you gotta do is search these forums and you shall be rewarded with many examples

I'll never use BTRFS again unless it comes out that they've admitted and fixed whatever it is that keeps making it fail.

@ich777@steini84@Joly0 Both my systems seem to still be running well on latest unraid and latest ZFS. Both docker.img set as xfs and residing on ZFS SSD Mirror. I am nervous as to why or what changed and wondering what's different between mine and Joly0 - nevertheless it seems like we have a couple of different scenarios we should be able to work it out from.

There was some time ago, issue with something on custom docker networks, should probably disable that and check. I've updated my test system now and it 'seems' to be OK with latest ZFS and latest unraid. If it is, I will try adding those few docker containers in common and see if there's an impact.

Hey thanks so much for your help Arragon, this is great! So if Joly0 has those 3 dockers in common, of those I have nextcloud and mariadb. There are plenty of others that I have in common with Arragon though, but I'll leave them out as they seem to be irrelevant. Also, I have a test box that has unraid but is basically only used for encoding so has nothing in common with what either of you have listed in regards to dockers used, so I will upgrade it all to latest shortly and see if it's impacted. As far as ZFS setups, on my test box I have: AMD Threadripper 1950X with 128G RAM RaidZ1 3x8TB Seagate HDD's SATA Mirror 4x240G M.2 intel SSD's in a mirror (e.g 480 total space) SATA XFS 128G Intel NVME for testing docker.img. NVME The other box is similar with Intel Xeon E5-2620 v3 with 96GB ECC RAM Raidz1 4x16TB Seagates SATA Mirror 2x150G+2x480G Intels SATA Mirror + 1x960G as a standalone ZFS. SATA This box has some of the disks on Dell Perc H310 in IT mode. Both of these boxes have exhibited the issue. Both systems have docker.img on mirrored zfs shared with active docker configs. One thing I don't see listed above is fstrim, which I do have enabled on all my ssd's. I also have xattr set to sa and I am using the new compression type zstd-1. I have attached output of zpool get all and one zfs get all from one snapshot Also screenshot of my docker settings below: What is your host access to custom networks set to? That's something else I've enabled from default. ZpoolGetAll.rtf ZFSGetAll.rtf

I wonder if this should be asked on ZFS forums first to confirm what kind of activity happens at how often? It may be that e.g. some kind of metadata check happens every 10 mins or something. I used to wonder about this too, but I've recently upgraded to 16TB seagate that have some kind of magic 'nearly the same as spindown' at idle thing so am not so concerned now. It's a good question though!

Found the problem - apparently unraid will do this when you have added a docker container not via the unraid gui. Perhaps you could get around it by adding something else into the docker container via the GUI, but deleting the other docker containers got around the issue. Go figure.

Yeah, so that issue will be over once it's baked in which is exciting. And yes again, DVD hoarders lol, I remember guys that were like that, whole walls of spent money on CD's and such. The thing I don't get is there's a whole market there in that file and print space (and even as a home market) that unraid almost completely flips the bird at, maybe they don't want to be compared with FreeNAS / TrueNAS (which is looking pretty exciting at the moment). But, unraid still wins or has the up and coming tag on it when you add in the community support - unraid has that area nailed and I doubt anyone will ever be able to even compete with it. Actually I don't think I've ever seen it on any product online ever. Other thing on my wishlist would be a proper software network stack - it doesn't work well when you start swapping around network cards (to much disbelief by others that have looked at my message one day when I decided to rant lol). One day I'll document it down properly - when I have the will to go up against the man again.

Ive upgrade to 6.9 final on another machine today - I notice some of the docker s have my previous pinning on - however I'm not able to set it on 4 dockers I'm trying to limit. Haven't tried the others yet. When I say I'm not able to, what actually happens is I can select the threads I want, I can apply them, it looks like it applies after I have pressed apply, but if I refresh the screen / go out, they're all gone again. It's back to how it was originally. Perhaps it's some kind of permissions issue, it's behaving like it's read only but doesn't know it. I can confirm it's working on my main machine on latest 6.9 so it's likely not 6.9 specific. I can also confirm it is working for VM's fine. Weird.

Limetech wasn't expressing frustration with BTRFS as far as I know, they did however hear some others frustration with BTRFS, which in one interview was referred to as a 'certain group whom remain vocal about the issues' or words to that effect. One has to wonder how many failures a file system has to go through before you will agree it's problematic (there are plenty recorded on these forums) and in my world one failure is one too many. Anyway, to answer your question, I do believe LimeTech said that they are most definitely looking at bringing it in, most definitely not in this latest version, but in the future. No date, nor release has been provided and no guarantee either, just a good whole hearted we're looking at it - which is very exciting and hopeful sounding. I doubt it's very hard to be honest. My one hope is that it's not limited to existing within their own unraid array as it's main benefits would largely be unavailable. And that things like docker and vm's will run on it. Personally I think the future of unraid lies in it's unraid driver being for mass storage and zfs for everything else. However I've migrated completely away from the unraid raid driver, I found it to make my system too sluggish. But there are many advantages to unraid, over and above it's disk setup, so it's still worth it for those I think. I do still think about the enterprise features that are missing though. The basic file and print, backup, user accounts, directory management etc would be a great start. Hopefully some day in the future.

That link takes me to a page which does not have any reference to SMB that I can see - where exactly is the workaround on SMB? Many thanks.

@Arragon what @steini84 is suggesting is to mount your docker to a folder rather than a .img file. I'm not sure if the bit above is new since the previous beta though, but it would pay to check. If it works, that could solve the problem and enable us to run later versions of ZFS. The problem I see is we still don't know why .img files won't run on ZFS filesystem above 2.0.0 and seemikly this is specific to Unraid. No reports from any other system. I'll try it later when I have some time. Also, I'm not sure if that zfs storage driver refers to zfs as an underlying file system, or within the docker container - I assume the latter.

I've just run this script, and now everything is crawling slow. I guess because lancache probably wasn't working before, or because to function efficiently it really needs sendfile. I'm going to have to disable it if I can't fix it. I remember this slow speed happened a long time ago so perhaps it hasn't been working since then at all and it's just how it is. As a guide, I'm on Gigabit fibre and my initial download starts off fast e.g. 60 MB/s then get's down to kb/s and in fact steam is telling me at one point it will be completed in 'years' lol. It bounces up again, but not more than about 2MB/s. Also noticing problems with my MS teams client for some reason, not sure if it's related but it's a mighty coincidence.

@ConnectivIT @Josh.5 Hey, so can't we just mount the /etc/ folder into local server e.g. /mnt/SSDPool1/docker/lancache/etc? I assume it's resetting the sendfile setting because the container get's updated or something? I haven't tested if this impacts me or not, but I believe it will be since I have the same kind of setup. And while looking into it, I noted there is not persistent store for lancache configuration, only logs and data. Thanks.

There is no issue with ZFS 2.0.0-1 running the docker image on ZFS, it's only when you get to the next version and above. I think I saw someone else talking about this lancache issue and I do run it myself, so should really look into it. Thanks.