Hi, for years I've begrudgingly run a btrfs docker image, which was hosted on an SSD with underlying XFS or ZFS no problem.

 

There has been for a while the option of instead storing this into a folder, which I'd prefer for various reasons.

 

However, it seems that docker is deciding that it won't store files on the file system, I assume because it's currently formatted ZFS and is looking for XFS or BTRFS underneath.  I don't understand why it would care, given that it's no longer an image what file system is underneath?

 

The below from the docker.log file

 

failed to start daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)

 

Anyone know why?  I'd like to log a ticket, but thought I'd ask here first.

 

Thanks.

You probably just aren't specifying the path correctly to get it to use a folder. Had to dig back a little to find when this was introduced in 6.9b24:

 

Thanks for the reply and link, I'm just setting it up under a folder on a ZFS drive via it's location, so /mnt/mountpoint/docker/docker/

 

When I point it at non zfs i.e. /mnt/user/docker/docker/ it works, but if I change it to the zfs one I get the error.

 

So based on the error in the log, I assume it's detecting ZFS somehow, however, I didn't check to see what the log says on XFS, perhaps it's the same and the cause is something else.  Either way, I strongly suspect it's some unraid config / bug here.

There's different drivers for docker to be on different filesystems.  AFAIK, there's only the 2 included in the OS -> btrfs and xfs.  You could probably do a feature req to have the driver for zfs included for docker

 

Your other option is either a btrfs or xfs image stored on the zfs pool.

Thanks Squid.  I half thought I read that, but just got lost wondering why it even needed to know it's underlying filesystem and therefore a driver, therefore thought I must have misunderstood.  Do you know why it needs a driver for an underlying filesystem?  I'll do a feature request, if Unraid actually does end up including ZFS it won't harm to have it done already.

@Marshalleq I'm very new to UnRaid & ZFS, but the material here is AMAZING!  Squid mentioned above that we can create a .xfs/.btrfs/ext4 volume on top of a ZFS zvol.  I've done it by creating both a btrfs & xfs volume that doesn't appear to have any issues, currently using the btrfs one.  There was some fun in getting it mounted, but this link set me straight finally!  Other sites said to mkfs.btrfs /dev/zd0p1 (which didn't work) vs /dev/zvol/POOL/ZVOL.  I did run into issues on how to mount the damned ZVOL after...kinda forgot which worked now, but it was a bit of a hassle. Hope this helps, as I can now has any FS I need on the frontend, but with all of the power of ZFS on the backend.  

 

 

 

Edited March 17, 20215 yr by OneMeanRabbit

Not sure if this is still relevant as 2 days ago (not knowing about this post) i moved from docker image to folder on zfs (dataset)and everything works just fine.
I run 6.9.1 build with zfs included via kernel helper docker.

edit : directly on zfs so no zvol stuff as mentioned in other posts

14 hours ago, OneMeanRabbit said:

@Marshalleq I'm very new to UnRaid & ZFS, but the material here is AMAZING!  Squid mentioned above that we can create a .xfs/.btrfs/ext4 volume on top of a ZFS zvol.  I've done it by creating both a btrfs & xfs volume that doesn't appear to have any issues, currently using the btrfs one.  There was some fun in getting it mounted, but this link set me straight finally!  Other sites said to mkfs.btrfs /dev/zd0p1 (which didn't work) vs /dev/zvol/POOL/ZVOL.  I did run into issues on how to mount the damned ZVOL after...kinda forgot which worked now, but it was a bit of a hassle. Hope this helps, as I can now has any FS I need on the frontend, but with all of the power of ZFS on the backend.  

 

 

 

You have a functioning docker 'folder' on ZFS filesystem, rather than a docker folder on BTRFS / XFS and then on ZFS?

Are there any benefits in migrating from btrfs docker.img to directory?

Will it use less space, because a fixed 20 GB allocation is not required anymore?

Is it possible to transfer all the data in existing docker.img to docker directory?

2 hours ago, Forusim said:

Are there any benefits in migrating from btrfs docker.img to directory?

Will it use less space, because a fixed 20 GB allocation is not required anymore?

Is it possible to transfer all the data in existing docker.img to docker directory?

 

I have the exact same question as your first one, but I can reply to the other 2 since I made the switch following a recommendation in one of SpaceInvader's video (he doesn't say why however).

 

It doesn't really use less space, at least for me. I'm not sure what's using 17GB of space honestly, I'm running 4 dockers: qbittorentvpn, plex, youtube-dl and duplicati.

 

root@server:~# du -h /mnt/cache/system/docker/docker/ | sort -hr | head
17G     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes
17G     /mnt/cache/system/docker/docker/docker/btrfs
17G     /mnt/cache/system/docker/docker/docker
17G     /mnt/cache/system/docker/docker/
986M    /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init
986M    /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3
986M    /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/681dbb488afc61adc745b592f266b76ac9cd52b6689fe7a7bb9acd3eb117ad4f
964M    /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3/usr
964M    /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr
964M    /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/681dbb488afc61adc745b592f266b76ac9cd52b6689fe7a7bb9acd3eb117ad4f/usr

 

As for whether you can transfer all the data... well, there is no data in your docker.img file technically, all you need are your customized user templates located in /boot/config/plugins/dockerMan/templates-user/. Then you just add your containers again, it should remember their configuration and boot normally.

One of your dockers almost certainly has a cache / temporary / cancelled files or similar contained within in.  My guess would be qbittorrent vpn.  Since you have it in a folder, you should be able to figure that out with du -sh /folderpath.

 

Also, I do believe that you would save space - because the difference between the 20GB image file and the actual docker space (in your case 3GB difference) is still a difference.  The main exception to this is when you have compression on the filesystem hosting the image such as ZFS - it will recognise the image files blank space and compress it effectively to zero.

4 minutes ago, Marshalleq said:

One of your dockers almost certainly has a cache / temporary / cancelled files or similar contained within in.  My guess would be qbittorrent vpn.  Since you have it in a folder, you should be able to figure that out with du -sh /folderpath.

 

Well if I go deeper I can see from one of the directory to another that similar lib files are here.

 

root@server:~# ls -l /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/ | wc -l
52

root@server:~# du -sh /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/* | sort -hr | head
95M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/python3.9
90M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/libLLVM-11.so
53M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/libgo.so.16.0.0
36M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/dri
32M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/libgphobos.so.1.0.0
28M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/perl5
28M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/libicudata.so.69.1
20M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/libstdc++.so.6.0.28
19M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/udev
16M     /mnt/cache/system/docker/docker/docker/btrfs/subvolumes/f241e33ad5c10ab40fdc2f4c33e5a3cc6b8a0b7dffa7cad22188a348a9de96c3-init/usr/lib/d3d

 

I'm just very confused by what are the subvolumes exactly? I have 4 dockers but there are 52 subvolumes directories? Also, how do I know which subvolume belongs to which docker? At any rate, I don't really mind docker consuming 17G of space, it's curiosity more than anything else considering that all my dockers are configured to have their data into the appdata and they do consume 17GB right off after booting which seems like an insane amount of space for something that should theoretically be lightweight.

I’m pretty sure I was able to run about 30 dockers in less than 5G of space - but I’d have to check that. What you can do is just stop the docker service, rename the image file / directory and restart the service. If configured as you say the image file / directory should be much smaller. I have seen some docker images not playing nice with the appdata by default so with this you can see which folder is growing amd investigate from there. 
 

if one folder is growing you will likely find you’ll have to increase your image file size or allow more space for the folder Indefinitely. 

2 hours ago, Marshalleq said:

I’m pretty sure I was able to run about 30 dockers in less than 5G of space - but I’d have to check that. What you can do is just stop the docker service, rename the image file / directory and restart the service. If configured as you say the image file / directory should be much smaller. I have seen some docker images not playing nice with the appdata by default so with this you can see which folder is growing amd investigate from there. 
 

if one folder is growing you will likely find you’ll have to increase your image file size or allow more space for the folder Indefinitely. 

 

I deleted the docker directory and downloaded every docker 1 by 1.

 

youtube-dl: 1.1 G

duplicati: 3.4 G

qbittorrentvpn: 7.5 G

linuxserver/plex: 4 G

 

Total of 17GB exactly. I'm a little bit confused because with the same dockers my docker.img was 4.1GB before updating to 6.9.X. I am... unsure what's going on exactly to say the least, especially if you have 30 dockers in less than 5G, that's insane in comparison.