Managed to fill docker.img and didnt notice, now all dockers gone on reboot


Recommended Posts

Title says it all really.

 

so... I suspect it was deluge or plex, not sure, but from the looks of it although .img was full (now expanded to 50gb and 500mb x3 logs). All appdata is there and accessible within cache mirror.

 

Anyone know how I can remap the docker containers? profiles still there but don't want to overwrite data on new container creation.

 

I have full nightly backups but just extracting 25gig tar would take a night probably.

 

any help appreciated.

Link to comment

Delete docker.img and recreate, adding dockers back in from My Dockers list, except this time actually look at each mapping of each docker to make sure its still not fraked up like before.

The flow is similiar to that, might not be exact, but it's close enough for anyone to just poke around and get it going.

Link to comment

thanks, looks like unraid did that for me, only 500mb used when expanding size, so ended up moving existing data into subdir rather than extract from backup, after going through all templates (seems like unraid only handles 5 adds at once) setting and mappings were preserved which is nice, and now I've stopped docker and I'm moving data back into dir's.

 

I'm still assuming that it was the 20gig that filled up, if I didn't force delete the .img will unraid be happy on next boot, don't feel like going through restore again.

Link to comment
55 minutes ago, Mizerka said:

thanks, looks like unraid did that for me, only 500mb used when expanding size, so ended up moving existing data into subdir rather than extract from backup, after going through all templates (seems like unraid only handles 5 adds at once) setting and mappings were preserved which is nice, and now I've stopped docker and I'm moving data back into dir's.

Much easier to simply go to the Apps tab, check off everything you want (more than 5 possible), and then hit install Multiple.

Link to comment
9 hours ago, Squid said:

Much easier to simply go to the Apps tab, check off everything you want (more than 5 possible), and then hit install Multiple.

next morning, so restarted docker and its all gone again...  fml.

 

So deleted docker.img this time properly and will try adding through apps and see how it goes.

 

edit;

so adding through apps is more convenient (despite being paged limited) but not faster, running a single docker add at a time compared to 5 tabs etc.

 

edit2;

okay, so manually deleting .img seems to have to worked, after reinstall and docker restart, everything still there. 10gig used after just install of what I acutally use (16, typical plex/sonarr/deluge setup etc). Restoring appdata now and will see what happens.

 

plex needs to chill with their file structure;

image.png.09479165f880736fcaf26c0901ceed4d.png

1.25m folders and 350k files and still going...

Edited by Mizerka
Link to comment

The reason I said to do it one at a time is you obviously have something wrongly mapped on at least one of the dockers. There's no reason to need more space on the actual docker.img. Everything that requires large amounts of space should be mapped through outside the docker container. If you dont review and correct the wrong mapping(s) you will continue to have your original issue.

Link to comment
54 minutes ago, BRiT said:

The reason I said to do it one at a time is you obviously have something wrongly mapped on at least one of the dockers. There's no reason to need more space on the actual docker.img. Everything that requires large amounts of space should be mapped through outside the docker container. If you dont review and correct the wrong mapping(s) you will continue to have your original issue.

In fact, you should recreate docker image with 20G. Making it larger will just take longer to fill. It won't fix the problem. I've never seen anyone needing more than 20G.

 

The usual cause of filling docker image is an application writing to a path that isn't mapped. Your mappings might be correct, but if the application isn't using those mapped paths, it is going to write into the image.

 

Typical mistakes are using different upper/lower case when specifying paths, or not using an absolute path (must start with / ).

Link to comment
1 hour ago, BRiT said:

The reason I said to do it one at a time is you obviously have something wrongly mapped on at least one of the dockers. There's no reason to need more space on the actual docker.img. Everything that requires large amounts of space should be mapped through outside the docker container. If you dont review and correct the wrong mapping(s) you will continue to have your original issue.

Thanks, and ye I get that, after adding fresh templates, no data or anything, it filled 10gb, all dockers should have their own appdata paths for config etc and anything pulling data is using main storage pool as well. I will go over all of them regardless, the only one I can think of would be influxdb which is storing sensors for grafana etc, I believe it should keep its db within appdata as well thought.

 

31 minutes ago, Squid said:

Future reference:  It's not page limited

What I meant is that I've a number of dockers in use and tested tripple that before as well. so I've 4 pages of old apps, unless im blind or missed some config somewhere I can't display more than 15 items per page under previous apps view. so could do 5-6 at a time (alphabetical) and also again seems to run them in sequence, which is probably intended method of install, but using multiple tabs I could add 5-6 at a time, sure they'd time each other out and cap wan link etc. using apps did "feel" faster though either way, thanks.

 

18 minutes ago, trurl said:

In fact, you should recreate docker image with 20G. Making it larger will just take longer to fill. It won't fix the problem. I've never seen anyone needing more than 20G.

 

The usual cause of filling docker image is an application writing to a path that isn't mapped. Your mappings might be correct, but if the application isn't using those mapped paths, it is going to write into the image.

 

Typical mistakes are using different upper/lower case when specifying paths, or not using an absolute path (must start with / ).

I'd agree, and haven't had issues for a long time, like above, after fresh container reinstall from templates, it filled 10gb. I will go over all of them but can't imagine any of them being big enough to need 10gb and even then that'd take like an hour to dl which I'd notice on reinstall.

 

 

 

what it looks like, ignore disks, need to move some data around to reduce spinups, but ye docker 9.3g fresh installs with no configs

root@..:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs           48G  762M   47G   2% /
tmpfs            32M  488K   32M   2% /run
devtmpfs         48G     0   48G   0% /dev
tmpfs            48G     0   48G   0% /dev/shm
cgroup_root     8.0M     0  8.0M   0% /sys/fs/cgroup
tmpfs           128M   64M   65M  50% /var/log
/dev/sda1        29G  447M   29G   2% /boot
/dev/loop0      8.7M  8.7M     0 100% /lib/modules
/dev/loop1      5.9M  5.9M     0 100% /lib/firmware
/dev/md1        3.7T  341G  3.4T  10% /mnt/disk1
/dev/md2        3.7T  841G  2.9T  23% /mnt/disk2
/dev/md3        3.7T  1.6T  2.2T  43% /mnt/disk3
/dev/md4        3.7T  232G  3.5T   7% /mnt/disk4
/dev/md5        7.3T  2.7T  4.7T  37% /mnt/disk5
/dev/md6        7.3T  3.6T  3.7T  50% /mnt/disk6
/dev/md7        7.3T  3.5T  3.9T  47% /mnt/disk7
/dev/md8        7.3T  2.4T  5.0T  33% /mnt/disk8
/dev/md9        9.1T   11G  9.1T   1% /mnt/disk9
/dev/md10       932G  290G  642G  32% /mnt/disk10
/dev/md11       932G  983M  931G   1% /mnt/disk11
/dev/md12       932G  983M  931G   1% /mnt/disk12
/dev/md13       932G  983M  931G   1% /mnt/disk13
/dev/md14       932G  983M  931G   1% /mnt/disk14
/dev/md15       932G  983M  931G   1% /mnt/disk15
/dev/md16       932G  983M  931G   1% /mnt/disk16
/dev/sdz1       466G  103G  363G  23% /mnt/cache
shfs             60T   16T   44T  26% /mnt/user0
shfs             60T   16T   45T  26% /mnt/user
/dev/loop3      1.0G   17M  905M   2% /etc/libvirt
/dev/loop2       50G  9.3G   41G  19% /var/lib/docker
shm              64M     0   64M   0% /var/lib/docker/containers/97408ac7186b33a59b3af02cdef729f733f03899aea369a78caee2130a7c9fb4/mounts/shm

 

Edited by Mizerka
Link to comment
8 minutes ago, Mizerka said:

I'd agree, and haven't had issues for a long time, like above, after fresh container reinstall from templates, it filled 10gb. I will go over all of them but can't imagine any of them being big enough to need 10gb and even then that'd take like an hour to dl which I'd notice on reinstall.

He's not talking about the fresh install being that big, it's the ongoing activity where the paths defined INSIDE the container app don't match the paths defined by your docker config. If you have /media mapped to /mnt/user/Media, and you tell your docker app to download to /Media, it's going to fill up the docker image and not end up on the array.

Link to comment
1 hour ago, Squid said:

What I meant is that you can go from page 1 to 2, and it'll remember what you had checked off and install

My bad, didn't assume to try it, given it feels compelled to refresh apps on every new view. thanks.

 

1 hour ago, jonathanm said:

He's not talking about the fresh install being that big, it's the ongoing activity where the paths defined INSIDE the container app don't match the paths defined by your docker config. If you have /media mapped to /mnt/user/Media, and you tell your docker app to download to /Media, it's going to fill up the docker image and not end up on the array.

Ye and I understand that, but again, this hasn't been an issue for ages, I will go over all mounts but all of them are being used correctly from what I could see, since I'd see data dissapearing into .img instead of my libraries and have messed up cases in the past that'd cause this.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.