Mizerka Posted July 8, 2019 Share Posted July 8, 2019 Title says it all really. so... I suspect it was deluge or plex, not sure, but from the looks of it although .img was full (now expanded to 50gb and 500mb x3 logs). All appdata is there and accessible within cache mirror. Anyone know how I can remap the docker containers? profiles still there but don't want to overwrite data on new container creation. I have full nightly backups but just extracting 25gig tar would take a night probably. any help appreciated. Quote Link to comment
BRiT Posted July 8, 2019 Share Posted July 8, 2019 Delete docker.img and recreate, adding dockers back in from My Dockers list, except this time actually look at each mapping of each docker to make sure its still not fraked up like before. The flow is similiar to that, might not be exact, but it's close enough for anyone to just poke around and get it going. Quote Link to comment
Mizerka Posted July 8, 2019 Author Share Posted July 8, 2019 thanks, looks like unraid did that for me, only 500mb used when expanding size, so ended up moving existing data into subdir rather than extract from backup, after going through all templates (seems like unraid only handles 5 adds at once) setting and mappings were preserved which is nice, and now I've stopped docker and I'm moving data back into dir's. I'm still assuming that it was the 20gig that filled up, if I didn't force delete the .img will unraid be happy on next boot, don't feel like going through restore again. Quote Link to comment
Squid Posted July 8, 2019 Share Posted July 8, 2019 55 minutes ago, Mizerka said: thanks, looks like unraid did that for me, only 500mb used when expanding size, so ended up moving existing data into subdir rather than extract from backup, after going through all templates (seems like unraid only handles 5 adds at once) setting and mappings were preserved which is nice, and now I've stopped docker and I'm moving data back into dir's. Much easier to simply go to the Apps tab, check off everything you want (more than 5 possible), and then hit install Multiple. Quote Link to comment
Mizerka Posted July 9, 2019 Author Share Posted July 9, 2019 (edited) 9 hours ago, Squid said: Much easier to simply go to the Apps tab, check off everything you want (more than 5 possible), and then hit install Multiple. next morning, so restarted docker and its all gone again... fml. So deleted docker.img this time properly and will try adding through apps and see how it goes. edit; so adding through apps is more convenient (despite being paged limited) but not faster, running a single docker add at a time compared to 5 tabs etc. edit2; okay, so manually deleting .img seems to have to worked, after reinstall and docker restart, everything still there. 10gig used after just install of what I acutally use (16, typical plex/sonarr/deluge setup etc). Restoring appdata now and will see what happens. plex needs to chill with their file structure; 1.25m folders and 350k files and still going... Edited July 9, 2019 by Mizerka Quote Link to comment
BRiT Posted July 9, 2019 Share Posted July 9, 2019 The reason I said to do it one at a time is you obviously have something wrongly mapped on at least one of the dockers. There's no reason to need more space on the actual docker.img. Everything that requires large amounts of space should be mapped through outside the docker container. If you dont review and correct the wrong mapping(s) you will continue to have your original issue. Quote Link to comment
Squid Posted July 9, 2019 Share Posted July 9, 2019 4 hours ago, Mizerka said: (despite being paged limited) Future reference: It's not page limited Quote Link to comment
trurl Posted July 9, 2019 Share Posted July 9, 2019 54 minutes ago, BRiT said: The reason I said to do it one at a time is you obviously have something wrongly mapped on at least one of the dockers. There's no reason to need more space on the actual docker.img. Everything that requires large amounts of space should be mapped through outside the docker container. If you dont review and correct the wrong mapping(s) you will continue to have your original issue. In fact, you should recreate docker image with 20G. Making it larger will just take longer to fill. It won't fix the problem. I've never seen anyone needing more than 20G. The usual cause of filling docker image is an application writing to a path that isn't mapped. Your mappings might be correct, but if the application isn't using those mapped paths, it is going to write into the image. Typical mistakes are using different upper/lower case when specifying paths, or not using an absolute path (must start with / ). Quote Link to comment
Mizerka Posted July 9, 2019 Author Share Posted July 9, 2019 (edited) 1 hour ago, BRiT said: The reason I said to do it one at a time is you obviously have something wrongly mapped on at least one of the dockers. There's no reason to need more space on the actual docker.img. Everything that requires large amounts of space should be mapped through outside the docker container. If you dont review and correct the wrong mapping(s) you will continue to have your original issue. Thanks, and ye I get that, after adding fresh templates, no data or anything, it filled 10gb, all dockers should have their own appdata paths for config etc and anything pulling data is using main storage pool as well. I will go over all of them regardless, the only one I can think of would be influxdb which is storing sensors for grafana etc, I believe it should keep its db within appdata as well thought. 31 minutes ago, Squid said: Future reference: It's not page limited What I meant is that I've a number of dockers in use and tested tripple that before as well. so I've 4 pages of old apps, unless im blind or missed some config somewhere I can't display more than 15 items per page under previous apps view. so could do 5-6 at a time (alphabetical) and also again seems to run them in sequence, which is probably intended method of install, but using multiple tabs I could add 5-6 at a time, sure they'd time each other out and cap wan link etc. using apps did "feel" faster though either way, thanks. 18 minutes ago, trurl said: In fact, you should recreate docker image with 20G. Making it larger will just take longer to fill. It won't fix the problem. I've never seen anyone needing more than 20G. The usual cause of filling docker image is an application writing to a path that isn't mapped. Your mappings might be correct, but if the application isn't using those mapped paths, it is going to write into the image. Typical mistakes are using different upper/lower case when specifying paths, or not using an absolute path (must start with / ). I'd agree, and haven't had issues for a long time, like above, after fresh container reinstall from templates, it filled 10gb. I will go over all of them but can't imagine any of them being big enough to need 10gb and even then that'd take like an hour to dl which I'd notice on reinstall. what it looks like, ignore disks, need to move some data around to reduce spinups, but ye docker 9.3g fresh installs with no configs root@..:~# df -h Filesystem Size Used Avail Use% Mounted on rootfs 48G 762M 47G 2% / tmpfs 32M 488K 32M 2% /run devtmpfs 48G 0 48G 0% /dev tmpfs 48G 0 48G 0% /dev/shm cgroup_root 8.0M 0 8.0M 0% /sys/fs/cgroup tmpfs 128M 64M 65M 50% /var/log /dev/sda1 29G 447M 29G 2% /boot /dev/loop0 8.7M 8.7M 0 100% /lib/modules /dev/loop1 5.9M 5.9M 0 100% /lib/firmware /dev/md1 3.7T 341G 3.4T 10% /mnt/disk1 /dev/md2 3.7T 841G 2.9T 23% /mnt/disk2 /dev/md3 3.7T 1.6T 2.2T 43% /mnt/disk3 /dev/md4 3.7T 232G 3.5T 7% /mnt/disk4 /dev/md5 7.3T 2.7T 4.7T 37% /mnt/disk5 /dev/md6 7.3T 3.6T 3.7T 50% /mnt/disk6 /dev/md7 7.3T 3.5T 3.9T 47% /mnt/disk7 /dev/md8 7.3T 2.4T 5.0T 33% /mnt/disk8 /dev/md9 9.1T 11G 9.1T 1% /mnt/disk9 /dev/md10 932G 290G 642G 32% /mnt/disk10 /dev/md11 932G 983M 931G 1% /mnt/disk11 /dev/md12 932G 983M 931G 1% /mnt/disk12 /dev/md13 932G 983M 931G 1% /mnt/disk13 /dev/md14 932G 983M 931G 1% /mnt/disk14 /dev/md15 932G 983M 931G 1% /mnt/disk15 /dev/md16 932G 983M 931G 1% /mnt/disk16 /dev/sdz1 466G 103G 363G 23% /mnt/cache shfs 60T 16T 44T 26% /mnt/user0 shfs 60T 16T 45T 26% /mnt/user /dev/loop3 1.0G 17M 905M 2% /etc/libvirt /dev/loop2 50G 9.3G 41G 19% /var/lib/docker shm 64M 0 64M 0% /var/lib/docker/containers/97408ac7186b33a59b3af02cdef729f733f03899aea369a78caee2130a7c9fb4/mounts/shm Edited July 9, 2019 by Mizerka Quote Link to comment
JonathanM Posted July 9, 2019 Share Posted July 9, 2019 8 minutes ago, Mizerka said: I'd agree, and haven't had issues for a long time, like above, after fresh container reinstall from templates, it filled 10gb. I will go over all of them but can't imagine any of them being big enough to need 10gb and even then that'd take like an hour to dl which I'd notice on reinstall. He's not talking about the fresh install being that big, it's the ongoing activity where the paths defined INSIDE the container app don't match the paths defined by your docker config. If you have /media mapped to /mnt/user/Media, and you tell your docker app to download to /Media, it's going to fill up the docker image and not end up on the array. Quote Link to comment
Squid Posted July 9, 2019 Share Posted July 9, 2019 What I meant is that you can go from page 1 to 2, and it'll remember what you had checked off and install 1 Quote Link to comment
Mizerka Posted July 9, 2019 Author Share Posted July 9, 2019 1 hour ago, Squid said: What I meant is that you can go from page 1 to 2, and it'll remember what you had checked off and install My bad, didn't assume to try it, given it feels compelled to refresh apps on every new view. thanks. 1 hour ago, jonathanm said: He's not talking about the fresh install being that big, it's the ongoing activity where the paths defined INSIDE the container app don't match the paths defined by your docker config. If you have /media mapped to /mnt/user/Media, and you tell your docker app to download to /Media, it's going to fill up the docker image and not end up on the array. Ye and I understand that, but again, this hasn't been an issue for ages, I will go over all mounts but all of them are being used correctly from what I could see, since I'd see data dissapearing into .img instead of my libraries and have messed up cases in the past that'd cause this. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.