Siwat2545 Posted January 7, 2019 Share Posted January 7, 2019 So This happen like 2 month ago, sometime, all the docker container will halt (It just stop doing whatever it's doing and freeze) and after a while, it will start working again. One time that this happen, I was running apt-get in one of the container and apt-get freeze but it continue extracting the packages after it unfreeze Usually this isn't a problem for me, but recently, I migrated my Home Assistant Server From vSphere to unRaid and I realize that this is and huge issue when I wake up at the middle of the night and I can't turn the lights on or when I'm trying to drive to work, and my front door won't open Tower Diagnostic Attached tower-diagnostics-20190107-2052.zip Quote Link to comment
earthworm Posted January 7, 2019 Share Posted January 7, 2019 I'm experiencing this issue as well. I will add that while all the dockers are unresponsive, all my drives are spun up. Quote Link to comment
Siwat2545 Posted January 8, 2019 Author Share Posted January 8, 2019 8 hours ago, earthworm said: I'm experiencing this issue as well. I will add that while all the dockers are unresponsive, all my drives are spun up. I tried recreating the dockermachine img, haven't freeze so far. PS. My drive are all spun up and VMs are working as well. Quote Link to comment
Siwat2545 Posted January 8, 2019 Author Share Posted January 8, 2019 UPDATE: It's freezing again!!! Quote Link to comment
Siwat2545 Posted January 8, 2019 Author Share Posted January 8, 2019 (edited) Another UPDATE: All of my cpu and memmory usage seems to be at almost 100% when it froze and go down to 60% when it stop freezing Edited January 8, 2019 by Siwat2545 Quote Link to comment
bonienl Posted January 8, 2019 Share Posted January 8, 2019 A memory usage of 99% means overcommitting the system. What happens if you let only a few docker containers run? Quote Link to comment
earthworm Posted January 8, 2019 Share Posted January 8, 2019 Do you think it's one of your VM's (eg. "Siwat PC") hogging the system? Looking though my own syslog I found the following: Jan 7 09:54:55 StorageMain kernel: Out of memory: Kill process 8878 (mono) score 456 or sacrifice child Jan 7 09:54:55 StorageMain kernel: Killed process 8878 (mono) total-vm:7063192kB, anon-rss:3708676kB, file-rss:0kB, shmem-rss:4kB Jan 7 09:54:56 StorageMain kernel: oom_reaper: reaped process 8878 (mono), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB I'm thinking maybe one of my dockers is misbehaving. Quote Link to comment
Siwat2545 Posted January 9, 2019 Author Share Posted January 9, 2019 Interesting, I've assigned a sum of 30GB to VMs and all of the leftover for docker, I will now try running only one docker, will get back to you, PS. The freezing happen more often after running "docker build" command Quote Link to comment
Siwat2545 Posted January 9, 2019 Author Share Posted January 9, 2019 (edited) 7 hours ago, earthworm said: Do you think it's one of your VM's (eg. "Siwat PC") hogging the system? Looking though my own syslog I found the following: Jan 7 09:54:55 StorageMain kernel: Out of memory: Kill process 8878 (mono) score 456 or sacrifice child Jan 7 09:54:55 StorageMain kernel: Killed process 8878 (mono) total-vm:7063192kB, anon-rss:3708676kB, file-rss:0kB, shmem-rss:4kB Jan 7 09:54:56 StorageMain kernel: oom_reaper: reaped process 8878 (mono), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB I'm thinking maybe one of my dockers is misbehaving. "Siwat PC" CPU Usage is at 16% when the docker freeze and stay at 6% even when the unraid dashboard says it is at 100% Edited January 9, 2019 by Siwat2545 Quote Link to comment
Siwat2545 Posted January 9, 2019 Author Share Posted January 9, 2019 (edited) I will try to screenshot the "Processes" page when it freeze, is there a way to view the docker cpu usage? Edited January 9, 2019 by Siwat2545 Quote Link to comment
Siwat2545 Posted January 9, 2019 Author Share Posted January 9, 2019 (edited) I ran top command and see user 998 taking up a lot of the memmory and with pid, I pin it down to gitlab and when I stop it, my memmory usage went down a ton(top, mem graph attached), will see if it freeze again( when I stop it, my home assistant automations fire a lot quicker, and my webserver is more responsive, plex media server decode in an instant) Here's the after Edited January 9, 2019 by Siwat2545 Quote Link to comment
Siwat2545 Posted January 13, 2019 Author Share Posted January 13, 2019 It's freezing Again!!! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.