Mikael
-
Posts
8 -
Joined
-
Last visited
Mikael's Achievements
Noob (1/14)
0
Reputation
-
I am indeed using Arq. It's the windows based agent that seems to cause the highest memory consumption. It is also the largest backup, so the Windows part may not be a significant factor.
-
Hi @springsman I didn't find a solution to this problem. I ended up not backing up one of my computers, since that one was responsible for the most rapidly increasing memory usage. Probably because the backup size was quite large.
-
Hi @johnnie.black, it only happens with dockers running. As far as I can tell only with the minio docker. That's the only one that is able to produce over 40 GB of memory usage for shfs in a couple hours.
-
Anyone have any other suggestions? The problem is still here. I am able to reproduce the issue when validating backups as well, which presumably doesn't write to the server, only read.
-
I recreated the docker image with a lower size and moved it and the mounted paths to unRAID shares. The shfs memory usage still grows, although possibly a bit slower, when backing up to the mino docker container. When stopping the backup, the memory usage drops quite a bit now, which I don't remember happening previously. I monitored the size used by the docker containers, and there was no growth there. The high shfs memory usage produced yesterday, still remained today, even after shutting down the docker and vm services.
-
Hi @trurl, it does not seem to happen when I turn off the docker containers that use the unRAID shares. The 100G docker image was just because I could. Only around 3G is in use. Prior to testing for this issue, all docker container paths were mounted to unRAID shares. The docker image was probably still on the unassigned device. I can lower the size of the docker container image and move it to the unRAID shares again tomorrow, to see if that helps.
-
During the last two weeks or so, my unRAID server has started to shut down all shares due to shfs using too much memory. Here is a snippet from an earlier log where shfs ended up using so much memory that it was shut down: My quickest way of reproducing it so far has been to turn on my minio docker container and run backups to it from other machines. That way shfs builds its memory usage with several gigabytes in a matter of minutes. I've uninstalled all plugins but Unassigned Devices, which I need for my VM's, in an attempt to see if that was the cause. The attached diagnostics is from today, after I uninstalled all plugins, and limited the amount of running VMs and docker containers. It hasn't crashed yet in this log but it is only a matter of time, seeing as shfs was using 20 GB of memory and growing. By disabling most of my docker containers and turning off some VMs, I am able to run the system for a few days before the shares are cut off, but I really hope to find a fix rather quickly. nas-diagnostics-20200322-1242.zip