Maximilian-Tek

Okay kinda a weird problem I do have but I am stuck. I used this plugin for the last years but have since ~6 Months the same bug, and could narrow it down. The Backup is scheduled every Tuesday at 3:00 AM, and every Tuesday morning my server crashes and I have to kill the power to the server to reboot. I am running a weekly backup of my appdata and I am hard resetting the power connection to the server every week (that des hurt emotionally). The backups are always complete and fine. I tested a manual backup which I monitored which went flawless, as always, the output showed me ("Done.") Something I noted, was that my RAM (32GB) was at 75-90 % when it is usually around 30 %, so I added a SWAP file via the SwapFile Plugin with around 20GB and went on my way, thinking that the problem was fixed. Nope, the following weeks the Server was still unresponsive (NO Gui via HDMI, NO WebGUI, NO Response of my Docker Containers) Today I tested it again manual backup, that finished well, all Containers started back up and I could use them for like 15min? and after that the RAM was againg idling at 75-90 % and suddenly the WebGUI was unresponsive and I couldn't reach any of the Docker Container. If I do the reboot via a power plugout and start the server, everything is running fine for the rest of the week until next Tuesday morning... So anyone does have an idea why that keeps happening? I will attach logfiles maybe someone can help me. Edit: My reboot process looks like the following: Server is unresponsive -> Power plug reboot -> Then because the Docker Service will always fail to start after this kind of reboot (idk why) I do a manual reboot via the WebGUI and voila everything runs again - this is a workaround but not a nice one tower-diagnostics-20240719-0119.zip

Edit: I dont know if this is the solution, but maybe my last paragraph here has the possible solution (I didn't find the error, just updated all my outstanding containers) Okay i had some time and tried my best, but this are the results (cannot find the cause) 1. Started the server 2. Straight the Settings and stopped the autostart of the general Docker Service 3. Waited some time, but Ram was idling at 5-10% 4. Started up the Docker Service, and immideatily disabled the autostart for most of the apps 4.1. Said apps went on and server was running at 20 %, until suddenly the RAM got up to 50,60,70,80 4.2. Started to power down some apps - still at 80 % (seemed to only slow down the process of the RAM 'growing') 4.3. Powered down all Docker Container by Stop All in the Docker View 4.4 RAM still at 97 % which does mean 30 GB of RAM (with no active apps (triple checked)) 5. Waited some time ~5min RAM was +-1% at 96-98 % 5.1. captured the following line in the process overview in the settings: root 21802 171 91.0 33754864 29626460 ? Sl 21:50 58:13 /usr/bin/dockerd -p /var/run/dockerd.pid --log-opt max-size=50m --log-opt max-file=1 --log-level=fatal --storage-driver=btrfs the 91.0 is the RAM value for this service 6. Turned of the general Docker Service 6.1. RAM dropped immediatly to 4-5 % Okay so next thing to try: I am notorius to just leave my containers on the current version and forget about updating them. Yes I know that has to change. Just updated everything and now tried to start everything one on one. Start the Docker Service and start up one container after each other with time breaks of around 5min between each! --- author's note: will send this posting when I am done doing this process - all in all it took around 3 hours---- Docker Service is started but all containers are set to do not auto start Waited 10 min and set everything one after one other to start up. Now all my Docker Containers are started up, each operating wth a reasanable amount of RAM, if something will break again I will report it back here. Possible Solution: Didn't update my containers and something there did break... Or I did something accidental that fixed the problem. Will mark the comment from JorgeB to be the solution, because impatience was not helping, and the slow but steady process was. Thank you!

will try this, thank you! but propably not before the weekend, will come back and report

Here I am again I did fix the error you mentionend, thanks! But I do run out of memory relatively fast after a reboot. Server Crashed is the situation 1. Cold Reboot (wait 1 min) 2. Open Web GUI - Have to reboot, because Docker Service does fail to start 3. Restart 4. Wait 1 min 5. Server is back online, Docker Service's start up 6. RAM (32 GB) is at 30-40 % 7. All Services started up (minimal start up, unnecessary services were stopped and cut of autostart, for the time until a fix is found) 8. wait 5 min 9. RAM starts to go up to 80,87,95 % 9.1. meanwhile - I do monitor my Container and each of them operates in a good amount of RAM (all together less than 10 GB, viewed in the Docker Page) 9.2. but under the tab for the processes, it is shown that almost all of the RAM that is used is allocated to the dockerd service (around 77% of all memory, odd because auf 9.2.) 10. Nevertheless the WEB GUI portraits 99% RAM Allocation, and at this point the server is unresponsive I plan to create a swap file (Plugin) on an extra SSD which will not be in a RAID, like my Cache Pool - for that I have ordered an extra cable (run out of enough power cables for SATA Connection (lol)) - maybe this will help, but if you would like to look into it, I couldn't find a solution in the diagnostics (Snapshot at rougly 85% RAM Allocation) be my guest. I do think that this swap file might be not enough (32 GB of RAM where only 10 GB are in active use by the Docker Services, the remaining 20 are usually enough for the overhead - I do not have any VM's in use) This issue was reproducable arround 3 times, after which I am writing this Thank you again! tower-diagnostics-20240425-0409.zip

Did change it to ipvlan, i guess it worked - if otherwise i will mention it here @itimpi thank you very much!

Hello there, I do use this server since 2 years and could troubleshoot most of the problems by myself, until now. I might have traced back the error to a memory leak, but there ends it. My server does crash, that I have to A) Do a cold reboot by flicking the power switch B) Wait and hope that the server does get back up on its feet, so that I can access the WebUI, but by then the Docker App crashed and I have to reboot. I can not access the Shares via SMB in the local network, and even the Web UI Page is no longer accessible if the server crashes. I do have my diagnostics attached, and would be glad if someone could help me with identifying the problem. Kind Regards Max tower-diagnostics-20240423-2145.zip