All dockers stopped suddenly

[spacezmonkey]

Hello, from time to time ( once after a few week ) , I am experiencing the following problem.

 

All my dockers are turn off overnight and i have to switch on all of them in the next morning. 

 

I have attached diagnosis file.  May I know what is wrong with my server?

homenas-diagnostics-20220118-0842.zip

[itimpi]

I suspect this is happening when CA Backup runs and stops all containers, but then fails in restarting them all afterwards.   It should be easy enough to check if this problem co-insides with when you have configured that to run.

[spacezmonkey]

Today almost all of my dockers went off again. At first i thought it is due to one of my harddisk storage space. ( one of unassigned harddisk that i use for CCTV record is full ). I remove all my CCTV records and wait for a few days. And it happens again. 

 

after reading this post, i check my CA backup.

Quote

I suspect this is happening when CA Backup runs and stops all containers, but then fails in restarting them all afterwards.   It should be easy enough to check if this problem co-insides with when you have configured that to run.

i have scheduled it to run at 3am daily and now is 10am and when i press on status, it show it is still running.   I think i will stop using CA backup. 

 

 

 

[trurl]

3 minutes ago, spacezmonkey said:

I think i will stop using CA backup. 

Better would be to figure out why it doesn't finish. How large is your appdata?

[MrGrey]

I agree with trurl and itimpi. This reminds me of a Plex (mis)configuration that had movies/tvshows being saved in "appdata" instead of the "media" share they were supposed to be in.

You'll find the solution/reason. Take your time.

[spacezmonkey]

Hello , is it because of the following issue. 

 

I have this problem when my cache disk is almost full. That is when i tried to change the " Use cache pool (for new files/directories) "  to yes and activate mover. I tried to move the appdata folder to array. 

 

After a day, i realised that i have to use cache pool as " only"  instead of "yes". So i change the setting again. 

 

Not sure if these will affect dockers.

 

currently my setup is as following

 

 

and disk space use is 

Edited January 21, 2022 by spacezmonkey

[itimpi]

5 hours ago, spacezmonkey said:

After a day, i realised that i have to use cache pool as " only"  instead of "yes". So i change the setting again. 

You do realise that any files for the appdata share that are on the array will be left there as mover ignores any shares with the Only setting?    If you want files to be moved from array to the cache drive you need the Prefer setting.

[trurl]

32 minutes ago, itimpi said:

You do realise that any files for the appdata share that are on the array will be left there as mover ignores any shares with the Only setting?    If you want files to be moved from array to the cache drive you need the Prefer setting.

And nothing can move open files, so you need to disable Docker in Settings.

 

Your earlier diagnostics showed appdata, system (and domains) shares with files on the array. Ideally, these would have all files on fast storage (cache or other pool) and configured to stay there so your dockers/VMs won't have performance impacted by slower array, and won't keep array disks spunup since these files are always open.