/var/log is getting full (currently 100 % used)

[JNP]

Hi,

 

var/log/docker.log is maxing out after about a week of server up time. Diagnostics attached.

 

Fix Common Problems alerted me to the issue. Have uninstalled Nerd Tools as recommended and issue was not solved. I'm fairly new to unraid any unsure how best to troubleshoot this issue.

 

Thanks for any support

 

Edited September 10, 2022 by JNP

[JorgeB]

Please post output of:

 

ls -lh /var/log

 

[JNP]

3 hours ago, JorgeB said:

Please post output of:

 

ls -lh /var/log

 

 

total 128M
-rw------- 1 root   root    0 Apr  8  2021 btmp
-rw-r--r-- 1 root   root    0 Apr 10  2020 cron
-rw-r--r-- 1 root   root    0 Apr 10  2020 debug
-rw-rw-rw- 1 root   root  54K Sep  1 21:26 dmesg
-rw-rw-rw- 1 root   root    0 Sep  3 04:40 docker.log
-rw-rw-rw- 1 root   root 128M Sep  9 15:13 docker.log.1
-rw-r--r-- 1 root   root    0 Jun 17  2020 faillog
-rw-r--r-- 1 root   root    0 Apr  8  2000 lastlog
drwxr-xr-x 3 root   root   60 Jul 17  2020 libvirt/
-rw-r--r-- 1 root   root    0 Apr 10  2020 maillog
-rw-r--r-- 1 root   root    0 Apr 10  2020 messages
drwxr-xr-x 2 root   root   40 May 16  2001 nfsd/
drwxr-x--- 2 nobody root   60 Sep  1 21:27 nginx/
lrwxrwxrwx 1 root   root   24 Apr  8  2021 packages -> ../lib/pkgtools/packages/
drwxr-xr-x 4 root   root   80 May 15  2020 pkgtools/
drwxr-xr-x 2 root   root  380 Sep  1 21:26 plugins/
drwxr-xr-x 2 root   root   40 Sep  1 21:27 pwfail/
lrwxrwxrwx 1 root   root   25 Apr  8  2021 removed_packages -> pkgtools/removed_packages/
lrwxrwxrwx 1 root   root   24 Apr  8  2021 removed_scripts -> pkgtools/removed_scripts/
drwxr-xr-x 3 root   root  180 Sep  1 21:30 samba/
lrwxrwxrwx 1 root   root   23 Apr  8  2021 scripts -> ../lib/pkgtools/scripts/
-rw-r--r-- 1 root   root    0 Apr 10  2020 secure
lrwxrwxrwx 1 root   root   21 Apr  8  2021 setup -> ../lib/pkgtools/setup/
-rw-r--r-- 1 root   root    0 Apr 10  2020 spooler
drwxr-xr-x 3 root   root   60 Jul 17  2020 swtpm/
-rw-r--r-- 1 root   root 244K Sep  9 21:38 syslog
-rw-rw-rw- 1 root   root    0 Sep  1 21:26 vfio-pci
-rw-rw-r-- 1 root   utmp 6.8K Sep  1 21:26 wtmp

[JorgeB]

8 minutes ago, JNP said:

128M Sep  9 15:13 docker.log.1

This is the problem.

[JNP]

Saw that too. Do you know how I can locate that file to check what is spamming it with 128MB worth of junk? Can't seem to find it in file navigation

[trurl]

It is in the OS files you would have to go to the command line to access it.

 

Create a new user share named "logs" then at the command line

cp /var/log/docker.log.1 /mnt/user/logs

Then you can get it from the logs user share, zip and post it.

 

[JNP]

Great! The log is indeed spam. Majority of line items read as below

 

time="2022-09-03T00:01:35.524583688+10:00" level=error msg="collecting stats for e0d0053c99d594c85c2b0343fa6733de08478c9d8ea936d6c482513fa72fa77f: Could not get container for ebc3948ae41f404b0e7e4fa1a7db21324f41d1a068a41428d6357242ff511136: No such container: ebc3948ae41f404b0e7e4fa1a7db21324f41d1a068a41428d6357242ff511136"

 

Edited September 9, 2022 by JNP

[trurl]

Do you know what caused it? You will have to reboot to get log cleared.