For the last couple weeks about twice per week I have had to reboot my server because the /var/lib/docker/tmp/ folder shows as write only.  After a reboot it fixes the issue and i can then start using my docker containers again.  This has been annoying to say the least, but I have survived.  Today it happened again and I rebooted. Now the server is showing no docker containers and my cache drive is empty. Smart is still showing my cache drive as active and healthy. I don't know what happened or how to fix it. I was running around 25 containers some of them were DB's.  Is there a way to also recover?

Go to Tools - Diagnostics and attach the complete Diagnostics ZIP file to your NEXT post in this thread.

Here is the diag report

ds-unraid-diagnostics-20201027-1406.zip

20 hours ago, twistedjackal509 said:

my cache drive is empty.

No, your cache drive is unmountable. Totally different thing than an empty drive.

 

And since cache is corrupt and unmountable, all the things associated with dockers and VMs were recreated from scratch on disk10.

 

You will want to get rid of all that and see if you can get cache going again. Go to Settings - Docker and disable then delete docker.img. Do the same for Settings - VM Manager. Leave them disabled until we see if cache can be recovered.

 

Go to Unraid command line and post output of this

lsblk

 

root@DS-Unraid:~# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0         7:0    0  22.7M  1 loop /lib/modules
loop1         7:1    0   7.1M  1 loop /lib/firmware
loop2         7:2    0    40G  0 loop /var/lib/docker
loop3         7:3    0     1G  0 loop /etc/libvirt
sda           8:0    1  14.6G  0 disk 
└─sda1        8:1    1  14.6G  0 part /boot
sdb           8:16   1  14.6G  0 disk 
├─sdb1        8:17   1   800K  0 part 
├─sdb2        8:18   1 766.9M  0 part 
└─sdb3        8:19   1    36M  0 part 
sdc           8:32   0   3.7T  0 disk 
└─sdc1        8:33   0   3.7T  0 part 
sdd           8:48   0   9.1T  0 disk 
sde           8:64   0   3.7T  0 disk 
└─sde1        8:65   0   3.7T  0 part 
sdf           8:80   0   3.7T  0 disk 
└─sdf1        8:81   0   3.7T  0 part 
sdg           8:96   0   3.7T  0 disk 
└─sdg1        8:97   0   3.7T  0 part 
sdh           8:112  0   3.7T  0 disk 
└─sdh1        8:113  0   3.7T  0 part 
sdi           8:128  0   3.7T  0 disk 
└─sdi1        8:129  0   3.7T  0 part 
sdj           8:144  0  10.9T  0 disk 
└─sdj1        8:145  0  10.9T  0 part 
sdk           8:160  0   3.7T  0 disk 
└─sdk1        8:161  0   3.7T  0 part 
sdl           8:176  0   3.7T  0 disk 
└─sdl1        8:177  0   3.7T  0 part 
sdm           8:192  0   3.7T  0 disk 
└─sdm1        8:193  0   3.7T  0 part 
sdn           8:208  0  10.9T  0 disk 
└─sdn1        8:209  0  10.9T  0 part 
sdo           8:224  0   3.7T  0 disk 
└─sdo1        8:225  0   3.7T  0 part 
sdp           8:240  0 931.5G  0 disk 
└─sdp1        8:241  0 931.5G  0 part 
md1           9:1    0   3.7T  0 md   /mnt/disk1
md2           9:2    0   3.7T  0 md   /mnt/disk2
md3           9:3    0   3.7T  0 md   /mnt/disk3
md4           9:4    0   3.7T  0 md   /mnt/disk4
md5           9:5    0   3.7T  0 md   /mnt/disk5
md6           9:6    0   3.7T  0 md   /mnt/disk6
md7           9:7    0   3.7T  0 md   /mnt/disk7
md8           9:8    0   3.7T  0 md   /mnt/disk8
md9           9:9    0   3.7T  0 md   /mnt/disk9
md10          9:10   0   3.7T  0 md   /mnt/disk10
nvme1n1     259:0    0 465.8G  0 disk 
└─nvme1n1p1 259:1    0 465.8G  0 part 

Doesn't look like you have CA Backup plugin installed, so I am assuming you don't have backups of appdata or libvirt.

 

12 hours ago, twistedjackal509 said:

nvme1n1p1

Just wanted to confirm the partition for your cache, since this FAQ assumes sdX1 and that isn't what you have with that nvme: