Docker containers corrupt once daily

[chris_b_chicken]

Hey guys, I'm at kind of a loss as to what's going on with my docker containers (and VMs for that matter) so I thought I would reach out and see if anyone has any thoughts on the subject.

 

I first noticed this when the Fix Common Problems plugin told me it couldn't write to the docker.img file, then when I looked into the system logs, I get a lot of BTRFS file system errors that results in the dockers going down and being generally unusable. This also shuts down my VM. At first, I thought it was a weird fluke, so I deleted docker.img, recreated it, downloaded all of my containers again, and then everything was fine for a day. I've also tried reformatting my cache drive to XFS and then back to BTRFS, as I (like most) use my cache SSD to run my VMs, dockers, and anything written to my other shares (temporarily for a day until it's offloaded to the array around 1 AM every night). This, again, worked for a day, and then doesn't now. And now I'm truly baffled as to why I'm still getting BTRFS errors as my docker.img file is currently formatted as an XFS vdisk.

 

However it seems like every day this corruption sets in, so that I either get the error from Fix Common Problems, or I go to access one of my dockers, find it's unavailable, and then I look at the log and see the BTRFS errors again. I don't lose any docker container data in /mnt/user (or cache)/appdata, and my VM's vdisk is still intact, so I don't know what's going on. I'm starting to think it might be the SSD itself getting on in life (I've had it for a few years now), but the SMART reports are all coming back OK. So this is completely new territory for me.

 

Anyway, I've posted my diagnostics, if there's anyone out there who knows a bit more than me on this topic, I would appreciate the help, even if it means I have to buy a new SSD. I'm sure I've left out a few important details, so if you have any questions about my setup, let me know.

unraid-diagnostics-20210319-1433.zip

Edited March 19, 2021 by chris_b_chicken

[trurl]

Corruption on btrfs cache

[chris_b_chicken]

OK that's what it looked like to me, would this go away with a reformat of the cache drive or are there hardware problems too?

[JorgeB]

It would go away by reformatting cache, but if it's recurring it suggests an underlying problem, unlikely to be the SSD, are there any regular server crashes? I did see a macvlan call trace on the log.