(Solved) Docker Read Only Filesystem errors

[Sousapro]

Hello,

 

I have a Unraid system that is constantly generating read only file system errors in docker after being online for a short period of time. troubleshooting steps so far:

 

1. checked 2x Cache SSD BTRFS with a scrub (no errors found)
2. removed and rebuilt docker.img
3. XFS_Repair on all XFS volumes in array maintenance mode
4. all SMART data is showing green

 

Logs are attached to post. I'm flexible with any needed troubleshooting but this one has kind of stumped me. this occurred after a drive (that had no data on it) started showing pre-fail and was replaced with cleared new drive. thanks for the help and looking at this post.

 

edit: I recreated docker one more time and made sure that I'm running my minimal set of images and that none of them are privileged at this time. going to run it like this for a few hours and see if the error recurs.

 

edit 2: failed within 15 minutes to read only file system.

unraid-diagnostics-20200330-1840.zip

Edited April 5, 2020 by Sousapro

[JorgeB]

Cache filesystem has issues, best bet is to backup all data, re-format the pool and restore the data.

[Sousapro]

Thanks @johnnie.black, this resolved it. I'm a little concerned about BTRFS Scrub not finding anything before but drive health is good and the array has behaved normally since rebuilding the cache drives.

[JorgeB]

14 hours ago, Sousapro said:

I'm a little concerned about BTRFS Scrub not finding anything before

Scrub only checks the data, not the filesystem consistency, you can check that with btrfs check (but never use repair mode unless you know what you're doing)