I noticed this am that my dockers were not started saw this error when attempting to start:

Mar 21 11:19:32 unRAID emhttpd: Starting services...
Mar 21 11:19:32 unRAID emhttpd: shcmd (1053): /usr/local/sbin/mount_image '/mnt/cache/docker1.img' /var/lib/docker 40
Mar 21 11:19:32 unRAID root: Creating new image file: /mnt/cache/docker1.img size: 40G
Mar 21 11:19:32 unRAID kernel: BTRFS critical (device sdn1): corrupt leaf: root=2 block=27387691008 slot=140, bad key order, prev (18446612132635222168 169 0) current (27385397248 169 0)
Mar 21 11:19:32 unRAID kernel: BTRFS critical (device sdn1): corrupt leaf: root=2 block=27387691008 slot=140, bad key order, prev (18446612132635222168 169 0) current (27385397248 169 0)
Mar 21 11:19:32 unRAID kernel: BTRFS: error (device sdn1) in __btrfs_free_extent:6953: errno=-5 IO failure
Mar 21 11:19:32 unRAID kernel: BTRFS info (device sdn1): forced readonly
Mar 21 11:19:32 unRAID kernel: BTRFS: error (device sdn1) in btrfs_run_delayed_refs:3058: errno=-5 IO failure
Mar 21 11:19:32 unRAID kernel: BTRFS warning (device sdn1): block group 29013049344 has wrong amount of free space
Mar 21 11:19:32 unRAID kernel: BTRFS warning (device sdn1): failed to load free space cache for block group 29013049344, rebuilding it now
Mar 21 11:19:33 unRAID root: failed to create image file
Mar 21 11:19:33 unRAID emhttpd: shcmd (1053): exit status: 1
Mar 21 11:19:33 unRAID emhttpd: shcmd (1066): /usr/local/sbin/mount_image '/mnt/user/system/libvirt/libvirt.img' /etc/libvirt 1
Mar 21 11:19:33 unRAID root: truncate: cannot open '/mnt/cache/system/libvirt/libvirt.img' for writing: Read-only file system
Mar 21 11:19:33 unRAID kernel: BTRFS info (device loop2): disk space caching is enabled
Mar 21 11:19:33 unRAID kernel: BTRFS info (device loop2): has skinny extents
Mar 21 11:19:33 unRAID root: ERROR: unable to resize '/etc/libvirt': Read-only file system

 

Ran a check filesystem on the cache drive and got this:

 

Fixed 0 roots. checking extents bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad block 27387691008 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache there is no free space entry for 27385380864-27386052608 there is no free space entry for 27385380864-27939307520 cache appears valid but isn't 26865565696 block group 29013049344 has wrong amount of free space, free space cache has 50647040 block group has 51695616 failed to load free space cache for block group 29013049344 ERROR: errors found in free space cache enabling repair mode Checking filesystem on /dev/sdn1 UUID: 17ddcee2-fac4-4e72-a4c5-54dddb7b8e94 found 135691198464 bytes used, error(s) found total csum bytes: 0 total tree bytes: 71122944 total fs tree bytes: 0 total extent tree bytes: 70959104 btree space waste bytes: 15872936 file data blocks allocated: 48234496 referenced 48234496

 

 

Not really sure what I should do from here out?  any advice would be lovely!

Nest time please post the complete diagnostics but based on that you need to backup, re-format and restore the data on cache, see here if you need more help with the recovery:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=543490

 

what part of which log did I miss?  Ill add it

When asking for help you should download the diagnostics zip from Tools -> Diagnostics and attach it to the post.

Ok great.  When I get home I'll attach them. Thanks for the tips

Here are my diag zip, please advice and thank you in advance!

unraid-diagnostics-20190321-1507.zip

Diags confirm metadata corruption on the cache filesystem, advice is the same as posted above.

Hey just an update!  After many hours of poking and proding, I was able to follow all the steps and restored my cache drive!!  Thanks again "That's what I do" 😃