shaunbarrow Posted March 21, 2019 Share Posted March 21, 2019 I noticed this am that my dockers were not started saw this error when attempting to start: Mar 21 11:19:32 unRAID emhttpd: Starting services... Mar 21 11:19:32 unRAID emhttpd: shcmd (1053): /usr/local/sbin/mount_image '/mnt/cache/docker1.img' /var/lib/docker 40 Mar 21 11:19:32 unRAID root: Creating new image file: /mnt/cache/docker1.img size: 40G Mar 21 11:19:32 unRAID kernel: BTRFS critical (device sdn1): corrupt leaf: root=2 block=27387691008 slot=140, bad key order, prev (18446612132635222168 169 0) current (27385397248 169 0) Mar 21 11:19:32 unRAID kernel: BTRFS critical (device sdn1): corrupt leaf: root=2 block=27387691008 slot=140, bad key order, prev (18446612132635222168 169 0) current (27385397248 169 0) Mar 21 11:19:32 unRAID kernel: BTRFS: error (device sdn1) in __btrfs_free_extent:6953: errno=-5 IO failure Mar 21 11:19:32 unRAID kernel: BTRFS info (device sdn1): forced readonly Mar 21 11:19:32 unRAID kernel: BTRFS: error (device sdn1) in btrfs_run_delayed_refs:3058: errno=-5 IO failure Mar 21 11:19:32 unRAID kernel: BTRFS warning (device sdn1): block group 29013049344 has wrong amount of free space Mar 21 11:19:32 unRAID kernel: BTRFS warning (device sdn1): failed to load free space cache for block group 29013049344, rebuilding it now Mar 21 11:19:33 unRAID root: failed to create image file Mar 21 11:19:33 unRAID emhttpd: shcmd (1053): exit status: 1 Mar 21 11:19:33 unRAID emhttpd: shcmd (1066): /usr/local/sbin/mount_image '/mnt/user/system/libvirt/libvirt.img' /etc/libvirt 1 Mar 21 11:19:33 unRAID root: truncate: cannot open '/mnt/cache/system/libvirt/libvirt.img' for writing: Read-only file system Mar 21 11:19:33 unRAID kernel: BTRFS info (device loop2): disk space caching is enabled Mar 21 11:19:33 unRAID kernel: BTRFS info (device loop2): has skinny extents Mar 21 11:19:33 unRAID root: ERROR: unable to resize '/etc/libvirt': Read-only file system Ran a check filesystem on the cache drive and got this: Fixed 0 roots. checking extents bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad key ordering 139 140 bad block 27387691008 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache there is no free space entry for 27385380864-27386052608 there is no free space entry for 27385380864-27939307520 cache appears valid but isn't 26865565696 block group 29013049344 has wrong amount of free space, free space cache has 50647040 block group has 51695616 failed to load free space cache for block group 29013049344 ERROR: errors found in free space cache enabling repair mode Checking filesystem on /dev/sdn1 UUID: 17ddcee2-fac4-4e72-a4c5-54dddb7b8e94 found 135691198464 bytes used, error(s) found total csum bytes: 0 total tree bytes: 71122944 total fs tree bytes: 0 total extent tree bytes: 70959104 btree space waste bytes: 15872936 file data blocks allocated: 48234496 referenced 48234496 Not really sure what I should do from here out? any advice would be lovely! Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 Nest time please post the complete diagnostics but based on that you need to backup, re-format and restore the data on cache, see here if you need more help with the recovery: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=543490 Quote Link to comment
shaunbarrow Posted March 21, 2019 Author Share Posted March 21, 2019 what part of which log did I miss? Ill add it Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 When asking for help you should download the diagnostics zip from Tools -> Diagnostics and attach it to the post. Quote Link to comment
shaunbarrow Posted March 21, 2019 Author Share Posted March 21, 2019 Ok great. When I get home I'll attach them. Thanks for the tips Quote Link to comment
shaunbarrow Posted March 21, 2019 Author Share Posted March 21, 2019 Here are my diag zip, please advice and thank you in advance! unraid-diagnostics-20190321-1507.zip Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 Diags confirm metadata corruption on the cache filesystem, advice is the same as posted above. Quote Link to comment
shaunbarrow Posted March 22, 2019 Author Share Posted March 22, 2019 Hey just an update! After many hours of poking and proding, I was able to follow all the steps and restored my cache drive!! Thanks again "That's what I do" 😃 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.