1activegeek Posted February 13, 2020 Share Posted February 13, 2020 (edited) Not sure what happened last night but something went wonky in the middle of the night. I got notification that my services weren't responding then were, but my site pings were failing. Checked this AM, nothing seems to be up, and docker daemon not responding. Restarted the host and now it seems the cache is in a RO mounted state and docker won't start (obviously). Diagnostics posted below - any help would be greatly appreciated to get this back up and running. atlantis-diagnostics-20200213-0955.zip Edited March 1, 2020 by 1activegeek adding solved notation Quote Link to comment
1activegeek Posted February 13, 2020 Author Share Posted February 13, 2020 Just in case, this is also the diagnostics I grabbed after running myself manually after the boot with RO mode atlantis-diagnostics-20200213-1014.zip Quote Link to comment
JorgeB Posted February 13, 2020 Share Posted February 13, 2020 Docker image is corrupt, delete and re-create, cache filesystem looks OK for now. P.S. there were several sync errors during last parity check, were those expected? Quote Link to comment
JorgeB Posted February 13, 2020 Share Posted February 13, 2020 Forgot to mention, there are several ATA errors on the parity disk, looks like a connection issue, check/replace cables. Quote Link to comment
1activegeek Posted February 14, 2020 Author Share Posted February 14, 2020 I feel really blind here. I thought that would be the "quick" resolution. Unfortunately having deleted the docker image, restarted multiple times, unmount and mounting again - I still get the same message that the docker service failed to start. Updated diagnostics attached. PS - is there something in particular I can start searching for if running into these issue? Perhaps a FAQ around troubleshooting for unRAID as a general rule? Hate relying on volunteers to problem solve. atlantis-diagnostics-20200213-2045.zip Quote Link to comment
1activegeek Posted February 14, 2020 Author Share Posted February 14, 2020 It still appears to be mounting as a read only filesystem, thus the delete of the docker.img doesn't seem to be working. I checked locally via an ssh session to the server after attempting to delete the docker.img and it's still there. Manual attempt and it tell me it is a RO filesystem. Quote Link to comment
JorgeB Posted February 14, 2020 Share Posted February 14, 2020 I missed it earlier but there's also corruption on the cache filesystem, best way forward is to backup and reformat cache, then recreate docker image once more. Quote Link to comment
1activegeek Posted February 15, 2020 Author Share Posted February 15, 2020 Thanks for the direction. Took a bit of work to make my way through it, but did get the disk wiped and reformatted. When I went to delete the docker image (from the backup I made of the cache and put back) there was a notice that the docker image was corrupted from something in a previous Beta. I should have screenshot the message. Deleted and now in the process of rebuilding my docker setup. Unfortunately somehow one of my custom networks was deleted so the containers defaulted to none for network. 🙁 But thank you again for the swift responses and assistance. I'm almost back up and running. Quote Link to comment
Squid Posted February 15, 2020 Share Posted February 15, 2020 4 minutes ago, 1activegeek said: process of rebuilding my docker setup. Since you said this the way you did, you're probably doing it the hard way. Easy way is here https://forums.unraid.net/topic/57181-docker-faq/#comment-564309 6 minutes ago, 1activegeek said: Unfortunately somehow one of my custom networks was deleted so the containers defaulted to none for network That happens because the network information is stored within the docker.img. 7 minutes ago, 1activegeek said: a notice that the docker image was corrupted from something in a previous Beta. Known "issue" from moving around the docker.img file. As you've already noticed, it's easiest to simply recreate it, and then reinstall everything. Quote Link to comment
1activegeek Posted February 15, 2020 Author Share Posted February 15, 2020 @Squid The only thing I’m confused about though, is that my custom macvlan network survived, but the custom bridge did not. So that inconsistency threw me. As for recreating, I did go that route - prob is the custom bridge wasn’t there and thus many of the containers didn’t have the right setup. So consequently the easiest option was Removing them all (manual work) and then re-create again with previous after I brought over backups of my container xml templates from a backup. Just lot of steps to get it right. Appreciate the guidance though, thank you! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.