Docker Failed to Start After Update All - Array Won't Stop Now


Recommended Posts

Unraid 6.6.7

This has happened twice in as many weeks now.

  • I update a docker application or update all and the update completes fine 
  • I only have two docker applications, NZBGet and Plex
  • After clicking Done in the completed update window I get the "Docker Service Failed to Start" error
  • Both my NZBGet and Plex are functioning still though
  • I try to stop the array to do a clean reboot, and I keep getting stuck at "Array Stopping - Retry unmounting disk share(s)..."

 

First time this happened I performed a dirty shutdown and had to do a parity check, which ended up valid. I don't want to have to do a dirty shutdown again is my biggest issued here. Next is why the Docker service keeps failing to start. The last time I had to edit the config from "yes" to "no", reboot, delete docker image, recreate and reboot. I actually had to to this three times before I stopped getting the Docker service failed to start error.

 

Please help, I ran a diagnostic, do I just upload the zip it created?

 

Thanks in advance for anyone willing to help me.

 

Edit: tower-diagnostics-20190507-0942.zip

Edited by KR1SeS
Added diagnostics
Link to comment

The syslog shows that you are having trouble communicating with the SSD.    This is probably why the system is not shutting down as it cannot successfully unmount the cache drive.    The error looks as if it is probably related to the SATA cabling for that drive.  The drive has a large number of CRC errors and that type of error is nearly always cable related.

  • Like 1
Link to comment
On 5/7/2019 at 3:06 PM, itimpi said:

The syslog shows that you are having trouble communicating with the SSD.    This is probably why the system is not shutting down as it cannot successfully unmount the cache drive.    The error looks as if it is probably related to the SATA cabling for that drive.  The drive has a large number of CRC errors and that type of error is nearly always cable related.

Thanks for the reply. I've tried changing the SATA cable twice now and I get the CRC errors still. I've also tried different SATA ports on the motherboard and still get the same errors. Is it possible the drive is bad?

 

It's a Samsung SSD under a year old, do you think they'd honor warranty? I don't know how to prove to them it's bad though as it does run.

Link to comment

I'm still having issues. I've replaced the SSD with a mechanical drive and sent the SSD away for RMA.

 

With the mechanical drive installed and formatted, I deleted the docker.img and recreated it. I still got the Docker Service Failed To Start error. I again used the method @Squid detailed in other posts, and edited the docker config to tell the service not to run. I restart the server, delete the docker.img and recreate and I was back in business after installing Previous Apps.

Things worked well for a day or so, when I noticed Plex not responding. Went into the UI for UnRAID and on the docker tab I see I have the Docker Service Failed To Start again. I have no errors on my cache drive this time. I went through the steps again to recreate the docker image and sure enough it failed again within a few days. I attempted one last time last night to recreate it but this time it wouldn't even work using @Squid's steps.

Please help me. My server is basically unusable without having NZBGet and Plex docker containers running.

 

tower-diagnostics-20190516-0814.zip

Edited by KR1SeS
Link to comment

yes, bump. Array just won't stop here. How do I force it to stop Moving files around? It keeps saying in the left bottom corner;

Array Stopping•Retry unmounting disk share(s)...

I'm afraid to force reboot the machine because of possible data loss of some sort.

 

I noticed stuff freezes when I force the Mover to clear cache drives or when I've stopped the Array.

Just can't do anything, and many dockers fail to even work while I'm forced to just wait it out, with no ETA in sight anywhere.

I have no idea what it is doing. I constantly hear drives busy writing/reading.

 

OK, found out one disk was behaving faulty, which is weird, since I didn't get warnings about it beforehand, just found out by viewing the system logs.

Edited by fluisterben
Link to comment
I have the same problem. I think it happened when the log space was full. I couldn't update any dockers with an error message  "open /var/lib/docker/tmp/getimageblob 361152166: read-only file system"
 
746275636_Screenshot(209).png.d74a4388f5608a6d6770b58b9832e8d2.png
Post diagnostics your screen shot shows that the docker.img is read only, not that the log is full

Sent via telekinesis

Link to comment
1 hour ago, Squid said:

Post diagnostics your screen shot shows that the docker.img is read only, not that the log is full

Sent via telekinesis
 

The log wasn't full after I restarted. I was trying to stop the array, but it hasn't finished since a couple of hours. I'm only getting "Array Stopping•Retry unmounting disk share(s)..."

I added the diagnostics.

tower-diagnostics-20190526-1719.zip

Edited by Heciruam
Link to comment

60 Gig is insane for the size of a docker.img  Vast majority of users only require 20Gig Max (assuming that everything is configured correctly),

 

But either way, the docker.img is getting write errors.  Because the system is in the process of shutting down, its hard to tell exactly where its being stored, (whether on the cache or disk3), but either way your initial trouble shooting is to 

 

disable the docker service, delete the image file, re-enable the service, then hit Apps, Previous Apps, and check off everything you had installed.

 

  • Like 1
Link to comment
16 hours ago, Squid said:

Personally, I would update to 6.7.0  I had the odd problem throughout 6.6.x and early 6.7-rc's where containers wouldn't stop properly.  At some point in the middle of 6.7 rc the problems disappeared.

Thanks for the advice, I'll update tonight.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.