Docker Service failed to start.

[Adrian]

I recently updated to 6.12 and found my docker down. When I access the Docker tab I see the following message, Docker Service failed to start.

I noticed there was an update to 6.12.1 and upgraded thinking maybe it had a fix for this issue. Same error message.

 

It's been forever since I set all this up, so not even sure where to start to troubleshoot. I do see the drive listed under unassigned devices and if I mount it I can browse the contents and I see everything is there, including the docker image.

 

Thanks in advance,

Adrian

mediaserver-diagnostics-20230627-2051.zip

[JorgeB]

Jun 27 20:47:14 MEDIASERVER emhttpd: no mountpoint along path: /mnt/cache/docker

 

Looks like this path does not exist, you need to create or correct it.

[Adrian]

5 hours ago, JorgeB said:

Jun 27 20:47:14 MEDIASERVER emhttpd: no mountpoint along path: /mnt/cache/docker

 

Looks like this path does not exist, you need to create or correct it.

 

Yes I saw that, but it's been years since I set all this up and I don't know how to fix it. Do you do this under unassigned devices in the "Change Disk Mount Point" dialog that pops up when I click the drive? When I tried to change that to "cache" I get an error saying, "Jun 28 08:39:45 MEDIASERVER unassigned.devices: Error: Device '/dev/sdc1' mount point 'cache' - name is reserved, used in the array or a pool, or by an unassigned device."

 

It sounds like 6.12 introduced some breaking changes. I can name it something else and mount it, but now I see it as /mnt/disks/ssd so looks like "disks" got added to the mount path. This should let me fix the Docker vDisk location, but not the Default appdata storage location which I had on the SSD also. It's only listing the user shares on the array, so I can't set it to /mnt/disks/ssd/appdata.

 

Looks like other users are having issues after the upgrade.  

I reading up now to see if what solutions are already out there.

Edited June 28, 2023 by Adrian

[JorgeB]

Problem is that your cache pool is not mounting, so /mnt/cache does not exist, there's only one device assigned to the pool, and pool current consists of two devices, unassign the pool device, start array, stop array, assign both pool devices to the pool, start array, post new diags.

[Adrian]

39 minutes ago, JorgeB said:

Problem is that your cache pool is not mounting, so /mnt/cache does not exist, there's only one device assigned to the pool, and pool current consists of two devices, unassign the pool device, start array, stop array, assign both pool devices to the pool, start array, post new diags.

 

While the cache pool not mounting is a problem, the real problem is 6.12 which obviously has some changes that somehow is breaking it. I may have jumped the gun too fast and I'm not really ready to deal with working through the issues. I'll try it again when I have some more time so I can provide more info to help sort this out, but for now I've reverted back to 6.11.5 and everything is working again. I'll try to read up on the changes and post in the release forum. Maybe there is something I can change in preparation to upgrading to 6.12.

[JorgeB]

28 minutes ago, Adrian said:

the real problem is 6.12 which obviously has some changes that somehow is breaking it

Did you have both cache devices assigned before upgrading?

[Adrian]

1 hour ago, JorgeB said:

Did you have both cache devices assigned before upgrading?

 

It's been years since I set this up, but I've uploaded the latest logs where everything is working. If you need to know any of configuration setting values, let me know which ones specifically and I'll reply back with them.

It looks like I have 2 physical SSDs in my server. Not really sure what I was trying to do. I see one listed under Pool Devices and the other under Unassigned Devices. I think I was trying to have one for cache and one for docker.

 

Please note, there's a high possibility that I might not have had all this configured in a typical way.

mediaserver-diagnostics-20230628-1139.zip

[JorgeB]

OK, so this issue is actually a v6.11 bug, you only have one device assigned to the pool, but your pool consists of two devices:

 

             Data     Metadata  System                              
Id Path      RAID1    RAID1     RAID1    Unallocated Total     Slack
-- --------- -------- --------- -------- ----------- --------- -----
 1 /dev/sdb1 94.00GiB   2.00GiB 32.00MiB    23.21GiB 119.24GiB     -
 2 /dev/sdc1 94.00GiB   2.00GiB 32.00MiB   142.44GiB 238.47GiB     -

 

v6.11 doesn't check this and it still mounting the single device as a pool, v6.12 is detecting that the device is not a single pool a it doesn't mount it, the fix is what I posted above.

[JorgeB]

Also note that because one of the devices is smaller, and it's using raid1, pool can only use up to the smallest device size (128GB).

[Adrian]

35 minutes ago, JorgeB said:

Also note that because one of the devices is smaller, and it's using raid1, pool can only use up to the smallest device size (128GB).

 

Ok, so if I wanted to just fix everything and get this setup correctly, could I simply stop docker, then copy all the contents from /root/media/cache, that's where I see the appdata, docker, and some other folders that are on the SSD to a backup folder. Then replace the 128GB drive with another 256GB drive and just recreate the raid1 cache and copy the folders back?

[JorgeB]

9 minutes ago, Adrian said:

/root/media/cache

Not sure what you mean by this path, but if it's on cache no need to copy anything, first do what I mentioned above to correct the current pool, once you have both devices assigned you can upgrade the smaller device, with v6.11 or v6.12.

 

 

[Adrian]

Ok, I swapped out the 128GB drive with a matching 256GB drive, added the missing drive to the pool and started it up. Looks like everything is good now, but please confirm.

 

Thank you,

Adrian

mediaserver-diagnostics-20230629-1249.zip

[JorgeB]

Looks good, if you now upgrade to v6.12 pool should continue to work normally.