Docker fails to start

[khager]

I've searched the forum and can't find anyone who's had an issue like this (or the posts are several versions old).

I'm adding larger disks to my array. So far I've only replaced the parity drive. Then I noticed Docker wasn't running. I get this screen when I click the Docker tab

 

The Cache drive (Raid1) is up and available / share works / I can access files so I changed my cache settings from "Only" to "Yes" and ran mover. I tried creating a new docker image but that didn't help. Obviously I'm out of my league here.

 

I'd really like to get my docker apps back and running with the same settings and data I had before (one of them is Plex and I really don't want to rebuild that from scratch since I have all the data - plus backups).

 

Any help would be most appreciated. I've attached diagnostics.

 

Thanks

hunraid01-diagnostics-20220102-0611.zip

[Squid]

The docker image has issues probably due to the cache having some issues

Jan  2 05:52:51 HunRAID01 kernel: BTRFS info (device sdl1): bdev /dev/sdl1 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0

 

Not a BTRFS guy, so others (eg: @JorgeB) will help there.

 

I'd still recreate the image though after the cache issue is fixed up.

Also, why a 256G docker image?

[JorgeB]

Btrfs is detecting data corruption, start by running memtest.

[khager]

I've run Balance and Scrub but no dice.

 

16 minutes ago, JorgeB said:

Btrfs is detecting data corruption, start by running memtest.

is memtest a RAM tester? Can you provide the syntax? It's not a valid bash command on my server.

 

19 minutes ago, Squid said:

Also, why a 256G docker image?

I'm fine with a 128GB - I just thought it might have run out of space (forgot I did that).

 

Is it possible to reformat, recreate the docker image, and copy the apps/files back over to the cache disk?

[itimpi]

2 minutes ago, khager said:

s memtest a RAM tester? Can you provide the syntax? It's not a valid bash command on my server.

Yes it is a RAM tester.   To run it you select that option from the Unraid boot menu.

[Squid]

Just now, khager said:

is memtest a RAM tester? Can you provide the syntax? It's not a valid bash command on my server.

 

It's on the boot menu.  If it doesn't load, then temporarily switch your boot method to Legacy in the BIOS

1 minute ago, khager said:

I'm fine with a 128GB - I just thought it might have run out of space (forgot I did that).

 

20G is good enough for the vast majority of users

 

2 minutes ago, khager said:

Is it possible to reformat, recreate the docker image, and copy the apps/files back over to the cache disk?

Use Mover to get everything off of the cache drive (stop the docker service and set the appdata share to be useCache: yes

 

Alternatively, back up the appdata with the appdata backup plugin (destination on the array) and when ready, restore it back to the cache drive

[khager]

Just now, itimpi said:

To run it you select that option from the Unraid boot menu.

Gotcha. I'll have to dig out a monitor and keyboard - it'll be a minute.

[khager]

2 minutes ago, Squid said:

Use Mover to get everything off of the cache drive (stop the docker service and set the appdata share to be useCache: yes

 

Alternatively, back up the appdata with the appdata backup plugin (destination on the array) and when ready, restore it back to the cache drive

 

I've done both. How do I reformat the cache drive?

[khager]

memtest showed no errors

 

I didn't have to reformat my cache drive(s). I just deleted the old docker.img and created a new one.

Then I disabled docker, restored appdata from backup (using the plugin), enabled docker, and rebooted.

Now I get this on my docker tab:

 

How do I get my docker apps running again. Do I have to install them fresh (then maybe overwrite appdata again?)

 

[Squid]

Apps - Previous Apps and check the stuff off

[khager]

Thanks everyone for lending a hand on this.

 

Just closing the loop with this post. I got my dockers up and running. I finally understand this in my brain. The docker image ONLY contains the dockers as released. NO configuration/user data is there. ALL configuration/user data is in appdata. For a long time, that never sank in.

 

Once I understood this, it was an easy fix since I have appdata backed up. Create a docker image, tick off previous apps & install, restore appdata, Bob's your uncle - or, in the words of the late, great John Madden, "BOOM! It's over."

 

A few notes:

- The trick of setting cache on the shares to "Yes" and using Mover didn't work for me. It took forever and files were left behind. Setting it back to "Only" afterward didn't move the files back. I had to clean this up manually both directions. Using "CA Backup / Restore Appdata" to restore my backup was much cleaner and works a peach.

- I need a new motherboard & SATA3 controller. This is completely unrelated to my problem but now that I've recovered my dockers I'm on to expanding the array. 17-19 hours to reconstruct a disk is awful.

- That new motherboard needs a couple M.2 slots for cache drives. 

 

Thanks again.

[ChatNoir]

4 minutes ago, khager said:

The trick of setting cache on the shares to "Yes" and using Mover didn't work for me. It took forever and files were left behind.

Appdata is often tons of very small files, so a long operation is expected.

As for the files not moved, did you disable Dockers and VMs in Settings ? The mover cannot move open files.

 

6 minutes ago, khager said:

Setting it back to "Only" afterward didn't move the files back.

Shares set to ONLY or NO are ignored by the Mover, only those with YES or PREFER are processed. You should have used Prefer I think.

[khager]

Both valid points. I did disable the dockers & VMs before running Mover. maybe I got something in the wrong order. I did wait long enough - or at least the UI said docker wasn't running anymore. All that got relegated to the curious when I realized my appdata backup would suffice.