Docker Images 'gone/missing' after 6.10.3 to 6.11.5 update


Outlaws
Go to solution Solved by Squid,

Recommended Posts

Hey all,

 

Why can't upgrades go smoothly? They normally have - just not this time :(

 

I tried to update my server from 6.10.3 to 6.11.5 this morning, and have found a strange issue. Update Assistant showed no concerns before the update, and I was able to access the machine again after a reboot. I experienced a separate issue when trying to start the array, where it showed 'array stopped - stale configuration'. Googling showed this to be a Firefox/webui issue, and I was able to start the array after restarting the server and using Edge to sign in.

 

Now, the issue is that the update appears to have removed/corrupted the install of all my Docker images? The images still show on the page and are able to start, and appear to be working as expected - but I am not able to edit/update any of the Docker containers. So I am not able to modify the settings on any docker container that appears, and selecting 'Force Update' on the container gives a more ominous error:

 

"Configuration not found. Was this container created using this plugin?"

 

Cool. So I install a random new Docker image from the store, and that one works as expected - I can select the container name and modify settings. So somehow, it seems to no longer recognize the previously installed containers. I have restarted the server again, toggled the Docker service off/on, and can still view the folders in /appdata - but still the issue persists.

 

Any recommendations on how to correct this, or just drop back to 6.10? Images below for reference

 

https://imgur.com/a/DcyozwK

snip.JPG

snip2.JPG

snip3.JPG

snip4.JPG

Link to comment
  • Solution
5 minutes ago, Outlaws said:

"Configuration not found. Was this container created using this plugin?"

 

 

5 minutes ago, Outlaws said:

previously installed containers.

 

Because the folder /config/plugins/dockerMan/templates-user is corrupted and/or everything is deleted from it.  This is not anything (at all) related to the upgrade.

 

What is the contents of that folder?

  • Like 1
Link to comment

I really appreciate the quick responses! So I did take a backup of the flash before the upgrade, and there I can see plenty of files in the /dockerMan/templates-user/ folder. I'm having a harder time navigating to that folder on the running server - what would that file path be?

 

I typically use Krusader to interact with the unraid files - is the /config/plugins/dockerMan/templates-user on the USB drive accessible from here, or do I need to navigate there another way?

snip5.JPG

Link to comment

root@Unii:/boot/config/plugins/dockerMan/templates-user# ls -la
total 24
drwx------ 2 root root 8192 Dec 22 02:46 ./
drwx------ 4 root root 8192 Dec 22 02:28 ../
-rw------- 1 root root 3219 Dec 22 02:46 my-Duplicacy.xml
root@Unii:/boot/config/plugins/dockerMan/templates-user#

 

 

Found it - appears to be empty lol. So what would be the best way to restore at this point, copy the template files from the flash backup to here and reload the server?

Link to comment

That is a very good, and concerning, question that I'm thinking about. I didn't take any actions this morning except what was described, and also using the 'Cleanup Appdata' plugin to remove four unused appdata files, but don't recall if that was before or after running the flash backup where I am still seeing the templates.

 

Also, I had noticed that the hostname of the server reset to 'Tower'. This was prior to the move to 6.11.5 as well, and shows the backup as 'tower-flash-backup'. (I performed a hardware update, new chassis/cpu cooler, on the server in October, which is the last time it was restarted - so this name change may have occurred then, however I only noticed the change this morning after creating the flash backup.)

 

I created another flash backup now, and am comparing the 6.11.5 backup vs the 6.10.3 version this morning, and an older 6.10.3 backup from August. The only differences I see (aside from the bz system files) are in that config/plugins/dockerMan/templates-user folder. I can share these 3 if you would be interested in looking at them as well, or any other diag files.

 

Edited by Outlaws
Link to comment

Okay, fun times. I still wasn't able to find much difference between the 3 backup files other than the latest one missing the templates. I've run a few health checks on the USB drive using Hard Disk Sentinel, and it doesn't seem to be spitting out errors or bad blocks there. I've restored the template files from the last flash backup containing them, and seem to be back in business.

 

I have some spare USB drives ready for if that main one fails, just in case.

 

Thanks again for your help - I've seen you both in countless other forum posts, and I can't imagine the amount of headaches you've saved throughout the years. Have a great holiday 😃

snip6.JPG

Link to comment
5 hours ago, Outlaws said:

hostname of the server reset to 'Tower'

this suggests config/ident.cfg was corrupt or missing and had to be recreated. And makes me wonder what else might be corrupt or missing.

 

Did you copy all of config from your backup?

 

Are there any FSCK files on your flash drive? Those would have been created by filesystem repair and if it had to be repaired then that would explain these losses.

Link to comment

Hey trurl, thanks for the follow up. Comparing the three backup files I have, (Aug, this morning pre-upgrade, post upgrade), I do see where the August/oldest ident.cfg has my original configuration edits. I will copy the lines that are different below.

 

Oldest/original:

 

# Generated settings:
NAME="Unii"
timeZone="America/New_York"
COMMENT="Unii"

enableFruit="no"           ---------------- what is this line?
USE_NETBIOS="yes"

NTP_SERVER1="<private>"
NTP_SERVER2="time1.google.com"
NTP_SERVER3="time3.google.com"
NTP_SERVER4="time4.google.com"

USE_SSH="yes"
START_PAGE="Dashboard"

 

This morning, post hardware upgrade but pre-6.11.5:

 

# Generated settings:
NAME="Tower"
timeZone="America/Los_Angeles"
COMMENT=""

enableFruit="yes"
USE_NETBIOS="no"

USE_NTP="no"
NTP_SERVER1=""
NTP_SERVER2=""
NTP_SERVER3=""
NTP_SERVER4=""

USE_SSH="no"
START_PAGE="Main"

 

 

 

This is more concerning, for sure. What other files would you recommend reviewing on the flash backups that I have retained?

Link to comment

All of your configuration (settings from the webUI) is in the config folder.

 

Did you look for FSCK files?

 

Do your backups have all files from flash or just config folder? FSCK files would be in the root of flash or any full flash backup. You might be able to open them in a text editor to see what files might have been reset.

Link to comment

Gotcha, thanks for the quick response. I do not see any *.fsck files in any of the 3 backup folders, no. However, in the pre and post update files from today, I do see one FSCK0000.REC file - is that the smoking gun?

 

 

I am looking thru the other .cfg files in that folder, so far I'm not finding any anomalies between the August & latest backups in the domain.cfg, smart-one.cfg, share.cfg, network-rules.cfg (one change, but likely due to the Oct hardware upgrade), network.cfg, disk.cfg, or flash.cfg.

 

I do see differences in docker.cfg, but this may be my initial troubleshooting before the original post.

 

Are there other diagnostics or .cfg files you would recommend checking?

Edited by Outlaws
Link to comment
1 hour ago, Outlaws said:

I do see one FSCK0000.REC file - is that the smoking gun?

yes, but if there is only that one maybe not much problem, maybe that is the missing ident.cfg. You could see if it will open in a text editor.

1 hour ago, Outlaws said:

.cfg files you would recommend checking?

Each of your user shares has their own .cfg in /config/shares. Shares will still exist even without a .cfg since user shares are just the combined top level folders, but any without a .cfg file will have default settings.

 

 

Link to comment

Understood. You are correct, when I open that FSCK0000 file in Notepad++, it's the ident.cfg file and then scrambled/unrendered text beneath it. I've gone thru the /config/shares for the August and today backups, and none of the files differ. The newer backups have this 'private.cfg' file which is not in the August version, though.

 

I've attached the FSCK file, if it's of any use to you. With this chain of events, would you have any other recommendations? Replace the usb drive, or not necessarily? Thanks again

FSCK0000.REC

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.