Docker settings returning after update


xorinzor

Recommended Posts

I've had a few occurrances lately where some of my docker containers would be stopped. Because after they were automatically updated, some of the port assignments that I removed came back.

 

I found that this apparently has been discussed before, but there doesn't really seem to have been implemented a fix. Rather more of a workaround was discussed.

 

 

Are there any plans on creating an actual fix for this? I don't want to create unnecessary port assignments to my docker containers just so they don't keep re-appearing and causing issues for my containers. Perhaps if a user removes a port, it should be marked as "deleted" in the template and thus should not re-appear even if there is an update for the template (this would require actually processing the template file and comparing changes during an update).

 

If an actual solution (not a workaround) exists already I'd be happy to hear that too.

Link to comment
17 minutes ago, xorinzor said:

If an actual solution (not a workaround) exists already I'd be happy to hear that too.

Go to: "/boot/config/plugins/dockerMan/templates-user" and open up the template that you are having issues with and editing the line that starts with: <TemplateURL> and set it to:

<TemplateURL>false</TemplateURL>

 

 

19 minutes ago, xorinzor said:

Perhaps if a user removes a port, it should be marked as "deleted" in the template and thus should not re-appear even if there is an update for the template (this would require actually processing the template file and comparing changes during an update).

But what if the author of the template adds a variable or port because it is now necessary that wasn't necessary before, this is basically the same and @Squid can't take this into account.

 

 

25 minutes ago, xorinzor said:

some of the port assignments that I removed came back.

May I ask why you are deleting those ports and not editing them?

  • Thanks 1
Link to comment
1 hour ago, ich777 said:

Go to: "/boot/config/plugins/dockerMan/templates-user" and open up the template that you are having issues with and editing the line that starts with: <TemplateURL> and set it to:

<TemplateURL>false</TemplateURL>

 

That could theoretically be every container with the current way the templates work in Unraid, especially with the new fields potentially being added, it'd be more of a hassle to figure out why a container suddenly breaks because a new field got added that I then don't have, vs having to re-remove a port (although the downtime is annoying).

 

1 hour ago, ich777 said:

But what if the author of the template adds a variable or port because it is now necessary that wasn't necessary before, this is basically the same and @Squid can't take this into account.

 

If a specific field already existed before, and I chose to remove if, that should take precedence over any future updates for that specific field. That's why I think they should be keeping track of this and process it when the template updates (so every field that's marked as removed, will stay removed). Obviously new variables and ports should still be added, but existing ones should conform to whatever the user configures it to be (including a removed state).

 

1 hour ago, ich777 said:

May I ask why you are deleting those ports and not editing them?

 

Because I use a lot of networks between my containers to further isolate them from each other, and they generally use internal communication. Exposing the port on the host is of no benefit to me, and I do not want to add an additional firewall to my unraid server to isolate it as a host. Rather I'd just prevent any ports from opening up, and only having to open up ports (if required) to my WAN in the router.

Link to comment

Admittedly I've always been a little bit confused over what the actual issue here is.

 

Just because a port exists in the template does not mean that it's exposed to the WAN.  And, if you're using custom networks (eg BR0) or host, then any and all port mappings are ignored anyways.  (ie: install the default cAdvisor template from CA - You'll hit the webUI on port 8081.  Delete the port and you can't hit the webUI.  Now move the container to br0 and you'll hit the webUI on port 8080 even though the port doesn't exist in the template)  Whether or not the port exists in the template (or the docker run command) does not stop the container from opening the port.  All it does is tell the docker system what to do when it's attempted to be accessed from outside the container.  And in the case of host / br0 the mappings are ignored by design from  docker itself.  It's why if you install any of the Plex Containers they do not have any port mappings in the template, yet they work as Host / br0, but if you switch the template to bridge they will fail)

 

Because @ich777 told me that there could be issues (once again I do not understand why), provisions were put into place to prevent the template from being updated if the container updates.  Note that simply stopping and restarting a container will NOT ever update the template.  Only an update to the container itself (not to the app within) will also have the template updated.  This provision is not a work-around, but is a by-design feature.  It's just not accessible via the GUI to prevent users from inadvertently changing the entry to something that doesn't work (80% of the maintainers can't figure out the proper URL to put in there, so CA's appfeed fills it in automatically)

 

As mentioned, there is no real way to determine if a port was deleted and you don't want it added again during an update or if the port isn't existing in the first place and needs to be added.

 

All in all, the update system solves far far more problems than it creates and if (once again "if") there are issues with ports being deleted and causing an actual issue with the container if it gets re-added, then the provisions exist for the maintainer of the template (or the user) to be able to easily flag the template as "do not update this"

Link to comment
19 minutes ago, Squid said:

Just because a port exists in the template does not mean that it's exposed to the WAN. 

 

I am aware of this, but it is exposed on the host. From a security standpoint of view it really is a no-go to expose services that shouldn't (have to) be exposed. Even if it's on the local network.

 

19 minutes ago, Squid said:

if you're using custom networks (eg BR0) or host, then any and all port mappings are ignored anyways.

I'm not using either br0 or host. I have custom bridge docker networks. Ports from the containers themselves are available to other containers within the same network, but as long as I do not have a port mapping defined, it will not be exposed on the host and available from the network that my unraid server is in.

 

For example, I have a network called "webhost" in which I have my nginx, php and mysql containers. I only have to expose nginx on the host, but do not want my mysql service to be exposed.

 

19 minutes ago, Squid said:

provisions were put into place to prevent the template from being updated if the container updates. 

 

19 minutes ago, Squid said:

Only an update to the container itself (not to the app within) will also have the template updated. 

 

I'm confused about this, these 2 statements seem contradictory. I use the Auto Update Applications plugin to automatically update the containers and have had a couple of times now that some docker containers are unable to start due to conflicting Port mappings that have been re-added after they got updated.

 

19 minutes ago, Squid said:

there is no real way to determine if a port was deleted and you don't want it added again during an update or if the port isn't existing in the first place and needs to be added

 

I'm not really familiar with how the template system works. But was under the impression that Unraid keeps track of these fields in the XML on a per-container basis. Perhaps an solution would be to instead of removing the field, marking them as "disabled" or leaving their value empty? This way, if the template updates, the field still exists, but is just "disabled".

Edited by xorinzor
Link to comment
3 hours ago, xorinzor said:

That could theoretically be every container with the current way the templates work in Unraid

27 minutes ago, xorinzor said:

start due to conflicting Port mappings that have been re-added after they got updated.

That's why I disabled the TemplateURL in most of my containers and that's basically the only way that I know to avoid adding the ports again.

 

Many users got issues with the game server containers because if someone want's for example two CS:Source containers you can't simply "NAT" the port by changing it in the template because that won't work.

What you have to do is delete the port mapping in the template, add a new one with the same ports for the container/host, edit the game config file with the corresponding port to the added port mapping in the template, done.

Link to comment
3 minutes ago, ich777 said:

That's why I disabled the TemplateURL in most of my containers and that's basically the only way that I know to avoid adding the ports again.

 

If that's the only solution, then I'll have to do that. But it really feels more like a hack then a fix.

 

3 minutes ago, ich777 said:

if someone want's for example two CS:Source containers you can't simply "NAT" the port by changing it in the template because that won't work.

 

Never experienced such issues in any of my minecraft containers. AFAIK the service inside the container is none the wiser about the mapped ports.

Link to comment
8 minutes ago, xorinzor said:

Never experienced such issues in any of my minecraft containers. AFAIK the service inside the container is none the wiser about the mapped ports.

I'm talking about Steam Ports, some games actually connect through a query port and have also other ports for the game traffic (ARK, ConanExiles, Valheim).

 

They initially connection is done through the query port and the answer from the query port actually provides the game ports which are read from the game config files and if you have different port inside and outside the container that simply can't work.

The issue here is that you can basically see the game in the Server Browser but you can't connect to it because the port is the wrong one.

 

8 minutes ago, xorinzor said:

But it really feels more like a hack then a fix.

I would not say it's a hack... :D

It's more a complete custom thing, since such things are not requested that often and not every user has such a setup like you are running.

Link to comment
25 minutes ago, ich777 said:

I'm talking about Steam Ports, some games actually connect through a query port and have also other ports for the game traffic (ARK, ConanExiles, Valheim).

 

They initially connection is done through the query port and the answer from the query port actually provides the game ports which are read from the game config files and if you have different port inside and outside the container that simply can't work.

That makes sense as to why there's an issue, but it always implies that the apps aren't friendly to a docker system.

 

26 minutes ago, ich777 said:

I would not say it's a hack... :D

It's more a complete custom thing, since such things are not requested that often and not every user has such a setup like you are running.

Absolutely.

  • Like 1
Link to comment
Just now, Squid said:

That makes sense as to why there's an issue, but it always implies that the apps aren't friendly to a docker system.

Exactly...

But that also doesn't apply to every game server but I always recommend to not "NAT" the port through Docker to avoid issues, at least for my game server containers.

 

I know people that are running a cluster from ARK with about 10 maps on the server with only one data directory, that means 10 ARK Docker containers on unRAID... :D

Don't ask how they do it because I'm not really into modding ARK but here a big thanks go to @Cyd who has written a tutorial on that. :)

  • Haha 1
Link to comment
32 minutes ago, ich777 said:

I'm talking about Steam Ports, some games actually connect through a query port and have also other ports for the game traffic (ARK, ConanExiles, Valheim).

 

I agree with @Squid here.

 

32 minutes ago, ich777 said:

I would not say it's a hack... :D

It's more a complete custom thing, since such things are not requested that often and not every user has such a setup like you are running.

 

Well, for docker containers, my use case isn't really all that weird ;) It might in the unraid userspace not be all that common, but this is quite a normal way to use docker containers.

Which isn't an issue, but I do think there's room for improvement in this particular case. Overall my experience with managing docker containers in unraid has been really positive.

Edited by xorinzor
Link to comment
3 minutes ago, xorinzor said:

Which isn't an issue, but I do think there's room for improvement in this particular case. Overall my experience with managing docker containers in unraid has been really positive.

Maybe a bad idea but would it be possible to add a checkbox, when Advanced View on the template page is turned on, if the template should be updated or not and change the value in the xml based on that?

 

Would that be somehow possible @Squid or is this hard to implement? Don't know how CA finds out the template url or when it's added to the template when it's not filled in from the maintainer.

  • Like 2
Link to comment

This has been irritated the hell out of me for the last few weeks and has caused me to chase my tail repeatedly as I have multiple containers I built based on the same original template that originally came from CA.  The problem is it constantly overwrites my metadata (links, icons etc) and adds paths and keys from the original template (that have been removed or renamed) that in my case is causing stability and security issues due to the misconfiguration.

 

I'd be super thankful if you could implement a gui way to disable template updates, instead of needing to poke around the backend configuration files.

 

Link to comment
  • 4 months later...

Tried this method 

But only after enabling the Template Authoring mode and not finding the TemplateURL field I continued reading the thread only to read that that field got removed (does the Template Authoring Mode even serve a purpose now?).

 

It'd be really great if you could re-implement any kind of way for us to take manual control, or just allow empty fields (which are then ignored entirely).

 

Link to comment

6.10 Does not update the templates any more.  TemplateURL has been since 6.9 always present in the xml's but hidden and no way to directly access it.  This is because 90% of people filled it out wrong and the application feed overwrote whatever was filled out anyways.  The way that's always been reiterated to stop 6.9 from updating the templates is to manually edit the xml and change the TemplateURL entry to be false

 

And yes, authoring mode does still serve a purpose as there are extra fields in there

  • Thanks 1
Link to comment
  • 10 months later...

Hey there,

 

I've been searching about this subject for a few days and found a couple of posts about it.

 

As a result of the new behavior in unRAID 6.10+, few dockers is not recognized as installed in the CA due to a change in the template. For example, I have swag installed since 2020 and currently the template is different (I guess even the repository is changed). Therefore, it shows as not installed in CA.

 

Is there a way to force updating the templates?

 

Thanks in advance for your support.

Link to comment
4 hours ago, Zuerrex said:

Therefore, it shows as not installed in CA

CA shows everything installed so long as the user-template exists on the flash drive (/config/plugins/dockerMan/templaes-user)

 

If you've deleted it or it became corrupt or if you installed a container manually via the command line or docker compose then it will not show it as being installed, because CA is unable to manage it.

 

IE:  Does "Edit" appear in the drop down on the docker tab, and if it does will it return an error when selecting it

Link to comment

I just checked the templates folder and I have 2 files there for swag. One ends with .xml and the other one ends with .xml.bak.

 

Moreover, the "Edit" option is available when I click on swag docker icon in docker tab.

 

Not sure why I'm facing this issue with multiple containers.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.