[Plugin] Docker Compose Manager

[nearcatch]

15 hours ago, Kilrah said:

 

The ones with issues are indeed also bound to the br0 macvlan. Seems bringing the stack up creates a binding that doesn't survive reboots, so when it starts up again after rebooting and expects it to still be there it isn't and that fails... Have to do a "down" instead of a "stop" for those so that the existing missing binding is forgotten and recreated on the next "up"...

 

  

OK, just seemed strange since it only does that on 2 of the 7 I have

 

You could try setting "Preserve user defined networks" to "Yes" in docker settings. I remember having the issue you're experiencing and I *think* changing this setting fixed it.

[Kilrah]

Oh makes sense it might, turned it on now, had to down/up again afterwards obviously but will see if next restart goes smooth.

 

Made me notice something else though, while the autostart stacks do start on reboot they don't seem to when disabling/reenabling docker which I had to do to turn the option on. Haven't tested with array start yet...

 

EDIT: Well, next start also didn't work...

Edited July 21, 2022 by Kilrah

[nearcatch]

On 7/21/2022 at 12:16 AM, Kilrah said:

Oh makes sense it might, turned it on now, had to down/up again afterwards obviously but will see if next restart goes smooth.

 

Made me notice something else though, while the autostart stacks do start on reboot they don't seem to when disabling/reenabling docker which I had to do to turn the option on. Haven't tested with array start yet...

 

EDIT: Well, next start also didn't work...

 

The other thing I've done is disable "IPv4 custom network on interface br0 (optional):", which is only visible after stopping the docker service. Iirc this, combined with "Docker custom network type: macvlan", creates a macvlan network via unRAID, but unRAID recreates it on every boot. This means the network id changes even if the network seems the same. I instead manually create a new docker macvlan network, and that gets preserved with the setting I mentioned in my previous post. I used something like this to create it months ago, and haven't had to recreate it since, even after more than a dozen reboots:
 

docker network create -d macvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 --ip-range 192.168.1.160/28 -o parent=br0 --aux-address="host=192.168.1.160" lan_macvlan

Use this in the compose file to make it available to your containers:

networks:
  lan_macvlan:
    external: true
    name: lan_macvlan

 

Sorry for the guesswork here. I set this up almost a year ago now and don't quite remember how I got past the problem.

Edited July 23, 2022 by hasown

[Kilrah]

Thanks, that worked. 

 

Now last thing to try to understand is why some stacks that are set to start automatically do so but some don't (at least when disabling/enabling docker)...

Edited July 23, 2022 by Kilrah

[primeval_god]

A new version of the plugin is available. This version allows choosing alternate project directories for storing more complex docker stacks. This version also implements functionality for managing webui and icon labels for unRAID webui integration. 

 

[NAS]

 

Addon works great.

 

What are your plans for patching compose. I can see that the opening posts says version 2.1.1

 

On 10/3/2021 at 8:22 PM, primeval_god said:

...

This plugin installs docker compose 2.1.1 and compose switch 1.0.3.

...

 

however the plugin change logs says:

2022.05.14

Change default output style to terminal.

Added theme support.

Docker Compose v2.5.0

 

which matches what is included:

# docker-compose -v
Docker Compose version v2.5.0

 

but according to https://docs.docker.com/compose/release-notes/#250

 

this is three months and three releases out of date.

[primeval_god]

5 hours ago, NAS said:

What are your plans for patching compose. I can see that the opening posts says version 2.1.1

Yeah its been a while since i updated the opening post (not sure many people read it anyway).

 

If there are any security issues or important bug fixes in the packages i include feel free to bring them to my attention and i will try to patch them asap. Otherwise the answer is probably sporadically or when i remember to check my dependencies. For this last release I forgot, I will try to get an update out soon.

[nearcatch]

On 7/28/2022 at 8:44 AM, primeval_god said:

Yeah its been a while since i updated the opening post (not sure many people read it anyway).

 

If there are any security issues or important bug fixes in the packages i include feel free to bring them to my attention and i will try to patch them asap. Otherwise the answer is probably sporadically or when i remember to check my dependencies. For this last release I forgot, I will try to get an update out soon.

Yeah I would welcome an update to the current compose version. The last couple of releases have added extra features for docker secrets.

[primeval_god]

@hasown@NAS A new version is available with an updated version of compose.

[shadowbert]

I'm a bit confused. Everything seems to be working well - but the UI and the command line don't seem to agree on what containers are running.

 

That is, if I run `docker compose up -d`, the webui still thinks everything is donw.

If I press "up" on the webui, the docker compose logs -f gives me nothing (as does docker compose ps)

 

If I do both, I get two sets of the same containers.

 

Is this a known thing? Or has something gone crazy wrong on my  end?

[primeval_god]

4 hours ago, shadowbert said:

I'm a bit confused. Everything seems to be working well - but the UI and the command line don't seem to agree on what containers are running.

 

That is, if I run `docker compose up -d`, the webui still thinks everything is donw.

If I press "up" on the webui, the docker compose logs -f gives me nothing (as does docker compose ps)

 

If I do both, I get two sets of the same containers.

 

Is this a known thing? Or has something gone crazy wrong on my  end?

The webui portion of this plugin is designed only to show compose stacks created and launched via the webui. I think your issue specifically is that you are launching the stack differently than the plugin does. The plugin uses something like 

docker compose up -f docker.compose.yml -f <possibly other compose files> -p <projectname>

Where project name is a sanitized version of the stack name assigned in the webui. The project name is what the plugin uses to match against and display the status of stacks.

[sjtuross]

Unraid keeps saying "update ready" for one container created by stack. I tried below but no luck. Any suggestion?

1. Compose pull, down and then up through webui.
2. Manually run docker pull and then restart stack.
3. Deleted this state file /usr/local/emhttp/state/plugins/dynamix.docker.manager/docker.json

Edited August 11, 2022 by sjtuross

[primeval_god]

4 hours ago, sjtuross said:

Unraid keeps saying "update ready" for one container created by stack. I tried below but no luck. 

This is normal behavior. Update status tracking only works for containers created by Dockerman. For containers created by other means including compose manager the update status is not meaningful. 

[sjtuross]

I run `docker images --digests --no-trunc` and found out the same image with 2 different digests. Most probably the same image was republished to docker hub. So I deleted the image, restart the stack and then click "check for updates" in webui. Now it says up-to-date.

 

It's a false alarm to docker compose manager which is awesome.

Edited August 11, 2022 by sjtuross

[Not liam]

im having an isue with docker copose.

i had prviously installed a old vertion witch worked but no with the compose file so then i found out about this plugin and now its broken. ive tryed to uninstall and re-install multipel times witch proberly is doing be no favours.

when i typen both docker-compose vertion and without the dash i get not reconised docker comand with the plugin installed.

if any one has any ideas that would be grately apretiated. 

[Agent531C]

I have an odd problem, and perhaps its related to ShadowBerts problem, but I just noticed that some of my compose stacks are missing from the web UI, even though they are in the compose.manager/project folder and are still being autostarted when the array starts. I am on 6.11-rc3, and the 'invisible stacks' all start fine, but have no representation in the UI for pull/up/down/editing.

The Web UI:

 

The Folder:

[primeval_god]

16 hours ago, Not liam said:

im having an isue with docker copose.

i had prviously installed a old vertion witch worked but no with the compose file so then i found out about this plugin and now its broken. ive tryed to uninstall and re-install multipel times witch proberly is doing be no favours.

when i typen both docker-compose vertion and without the dash i get not reconised docker comand with the plugin installed.

if any one has any ideas that would be grately apretiated. 

First and foremost I would remove the compose manager plugin and whatever you previously used to install compose then restart your server to make sure not traces of compose are left. Then try re-installing the compose manager plugin. How were you installing compose previously by the way? And what version of unRAID are you running?

[primeval_god]

3 minutes ago, Agent531C said:

I have an odd problem, and perhaps its related to ShadowBerts problem, but I just noticed that some of my compose stacks are missing from the web UI, even though they are in the compose.manager/project folder and are still being autostarted when the array starts. I am on 6.11-rc3, and the 'invisible stacks' all start fine, but have no representation in the UI for pull/up/down/editing.

 

I dont know for sure why this is happening but it may have something to do with 6.11 (currently i cant test against 6.11). Based on the screenshots you have though i am going to say it probably has to do with the fact that there are spaces in the folder paths of the missing stacks (well there is one without spaces but it may be shadowed by the one with the same prefix before the space). I would suggest manually removing the containers for the missing stacks (with dockerman) then in the terminal rename the problem stack folders with '_' instead of spaces and then try refreshing the page (or maybe rebooting).

[Agent531C]

4 hours ago, primeval_god said:

I dont know for sure why this is happening but it may have something to do with 6.11 (currently i cant test against 6.11). Based on the screenshots you have though i am going to say it probably has to do with the fact that there are spaces in the folder paths of the missing stacks (well there is one without spaces but it may be shadowed by the one with the same prefix before the space). I would suggest manually removing the containers for the missing stacks (with dockerman) then in the terminal rename the problem stack folders with '_' instead of spaces and then try refreshing the page (or maybe rebooting).

Ended up shutting down the array, renaming them, and then restarting the array, but it cause a weird docker crash where the manager wouldn't start. So I uninstalled the compose plugin, rebooted, and everything came back up fine. Added the Compose plugin again (with the renamed folders), and all of the ones in the originally attached image showed up fine, but none of the others are in the web ui, even though their containers are started. Should I try/have tried removing the containers completely, orphan images and all?

[primeval_god]

1 hour ago, Agent531C said:

Should I try/have tried removing the containers completely, orphan images and all?

You need to remove the running or stopped containers belonging to each of those stacks, the orphan images dont matter. Shutting down docker/the array will not remove the containers only stop them.

[Not liam]

13 hours ago, primeval_god said:

First and foremost I would remove the compose manager plugin and whatever you previously used to install compose then restart your server to make sure not traces of compose are left. Then try re-installing the compose manager plugin. How were you installing compose previously by the way? And what version of unRAID are you running?

i have fixed it by modifing the defult install comand then installing the docker compose plugin again

 

defult : "curl -SL https://github.com/docker/compose/releases/download/v2.7.0/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose" ,  "chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose"

 

simple modifcation to diretorys : "curl -SL https://github.com/docker/compose/releases/download/v2.9.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose" ,  "chmod +x /usr/local/bin/docker-compose"

 

the v2.9.0 inside the curl you can modify to curent vertion

[primeval_god]

9 hours ago, Not liam said:

i have fixed it by modifing the defult install comand then installing the docker compose plugin again

 

defult : "curl -SL https://github.com/docker/compose/releases/download/v2.7.0/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose" ,  "chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose"

 

simple modifcation to diretorys : "curl -SL https://github.com/docker/compose/releases/download/v2.9.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose" ,  "chmod +x /usr/local/bin/docker-compose"

 

the v2.9.0 inside the curl you can modify to curent vertion

I really dont understand what you are doing here...

This plugin should handle properly installing docker compose. To use this plugin you should ensure that you do not have compose installed on your system from any other installation method.

[Agent531C]

21 hours ago, primeval_god said:

You need to remove the running or stopped containers belonging to each of those stacks, the orphan images dont matter. Shutting down docker/the array will not remove the containers only stop them.

Ended up doing that, renaming, and rebooting, but still no luck. I ended up just pulling the compose/.env files out of the config, and then deleting the folders and recreating them through the web gui with no spaces. Seems to have no issues so far

[sportsziggy]

Just re-installed the plugin as I was getting a

Quote

"service "..." has neither an image nor a build context specified: invalid compose project"

error. However if I were to go to the docker-compose folder and run "docker-compose up" in terminal it would work just fine.
Upon reinstall now I'm getting a

Quote

"-f/boot/config/plugins/compose.manager/projects/setup/docker-compose.yml" is not a valid project name

any idea what I need to do to fix it?

Edited August 17, 2022 by sportsziggy

[primeval_god]

1 hour ago, sportsziggy said:

Just re-installed the plugin as I was getting a

error. However if I were to go to the docker-compose folder and run "docker-compose up" in terminal it would work just fine.
Upon reinstall now I'm getting a

any idea what I need to do to fix it?

What do the content of the project directory look like?

As for your original error about the invalid compose file, that would be due to a service within the compose file that was missing an image name. Could be a mistake in the compose file or a problem with the override file the plugin generates to add unRAID compatibility labels. Just before you got that error had you edited any compose file outside of the GUI? Or had you renamed any service within a compose file?