[Plugin] Docker Compose Manager


Recommended Posts

Hello guys,   could you please clear my mind? 

Just setted up  all with 6.10 and portainer, the fact that I have some CA apps and others managed via portainer it's driving me crazy (most a OCD than a real issue) with some apps without icons ( should be fixable with last update ), some that I can update via unraid others via portainer and always  the label  "update ready"  on all portainer's container.

 

Using this plugin would allow to update all apps via webgui and ditch portainer?  I just want a clean and stable solution.

Thanks

 

 

Link to comment
3 hours ago, Hitokiri said:

some that I can update via unraid others via portainer and always  the label  "update ready"  on all portainer's container.

This will not change if you switch to this plugin. Containers managed via this plugin can be updated via a button in the gui however their update status does not integrate with unRAID's native dockerman ui and so the "update ready" label will continue to be present.

Link to comment

Ive got some really weird behavior surrounding changing labels after setting them initially. If I set an icon/webui port option, and then change it later, the old one sticks for some reason. Restarting the server doesn't clear them out, deleting the stack and rebuilding it under a different name doesn't clear them out either.

Also, a great feature would be a 'restart' button. With how the unraid gui loads, I spent a lot of time waiting for things to load while editing docker compose files. Id change one thing, save, load. Compose down, load. Compose up, load. Itd be nice to have a restart button which does down followed by up for when you're editing compose files and testing things out.

Link to comment
11 hours ago, Agent531C said:

Ive got some really weird behavior surrounding changing labels after setting them initially. If I set an icon/webui port option, and then change it later, the old one sticks for some reason. Restarting the server doesn't clear them out, deleting the stack and rebuilding it under a different name doesn't clear them out either.

When you say they stick do you mean a docker inspect shows that the labels are not updated or that the webui doesnt pick up the changes and modify its links? What version of unRAID are you running? The translating of labels to webui links is a feature of the unRAID os rather than this plugin specifically, if thats the issue we may have to raise this to limetech.

 

11 hours ago, Agent531C said:

Compose up, load. Itd be nice to have a restart button which does down followed by up for when you're editing compose files and testing things out.

There should be no reason to reload the page, it should auto refresh when the editor closes. Also you shouldnt have to do compose down and then compose up again. Running compose up on an already running stack should recreate its containers if anything has changed in its config file.

Link to comment
49 minutes ago, primeval_god said:

When you say they stick do you mean a docker inspect shows that the labels are not updated or that the webui doesnt pick up the changes and modify its links? What version of unRAID are you running? The translating of labels to webui links is a feature of the unRAID os rather than this plugin specifically, if thats the issue we may have to raise this to limetech.

 

There should be no reason to reload the page, it should auto refresh when the editor closes. Also you shouldnt have to do compose down and then compose up again. Running compose up on an already running stack should recreate its containers if anything has changed in its config file.

 

I honestly didn't know that about compose-up.

As for the first part, the actual webui doesn't pick up the changes for any label changes. If I set a label for an icon, and change it, it remains the first one even through reboots, deleting the cached image, and deleting/rebuilding the stack alltogether. Same goes for the 'webui' button on the popup. If I set a 'webui' label, and then go change it later, it remains whatever that first label was set on.

 

I am on 6.10.2, but this very well could be a limetech issue with the way labels work at an OS level as opposed to the compose plugin.

Link to comment

Sorry, but I'm a bit confused how to get my docker-compose dockers to be listed under this manager.  I'm glad it made installing docker-compose easy, but now how do i get my dockers in there after "creating a stack?"

 

I followed a tutorial to install this docker:

cd /mnt/user/appdata/
git clone -b release https://github.com/TheSpaghettiDetective/obico-server.git
cd obico-server && docker-compose up -d

 

Everything works fine, but I'd like to get that docker stack viewable under the docker-compose manager so I can use the up/down/pull buttons.

 

Can someone kindly point me in the right direction where to read up more on this?

 

Thanks!

Link to comment
6 hours ago, tofu- said:

Sorry, but I'm a bit confused how to get my docker-compose dockers to be listed under this manager.  I'm glad it made installing docker-compose easy, but now how do i get my dockers in there after "creating a stack?"

At this time compose manager only manages stacks created from its interface. When you add a stack in the webui it creates a folder on the flash drive and populates it with a new compose.yml. You could try copying the contents of your obico-server folder into the folder created by a compose manager stack.

Link to comment
  • 2 weeks later...

I searched the thread but couldnt' see anything similar, this doesn't seem to work well with macvlan networks after a reboot :(

 

I'm running LibreNMS thought a compose file with each container having a static IP on a vlan that's different from the host. Nothing starts after a reboot. If I try to run 'Compose up' after a reboot I get the following:


[+] Running 4/8
 ⠿ Container librenms_msmtpd      Starting                                                                                                                            0.1s
 ⠿ Container librenms_db          Starting                                                                                                                            0.1s
 ⠿ Container librenms_memcached   Starting                                                                                                                            0.1s
 ⠿ Container librenms             Created                                                                                                                             0.0s
 ⠿ Container librenms_snmptrapd   Created                                                                                                                             0.0s
 ⠿ Container librenms_redis       Starting                                                                                                                            0.1s
 ⠿ Container librenms_dispatcher  Created                                                                                                                             0.0s
 ⠿ Container librenms_syslogng    Created                                                                                                                             0.0s
Error response from daemon: network 01b3083bfada0eacc06028b994c51fcee636dfa299ca5742895ff5cdc27aca3a not found
 

If I click 'Compose Down' and wait for that to complete and then run 'Compose Up' everything starts up fine. Any ideas?

Link to comment
On 6/2/2022 at 9:36 PM, primeval_god said:

When you say they stick do you mean a docker inspect shows that the labels are not updated or that the webui doesnt pick up the changes and modify its links? What version of unRAID are you running? The translating of labels to webui links is a feature of the unRAID os rather than this plugin specifically, if thats the issue we may have to raise this to limetech.

 

On 6/2/2022 at 10:28 PM, Agent531C said:

As for the first part, the actual webui doesn't pick up the changes for any label changes. If I set a label for an icon, and change it, it remains the first one even through reboots, deleting the cached image, and deleting/rebuilding the stack alltogether. Same goes for the 'webui' button on the popup. If I set a 'webui' label, and then go change it later, it remains whatever that first label was set on.

 

I am on 6.10.2, but this very well could be a limetech issue with the way labels work at an OS level as opposed to the compose plugin.

 

The problem is indeed in the OS's docker manager plugin. It stores all the information in RAM and only supports changes to the XML template file. An quick and easy fix would be to delete `/usr/local/emhttp/state/plugins/dynamix.docker.manager/docker.json` and have it build again on reload every time you make a change to the compose file. Another approach is to include a small script within this plugin to remove the service entry from the same file every time `docker-compose down` is executed.

 

You can use `docker compose ps -a` to get the names of all the containers of a stack, pipe that into `jq 'del(.CONTAINER_NAME)' /usr/local/emhttp/state/plugins/dynamix.docker.manager.docker.json` to remove the entries, and then let the server build the file again once the stack is spun up.

Edited by arifer
Formatting
Link to comment

Adding the following snippet under the "up" and "down" lines in "/usr/local/emhttp/plugins/compose.manager/scripts/compose.sh" successfully settled the problem raised by @Agent531C for the case of updating the web UI label. I can get the same result by first spinning down a stack, editing it, and then spinning it back up; as well as editing a stack and then recreating it by clicking on the "compose up" button on the web GUI.

 

docker compose -f "$2" -p "$3" ps -a |                           
    awk '{if (NR!=1) {printf("%s.\"%s\"", sep, $1); sep=", "}}' |
    xargs -0 -I {} jq 'del({})' $DOCKER_JSON > $DOCKER_JSON      
docker compose -f "$2" -p "$3" up -d 2>&1                        

 

The following snippet is a cheap and dirty way of forcing unRAID to delete cached icons and retrieve them from the `net.unraid.docker.icon` label. I placed this udner the "up" "down" lines in the same file above.

docker compose -f "$2" -p "$3" ps -a |                               
  awk '{if (NR!=1) {print $1}}' |
  xargs -I {} find $DOCKER_IMAGES $UNRAID_IMAGES -name {}.png -delete

 

The environments used in the snippets are declared at the top of the file:

DOCKER_MANAGER=/usr/local/emhttp/state/plugins/dynamix.docker.manager
DOCKER_JSON=$DOCKER_MANAGER/docker.json
DOCKER_IMAGES=$DOCKER_MANAGER/images
UNRAID_IMAGES=/var/lib/docker/unraid/images

 

Edited by arifer
Clarity
Link to comment

Hey, had this plugin running for some months on Unraid 6.9.3 and upgraded to 6.10.3 today. unfortunately the command "docker-compose" cannot be found in terminal anymore.

I also tried removing and reinstalling the plugin with restart but it did not work.

 

Does anyone has an idea on how to fix this?

Link to comment
5 hours ago, afl said:

Hey, had this plugin running for some months on Unraid 6.9.3 and upgraded to 6.10.3 today. unfortunately the command "docker-compose" cannot be found in terminal anymore.

I also tried removing and reinstalling the plugin with restart but it did not work.

 

Does anyone has an idea on how to fix this?

Try 

docker compose

instead of docker-compose

Link to comment
11 hours ago, primeval_god said:

Try 

docker compose

instead of docker-compose

 

to elaborate on this, docker-compose is no longer a separate program but is a plugin to docker itself. So that's why it's `docker compose` now - you're passing the compose option to docker.

Link to comment
  • 2 weeks later...

Hi Guys

 

A bit of a dumb question probably but how do I map folders to compose.yml?

 

I have this:

 volumes:
     - redis_data:/mnt/user/appdata/myapp

 

And at the bottom this:

volumes:
   redis_data:
    driver: local

 

Am I missing something because the folder never gets filled up and the data just goes to:

/boot/config/plugins/compose.manager/projects/Compose

 

Is there something I'm missing?

Link to comment

Hi All, been a very long term unraid customer but have started to use it for much more.  I wanted to load teslamate but there was no unraid template anyone made so I loaded docker compose manager, found the stack and got everything working now.  Works great, but I just lost a couple weeks of data as I had a corruption and could not restore from backup.  I don't see any of the new containers in appdata and I'm not sure where it (I) put it or how to back it up.  I assume this is related to mapping as in the post above, but not sure.  I attached the txt file I used to compose.  Can anyone point me to an unraid/compose primer or help me understand this better.  I am pretty technically competent but not with linux or dockers, etc.  Mostly through googlefu I've been able to keep things running.

teslamate compose.txt

Link to comment
1 hour ago, howardis said:

Can anyone point me to an unraid/compose primer or help me understand this better.  

 

1 hour ago, howardis said:

I don't see any of the new containers in appdata and I'm not sure where it (I) put it or how to back it up.  I assume this is related to mapping as in the post above, but not sure.  I attached the txt file I used to compose. 

Indeed, the compose file you attached uses volumes to store data instead of bind mounts (host directories). You will want to remove the 'volumes' section at the bottom and then replace each volume name in the rest of the file with a host path.

This 

    volumes:
      - mosquitto-conf:/mosquitto/config
      - mosquitto-data:/mosquitto/data

becomes something like

    volumes:
      - /mnt/user/appdata/mosquitto/config:/mosquitto/config
      - /mnt/cache/appdata/mosquitto/data:/mosquitto/data

 

  • Upvote 1
Link to comment

Not sure if it's coming from Compose Manager or Unraid's Docker app, but 2 of my containers from Compose stacks keep showing "update available" on the Docker page. Both stacks use the "latest" tags for their respective containers, and it stays that way even if I "down", "docker image prune -a" to remove any images present and "up" again redownloading everything.

 

1999686401_Capturedcran(25).png.f49a1257d7b6a634c063d98ffc0b575b.png

 

Also some of the stacks I have set to autostart won't do so, failing with "network doesn't exist" until I "down" then "up" to recreate them. Maybe the plugin should do that when autostarting, or present an option for it?

Edited by Kilrah
Link to comment
On 6/16/2022 at 7:51 PM, opticon said:

I searched the thread but couldnt' see anything similar, this doesn't seem to work well with macvlan networks after a reboot :(

 

I'm running LibreNMS thought a compose file with each container having a static IP on a vlan that's different from the host. Nothing starts after a reboot. If I try to run 'Compose up' after a reboot I get the following:


[+] Running 4/8
 ⠿ Container librenms_msmtpd      Starting                                                                                                                            0.1s
 ⠿ Container librenms_db          Starting                                                                                                                            0.1s
 ⠿ Container librenms_memcached   Starting                                                                                                                            0.1s
 ⠿ Container librenms             Created                                                                                                                             0.0s
 ⠿ Container librenms_snmptrapd   Created                                                                                                                             0.0s
 ⠿ Container librenms_redis       Starting                                                                                                                            0.1s
 ⠿ Container librenms_dispatcher  Created                                                                                                                             0.0s
 ⠿ Container librenms_syslogng    Created                                                                                                                             0.0s
Error response from daemon: network 01b3083bfada0eacc06028b994c51fcee636dfa299ca5742895ff5cdc27aca3a not found
 

If I click 'Compose Down' and wait for that to complete and then run 'Compose Up' everything starts up fine. Any ideas?

 

@primeval_god, Any ideas?

Link to comment
17 hours ago, Kilrah said:

Not sure if it's coming from Compose Manager or Unraid's Docker app, but 2 of my containers from Compose stacks keep showing "update available" on the Docker page. 

This is normal. The unRAID web ui will not properly show the update status of containers created via compose. For any container created with compose the update status displayed in the webui is meaningless.

 

17 hours ago, Kilrah said:

Also some of the stacks I have set to autostart won't do so, failing with "network doesn't exist" until I "down" then "up" to recreate them. Maybe the plugin should do that when autostarting, or present an option for it?

The compose plugin run compose up on startup and compose stop on shutdown. What do your network configurations look like in your compose files?

Link to comment
33 minutes ago, primeval_god said:

The compose plugin run compose up on startup and compose stop on shutdown. What do your network configurations look like in your compose files?

 

The ones with issues are indeed also bound to the br0 macvlan. Seems bringing the stack up creates a binding that doesn't survive reboots, so when it starts up again after rebooting and expects it to still be there it isn't and that fails... Have to do a "down" instead of a "stop" for those so that the existing missing binding is forgotten and recreated on the next "up"...

 

  

33 minutes ago, primeval_god said:

This is normal. The unRAID web ui will not properly show the update status of containers created via compose. For any container created with compose the update status displayed in the webui is meaningless.

OK, just seemed strange since it only does that on 2 of the 7 I have

Edited by Kilrah
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.