[Plugin] Docker Compose Manager

[primeval_god]

33 minutes ago, L0rdRaiden said:

@primeval_god

If I add my the "compose manager" labels to my docker compose file, should I delete the docker-compose.override.yml? is there a way to override this file and use exclusively whatever I set in the compose label?

 

I see in some cases inconsistencies where the content of this file does not match the labels defined for it in docker compose.

 

I am using labels like this

lables:

      - "com.centurylinklabs.watchtower.enable=true"

      - "net.unraid.docker.managed=composeman"

      - "net.unraid.docker.shell=sh"

      - "net.unraid.docker.icon=https://raw.githubusercontent.com/ibracorp/unraid-templates/master/icons/traefik.png"

      - "net.unraid.docker.webui=https://traefik.dsaasddsa.com.es/dashboard/#/"

Off hand i think deleting the override files wont cause a problem. I am pretty sure they will get recreated any time the compose stack is edited in the GUI. 

[OrdinaryButt]

I run into write permission issues while installing a grafana as part of compos bundle of containers. Was able to "resolve" by adding 'user:"0" '  (root) to compose file. I feel like this is a bad practice, but I don't know if I am actually compromising anything by doing that. Anyone knows if this is a potential week point and if there might a different way to solve it?

[mtongnz]

I admit I haven't read all of the 16 pages here but have tried searching...

 

Is there a Github repo for this?  I've modified some of this plugin locally to allow use of a "master" .env with multiple projects and I'd like to share it back.

 

I'm slowly moving alll my services to compose.  I was getting annoyed having to maintain multiple identical .env files across different folders.  It seems to work flawlessly and allows to easily switch between a "local" and "master" .env.  If you maintain the compose files in code server (as I do) then you can modify the file to use any .env file you'd like (so you could have a few "masters")

[hrv231]

Hello,

 

I just want to check if this is normal in this plugin?
I'm running a normal docker compose to test the plugin.

Each of my services are running like this, example:
 

graylog:

    networks:
      eth0:
        ipv4_address: 10.10.10.20

networks:
  eth0:
    external: true
    name: eth0

 

The Docker tab doesn't show the name of my network, instead it shows the id of it.

I'm using macvlan, "no bridge" and "no bonding"

 

root@Tower:~# docker network inspect eth0
[
    {
        "Name": "eth0",
        "Id": "a38b1ae9bf3865b59f1c58c33308a3885528d2d57241ed44fe812caa63aae6d3",
        "Created": "2023-11-18T17:02:18.576907217-08:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {

 

[Kilrah]

Why do you name a custom network "eth0'" when there's likely already a physical interface named that it would conflict with?

[L0rdRaiden]

6 hours ago, hrv231 said:

Hello,

 

I just want to check if this is normal in this plugin?
I'm running a normal docker compose to test the plugin.

Each of my services are running like this, example:
 

graylog:

    networks:
      eth0:
        ipv4_address: 10.10.10.20

networks:
  eth0:
    external: true
    name: eth0

 

The Docker tab doesn't show the name of my network, instead it shows the id of it.

I'm using macvlan, "no bridge" and "no bonding"

 

root@Tower:~# docker network inspect eth0
[
    {
        "Name": "eth0",
        "Id": "a38b1ae9bf3865b59f1c58c33308a3885528d2d57241ed44fe812caa63aae6d3",
        "Created": "2023-11-18T17:02:18.576907217-08:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {

 

I have the same config with the same result. Everything is fine the problem is unRAID compatibility with docker compose to represent the config in the webui

[hrv231]

10 hours ago, Kilrah said:

Why do you name a custom network "eth0'" when there's likely already a physical interface named that it would conflict with?

I didn't name it, it is the one by default when you turn off bridging and binding.

[hrv231]

6 hours ago, L0rdRaiden said:

I have the same config with the same result. Everything is fine the problem is unRAID compatibility with docker compose to represent the config in the webui

Thank you.

So.. you just use it like that?

[sjtuross]

14 hours ago, L0rdRaiden said:

I have the same config with the same result. Everything is fine the problem is unRAID compatibility with docker compose to represent the config in the webui

It can show the network name correctly in the past, at least when I upgraded to Unraid 6.12.0. Not sure when and why it started showing network id only. Everything works, but it just looks odd.

Edited December 17, 2023 by sjtuross

[primeval_god]

On 12/16/2023 at 10:16 PM, sjtuross said:

It can show the network name correctly in the past, at least when I upgraded to Unraid 6.12.0. Not sure when and why it started showing network id only. Everything works, but it just looks odd.

It was an update to docker compose. They changed something about how they reference networks, which interacts badly with how unRAIDs webui detects network names.

[Caleb Bassham]

Sorry to be a nuisance, but I can't find a lot of info for this plugin. Hoping for an update soon to get docker compose up to > v2.22 for the compose file extending/merging/importing new hotness. Any idea of when this may be?

 

Also, is there some sort of repo/homepage for this project besides this thread?

[primeval_god]

52 minutes ago, Caleb Bassham said:

Sorry to be a nuisance, but I can't find a lot of info for this plugin. Hoping for an update soon to get docker compose up to > v2.22 for the compose file extending/merging/importing new hotness. Any idea of when this may be?

There is no ETA for updates but likely not until after the holidays.

 

56 minutes ago, Caleb Bassham said:

Also, is there some sort of repo/homepage for this project besides this thread?

The code is on github but this thread is all there is so far as support and info.

[aglyons]

Haven't been working with Docker for very long and even less with compose.

 

I have a project that I was trying to get working in compose as there's no template for it is the UR app store.

 

https://github.com/JesseWebDotCom/homepage-for-tesla

 

I have a MACVLAN docker network already in place (eth0.2) for some other external services so I was trying to interpret what I found on other sites on how to add those settings to the compose YAML. I must have something wrong as it's not working at all. It doesn't crash but I can't access anything from the IP I assigned it.

 

If it's easier to create an UR template rather than crack the compose mystery then I'm all for that. Mind you, I don't have a clue how to do that yet either lol.

 

version: '3'
services:
  app:
    image: jessewebdotcom/homepage-for-tesla:latest
    volumes:
      - /mnt/user/appdata/homepage-for-tesla/bookmarks/bookmarks.json:/app/public/bookmarks.json
      - /mnt/user/appdata/homepage-for-tesla/images:/app/public/images
    ports:
      - "80:80"
    networks:
      eth0.2:
        ipv4_address: 192.168.202.70
networks:
  eth0.2:
    external: true

 

[nearcatch]

On 12/24/2023 at 2:13 PM, Caleb Bassham said:

Sorry to be a nuisance, but I can't find a lot of info for this plugin. Hoping for an update soon to get docker compose up to > v2.22 for the compose file extending/merging/importing new hotness. Any idea of when this may be?

 

Also, is there some sort of repo/homepage for this project besides this thread?

 

If you don't want to wait for the plugin to update, you can update docker-compose with this function I wrote and shared earlier in this topic. You can add it to your profile.sh.

Edited December 29, 2023 by hasown

[Caleb Bassham]

On 12/29/2023 at 4:03 PM, hasown said:

 

If you don't want to wait for the plugin to update, you can update docker-compose with this function I wrote and shared earlier in this topic. You can add it to your profile.sh.

Wow cool tool. Thanks for letting me know it exists! I'll check it out.

[Caleb Bassham]

On 12/26/2023 at 10:42 PM, aglyons said:

Haven't been working with Docker for very long and even less with compose.

 

I have a project that I was trying to get working in compose as there's no template for it is the UR app store.

 

https://github.com/JesseWebDotCom/homepage-for-tesla

 

I have a MACVLAN docker network already in place (eth0.2) for some other external services so I was trying to interpret what I found on other sites on how to add those settings to the compose YAML. I must have something wrong as it's not working at all. It doesn't crash but I can't access anything from the IP I assigned it.

 

If it's easier to create an UR template rather than crack the compose mystery then I'm all for that. Mind you, I don't have a clue how to do that yet either lol.

 

version: '3'
services:
  app:
    image: jessewebdotcom/homepage-for-tesla:latest
    volumes:
      - /mnt/user/appdata/homepage-for-tesla/bookmarks/bookmarks.json:/app/public/bookmarks.json
      - /mnt/user/appdata/homepage-for-tesla/images:/app/public/images
    ports:
      - "80:80"
    networks:
      eth0.2:
        ipv4_address: 192.168.202.70
networks:
  eth0.2:
    external: true

 

You may need to use a name without the `.` in it. So like eth02 or something. Then specify the name with the name key.

 

So:

 

networks:
	eth02:
		name: "eth0.2"
		external: true

 

[aglyons]

4 hours ago, Caleb Bassham said:

You may need to use a name without the `.` in it. So like eth02 or something. Then specify the name with the name key.

 

So:

 

networks:
	eth02:
		name: "eth0.2"
		external: true

 

 

Unraid itself named the VLAN eth0.2 when I created it in the UR network settings. Perhaps if that is creating an invalid naming convention the UR devs should be notified that this needs fixing @JorgeB.

 

I did manage to get this working with some help from a Reddit post I put up a few days after my post here.

 

While it is working, it is presented differently in the UR Docker list. The network shown is the hex ID of what appears to be a new network. But it does work.

 

Edited January 2 by aglyons

[mtongnz]

I've been playing with the code behind this plugin and have made a couple of tweaks.  I'd be interested on peoples thoughts and suggestions.

 

The first allows you to specify any .env file.  This allows use of a central env for all your compose files.

https://github.com/dcflachs/compose_plugin/pull/24

 

Update the built in unRaid image versions so the GUI correctly shows updates for containers.  I'm still testing this before doing a PR.

https://github.com/dcflachs/compose_plugin/commit/4297b46fdfeaa077f405a328ee074606363d5f9b

 

I've tweaked how the network info is shown in the GUI.  It now shows all network names with the associated IP.  This is in the dynamix.docker.manager plugin which is a core part of unRaid's GUI - so not technically part of this plugin.

You need to set the label net.unraid.docker.managed: 'composeman' for this to work (this is usually done with the overide yaml).

https://github.com/unraid/webgui/issues/1569

Here's a screenshot of it:  I've contemplated making the network name bold but I don't think it's needed.

 

 

At present, these all need to be implemented by overwriting the files in /usr/local/emhttp/plugins/.  Look at the commits to see what files need to be edited.

Edited January 3 by mtongnz

[OddMagnet]

Is the viewing of logs on a container basis bugged? Or am I doing something wrong?

 

I have a stack with Traefik and Homepage, so nothing too fancy yet.

When I try to check the logs for Traefik (clicking on Traefik's icon, then clicking "Logs"), I only get one line:

time="2024-01-05T21:51:16+01:00" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"

 

When I check the log file from Traefik however, everything is logged as it should be.

[jcbshw]

Thanks @mtongnz for the fixes! I am new to Unraid and my biggest annoyances was the docker gui when using compose.

 

Everything is mostly working as expected. Only my gluetun container keeps showing an update after doing "update stack". All other updates have disappeared.

 

 

Edited January 8 by jcbshw

[L0rdRaiden]

2 hours ago, jcbshw said:

Thanks @mtongnz for the fixes! I am new to Unraid and my biggest annoyances was the docker gui when using compose.

 

Everything is mostly working as expected. Only my gluetun container keeps showing an update after doing "update stack". All other updates have disappeared.

 

 

It's a general issue, the integration between compose and docker is not native. I guess unRAID should do some changes to accommodate better docker compose while using compose manager

[Ed Wardo]

I installed this plugin because I wanted to get musicbrainz running and there doesn't seem to be a supported App/Docker for musicbrainz. There is a supported way to install it with docker-compose

https://github.com/metabrainz/musicbrainz-docker

Musicbrainz isn't the easiest thing to get (or keep) running, but I decided to give it a shot. Its easy enough to use the plugin to create a compose stack and clone the repo, but the build process fails quite quickly.

 

I started editing Dockerfile(s) and posted an issue on github, and finally got it running.  In a nutshell, most issues were due to Unraid not allowing the cloned project files to keep their (execute) permissions. When docker-compose is building and it comes across a COPY command in a Dockerfile, it copies then without execute permissions and then one (or more) services don't start.

 

I understand the security reasons why Unraid won't all executable files in /boot/config/plugins but how do others work around this? I tried to symlink out ot a volume in /mnt/user/appdata but can't symlink either.

[itimpi]

6 hours ago, Ed Wardo said:

I understand the security reasons why Unraid won't all executable files in /boot/config/plugins but how do others work around this? I tried to symlink out ot a volume in /mnt/user/appdata but can't symlink either.

You can either copy them elsewhere during system startup in the 'go' file or using the User Scripts plugin and add execute permission or store them elsewhere in the first place (perhaps something like a folder within the musicbrainz appdata folder?).   

 

Note that if running a script via the User Scripts plugin it takes care of this for you.

[primeval_god]

9 hours ago, Ed Wardo said:

I understand the security reasons why Unraid won't all executable files in /boot/config/plugins but how do others work around this? I tried to symlink out ot a volume in /mnt/user/appdata but can't symlink either.

When creating a stack there is an advanced option (in a dropdown at the bottom) for specifying the project location. If you need to add extra files to the compose project (such as a build directory) you should select a project location on a pool or array.

Edited January 9 by primeval_god

[aglyons]

I just happened across this the other day. I haven't spun one up as yet but it looks interesting.

 

One interesting tool is is ability to convert a run command to a compose YAML.

 

https://github.com/louislam/dockge