[Plugin] Docker Compose Manager

[mrpops2ko]

6 hours ago, primeval_god said:

I dont know what the above section does but it it seems to break things. The following works just fine with a 

version: "3.9"
services:
  tautulli:
    image: lscr.io/linuxserver/tautulli:latest
    container_name: tautulli
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=${TZ}
    ports:
      - 8181:8181
    restart: unless-stopped
    networks:
      br0:
        ipv4_address: 192.168.1.133
    volumes:
      - ${DOCKERDIR}/tautulli/config:/config

with an env file like this 

TZ=UTC
DOCKERDIR=/mnt/cache/appdata

placed in /tmp/env-test-2/

 

just checked it all again and seems to work now. 


so first i tried

 

version: "3.9"
services:
  tautulli:
    image: lscr.io/linuxserver/tautulli:latest
    container_name: tautulli
    env_file:
      - /boot/config/plugins/compose.manager/projects/test1/.env
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=${TZ}
    ports:
      - 8181:8181
    restart: unless-stopped
    networks:
      br0:
        ipv4_address: 192.168.1.133
    volumes:
      - ${DOCKERDIR}/tautulli/config:/config

networks:
  br0:
    external: true

which results in 

WARN[0000] The "TZ" variable is not set. Defaulting to a blank string. 
WARN[0000] The "DOCKERDIR" variable is not set. Defaulting to a blank string. 

 

then i tried the same but
    env_file:
      - /boot/config/plugins/compose.manager/projects/.env

which results in the same

then i tried 

 

 

using .env: /boot/config/plugins/compose.manager/projects/.env
[+] Running 1/1
 ✔ Container tautulli  Started                                                                                                   0.2s 

 

this worked without needing to declare the env_file. 

sorry about wasting your time, it seems to have always worked. Would it be possible to have a drop down menu or some kind of sticky location to the stack settings env file path? or maybe some kind of config file where we can store say multiple variables with an auto enabled: true / false flag? 

 

[OddMagnet]

13 hours ago, primeval_god said:

The cli equivalent is docker compose up, not docker pull. The images are pulled as part of the up process.

Ah, I wasn't aware that that was a limitation of the compose command.

Would be really cool if compose could also show extraction progress, so I know it's still working, just extracting something really slowly, lol

[OddMagnet]

Since my project has grown to include quite a sizeable amount of containers I noticed that docker compose struggles with that.

For context, I have a single "docker-compose.yml" which has includes to the sub-compose files that make up my project.

Ideally I'd like to be able to just use that main file to start up the whole project. 

 

By default it tries to do everything in parallel, which makes it impossible for me to start my project from scratch.

It's possible to use the --parallel option like this:

docker compose --parallel 3 up

That way docker compose can handle huge compose files or projects consisting out of a multitude of included sub-compose files

 

Suggestion: would it be possible to add that as an option in the Docker Compose Manager settings?

[primeval_god]

2 hours ago, OddMagnet said:

Since my project has grown to include quite a sizeable amount of containers I noticed that docker compose struggles with that.

For context, I have a single "docker-compose.yml" which has includes to the sub-compose files that make up my project.

Ideally I'd like to be able to just use that main file to start up the whole project. 

 

By default it tries to do everything in parallel, which makes it impossible for me to start my project from scratch.

It's possible to use the --parallel option like this:

docker compose --parallel 3 up

That way docker compose can handle huge compose files or projects consisting out of a multitude of included sub-compose files

 

Suggestion: would it be possible to add that as an option in the Docker Compose Manager settings?

The compose manager plugin is really designed with simple stacks in mind, it was not my intention to support all possible compose cli settings. Does settings COMPOSE_PARALLEL_LIMIT in the .env file have any effect?

Edited January 17 by primeval_god

[OddMagnet]

Based on the docs that should work as well (didn't test it yet)

 

I guess that's probably more reasonably, to expect people like me to just set that variable

[mtongnz]

20 hours ago, mrpops2ko said:

 

just checked it all again and seems to work now. 


so first i tried

 

version: "3.9"
services:
  tautulli:
    image: lscr.io/linuxserver/tautulli:latest
    container_name: tautulli
    env_file:
      - /boot/config/plugins/compose.manager/projects/test1/.env
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=${TZ}
    ports:
      - 8181:8181
    restart: unless-stopped
    networks:
      br0:
        ipv4_address: 192.168.1.133
    volumes:
      - ${DOCKERDIR}/tautulli/config:/config

networks:
  br0:
    external: true

which results in 

WARN[0000] The "TZ" variable is not set. Defaulting to a blank string. 
WARN[0000] The "DOCKERDIR" variable is not set. Defaulting to a blank string. 

 

then i tried the same but
    env_file:
      - /boot/config/plugins/compose.manager/projects/.env

which results in the same

then i tried 

 

 

using .env: /boot/config/plugins/compose.manager/projects/.env
[+] Running 1/1
 ✔ Container tautulli  Started                                                                                                   0.2s 

 

this worked without needing to declare the env_file. 

sorry about wasting your time, it seems to have always worked. Would it be possible to have a drop down menu or some kind of sticky location to the stack settings env file path? or maybe some kind of config file where we can store say multiple variables with an auto enabled: true / false flag? 

 

 

What you're describing is the issue I had (and that was fixed in this latest version).

 

The Docker Compose documentation doesn't really explain that specifying the env-file in the compose.yaml just sends the entire .env to the container - hence you can't access those values in the compose.yaml file.

Having the .env file in the compose.yaml directory (or specifying it in the CLI with the --env-file flag) loads the env into the compose.yaml and you can access the values.

Very confusing and annoying!

 

I have around 30 containers and wanted to be able to easily keep all the "config" values such as appdata & document directories, database passwords, domains... all in one .env file so it's easier to update, change and migrate everything.

 

This plufgin, with the Folder View plugin, make the unRaid docker management much easier.

[nearcatch]

On 1/16/2024 at 7:26 PM, primeval_god said:

The cli equivalent is docker compose up, not docker pull. The images are pulled as part of the up process.

wouldn't the equivalent be `docker compose pull SERVICENAME`? I always get extraction progress when pulling via docker compose.

Edited January 18 by nearcatch

[primeval_god]

1 hour ago, nearcatch said:

wouldn't the equivalent be `docker compose pull SERVICENAME`? I always get extraction progress when pulling via docker compose.

No the "Compose Up" button in the webui executes 'docker compose up'. Images are pulled as part of the up command not with a separate command.

Edited January 18 by primeval_god

[nearcatch]

On 1/16/2024 at 6:41 AM, L0rdRaiden said:

Could you please include docker scout cli binaries as part of compose manager?

 

I modified my docker compose update script to create a script to install docker scout on unraid. Save the script somewhere, source it in your profile.sh with `source /YOURPATHTOSCRIPT/dsupdate.source`, and then run with `dsupdate` or `dsupdate check`.

 

This works for me on a linux x86 system. If your system is different then you may need to edit line 12 to pull the proper filename from the release page.

 

#!/bin/bash
alias notify='/usr/local/emhttp/webGui/scripts/notify'
dsupdate() {
  SCOUT_LOCAL=$(docker scout version 2>/dev/null | grep version | cut -d " " -f2)
  SCOUT_LOCAL=${SCOUT_LOCAL:-"none"}
  echo Current: ${SCOUT_LOCAL}
  SCOUT_REPO=$(curl -s https://api.github.com/repos/docker/scout-cli/releases/latest | grep 'tag_name' | cut -d '"' -f4)
  if [ ${SCOUT_LOCAL} != ${SCOUT_REPO} ]; then
    dsdownload() {
      echo Repo: ${SCOUT_REPO}
      # curl -L "https://github.com/docker/scout-cli/releases/download/${SCOUT_REPO}/docker-scout_${SCOUT_REPO/v/}_$(uname -s)_$(uname -m).tar.gz" --create-dirs -o /tmp/docker-scout/docker-scout.tar.gz
      curl -L "https://github.com/docker/scout-cli/releases/download/${SCOUT_REPO}/docker-scout_${SCOUT_REPO/v/}_linux_amd64.tar.gz" --create-dirs -o /tmp/docker-scout/docker-scout.tar.gz
      tar -xf "${_}" -C /tmp/docker-scout/ --no-same-owner
      mkdir -p /usr/local/lib/docker/scout
      mv -T /tmp/docker-scout/docker-scout /usr/local/lib/docker/scout/docker-scout && chmod +x "${_}"
      rm -r /tmp/docker-scout
      cat "$HOME/.docker/config.json" | jq '.cliPluginsExtraDirs[]' 2>/dev/null | grep -qs /usr/local/lib/docker/scout 2>/dev/null
      if [ $? -eq 1 ]; then
        echo "Scout entry not found in .docker/config.json. Creating a backup and adding the scout entry."
        cp -vnT "$HOME/.docker/config.json" "$HOME/.docker/config.json.bak"
        cat "$HOME/.docker/config.json" | jq '.cliPluginsExtraDirs[.cliPluginsExtraDirs| length] |= . + "/usr/local/lib/docker/scout"' >"$HOME/.docker/config.json.tmp"
        mv -vT "$HOME/.docker/config.json.tmp" "$HOME/.docker/config.json"
      fi
      echo "Installed: $(docker scout version | grep version | cut -d " " -f2)"
      notify -e "docker-scout updater" -s "Update Complete" -d "New version: $(docker scout version | grep version | cut -d " " -f2)<br>Previous version: ${SCOUT_LOCAL}" -i "normal"
    }
    if [ -n "${1}" ]; then
      if [ "${1}" = "check" ]; then
        echo "Update available: ${SCOUT_REPO}"
        notify -e "docker-scout updater" -s "Update Available" -d "Repo version: ${SCOUT_REPO}<br>Local version: ${SCOUT_LOCAL}" -i "normal"
      else
        dsdownload
      fi
    else
      dsdownload
    fi
  else
    echo Repo: ${SCOUT_REPO}
    echo "Versions match, no update needed"
  fi
  unset SCOUT_LOCAL
  unset SCOUT_REPO
}

 

Edited January 18 by nearcatch

[nearcatch]

2 hours ago, primeval_god said:

No the "Compose Up" button in the webui executes 'docker compose up'. Images are pulled as part of the up command not with a separate command.

Ah sorry, I misunderstood. I thought the other person was asking for the cli analogue, not the plugin analogue.

[stayupthetree]

I have a small compose stack setup and it working great for the most part. Problem is every night when I do a backup of my standard unraid docker containers, the stack is restarting. I've gone into the specific plugin and set the containers made from the stack to be skipped, which setting skip works for the standard unraid docker containers, but the stack keeps restarting every night. is there something somewhere I am overlooking causing this???

[primeval_god]

4 hours ago, stayupthetree said:

I have a small compose stack setup and it working great for the most part. Problem is every night when I do a backup of my standard unraid docker containers, the stack is restarting. I've gone into the specific plugin and set the containers made from the stack to be skipped, which setting skip works for the standard unraid docker containers, but the stack keeps restarting every night. is there something somewhere I am overlooking causing this???

It doesnt sound like an with the compose manager plugin. Aside from boot and shutdown the compose manager plugin does not do anything with containers in the background (unless you click buttons). The problem is more likely with whatever you are using to backup your containers.

[rickYang]

Hello primeval_god, first of all, thank you very much for developing such a useful plugin for docker-compose. However, there is a problem that has been bothering me. I added an icon for Docker through the Edit Stack UI Labels' icon, but strangely, some icons display normally while others do not. I would like to know if there are any requirements for the icons? Or is there somewhere I can view related logs to help me troubleshoot the issue?

[primeval_god]

17 hours ago, rickYang said:

 I would like to know if there are any requirements for the icons? 

The requirements for the icons are the same as those of unRAID's Dockerman ui (whatever they are). All the compose manager does is apply a label to each container with the icon url you specify. Everything after that is built in unRAID webui functionality. There is a known issue where updating the icon url does not actually change the icon of that container (thus you kind of only have one shot at specifying the icon). The problem is with the way dockerman downloads and caches icons. I attempted a pull request for a fix but never got any traction. You can manually go in and delete the cached icons to force unRAID to redownload them.

[TheMarf]

On 10/29/2023 at 5:31 AM, nasbox said:

I've recently started using Compose Manger to orchestrate my containers in Unraid. I've used docker-compose for a long time on old servers, but moved away from it when I started using Unraid, as I preferred the GUI. I have recently shifted back to compose as my container stack is getting complicated and, frankly, it's easier to manage complicated stacks with docker compose.

 

Compose Manager has some quirks which I've come across. I wasn't able to easily find the answers to these when researching the quirks. In case it helps anyone else, I thought I'd share some of these below: 

 

1) WebUI Link & Icon: 

It is possible to automatically add WebUI Links and Icons from a docker-compose.yml, this saves you having to add them manually via the UI Labels section. Just add the following labels to your docker-compose.yml:

 

    labels:
      # Unraid Labels #
      net.unraid.docker.managed: "composeman"
      net.unraid.docker.icon: "https://raw.githubusercontent.com/binhex/docker-templates/master/binhex/images/deluge-icon.png"
      net.unraid.docker.webui: "http://[IP]:[PORT:8112]/"
      net.unraid.docker.shell: "shell"

 

I haven't managed to get the Console working (automatically closes for me), let me know if anyone has a solution for this. 

 

 

2) Creating a new custom docker network using Compose, and naming it as you wish:

You can create Custom Docker Networks using compose.yml files and Compose Manager. To do so, just add the "external: false" variable to the network. 

If you do this, you will notice that the network name ends up being "[STACK_NAME]_[THE NAME YOU CHOSE]".

You can change this behaviour by adding a "name: XXX" variable in your compose file:

networks:
  nassy:
    driver: bridge
    external: false
    name: nassy

This makes the network name become just [THE NAME YOU CHOSE].

 

3) Custom Docker Networks: Network Name in Unraid UI showing Network ID vs Network Name:

When using Custom Docker Networks which have already been created (ie. external = true), if you define the network, and use the 'networks: - nassy' command to specify what the container should use (shown below):

networks:
  nassy:
    external: true
    name: nassy
    
services:

# ----- Core Apps -----#

## watchtower ##
  
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    networks: 
      - nassy

 

You will find that that Unraid's Docker UI will show the Network ID (rather than the Network Name) in the UI:

 

This is not the case if: a) you define the network as "external: false" (having Compose create the new network, even if it's already created) or b) you just use the "network_mode: XXXX" command in docker compose: 

services:

# ----- Core Apps -----#

## watchtower ##
  
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    network_mode: "nassy"

 

Under both a) and b) you will see the correct name in the Web UI:

 

I am not sure if this is a Compose Manager or Unraid bug, but when inspecting the Containers using the Command Line, if you set "external: false" the Container's "NetworkMode" lable is the Network ID. Under both a & b described above, "NetworkMode" is the Network Name. 

 

That is all for now, I may update this as I come up with more. 

 

I hope this helps someone 

 

 

 

Hello.  Brand new to docker-compose and trying to decipher #1 about WebUI links.  I'm currently running Immich installed via docker compose as that was recommended over the CA version.  If I wanted to add the labels with the WebUI link do I need to add it to the .yml for the entire stack, or do I need to add in just one section (the server)?  Below is my current compose.

 

version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
    #   file: hwaccel.transcoding.yml 
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:afb290a0a0d0b2bd7537b62ebff1eb84d045c757c1c31ca2ca48c79536c0de82
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Here is how my immich dockers currently look thanks to FolderView. 

[primeval_god]

1 hour ago, TheMarf said:

 

Hello.  Brand new to docker-compose and trying to decipher #1 about WebUI links.  I'm currently running Immich installed via docker compose as that was recommended over the CA version.  If I wanted to add the labels with the WebUI link do I need to add it to the .yml for the entire stack, or do I need to add in just one section (the server)?  Below is my current compose.

 

Each container needs its own label.

[sonofdbn]

I managed to install a container (virtual-dsm) and it seems to be running fine. Now I'd like to set up a reverse proxy to the container via Swag. There's no prebuilt nginx config file, but I'm happy to fiddle around with this.

 

Is there anything different about a docker container installed in the "normal" unRAID way (download from Community Apps and click install) vs a container installed via docker compose manager? Particularly anything that might affect what needs to be done for a reverse proxy?

 

(I'm not well-versed in linux, docker containers or reverse proxying.)

[okkies]

hi, maybe im just a docker-compose noob. 
but could we ad a functionality to add a dockerfile to a project ?

or am i doing it wrong anyways ?dockerfile.dockerfile

 

docker-compose.yml

[primeval_god]

On 2/10/2024 at 11:48 AM, sonofdbn said:

I managed to install a container (virtual-dsm) and it seems to be running fine. Now I'd like to set up a reverse proxy to the container via Swag. There's no prebuilt nginx config file, but I'm happy to fiddle around with this.

 

Is there anything different about a docker container installed in the "normal" unRAID way (download from Community Apps and click install) vs a container installed via docker compose manager? Particularly anything that might affect what needs to be done for a reverse proxy?

 

(I'm not well-versed in linux, docker containers or reverse proxying.)

Not really except that containers run via compose can more easily be attached to multiple networks which can be useful for reverse proxy containers in some configurations.

[primeval_god]

10 hours ago, okkies said:

hi, maybe im just a docker-compose noob. 
but could we ad a functionality to add a dockerfile to a project ?

or am i doing it wrong anyways ?dockerfile.dockerfile

 

docker-compose.yml 982 B · 0 downloads

Projects that need to build their own containers are a more advanced usecase than I had intended on supporting in the webui.

[Chris H]

Hi all. 

 

New to unRAID and the Docker Compose Manager (not new to docker or containers or compose).

 

I didn't see this in the configuration but instructions might be elsewhere - how do I tell Docker Compose Manager to put the docker-compose.yml in a different directory than 

/boot/config/plugins/compose.manager/projects/

 

I apparently did this once but can't figure out how I did it so my compose files are stored there rather than where I can easily back them up.

 

Thanks in advance for any insight!

[nickniu001]

Hi all,

I found a strange issue that the Docker Compose Manager is pending in Action Centre of Unraid.

How do i deal it?

 

Thanks very much.

[trurl]

14 hours ago, Chris H said:

rather than where I can easily back them up.

You should always keep a current backup of your Unraid flash drive. 

[primeval_god]

17 hours ago, Chris H said:

Hi all. 

 

New to unRAID and the Docker Compose Manager (not new to docker or containers or compose).

 

I didn't see this in the configuration but instructions might be elsewhere - how do I tell Docker Compose Manager to put the docker-compose.yml in a different directory than 

/boot/config/plugins/compose.manager/projects/

 

I apparently did this once but can't figure out how I did it so my compose files are stored there rather than where I can easily back them up.

 

Thanks in advance for any insight!

When creating a new stack via the webui there is a dropdown for advanced options that allows you to specify the location of the project folder for that stack. 

In a future release an option to place the whole projects folder elsewhere will exist, no ETA.

[primeval_god]

3 hours ago, nickniu001 said:

I found a strange issue that the Docker Compose Manager is pending in Action Centre of Unraid.

I have no idea what this means.