[Plugin] Docker Compose Manager

[sunwind]

do I need to edit the yml file or something? so confused on how this works.

[primeval_god]

12 hours ago, sunwind said:

do I need to edit the yml file or something? so confused on how this works.

It looks like you have a stack with an empty yml file. You would need to add something to it if you expect it to do anything.

[Jarik]

Hi. Can you fix something from the issues which are addressed in the topic [6.12.4] 'DOCKER' TAB SHOWS INCORRECT\UNREADABLE INFORMATION ?

[primeval_god]

6 hours ago, Jarik said:

Hi. Can you fix something from the issues which are addressed in the topic [6.12.4] 'DOCKER' TAB SHOWS INCORRECT\UNREADABLE INFORMATION ?

At this time i have no plans to address these issues. 

Modifying the built in dockerman page is a bit of a pain and not something I want to muck around in right now. 

[sohailoo]

does anyone know how can 'i --force-recreate' manually ? where do i run the command for docker compose ?

Edited October 19, 2023 by sohailoo

[Nchina]

Why does the download from the unriad store appear like this when it starts?

[JudMeherg]

Hello fair users and awesome developers!

 

Loving the plugin for immich and paperless-ngx! There is just one thing I cannot figure out. On my previous paperless-ngx install via docker compose on ubuntu server I had a cron job running to call the paperless exporter and make a backup of all the files in paperless.

 

#!/bin/sh
cd /home/server1/paperless-ng
/usr/local/bin/docker-compose exec webserver document_exporter ../export

 

However, I cannot quite get it to work with compose manager on Unraid.

 

Whenever I try to call the webserver document_exporter, docker compose says "service webserver isn't running."

 

root@Tower:/mnt/user/backup# docker-compose -f '/boot/config/plugins/compose.manager/projects/paperless-ngx/docker-compose.yml' exec -T webserver document_exporter ../paperless-ng

Or

root@Tower:/boot/config/plugins/compose.manager/projects/paperless-ngx# docker compose exec -T webserver document_exporter /mnt/user/backup/paperless-ng

 

Above are the commands I have tried using to call the export, but the same result.

 

Is there some special trick to address a container started via the compose manager?

 

 

[Kilrah]

The containers created by compose will be on the Docker page, see what it's called and use that name.

[L0rdRaiden]

Could you add an option to select several docker compose stacks to execute de actions Start / Stop / Update. Executing each stack in parallel if it's possible or in series.

I have quite a few right now and going 1 by 1 waiting for each to go with the next one is... slow.

[JudMeherg]

14 hours ago, Kilrah said:

The containers created by compose will be on the Docker page, see what it's called and use that name.

 

Thank you for the reply! I had tried your suggestion, maybe I should have added more information.

 

Here are the results of docker compose ls

root@Tower:~# docker compose ls
NAME                STATUS              CONFIG FILES
immich              running(8)          /boot/config/plugins/compose.manager/projects/immich/docker-compose.yml,/boot/config/plugins/compose.manager/projects/immich/docker-compose.override.yml
paperless_ngx       running(3)          /boot/config/plugins/compose.manager/projects/paperless-ngx/docker-compose.yml,/boot/config/plugins/compose.manager/projects/paperless-ngx/docker-compose.override.yml

 

This is a screenshot of the stack folder:

 

And here are a few more ways I have tried to run the command:

root@Tower:~# docker compose -f '/boot/config/plugins/compose.manager/projects/paperless-ngx/docker-compose.yml' exec -T paperless_ngx-webserver-1 document_exporter /mnt/user/backup/paperless-ng
service "paperless_ngx-webserver-1" is not running

root@Tower:~# docker compose -f '/boot/config/plugins/compose.manager/projects/paperless-ngx/docker-compose.yml' exec -T webserver-1 document_expo
rter /mnt/user/backup/paperless-ng
service "webserver-1" is not running

root@Tower:~# docker compose exec -T paperless_ngx-webserver-1 document_exporter /mnt/user/backup/paperless-ng
no configuration file provided: not found

 

Here is my compose file:

 

Spoiler

# docker-compose file for running paperless from the Docker Hub.
# This file contains everything paperless needs to run.
# Paperless supports amd64, arm and arm64 hardware.
#
# All compose files of paperless configure paperless in the following way:
#
# - Paperless is (re)started on system boot, if it was running before shutdown.
# - Docker volumes for storing data are managed by Docker.
# - Folders for importing and exporting files are created in the same directory
#   as this file and mounted to the correct folders inside the container.
# - Paperless listens on port 8000.
#
# In addition to that, this docker-compose file adds the following optional
# configurations:
#
# - Instead of SQLite (default), PostgreSQL is used as the database server.
#
# To install and update paperless with this file, do the following:
#
# - Copy this file as 'docker-compose.yml' and the files 'docker-compose.env'
#   and '.env' into a folder.
# - Run 'docker-compose pull'.
# - Run 'docker-compose run --rm webserver createsuperuser' to create a user.
# - Run 'docker-compose up -d'.
#
# For more extensive installation and update instructions, refer to the
# documentation.

version: "3.4"
services:
  broker:
    image: docker.io/library/redis:7
    restart: unless-stopped
    volumes:
      - /mnt/user/appdata/paperless-ngx/redisdata:/data

  db:
    image: docker.io/library/postgres:13
    restart: unless-stopped
    volumes:
      - /mnt/user/appdata/paperless-ngx/pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: paperless
      POSTGRES_USER: paperless
      POSTGRES_PASSWORD: paperless

  webserver:
    image: ghcr.io/paperless-ngx/paperless-ngx:latest
    restart: unless-stopped
    depends_on:
      - db
      - broker
    ports:
      - 8001:8000
    healthcheck:
      test: ["CMD", "curl", "-fs", "-S", "--max-time", "2", "http://localhost:8000"]
      interval: 30s
      timeout: 10s
      retries: 5
    volumes:
      - /mnt/user/appdata/paperless-ngx/data:/usr/src/paperless/data
      - /mnt/user/documents/paperless-ngx/media:/usr/src/paperless/media
      - /mnt/user/documents/paperless-ngx/export:/usr/src/paperless/export
      - /mnt/user/paperless-import:/usr/src/paperless/consume
    environment:
      PAPERLESS_REDIS: redis://broker:6379
      PAPERLESS_DBHOST: db

# The UID and GID of the user used to run paperless in the container. Set this
# to your UID and GID on the host so that you have write access to the
# consumption directory.
      USERMAP_UID: 1000
      USERMAP_GID: 100

volumes:
  data:
  media:
  pgdata:
  redisdata:

 

I am still not sure what I am missing.

[Jarik]

6 hours ago, JudMeherg said:

I am still not sure what I am missing.

If you use docker compose to call your containers than you should use service's names instead of container's names from the docker-compose.yml file. In your case:

docker compose -f '/boot/config/plugins/compose.manager/projects/paperless-ngx/docker-compose.yml' exec -T webserver document_exporter /mnt/user/backup/paperless-ng

 

or simple use docker directly with the name of your container:

 

docker exec -it paperless_ngx-webserver-1 document_exporter /mnt/user/backup/paperless-ng

 

 

Edited October 24, 2023 by Jarik

[Kilrah]

Also since that command is executed in the container then the path to backup needs to be a container path. /mnt/user/backup/paperless-ng doesn't exist in the container, you'd need to add a mapping from that to whatever folder in the webserver container, then reference the container folder in the command. Or use one of the existing locations, seems in your old setup that's what the "export" one was used for.

Edited October 24, 2023 by Kilrah

[JudMeherg]

16 hours ago, Jarik said:

docker exec -it paperless_ngx-webserver-1 document_exporter /mnt/user/backup/paperless-ng

 

 

 

For some reason "docker exec -it paperless_ngx-webserver-1 document_exporter ../export".

 

I guess the -i flag made all the difference.

 

13 hours ago, Kilrah said:

Also since that command is executed in the container then the path to backup needs to be a container path. /mnt/user/backup/paperless-ng doesn't exist in the container, you'd need to add a mapping from that to whatever folder in the webserver container, then reference the container folder in the command. Or use one of the existing locations, seems in your old setup that's what the "export" one was used for.

 

Thank you for pointing this point! You are right it should point to the export folder mounted inside the container.

[nasbox]

I've recently started using Compose Manger to orchestrate my containers in Unraid. I've used docker-compose for a long time on old servers, but moved away from it when I started using Unraid, as I preferred the GUI. I have recently shifted back to compose as my container stack is getting complicated and, frankly, it's easier to manage complicated stacks with docker compose.

 

Compose Manager has some quirks which I've come across. I wasn't able to easily find the answers to these when researching the quirks. In case it helps anyone else, I thought I'd share some of these below: 

 

1) WebUI Link & Icon: 

It is possible to automatically add WebUI Links and Icons from a docker-compose.yml, this saves you having to add them manually via the UI Labels section. Just add the following labels to your docker-compose.yml:

 

    labels:
      # Unraid Labels #
      net.unraid.docker.managed: "composeman"
      net.unraid.docker.icon: "https://raw.githubusercontent.com/binhex/docker-templates/master/binhex/images/deluge-icon.png"
      net.unraid.docker.webui: "http://[IP]:[PORT:8112]/"
      net.unraid.docker.shell: "shell"

 

I haven't managed to get the Console working (automatically closes for me), let me know if anyone has a solution for this. 

 

 

2) Creating a new custom docker network using Compose, and naming it as you wish:

You can create Custom Docker Networks using compose.yml files and Compose Manager. To do so, just add the "external: false" variable to the network. 

If you do this, you will notice that the network name ends up being "[STACK_NAME]_[THE NAME YOU CHOSE]".

You can change this behaviour by adding a "name: XXX" variable in your compose file:

networks:
  nassy:
    driver: bridge
    external: false
    name: nassy

This makes the network name become just [THE NAME YOU CHOSE].

 

3) Custom Docker Networks: Network Name in Unraid UI showing Network ID vs Network Name:

When using Custom Docker Networks which have already been created (ie. external = true), if you define the network, and use the 'networks: - nassy' command to specify what the container should use (shown below):

networks:
  nassy:
    external: true
    name: nassy
    
services:

# ----- Core Apps -----#

## watchtower ##
  
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    networks: 
      - nassy

 

You will find that that Unraid's Docker UI will show the Network ID (rather than the Network Name) in the UI:

 

This is not the case if: a) you define the network as "external: false" (having Compose create the new network, even if it's already created) or b) you just use the "network_mode: XXXX" command in docker compose: 

services:

# ----- Core Apps -----#

## watchtower ##
  
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    network_mode: "nassy"

 

Under both a) and b) you will see the correct name in the Web UI:

 

I am not sure if this is a Compose Manager or Unraid bug, but when inspecting the Containers using the Command Line, if you set "external: false" the Container's "NetworkMode" lable is the Network ID. Under both a & b described above, "NetworkMode" is the Network Name. 

 

That is all for now, I may update this as I come up with more. 

 

I hope this helps someone 

 

 

[primeval_god]

On 10/29/2023 at 5:31 AM, nasbox said:

    labels:
      # Unraid Labels #
      net.unraid.docker.managed: "composeman"
      net.unraid.docker.icon: "https://raw.githubusercontent.com/binhex/docker-templates/master/binhex/images/deluge-icon.png"
      net.unraid.docker.webui: "http://[IP]:[PORT:8112]/"
      net.unraid.docker.shell: "shell"

 

I haven't managed to get the Console working (automatically closes for me), let me know if anyone has a solution for this. 

I believe the valid options for this are "bash" and "sh", shell is not.

[primeval_god]

On 10/29/2023 at 5:31 AM, nasbox said:

3) Custom Docker Networks: Network Name in Unraid UI showing Network ID vs Network Name:

 

I am not sure if this is a Compose Manager or Unraid bug, but when inspecting the Containers using the Command Line, if you set "external: false" the Container's "NetworkMode" lable is the Network ID. Under both a & b described above, "NetworkMode" is the Network Name. 

It is an unRAID issue since composeman does not modify the dockerman containers page. More specifically docker compose changed something about the way network naming is handled which throws a wrench in the way dockerman extracts network names for some network types. A fix is not going to be a high priority as it only effects containers created by compose and composeman is a third party plugin.

[L0rdRaiden]

more elegant now with the labels in the compose

 

###############################################################

# Adguard Home 6

###############################################################

 

# Common settings #############################################

 

x-default: &config

  restart: unless-stopped

  cpuset: 12,13,14,15

  security_opt:

    - no-new-privileges:true

  #dns:

  #  - 10.10.50.5

  #  - 10.10.50.6

x-labels: &labels

  com.centurylinklabs.watchtower.enable: "true"

  net.unraid.docker.managed: "composeman"

  net.unraid.docker.shell: "sh"

  net.unraid.docker.icon: "https://raw.githubusercontent.com/SiwatINC/unraid-ca-repository/master/icons/adguard.png"

 

# Services ####################################################

 

services:

 

# Adguard Home 6 Internal ######################################

 

  adguardhomeint6:

    container_name: AdGuardHomeINT6

    image: adguard/adguardhome

    <<: *config

    networks:

      eth1:

        ipv4_address: 10.10.40.6

    ports:

      - "53:53/tcp" # Plain DNS

      - "53:53/udp" # Plain DNS

    # - "67:67/udp" # DHCP

    # - "68:68/tcp" # DHCP

    # - "68:68/udp" # DHCP

      - "80:80/tcp" # HTTPS/DNS-over-HTTPS server & admin panel

      - "443:443/tcp" # HTTPS/DNS-over-HTTPS server & admin panel

      - "443:443/udp" # HTTPS/DNS-over-HTTPS server & admin panel

      - "3000:3000/tcp" # admin panel

    # - "853:853/tcp" # DNS-over-TLS server

    # - "5443:5443/tcp" # DNSCrypt server

    # - "5443:5443/udp" # DNSCrypt server

    volumes:

      - /mnt/services/docker/AdguardINT6/config:/opt/adguardhome/conf

      - /mnt/services/docker/AdguardINT6/workingdir:/opt/adguardhome/work

    labels:

      <<: *labels

      net.unraid.docker.webui: "http://10.10.40.6:80/"

 

# Adguard Home 6 DMZ ###########################################

 

  adguardhomedmz6:

    container_name: AdGuardHomeDMZ6

    image: adguard/adguardhome

    <<: *config

    networks:

      eth2:

        ipv4_address: 10.10.50.6

    ports:

      - "53:53/tcp" # Plain DNS

      - "53:53/udp" # Plain DNS

    # - "67:67/udp" # DHCP

    # - "68:68/tcp" # DHCP

    # - "68:68/udp" # DHCP

      - "80:80/tcp" # HTTPS/DNS-over-HTTPS server & admin panel

      - "443:443/tcp" # HTTPS/DNS-over-HTTPS server & admin panel

      - "443:443/udp" # HTTPS/DNS-over-HTTPS server & admin panel

      - "3000:3000/tcp" # admin panel

    # - "853:853/tcp" # DNS-over-TLS server

    # - "5443:5443/tcp" # DNSCrypt server

    # - "5443:5443/udp" # DNSCrypt server

    volumes:

      - /mnt/services/docker/AdguardDMZ6/config:/opt/adguardhome/conf

      - /mnt/services/docker/AdguardDMZ6/workingdir:/opt/adguardhome/work

    labels:

      <<: *labels

      net.unraid.docker.webui: "http://10.10.50.6:80/"

 

# Networks ####################################################

 

networks:

  eth1:

    name: eth1

    external: true

  eth2:

    name: eth2

    external: true

Edited October 30, 2023 by L0rdRaiden

[SamDu]

Super low importance, but there's a typo in the description of this plugin. It says:

"This plugin installs the docker compose package on you unRAID..."

It should read:

"This plugin installs the docker compose package on your unRAID..."

[Droppisalt]

Trying to "compose up" my docker-compose.yml and getting this error: unknown shorthand flag: 'f' in -f

 

EDIT: also getting this error. Not sure what's up with my permissions:

Invalid Plugins:
  compose     failed to fetch metadata: fork/exec /root/.docker/cli-plugins/docker-compose: permission denied

 

Any clue on what is going on?

Edited November 4, 2023 by Droppisalt

[SOULV1CE]

On 10/29/2023 at 4:31 AM, nasbox said:

1) WebUI Link & Icon: 

 

It is possible to automatically add WebUI Links and Icons from a docker-compose.yml, this saves you having to add them manually via the UI Labels section. Just add the following labels to your docker-compose.yml:

 

 

    labels:
      # Unraid Labels #
      net.unraid.docker.managed: "composeman"
      net.unraid.docker.icon: "https://raw.githubusercontent.com/binhex/docker-templates/master/binhex/images/deluge-icon.png"
      net.unraid.docker.webui: "http://[IP]:[PORT:8112]/"
      net.unraid.docker.shell: "shell"

 

Thanks for this! I think it might not be necessary to have to define net.unraid.docker.managed: "composeman" because I saw it already existed by default in the docker-compose.override.yml that gets created for a project.

 

Also I just want to mention that it works fine with using environment variables to define these as well - my setup is working correctly like this with the actual values defined in a .env file:

net.unraid.docker.shell: ${LABEL_SHELL}
net.unraid.docker.icon: ${LABEL_ICON_CODE-SERVER}
net.unraid.docker.webui: ${LABEL_WEBUI_CODE-SERVER}

 

 

On 10/29/2023 at 4:31 AM, nasbox said:

 

This is not the case if: a) you define the network as "external: false" (having Compose create the new network, even if it's already created) or b) you just use the "network_mode: XXXX" command in docker compose: 

services:

# ----- Core Apps -----#

## watchtower ##
  
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    network_mode: "nassy"

 

Under both a) and b) you will see the correct name in the Web UI:

 

Setting external: false did not work for me for an already created network in Unraid - when I tried to do this I got the error:

Quote

WARN[0000] a network with name skynet exists but was not created by compose.
Set `external: true` to use an existing network
network skynet was found but has incorrect label com.docker.compose.network set to ""

However using network_mode: "skynet" was successful.

[SOULV1CE]

Is redirecting to https when inputting a URL with http a known issue for the WebUI?

I saw one previous comment asking about this back in January but havent seen anything since.

 

My example:

 

I put in this for WebUI label: http://[IP]:[PORT:8443]/

When I click WebUI on the container it goes to https://10.0.0.100:8443/

[primeval_god]

10 hours ago, SOULV1CE said:

Is redirecting to https when inputting a URL with http a known issue for the WebUI?

I saw one previous comment asking about this back in January but havent seen anything since.

 

My example:

 

I put in this for WebUI label: http://[IP]:[PORT:8443]/

When I click WebUI on the container it goes to https://10.0.0.100:8443/

The answer is in the comment directly after the one you linked. It has nothing to do with the link itself. The webui isnt using the updated label.

[momsi]

Is it somehow possible to have the containers of a compose stack to not show up in the unraid docker webui?

[primeval_god]

21 hours ago, momsi said:

Is it somehow possible to have the containers of a compose stack to not show up in the unraid docker webui?

No not at this time.

[L0rdRaiden]

@primeval_god

If I add my the "compose manager" labels to my docker compose file, should I delete the docker-compose.override.yml? is there a way to override this file and use exclusively whatever I set in the compose label?

 

I see in some cases inconsistencies where the content of this file does not match the labels defined for it in docker compose.

 

I am using labels like this

lables:

      - "com.centurylinklabs.watchtower.enable=true"

      - "net.unraid.docker.managed=composeman"

      - "net.unraid.docker.shell=sh"

      - "net.unraid.docker.icon=https://raw.githubusercontent.com/ibracorp/unraid-templates/master/icons/traefik.png"

      - "net.unraid.docker.webui=https://traefik.dsaasddsa.com.es/dashboard/#/"

Edited December 4, 2023 by L0rdRaiden