[Plugin] Docker Compose Manager


Recommended Posts

8 hours ago, Glassed Silver said:

👀

 

Are you gonna attempt to hack some way into this so I can manage compose'd containers in DockerMan? Or what is that reference to WebUI supposed to mean in the template description?

Also interested in how the WebUI integration will will.

 

Also, curious if it would be possible to support compose v3

Link to comment
2 hours ago, primeval_god said:

I am not certain what compose v3 is. This plugin installs compose v2.0.1 https://github.com/docker/compose which to my knowledge is the latest release.

 

Sorry, I meat is it gonna support v3 of the docker-compose.yaml files? For a while portainer only supported v2 of the .yaml files and most things you see online use a v3 version and they don't ALWAYS convert well.

Link to comment
10 minutes ago, neruve said:

 

Sorry, I meat is it gonna support v3 of the docker-compose.yaml files? For a while portainer only supported v2 of the .yaml files and most things you see online use a v3 version and they don't ALWAYS convert well.

Any gui component i implement will be little more than a text editor for compose files. As for what versions of compose files are supported by the version of compose i have baked in I think the answer is all, so long as the version of Docker on unraid also supports that version of the compose spec. https://docs.docker.com/compose/compose-file/compose-versioning/ If i understood that fairly confusing page correctly the latest version of the compose spec has been supported by compose since v1.27.0

Link to comment
On 10/4/2021 at 11:10 AM, neruve said:

 

Sorry, I meat is it gonna support v3 of the docker-compose.yaml files? For a while portainer only supported v2 of the .yaml files and most things you see online use a v3 version and they don't ALWAYS convert well.

 

FYI, the current version of Docker Compose supports both v2 and v3. The recommended way to write compose files is to omit the "version" tag at the top entirely. Compose will then read the yml and interpret all v2 and v3 features.

Link to comment
On 10/4/2021 at 11:42 AM, Glassed Silver said:

Are you gonna attempt to hack some way into this so I can manage compose'd containers in DockerMan? Or what is that reference to WebUI supposed to mean in the template description?

If you use the docker folder plugin you can already add your compose commands to the parent folder i.e. compose up, compose down. as context menu items like the native Unraid UI.
With the label support coming in 6.10, it should start to integrate nicely with the UI.

  • Like 1
Link to comment
On 10/3/2021 at 2:22 PM, primeval_god said:

Compose Manager

Beta Release!

This plugin installs docker compose 2.0.1 and compose switch 1.0.2.

Use "docker compose" or "docker-compose" from the command line.

See https://docs.docker.com/compose/cli-command/ for additional details.

Install via Community Applications

 

Future Work:

A simple unRAID web-gui integration is in the works.

 

I've installed and rebooted but all I get is the following when I try running "docker compose" or "docker-compose":
 

docker: 'compose' is not a docker command.

 

I see this when I run "docker"

[...]
Management Commands:
  builder     Manage builds
  compose*    Docker Compose (Docker Inc., v2.0.1)
  config      Manage Docker configs
  container   Manage containers
  context     Manage contexts
[...]

 

 

EDIT: Nevermind, got it working. I'd earlier manually created the cli-plugins folder in my ~/.docker in an attempt to install Compose v2 manually, and it seemed to cause some issues. Deleted it and everything works now.

Edited by hasown
Link to comment
On 10/6/2021 at 1:53 AM, tjb_altf4 said:

If you use the docker folder plugin you can already add your compose commands to the parent folder i.e. compose up, compose down. as context menu items like the native Unraid UI.
With the label support coming in 6.10, it should start to integrate nicely with the UI.

Hmm, I'm hesitant to experiment and run into troubles and such.

 

I'll wait for a spaceinvaderone video for this one, mostly because once he does one, it'll be a lot more used and hence (implicitly) expected functionality that Limetech might have to test for as well with future releases.

 

I'd rather ride it slowly and know that docker-compose works PROPERLY before I fully commit to it.

 

Good news for sure though!

Link to comment
On 10/3/2021 at 11:22 PM, primeval_god said:

Compose Manager

Beta Release!

This plugin installs docker compose 2.0.1 and compose switch 1.0.2.

Use "docker compose" or "docker-compose" from the command line.

See https://docs.docker.com/compose/cli-command/ for additional details.

Install via Community Applications

 

Future Work:

A simple unRAID web-gui integration is in the works.

@primeval_god

Thank you for producing this plugin. Works fine for me; new compose is significantly faster and just better.

 

Despite the Switch, there are some changes to the yaml format that need modification. For instance gpu pass through; however, it's all in the excellent documentation.

Edited by juan11perez
Link to comment
On 10/11/2021 at 10:05 PM, Glassed Silver said:

Hmm, I'm hesitant to experiment and run into troubles and such.

 

I'll wait for a spaceinvaderone video for this one, mostly because once he does one, it'll be a lot more used and hence (implicitly) expected functionality that Limetech might have to test for as well with future releases.

 

I'd rather ride it slowly and know that docker-compose works PROPERLY before I fully commit to it.

 

Good news for sure though!

If you are afraid to get in truble with your main system, check out this video from IBRACORP. Based on that you can create a VM with unraid trial and experiment as much as you need before putting anything on production.

 

I love SpaceInvader with all my heart, but IBRACOORP is doing super job and they are also focused on unRaid only. 

 

 

  • Like 1
Link to comment
On 10/13/2021 at 9:16 PM, enerdo said:

If you are afraid to get in truble with your main system, check out this video from IBRACORP. Based on that you can create a VM with unraid trial and experiment as much as you need before putting anything on production.

 

I love SpaceInvader with all my heart, but IBRACOORP is doing super job and they are also focused on unRaid only. 

 

 

 

No urgent need so I'd rather just wait. Got plenty of other stuff to take care of, so I'm gonna skip spending time on something that will for sure be figured out by someone else and that I'm gonna learn about relatively soon, so... yeah.

 

But thank you for the feedback. Also, not a bad thing to have more unRAID learning material at your disposal. :)

Edited by Glassed Silver
typo
Link to comment
  • 3 weeks later...

@primeval_god

suggestion for "GUI".

I just prepped a dockerdev container for my unraid compose management as a supplement for your plugin. Why not include just a vscode server container for editing the files.

I have a working compose file that setups a vscode server container based on linuxservers/code-server with docker, docker compose with same version as you provide in the plugin and some suggar.

 

Features:

  • vscode extension: docker
    • support for managing/debug docker via vscode docker explorer.
    • support for compose up down commands.
    • synatx highlighting and help for compose and docker files
  • vscode extension: dotenv
    for all the people using .env files
  • bash completion in the vscode terminal for docker and docker-compose

I'm happy if you would integrate that with your plugin, I assume you would have less effort to bring this up and running and support with more features than self dev a gui.

Maybe it needs a bit of cleanup und convert to a template but should be almost 1:1 mapping.

 

Explanation:

The docker file builds the sugar into the code-server image, they have a "mod" for integrating docker into their images, but hard to maintain with custom version etc.

The settings.json of the workspace is currently manually created after container is running... but could be easy integrated into the docker image as base. And maybe I missed some docker command settings for all possible cases. and currently also hardcoded for using .env file in these commands (couldn't get it to pick it up automatically).

Also the image needs some cleanups like removing apt cache etc. to make it smaller.

 

dockerfile for codeserver with sugar:

Spoiler
ARG DOCKER_VERSION=20.10.5

# get docker binaries
FROM docker:${DOCKER_VERSION} AS docker

# runtime image
FROM lscr.io/linuxserver/code-server

# copy docker binary
COPY --from=docker /usr/local/bin/docker /usr/local/bin/

ARG DOCKER_COMPOSE_VERSION="v2.0.1"
ARG DOCKER_COMPOSE_SWITCH_VERSION="v1.0.2"
ARG DOCKER_COMPOSE_COMPLETION_VERSION="1.29.2"
ARG DOCKER_VERSION

# install compose
RUN mkdir -p /usr/local/lib/docker/cli-plugins && \
    curl -fSL https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-linux-x86_64 -o /usr/local/lib/docker/cli-plugins/docker-compose && \
    chmod +x /usr/local/lib/docker/cli-plugins/docker-compose
# install compose switch
RUN curl -fSL https://github.com/docker/compose-switch/releases/download/${DOCKER_COMPOSE_SWITCH_VERSION}/docker-compose-linux-amd64 -o /usr/local/bin/compose-switch && \
    chmod +x /usr/local/bin/compose-switch

# alias for legacy docker-compose
RUN update-alternatives --install /usr/local/bin/docker-compose docker-compose /usr/local/bin/compose-switch 99

# install bash completion
RUN apt-get update && apt-get install bash-completion
RUN curl \
    -fSL https://raw.githubusercontent.com/docker/cli/v${DOCKER_VERSION}/contrib/completion/bash/docker \
    -o /etc/bash_completion.d/docker
RUN curl \
    -fSL https://raw.githubusercontent.com/docker/compose/${DOCKER_COMPOSE_COMPLETION_VERSION}/contrib/completion/bash/docker-compose \
    -o /etc/bash_completion.d/docker-compose

# enable bash completion globally (we could uncomment existings lines but this is harder to automate)
RUN echo "" >>/etc/bash.bashrc && \
    echo "# enable bash completion in interactive shells" >>/etc/bash.bashrc && \
    echo "if ! shopt -oq posix; then" >>/etc/bash.bashrc && \
    echo "  if [ -f /usr/share/bash-completion/bash_completion ]; then" >>/etc/bash.bashrc && \
    echo "    . /usr/share/bash-completion/bash_completion" >>/etc/bash.bashrc && \
    echo "  elif [ -f /etc/bash_completion ]; then" >>/etc/bash.bashrc && \
    echo "    . /etc/bash_completion" >>/etc/bash.bashrc && \
    echo "  fi" >>/etc/bash.bashrc && \
    echo "fi" >>/etc/bash.bashrc

# install extensions
RUN mkdir -p  /config/custom-cont-init.d && \
    echo "code-server --user-data-dir /config/data --extensions-dir /config/extensions --install-extension ms-azuretools.vscode-docker" >/config/custom-cont-init.d/extensions.sh && \
    echo "code-server --user-data-dir /config/data --extensions-dir /config/extensions --install-extension mikestead.dotenv" >>/config/custom-cont-init.d/extensions.sh && \
    chmod +x /config/custom-cont-init.d/extensions.sh

 

 

docker-compose:

Spoiler
---
services:

  code-server:
    build:
      context: .
      dockerfile: code-server.dockerfile
      args:
        DOCKER_COMPOSE_VERSION: "v2.0.1"
        DOCKER_COMPOSE_COMPLETION_VERSION: "1.29.2"
        DOCKER_COMPOSE_SWITCH_VERSION: "v1.0.2"
        DOCKER_VERSION: "20.10.5"
    image: dockerdev-code-server:latest
    restart: unless-stopped
    ports:
      - 8443:8443
    environment:
      TZ: ${TIMEZONE_TZ:?}
      PUID: 0
      PGID: 0
      UMASK: "000"
      #PROXY_DOMAIN: ${DOMAIN_NAME:?}
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - code-server-config:/config
    labels:
      traefik.enable: true
      traefik.http.routers.dockerdev.rule: Host(`dockerdev.${DOMAIN_NAME:?}`)
      net.unraid.docker.webui: https://dockerdev.${DOMAIN_NAME:?}
    networks:
      - proxy

volumes:
  code-server-config:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /mnt/user/docker_ssd_cached/dockerdev/code-server/config

networks:
  proxy:
    name: proxy
    external: true

 

vscode workspace settings file:

Spoiler

 

{
    "docker.enableDockerComposeLanguageService": true,
    "docker.commands.composeUp": [
        {
            "label": "Compose Up",
            "template": "docker-compose --env-file .env ${configurationFile} up ${detached} ${build}"
        }
    ],
    "docker.commands.composeUpSubset": [
        {
            "label": "Compose Up",
            "template": "docker-compose --env-file .env ${profileList} ${configurationFile} up ${detached} ${build} ${serviceList}"
        }
    ],
    "docker.commands.composeDown": [
        {
            "label": "Compose Down",
            "template": "docker-compose --env-file .env ${configurationFile} down"
        },
    ],
    "editor.formatOnSave": true
}

 

Edited by spali
Link to comment
7 hours ago, hasown said:

Maybe put in a way for people to choose whether to use the stable version of docker-compose or pre-releases. Stable is 2.0.1 right now, but pre-release is 2.1.0.

In the compose you can change the variables in the compose file for the versions. Just make sure to use the right notation, because they are github tags/releases some of them with "v" prefixed. And the DOCKER_VERSION argument is also referenced for the docker image used to pick the docker binary.

 

But I'm sure if primeval_god decides to integrate it in his plugin, he will make it more polished and configurable.

 

Link to comment
  • 2 weeks later...

i see what you're trying to do here and wonder if you're taking on more complexity than is necessary?

 

the frustration i have with the docker ui in unraid is that there is a lot of manual clicking and form filling to add containers. this is especially annoying when you see projects with demo docker-compose.yaml files just sitting there ready to use. perhaps a good solution would be to create a docker compose yaml parser, which imports the docker compose yaml into an unraid docker configuration, which you can then start in the ui as if you did all the manual clicking and form filling legitimately?

 

bonus points for changing the existing basic/advanced view toggle in the container configuration UI to include 'edit as docker compose yaml'!

 

do you see the problem the same way i do?

Link to comment
4 minutes ago, marengaz said:

 

the frustration i have with the docker ui in unraid is that there is a lot of manual clicking and form filling to add containers. this is especially annoying when you see projects with demo docker-compose.yaml files just sitting there ready to use.

The issue here is that Unraid runs in RAM, so "normal" locations for storage that work for 99.9% of the linux world by default will either be in RAM or otherwise not valid. You must manually assign storage locations to valid spots, which can vary based on how each specific Unraid setup has been customized.

Link to comment

im not saying it would work 100% of the time, volume mount locations are always something i check before running a new compose stack.

 

im still trialing unraid and was brought here by wanting to run an instance of apache superset (quite a daunting amount of manual clicking and form filling). im just saying that it would be great if there was an interface to get a compose stack into unraid in a less painful way.

 

i acknowledge that running docker compose cli and portainer would give me a nice solution, but that kinda defeats the point of unraid's ui!

Link to comment

I think the main problem is that we have at least two use-cases for people using docker compose.

1. the people which just want no special features, but reuse compose configs.

2. the people which want most, if not all compose features, which usually also create their own compose files. 

 

I count myself to the second group. So that's why this kind of "compose - dev - container" works perfectly for me.

 

What I don't get is what's the plan with the step with the 6.10 feature of the container labels for just icons and webgui url.

I don't know their thoughts and plans, but I would be consequent and just replace the templates with compose-files. They would only need to support the features they want in the GUI and just but unraid specific stuff in labels (as with the comming changes for icon and webgui url).

Even if they would decide to support only one container in the GUI per compose file. It would be a lot more compatible with custom compose files, especially that (at least my impression) a lot of users here use them. It would also allow for people in the second group to to at least basic stuff in the GUI with their custom compose files.

 

I see the point with the storage location for volumes... but that's something a "power-user" anyway has to deal with. A so called compose based template would just predefine the unraid paths, as the current templates do. But it would be an easy adaption from a compose file that I copy&paste from the internet.

 

Maybe the labels they implement in 6.10 is only a small step in their plan towards this direction, who knows....

 

 

 

 

 

Link to comment

So far as i know there are no plans to replace the unRAIDs template format with compose yaml files. As for my goal with this plugin, it is a lot simpler than people are speculating. The interface I am working on (very slowly i know but I have been busy lately) is little more than a way to create and edit compose files in the webui, plus a few buttons to trigger compose up and compose down. Basically the user.scripts plugin for compose files.

 

Personally I think that docker compose will always be secondary to templates on unRAID. There are a few useful application stacks that are just hard to build up with unRAID templates however for the most part the applications that unRAID users use are single container and easily added via unRAIDs template interface. 

Link to comment
3 hours ago, spali said:

6.10 feature of the container labels for just icons and webgui url

i upgraded my unraid to 6.10rc2 and did some digging. looks like this is what they meant in the release notes (spoilers: its nothing like what we want, but is a slight quality of life improvement!)

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.