[Solved] Dockerfile > Volume > Unraid appdata

[Forusim]

Hello,

I appologies, if my question is too nooby, but I was not able to figure out, what I am doing wrong.

I am running Unraid 6.9.2 and was mostly a docker user, but not a developer.

I want to create a custom docker, which pulls some python apps from git (not my repo) and run it there.

This python app has some hardcoded config / logs paths, which I would like to route via a volume to a persistent store on /mnt/cache/appdata/my-docker.

I tried it via Dockerfile, like it is described here, but the created files are not initialy in the volume, when I start the docker contrainer.

 

I made a sample project to show my issue.

 

Dockerfile:

FROM alpine
RUN apk add --no-cache bash
RUN mkdir /config
RUN echo "test" > /config/config.yaml
VOLUME /config
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["bash", "/entrypoint.sh"]

 

entrypoint.sh

#!/bin/bash
echo "Run in loop"
while true; do sleep 30; done;

Docker config in Unraid

 

From above I would expect on first start of the container the "config.yaml" file to be inside /mnt/cache/appdata/test.

However the file is not created in the contrainers /config and therefore is not inside  /mnt/cache/appdata/test.

When I execute the command 'echo "test" > /config/config.yaml' in the running container the file shows up in /mnt/cache/appdata/test.

 

Do I missunderstand, how the volume is supposed to work?

 

Edited May 7, 2021 by Forusim
Question answered

[itimpi]

1 minute ago, Forusim said:

Hello,

I appologies, if my question is too nooby, but I was not able to figure out, what I am doing wrong.

I am running Unraid 6.9.2 and were mostly a docker user, but not a developer.

I want to create a custom docker, which pulls some python apps from git (not my repo) and run it there.

This python app has some hardcoded config / logs paths, which I would like to route via a volume to a persistent store on /mnt/cache/appdata/my-docker.

I tried it via Dockerfile, like it is described here, but the created files are not initialy in the volume, when I start the docker contrainer.

 

I made a sample project to show my issue.

 

Dockerfile:

FROM alpine
RUN apk add --no-cache bash
RUN mkdir /config
RUN echo "test" > /config/config.yaml
VOLUME /config
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["bash", "/entrypoint.sh"]

 

entrypoint.sh

#!/bin/bash
echo "Run in loop"
while true; do sleep 30; done;

Docker config in Unraid

 

From above I would expect on first start of the container the "config.yaml" file to be inside /mnt/cache/appdata/test.

However the file is not created in the contrainers /config and therefore is not inside  /mnt/cache/appdata/test.

When I execute the command 'echo "test" > /config/config.yaml' in the running container the file shows up in /mnt/cache/appdata/test.

 

Do I missunderstand, how the volume is supposed to work?

 

what you describe is what I would expect?   The /config folder is mapped to an external location but until a file is put into that folder within the container it will not appear at the host level.  

[Forusim]

From the linked reference I understood, that you can "put" files into the volume during the build time via Dockerfile.

Somehow this only works only during the runtime, but then I do not see a point in volume declaration in Dockerfile and can directly map it on docker run.

However I want to have all required files routed to one location and map in the Unraid gui only this one /config location.

 

Edited May 6, 2021 by Forusim

[itimpi]

Not quite sure what you are trying to say :(.     The UnRaid GUI is just a graphical front-end to help with setting up the parameters to be passed to a Docker run command.    I believe the DockerFile you mention in the link is what is used to create the container that is later used with the Docker run command so is not really relevant to the UnRaid side of things.

[primeval_god]

Ah yes you have found one of the classic stumbling blocks for new docker developers. This isnt an unRAID issue rather a non-intuitive specific of the Docker volume/bind mount system.

8 hours ago, Forusim said:

From the linked reference I understood, that you can "put" files into the volume during the build time via Dockerfile.

You understood correctly however there are caveats that are not obvious. The VOLUME directive in a dockerfile tells docker that a volume needs to be mounted at the provided path. When the container is run if no volume flag is specified (-v) docker will automatically attach an anonymous volume to that path, alternatively you can use the -v flag to attach a named volume to your container. These volumes are managed by docker, stored somewhere in dockers system files (which on unRAID is in the docker image) and aren't easily accessible from outside. As the documentation suggests you can copy files to those volumes in your docker file and they will be placed into the volume at runtime. The caveat comes when we start talking about bind-mounts (which we typically use in unRAID). As an alternative to docker volumes you can use the -v flag to tell docker to mount a host directory in place of a volume (or really in place of any directory in a container). This is what you are doing when you use the path opton in unRAID's docker template. The issue you are seeing is that the files you added to the volume at build time are not made available when a folder is bind mounted into the container over that directory, this is just how docker does things. If you were to remove the path mapping in your unraid docker template and then try your test you would be able to see the yaml file which would be saved in an anonymous volume. 

 

As to how to achieve your original goal of pre-populating the config folder for your app, your best bet is to add those files to a different folder in your dockerfile and have a startup script in your container copy them over to your config volume on init if they are not already there.

 

 

Edited May 6, 2021 by primeval_god

[Forusim]

@primeval_god Thanks for clarification. Non-intuitive is an understatement to say at least.

Maybe there is a good reason, why it is working the way it is, but I find my use case reasonable.

I have now solved it, like you suggested in the entrypoint.sh.

[jxjelly]

On 5/6/2021 at 2:25 PM, primeval_god said:

As to how to achieve your original goal of pre-populating the config folder for your app, your best bet is to add those files to a different folder in your dockerfile and have a startup script in your container copy them over to your config volume on init if they are not already there.

So, could you tell me if the following is a bad idea or not?

A docker I want to run on dockerhub copies all of its working files to /sd in the Dockerfile.
And because of what you explained above by mapping /sd:%appdata%/sd the container doesn't see its files anymore because the files are covered up by the bind-mount.

If I add a Post Argument of:

mv -f /sd/* /appdata && ln -s /appdata /sd

and change my mapping to /appdata:%appdata%/sd

would it read across the link fine?

Edited October 7, 2022 by jxjelly