Recommended docker setup


Nem

Recommended Posts

Looking for some advice on a recommended way to set up my server, particularly as it pertains to docker setup.

 

I have:

- a 64gb SSD as cache

- 3TB parity

- 3TB data drive for my media

- 3TB data drive for backups and other random things

 

I'm planning on running the following:

- plex

- mariadb

- couch potato

- sonarr

- deluge + openvpn

- kodi/xbmc

- steam os

 

So, with docker, I understand there is a docker image file that needs to go somewhere, and an appdata directory? I have a few questions:

1. I read in the docks that its typical to put the docker image on the cache drive, but what about appdata? In my mind, cache = temporary, so it feels a bit strange to put appdata on the cache maybe?

2. Am I correct in thinking that appdata should not be a share, but rather should be some root level directory, wherever you decide to put it?

3. Any downsides to putting appdata on slower (5400rpm WD red) data drives? What about the docker image on a slower drive?

4. Is it more typical to just have a single appdata for all applications? Or to have different ones, one for each application you intend to use?

5. I'm assuming that if you have the above applications running 24/7, the docker image/appdata will be in constant use. Therefore, if I put those files on my "random stuff" data drive, will it just keep that drive constantly in use? maybe causing some slowdown for either the applications or transferring files to/from the shares on that drive?

6. Does putting docker/appdata on the cache drive basically then require the use of a cache pool in case the cache SSD goes down and taking all the applications with it?

 

I'm trying to be conservative with my use of the cache drive due to it only being 64gb, but I don't want to slow down the server by dumping everything on the slower WD red drives either.

 

How would you suggest I go about organizing the docker related files?

Link to comment

1. I read in the docks that its typical to put the docker image on the cache drive, but what about appdata? In my mind, cache = temporary, so it feels a bit strange to put appdata on the cache maybe?

The cache drive is the one that has the highest performance as writes to it do not involve parity which is why it is typically used to host applications.

2. Am I correct in thinking that appdata should not be a share, but rather should be some root level directory, wherever you decide to put it?
All top level directories automatically become shares.    If you have the appdata on the cache drive then you want to make sure that is set to be "Cache only" to stop mover trying to move its contents to the data drives.

3. Any downsides to putting appdata on slower (5400rpm WD red) data drives? What about the docker image on a slower drive?
It should work fine.  You just might end up with lower performance.

4. Is it more typical to just have a single appdata for all applications? Or to have different ones, one for each application you intend to use?

Most users have one appdata for all applications and then give each application a sub-folder within that.

5. I'm assuming that if you have the above applications running 24/7, the docker image/appdata will be in constant use. Therefore, if I put those files on my "random stuff" data drive, will it just keep that drive constantly in use? maybe causing some slowdown for either the applications or transferring files to/from the shares on that drive?

Yes it will keep that drive in constant use, and almost certainly the parity drive as well if (as is likely) any writes are occurring.  this is likely to affect overall performance.

6. Does putting docker/appdata on the cache drive basically then require the use of a cache pool in case the cache SSD goes down and taking all the applications with it?

That is really up to you.  The configuration information for dockers is held on the flash drive, so it is normally easy to re-instate dockers and their configuration if the cache drive fails (or is corrupted for any reason).  Many applications are simply downloading data and then writing the results to a user share so that it ends up on the data drives for longer term storage.

 

With such a small SSD it mioght be worth dedicating it to running your apps, and nor for caching writes to user shares.  This will give slower writes to User shares (for most people they area relatively small amount of the overall I/O), but may give better overall system performance.

Link to comment

That is really up to you.  The configuration information for dockers is held on the flash drive, so it is normally easy to re-instate dockers and their configuration if the cache drive fails (or is corrupted for any reason).  Many applications are simply downloading data and then writing the results to a user share so that it ends up on the data drives for longer term storage.

 

Most containers ask for a config path. I assume this is config information for the application within the docker container. Should this path point to the cache on the host? Or one of the data drives? Or the flash?

 

It's easy enough to reinstall a a docker container if the cache drive dies, but some applications might have more complex settings. For example, with deluge theres the entire preferences pane you need to set up for the application itself. I assume those settings are stored wherever you point your config file to during docker container installation.

 

I guess my specific questions are about where different types of docker information are stored:

 

You mentioned config info is held on the flash drive. Is this just template information for how to do the initial docker container setup (i.e. when you add a container on unraid)?

 

I believe the container image is stored in the docker image file when docker was first started. I assume this doesn't contain any actual config info?

 

The the config for each application (e.g. deluge) are stored wherever you point /config to during setup. I assume this is pretty small and would not actually benefit from a faster drive (i.e. cache) and can/should be stored on a data drive if you want to retain the application config info if the cache drive dies?

Link to comment

I guess my specific questions are about where different types of docker information are stored:

 

You mentioned config info is held on the flash drive. Is this just template information for how to do the initial docker container setup (i.e. when you add a container on unraid)?

Yes.  It will also contain your customised templates with the settings for a particular docker.  This makes it easy to restore a docker if for any reason you have to "blow-away" its container.

 

I believe the container image is stored in the docker image file when docker was first started. I assume this doesn't contain any actual config info?

This is best practise although nothing technically stops you storing config information there.  The basic idea is to try and keep the docker containers in the docker image file as static information with all dynamic information held externally.  This stops the container image from continually growing in size.    It also make it much easier if for any reason the docker image file needs rebuilding.

 

The the config for each application (e.g. deluge) are stored wherever you point /config to during setup. I assume this is pretty small and would not actually benefit from a faster drive (i.e. cache) and can/should be stored on a data drive if you want to retain the application config info if the cache drive dies?

This really varies by docker.  Some docker images store a LOT of information under the config location and also are updating it dynamically.  You can always set up a cache pool to provide resilience to this data without involving the data drives.
Link to comment

if, for all of my containers, I point config to some data disk, then the only thing stored on cache would be the docker images right? In other words, config = appdata? No appdata is created by docker unless you explicitly point something to it during container config?

Link to comment

if, for all of my containers, I point config to some data disk, then the only thing stored on cache would be the docker images right? In other words, config = appdata? No appdata is created by docker unless you explicitly point something to it during container config?

For example:

 

Plex dockers will actually store the Plex library in the config location. By the Plex library, I don't mean your media files. Instead, I mean the database that Plex stores data about your media files. This can become quite large if you have a lot of media.

 

Also, Plex will often read and write to its library. If you store it on a data drive in the array, every time Plex wants to write to its library, it will involve also updating parity, causing 2 drives to spin, and slowing things down. If instead, you store it on cache, then writing the library will not involve the parity drive, so only 1 disk will spin and it will be faster. Even faster if cache is SSD of course.

 

The recommended way to handle the config for all your dockers is to create an appdata user share, set to Use cache only, and make each docker use a subfolder of that appdata folder, named after the docker. For example, /mnt/cache/appdata/plex.

Link to comment

makes perfect sense, thanks

 

In that case, is there a plugin that will create a backup of the cache drive so I don't risk losing my appdata/config data in the event of a cache drive failure? I've seen some scripts floating around the forum, but something like a plugin would be useful

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.