***OFFICIAL GUIDE*** Using Docker

[trurl]

Dockers and other addons often keep the drive they are using spinning, and possibly write to it often. If you use an array drive, then you will also be writing parity. I highly recommend using a cache drive for this. That is the simplest thing, since unRAID already has support for the cache drive, and it is not in the array so parity is not involved.

 

There is only one docker img file, and it will be used by all of your dockers to store each container's code and any storage the container needs which is not mapped to unRAID storage. I have about a dozen dockers with a 20GB docker img, and it is just over half full.

 

Many dockers will access unRAID storage, such as when the container is doing something with your media files. Also most dockers will need a place to store any persistent data that it uses, similar to plugins. This is usually put on your cache drive also. You should create a user share for this purpose, and set it to Use cache disk: Only so mover will leave it alone. Many of the docker templates are preconfigured to use a user share named appdata for this purpose.

 

Your hardware should be fine for trying out docker. Like anything, it depends on what you try to make the dockers do whether you will have any performance issues.

[mr-hexen]

I would agree with trurl in all comments.

 

Your hardware is enough to play with docker (minus the lack of a cache drive). I run 4 containers on my setup and yours has more hosepower than mine.

 

The only thing i noticed is that i quickly chewed through 2GB of ram that I had initially and upgraded to 8GB, which is what you already have.

 

I've even run a win7 and ubuntu server VM on this hardware sucessfully.

[opentoe]

Dockers and other addons often keep the drive they are using spinning, and possibly write to it often. If you use an array drive, then you will also be writing parity. I highly recommend using a cache drive for this. That is the simplest thing, since unRAID already has support for the cache drive, and it is not in the array so parity is not involved.

 

There is only one docker img file, and it will be used by all of your dockers to store each container's code and any storage the container needs which is not mapped to unRAID storage. I have about a dozen dockers with a 20GB docker img, and it is just over half full.

 

Many dockers will access unRAID storage, such as when the container is doing something with your media files. Also most dockers will need a place to store any persistent data that it uses, similar to plugins. This is usually put on your cache drive also. You should create a user share for this purpose, and set it to Use cache disk: Only so mover will leave it alone. Many of the docker templates are preconfigured to use a user share named appdata for this purpose.

 

Your hardware should be fine for trying out docker. Like anything, it depends on what you try to make the dockers do whether you will have any performance issues.

 

But you still can install another data drive just for dockers and not make it part of the array so you don't have to mess with your cache drive, right? In my current box I'm all out of SATA connections, but can purge quite a bit and convert a data drive into a cache drive or just a plain data (no parity) drive for dockers. Has anyone done that yet?

 

 

[trurl]

Dockers and other addons often keep the drive they are using spinning, and possibly write to it often. If you use an array drive, then you will also be writing parity. I highly recommend using a cache drive for this. That is the simplest thing, since unRAID already has support for the cache drive, and it is not in the array so parity is not involved.

 

There is only one docker img file, and it will be used by all of your dockers to store each container's code and any storage the container needs which is not mapped to unRAID storage. I have about a dozen dockers with a 20GB docker img, and it is just over half full.

 

Many dockers will access unRAID storage, such as when the container is doing something with your media files. Also most dockers will need a place to store any persistent data that it uses, similar to plugins. This is usually put on your cache drive also. You should create a user share for this purpose, and set it to Use cache disk: Only so mover will leave it alone. Many of the docker templates are preconfigured to use a user share named appdata for this purpose.

 

Your hardware should be fine for trying out docker. Like anything, it depends on what you try to make the dockers do whether you will have any performance issues.

 

But you still can install another data drive just for dockers and not make it part of the array so you don't have to mess with your cache drive, right? In my current box I'm all out of SATA connections, but can purge quite a bit and convert a data drive into a cache drive or just a plain data (no parity) drive for dockers. Has anyone done that yet?

unRAID already manages your cache drive for you. If you try to use a drive unRAID doesn't manage you will have to do it yourself. Partitioning, formatting, mounting, unmounting, etc. You also have to be concerned about when in the boot up your drive becomes accessible for dockers and other addons to use.

 

I have never seen any need for the additional complexity.

[opentoe]

Thanks. I'm still debating weather or not to activate a cache drive or not. The reason I suggested to add an additional drive not part of the array was so I can put the docker image on it and not have unraid keep parity on it. If I were to activate dockers right now I'd be forced to put the docker image on a parity data drive which would make it pretty slow. Since I never had a cache drive I'll need to find out how it works. I like to know how it all works before I just blindly add it or convert a drive to it. I know files are first moved to the cache drive, then a script or "mover" moves them to the appropriate usershares/mounts. Just wondering the latency on the move. Is it instant or do files sit on the cache drive for a long time? Sorry, not appropriate for this thread, can move it. But it does bring up docker questions I had.

[trurl]

Thanks. I'm still debating weather or not to activate a cache drive or not. The reason I suggested to add an additional drive not part of the array was so I can put the docker image on it and not have unraid keep parity on it. If I were to activate dockers right now I'd be forced to put the docker image on a parity data drive which would make it pretty slow. Since I never had a cache drive I'll need to find out how it works. I like to know how it all works before I just blindly add it or convert a drive to it. I know files are first moved to the cache drive, then a script or "mover" moves them to the appropriate usershares/mounts. Just wondering the latency on the move. Is it instant or do files sit on the cache drive for a long time? Sorry, not appropriate for this thread, can move it. But it does bring up docker questions I had.

You have complete control over each user share regarding whether it is cached or not. You can also control whether or not a user share gets moved from cache to array. In the case of shares used for storing dockers and application data, you should set these to Use cache disk: Only. You also have control of the mover schedule. By default, mover runs daily at 03:40, but you can change this. All of this can be configured in the webGUI.

[opentoe]

Thanks. I'm still debating weather or not to activate a cache drive or not. The reason I suggested to add an additional drive not part of the array was so I can put the docker image on it and not have unraid keep parity on it. If I were to activate dockers right now I'd be forced to put the docker image on a parity data drive which would make it pretty slow. Since I never had a cache drive I'll need to find out how it works. I like to know how it all works before I just blindly add it or convert a drive to it. I know files are first moved to the cache drive, then a script or "mover" moves them to the appropriate usershares/mounts. Just wondering the latency on the move. Is it instant or do files sit on the cache drive for a long time? Sorry, not appropriate for this thread, can move it. But it does bring up docker questions I had.

You have complete control over each user share regarding whether it is cached or not. You can also control whether or not a user share gets moved from cache to array. In the case of shares used for storing dockers and application data, you should set these to Use cache disk: Only. You also have control of the mover schedule. By default, mover runs daily at 03:40, but you can change this. All of this can be configured in the webGUI.

 

Thanks. I guess since I don't have a cache drive now I don't see these options in the gui. Now that I know you can control a share NOT to use the cache drive this would be nice to setup a docker share.

[trurl]

... Now that I know you can control a share NOT to use the cache drive this would be nice to setup a docker share.

You would setup your docker share to use the cache drive ONLY so it doesn't get moved to the array.

 

I don't actually cache writes to any of my normal user shares. My cache drive(s) are strictly used for dockers.

 

Maybe this is what you meant above.

[mr-hexen]

... Now that I know you can control a share NOT to use the cache drive this would be nice to setup a docker share.

You would setup your docker share to use the cache drive ONLY so it doesn't get moved to the array.

 

I don't actually cache writes to any of my normal user shares. My cache drive(s) are strictly used for dockers.

Maybe this is what you meant above.

 

That is how I have mine setup too.

[opentoe]

... Now that I know you can control a share NOT to use the cache drive this would be nice to setup a docker share.

You would setup your docker share to use the cache drive ONLY so it doesn't get moved to the array.

 

I don't actually cache writes to any of my normal user shares. My cache drive(s) are strictly used for dockers.

 

Maybe this is what you meant above.

 

Did you plan on doing that when version 6 came out or did you just find using a cache disk for dockers only was a good idea? Did you ever use a cache drive for writes? So that cache drive is part of your parity? And how come you didn't just setup a regular data drive that wasn't part of the parity? Sorry for all the questions, makes for good learning mentally here.

[trurl]

... Now that I know you can control a share NOT to use the cache drive this would be nice to setup a docker share.

You would setup your docker share to use the cache drive ONLY so it doesn't get moved to the array.

 

I don't actually cache writes to any of my normal user shares. My cache drive(s) are strictly used for dockers.

 

Maybe this is what you meant above.

Did you plan on doing that when version 6 came out or did you just find using a cache disk for dockers only was a good idea? Did you ever use a cache drive for writes? So that cache drive is part of your parity? And how come you didn't just setup a regular data drive that wasn't part of the parity? Sorry for all the questions, makes for good learning mentally here.

I did cache things when I first started using unRAID, but with experience I realized I didn't care about write speed. Most of the data that gets written to unRAID originates from automated backups from other PCs on the network in the middle of the night, or applications such as transmission or NZBGet running on unRAID getting things from the internet, also totally automated and mostly happening at night. So I am seldom involved in writing to the array. Of course I read the data on unRAID all the time but caching does not help with that.

 

With v5 I was using cache for plugin data, but now with v6 dockers have totally replaced all of those apps that used to be plugins, and there are a lot more dockers than there ever were plugins and I use more apps now than ever.

 

The reason you use cache instead of an array disk for this purpose is because the apps are often reading and writing their data storage, such as Plex libraries. This keeps the disk spun up all the time, and if these writes were to an array drive then parity would be involved also.

 

The reason my cache disk(s) is plural is because unRAID v6 supports btrfs raid cache pools. I actually have 2 x 120GB SSDs in a raid1 pool for cache. These are not array drives so parity is not involved, but since it is raid1 it has its own redundancy. Also, since they are SSDs they are faster and don't really spin.

[hgeorges]

Hello;

I'm a long time user of unraid and new to the latest features, including docker.

So first I want to thank you very much for creating the user guide, that's quite helpful.

 

In my particular situation, I tried a few things so far, and I'm going back and forth on my experiments, installing and uninstalling docker containers. And doing that I noticed that the cleanup part is not really working too well, as I still find things left behind after I deliberately chose to delete images and containers. Would be helpful if the guide can be enhanced to cover the entire life-cycle of the docker environment:

- installing and uninstalling docker

- installing and uninstalling applications in docker

- different upgrade scenarios of  the environment

etc

 

Thank you

hg

[wgstarks]

@jonp

Request

The Docker Guide answered all my questions except one. HOW TO SETUP THE APPDATA SHARE. With help from trurl I was able to work this out, but after reading through this thread I can see I'm not the only person with questions about this. Perhaps a step-by-step for creating this share along with some discussion of best location and proper share and security settings could be added to the guide.

[armbrust]

Has something changed since when the guide was written?

 

i don't have a main docker tab, only a docker option under settings.  I change enable to yes, and hit apply, but nothing seems to happen.

 

[Squid]

Has something changed since when the guide was written?

 

i don't have a main docker tab, only a docker option under settings.  I change enable to yes, and hit apply, but nothing seems to happen.

Do you have a cache drive?  if you don't, change the path to be something like /mnt/disk1/docker.img

[armbrust]

Thanks for the reply. Yes, I do have a cache drive.  Should I see a docker tab in the ui?

[Squid]

Thanks for the reply. Yes, I do have a cache drive.  Should I see a docker tab in the ui?

Yes you should.  After hitting apply, it will take a minute and create a 10G docker.img file.  After the the docker tab should be present.

 

If its not, post your diagnostics

 

[armbrust]

Thanks, figure it out.  Had to manually create the "docker" sub folder on the cache drive.  The tab shows up now, so I assume I'm good to go.    Thanks again

 

[FredG89]

Thanks. I'm still debating weather or not to activate a cache drive or not. The reason I suggested to add an additional drive not part of the array was so I can put the docker image on it and not have unraid keep parity on it. If I were to activate dockers right now I'd be forced to put the docker image on a parity data drive which would make it pretty slow. Since I never had a cache drive I'll need to find out how it works. I like to know how it all works before I just blindly add it or convert a drive to it. I know files are first moved to the cache drive, then a script or "mover" moves them to the appropriate usershares/mounts. Just wondering the latency on the move. Is it instant or do files sit on the cache drive for a long time? Sorry, not appropriate for this thread, can move it. But it does bring up docker questions I had.

 

NVM...I had to click the box a few times to get it to come up. So what should I set the size to? If I can make the Docker Image smaller than I'd be more than happy to.

[admsteck]

I've been looking around and it would be really nice if the guide would include a little more detail on general maintenance.  For example, the guide says you should start with a 10G size and can increase it later, but doesn't say how.  Along the same lines, how can we tell how much of the 10G each of the installed containers are using?

[Squid]

I've been looking around and it would be really nice if the guide would include a little more detail on general maintenance.  For example, the guide says you should start with a 10G size and can increase it later, but doesn't say how.

That's in the docker FAQ

Along the same lines, how can we tell how much of the 10G each of the installed containers are using?

The resource monitor within CA will sort of show you that, but unfortunately docker doesn't seem to take into account any new files which get written to the image (ie: misconfigured and downloads wind up going into the docker.img are not taken into account)

 

This official guide has pretty much fallen by the wayside, and doesn't take into consideration the new features within 6.2.  The Docker FAQ has picked up most of the slack on questions regarding setup etc, and unRaid's manual (click the manual at the bottom right of any screen) does a decent job of walking through the initial set up, and as usual this forum is amazing at answering the questions.

[kode54]

The screen shots in the docker-guide are in serious need of update. For instance, there's one that shows the default path for the Docker image to be /mnt/cache/docker.img and sized to 10GB. Another screen shot shows a /config mount point directed at /opt/appdata/appname.

[JonathanM]

The screen shots in the docker-guide are in serious need of update. For instance, there's one that shows the default path for the Docker image to be /mnt/cache/docker.img and sized to 10GB. Another screen shot shows a /config mount point directed at /opt/appdata/appname.

The 10GB file size is fine, I've got 18 dockers, several different bases, and am only using 11GB. It's much better to find out sooner rather than later if you have a misconfigured app writing inside the container, and it's really simple to expand if you really need a bigger file.

 

The paths should be updated to reflect current defaults though.

[gassie]

Any answer for this (old post)? I try it with the Network Type: bridge but without succes....

 

How would I set certain docker, that it would get different IP than unRAID?

 

As in an IP from your DHCP server or a NAT'd IP from the bridge?

IP from DHCP. unRAID server has IP 10.1.0.12. For docker (ruTorrent) I would like to get different IP, so I can set up exception in firewall and Snort.

[Squid]

Look at using the Pipeworks docker app to help you with that