[OUTDATED] Extended Docker configuration page

[devanchya]

Can we get an option to force a docker to run in -u nobody.  The Media Browser service can be installed by command line by running -u but appears to fail in the configuration web option due to not running in -u mode.

[limetech]

/var/lib/docker/unraid/templates/       - persistent storage of templates;

What goes in here and does it get in here?

 

There's also one thing we'll need to make GitHub API download doable: an OAUTH-TOKEN with tons of hits available; currently, GitHub only allows 60 unauthenticated API hits per hour, which isn't enough.

Nice catch, I forgot about that.

 

PS. there's another approach: use the git command line to download this stuff.

Also a nice idea.

[JustinChase]

I remember the update process is pretty vague, however I think I got a 4th line that said "running as anonymous" or something equivalent. The only way I knew it was updated properly was when I go to the extensions page I see the Docker Apps entries, and now if I go to extensions/plugins I see Docker Manager and can update it (with just beta8 you only see UnRAID Server OS under plugins).

 

I am not sure why the update would fail, but maybe gfjardim or Tom could help troubleshoot. Also, I've never looked, but I would guess there are some log entries tied to this that may help.

 

One other thought... when moving to beta8 Tom had us remove the old dockerman.plg file, but other mentioned there were 1-2 other files some had, and some didn't. Have you verified everything docker related under the plugins folder is gone? You may have parts of both GUIs?

 

The Docker Manager aside, are you able to create the new docker image file and start docker without issue? It may be worth trying to add a docker app to see what happens.

 

I have removed the old plugin, and have renamed everything related to the old plugin system, but still have the same issue.  I just tried to install HTPC manager (never had it installed, no my-* info), and it failed in the same way.

 

I'm sure the plugin is just not installed correctly.  I wonder if the fact that I named it "docker" instead of "docker.img" is the problem.  Tom's instructions say it's okay to have no extension.  I guess I'll start over again, and call it docker.img and see if that helps.

 

Hopefully gfjardim or Tom can chime in on this, in case I'm just doing it wrong.

[gfjardim]

 

/var/lib/docker/unraid/templates/       - persistent storage of templates;

What goes in here and does it get in here?

 

The templates are downloaded there, and copied to /usr/local/emhttp/plugins/dockerMan/templates for faster access.

 

$dockerManPaths = array(
'plugin'         => '/usr/local/emhttp/plugins/dockerMan',
'autostart-file' => '/var/lib/docker/unraid-autostart',
'update-file'    => '/tmp/dockerUpdateStatus.json',
'template-repos' => '/boot/config/plugins/dockerMan/template-repos',
'templates-user' => '/var/lib/docker/unraid-templates',
'templates-ram'  => '/usr/local/emhttp/plugins/dockerMan/templates',
'templates-stor' => '/var/lib/docker/unraid/templates',
'images-ram'     => '/usr/local/emhttp/plugins/dockerMan/assets/images',
'images-stor'    => '/var/lib/docker/unraid/images',
'github_oauth'   => '',
);

 

I'm consolidating all paths in one variable. What do you think?

 

About the GitHub API: there must be a way to circumvent this limit. Is there a way LT get a large oauth key? The github code was already done...

 

[JustinChase]

starting over, calling it docker.img did not help.  I tried to install the plugin again, and the first line shown when it tries to install says this...

 

/usr/local/sbin/plugin install https://raw.githubusercontent.com/gfjardim/dockers/beta8/dockerMan/dockerMan.plg 2>&1 2>&1

 

When I go to putty, then open Midnight Commander, then I try to navigate to /usr/local/sbin/plugin but that folder does not exist, as far as mc shows.  it shows @plugin, but not just plain "plugin" as a directory.

 

When I exit mc, then run ls, it seems to be there, but with 'special' permissions (from what I can see), and maybe a redirect??...

 

root@media:/usr/local/sbin# ls -l

total 252

-rwxr-xr-x 1 root root 147296 Aug 31 18:33 emhttp*

-rwxr-xr-x 1 root root  1862 Aug 31 18:33 emhttp_event*

-rwxr-xr-x 1 root root    410 Aug 31 18:33 emhttp_identify*

-rwxr-xr-x 1 root root    666 Aug 31 18:33 initconfig*

-rwxr-xr-x 1 root root    103 Aug 31 18:33 installplg*

-rwxr-xr-x 1 root root  2301 Aug 31 18:33 mover*

-rwxr-xr-x 1 root root  1415 Aug 31 18:33 newperms*

lrwxrwxrwx 1 root root    39 Aug 31 18:33 plugin -> /usr/local/emhttp/plugins/plgMan/plugin*

-r-xr-x--- 1 root root  1544 Sep  5 00:04 powerdown*

-rwxr-xr-x 1 root root  1242 Aug 31 18:33 set_ncq*

-rwxr-xr-x 1 root root  66856 Aug 31 18:33 shfs*

-rwxr-xr-x 1 root root    393 Aug 31 18:33 unraid_powerdown*

-rwxr-xr-x 1 root root    552 Aug 31 18:33 vfio-bind*

lrwxrwxrwx 1 root root    39 Aug 31 18:33 xenman -> /usr/local/emhttp/plugins/xenMan/xenman*

 

I'm not sure how to proceed, but would very much like to get my dockers running again.

 

[ljm42]

starting over, calling it docker.img did not help.  I tried to install the plugin again

 

Just wanted to make sure you were aware... there is no need to install the plugin unless you want the latest cutting edge code.  If you are just trying to get up and running in b8, stick with the version of this plugin that was included there by default. 

[JustinChase]

starting over, calling it docker.img did not help.  I tried to install the plugin again

 

Just wanted to make sure you were aware... there is no need to install the plugin unless you want the latest cutting edge code.  If you are just trying to get up and running in b8, stick with the version of this plugin that was included there by default.

 

Yeah, that's my understanding also.  That wasn't working, so I tried to install the plugin, to get the updates, but that also didn't work.  I'm sure that once I get the built-in version working, installing the plugin will continue to work also.

[bkastner]

starting over, calling it docker.img did not help.  I tried to install the plugin again, and the first line shown when it tries to install says this...

 

/usr/local/sbin/plugin install https://raw.githubusercontent.com/gfjardim/dockers/beta8/dockerMan/dockerMan.plg 2>&1 2>&1

 

When I go to putty, then open Midnight Commander, then I try to navigate to /usr/local/sbin/plugin but that folder does not exist, as far as mc shows.  it shows @plugin, but not just plain "plugin" as a directory.

 

When I exit mc, then run ls, it seems to be there, but with 'special' permissions (from what I can see), and maybe a redirect??...

 

root@media:/usr/local/sbin# ls -l

total 252

-rwxr-xr-x 1 root root 147296 Aug 31 18:33 emhttp*

-rwxr-xr-x 1 root root  1862 Aug 31 18:33 emhttp_event*

-rwxr-xr-x 1 root root    410 Aug 31 18:33 emhttp_identify*

-rwxr-xr-x 1 root root    666 Aug 31 18:33 initconfig*

-rwxr-xr-x 1 root root    103 Aug 31 18:33 installplg*

-rwxr-xr-x 1 root root  2301 Aug 31 18:33 mover*

-rwxr-xr-x 1 root root  1415 Aug 31 18:33 newperms*

lrwxrwxrwx 1 root root    39 Aug 31 18:33 plugin -> /usr/local/emhttp/plugins/plgMan/plugin*

-r-xr-x--- 1 root root  1544 Sep  5 00:04 powerdown*

-rwxr-xr-x 1 root root  1242 Aug 31 18:33 set_ncq*

-rwxr-xr-x 1 root root  66856 Aug 31 18:33 shfs*

-rwxr-xr-x 1 root root    393 Aug 31 18:33 unraid_powerdown*

-rwxr-xr-x 1 root root    552 Aug 31 18:33 vfio-bind*

lrwxrwxrwx 1 root root    39 Aug 31 18:33 xenman -> /usr/local/emhttp/plugins/xenMan/xenman*

 

I'm not sure how to proceed, but would very much like to get my dockers running again.

 

That appears normal, mine is the same:

 

root@CydStorage:/usr/local/sbin# ls -al

total 252

drwxr-xr-x  2 root root    320 Sep  3 11:39 ./

drwxr-xr-x 10 root root    200 Aug 31 19:33 ../

-rwxr-xr-x  1 root root 147296 Aug 31 19:33 emhttp*

-rwxr-xr-x  1 root root  1862 Aug 31 19:33 emhttp_event*

-rwxr-xr-x  1 root root    410 Aug 31 19:33 emhttp_identify*

-rwxr-xr-x  1 root root    666 Aug 31 19:33 initconfig*

-rwxr-xr-x  1 root root    103 Aug 31 19:33 installplg*

-rwxr-xr-x  1 root root  2301 Aug 31 19:33 mover*

-rwxr-xr-x  1 root root  1415 Aug 31 19:33 newperms*

lrwxrwxrwx  1 root root    39 Aug 31 19:33 plugin -> /usr/local/emhttp/plugins/plgMan/plugin*

-r-xr-x---  1 root root  1544 Sep  3 11:39 powerdown*

-rwxr-xr-x  1 root root  1242 Aug 31 19:33 set_ncq*

-rwxr-xr-x  1 root root  66856 Aug 31 19:33 shfs*

-rwxr-xr-x  1 root root    393 Aug 31 19:33 unraid_powerdown*

-rwxr-xr-x  1 root root    552 Aug 31 19:33 vfio-bind*

lrwxrwxrwx  1 root root    39 Aug 31 19:33 xenman -> /usr/local/emhttp/plugins/xenMan/xenman*

 

I am guessing the folder is actually plugin, but the trailing @ denotes it's a redirect. It shouldn't cause any issues.

[limetech]

 

/var/lib/docker/unraid/templates/       - persistent storage of templates;

What goes in here and does it get in here?

 

The templates are downloaded there, and copied to /usr/local/emhttp/plugins/dockerMan/templates for faster access.

How about we store those outside the image file, maybe in /boot/config/plugins/dockerMan/templates ?  Why? Suppose someone deletes their image file, or they create a new one - now you have to download all the templates again.

 

Good idea about the paths variable, I guess it would look like this:

 

$dockerManPaths = array(
'plugin'         => '/usr/local/emhttp/plugins/dockerMan',
'autostart-file' => '/var/lib/docker/unraid/autostart',
'update-file'    => '/tmp/dockerUpdateStatus.json',
'template-repos' => '/boot/config/plugins/dockerMan/template-repos',
'templates-user' => '/var/lib/docker/unraid/templates',
'templates-ram'  => '/usr/local/emhttp/plugins/dockerMan/templates',
'templates-stor' => '/boot/config/plugins/dockerMan/templates',
'images-ram'     => '/usr/local/emhttp/plugins/dockerMan/assets/images',
'images-stor'    => '/var/lib/docker/unraid/images',
'github_oauth'   => '',
);

 

About the GitHub API: there must be a way to circumvent this limit. Is there a way LT get a large oauth key? The github code was already done...

I have to study this...

 

Is this what we want to do then?  Change from:

 

/var/lib/docker/unraid-autostart

/var/lib/docker/unraid-templates

/var/lib/docker/images

 

to (option 1)?

 

/var/lib/docker/unraid/autostart

/var/lib/docker/unraid/templates

/var/lib/docker/unraid/images

 

or to (option 2)?

 

/var/lib/docker/unraid-autostart

/var/lib/docker/unraid-templates

/var/lib/docker/unraid-templates/images

 

 

If I was writing all from scratch I would pick option 1, but option 2 is compatible with users existing images and probably "good enough".  Here is one of those tricky design decisions.... what do you think?

 

[eroz]

starting over, calling it docker.img did not help.  I tried to install the plugin again

 

Just wanted to make sure you were aware... there is no need to install the plugin unless you want the latest cutting edge code.  If you are just trying to get up and running in b8, stick with the version of this plugin that was included there by default.

 

Yeah, that's my understanding also.  That wasn't working, so I tried to install the plugin, to get the updates, but that also didn't work.  I'm sure that once I get the built-in version working, installing the plugin will continue to work also.

Did you delete the docker plugin from your flashdrive before you updated to b8?

[JustinChase]

I never installed beta7, I went from beta6 directly to beta8.  Was anything changed/added in beta7 that is necessary to function properly in beta8?

 

I've just removed or renamed everything related to docker, shutdown unRAID, then started new.

 

I've not tried to install this plugin again yet, as I suspect there's something 'wrong' and I'm just going to wait for guidance before proceeding again. (since I've already done this once before).

[JustinChase]

Did you delete the docker plugin from your flashdrive before you updated to b8?

 

yes.

[ljm42]

Did you delete the docker plugin from your flashdrive before you updated to b8?

 

yes.

 

Sorry to be pedantic, but I had three files I had to delete:

http://lime-technology.com/forum/index.php?topic=34955.msg325034#msg325034

Did you get all of them?

[JustinChase]

Did you delete the docker plugin from your flashdrive before you updated to b8?

 

yes.

 

Sorry to be pedantic, but I had three files I had to delete:

http://lime-technology.com/forum/index.php?topic=34955.msg325034#msg325034

Did you get all of them?

 

Yeah, here is my current /boot/config/plugins/ folder...

 

root@media:/boot/config/plugins# ls -l
total 96
drwxrwxrwx 2 root root 16384 Aug  5 21:41 OLDDocker/
drwxrwxrwx 2 root root 16384 Sep  5 12:25 apcupsd/
-rwxrwxrwx 1 root root 21122 Jul  1 23:37 apcupsd_64.plg*
drwxrwxrwx 3 root root 16384 Sep  5 12:25 powerdown/
drwxrwxrwx 2 root root 16384 Sep  5 12:25 webGui/

 

and contents of OLDDocker...

 

root@media:/boot/config/plugins/OLDDocker# ls -l
total 80
-rwxrwxrwx 1 root root 1042 Aug  6 09:06 my-Deluge.xml*
-rwxrwxrwx 1 root root  756 Aug 29 16:26 my-MovieGrabber.xml*
-rwxrwxrwx 1 root root  769 Aug 29 16:20 my-NzbDrone.xml*
-rwxrwxrwx 1 root root  903 Aug  6 09:10 my-SABnzbd.xml*
-rwxrwxrwx 1 root root  862 Aug  6 09:13 my-SickRage.xml*

 

and finally, my cache drive contents...

 

root@media:/mnt/cache# ls -l
total 0
drwxrwxrwx 1 nobody users  24 Sep  5 12:02 JRiver/
drwxrwxrwx 1 nobody users 194 Sep  5 11:41 OLDappdata/
drwxrwxrwx 1 nobody users 148 Aug 29 16:33 OLDdocker/
drwxrwxrwx 1 nobody users  42 Jun 27 17:30 VM/
drwxrwxrwx 1 nobody users 330 Sep  1 19:24 downloads/

[JustinChase]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

[ljm42]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

 

I'm glad you were able to figure it out!

 

[gfjardim]

Tom, why not store all these things in the flash drive then?

 

It will use less then 1MB, will be rarely written/rewritten, will always be available and can be backed up with the rest of unRAID configuration.

 

'template-repos' => '/boot/config/plugins/dockerMan/template-repos',

'templates-user' => '/boot/config/plugins/dockerMan/user-templates',

'templates-stor' => '/boot/config/plugins/dockerMan/templates',

'images-stor'    => '/boot/config/plugins/dockerMan/images',

 

[eroz]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

Glad you figured it out.   I was starting to think you just have some terrible luck.  ;D

[JustinChase]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

Glad you figured it out.   I was starting to think you just have some terrible luck.  ;D

 

ha, you and me both!!  (to be fair, it is pretty terrible luck that unRAID reverted my DNS settings behind my back!)

[bkastner]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

Glad you figured it out.   I was starting to think you just have some terrible luck.  ;D

 

ha, you and me both!!  (to be fair, it is pretty terrible luck that unRAID reverted my DNS settings behind my back!)

 

True, but at least it was an easy fix as opposed to digging in deep into Docker to figure out where things were failing.

 

Are you now able to install containers without issue as well? I am assuming that was just the DNS issue too?

[limetech]

Tom, why not store all these things in the flash drive then?

 

It will use less then 1MB, will be rarely written/rewritten, will always be available and can be backed up with the rest of unRAID configuration.

 

'template-repos' => '/boot/config/plugins/dockerMan/template-repos',

'templates-user' => '/boot/config/plugins/dockerMan/user-templates',

'templates-stor' => '/boot/config/plugins/dockerMan/templates',

'images-stor'    => '/boot/config/plugins/dockerMan/images',

 

Except 'template-user'.  Correct me if I'm wrong: these are the "my-*" templates right?  They start out being initialized from one of the 'built-in' templates, and then any changes made by the user for that particular container are recorded in the 'my-' file, correct?  If that's the case, then the 'my-' files are specific to particular containers, which themselves are specific to a particular docker vdisk file, correct?  If my understanding is correct, then they need to be kept with the docker vdisk file.

 

As for 'images', yes they can be kept on flash, but I thought they too could be customized?

 

Anything "customized" should be kept with the thing the customizations pertain to.

[JustinChase]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

Glad you figured it out.   I was starting to think you just have some terrible luck.  ;D

 

ha, you and me both!!  (to be fair, it is pretty terrible luck that unRAID reverted my DNS settings behind my back!)

 

True, but at least it was an easy fix as opposed to digging in deep into Docker to figure out where things were failing.

 

Are you now able to install containers without issue as well? I am assuming that was just the DNS issue too?

 

"easy" is a matter of opinion

 

I copied my templates over again, and was able to install them just fine using the updated plugin.  I'm just confirming everything is working, but so far, it all looks fine.

 

Maybe limetech can figure out why the DNS got overwritten, so I can prevent it from happening again.

[bkastner]

AAaaarrrggghhhhh!!!

 

Somehow my DNS settings reverted to a server that doesn't exist, so I had no internet access in beta8.  Once I changed them back to correct settings, installing the plugin worked fine.

Glad you figured it out.   I was starting to think you just have some terrible luck.  ;D

 

ha, you and me both!!  (to be fair, it is pretty terrible luck that unRAID reverted my DNS settings behind my back!)

 

True, but at least it was an easy fix as opposed to digging in deep into Docker to figure out where things were failing.

 

Are you now able to install containers without issue as well? I am assuming that was just the DNS issue too?

 

"easy" is a matter of opinion

 

I copied my templates over again, and was able to install them just fine using the updated plugin.  I'm just confirming everything is working, but so far, it all looks fine.

 

Maybe limetech can figure out why the DNS got overwritten, so I can prevent it from happening again.

 

The fix was very easy - once you knew where to look.

 

You raise a good point though about the DNS issue, and also I think it would be helpful if the Install Extension was a bit more verbose when an issue arises. If it had clearly stated it couldn't reach the specified server you likely would have figured this out in mere minutes as opposed to all the time it took.

 

[JustinChase]

The fix was very easy - once you knew where to look.

 

You raise a good point though about the DNS issue, and also I think it would be helpful if the Install Extension was a bit more verbose when an issue arises. If it had clearly stated it couldn't reach the specified server you likely would have figured this out in mere minutes as opposed to all the time it took.

 

I agree completely!

[gfjardim]

Tom, why not store all these things in the flash drive then?

 

It will use less then 1MB, will be rarely written/rewritten, will always be available and can be backed up with the rest of unRAID configuration.

 

'template-repos' => '/boot/config/plugins/dockerMan/template-repos',

'templates-user' => '/boot/config/plugins/dockerMan/user-templates',

'templates-stor' => '/boot/config/plugins/dockerMan/templates',

'images-stor'    => '/boot/config/plugins/dockerMan/images',

 

Except 'template-user'.  Correct me if I'm wrong: these are the "my-*" templates right?  They start out being initialized from one of the 'built-in' templates, and then any changes made by the user for that particular container are recorded in the 'my-' file, correct?  If that's the case, then the 'my-' files are specific to particular containers, which themselves are specific to a particular docker vdisk file, correct?  If my understanding is correct, then they need to be kept with the docker vdisk file.

 

As for 'images', yes they can be kept on flash, but I thought they too could be customized?

 

Anything "customized" should be kept with the thing the customizations pertain to.

 

The my-* files are runtime variables. They store the user customizations of the original templates. For users, it's more important to keep those my-* files safe then the rest of the content. Why, you could ask me? Because:

 

1) They are used to update containers. When we update a container, in fact we're removing it, pulling the new image and recreating the container with the same runtime variables, stored on the my-* templates.

 

2) The user can remove all containers and images, and latter he can restore any previous container with only 4 clicks, using the same values as before. He can update the whole server, plug the flash drive on it, and again recreate all containers exactly as they were.

 

Because of that, I think we should keep them in the flash drive.