Mount a permanent ftp/sftp volume on boot?

[mbc0]

Hi,

 

As the title suggests, I need to mount a permanent ftp/sftp volume on boot... this is the command I am using but get the error listed below, is there another way to do this?

 

sshfs ***@*****.****:/home/hd11/*** /mnt/user/processingfolder/seedbox2

 

I am running Sonarr & Couchpotato from my unRAID and all is well except I want to be able to use my seedbox/VPS to handle all the downloading.  This option is available in both Couchpotato & Sonarr but they require a local path.  I have used SSHFTPS to mount an ftp share on unRAID and it works perfectly but if I add that path to either Sonarr or Couchpotato dockers it crashes with this error

 

PROBLEMS

invalid value "/mnt/user/processingfolder/seedbox2/:deluge4cp:rw" for flag -v: deluge4cp:rw is not an absolute path

See '/usr/bin/docker run --help'.

 

is there a way of making a permanent (absolute path) to my VPS to make this work?

 

Many Thanks

[mbc0]

*edited to make more sense* 

[trurl]

I don't know but I'm not sure you should be mixing the user shares shfs at /mnt/user with the other mount. You might try mounting to a folder on one of your disks, such as /mnt/cache/appdata/processingfolder for example.

 

I think your error though is because you are not specifying the mapped volume. Dockers do not know anything about the unRAID mounts, they can only access the mapped volumes. You would have to map the processingfolder to a container volume before a container can see it. And you would probably have to restart the docker service (Settings - Docker) after doing the mount since docker cannot see any mount made after the docker service starts.

[mbc0]

I don't know but I'm not sure you should be mixing the user shares shfs at /mnt/user with the other mount. You might try mounting to a folder on one of your disks, such as /mnt/cache/appdata/processingfolder for example.

 

I think your error though is because you are not specifying the mapped volume. Dockers do not know anything about the unRAID mounts, they can only access the mapped volumes. You would have to map the processingfolder to a container volume before a container can see it. And you would probably have to restart the docker service (Settings - Docker) after doing the mount since docker cannot see any mount made after the docker service starts.

 

That was a great idea and although it prevented the errors by mounting it is still not functional 

 

sshfs ***@*****.****:/home/hd11/*** /mnt/cache/seedboxmnt works and I can see all the files on my ftp server

 

When setting the volume mappings for the container I am able to navigate to the folders on my ftp server (/mnt/cache/seedboxmnt/data/downloaded etc etc)

 

when accessing seedboxmnt from within the container the directory is empty

 

Is this because the docker/container cannot access the unRAID files system? because of this I can see no way to map the mount to the countainer

 

Thanks for your help so far! 

[CHBMB]

I don't know but I'm not sure you should be mixing the user shares shfs at /mnt/user with the other mount. You might try mounting to a folder on one of your disks, such as /mnt/cache/appdata/processingfolder for example.

 

I think your error though is because you are not specifying the mapped volume. Dockers do not know anything about the unRAID mounts, they can only access the mapped volumes. You would have to map the processingfolder to a container volume before a container can see it. And you would probably have to restart the docker service (Settings - Docker) after doing the mount since docker cannot see any mount made after the docker service starts.

 

That was a great idea and although it prevented the errors by mounting it is still not functional 

 

sshfs ***@*****.****:/home/hd11/*** /mnt/cache/seedboxmnt works and I can see all the files on my ftp server

 

When setting the volume mappings for the container I am able to navigate to the folders on my ftp server (/mnt/cache/seedboxmnt/data/downloaded etc etc)

 

when accessing seedboxmnt from within the container the directory is empty

 

Is this because the docker/container cannot access the unRAID files system? because of this I can see no way to map the mount to the countainer

 

Thanks for your help so far! 

 

Try mounting your shares then restarting the docker service (not just the app, the whole service via  settings ==> docker ==> enable)  I don't think docker can see stuff that has been mounted after the service starts.

[mbc0]

Hi,

 

Thanks for that, I already stopped/started the docker service... I am wondering now if it is a permission issue as I just noticed this but cannot find a way to change the permissions of an SSHFS mount?

 

 

drwxrwxrwx 1 nobody users          58 Jun 14 00:14 appdata/

-rw-rw-rw- 1 nobody users 107374182400 Jun 22 00:07 docker.img

drwxrwxrwx 1 nobody users          10 Jun 19 10:34 plexmedia/

drw-rw-rw- 1 nobody users          18 Jun 21 03:47 processingfolder/

drwx------ 1  1045  1045        4096 Jun 21 10:22 seedboxmnt/

root@UNRAIDSERVER:/mnt/cache#

 

[mbc0]

*Some Progress* So Close!

 

if I docker exec into the container I am able to access the sshfs mount fully and copy files etc, all working as hoped but for some reason within Sonarr & CouchPotato the directory is not visable?

 

here is the config of the volume mounts for CouchPotato

 

 

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name="CouchPotato" --net="bridge" -e PUID="99" -e PGID="100" -e TZ="Europe/London" -p 5050:5050/tcp -v "/mnt/cache/seedboxmnt/":"/seedboxmnt":rw -v "/mnt/user/processingfolder/couchpotato/manualproc/":"/cpmanualproc":rw -v "/mnt/user/processingfolder/rutorrent/downloads/cp/":"/cp":rw -v "/mnt/user/appdata/couchpotato/config/":"/config":rw -v "/mnt/user/plexmedia/Video/Movies/":"/movies":rw linuxserver/couchpotato

017af27cb3800e201d7bd878793f7ff10f1ae0d8782517b4d59f5fe71c61acd9

 

The command finished successfully!

 

 

Any clues as to the final piece of this puzzle?

 

[mbc0]

Just in case anyone else tries this.... I gave up!

 

Installed lubuntu on a VM, all working now... shame to have to do it but it works nicely at least...

[weirdcrap]

On 6/23/2016 at 5:49 AM, mbc0 said:

Just in case anyone else tries this.... I gave up!

 

Installed lubuntu on a VM, all working now... shame to have to do it but it works nicely at least...

I found this thread trying to do the same thing you mentioned and I didn't have any issues, everything seems to be working as expected after restarting the docker service. I am still testing everything to be sure there are no bugs but so far so good. I have episodes from Sonarr importing correctly and movies from couchpotato are being pulled over fine as well.

 

I followed some advice from a thread on TorrentInvites http://www.torrent-invites.com/bittorrent/286522-ultimate-automated-home-media-setup-torrents-plex-xmbc.html where someone was trying to do something very similar to get the settings tweaked correctly in CouchPotato and Sonarr.

 

This is the command I used with the private bits removed (one each for sonarr and couchpotato):

sshfs [email protected]:pathonremotemachine/ /localpath/ -o allow_other -o Ciphers=arcfour -o Compression=no -o IdentityFile=/mnt/cache/.watch/privatekeyfilehere

 

I set the cipher to arcfour because I am not trying to transmit any truly sensitive data and turned off compression to speed up the process of downloading the remote files to my machine. My Speeds went from about 10Mb/s to 40Mb/s just by adding those two options. The allow_other option makes it so other users who aren't the owner of the folders I'm mapping can interact with the data inside (this is necessary since my seedbox doesn't have the same UID and GUID as my UnRAID server).

 

If anyone is interested in more details or has specific questions I will do my best to answer them.

Edited October 11, 2021 by weirdcrap

[Frostyfruit]

Hi wierdcrap,

 

I have been searching high and low for this as I am in the exact situation. so will try it tonight.

 

I wanted to ask a couple of questions hoping you can guide me as I am new to docker

 

1. Does the above command go into the extra parameters field in my docker app i.e. Sonarr?

2. Does my local path need to be a real folder i.e. /mnt/user/downloads/complete  - or does it create the local folder?

3. what does the identify file do? is this the sshkey file? is identifyFile= where you define where to store it or do you need to place a identify file in that folder?

 

Appreciate your help! 

[Frostyfruit]

Hi WC,

 

I had a look at the link you originally posted - https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh

 

I assume that you would need to install sshfs via unraid CLI -

sudo apt-get install sshfs

 

[weirdcrap]

-You install SSHFS from the Unraid nerd pack plugin: 

 

https://lime-technology.com/forums/topic/35866-unraid-6-nerdpack-cli-tools-iftop-iotop-screen-kbd-etc/

 

-The command I provided above needs to be run at array start and stop. If your SSHFS share isn't unmounted when stopping the array or powering down it will cause your system to hang. I use one script that mounts my SSHFS remote dirs at array start and another to unmount them at array stop. Squid's excellent user script plugin is perfect for this: 

https://lime-technology.com/forums/topic/48286-plugin-ca-user-scripts/

 

My mount script is just this (copy the line and change the path for however many directories):

#! /bin/bash
sshfs [email protected]:/remotePath/ /localPath/ -o StrictHostKeyChecking=no -o allow_other -o Compression=no -o IdentityFile=/mnt/cache/.watch/rsaKeyName

 

Unmount just looks like this (copy the line and change the path for however many directories):

#! /bin/bash
logger "My unmount script is running..."
umount /my/local/folder/path
umount /my/local/folder/path2

 

-You need to have the folder already created IIRC.  SSHFS essentially mounts the remote system's path inside of the local folder.

 

-identity file is your SSH private key file used to authenticate to the remote system without the need for user input. You need to have already generated your ssh key pair and added your public key to the remote system's authorized_keys folder. The file path is just a path you choose to store the key file at. It needs to be somewhere outside of your array like inside a hidden dotted directory so the mover doesn't mess with it (like /mnt/cache/.mykeyfolder/mykey)  

 

Edited July 16, 2019 by weirdcrap
updated provided SSHFS mounting command to reflect changes I have mode over the years.

[Frostyfruit]

Thanks for this!

 

I'll give it a go and document anything further to help others. 

 

I installed SSHSF from nerd pack but when i run sshsf from CLI it dosnt recognize it so i'll give the server a reboot and try the following tonight.

 

I have also found some instructions on creating the SSH Private key / public key so will post this for others using a remote server

[Frostyfruit]

Wierdcrap,

 

So just to confirm the mount point for SSHFS is within UNRAID not the docker container itself. 

 

Have you tried or could you mount the remote directory directly within the docker container?

[weirdcrap]

--Yes I believe you need to reboot for the CLI tools you install to be available.

--Key pair creation is a simple task and you can find instructions everywhere. The most important thing is to not set a passphrase to protect the key, if you do the script will fail to mount your directories because I am not aware of an easy way to pass a key passphrase to ssh as part of it's calling argument.

--Yes the mount points all operate from within UnRAID. You may be able to mount directly in the docker container, I have never tried. Fairly certain you will lose any customizations you make to the docker when you update though. I honestly never modify anything in the dockers themselves.

Edited January 17, 2018 by weirdcrap

[Frostyfruit]

Ok! 

 

I got the mount working and tested it. I had to change the cipher as my remote host didnt support the one on the above command. if you run executing sshfs - run the -o debug switch to help troubleshot.

 

I spent a lot of my time setting up the public key on the remote server and have it tested. For those interested I had to do the following

• ssh-keygen -t rsa
  

• enter the public key in ~/.ssh/authorized_keys in one line (needs to start with ssh-rsa)
• chmod 700 ~/.ssh
• chmod 600 ~/.ssh/authorized_keys

Depending on your distro you might need to play around with .ssh/authorized_keys or authorized_keys2

 

I would suggest testing this with Putty first and then going from there..

[Frostyfruit]

Wierdcrap a couple of questions for your setup

 

1. Are you mapping the remote mount via unraid (not docker containers)

2. Are you just doing one mount i.e. /remotefolder and then mapping the respective folders to your Docker containers i.e. remotefolder/container1 - remotefolder/container2? or are you creating separate mount points per docker?

 

[weirdcrap]

On 1/17/2018 at 3:35 PM, Frostyfruit said:

Wierdcrap a couple of questions for your setup

 

1. Are you mapping the remote mount via unraid (not docker containers)

2. Are you just doing one mount i.e. /remotefolder and then mapping the respective folders to your Docker containers i.e. remotefolder/container1 - remotefolder/container2? or are you creating separate mount points per docker?

 

5

1. You issue the mounting commands from UnRAID via terminal/ssh. Once I had my commands for mounting figured out I turned them into a user script that Squid's plugin can run on the server for me.

2. I use two one for radarr and another for sonarr. There is no reason you couldn't use a single remote mount point that I can think of.

Edited July 16, 2019 by weirdcrap