Jump to content
DZMM

Guide: How To Use Rclone To Mount Cloud Drives And Play Files

2152 posts in this topic Last Reply

Recommended Posts

I have a questions that has more to do w/ mergerfs and a little to do with the script which I am modifying for my needs. If you created identical directory structures for LocalFilesShare1 (located on Unassigned Disk 1) and LocalFilesShare2 (located on Unassigned Disk 2) whatever docker you used that would be pointed to /mnt/disks/mount_mergerfs/gdrive_media_vfs/downloads (for example in my case) would see the local directories as one and mergerfs would manage which physical disk the local data (seeded downloads) is written to based on available space in the localfileshare#/gdrive_media_vfs/download locations correct?

Edited by lzrdking71

Share this post


Link to post
16 hours ago, DZMM said:

Have you tried looking at the folders to see if the files are actually there?  Maybe your mount is dropping

So it happened again today and i went and looked and my local folders are empty no contents at all. I went ahead and ran the mount script and I could see files on the mergerfs mount. So its losing its mount for whatever reason. Any ideas? I have been running this same script for months now with no problems until recently. 

Share this post


Link to post

Does anyone else having problems with plex autoscan lately not updating posters / thumbnails / season pictures / etc.. on new movies and tv shows being added? Does anyone have a fix? it adds them but the pictures are blank

Share this post


Link to post
13 hours ago, lzrdking71 said:

I have a questions that has more to do w/ mergerfs and a little to do with the script which I am modifying for my needs. If you created identical directory structures for LocalFilesShare1 (located on Unassigned Disk 1) and LocalFilesShare2 (located on Unassigned Disk 2) whatever docker you used that would be pointed to /mnt/disks/mount_mergerfs/gdrive_media_vfs/downloads (for example in my case) would see the local directories as one and mergerfs would manage which physical disk the local data (seeded downloads) is written to based on available space in the localfileshare#/gdrive_media_vfs/download locations correct?

I don't think so.  My scripts do first found (ff) so it would write the file to LocalFileShare1 - I'm not sure what mergerfs does if LocalFileShare1 is full.  If LocalFileShare1 was a normal unRAID share, then UnRAID would control where the file is written.

 

You'll have to read up on mergerfs options, but that's outside the scope of this thread.

Share this post


Link to post
8 hours ago, Hypner said:

So it happened again today and i went and looked and my local folders are empty no contents at all. I went ahead and ran the mount script and I could see files on the mergerfs mount. So its losing its mount for whatever reason. Any ideas? I have been running this same script for months now with no problems until recently. 

post your mount script and rclone config.

Share this post


Link to post

i am still having troubling trying to resolve the error in my logs upon mergerfs mounting,

Quote

Sep 15 22:37:40 HTPC-unRAID emhttpd: error: share_luks_status, 6256: Operation not supported (95): getxattr: /mnt/user/mount_mergerfs

 

I previously had everything running smoothly, but since the above issue started, playback has been poor, as well as not being able to use the upload scripts as i believe my mergerfs is wrong in some way.

 

Upon mount, i have noticed a "mount_mergerfs" folder is being created in my LocalFilesShare. This didnt use to happen.

Files are not being stored in the LocalFilesShare (awaiting the upload script to move them to the relative tdrive_upload share) Instead it seems i find the files have been moved into the relative /mount_rclone without the upload script running.

 

Previously in my LocalFilesShare downloaded files would be LocalFilesShare/tv_shows, upload script would upload to tdrive_upload/tv_shows. Lastly the mover would run. 

 

I have since started fresh by re editing the mount script in hope i missed something, this is not the case..

 

Quote

#!/bin/bash

 

######################

#### Mount Script ####

######################

### Version 0.96.7 ###

######################

 

####### EDIT ONLY THESE SETTINGS #######

 

# INSTRUCTIONS

# 1. Change the name of the rclone remote and shares to match your setup

# 2. NOTE: enter RcloneRemoteName WITHOUT ':'

# 3. Optional: include custom command and bind mount settings

# 4. Optional: include extra folders in mergerfs mount

 

# REQUIRED SETTINGS

RcloneRemoteName="tdrive_upload_vfs" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data

RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash  e.g. /mnt/user/mount_rclone

LocalFilesShare="/mnt/disks/Hitachi_HDS723020BLA642_MN1220FA014G6D/local" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable

MergerfsMountShare="/mnt/user/mount_mergerfs" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable

DockerStart="tautulli plex" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page

MountFolders=\{"downloads/blackhole,downloads/complete,downloads/intermediate,downloads/complete/books,downloads/complete/comic,downloads/complete/movies,downloads/complete/music,downloads/complete/tv"\} # comma separated list of folders to create within the mount

 

# Note: Again - remember to NOT use ':' in your remote name above

 

# OPTIONAL SETTINGS

 

# Add extra paths to mergerfs mount in addition to LocalFilesShare

LocalFilesShare2="/mnt/user/mount_rclone/tdrive_upload_vfs" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder.  Enter 'ignore' to disable

LocalFilesShare3="/mnt/user/mount_rclone/tdrive_tv_shows_vfs"

LocalFilesShare4="/mnt/user/mount_rclone/tdrive_tv_comic_book_vfs"

LocalFilesShare5="/mnt/user/mount_rclone/tdrive_tv_kids_vfs"

 

# Add extra commands or filters

Command1="--rc"

Command2=""

Command3=""

Command4=""

Command5=""

Command6=""

Command7=""

Command8=""

 

CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter

RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address

NetworkAdapter="eth0" # choose your network adapter. eth0 recommended

VirtualIPNumber="2" # creates eth0:x e.g. eth0:1.  I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them

 

####### END SETTINGS #######

 

###############################################################################

#####   DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING   #######

###############################################################################

 

####### Preparing mount location variables #######

RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location for rclone mount

LocalFilesLocation="$LocalFilesShare/" # Location for local files to be merged with rclone mount

MergerFSMountLocation="$MergerfsMountShare/" # Rclone data folder location

 

####### create directories for rclone mount and mergerfs mounts #######

#mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName #for script files

mkdir -p /mnt/user/appdata/other/rclone/remotes/tdrive_upload_vfs # 172

mkdir -p /mnt/user/appdata/other/rclone/remotes/tdrive_tv_shows_vfs # 250

mkdir -p /mnt/user/appdata/other/rclone/remotes/tdrive_tv_comic_book_vfs #328

mkdir -p /mnt/user/appdata/other/rclone/remotes/tdrive_tv_kids_vfs #406

if [[  $LocalFileShare == 'ignore' ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested."

else

    echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders."

    eval mkdir -p $LocalFilesLocation/"$MountFolders"

fi

#mkdir -p $RcloneMountLocation

mkdir -p /mnt/user/mount_rclone/tdrive_upload_vfs

mkdir -p /mnt/user/mount_rclone/tdrive_tv_shows_vfs

mkdir -p /mnt/user/mount_rclone/tdrive_tv_comic_book_vfs

mkdir -p /mnt/user/mount_rclone/tdrive_tv_kids_vfs

mkdir -p $MergerFSMountLocation

 

#######  Create tdrive_upload_vfs Mount  #######

#######  Check if script is already running  #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote tdrive_upload_vfs"

echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."

if [[ -f "/mnt/user/appdata/other/rclone/remotes/tdrive_upload_vfs/mount_running" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."

    exit

else

    echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."

    touch /mnt/user/appdata/other/rclone/remotes/tdrive_upload_vfs/mount_running

fi

 

####### Checking have connectivity #######

 

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"

ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform

if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

    echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"

else

    echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"

    rm /mnt/user/appdata/other/rclone/remotes/tdrive_upload_vfs/mount_running

    exit

fi

 

#######  Create Rclone Mount  #######

 

# Check If Rclone Mount Already Created

if [[ -f "/mnt/user/mount_rclone/tdrive_upload_vfs/mountcheck" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Success tdrive_upload_vfs remote is already mounted."

else

    echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount tdrive_upload_vfs remote."

# Creating mountcheck file in case it doesn't already exist

    echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for tdrive_upload_vfs remote."

    touch mountcheck

    rclone copy mountcheck tdrive_upload_vfs: -vv --no-traverse

# Check bind option

    if [[  $CreateBindMount == 'Y' ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote tdrive_upload_vfs"

        ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform

        if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

            echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote tdrive_upload_vfs"

        else

            echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote tdrive_upload_vfs"

            ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber

        fi

        echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote tdrive_upload_vfs"

    else

        RCloneMountIP=""

        echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote tdrive_upload_vfs"

    fi

# create rclone mount

    rclone mount \

    --allow-other \

    --buffer-size 256M \

    --dir-cache-time 720h \

    --drive-chunk-size 512M \

    --log-level INFO \

    --vfs-read-chunk-size 128M \

    --vfs-read-chunk-size-limit off \

    --vfs-cache-mode writes \

    --bind=$RCloneMountIP \

    tdrive_upload_vfs: /mnt/user/mount_rclone/tdrive_upload_vfs &

 

# Check if Mount Successful

    echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"

# slight pause to give mount time to finalise

    sleep 5

    echo "$(date "+%d.%m.%Y %T") INFO: continuing..."

    if [[ -f "/mnt/user/mount_rclone/tdrive_upload_vfs/mountcheck" ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of tdrive_upload_vfs mount."

    else

        echo "$(date "+%d.%m.%Y %T") CRITICAL: tdrive_upload_vfs mount failed - please check for problems.  Stopping dockers"

        docker stop $DockerStart

        rm /mnt/user/appdata/other/rclone/remotes/tdrive_upload_vfs/mount_running

        exit

    fi

fi

 

#######  Create tdrive_tv_shows_vfs Mount  #######

#######  Check if script is already running  #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote tdrive_tv_shows_vfs"

echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."

if [[ -f "/mnt/user/appdata/other/rclone/remotes/tdrive_tv_shows_vfs/mount_running" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."

    exit

else

    echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."

    touch /mnt/user/appdata/other/rclone/remotes/tdrive_tv_shows_vfs/mount_running

fi

 

####### Checking have connectivity #######

 

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"

ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform

if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

    echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"

else

    echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"

    rm /mnt/user/appdata/other/rclone/remotes/tdrive_tv_shows_vfs/mount_running

    exit

fi

 

#######  Create Rclone Mount  #######

 

# Check If Rclone Mount Already Created

if [[ -f "/mnt/user/mount_rclone/tdrive_tv_shows_vfs/mountcheck" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Success tdrive_tv_shows_vfs remote is already mounted."

else

    echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount tdrive_tv_shows_vfs remote."

# Creating mountcheck file in case it doesn't already exist

    echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for tdrive_tv_shows_vfs remote."

    touch mountcheck

    rclone copy mountcheck tdrive_tv_shows_vfs: -vv --no-traverse

# Check bind option

    if [[  $CreateBindMount == 'Y' ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote tdrive_tv_shows_vfs"

        ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform

        if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

            echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote tdrive_tv_shows_vfs"

        else

            echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote tdrive_tv_shows_vfs"

            ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber

        fi

        echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote tdrive_tv_shows_vfs"

    else

        RCloneMountIP=""

        echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote tdrive_tv_shows_vfs"

    fi

# create rclone mount

    rclone mount \

    --allow-other \

    --buffer-size 256M \

    --dir-cache-time 720h \

    --drive-chunk-size 512M \

    --log-level INFO \

    --vfs-read-chunk-size 128M \

    --vfs-read-chunk-size-limit off \

    --vfs-cache-mode writes \

    --bind=$RCloneMountIP \

    tdrive_tv_shows_vfs: /mnt/user/mount_rclone/tdrive_tv_shows_vfs &

 

# Check if Mount Successful

    echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"

# slight pause to give mount time to finalise

    sleep 5

    echo "$(date "+%d.%m.%Y %T") INFO: continuing..."

    if [[ -f "/mnt/user/mount_rclone/tdrive_tv_shows_vfs/mountcheck" ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of tdrive_tv_shows_vfs mount."

    else

        echo "$(date "+%d.%m.%Y %T") CRITICAL: tdrive_tv_shows_vfs mount failed - please check for problems.  Stopping dockers"

        docker stop $DockerStart

        rm /mnt/user/appdata/other/rclone/remotes/tdrive_tv_shows_vfs/mount_running

        exit

    fi

fi

 

#######  Create tdrive_tv_comic_book_vfs Mount  #######

#######  Check if script is already running  #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote tdrive_tv_comic_book_vfs"

echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."

if [[ -f "/mnt/user/appdata/other/rclone/remotes/tdrive_tv_comic_book_vfs/mount_running" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."

    exit

else

    echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."

    touch /mnt/user/appdata/other/rclone/remotes/tdrive_tv_comic_book_vfs/mount_running

fi

 

####### Checking have connectivity #######

 

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"

ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform

if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

    echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"

else

    echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"

    rm /mnt/user/appdata/other/rclone/remotes/tdrive_tv_comic_book_vfs/mount_running

    exit

fi

 

#######  Create Rclone Mount  #######

 

# Check If Rclone Mount Already Created

if [[ -f "/mnt/user/mount_rclone/tdrive_tv_comic_book_vfs/mountcheck" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Success tdrive_tv_comic_book_vfs remote is already mounted."

else

    echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount tdrive_tv_comic_book_vfs remote."

# Creating mountcheck file in case it doesn't already exist

    echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for tdrive_tv_comic_book_vfs remote."

    touch mountcheck

    rclone copy mountcheck tdrive_tv_comic_book_vfs: -vv --no-traverse

# Check bind option

    if [[  $CreateBindMount == 'Y' ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote tdrive_tv_comic_book_vfs"

        ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform

        if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

            echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote tdrive_tv_comic_book_vfs"

        else

            echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote tdrive_tv_comic_book_vfs"

            ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber

        fi

        echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote tdrive_tv_comic_book_vfs"

    else

        RCloneMountIP=""

        echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote tdrive_tv_comic_book_vfs"

    fi

# create rclone mount

    rclone mount \

    --allow-other \

    --buffer-size 256M \

    --dir-cache-time 720h \

    --drive-chunk-size 512M \

    --log-level INFO \

    --vfs-read-chunk-size 128M \

    --vfs-read-chunk-size-limit off \

    --vfs-cache-mode writes \

    --bind=$RCloneMountIP \

    tdrive_tv_comic_book_vfs: /mnt/user/mount_rclone/tdrive_tv_comic_book_vfs &

 

# Check if Mount Successful

    echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"

# slight pause to give mount time to finalise

    sleep 5

    echo "$(date "+%d.%m.%Y %T") INFO: continuing..."

    if [[ -f "/mnt/user/mount_rclone/tdrive_tv_comic_book_vfs/mountcheck" ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of tdrive_tv_comic_book_vfs mount."

    else

        echo "$(date "+%d.%m.%Y %T") CRITICAL: tdrive_tv_comic_book_vfs mount failed - please check for problems.  Stopping dockers"

        docker stop $DockerStart

        rm /mnt/user/appdata/other/rclone/remotes/tdrive_tv_comic_book_vfs/mount_running

        exit

    fi

fi

 

#######  Create tdrive_tv_kids_vfs Mount  #######

#######  Check if script is already running  #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote tdrive_tv_kids_vfs"

echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."

if [[ -f "/mnt/user/appdata/other/rclone/remotes/tdrive_tv_kids_vfs/mount_running" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."

    exit

else

    echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."

    touch /mnt/user/appdata/other/rclone/remotes/tdrive_tv_kids_vfs/mount_running

fi

 

####### Checking have connectivity #######

 

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"

ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform

if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

    echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"

else

    echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"

    rm /mnt/user/appdata/other/rclone/remotes/tdrive_tv_kids_vfs/mount_running

    exit

fi

 

#######  Create Rclone Mount  #######

 

# Check If Rclone Mount Already Created

if [[ -f "/mnt/user/mount_rclone/tdrive_tv_kids_vfs/mountcheck" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Success tdrive_tv_kids_vfs remote is already mounted."

else

    echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount tdrive_tv_kids_vfs remote."

# Creating mountcheck file in case it doesn't already exist

    echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for tdrive_tv_kids_vfs remote."

    touch mountcheck

    rclone copy mountcheck tdrive_tv_kids_vfs: -vv --no-traverse

# Check bind option

    if [[  $CreateBindMount == 'Y' ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote tdrive_tv_kids_vfs"

        ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform

        if [ $? -eq 0 ]; then # ping returns exit status 0 if successful

            echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote tdrive_tv_kids_vfs"

        else

            echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote tdrive_tv_kids_vfs"

            ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber

        fi

        echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote tdrive_tv_kids_vfs"

    else

        RCloneMountIP=""

        echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote tdrive_tv_kids_vfs"

    fi

# create rclone mount

    rclone mount \

    --allow-other \

    --buffer-size 256M \

    --dir-cache-time 720h \

    --drive-chunk-size 512M \

    --log-level INFO \

    --vfs-read-chunk-size 128M \

    --vfs-read-chunk-size-limit off \

    --vfs-cache-mode writes \

    --bind=$RCloneMountIP \

    tdrive_tv_kids_vfs: /mnt/user/mount_rclone/tdrive_tv_kids_vfs &

 

# Check if Mount Successful

    echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"

# slight pause to give mount time to finalise

    sleep 5

    echo "$(date "+%d.%m.%Y %T") INFO: continuing..."

    if [[ -f "/mnt/user/mount_rclone/tdrive_tv_kids_vfs/mountcheck" ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of tdrive_tv_kids_vfs mount."

    else

        echo "$(date "+%d.%m.%Y %T") CRITICAL: tdrive_tv_kids_vfs mount failed - please check for problems.  Stopping dockers"

        docker stop $DockerStart

        rm /mnt/user/appdata/other/rclone/remotes/tdrive_tv_kids_vfs/mount_running

        exit

    fi

fi

 

####### Start MergerFS Mount #######

 

if [[  $MergerfsMountShare == 'ignore' ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested."

else

    if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then

        echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place."

    else

# check if mergerfs already installed

        if [[ -f "/bin/mergerfs" ]]; then

            echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount"

        else

# Build mergerfs binary

            echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now."

            mkdir -p /mnt/user/appdata/other/rclone/mergerfs

            docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build

            mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin

# check if mergerfs install successful

            echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds"

            sleep 5

            if [[ -f "/bin/mergerfs" ]]; then

                echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount."

            else

                echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully.  Please check for errors.  Exiting."

                rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running

                exit

            fi

        fi

# Create mergerfs mount

        echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount."

# Extra Mergerfs folders

        if [[  $LocalFilesShare2 != 'ignore' ]]; then

            echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount."

            LocalFilesShare2=":$LocalFilesShare2"

        else

            LocalFilesShare2=""

        fi

        if [[  $LocalFilesShare3 != 'ignore' ]]; then

            echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount."

            LocalFilesShare3=":$LocalFilesShare3"

        else

            LocalFilesShare3=""

        fi

        if [[  $LocalFilesShare4 != 'ignore' ]]; then

            echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount."

            LocalFilesShare4=":$LocalFilesShare4"

        else

            LocalFilesShare4=""

        fi

        if [[  $LocalFilesShare5 != 'ignore' ]]; then

            echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare5} to ${RcloneRemoteName} mergerfs mount."

            LocalFilesShare5=":$LocalFilesShare5"

        else

            LocalFilesShare5=""

        fi

# make sure mergerfs mount point is empty

        mv $MergerFSMountLocation $LocalFilesLocation

        mkdir -p $MergerFSMountLocation

# mergerfs mount command

        mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4$LocalFilesShare5 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

# check if mergerfs mount successful

        echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created."

        if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then

            echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created."

        else

            echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed.  Stopping dockers."

            docker stop $DockerStart

            rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running

            exit

        fi

    fi

fi

 

####### Starting Dockers That Need Mergerfs Mount To Work Properly #######

 

# only start dockers once

if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started" ]]; then

    echo "$(date "+%d.%m.%Y %T") INFO: dockers already started."

else

    touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started

    echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers."

    docker start $DockerStart

fi

 

rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running

echo "$(date "+%d.%m.%Y %T") INFO: Script complete"

 

exit

 

 

Share this post


Link to post
2 hours ago, KeyBoardDabbler said:

LocalFilesShare="/mnt/disks/Hitachi_HDS723020BLA642_MN1220FA014G6D/local"

 

I'm guessing that Mergerfs doesn't like using an unassigned drive.  Maybe try using a normal user share and see if the error goes away?

Share this post


Link to post
24 minutes ago, DZMM said:

I'm guessing that Mergerfs doesn't like using an unassigned drive.  Maybe try using a normal user share and see if the error goes away?

Looks like you got it right, changing LocalFilesShare to /mnt/user/local has cleared the error from the logs. I will test the uploading tomorrow as i have work early in the morning.

 

Makes me feel a little better that it wasn't something i changed by mistake, after trying to figure it out for the last week. Any idea why for this has been working, now all of a sudden stopped?

Also i thought it was best practice to download to a unassigned device to prevent unnecessary read/writes on the array.

Share this post


Link to post

[NOW RESOLVED]

 

I recently moved hardware (drives remained the same just moved to a new mobo and cpu) and before the move everything ran smoothly and now it keeps on having issues with the mount. Having  said that I believe Rclone also received an update around the same time so I'm not sure what could be the issue. 
 

 

 

Script location: /tmp/user.scripts/tmpScripts/Rclone_mount/script
Note that closing this window will abort the execution of this script
27.09.2020 21:05:38 INFO: Creating local folders.
mkdir: cannot stat '/mnt/user/mount_rclone/gdrive_media_vfs': Transport endpoint is not connected
27.09.2020 21:05:38 INFO: *** Starting mount of remote gdrive_media_vfs
27.09.2020 21:05:38 INFO: Checking if this script is already running.
27.09.2020 21:05:38 INFO: Script not running - proceeding.
27.09.2020 21:05:38 INFO: *** Checking if online
27.09.2020 21:05:39 PASSED: *** Internet online
27.09.2020 21:05:39 INFO: Mount not running. Will now mount gdrive_media_vfs remote.
27.09.2020 21:05:39 INFO: Recreating mountcheck file for gdrive_media_vfs remote.
2020/09/27 21:05:39 DEBUG : rclone: Version "v1.53.1" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "mountcheck" "gdrive_media_vfs:" "-vv" "--no-traverse"]
2020/09/27 21:05:39 DEBUG : Creating backend with remote "mountcheck"
2020/09/27 21:05:39 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2020/09/27 21:05:39 DEBUG : fs cache: adding new entry for parent of "mountcheck", "/"
2020/09/27 21:05:39 DEBUG : Creating backend with remote "gdrive_media_vfs:"
2020/09/27 21:05:39 DEBUG : Creating backend with remote "gdrive:crypt"
2020/09/27 21:05:41 DEBUG : mountcheck: Modification times differ by -12h2m28.852390437s: 2020-09-27 21:05:39.806390437 +0100 BST, 2020-09-27 08:03:10.954 +0000 UTC
2020/09/27 21:05:43 INFO : mountcheck: Copied (replaced existing)
2020/09/27 21:05:43 INFO :
Transferred: 32 / 32 Bytes, 100%, 13 Bytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 3.9s

2020/09/27 21:05:43 DEBUG : 4 go routines active
27.09.2020 21:05:43 INFO: *** Creating mount for remote gdrive_media_vfs
27.09.2020 21:05:43 INFO: sleeping for 5 seconds
2020/09/27 21:05:44 Fatal error: Can not open: /mnt/user/mount_rclone/gdrive_media_vfs: open /mnt/user/mount_rclone/gdrive_media_vfs: transport endpoint is not connected
27.09.2020 21:05:48 INFO: continuing...
27.09.2020 21:05:48 CRITICAL: gdrive_media_vfs mount failed - please check for problems. Stopping dockers

If I reboot the server then they will mount for a short while before then getting terminated and not mounting again until I restart it. 

 

Any help would be greatly appreciated in trying to figure out what has gone wrong or that I have set incorrectly. 

 

Not sure what I had done wrong but now it seems to work again after being down for a couple of days.

tower-diagnostics-20200927-2104.zip

Edited by GreenGoblin
Resolved itself.

Share this post


Link to post
21 hours ago, KeyBoardDabbler said:

Looks like you got it right, changing LocalFilesShare to /mnt/user/local has cleared the error from the logs. I will test the uploading tomorrow as i have work early in the morning.

 

Makes me feel a little better that it wasn't something i changed by mistake, after trying to figure it out for the last week. Any idea why for this has been working, now all of a sudden stopped?

Also i thought it was best practice to download to a unassigned device to prevent unnecessary read/writes on the array.

Dunno - have you upgraded unRAID?  If not, maybe upgrade unRAID and make your download drive a pool drive rather than UD.

Share this post


Link to post

Great script, thank you.

If we want to mount more then one thing, can we do that by just duplicating the script?

Share this post


Link to post
3 hours ago, Dimtar said:

Great script, thank you.

If we want to mount more then one thing, can we do that by just duplicating the script?

 

Yes.  If you want to add more folders to an existing mergerfs mount, you can add the extra paths in the script

Share this post


Link to post

Hi guys 

 

I need help please.

 

I'm trying to switch from unionfs to merge fs.

 

But i'm lost with config files etc.

 

Currently i just need an upload folder for the upload script and a gdrive mount.

 

So what should i strip in the config exactly ?

 

 

Is it ok

 

RcloneRemoteName="crypt" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data
RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash  e.g. /mnt/user/mount_rclone
LocalFilesShare="/mnt/user/mount_rclone_upload" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable
MergerfsMountShare="ignore" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable
DockerStart="plex" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page

 

Do we really need that ?

MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount 

 

 

thx

Edited by francrouge

Share this post


Link to post
On 9/28/2020 at 8:45 PM, DZMM said:

Dunno - have you upgraded unRAID?  If not, maybe upgrade unRAID and make your download drive a pool drive rather than UD.

I am running on version: beta 29, possible something broke in recent update around 25 maybe. Should i report this as a bug?

 

Everything is working now pointed to a user share but nzbget does not like the change. I have notice a big difference in my speeds...

 

I was hoping to eventually move (sonarr, nzbget ect) off my unraid server and onto a nuc, that i have spare. Do you know if this would be a option with the mergerfs mount or would i run into the same issues. So basically my "LocalFilesShare" would point to the nuc running on the network.

something like LocalFilesShare="//192.168.1.10/seedbox/downloads"??

Edited by KeyBoardDabbler

Share this post


Link to post
On 9/28/2020 at 8:41 PM, francrouge said:

MergerfsMountShare="ignore" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable

you need to specify wheere you want the Mergerfs mount to be

 

On 9/28/2020 at 8:41 PM, francrouge said:

Do we really need that ?

MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount 

 

you don't have to create your folders this way - I was just trying to make life easier for new users.  I think though leaving this empty does create problems - so, just list two or more of your current folders (again I think there's a bug if you only list one folder)

Share this post


Link to post
59 minutes ago, KeyBoardDabbler said:

I am running on version: beta 29, possible something broke in recent update around 25 maybe. Should i report this as a bug?

 

Everything is working now pointed to a user share but nzbget does not like the change. I have notice a big difference in my speeds...

If it was working before then my suspicion is that beta 29 could be the culprit.  I've just had to rollback from beta29 as I've had 3 machine lockouts in 24 hours.

 

Re speeds - rather than using UDs why not make your download drive a new pool drive with a cache/pool-only share 'downloads'?  For me the biggest benefit of the 6.9 betas are having more options to have drives with shares that don't need to touch the array (even though I don't have a parity drive) and potential R/W slave issues, without having to resort to UD.  E.g. here's my current structure, where only my cache has shares that are moved to the array

 

767097153_FireShotCapture358-Highlander_Main-1d087a25aac48109ee9a15217a105d14c06e02a6.unraid_net.thumb.png.32be437b043d8c437b6f849e84ea6a2a.png

Share this post


Link to post
1 hour ago, KeyBoardDabbler said:

I was hoping to eventually move (sonarr, nzbget ect) off my unraid server and onto a nuc, that i have spare. Do you know if this would be a option with the mergerfs mount or would i run into the same issues. So basically my "LocalFilesShare" would point to the nuc running on the network.

something like LocalFilesShare="//192.168.1.10/seedbox/downloads"??

I don't see why it would cause a problem.  To be safe, I'd probably add the SMB share using UD.

Share this post


Link to post
Posted (edited)
On 9/24/2020 at 10:14 AM, DZMM said:

If you just want to sync, then there's no point mounting as you've already got a local copy.  I would just use the upload script but set it to sync not move:

 


RcloneCommand="sync" # choose your rclone command e.g. move, copy, sync

 

EDIT:  I think I solved it, found the video from spaceinvaderone which used a much simpler script,  ended upp with this.
Gonna try syncing the share and doing a reboot to see if it's persistent and new files are uploaded/downloaded correctly 
 

 

#!/bin/bash
#----------------------------------------------------------------------------
#first section makes the folders for the mount in the /mnt/disks folder so docker containers can have access
#there are 4 entries below as in the video i had 4 remotes amazon,dropbox, google and secure
#you only need as many as what you need to mount for dockers or a network share

mkdir -p /mnt/user/NAS/OneDrive

#This section mounts the various cloud storage into the folders that were created above.

rclone sync OneDrive: /mnt/user/NAS/OneDrive

 

 

 

----------------------------------------------------------- 

 

 

 

Sorry for the late reply, didn't have time to focus on my private stuff cause of work =/ 

I now tried using the upload script: https://github.com/BinsonBuzz/unraid_rclone_mount/blob/latest---mergerfs-support/rclone_upload 

Changed the required settings to my paths, and to sync. 

 

# REQUIRED SETTINGS
RcloneCommand="sync" # choose your rclone command e.g. move, copy, sync
RcloneRemoteName="OneDrive" # Name of rclone remote mount WITHOUT ':'.
RcloneUploadRemoteName="OneDrive" # If you have a second remote created for uploads put it here.  Otherwise use the same remote as RcloneRemoteName.
LocalFilesShare="/mnt/user/NAS/OneDrive" # location of the local files without trailing slash you want to rclone to use
RcloneMountShare="/mnt/user/NAS/OneDrive" # where your rclone mount is located without trailing slash  e.g. /mnt/user/mount_rclone


Executing this script gives me the error that rclone is not installed.. 

Script location: /tmp/user.scripts/tmpScripts/rclone_Sync/script
Note that closing this window will abort the execution of this script
04.10.2020 08:58:15 INFO: *** Rclone move selected. Files will be moved from /mnt/user/NAS/OneDrive/OneDrive for OneDrive ***
04.10.2020 08:58:15 INFO: *** Starting rclone_upload script for OneDrive ***
04.10.2020 08:58:15 INFO: Script not running - proceeding.
04.10.2020 08:58:15 INFO: Checking if rclone installed successfully.
04.10.2020 08:58:15 INFO: rclone not installed - will try again later.

Executing the command "rclone listremotes" in an ssh to my unRAID server and it works as I think it would

 

root@Unraid:/mnt/user/NAS# rclone listremotes
GoogleDrive:
OneDrive:


Any pointers? 
I dont have any other rclone user script running at the moment, and the paths above are empty, just to try to verify functionallity, dont wanna mess up my onedrive :)

 

Edited by martikainen

Share this post


Link to post
2 hours ago, martikainen said:

EDIT:  I think I solved it, found the video from spaceinvaderone which used a much simpler script,  ended upp with this.
Gonna try syncing the share and doing a reboot to see if it's persistent and new files are uploaded/downloaded correctly 
 

 


#!/bin/bash
#----------------------------------------------------------------------------
#first section makes the folders for the mount in the /mnt/disks folder so docker containers can have access
#there are 4 entries below as in the video i had 4 remotes amazon,dropbox, google and secure
#you only need as many as what you need to mount for dockers or a network share

mkdir -p /mnt/user/NAS/OneDrive

#This section mounts the various cloud storage into the folders that were created above.

rclone sync OneDrive: /mnt/user/NAS/OneDrive

 

 

 

----------------------------------------------------------- 

 

 

 

Sorry for the late reply, didn't have time to focus on my private stuff cause of work =/ 

I now tried using the upload script: https://github.com/BinsonBuzz/unraid_rclone_mount/blob/latest---mergerfs-support/rclone_upload 

Changed the required settings to my paths, and to sync. 

 


# REQUIRED SETTINGS
RcloneCommand="sync" # choose your rclone command e.g. move, copy, sync
RcloneRemoteName="OneDrive" # Name of rclone remote mount WITHOUT ':'.
RcloneUploadRemoteName="OneDrive" # If you have a second remote created for uploads put it here.  Otherwise use the same remote as RcloneRemoteName.
LocalFilesShare="/mnt/user/NAS/OneDrive" # location of the local files without trailing slash you want to rclone to use
RcloneMountShare="/mnt/user/NAS/OneDrive" # where your rclone mount is located without trailing slash  e.g. /mnt/user/mount_rclone


Executing this script gives me the error that rclone is not installed.. 


Script location: /tmp/user.scripts/tmpScripts/rclone_Sync/script
Note that closing this window will abort the execution of this script
04.10.2020 08:58:15 INFO: *** Rclone move selected. Files will be moved from /mnt/user/NAS/OneDrive/OneDrive for OneDrive ***
04.10.2020 08:58:15 INFO: *** Starting rclone_upload script for OneDrive ***
04.10.2020 08:58:15 INFO: Script not running - proceeding.
04.10.2020 08:58:15 INFO: Checking if rclone installed successfully.
04.10.2020 08:58:15 INFO: rclone not installed - will try again later.

Executing the command "rclone listremotes" in an ssh to my unRAID server and it works as I think it would

 


root@Unraid:/mnt/user/NAS# rclone listremotes
GoogleDrive:
OneDrive:


Any pointers? 
I dont have any other rclone user script running at the moment, and the paths above are empty, just to try to verify functionallity, dont wanna mess up my onedrive :)

 

Ahhh, the logic in my upload script isn't quite right.  It looks for the mountcheck file in the mount location - if you don't mount, then there is no mountcheck file!

Share this post


Link to post
4 hours ago, DZMM said:

Ahhh, the logic in my upload script isn't quite right.  It looks for the mountcheck file in the mount location - if you don't mount, then there is no mountcheck file!

Been trying out the sync command now, have I understood it correctly that I'll need to run the sync command on a schedule to download/upload files? It cannot do that automatically when a file is either placed locally or in my onedrive? 

I choose to run the script "in background" thinking it would always be active, but no file changes are being made. 

So to make it always check for new files I shoud use "Custom" schedule and type in 2 * * * *, this would run the script every second minute. What happends if there's a huge file that haven't finished in 2 minutes, will it most likely crash the cron scedule, or just wait for the next time to run? :)
 

Share this post


Link to post
8 hours ago, Bolagnaise said:

 

GG it was good while it lasted 😭

Looks like existing users/legacy accounts will be ok..

Share this post


Link to post

#!/bin/bash
#----------------------------------------------------------------------------

#first
section makes the folders for the mount in the /mnt/disks folder so docker containers can have access

#there
are 4 entries below as in the video i had 4 remotes amazon,dropbox, google and secure

#you
only need as many as what you need to mount for dockers or a network share

mkdir -p /mnt/disks/tdrive
mkdir -p /mnt/disks/gdrive



#This
section mounts the various cloud storage into the folders that were created above.

rclone mount --max-read-ahead 1024k --allow-other tdrive: /mnt/disks/tdrive &
rclone mount --max-read-ahead 1024k --allow-other gdrive: /mnt/disks/gdrive &
This is my tdrive and gdrive mnt script.
I am only going to point plex to my tdrive for me and a few friends to use, uploading is handled by a cloud server. Do I need to use mergerfs or is it ok to carry on like this my drive is
/mnt/disks/tdrive/movies


Sent from my iPhone using Tapatalk

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.