Guide: How To Use Rclone To Mount Cloud Drives And Play Files


DZMM

Recommended Posts

Has anybody had any problems running the scripts on 6.11?  Does the mergerfs install still work without NerdPack?

So far, I haven’t had any issues. I actually thought it was loading some content faster, but that’s just my opinion with what I’ve tested. No proof in that statement.


Sent from my iPhone using Tapatalk
Link to comment

@Kaizac @Roudy Thanks.  I rebooted to upgrade yesterday, and everything stopped working and I panicked when I saw NerdPack was incompatible. 

 

After rolling back, I think the problem was actually because of changes I'd made recently to my mount script - I'd added about 5 more tdrives to get some of my tdrives under 100K files used, but my backup tdrive was still at 400k and was messing up rclone and wouldn't mount properly.  It was so messed up I couldn't even delete old backups to make room, so I just deleted it from gdrive and created a replacement tdrive. 

 

All seems ok, so I'm going to upgrade again once I've worked out how to replace the NerdPack elements I need for other non-rclone activites.

Link to comment

Hi, I used the provided scripts throughout the last couple of years. 

 

A few days ago I ran into ongoing issues with my crypt mount. After mounting the crypt remote, folders inside the remote are visible and accessible. However, after a few minutes the mount drops and I get the following error: 

root@Tower:~# ls -la /mnt/user/mount_rclone/gdrive_vfs
/bin/ls: cannot open directory '/mnt/user/mount_rclone/gdrive_vfs': Transport endpoint is not connected

 

If I now run the mount script without unmounting it does not give me any errors. However, I cannot acess the crypt mount.

 

After unmounting with fusermount -uz I can mount and access the crypt folder again. 

 

Any ideas to fix the issue are welcome. 

 

Many thanks in advance. 

 

 

Link to comment

I have added the following command to the script, because I like to follow the upload process:

 

Command2="--stats=2m --stats-log-level NOTICE --log-file=/mnt/user/torrent/rclone/log.txt"

 

and it ran for many weeks without problems. Filesize was maybe 30-40 MB.

 

Suddenly the file grew to a couple of gigabytes, so I deleted the file and rebooted. 

 

After an hour the log-file is now 2.5 GB, and filled with these lines (repeated MANY times):

 

2022/10/02 14:24:23 INFO  : vfs cache purgeClean item.Reset media/TV-serie/Stargate SG-1 (1997)/Season 05/Stargate SG-1 (1997) - S05E04 - The Fifth Man [DVD][8bit][x264][AC3 5.1]-MEECH.mkv: Empty item skipped, freed 0 bytes
2022/10/02 14:24:23 INFO  : vfs cache purgeClean item.Reset media/TV-serie/Stargate SG-1 (1997)/Season 05/Stargate SG-1 (1997) - S05E04 - The Fifth Man [DVD][8bit][x264][AC3 5.1]-MEECH.mkv: Empty item skipped, freed 0 bytes
2022/10/02 14:24:23 INFO  : vfs cache purgeClean item.Reset media/TV-serie/Stargate SG-1 (1997)/Season 05/Stargate SG-1 (1997) - S05E04 - The Fifth Man [DVD][8bit][x264][AC3 5.1]-MEECH.mkv: Empty item skipped, freed 0 bytes

 

 

Sometimes I have the same problems as the user in this thread, but I haven´t found any solution to that problem:
 

 

Any idea on how to fix this?

 

EDIT: Added mountscript in spoiler:

 

Spoiler
#!/bin/bash

######################
#### Mount Script ####
######################
## Version 0.96.9.3 ##
######################

####### EDIT ONLY THESE SETTINGS #######

# INSTRUCTIONS
# 1. Change the name of the rclone remote and shares to match your setup
# 2. NOTE: enter RcloneRemoteName WITHOUT ':'
# 3. Optional: include custom command and bind mount settings
# 4. Optional: include extra folders in mergerfs mount

# REQUIRED SETTINGS
RcloneRemoteName="cryptmedia" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data
RcloneMountShare="/mnt/user/cryptmedia" # where your rclone remote will be located without trailing slash  e.g. /mnt/user/mount_rclone
RcloneMountDirCacheTime="720h" # rclone dir cache time
LocalFilesShare="ignore" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable
RcloneCacheShare="/mnt/user0/rclonecache" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone
RcloneCacheMaxSize="400G" # Maximum size of rclone cache
RcloneCacheMaxAge="336h" # Maximum age of cache files
MergerfsMountShare="/mnt/user/mount_mergerfs" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable
DockerStart="binhex-plexpass binhex-rtorrentvpn binhex-jackett" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page
MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount

# Note: Again - remember to NOT use ':' in your remote name above

# OPTIONAL SETTINGS

# Add extra paths to mergerfs mount in addition to LocalFilesShare
LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder.  Enter 'ignore' to disable
LocalFilesShare3="ignore"
LocalFilesShare4="ignore"

# Add extra commands or filters
Command1="--rc"
Command2="--stats=2m --stats-log-level NOTICE --log-file=/mnt/user/torrent/rclone/log.txt"
Command3=""
Command4=""
Command5=""
Command6=""
Command7=""
Command8=""

CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter
RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended
VirtualIPNumber="2" # creates eth0:x e.g. eth0:1.  I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them

####### END SETTINGS #######

###############################################################################
#####   DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING   #######
###############################################################################

####### Preparing mount location variables #######
RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location for rclone mount
LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" # Location for local files to be merged with rclone mount
MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" # Rclone data folder location

####### create directories for rclone mount and mergerfs mounts #######
mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName # for script files
mkdir -p $RcloneCacheShare/cache/$RcloneRemoteName # for cache files
if [[  $LocalFilesShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested."
	LocalFilesLocation="/tmp/$RcloneRemoteName"
	eval mkdir -p $LocalFilesLocation
else
	echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders."
	eval mkdir -p $LocalFilesLocation/"$MountFolders"
fi
mkdir -p $RcloneMountLocation

if [[  $MergerfsMountShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating MergerFS folders as requested."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Creating MergerFS folders."
	mkdir -p $MergerFSMountLocation
fi


#######  Check if script is already running  #######
echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}"
echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."
	exit
else
	echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."
	touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
fi

####### Checking have connectivity #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"
ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform
if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
	echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"
else
	echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"
	rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
	exit
fi

#######  Create Rclone Mount  #######

# Check If Rclone Mount Already Created
if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote."
# Creating mountcheck file in case it doesn't already exist
	echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote."
	touch mountcheck
	rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse
# Check bind option
	if [[  $CreateBindMount == 'Y' ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}"
		ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform
		if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
			echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}"
			ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber
		fi
		echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}"
	else
		RCloneMountIP=""
		echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}"
	fi
# create rclone mount
	rclone mount \
	$Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \
	--allow-other \
	--umask 000 \
	--dir-cache-time $RcloneMountDirCacheTime \
	--attr-timeout $RcloneMountDirCacheTime \
	--log-level INFO \
	--poll-interval 10s \
	--cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \
	--drive-pacer-min-sleep 10ms \
	--drive-pacer-burst 1000 \
	--vfs-cache-mode full \
	--vfs-cache-max-size $RcloneCacheMaxSize \
	--vfs-cache-max-age $RcloneCacheMaxAge \
	--vfs-read-ahead 1G \
	--bind=$RCloneMountIP \
	$RcloneRemoteName: $RcloneMountLocation &

# Check if Mount Successful
	echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"
# slight pause to give mount time to finalise
	sleep 5
	echo "$(date "+%d.%m.%Y %T") INFO: continuing..."
	if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount."
	else
		echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems.  Stopping dockers"
		docker stop $DockerStart
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
		exit
	fi
fi

####### Start MergerFS Mount #######

if [[  $MergerfsMountShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested."
else
	if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place."
	else
# check if mergerfs already installed
		if [[ -f "/bin/mergerfs" ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount"
		else
# Build mergerfs binary
			echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now."
			mkdir -p /mnt/user/appdata/other/rclone/mergerfs
			docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build
			mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin
# check if mergerfs install successful
			echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds"
			sleep 5
			if [[ -f "/bin/mergerfs" ]]; then
				echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount."
			else
				echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully.  Please check for errors.  Exiting."
				rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
				exit
			fi
		fi
# Create mergerfs mount
		echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount."
# Extra Mergerfs folders
		if [[  $LocalFilesShare2 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare2=":$LocalFilesShare2"
		else
			LocalFilesShare2=""
		fi
		if [[  $LocalFilesShare3 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare3=":$LocalFilesShare3"
		else
			LocalFilesShare3=""
		fi
		if [[  $LocalFilesShare4 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare4=":$LocalFilesShare4"
		else
			LocalFilesShare4=""
		fi
# make sure mergerfs mount point is empty
		mv $MergerFSMountLocation $LocalFilesLocation
		mkdir -p $MergerFSMountLocation
# mergerfs mount command
		mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true
# check if mergerfs mount successful
		echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created."
		if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created."
		else
			echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed.  Stopping dockers."
			docker stop $DockerStart
			rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
			exit
		fi
	fi
fi

####### Starting Dockers That Need Mergerfs Mount To Work Properly #######

# only start dockers once
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: dockers already started."
else
# Check CA Appdata plugin not backing up or restoring
	if [ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ] || [ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ] ; then
		echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers."
	else
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started
		echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers."
		docker start $DockerStart
	fi
fi

rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
echo "$(date "+%d.%m.%Y %T") INFO: Script complete"

exit

 

 

User shares, all computed:

 

 

img%5D

 

I've also included my diagnostics.

tower-diagnostics-20221002-1559.zip

Edited by Halvliter
Added mountscript and diagnostics
Link to comment
On 9/29/2022 at 4:54 PM, unn4m3d said:

Hi, I used the provided scripts throughout the last couple of years. 

 

A few days ago I ran into ongoing issues with my crypt mount. After mounting the crypt remote, folders inside the remote are visible and accessible. However, after a few minutes the mount drops and I get the following error: 

root@Tower:~# ls -la /mnt/user/mount_rclone/gdrive_vfs
/bin/ls: cannot open directory '/mnt/user/mount_rclone/gdrive_vfs': Transport endpoint is not connected

 

If I now run the mount script without unmounting it does not give me any errors. However, I cannot acess the crypt mount.

 

After unmounting with fusermount -uz I can mount and access the crypt folder again. 

 

Any ideas to fix the issue are welcome. 

 

Many thanks in advance. 

 

 

Are you running the mount script on a continuous cron job? Can you share your specific mount script to see if you maybe configured something wrong?

 

22 minutes ago, undone said:

Hi, i have the rclone.config life in a different direction, is it somehow possible to tell the script where the new config file is located?

e.g. the new path is "/mnt/disk1/rclone/rclone.conf"

That is a question for the rclone plugin itself. But I don't think that's possible with Unraid because all the plugins are stored on the boot drive.

 

33 minutes ago, Halvliter said:

I have added the following command to the script, because I like to follow the upload process:

 

Command2="--stats=2m --stats-log-level NOTICE --log-file=/mnt/user/torrent/rclone/log.txt"

 

and it ran for many weeks without problems. Filesize was maybe 30-40 MB.

 

Suddenly the file grew to a couple of gigabytes, so I deleted the file and rebooted. 

 

After an hour the log-file is now 2.5 GB, and filled with these lines (repeated MANY times):

 

2022/10/02 14:24:23 INFO  : vfs cache purgeClean item.Reset media/TV-serie/Stargate SG-1 (1997)/Season 05/Stargate SG-1 (1997) - S05E04 - The Fifth Man [DVD][8bit][x264][AC3 5.1]-MEECH.mkv: Empty item skipped, freed 0 bytes
2022/10/02 14:24:23 INFO  : vfs cache purgeClean item.Reset media/TV-serie/Stargate SG-1 (1997)/Season 05/Stargate SG-1 (1997) - S05E04 - The Fifth Man [DVD][8bit][x264][AC3 5.1]-MEECH.mkv: Empty item skipped, freed 0 bytes
2022/10/02 14:24:23 INFO  : vfs cache purgeClean item.Reset media/TV-serie/Stargate SG-1 (1997)/Season 05/Stargate SG-1 (1997) - S05E04 - The Fifth Man [DVD][8bit][x264][AC3 5.1]-MEECH.mkv: Empty item skipped, freed 0 bytes

 

 

Sometimes I have the same problems as the user in this thread, but I haven´t found any solution to that problem:
 

 

Any idea on how to fix this?

You gave no information about your configurations, folders, disks, sizes, didn't attach the scripts you use. You made it pretty much impossible for us to be of any help.

Link to comment
14 minutes ago, Kaizac said:

Are you running the mount script on a continuous cron job? Can you share your specific mount script to see if you maybe configured something wrong?

 

Hey, I am running the script on a cronjob every 10 minutes. 

 

Here is a full qoute of my mount script. 

 

#!/bin/bash

######################
#### Mount Script ####
######################
## Version 0.96.9.3 ##
######################

####### EDIT ONLY THESE SETTINGS #######

# INSTRUCTIONS
# 1. Change the name of the rclone remote and shares to match your setup
# 2. NOTE: enter RcloneRemoteName WITHOUT ':'
# 3. Optional: include custom command and bind mount settings
# 4. Optional: include extra folders in mergerfs mount

# REQUIRED SETTINGS
RcloneRemoteName="gdrive_vfs" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data
RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash  e.g. /mnt/user/mount_rclone
RcloneMountDirCacheTime="720h" # rclone dir cache time
LocalFilesShare="/mnt/user/local" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable
RcloneCacheShare="/mnt/user0/mount_rclone" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone
RcloneCacheMaxSize="200G" # Maximum size of rclone cache
RcloneCacheMaxAge="336h" # Maximum age of cache files
MergerfsMountShare="/mnt/user/mount_mergerfs" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable
DockerStart="binhex-nzbhydra2 binhex-jackett binhex-lidarr binhex-radarr sonarr unpackerr nzbget overseerr" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page
MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,downloads/completed"\} # comma separated list of folders to create within the mount

# Note: Again - remember to NOT use ':' in your remote name above

# OPTIONAL SETTINGS

# Add extra paths to mergerfs mount in addition to LocalFilesShare
LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder.  Enter 'ignore' to disable
LocalFilesShare3="ignore"
LocalFilesShare4="ignore"

# Add extra commands or filters
Command1="--rc"
Command2=""
Command3=""
Command4=""
Command5=""
Command6=""
Command7=""
Command8=""

CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter
RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended
VirtualIPNumber="2" # creates eth0:x e.g. eth0:1.  I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them

####### END SETTINGS #######

###############################################################################
#####   DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING   #######
###############################################################################

####### Preparing mount location variables #######
RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location for rclone mount
LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" # Location for local files to be merged with rclone mount
MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" # Rclone data folder location

####### create directories for rclone mount and mergerfs mounts #######
mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName # for script files
mkdir -p $RcloneCacheShare/cache/$RcloneRemoteName # for cache files
if [[  $LocalFilesShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested."
	LocalFilesLocation="/tmp/$RcloneRemoteName"
	eval mkdir -p $LocalFilesLocation
else
	echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders."
	eval mkdir -p $LocalFilesLocation/"$MountFolders"
fi
mkdir -p $RcloneMountLocation

if [[  $MergerfsMountShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating MergerFS folders as requested."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Creating MergerFS folders."
	mkdir -p $MergerFSMountLocation
fi


#######  Check if script is already running  #######
echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}"
echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."
	exit
else
	echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."
	touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
fi

####### Checking have connectivity #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"
ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform
if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
	echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"
else
	echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"
	rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
	exit
fi

#######  Create Rclone Mount  #######

# Check If Rclone Mount Already Created
if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote."
# Creating mountcheck file in case it doesn't already exist
	echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote."
	touch mountcheck
	rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse
# Check bind option
	if [[  $CreateBindMount == 'Y' ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}"
		ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform
		if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
			echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}"
			ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber
		fi
		echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}"
	else
		RCloneMountIP=""
		echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}"
	fi
# create rclone mount
	rclone mount \
	$Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \
	--allow-other \
	--umask 000 \
	--dir-cache-time $RcloneMountDirCacheTime \
	--attr-timeout $RcloneMountDirCacheTime \
	--log-level INFO \
	--poll-interval 10s \
	--cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \
	--drive-pacer-min-sleep 10ms \
	--drive-pacer-burst 1000 \
	--vfs-cache-mode full \
	--vfs-cache-max-size $RcloneCacheMaxSize \
	--vfs-cache-max-age $RcloneCacheMaxAge \
	--vfs-read-ahead 1G \
	--bind=$RCloneMountIP \
	$RcloneRemoteName: $RcloneMountLocation &

# Check if Mount Successful
	echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"
# slight pause to give mount time to finalise
	sleep 5
	echo "$(date "+%d.%m.%Y %T") INFO: continuing..."
	if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount."
	else
		echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems.  Stopping dockers"
		docker stop $DockerStart
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
		exit
	fi
fi

####### Start MergerFS Mount #######

if [[  $MergerfsMountShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested."
else
	if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place."
	else
# check if mergerfs already installed
		if [[ -f "/bin/mergerfs" ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount"
		else
# Build mergerfs binary
			echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now."
			mkdir -p /mnt/user/appdata/other/rclone/mergerfs
			docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build
			mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin
# check if mergerfs install successful
			echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds"
			sleep 5
			if [[ -f "/bin/mergerfs" ]]; then
				echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount."
			else
				echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully.  Please check for errors.  Exiting."
				rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
				exit
			fi
		fi
# Create mergerfs mount
		echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount."
# Extra Mergerfs folders
		if [[  $LocalFilesShare2 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare2=":$LocalFilesShare2"
		else
			LocalFilesShare2=""
		fi
		if [[  $LocalFilesShare3 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare3=":$LocalFilesShare3"
		else
			LocalFilesShare3=""
		fi
		if [[  $LocalFilesShare4 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare4=":$LocalFilesShare4"
		else
			LocalFilesShare4=""
		fi
# make sure mergerfs mount point is empty
		mv $MergerFSMountLocation $LocalFilesLocation
		mkdir -p $MergerFSMountLocation
# mergerfs mount command
		mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true
# check if mergerfs mount successful
		echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created."
		if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created."
		else
			echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed.  Stopping dockers."
			docker stop $DockerStart
			rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
			exit
		fi
	fi
fi

####### Starting Dockers That Need Mergerfs Mount To Work Properly #######

# only start dockers once
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: dockers already started."
else
# Check CA Appdata plugin not backing up or restoring
	if [ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ] || [ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ] ; then
		echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers."
	else
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started
		echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers."
		docker start $DockerStart
	fi
fi

rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
echo "$(date "+%d.%m.%Y %T") INFO: Script complete"

exit

 

I cannot see any issues configuration wise, however I am glad for help on this. 

Link to comment
46 minutes ago, Kaizac said:

That is a question for the rclone plugin itself. But I don't think that's possible with Unraid because all the plugins are stored on the boot drive.

Well, that is what i've done, and it works by hand if I run rclone without script: 

 

 

Edited by undone
engrisch
Link to comment
28 minutes ago, unn4m3d said:

 

Hey, I am running the script on a cronjob every 10 minutes. 

 

Here is a full qoute of my mount script. 

 

#!/bin/bash

######################
#### Mount Script ####
######################
## Version 0.96.9.3 ##
######################

####### EDIT ONLY THESE SETTINGS #######

# INSTRUCTIONS
# 1. Change the name of the rclone remote and shares to match your setup
# 2. NOTE: enter RcloneRemoteName WITHOUT ':'
# 3. Optional: include custom command and bind mount settings
# 4. Optional: include extra folders in mergerfs mount

# REQUIRED SETTINGS
RcloneRemoteName="gdrive_vfs" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data
RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash  e.g. /mnt/user/mount_rclone
RcloneMountDirCacheTime="720h" # rclone dir cache time
LocalFilesShare="/mnt/user/local" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable
RcloneCacheShare="/mnt/user0/mount_rclone" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone
RcloneCacheMaxSize="200G" # Maximum size of rclone cache
RcloneCacheMaxAge="336h" # Maximum age of cache files
MergerfsMountShare="/mnt/user/mount_mergerfs" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable
DockerStart="binhex-nzbhydra2 binhex-jackett binhex-lidarr binhex-radarr sonarr unpackerr nzbget overseerr" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page
MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,downloads/completed"\} # comma separated list of folders to create within the mount

# Note: Again - remember to NOT use ':' in your remote name above

# OPTIONAL SETTINGS

# Add extra paths to mergerfs mount in addition to LocalFilesShare
LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder.  Enter 'ignore' to disable
LocalFilesShare3="ignore"
LocalFilesShare4="ignore"

# Add extra commands or filters
Command1="--rc"
Command2=""
Command3=""
Command4=""
Command5=""
Command6=""
Command7=""
Command8=""

CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter
RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended
VirtualIPNumber="2" # creates eth0:x e.g. eth0:1.  I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them

####### END SETTINGS #######

###############################################################################
#####   DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING   #######
###############################################################################

####### Preparing mount location variables #######
RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location for rclone mount
LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" # Location for local files to be merged with rclone mount
MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" # Rclone data folder location

####### create directories for rclone mount and mergerfs mounts #######
mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName # for script files
mkdir -p $RcloneCacheShare/cache/$RcloneRemoteName # for cache files
if [[  $LocalFilesShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested."
	LocalFilesLocation="/tmp/$RcloneRemoteName"
	eval mkdir -p $LocalFilesLocation
else
	echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders."
	eval mkdir -p $LocalFilesLocation/"$MountFolders"
fi
mkdir -p $RcloneMountLocation

if [[  $MergerfsMountShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating MergerFS folders as requested."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Creating MergerFS folders."
	mkdir -p $MergerFSMountLocation
fi


#######  Check if script is already running  #######
echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}"
echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running."
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running."
	exit
else
	echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."
	touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
fi

####### Checking have connectivity #######

echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online"
ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform
if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
	echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online"
else
	echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity.  Will try again on next run"
	rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
	exit
fi

#######  Create Rclone Mount  #######

# Check If Rclone Mount Already Created
if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote."
# Creating mountcheck file in case it doesn't already exist
	echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote."
	touch mountcheck
	rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse
# Check bind option
	if [[  $CreateBindMount == 'Y' ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}"
		ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform
		if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
			echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}"
			ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber
		fi
		echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}"
	else
		RCloneMountIP=""
		echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}"
	fi
# create rclone mount
	rclone mount \
	$Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \
	--allow-other \
	--umask 000 \
	--dir-cache-time $RcloneMountDirCacheTime \
	--attr-timeout $RcloneMountDirCacheTime \
	--log-level INFO \
	--poll-interval 10s \
	--cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \
	--drive-pacer-min-sleep 10ms \
	--drive-pacer-burst 1000 \
	--vfs-cache-mode full \
	--vfs-cache-max-size $RcloneCacheMaxSize \
	--vfs-cache-max-age $RcloneCacheMaxAge \
	--vfs-read-ahead 1G \
	--bind=$RCloneMountIP \
	$RcloneRemoteName: $RcloneMountLocation &

# Check if Mount Successful
	echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds"
# slight pause to give mount time to finalise
	sleep 5
	echo "$(date "+%d.%m.%Y %T") INFO: continuing..."
	if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount."
	else
		echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems.  Stopping dockers"
		docker stop $DockerStart
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
		exit
	fi
fi

####### Start MergerFS Mount #######

if [[  $MergerfsMountShare == 'ignore' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested."
else
	if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place."
	else
# check if mergerfs already installed
		if [[ -f "/bin/mergerfs" ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount"
		else
# Build mergerfs binary
			echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now."
			mkdir -p /mnt/user/appdata/other/rclone/mergerfs
			docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build
			mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin
# check if mergerfs install successful
			echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds"
			sleep 5
			if [[ -f "/bin/mergerfs" ]]; then
				echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount."
			else
				echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully.  Please check for errors.  Exiting."
				rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
				exit
			fi
		fi
# Create mergerfs mount
		echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount."
# Extra Mergerfs folders
		if [[  $LocalFilesShare2 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare2=":$LocalFilesShare2"
		else
			LocalFilesShare2=""
		fi
		if [[  $LocalFilesShare3 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare3=":$LocalFilesShare3"
		else
			LocalFilesShare3=""
		fi
		if [[  $LocalFilesShare4 != 'ignore' ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount."
			LocalFilesShare4=":$LocalFilesShare4"
		else
			LocalFilesShare4=""
		fi
# make sure mergerfs mount point is empty
		mv $MergerFSMountLocation $LocalFilesLocation
		mkdir -p $MergerFSMountLocation
# mergerfs mount command
		mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true
# check if mergerfs mount successful
		echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created."
		if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then
			echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created."
		else
			echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed.  Stopping dockers."
			docker stop $DockerStart
			rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
			exit
		fi
	fi
fi

####### Starting Dockers That Need Mergerfs Mount To Work Properly #######

# only start dockers once
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: dockers already started."
else
# Check CA Appdata plugin not backing up or restoring
	if [ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ] || [ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ] ; then
		echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers."
	else
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started
		echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers."
		docker start $DockerStart
	fi
fi

rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running
echo "$(date "+%d.%m.%Y %T") INFO: Script complete"

exit

 

I cannot see any issues configuration wise, however I am glad for help on this. 

Your script seems fine. You can switch logs INFO to DEBUG to maybe catch something. But I'm thinking it's not a Rclone issue, but something with the Unraid system. See:

 

29 minutes ago, undone said:

Well, that is what if done, and it works by hand if I run rclone without script: 

 

 

So why don't you put in that manual command in a seperate cron job to execute at start of the system? And if that overlaps with the rclone scripts, add a wait to those scripts.

Link to comment
53 minutes ago, Kaizac said:

You gave no information about your configurations, folders, disks, sizes, didn't attach the scripts you use. You made it pretty much impossible for us to be of any help.

 

That is true, my post was very hasted, I'm sorry.

 

I have edited my previous post, I hope that helps.

 

Thank you.

Link to comment
10 minutes ago, Halvliter said:

 

That is true, my post was very hasted, I'm sorry.

 

I have edited my previous post, I hope that helps.

 

Thank you.

Your shares are all over the place to be honest. And your disk 1 which you use for all your shares is almost completely full which will just lock up your whole system.

 

Right now your mount config seems to be fully cloudbased, without any local files. Is that correct? But you are using VFS cache on a disk which you use for all your other media/storage. That's really not advisable. Normally you use an unassigned drive for the cache so it does not interfere with your main storage/processes.

 

So I would say, you need to add either storage, or you make sure your disk 1 unloads a lot of used storage, so you have a big buffer. Practically I would just disable the use of a VFS cache. And I also don't see any use for the stats-log flag, when the upload script already arranges that the log shows which files are being uploaded.

  • Like 1
Link to comment
1 hour ago, Kaizac said:

Your shares are all over the place to be honest. And your disk 1 which you use for all your shares is almost completely full which will just lock up your whole system.

 

Right now your mount config seems to be fully cloudbased, without any local files. Is that correct? But you are using VFS cache on a disk which you use for all your other media/storage. That's really not advisable. Normally you use an unassigned drive for the cache so it does not interfere with your main storage/processes.

 

So I would say, you need to add either storage, or you make sure your disk 1 unloads a lot of used storage, so you have a big buffer. Practically I would just disable the use of a VFS cache. And I also don't see any use for the stats-log flag, when the upload script already arranges that the log shows which files are being uploaded.

 

Thank you!

 

1. Shares all over the place - How can I fix this? Can you give me an example, please?

 

2. Fully cloudbased - I'm not entirely sure what you mean here ? I am used to the "old" way of mounting rclone, I just had a startup-script for mounting, and had another rclone copy script for adding new media. 

 

 

3. Using VFS cache on seperate disk - Thank you very much for that tip, I will clean up the disk 1, and use that for VFS-cache. How do I ensure that only disk 1 is used for vfs-cache ?

 

In the meantime, I will disable it, will it be enough to remove these parts from the mountscript?
 

    --vfs-cache-mode full \
    --vfs-cache-max-size $RcloneCacheMaxSize \
    --vfs-cache-max-age $RcloneCacheMaxAge \


and change

    --vfs-read-ahead 1G \

to perhaps 

    --vfs-read-ahead 512MB \

 

4. Logging upload - I have not see the progress in the upload script. Where would I be able to view the progress ? The only thing I see is this:

 

02.10.2022 13:59:50 INFO: Script complete
Script Finished Oct 02, 2022 13:59.50

Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount/log.txt

 

 

Thank you again.

 

 

 

 

Edited by Halvliter
added question
Link to comment
32 minutes ago, Halvliter said:

 

Thank you!

 

1. Shares all over the place - How can I fix this? Can you give me an example, please?

 

2. Fully cloudbased - I'm not entirely sure what you mean here ? I am used to the "old" way of mounting rclone, I just had a startup-script for mounting, and had another rclone copy script for adding new media. 

 

 

3. Using VFS cache on seperate disk - Thank you very much for that tip, I will clean up the disk 1, and use that for VFS-cache. How do I ensure that only disk 1 is used for vfs-cache ?

 

In the meantime, I will disable it, will it be enough to remove these parts from the mountscript?
 

    --vfs-cache-mode full \
    --vfs-cache-max-size $RcloneCacheMaxSize \
    --vfs-cache-max-age $RcloneCacheMaxAge \


and change

    --vfs-read-ahead 1G \

to perhaps 

    --vfs-read-ahead 512MB \

 

4. Logging upload - I have not see the progress in the upload script. Where would I be able to view the progress ? The only thing I see is this:

 

02.10.2022 13:59:50 INFO: Script complete
Script Finished Oct 02, 2022 13:59.50

Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount/log.txt

 

 

Thank you again.

 

 

 

 

1. I see you have the shares Gsuite, Cryptmedia, Rclonecache, local, mount_mergerfs. I don't understand the purpose of these. Normally you would only have 3 or 4 if you also use VFS cache. In the traditional setup you would have 1 folder to which you mount your crypt of google drive (mount_rclone in the script). Then you have your local files (mnt/user/local in the script) and the merger folder of those 2 (mount_mergerfs in the script).

 

2. This is where your share structure also confuses me. You don't use the local share in your mount script, you put it on ignore. So your current setup will just use the mounted crypt rclone mount and the rclone cache. Your local files are not included in the merger. This means you are working fully in the cloud and don't have any of your media associated files locally. I personally store the small files like subtitles and nfo and artwork locally and merge that share with my cloud mount.

 

3. Like you did right now, the Share rclonecache only uses Disk 1. Just know that the depending on your setup/wan speed/amount of users the rclone cache advantage can be pretty small. I used to use it, but I stopped and don't notice any difference, even though it uses a lot of storage. And I even stored it on an SSD. Putting a cache on a HDD can work, but I'm not really sure you are really getting the advantages you desire.

 

To explain the cache: without cache you play a media file and it will write the buffer to your RAM and will play from there, thus initiating a new download every time the file is accessed.

With cache: the buffer will be written to your VFS cache-dir disk and from there it will play. If you access that file later again it will play from your cache instead of downloading again.

 

This can be useful when you have a multiple people accessing the same media and it prevents having to download the file every time when someone start the media playback and is easier on your RAM. But if you just have a few users or have different media interests it will just download Episode 1 to the disk, then you play Episode 2, Then episode 3, etc. And then once the cache is full it will just delete the unused cache files. There won't be any advantage because nobody is looking at Episode 1 within a short time span again. So take your situation in consideration before you take on the hassle to get a cache working.

 

vfs-read-ahead only works with vfs-cache-mode full, so when you don't use the cache you don't need this flag.

 

4. My upload script has -vv in it and this shows me what's going on when I look at the log within the User Scripts section of Unraid.

  • Like 1
Link to comment
1 hour ago, Kaizac said:

1. I see you have the shares Gsuite, Cryptmedia, Rclonecache, local, mount_mergerfs. I don't understand the purpose of these. Normally you would only have 3 or 4 if you also use VFS cache. In the traditional setup you would have 1 folder to which you mount your crypt of google drive (mount_rclone in the script). Then you have your local files (mnt/user/local in the script) and the merger folder of those 2 (mount_mergerfs in the script).

 

2. This is where your share structure also confuses me. You don't use the local share in your mount script, you put it on ignore. So your current setup will just use the mounted crypt rclone mount and the rclone cache. Your local files are not included in the merger. This means you are working fully in the cloud and don't have any of your media associated files locally. I personally store the small files like subtitles and nfo and artwork locally and merge that share with my cloud mount.

 

3. Like you did right now, the Share rclonecache only uses Disk 1. Just know that the depending on your setup/wan speed/amount of users the rclone cache advantage can be pretty small. I used to use it, but I stopped and don't notice any difference, even though it uses a lot of storage. And I even stored it on an SSD. Putting a cache on a HDD can work, but I'm not really sure you are really getting the advantages you desire.

 

To explain the cache: without cache you play a media file and it will write the buffer to your RAM and will play from there, thus initiating a new download every time the file is accessed.

With cache: the buffer will be written to your VFS cache-dir disk and from there it will play. If you access that file later again it will play from your cache instead of downloading again.

 

This can be useful when you have a multiple people accessing the same media and it prevents having to download the file every time when someone start the media playback and is easier on your RAM. But if you just have a few users or have different media interests it will just download Episode 1 to the disk, then you play Episode 2, Then episode 3, etc. And then once the cache is full it will just delete the unused cache files. There won't be any advantage because nobody is looking at Episode 1 within a short time span again. So take your situation in consideration before you take on the hassle to get a cache working.

 

vfs-read-ahead only works with vfs-cache-mode full, so when you don't use the cache you don't need this flag.

 

4. My upload script has -vv in it and this shows me what's going on when I look at the log within the User Scripts section of Unraid.

 

This is a new way of using Rclone for me, thank you for helping me! I must not have been paying attention to all the information when I installed it.

 

1. I have now tried to clean up all my shares, take a look below:

 

An overview:

V9WQfNg.png

 

Specific details of the shares:

 

cryptmedia:

 

qacg6s2.png

 

mount_mergerfs

vmMnPGY.png

 

rclonecache

 

mxYoRhW.png

 

system

 

UYgtjpF.png

 

torrent

 

IOWJxWb.png

 

 

 

 

 

2. Yes, I must have not utilized the local share properly, I see that now!

I use Plex, Radarr and Sonarr. I must make some adjustments so that it is used correct. Am I thinking correct when I want to:

 

OPTION 1

    1. Radarr sends torrent to rTorrent.
    2. rTorrent downloads torrent to /mnt/user/torrent/seeding
    3. Radarr sends hard-copy link to /mnt/user/local/movies/Movie 1 (2022)/Movie1.mkv
    4. Plex gets access to Movie 1 via /mnt/user/mount_mergerfs/Movie 1 (2022)/Movie1.mkv
    5. Upload script copies Movie1.mkv to crypt:media/movies/Movie 1 (2022)/Movie1.mkv
    6. When torrent is seeded long enough deletes file from /mnt/user/torrent/seeding/movie.mkv
    7. Plex continues to have access to moviefile

 

OPTION 2

 

    1. Radarr sends torrent to rTorrent.
    2. rTorrent downloads torrent to /mnt/user/torrent/seeding
    3. Radarr imports (COPY) movie to /mnt/user/local/movies/Movie 1 (2022)/Movie1.mkv
    4. Plex gets access to Movie 1 via /mnt/user/mount_mergerfs/Movie 1 (2022)/Movie1.mkv
    5. Upload script moves Movie1.mkv to crypt:media/movies/Movie 1 (2022)/Movie1.mkv
    6. When torrent is seeded long enough deletes file from /mnt/user/torrent/seeding/movie.mkv
    7. Plex continues to have access to moviefile

 

I have very limited upload speed only 20 mbit. 

 

What is the best way to setup Radarr and Sonarr?
 

I also have several Kodi-boxes accessing the crypt:media, so I would still like to have the .nfo's, subtitles and posters in the cloud.

This command, are these the folders inside /mnt/user/local that I want to be mounted in /mnt/user/mount_mergerfs?

 

    MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount

 

 

3. Thank you very much, by your explaination I would like to stop using rclone-cache. How do I disable it ?

Under the required settings, do I remove these? Or set them to "ignore"

 

RcloneCacheShare="/mnt/user0/rclonecache" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone
RcloneCacheMaxSize="400G" # Maximum size of rclone cache
RcloneCacheMaxAge="336h" # Maximum age of cache files

 

In the rclone mount-settings do I remove these ?

 

    --dir-cache-time $RcloneMountDirCacheTime \
    --attr-timeout $RcloneMountDirCacheTime \
    --cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \



After the share is empty, then I can delete the share "rclonecache" ?

 

4. Well, this is actually embarassing. I have not used the upload script at all. I have added a proposal to the script here. Does it look okay ?

 

Sorry for the bombardment of questions.... I have to rethink all the prosesses.

 

Spoiler
#!/bin/bash

######################
### Upload Script ####
######################
### Version 0.95.5 ###
######################

####### EDIT ONLY THESE SETTINGS #######

# INSTRUCTIONS
# 1. Edit the settings below to match your setup
# 2. NOTE: enter RcloneRemoteName WITHOUT ':'
# 3. Optional: Add additional commands or filters
# 4. Optional: Use bind mount settings for potential traffic shaping/monitoring
# 5. Optional: Use service accounts in your upload remote
# 6. Optional: Use backup directory for rclone sync jobs

# REQUIRED SETTINGS
RcloneCommand="copy" # choose your rclone command e.g. move, copy, sync
RcloneRemoteName="cryptmedia" # Name of rclone remote mount WITHOUT ':'.
RcloneUploadRemoteName="cryptmedia" # If you have a second remote created for uploads put it here.  Otherwise use the same remote as RcloneRemoteName.
LocalFilesShare="/mnt/user/local" # location of the local files without trailing slash you want to rclone to use
RcloneMountShare="/mnt/user/cryptmedia" # where your rclone mount is located without trailing slash  e.g. /mnt/user/mount_rclone
MinimumAge="15m" # sync files suffix ms|s|m|h|d|w|M|y
ModSort="ascending" # "ascending" oldest files first, "descending" newest files first

# Note: Again - remember to NOT use ':' in your remote name above

# Bandwidth limits: specify the desired bandwidth in kBytes/s, or use a suffix b|k|M|G. Or 'off' or '0' for unlimited.  The script uses --drive-stop-on-upload-limit which stops the script if the 750GB/day limit is achieved, so you no longer have to slow 'trickle' your files all day if you don't want to e.g. could just do an unlimited job overnight.
BWLimit1Time="01:00"
BWLimit1="off"
BWLimit2Time="08:00"
BWLimit2="15M"
BWLimit3Time="16:00"
BWLimit3="12M"

# OPTIONAL SETTINGS

# Add name to upload job
JobName="_daily_upload" # Adds custom string to end of checker file.  Useful if you're running multiple jobs against the same remote.

# Add extra commands or filters
Command1="--exclude downloads/**"
Command2="-vv"
Command3=""
Command4=""
Command5=""
Command6=""
Command7=""
Command8=""

# Bind the mount to an IP address
CreateBindMount="N" # Y/N. Choose whether or not to bind traffic to a network adapter.
RCloneMountIP="192.168.1.253" # Choose IP to bind upload to.
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended.
VirtualIPNumber="1" # creates eth0:x e.g. eth0:1.

# Use Service Accounts.  Instructions: https://github.com/xyou365/AutoRclone
UseServiceAccountUpload="N" # Y/N. Choose whether to use Service Accounts.
ServiceAccountDirectory="/mnt/user/appdata/other/rclone/service_accounts" # Path to your Service Account's .json files.
ServiceAccountFile="sa_gdrive_upload" # Enter characters before counter in your json files e.g. for sa_gdrive_upload1.json -->sa_gdrive_upload100.json, enter "sa_gdrive_upload".
CountServiceAccounts="15" # Integer number of service accounts to use.

# Is this a backup job
BackupJob="N" # Y/N. Syncs or Copies files from LocalFilesLocation to BackupRemoteLocation, rather than moving from LocalFilesLocation/RcloneRemoteName
BackupRemoteLocation="backup" # choose location on mount for deleted sync files
BackupRemoteDeletedLocation="backup_deleted" # choose location on mount for deleted sync files
BackupRetention="90d" # How long to keep deleted sync files suffix ms|s|m|h|d|w|M|y

####### END SETTINGS #######

###############################################################################
#####    DO NOT EDIT BELOW THIS LINE UNLESS YOU KNOW WHAT YOU ARE DOING   #####
###############################################################################

####### Preparing mount location variables #######
if [[  $BackupJob == 'Y' ]]; then
	LocalFilesLocation="$LocalFilesShare"
	echo "$(date "+%d.%m.%Y %T") INFO: *** Backup selected.  Files will be copied or synced from ${LocalFilesLocation} for ${RcloneUploadRemoteName} ***"
else
	LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName"
	echo "$(date "+%d.%m.%Y %T") INFO: *** Rclone move selected.  Files will be moved from ${LocalFilesLocation} for ${RcloneUploadRemoteName} ***"
fi

RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location of rclone mount

####### create directory for script files #######
mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName #for script files

#######  Check if script already running  ##########
echo "$(date "+%d.%m.%Y %T") INFO: *** Starting rclone_upload script for ${RcloneUploadRemoteName} ***"
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Exiting as script already running."
	exit
else
	echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."
	touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName
fi

#######  check if rclone installed  ##########
echo "$(date "+%d.%m.%Y %T") INFO: Checking if rclone installed successfully."
if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: rclone installed successfully - proceeding with upload."
else
	echo "$(date "+%d.%m.%Y %T") INFO: rclone not installed - will try again later."
	rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName
	exit
fi

####### Rotating serviceaccount.json file if using Service Accounts #######
if [[ $UseServiceAccountUpload == 'Y' ]]; then
	cd /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/
	CounterNumber=$(find -name 'counter*' | cut -c 11,12)
	CounterCheck="1"
	if [[ "$CounterNumber" -ge "$CounterCheck" ]];then
		echo "$(date "+%d.%m.%Y %T") INFO: Counter file found for ${RcloneUploadRemoteName}."
	else
		echo "$(date "+%d.%m.%Y %T") INFO: No counter file found for ${RcloneUploadRemoteName}. Creating counter_1."
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_1
		CounterNumber="1"
	fi
	ServiceAccount="--drive-service-account-file=$ServiceAccountDirectory/$ServiceAccountFile$CounterNumber.json"
	echo "$(date "+%d.%m.%Y %T") INFO: Adjusted service_account_file for upload remote ${RcloneUploadRemoteName} to ${ServiceAccountFile}${CounterNumber}.json based on counter ${CounterNumber}."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Uploading using upload remote ${RcloneUploadRemoteName}"
	ServiceAccount=""
fi

#######  Upload files  ##########

# Check bind option
if [[  $CreateBindMount == 'Y' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for upload to remote ${RcloneUploadRemoteName}"
	ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform
	if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
		echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for upload to remote ${RcloneUploadRemoteName}"
	else
		echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for upload to remote ${RcloneUploadRemoteName}"
		ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber
	fi
else
	RCloneMountIP=""
fi

#  Remove --delete-empty-src-dirs if rclone sync or copy
if [[  $RcloneCommand == 'move' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Using rclone move - will add --delete-empty-src-dirs to upload."
	DeleteEmpty="--delete-empty-src-dirs "
else
	echo "$(date "+%d.%m.%Y %T") INFO: *** Not using rclone move - will remove --delete-empty-src-dirs to upload."
	DeleteEmpty=""
fi

#  Check --backup-directory
if [[  $BackupJob == 'Y' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Will backup to ${BackupRemoteLocation} and use  ${BackupRemoteDeletedLocation} as --backup-directory with ${BackupRetention} retention for ${RcloneUploadRemoteName}."
	LocalFilesLocation="$LocalFilesShare"
	BackupDir="--backup-dir $RcloneUploadRemoteName:$BackupRemoteDeletedLocation"
else
	BackupRemoteLocation=""
	BackupRemoteDeletedLocation=""
	BackupRetention=""
	BackupDir=""
fi

# process files
	rclone $RcloneCommand $LocalFilesLocation $RcloneUploadRemoteName:$BackupRemoteLocation $ServiceAccount $BackupDir \
	--user-agent="$RcloneUploadRemoteName" \
	-vv \
	--buffer-size 512M \
	--drive-chunk-size 512M \
	--tpslimit 8 \
	--checkers 8 \
	--transfers 4 \
	--order-by modtime,$ModSort \
	--min-age $MinimumAge \
	$Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \
	--exclude *fuse_hidden* \
	--exclude *_HIDDEN \
	--exclude .recycle** \
	--exclude .Recycle.Bin/** \
	--exclude *.backup~* \
	--exclude *.partial~* \
	--drive-stop-on-upload-limit \
	--bwlimit "${BWLimit1Time},${BWLimit1} ${BWLimit2Time},${BWLimit2} ${BWLimit3Time},${BWLimit3}" \
	--bind=$RCloneMountIP $DeleteEmpty

# Delete old files from mount
if [[  $BackupJob == 'Y' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Removing files older than ${BackupRetention} from $BackupRemoteLocation for ${RcloneUploadRemoteName}."
	rclone delete --min-age $BackupRetention $RcloneUploadRemoteName:$BackupRemoteDeletedLocation
fi

#######  Remove Control Files  ##########

# update counter and remove other control files
if [[  $UseServiceAccountUpload == 'Y' ]]; then
	if [[ "$CounterNumber" == "$CountServiceAccounts" ]];then
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_*
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_1
		echo "$(date "+%d.%m.%Y %T") INFO: Final counter used - resetting loop and created counter_1."
	else
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_*
		CounterNumber=$((CounterNumber+1))
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_$CounterNumber
		echo "$(date "+%d.%m.%Y %T") INFO: Created counter_${CounterNumber} for next upload run."
	fi
else
	echo "$(date "+%d.%m.%Y %T") INFO: Not utilising service accounts."
fi

# remove dummy file
rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName
echo "$(date "+%d.%m.%Y %T") INFO: Script complete"

exit

 

 

Edited by Halvliter
added a question
Link to comment
49 minutes ago, Halvliter said:

 

This is a new way of using Rclone for me, thank you for helping me! I must not have been paying attention to all the information when I installed it.

 

1. I have now tried to clean up all my shares, take a look below:

 

An overview:

V9WQfNg.png

 

Specific details of the shares:
<deleted for readability>

 

2. Yes, I must have not utilized the local share properly, I see that now!

I use Plex, Radarr and Sonarr. I must make some adjustments so that it is used correct. Am I thinking correct when I want to:

 

    1. Radarr sends torrent to rTorrent.
    2. rTorrent downloads torrent to /mnt/user/torrent/seeding
    3. Radarr sends hard-copy link to /mnt/user/local/movies/Movie 1 (2022)/Movie1.mkv
    4. Plex gets access to Movie 1 via /mnt/user/mount_mergerfs/Movie 1 (2022)/Movie1.mkv
    5. Upload script copies Movie1.mkv to crypt:media/movies/Movie 1 (2022)/Movie1.mkv
    6. When torrent is seeded long enough deletes file from /mnt/user/torrent/seeding/movie.mkv
    7. Plex continues to have access to moviefile

 

I also have several Kodi-boxes accessing the crypt:media, so I would still like to have the .nfo's, subtitles and posters in the cloud.

This command, are these the folders inside /mnt/user/local that I want to be mounted in /mnt/user/mount_mergerfs?

 

    MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount

 

 

3. Thank you very much, by your explaination I would like to stop using rclone-cache. How do I disable it ?

Under the required settings, do I remove these? Or set them to "ignore"

 

RcloneCacheShare="/mnt/user0/rclonecache" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone
RcloneCacheMaxSize="400G" # Maximum size of rclone cache
RcloneCacheMaxAge="336h" # Maximum age of cache files

 

In the rclone mount-settings do I remove these ?

 

    --dir-cache-time $RcloneMountDirCacheTime \
    --attr-timeout $RcloneMountDirCacheTime \
    --cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \



After the share is empty, then I can delete the share "rclonecache" ?

 

4. Well, this is actually embarassing. I have not used the upload script at all. I have added a proposal to the script here. Does it look okay ?

 

  Hide contents
#!/bin/bash

######################
### Upload Script ####
######################
### Version 0.95.5 ###
######################

####### EDIT ONLY THESE SETTINGS #######

# INSTRUCTIONS
# 1. Edit the settings below to match your setup
# 2. NOTE: enter RcloneRemoteName WITHOUT ':'
# 3. Optional: Add additional commands or filters
# 4. Optional: Use bind mount settings for potential traffic shaping/monitoring
# 5. Optional: Use service accounts in your upload remote
# 6. Optional: Use backup directory for rclone sync jobs

# REQUIRED SETTINGS
RcloneCommand="copy" # choose your rclone command e.g. move, copy, sync
RcloneRemoteName="cryptmedia" # Name of rclone remote mount WITHOUT ':'.
RcloneUploadRemoteName="cryptmedia" # If you have a second remote created for uploads put it here.  Otherwise use the same remote as RcloneRemoteName.
LocalFilesShare="/mnt/user/local" # location of the local files without trailing slash you want to rclone to use
RcloneMountShare="/mnt/user/cryptmedia" # where your rclone mount is located without trailing slash  e.g. /mnt/user/mount_rclone
MinimumAge="15m" # sync files suffix ms|s|m|h|d|w|M|y
ModSort="ascending" # "ascending" oldest files first, "descending" newest files first

# Note: Again - remember to NOT use ':' in your remote name above

# Bandwidth limits: specify the desired bandwidth in kBytes/s, or use a suffix b|k|M|G. Or 'off' or '0' for unlimited.  The script uses --drive-stop-on-upload-limit which stops the script if the 750GB/day limit is achieved, so you no longer have to slow 'trickle' your files all day if you don't want to e.g. could just do an unlimited job overnight.
BWLimit1Time="01:00"
BWLimit1="off"
BWLimit2Time="08:00"
BWLimit2="15M"
BWLimit3Time="16:00"
BWLimit3="12M"

# OPTIONAL SETTINGS

# Add name to upload job
JobName="_daily_upload" # Adds custom string to end of checker file.  Useful if you're running multiple jobs against the same remote.

# Add extra commands or filters
Command1="--exclude downloads/**"
Command2="-vv"
Command3=""
Command4=""
Command5=""
Command6=""
Command7=""
Command8=""

# Bind the mount to an IP address
CreateBindMount="N" # Y/N. Choose whether or not to bind traffic to a network adapter.
RCloneMountIP="192.168.1.253" # Choose IP to bind upload to.
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended.
VirtualIPNumber="1" # creates eth0:x e.g. eth0:1.

# Use Service Accounts.  Instructions: https://github.com/xyou365/AutoRclone
UseServiceAccountUpload="N" # Y/N. Choose whether to use Service Accounts.
ServiceAccountDirectory="/mnt/user/appdata/other/rclone/service_accounts" # Path to your Service Account's .json files.
ServiceAccountFile="sa_gdrive_upload" # Enter characters before counter in your json files e.g. for sa_gdrive_upload1.json -->sa_gdrive_upload100.json, enter "sa_gdrive_upload".
CountServiceAccounts="15" # Integer number of service accounts to use.

# Is this a backup job
BackupJob="N" # Y/N. Syncs or Copies files from LocalFilesLocation to BackupRemoteLocation, rather than moving from LocalFilesLocation/RcloneRemoteName
BackupRemoteLocation="backup" # choose location on mount for deleted sync files
BackupRemoteDeletedLocation="backup_deleted" # choose location on mount for deleted sync files
BackupRetention="90d" # How long to keep deleted sync files suffix ms|s|m|h|d|w|M|y

####### END SETTINGS #######

###############################################################################
#####    DO NOT EDIT BELOW THIS LINE UNLESS YOU KNOW WHAT YOU ARE DOING   #####
###############################################################################

####### Preparing mount location variables #######
if [[  $BackupJob == 'Y' ]]; then
	LocalFilesLocation="$LocalFilesShare"
	echo "$(date "+%d.%m.%Y %T") INFO: *** Backup selected.  Files will be copied or synced from ${LocalFilesLocation} for ${RcloneUploadRemoteName} ***"
else
	LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName"
	echo "$(date "+%d.%m.%Y %T") INFO: *** Rclone move selected.  Files will be moved from ${LocalFilesLocation} for ${RcloneUploadRemoteName} ***"
fi

RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location of rclone mount

####### create directory for script files #######
mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName #for script files

#######  Check if script already running  ##########
echo "$(date "+%d.%m.%Y %T") INFO: *** Starting rclone_upload script for ${RcloneUploadRemoteName} ***"
if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Exiting as script already running."
	exit
else
	echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding."
	touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName
fi

#######  check if rclone installed  ##########
echo "$(date "+%d.%m.%Y %T") INFO: Checking if rclone installed successfully."
if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: rclone installed successfully - proceeding with upload."
else
	echo "$(date "+%d.%m.%Y %T") INFO: rclone not installed - will try again later."
	rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName
	exit
fi

####### Rotating serviceaccount.json file if using Service Accounts #######
if [[ $UseServiceAccountUpload == 'Y' ]]; then
	cd /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/
	CounterNumber=$(find -name 'counter*' | cut -c 11,12)
	CounterCheck="1"
	if [[ "$CounterNumber" -ge "$CounterCheck" ]];then
		echo "$(date "+%d.%m.%Y %T") INFO: Counter file found for ${RcloneUploadRemoteName}."
	else
		echo "$(date "+%d.%m.%Y %T") INFO: No counter file found for ${RcloneUploadRemoteName}. Creating counter_1."
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_1
		CounterNumber="1"
	fi
	ServiceAccount="--drive-service-account-file=$ServiceAccountDirectory/$ServiceAccountFile$CounterNumber.json"
	echo "$(date "+%d.%m.%Y %T") INFO: Adjusted service_account_file for upload remote ${RcloneUploadRemoteName} to ${ServiceAccountFile}${CounterNumber}.json based on counter ${CounterNumber}."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Uploading using upload remote ${RcloneUploadRemoteName}"
	ServiceAccount=""
fi

#######  Upload files  ##########

# Check bind option
if [[  $CreateBindMount == 'Y' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for upload to remote ${RcloneUploadRemoteName}"
	ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform
	if [ $? -eq 0 ]; then # ping returns exit status 0 if successful
		echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for upload to remote ${RcloneUploadRemoteName}"
	else
		echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for upload to remote ${RcloneUploadRemoteName}"
		ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber
	fi
else
	RCloneMountIP=""
fi

#  Remove --delete-empty-src-dirs if rclone sync or copy
if [[  $RcloneCommand == 'move' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Using rclone move - will add --delete-empty-src-dirs to upload."
	DeleteEmpty="--delete-empty-src-dirs "
else
	echo "$(date "+%d.%m.%Y %T") INFO: *** Not using rclone move - will remove --delete-empty-src-dirs to upload."
	DeleteEmpty=""
fi

#  Check --backup-directory
if [[  $BackupJob == 'Y' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Will backup to ${BackupRemoteLocation} and use  ${BackupRemoteDeletedLocation} as --backup-directory with ${BackupRetention} retention for ${RcloneUploadRemoteName}."
	LocalFilesLocation="$LocalFilesShare"
	BackupDir="--backup-dir $RcloneUploadRemoteName:$BackupRemoteDeletedLocation"
else
	BackupRemoteLocation=""
	BackupRemoteDeletedLocation=""
	BackupRetention=""
	BackupDir=""
fi

# process files
	rclone $RcloneCommand $LocalFilesLocation $RcloneUploadRemoteName:$BackupRemoteLocation $ServiceAccount $BackupDir \
	--user-agent="$RcloneUploadRemoteName" \
	-vv \
	--buffer-size 512M \
	--drive-chunk-size 512M \
	--tpslimit 8 \
	--checkers 8 \
	--transfers 4 \
	--order-by modtime,$ModSort \
	--min-age $MinimumAge \
	$Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \
	--exclude *fuse_hidden* \
	--exclude *_HIDDEN \
	--exclude .recycle** \
	--exclude .Recycle.Bin/** \
	--exclude *.backup~* \
	--exclude *.partial~* \
	--drive-stop-on-upload-limit \
	--bwlimit "${BWLimit1Time},${BWLimit1} ${BWLimit2Time},${BWLimit2} ${BWLimit3Time},${BWLimit3}" \
	--bind=$RCloneMountIP $DeleteEmpty

# Delete old files from mount
if [[  $BackupJob == 'Y' ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: *** Removing files older than ${BackupRetention} from $BackupRemoteLocation for ${RcloneUploadRemoteName}."
	rclone delete --min-age $BackupRetention $RcloneUploadRemoteName:$BackupRemoteDeletedLocation
fi

#######  Remove Control Files  ##########

# update counter and remove other control files
if [[  $UseServiceAccountUpload == 'Y' ]]; then
	if [[ "$CounterNumber" == "$CountServiceAccounts" ]];then
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_*
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_1
		echo "$(date "+%d.%m.%Y %T") INFO: Final counter used - resetting loop and created counter_1."
	else
		rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_*
		CounterNumber=$((CounterNumber+1))
		touch /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/counter_$CounterNumber
		echo "$(date "+%d.%m.%Y %T") INFO: Created counter_${CounterNumber} for next upload run."
	fi
else
	echo "$(date "+%d.%m.%Y %T") INFO: Not utilising service accounts."
fi

# remove dummy file
rm /mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running$JobName
echo "$(date "+%d.%m.%Y %T") INFO: Script complete"

exit

 

 

1. I would put all the shares you only use to mount cloud stored files on (so nothing locally, or not much) to Prefer cache. It prevents to disturb your array disks from sleeping or having to switch read/write which is slow on a HDD. Other than that it looks fine

 

2. Let me state beforehand I don't use torrents, so this is a part of your workflow I don't use personally. However I wonder why you want to upload files before seeding? You could decide to add the Torrent share to your merged folder of local/cryptmedia cloud/torrents. But you have to be aware of using the right folder structures. Maybe you can combine local and torrents and you only have (local/torrents) and (cloud) as 2 folders that are merged. Plex would still see the file the same since it's part of mount_mergerfs. It's up to you though.

 

Regarding the folders in the script, these are folders that will be created in local folder in case they are not already there. I don't use DZMM's mount script because I have a bit of a different use case. But I also don't create folders within the main folders. I only have local, mount_rclone and mount_mergerfs. But like I said, I don't upload everything like he and you do. So you can try it out, with or without that list of folders. If you don't want to use that command, just remove the folder names or put in 1 dummy folder name. Don't just delete the whole line, that will mess up the script.

 

3. Leave the top parameters, delete the flags further down in the script with the list of -- XX flags. Keep dir-cache-time and attr-timeout, but remove cache-dir. Dir-cache is different than VFS cache.

Delete:

--cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \

--vfs-cache-mode full \

--vfs-cache-max-size $RcloneCacheMaxSize \

--vfs-cache-max-age $RcloneCacheMaxAge \

--vfs-read-ahead 1G \

 

Just FYI, I use these values for my mount script which differ a bit from DZMM:

--allow-other --umask 002 --buffer-size 256M --dir-cache-time 9999h --drive-chunk-size 512M --attr-timeout 1s --poll-interval 1m --drive-pacer-min-sleep 10ms --drive-pacer-burst 500 --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off

 

After the share rclonecache is empty you can just delete it yes. Make sure you reboot after all the changes, so the cache is also actually stopped. Even better is to disable your dockers and mount script, reboot and then make the changes and delete the share and then start everything up.

 

4. You use copy instead of move, any reason for that?

These parameters are a bit ambiguous and can go wrong easily if you have a different setup/naming scheme. So to explain:

 

RcloneRemoteName= This is the foldername where everything is stored. So if your crypt mount name (cryptmedia) is the same as the folder in your local and mount_mergerfs folders  this is fine. But for example if your rclone mount is crypt_media: but your folder is actually mount_mergerfs/Crypt-Media you have to fill in Crypt-Media here.

RcloneUploadRemoteName= use your actual rclone mount name here, not the folder name.

LocalFilesShare= location of the local files, so yours is fine like this

RcloneMountShare= this is the main folder directory in which the folder to which you mount the rclone mount is nested. So in your case you used "/mnt/user/cryptmedia". But that means the script as you configured it now will upload files from "/mnt/user/cryptmedia/cryptmedia". If that is correct, then no problem. But if your media folder structure starts directly in the share cryptmedia you would have to fill in "/mnt/user" here.

 

Remove -vv at command 2. It's already part of the default upload script as you can see further down below.

  • Like 1
Link to comment
On 9/27/2022 at 5:51 PM, DZMM said:

Has anybody had any problems running the scripts on 6.11?  Does the mergerfs install still work without NerdPack?

So after I upgraded my main uploading unRAID server from 6.9 series to 6.11 I noticed how permissions could mess up using the scripts from GitHub.

My mount script is a bit different - I create a “backup” folder acting as sort of Dropbox for uploads to gdrive at:
/mnt/user/local/gdrive_vfs/backup

This folder would get deleted after rclone_upload completes and then get recreated with rclone_mount (with root:root).

In DZMM rclone_mount the MountFolders bit:

MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount


In DZMM rclone_upload:

Command1="--exclude downloads/**"


So the paths from MountFolders:

/mnt/user/local/gdrive_vfs/movies
/mnt/user/local/gdrive_vfs/tv


will get deleted (after uploading) - then the rclone_mount script will create them again with root:root (and dockers trying to use them will give permission errors.

Is there a way for the rclone_mount script to create the empty MountFolders with nobody:user permissions?

Link to comment
2 hours ago, live4ever said:

So after I upgraded my main uploading unRAID server from 6.9 series to 6.11 I noticed how permissions could mess up using the scripts from GitHub.

My mount script is a bit different - I create a “backup” folder acting as sort of Dropbox for uploads to gdrive at:
/mnt/user/local/gdrive_vfs/backup

This folder would get deleted after rclone_upload completes and then get recreated with rclone_mount (with root:root).

In DZMM rclone_mount the MountFolders bit:

MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount


In DZMM rclone_upload:

Command1="--exclude downloads/**"


So the paths from MountFolders:

/mnt/user/local/gdrive_vfs/movies
/mnt/user/local/gdrive_vfs/tv


will get deleted (after uploading) - then the rclone_mount script will create them again with root:root (and dockers trying to use them will give permission errors.

Is there a way for the rclone_mount script to create the empty MountFolders with nobody:user permissions?

 

Maybe just add a chown command at the end of the script somewhere before the last exit?

chown -R nobody:users /mnt/user/local

 

Link to comment
7 hours ago, francrouge said:

anyone having issue with permission i can't edit any file or folder on my windows with network share

It's a pretty common issue throughout the latest topics. It's probably because the files are owned by root. Did you check the files permissions which you try to open?

Link to comment
15 hours ago, Kaizac said:

It's a pretty common issue throughout the latest topics. It's probably because the files are owned by root. Did you check the files permissions which you try to open?

hi yes  on krusader i got no problem but on windows  with network shares its not working anymore

image.png.280f532277af0c32d9b6442ad9ece11f.png

image.png.8587cc2e10568da03f901685502de95b.png

i can't edit or rename delete etc. on the gdrive mount on windows my local shares are ok

image.png.a07182d78fc62551a8b954364d57cc38.png

 

 

 

I will add also my mount and upload script

 

Maybe i'm missing something

 

 

 

should i try the new permission feature you think ?

 

thx

upload.txt mount.txt unraid-diagnostics-20221004-1818.zip

Edited by francrouge
add scripts
Link to comment
On 10/2/2022 at 4:18 PM, Kaizac said:

So why don't you put in that manual command in a seperate cron job to execute at start of the system? And if that overlaps with the rclone scripts, add a wait to those scripts.

I finally found the right lines where I needed to insert it, thank you.

 

The mount script now also runs without error messages, but I cannot use rclone as I did with my old server (A).
There (A) I can enter the following line and have direct access to the files in the cloud, but this does not work on the new server (B). 

ls /mnt/gsuite/
>List of all connected cloud files< 

In the new server (B), I cannot find the appropriate /mnt/ to access the data.

 

Here is the comparison of the old- (A) und new- (B) server:

Old-Server (A) | 1. Start server (ssd mounted) → 2. cmd → 3. past mount line → 4. rclone is mounted

CONFIG:
[gsuite]
type = drive
client_id = ##########
client_secret = ##########
scope = drive
team_drive =
token = ##########

[gcrypt]
type = crypt
remote = gsuite:/crypt/media
filename_encryption = off
directory_name_encryption = false
password = ##########
password2 = ##########

MOUNT:
rclone mount --allow-other --no-check-certificate --cache-dir=/cache --dir-cache-time 168h --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 72h gcrypt: /mnt/gsuite


--------------------------------------------------
New-Server (B) | 1. Start server → 2. mount array → 3. run "rclone mount script" → 4. ... is rclone mounted?

CONFIG:
[gsuite]
type = drive
client_id = ##########new
client_secret = ##########new
scope = drive
team_drive =
token = ##########new

[gcrypt]
type = crypt
remote = gsuite:/crypt/media
filename_encryption = off
directory_name_encryption = false
password = ##########
password2 = ##########

MOUNT_SCRIPT (only the changes i have made):
######################
#### Mount Script ####
######################
## Version 0.96.9.3 ##
######################

# REQUIRED SETTINGS
RcloneRemoteName="gcrypt" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data
RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash  e.g. /mnt/user/mount_rclone
RcloneMountDirCacheTime="168h" # rclone dir cache time
LocalFilesShare="/mnt/user/local" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable
RcloneCacheShare="/mnt/user0/mount_rclone_cache" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone
RcloneCacheMaxSize="100G" # Maximum size of rclone cache
RcloneCacheMaxAge="168h" # Maximum age of cache files
MergerfsMountShare="/mnt/user/mount_mergerfs" # location without trailing slash  e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable
DockerStart="plex tautulli tinymedia manager sabnzbd nzbget sonarr radarr ombi overseerr" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page
MountFolders=\{"load/nzb,load/complete,temp,movies,tv,anime,docu,concert"\} # comma separated list of folders to create within the mount

# Add extra paths to mergerfs mount in addition to LocalFilesShare
LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder.  Enter 'ignore' to disable
LocalFilesShare3="ignore"
LocalFilesShare4="ignore"

# Add extra commands or filters
Command1="--rc"
Command2="--config=/mnt/disk1/rclone/rclone.conf"
Command3="--allow-non-empty"

CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter
RCloneMountIP="192.168.2.200" # My unraid IP is 172.30.12.2 so I create another similar IP address
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended
VirtualIPNumber="2" # creates eth0:x e.g. eth0:1.  I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them

####### END SETTINGS #######

#######  Create Rclone Mount  #######

# Check If Rclone Mount Already Created
if [[ -f "$RcloneMountLocation/mountcheck" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted."
else
	echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote."
# Creating mountcheck file in case it doesn't already exist
	echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote."
	touch mountcheck
	rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse --config=/mnt/disk1/rclone/rclone.conf

 

1. does the script work as described above if I do not get any errors when I click on "run script" and it shows "script complete" at the end?
2. How can I access my cloud mount from the command line?
3. how can I stop the mount again (my array cannot be stopped (stop-loop) when the array is still running/ after I started the script)?

 

I don't have any other scripts running.
The mount is not visible in the array.

Link to comment
18 hours ago, francrouge said:

hi yes  on krusader i got no problem but on windows  with network shares its not working anymore

image.png.280f532277af0c32d9b6442ad9ece11f.png

image.png.8587cc2e10568da03f901685502de95b.png

i can't edit or rename delete etc. on the gdrive mount on windows my local shares are ok

image.png.a07182d78fc62551a8b954364d57cc38.png

 

 

 

I will add also my mount and upload script

 

Maybe i'm missing something

 

 

 

should i try the new permission feature you think ?

 

thx

 

Krusader working is not surprising since it uses the root account, so it's not limited with permissions. I looked over your scripts quickly and I see nothing strange. So I think it's an samba issue. Did you upgrade to 6.11? There were changes to sambe in there, maybe check that out? Could explain why it stopped working.

 

1 hour ago, undone said:

I finally found the right lines where I needed to insert it, thank you.

 

The mount script now also runs without error messages, but I cannot use rclone as I did with my old server (A).
There (A) I can enter the following line and have direct access to the files in the cloud, but this does not work on the new server (B). 

ls /mnt/gsuite/
>List of all connected cloud files< 

In the new server (B), I cannot find the appropriate /mnt/ to access the data.

 

Here is the comparison of the old- (A) und new- (B) server:

 

 

1. does the script work as described above if I do not get any errors when I click on "run script" and it shows "script complete" at the end?
2. How can I access my cloud mount from the command line?
3. how can I stop the mount again (my array cannot be stopped (stop-loop) when the array is still running/ after I started the script)?

 

I don't have any other scripts running.
The mount is not visible in the array.

 

Wow, I'm amazed your script for Server A worked....

 

You configured your Gcrypt as : gsuite:/crypt/media. But the correct method would be gsuite:crypt/media.

Then you also had your cache-dir in your A script to /cache. Did that work? In unraid you will have to define the actual path. Seems you did that correctly in the new script.

On the A script you didn't close the mount script with a "&", in the new script this is fixed already by default.

In your new script you put in --allow-non-empty as extra command, this is very risky to do. So make sure you thought about doing that.

 

What I find most worrying is that your crypt doesn't actually encrypt anything. Is that by choice? If you do want to switch to an actually encrypted Crypt you will have to send all your files through the crypt to your storage. It won't automatically encrypt all the files within that mount.

 

Your specific questions:

1. Don't use "Run script" in User Scripts. Always use run script in the background when running the script. If you use the run script option it will just stop the script as soon as you close the popup. That might explain why your mount drops right away.

2. You have the rclone commands here: https://rclone.org/commands/. Other than that you can for example use "rclone size gcrypt:" to see how big your gcrypt mount is in the cloud.

3. You can unmount with fusermount -uz /path/to/remote. Make sure you don't have any dockers or transfers running with access to that mount though, because they will start writing in your merger folder causing problems when you mount again.

  • Like 1
Link to comment
12 minutes ago, Kaizac said:

Wow, I'm amazed your script for Server A worked....

 

You configured your Gcrypt as : gsuite:/crypt/media. But the correct method would be gsuite:crypt/media.

Then you also had your cache-dir in your A script to /cache. Did that work? In unraid you will have to define the actual path. Seems you did that correctly in the new script.

On the A script you didn't close the mount script with a "&", in the new script this is fixed already by default.

Well, after a few initial difficulties a couple of years ago, it has been running without any problems so far.

  • gsuite:/crypt/media works
  • /cache leads to /mnt/cache and also works
  • the "&" is included, it just was not copied
41 minutes ago, Kaizac said:

In your new script you put in --allow-non-empty as extra command, this is very risky to do. So make sure you thought about doing that.

I added it because of a failure running the script. After a reboot I seems to work without it.

 

42 minutes ago, Kaizac said:

What I find most worrying is that your crypt doesn't actually encrypt anything. Is that by choice? If you do want to switch to an actually encrypted Crypt you will have to send all your files through the crypt to your storage. It won't automatically encrypt all the files within that mount.

What do you mean with "anything"?

The file is encrypted with the password in the crypt config and the naming can be anything, nothing to hide behind my 'Merval Endgames 37 8K Premium 20.4 THEx.mp3.bin'.

 

45 minutes ago, Kaizac said:

Your specific questions:

1. Don't use "Run script" in User Scripts. Always use run script in the background when running the script. If you use the run script option it will just stop the script as soon as you close the popup. That might explain why your mount drops right away.

2. You have the rclone commands here: https://rclone.org/commands/. Other than that you can for example use "rclone size gcrypt:" to see how big your gcrypt mount is in the cloud.

3. You can unmount with fusermount -uz /path/to/remote. Make sure you don't have any dockers or transfers running with access to that mount though, because they will start writing in your merger folder causing problems when you mount again.

  1. That was obviously the problem. If it runs in the background, there is also a connection in the appropriate folder.
  2. With the background process as predefined, it can simply be found at /mnt/user/mount_rclone/gcrypt/ .
  3. With the path from 2. the following also works (fusermount -uz /mnt/user/mount_rclone/gcrypt && fusermount -uz /mnt/user/mount_mergerfs/gcrypt/) and the array can be stopped.

Thank you very much, I hope that is it for now with my problems.

Link to comment
52 minutes ago, undone said:

Well, after a few initial difficulties a couple of years ago, it has been running without any problems so far.

  • gsuite:/crypt/media works
  • /cache leads to /mnt/cache and also works
  • the "&" is included, it just was not copied

I added it because of a failure running the script. After a reboot I seems to work without it.

 

What do you mean with "anything"?

The file is encrypted with the password in the crypt config and the naming can be anything, nothing to hide behind my 'Merval Endgames 37 8K Premium 20.4 THEx.mp3.bin'.

 

  1. That was obviously the problem. If it runs in the background, there is also a connection in the appropriate folder.
  2. With the background process as predefined, it can simply be found at /mnt/user/mount_rclone/gcrypt/ .
  3. With the path from 2. the following also works (fusermount -uz /mnt/user/mount_rclone/gcrypt && fusermount -uz /mnt/user/mount_mergerfs/gcrypt/) and the array can be stopped.

Thank you very much, I hope that is it for now with my problems.

Glad it works now!

 

Regarding the encryption, I meant you don't do file or directory name encryption. And you made clear that's a deliberate choice. I was just surprised with that. Knowing that Google has been limiting new accounts and stopping unlimited storage and multiple stories of people who got their whole drive deleted because of copyright material. I personally don't want to take that risk.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.