Jump to content
DZMM

Guide: How To Use Rclone To Mount Cloud Drives And Play Files

1914 posts in this topic Last Reply

Recommended Posts

37 minutes ago, livingonline8 said:

So I have rclone setup and running...great! 

I used spaceinvador script to mount google drive as a share and i can access that share now... amazing! 

I installed emby and I tried to point the media path of to google share but I cannot see it... I see all my other shares that are physically on my unraid server but not the mounted google drive one ?!

 

Can anyone help me with this please? 

I’ve never used SpaceInvaders script, are you creating directories and mounting them using the script? I highly Highly recommend using the scripts created on page 1 by DZMM for mounting G drive, as it will stop you getting API bans from google. 

Share this post


Link to post
2 hours ago, livingonline8 said:

So I have rclone setup and running...great! 

I used spaceinvador script to mount google drive as a share and i can access that share now... amazing! 

I installed emby and I tried to point the media path of to google share but I cannot see it... I see all my other shares that are physically on my unraid server but not the mounted google drive one ?!

 

Can anyone help me with this please? 

You also go to McDonalds and ask for support on your Burger King menue? ;-)

Share this post


Link to post
On 10/12/2019 at 2:46 AM, Viperkc said:

Upgraded to 6.8rc1 and am getting this error.  Any help please

 

Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH

I just updated the rclone plugin and unraid to 6.8.0-rc1 and everything's working ok for me so far

Share this post


Link to post

yes confirmed working.

 

I did have question though.  For the past month or so my Radarr and Sonarr have been really slow to do the daily refresh.  At times the dockers will actually lockup and only a reboot will fix.  Anyone else had this issue?

 

I had read elsewhere of it happening because of something google is doing to throttle scans.

 

Thanks

 

Share this post


Link to post

BTW, i found something cool!

 

https://github.com/rclone/rclone-webui-react

 

Just run rclone rcd --rc-web-gui --rc-user=test --rc-pass=test --rc-addr IP:PORT in terminal and you have a GUI for rclone, wtf! xD

 

Then go to IP:Port in browser. Thats all! Crazy.

It seems liek you can change your config there. Theres also a "jobs" "status" and so on, but sadly that doesnt work, maybe someone know how to connect that dashboard to our rclone installs? :D (so it shows what its doing, i would really love that becasue most time i dont really understand why its downloading) [or does it only show stats from things done via GUI? :(]

1.png

Edited by nuhll

Share this post


Link to post
32 minutes ago, nuhll said:

BTW, i found something cool!

 

https://github.com/rclone/rclone-webui-react

 

Just run rclone rcd --rc-web-gui --rc-user=test --rc-pass=test --rc-addr IP:PORT in terminal and you have a GUI for rclone, wtf! xD

 

Then go to IP:Port in browser. Thats all! Crazy.

It seems liek you can change your config there. Theres also a "jobs" "status" and so on, but sadly that doesnt work, maybe someone know how to connect that dashboard to our rclone installs? :D (so it shows what its doing, i would really love that becasue most time i dont really understand why its downloading) [or does it only show stats from things done via GUI? :(]

1.png

it looks nice

 

maybe someone can make a update to install.

Share this post


Link to post

No, im just stupid... you can embed these commands into your mount and or move commands, im currently tryin to work it out.

Share this post


Link to post
48 minutes ago, nuhll said:

I got it working:

https://github.com/rclone/rclone-webui-react/issues/38

 

There are my 2 commands i use currently you need to add another script which only runs at the first start... now you can see what your mount is doing!

So to sum up, you have to create a new userscript that start at startup with:

rclone rcd --rc-web-gui --rc-addr :5555 &

 

and then add to your mount

--rc --rc-addr=YOURIP:YOURPORT --rc-web-gui --rc-user=test --rc-pass=test --rc-web-gui-update --stats=24h

And to your upload script

--rc --rc-addr=YOURIP:YOURPORT --rc-web-gui --rc-user=test --rc-pass=test --rc-web-gui-update --stats=24h

am I correct?

 

I don't understand the usage of the first script ? (rclone rcd --rc-web-gui --rc-addr :5555 &)

What is the purpose? Why a different port as the others?

Share this post


Link to post
11 hours ago, yendi said:

So to sum up, you have to create a new userscript that start at startup with:


rclone rcd --rc-web-gui --rc-addr :5555 &

 

and then add to your mount


--rc --rc-addr=YOURIP:YOURPORT --rc-web-gui --rc-user=test --rc-pass=test --rc-web-gui-update --stats=24h

And to your upload script


--rc --rc-addr=YOURIP:YOURPORT --rc-web-gui --rc-user=test --rc-pass=test --rc-web-gui-update --stats=24h

am I correct?

 

I don't understand the usage of the first script ? (rclone rcd --rc-web-gui --rc-addr :5555 &)

What is the purpose? Why a different port as the others?

Hey, yes that should be correct.

 

rclone rcd --rc-web-gui --rc-addr :5555 &

 

is just be used to download the latest web gui (bc unraid delete linux after ever restart) sadly the normal commands dont auto download the gui (it seems)

Share this post


Link to post

Just wanted to thank @DZMM for all of you work putting this together. I was using a different setup but yours fixes some of the issues I was having (upload speed) and generally seems to work better.

I haven't read all of the thread but I'm wondering if you have been able to set this up or switch it over to being cache only shares?

 

I have it working properly following your guide but when I switched it over to cache only shares it didn't work. The mounting script would complete but there was nothing in the rclone or union mounts.

 

I have a large fast SSD cache and wanted to limit activity to this. Any obvious reason why this doesn't work? Thanks

Share this post


Link to post

@nuhll: oh wow, that's cool. I shall try that tonight.

 

29 minutes ago, jude said:

I have it working properly following your guide but when I switched it over to cache only shares it didn't work. The mounting script would complete but there was nothing in the rclone or union mounts.

 

I have a large fast SSD cache and wanted to limit activity to this. Any obvious reason why this doesn't work? Thanks

Perhaps you need to change your mount script mount points to /mnt/cache/[something something]?

Share this post


Link to post
11 minutes ago, testdasi said:

Perhaps you need to change your mount script mount points to /mnt/cache/[something something]?

@jude /unionfs and /rclone_mount 'files' don't get stored anywhere - streaming is done in ram.  From memory trying to mount to /mnt/cache/....etc doesn't work - you have to use /mnt/user. 

 

If you want your pending upload files to only use the cache then set /mnt/user/rclone_upload to cache only.

Share this post


Link to post
7 minutes ago, DZMM said:

@jude /unionfs and /rclone_mount 'files' don't get stored anywhere - streaming is done in ram.  From memory trying to mount to /mnt/cache/....etc doesn't work - you have to use /mnt/user. 

 

If you want your pending upload files to only use the cache then set /mnt/user/rclone_upload to cache only.

Mounting to /mnt/cache won't work if docker is mapped to to /mnt/user. If both use /mnt/cache then it would work.

In other words, both the docker and rclone must be pointing to the exact same path, because as you said, streaming is done in RAM.

Share this post


Link to post
33 minutes ago, testdasi said:

@nuhll: oh wow, that's cool. I shall try that tonight.

 

Perhaps you need to change your mount script mount points to /mnt/cache/[something something]?

The mount script i posted seems to work, even if the "online since" stat is complete garbage xD

 

Finally i can see WHY i get disconnects  while gaming... today it was my music library... :/ maybe i should store these locally until i have my 1gbits.. :D

Share this post


Link to post
9 hours ago, testdasi said:

Mounting to /mnt/cache won't work if docker is mapped to to /mnt/user. If both use /mnt/cache then it would work.

In other words, both the docker and rclone must be pointing to the exact same path, because as you said, streaming is done in RAM.

I've switched it back to not cache only shares and its working well again. Sometimes its good to take the win rather than breaking it and dealing with the wrath of the family.

 

I've also noticed that sometimes my dockers are not seeing the mount points properly and disabling the dockers from the main docker settings page and then re-enabling brings it back to where Plex and Sonarr can access the unionfs mount. 

This isn't specifically related to these scripts or the unionfs mount as I was starting to have this issue prior to this. Not sure if its an issue with the current rclone beta or the unraid RC or a combination.

Share this post


Link to post
9 hours ago, testdasi said:

@nuhll: oh wow, that's cool. I shall try that tonight.

 

Perhaps you need to change your mount script mount points to /mnt/cache/[something something]?

Thats what I'm going to try the next chance I get to work on the server

Share this post


Link to post
6 hours ago, jude said:

I've switched it back to not cache only shares and its working well again. Sometimes its good to take the win rather than breaking it and dealing with the wrath of the family.

 

I've also noticed that sometimes my dockers are not seeing the mount points properly and disabling the dockers from the main docker settings page and then re-enabling brings it back to where Plex and Sonarr can access the unionfs mount. 

This isn't specifically related to these scripts or the unionfs mount as I was starting to have this issue prior to this. Not sure if its an issue with the current rclone beta or the unraid RC or a combination.

Just use my automatic script.

 

I dont have that problem with disconnecting mounts, but its my fail safe. I run it every minute, if mount is not working, stopping all mount related dockers, if mount is running again, starting dockers... (create/change directories to your liking)

 

#!/bin/bash
# Check if Mount workz
if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then
# Mount workz

echo "$(date "+%d.%m.%Y %T") INFO: Mount okay, starte Docker"

if [[ -f "/mnt/user/appdata/other/docker/docker_on" ]]; then
#docker schon gestartet.

echo "$(date "+%d.%m.%Y %T") INFO: Docker bereits gestartet."

exit

else
#docker noch nicht gestartet

logger $(date "+%d.%m.%Y %T") Docker noch nicht gestartet. Starte Docker...

docker start PlexMediaServer
docker start radarr
docker start Sonarr
docker start ombi
docker start lidarr

touch /mnt/user/appdata/other/docker/docker_on
rm -f /mnt/user/appdata/other/docker/docker_off

fi

else
if [[ -f "/mnt/user/appdata/other/docker/docker_off" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Docker bereits gestoppt."

exit

else

logger $(date "+%d.%m.%Y %T") Mount im Arsch. Docker stoppen...
docker stop PlexMediaServer -t 1
docker stop radarr -t 1
docker stop Sonarr -t 1
docker stop ombi -t 1
docker stop lidarr -t 1

rm -f /mnt/user/appdata/other/docker/docker_on
touch /mnt/user/appdata/other/docker/docker_off
fi
fi
 

Edited by nuhll

Share this post


Link to post
46 minutes ago, nuhll said:

Just use my automatic script.

 

I dont have that problem with disconnecting mounts, but its my fail safe. I run it every minute, if mount is not working, stopping all mount related dockers, if mount is running again, starting dockers... (create/change directories to your liking)

 

#!/bin/bash
# Check if Mount workz
if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then
# Mount workz

echo "$(date "+%d.%m.%Y %T") INFO: Mount okay, starte Docker"

if [[ -f "/mnt/user/appdata/other/docker/docker_on" ]]; then
#docker schon gestartet.

echo "$(date "+%d.%m.%Y %T") INFO: Docker bereits gestartet."

exit

else
#docker noch nicht gestartet

logger $(date "+%d.%m.%Y %T") Docker noch nicht gestartet. Starte Docker...

docker start PlexMediaServer
docker start radarr
docker start Sonarr
docker start ombi
docker start lidarr

touch /mnt/user/appdata/other/docker/docker_on
rm -f /mnt/user/appdata/other/docker/docker_off

fi

else
if [[ -f "/mnt/user/appdata/other/docker/docker_off" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Docker bereits gestoppt."

exit

else

logger $(date "+%d.%m.%Y %T") Mount im Arsch. Docker stoppen...
docker stop PlexMediaServer -t 1
docker stop radarr -t 1
docker stop Sonarr -t 1
docker stop ombi -t 1
docker stop lidarr -t 1

rm -f /mnt/user/appdata/other/docker/docker_on
touch /mnt/user/appdata/other/docker/docker_off
fi
fi
 

my mount script already manages automatic reconnects - just set the cron job to whatever frequency you want to check

Share this post


Link to post

Im not sure why i outcomented the start and stop commands of your script, but there was a reason. (or i just like it seperate, i cant remember)

Edited by nuhll

Share this post


Link to post

Looking at my system this morning and the mounts are all active (I can play files on Plex and browse folders). I did notice however that the upload script did not move three files to the cloud mount.

The log shows that in the hours proceeding other files were downloaded and then properly uploaded. Then the next time the script runs it finds a new local file and directories in rclone_upload but does not start uploading it

 

The script starts at 3:40am

The file was completed in NZBget at 3:27am and moved to rclone_upload 

The script requires minimum age of 20minutes

 

So I'm assuming it excluded the file due to its age but the problem is that the next time the script runs at 4am it again excludes the file and fails to upload it.

Since then another two files have been added to rclone_upload and they also get excluded when the upload script runs. 

If I manually start the script the same thing happens. Any ideas on why the script would work and then start failing after a certain amount of time?

 

edit

One of those files has since uploaded and i have test downloaded two more files and then run the upload script and both of those new files uploaded. The two old files are still sitting there and are excluded each time the upload script runs. Still have no idea why.

 

edit 

Turns out many of the weird issues were created by the rclone mount script creating a situation where I was running out of memory. I have reduced the memory demand to a --buffer-size 128M and now the rclone mount is stable and hasn't crashed once. I'm sure if I add more ram then I could bump this up. 

Edited by jude
update

Share this post


Link to post

Permissions problem?

 

I am having issues with Sonarr not being able to import and move files from the /downloads folder and then send them to the /mnt/cache/rclone_upload/google_vfs folder.

 

here is my mount script

 

else

echo "$(date "+%d.%m.%Y %T") INFO: mounting rclone vfs."

# create directories for rclone mount and unionfs mount

mkdir -p /mnt/user/appdata/other/rclone
mkdir -p /mnt/cache/mount_rclone/google_vfs
mkdir -p /mnt/cache/mount_unionfs/google_vfs
mkdir -p /mnt/cache/rclone_upload/google_vfs

rclone mount --allow-other --buffer-size 128M --attr-timeout 1000h --timeout 1h --dir-cache-time 1000h --fast-list --log-level INFO --umask 002 encrypted: /mnt/cache/mount_rclone/google_vfs &

# check if mount successful

# slight pause to give mount time to finalise

sleep 5

if [[ -f "/mnt/cache/mount_rclone/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check rclone gdrive vfs mount success."

else

echo "$(date "+%d.%m.%Y %T") CRITICAL: rclone gdrive vfs mount failed - please check for problems."

rm /mnt/user/appdata/other/rclone/rclone_mount_running

exit

fi

fi

#######  End rclone gdrive mount  ##########

#######  Start unionfs mount   ##########

if [[ -f "/mnt/cache/mount_unionfs/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs already mounted."

else

unionfs -o cow,direct_io,allow_other,auto_cache,sync_read /mnt/cache/rclone_upload/google_vfs=RW:/mnt/cache/mount_rclone/google_vfs=RO /mnt/cache/mount_unionfs/google_vfs

if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs mounted."

else

echo "$(date "+%d.%m.%Y %T") CRITICAL: unionfs Remount failed."

rm /mnt/user/appdata/other/rclone/rclone_mount_running

exit

fi

fi

#######  End Mount unionfs   ##########

############### starting dockers that need unionfs mount ######################

# only start dockers once

if [[ -f "/mnt/user/appdata/other/rclone/dockers_started" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: dockers already started"

rm /mnt/user/appdata/other/rclone/rclone_mount_running

else

touch /mnt/user/appdata/other/rclone/dockers_started

echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers."

#docker start plex 
#docker start 
#docker start 

fi

############### end dockers that need unionfs mount ######################

 

I was worried that this was an issue with the docker paths not matching correctly but I have those matching what has been recommended.

 

I stumbled across something odd. The downloaded files will be sitting in the downloads folder and Sonarr will not be able to import them. See below for typical error

 

Couldn't import episode /downloads/completed/tv/The.Amazing.Race.Canada.S06E01.720p.HDTV.x264-aAF/The.Amazing.Race.Canada.S06E01.720p.HDTV.x264-aAF.mkv: Access to the path "/media/The Amazing Race Canada/Season 6" is denied.

 

If I go to that pathway which is in  /mnt/cache/mount_unionfs/google_vfs and look at permissions type ls -al I find that

/mnt/cache/mount_unionfs/google_vfs/tv which maps to /media has permissions of

 

drwxrwxrwx 1 root   root          74 Oct 29 11:11 tv/

 

the folders within tv/ look like this

 

drwxrwxr-x 1 root   root     0 Sep  8 08:58 The\ Amazing\ Race\ Canada/

 

Now, if I go into /mnt/cache/rclone_upload/google_vfs and manually create the tv folder and series folder.

 

then Sonarr is able to import the show and move it to the rclone_upload folder

 

it now shows up in the union folder like this

root@Tower:/mnt/cache/mount_unionfs/google_vfs/tv/The Amazing Race Canada/Season 6# ls -al

 

drwxrwxrwx 1 root   root    22 Oct 29 11:47 The\ Amazing\ Race\ Canada/

and

-rw-r--r-- 1 nobody users 1706811776 Aug  8  2018 The\ Amazing\ Race\ Canada\ -\ S06E01\ -\ Just\ a\ Beaver\ Hero\ HDTV-720p.mkv

 

This is beyond my skill level to properly explain let alone fix so I hope that I have provided enough information.  

 

TLDR is that nothing is moved into rclone_upload until I manually create the category and series folder in rclone_upload. At that point Sonarr creates the season folder and moves the renamed file into it. At that point the upload script can do its job.

Unfortunately it then removes the category and series folder.

 

Are there unionfs flags that I can add or permissions code that needs to be run by the script.

Thanks

 

  

 

 

 

 

Edited by jude
update

Share this post


Link to post
23 hours ago, jude said:

/mnt/cache/rclone_upload/google_vfs

It sounds like you mapped Sonarr to the upload folder.

Sonarr should be mapped with the unionfs folder (and let unionfs control the use of upload folder).

 

 

 

 

Share this post


Link to post

I have an issue uploading files. Whenever the upload script is run, it complains that rclone isn't installed and halts. When I look in /user/mount_rclone, where previously the google_vfs directory was mounted containing a mount check file, there is now only the following - ?google_vfs. On using mc to click on this file, an error message appears stating "Cannot change directory".

 

I'm completely at a loss as to how to progress. All scripts are unmodified apart from the Mount script where the docker launches are removed.

Share this post


Link to post

I everbody

 

I need help since today i'm not able to watch anything that is uploaded to gdrive.

 

I got 2 server map with rclone and the script. When i tried to load something its just a cercle turning and nothing.

 

What can i do

 

I did reboot and manually start the script without problem

 

thx

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.