Guide: How To Use Rclone To Mount Cloud Drives And Play Files


DZMM

2468 posts in this topic Last Reply

Recommended Posts

  • Replies 2.5k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

3/1/20 UPDATE: TO MIGRATE FROM UNIONFS TO MERGERFS READ THIS POST.  New users continue reading 13/3/20 Update: For a clean version of the 'How To' please use the github site https://github.com/B

Multiple mounts, one upload and one tidy-up script.   @watchmeexplode5 did some testing and performance gets worse as you get closer to the 400k mark, so you'll need to do something like bel

so ive been reading for ages but theres so many pages here, my issue is that when the array is stopped the mergerfs mount is not unmounted casuing unraid to keep retrying indefinately to unmount share

Posted Images

2 hours ago, brasi said:

Anyone experiencing play back issues with nvidia shield plex client with this setup?  I keep getting "Your connection to the server is not fast enough to stream this video”

 

 

No, did you put the shield on wifi? If so that's probably the issue. Or are you playing 4k movies? The shield doesn't need transcode so it will play on full quality which can be taxing on your bandwidth depending on your file size.

Link to post
3 hours ago, brasi said:

Anyone experiencing play back issues with nvidia shield plex client with this setup?  I keep getting "Your connection to the server is not fast enough to stream this video”

 

 

I get this too. Its nothing to do with rclone and the setup, as local files do the same. Im wired with gig connection. There seems to be quite a bit on the forums about shield and plex and this issue. If i watch my network usage, i see good spike while it buffers to the shield, with bursts every few seconds then all of a sudden the bursts fall away and you get a low throughput rate until you get the message and playback stops. The same video will work some times, other times not. I can go weeks with it being ok, then i get it a few times a day. Search the nvidia and plex forums, its well documented with no consistent fix.

Link to post
8 hours ago, Kaizac said:

No, did you put the shield on wifi? If so that's probably the issue. Or are you playing 4k movies? The shield doesn't need transcode so it will play on full quality which can be taxing on your bandwidth depending on your file size.

This is hard wired 1Gbps link.  Am I right to assume with the gdrive mount the files are downloaded and then served to my nvidia shield locally from my plex server?  My WAN uplink should have no impact here correct?

Link to post
1 hour ago, brasi said:

This is hard wired 1Gbps link.  Am I right to assume with the gdrive mount the files are downloaded and then served to my nvidia shield locally from my plex server?  My WAN uplink should have no impact here correct?

Normally plex shoudl check its LAn and sent via LAN but it COULD go over WAN if it doesnt work (i dont know if that works with shield anyway)

 

Just check if you have upload on yoru router while you stream to shield.

Link to post

New problem popped up last night. Updated to 6.7, rebooted a few times, and ran into a new problem

 

/mount_rclone/google_vfs/[all shares] loads the few files I've put there, but /mount_unionfs/google_vfs/ loads the share folders (movies, tv shows) but doesn't load the contents. I tried deleting the folders in google_vfs but that didn't help.

 

Here is the new error (from the mount script):

 

INFO: mounting rclone vfs.
Fatal error: Can not open: /mnt/user/mount_rclone/google_vfs: open /mnt/user/mount_rclone/google_vfs: transport endpoint is not connected
CRITICAL: rclone gdrive vfs mount failed - please check for problems.
rm: cannot remove '/mnt/user/appdata/other/rclone/rclone_mount_running': No such file or directory
Script Finished Mon, 03 Jun 2019

 

after a reboot I get this error:

INFO: mounting rclone vfs.
INFO: Check rclone gdrive vfs mount success.
fuse: mountpoint is not empty
fuse: if you are sure this is safe, use the 'nonempty' mount option
CRITICAL: unionfs Remount failed.
rm: cannot remove '/mnt/user/appdata/other/rclone/rclone_mount_running': No such file or directory
Script Finished Mon, 03 Jun 2019 

 

It looks like I can see the mounted files inside /mount_clone but they aren't appearing in /mount_unionfs.

Edited by privateer
Link to post
17 hours ago, privateer said:

after a reboot I get this error:

INFO: mounting rclone vfs.
INFO: Check rclone gdrive vfs mount success.
fuse: mountpoint is not empty
fuse: if you are sure this is safe, use the 'nonempty' mount option

you've got something in /mount_unionfs/google_vfs/ prior to the unionfs mount being created so it fails.  Manually delete whatever's in there (be sure you don't want first!)

Link to post

hello, i just suscribed g suite for upload my plex file to google. I try your logs. But i don't understand everything

 

here is the log errors for mount : https://d.pr/i/TOnFob

 

the config file : https://d.pr/i/e1UKZ7

 

the mount script i copy cut from github

same for unmount script

 

the user scripts from your github : https://d.pr/i/zXBbPD

 

the mountcheck : https://d.pr/i/mPJajO

the # rclone config : https://d.pr/i/G4Nkjo https://d.pr/i/2zFfSt  https://d.pr/i/5wZd4M https://d.pr/i/xEM4h6  

  for encrypted : https://d.pr/i/3AeNBo

 

I am a noob with coding and totally lost :)

 

thanking you in advance,

 

regards

 

 

edit : i have an upload correct now  to gdrive but still have some issues

 

 

1- an error with gdrive vfs https://d.pr/i/giI75Z i thin i made something wrong with configuration https://d.pr/i/SIneHX or here https://d.pr/i/BRYENB need help on this point

2- in krusader i have this : https://d.pr/i/wPGRY6  and this in the crypt file https://d.pr/i/KTUGSU

 

first question what is the first strange file?

second question the file crypt appears in gdrive https://d.pr/i/yZjjd0 but i can see the file, is it normal or should be crypt?

3- my main objective is to upload all my plex files to gdrive with crypt for being invisible to google but accessible to me everywhere, is it ok with my configuration rclone?

4- how to upload everything from my unraid library to my gdrive crypt folder without copy from krusader to local? i want to save space in my server. As i understand, if i copy to disks->gdrive->crypt, it take space on my server, am i right?

5- last question, how i acces to this file from plex server to gdrive?

 

it took me 9 hours to have a connection so please be kind with me

 

thanks for your work and help

 

edit 2: i redo every thing and reboot, i have this : https://d.pr/i/QRZ2Gd

 

Edited by neow
Link to post
16 hours ago, neow said:

 

 

it took me 9 hours to have a connection so please be kind with me

 

thanks for your work and help

 

 

Everybody's nice on this forum!

 

Can you post your rclone config file (remember to remove any passwords), mount script and the log with any errors in the thread please.  It's easier to follow and I couldn't open your links anyway, although it could be because I'm on my work laptop which is seriously locked down.

Link to post
6 minutes ago, DZMM said:

Everybody's nice on this forum!

 

Can you post your rclone config file (remember to remove any passwords), mount script and the log with any errors in the thread please.  It's easier to follow and I couldn't open your links anyway, although it could be because I'm on my work laptop which is seriously locked down.

hello,

 

thanks for your reply i appreciate a lot

 

[gdrive]
type = drive
client_id = 
client_secret = 
scope = drive
token = {"access_token":""}

[gdrive_media_vfs]
type = crypt
remote = gdrive:crypt
filename_encryption = standard
directory_name_encryption = true
password = 
password2 = 

 

 

the erros in rclone mount plugin ;

 

Script location: /tmp/user.scripts/tmpScripts/rclone_mount_plugin/script
Note that closing this window will abort the execution of this script
12.06.2019 21:47:13 INFO: mounting rclone vfs.
12.06.2019 21:47:18 INFO: Check rclone gdrive vfs mount success.
/tmp/user.scripts/tmpScripts/rclone_mount_plugin/script: line 56: unionfs: command not found
12.06.2019 21:47:18 CRITICAL: unionfs Remount failed.
rm: cannot remove '/mnt/user/appdata/other/rclone/rclone_mount_running': No such file or directory

Edited by neow
Link to post
9 minutes ago, neow said:


/tmp/user.scripts/tmpScripts/rclone_mount_plugin/script: line 56: unionfs: command not found
12.06.2019 21:47:18 CRITICAL: unionfs Remount failed.
 

you need to install nerdtools to then install unionfs - see the first post for the 'optional plugins' section

 

10 minutes ago, neow said:

rm: cannot remove '/mnt/user/appdata/other/rclone/rclone_mount_running': No such file or directory

post your full mount script as this is easily fixed

Link to post
1 minute ago, DZMM said:

you need to install nerdtools to then install unionfs - see the first post for the 'optional plugins' section

 

post your full mount script as this is easily fixed

#!/bin/bash

#######  Start rclone gdrive mount  ##########

# check if gdrive mount already created

if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check rclone vfs already mounted."

else

echo "$(date "+%d.%m.%Y %T") INFO: mounting rclone vfs."

# create directories for rclone mount and unionfs mount

mkdir -p /mnt/user/appdata/other/rclone
mkdir -p /mnt/user/mount_unionfs/google_vfs
mkdir -p /mnt/user/rclone_upload/google_vfs

rclone mount --allow-other --buffer-size 256M --dir-cache-time 72h --drive-chunk-size 512M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs &

# check if mount successful

# slight pause to give mount time to finalise

sleep 5

if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check rclone gdrive vfs mount success."

else

echo "$(date "+%d.%m.%Y %T") CRITICAL: rclone gdrive vfs mount failed - please check for problems."

rm /mnt/user/appdata/other/rclone/rclone_mount_running

exit

fi

fi

#######  End rclone gdrive mount  ##########

#######  Start unionfs mount   ##########

if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs already mounted."

else

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs mounted."

else

echo "$(date "+%d.%m.%Y %T") CRITICAL: unionfs Remount failed."

rm /mnt/user/appdata/other/rclone/rclone_mount_running

exit

fi

fi

#######  End Mount unionfs   ##########

############### starting dockers that need unionfs mount ######################

# only start dockers once

if [[ -f "/mnt/user/appdata/other/rclone/dockers_started" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: dockers already started"

else

touch /mnt/user/appdata/other/rclone/dockers_started

echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers."

docker start plex
docker start ombi
docker start tautulli
docker start radarr
docker start sonarr

fi

############### end dockers that need unionfs mount ######################

exit

Link to post

ok i just installe nerd pack and the unionfs

 

now i have this in mount plugin

 

Script location: /tmp/user.scripts/tmpScripts/rclone_mount_plugin/script
Note that closing this window will abort the execution of this script
12.06.2019 22:02:44 INFO: mounting rclone vfs.
12.06.2019 22:02:50 INFO: Check rclone gdrive vfs mount success.
12.06.2019 22:02:50 INFO: Check successful, unionfs mounted.
12.06.2019 22:02:50 INFO: Starting dockers.
Error response from daemon: No such container: plex
Error: failed to start containers: plex
Error response from daemon: No such container: ombi
Error: failed to start containers: ombi
Error response from daemon: No such container: tautulli
Error: failed to start containers: tautulli
Error response from daemon: No such container: radarr
Error: failed to start containers: radarr
Error response from daemon: No such container: sonarr
Error: failed to start containers: sonarr

 

edit : i restart after this plex server package and krusader and now i have this :

 

Script location: /tmp/user.scripts/tmpScripts/rclone_mount_plugin/script
Note that closing this window will abort the execution of this script
12.06.2019 22:05:43 INFO: mounting rclone vfs.
2019/06/12 22:05:43 Fatal error: Can not open: /mnt/user/mount_rclone/google_vfs: open /mnt/user/mount_rclone/google_vfs: transport endpoint is not connected
12.06.2019 22:05:48 CRITICAL: rclone gdrive vfs mount failed - please check for problems.
rm: cannot remove '/mnt/user/appdata/other/rclone/rclone_mount_running': No such file or directory 

Edited by neow
Link to post
11 minutes ago, neow said:


rm: cannot remove '/mnt/user/appdata/other/rclone/rclone_mount_running': No such file or directory 

fixed - my mistake - an error sneaked in on github https://github.com/BinsonBuzz/unraid_rclone_mount/blob/master/rclone_mount.  you need to add a new section at the start of your mount script -

#######  Check if script is already running  ##########

if [[ -f "/mnt/user/appdata/other/rclone/rclone_mount_running" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Exiting script already running."

exit

else

touch /mnt/user/appdata/other/rclone/rclone_mount_running

fi

#######  End Check if script already running  ##########

 

13 minutes ago, neow said:

12.06.2019 22:02:44 INFO: mounting rclone vfs.
12.06.2019 22:02:50 INFO: Check rclone gdrive vfs mount success.
12.06.2019 22:02:50 INFO: Check successful, unionfs mounted.
12.06.2019 22:02:50 INFO: Starting dockers.
Error response from daemon: No such container: plex
Error: failed to start containers: plex
Error response from daemon: No such container: ombi
Error: failed to start containers: ombi
Error response from daemon: No such container: tautulli
Error: failed to start containers: tautulli
Error response from daemon: No such container: radarr
Error: failed to start containers: radarr
Error response from daemon: No such container: sonarr
Error: failed to start containers: sonarr

your first run was actually successful.  What failed was the docker start command as you don't have those dockers or they have slightly different names on your server.  Remove or edit

 

15 minutes ago, neow said:

12.06.2019 22:05:43 INFO: mounting rclone vfs.
2019/06/12 22:05:43 Fatal error: Can not open: /mnt/user/mount_rclone/google_vfs: open /mnt/user/mount_rclone/google_vfs: transport endpoint is not connected

I'm not really sure what happened on the 2nd run...I think your first mount is lingering somewhere.  Manually run:

 

fusermount -uz /mnt/user/mount_unionfs/google_vfs
fusermount -uz /mnt/user/mount_rclone/google_vfs

before running the mount script again.  Once you've edited the docker lines and added the missing section to the mount script you should be good to go as long as you've used the latest upload script from github 

Link to post
7 minutes ago, DZMM said:

fixed - my mistake - an error sneaked in on github https://github.com/BinsonBuzz/unraid_rclone_mount/blob/master/rclone_mount.  you need to add a new section at the start of your mount script -


#######  Check if script is already running  ##########

if [[ -f "/mnt/user/appdata/other/rclone/rclone_mount_running" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Exiting script already running."

exit

else

touch /mnt/user/appdata/other/rclone/rclone_mount_running

fi

#######  End Check if script already running  ##########

 

your first run was actually successful.  What failed was the docker start command as you don't have those dockers or they have slightly different names on your server.  Remove or edit

 

I'm not really sure what happened on the 2nd run...I think your first mount is lingering somewhere.  Manually run:

 


fusermount -uz /mnt/user/mount_unionfs/google_vfs
fusermount -uz /mnt/user/mount_rclone/google_vfs

before running the mount script again.  Once you've edited the docker lines and added the missing section to the mount script you should be good to go as long as you've used the latest upload script from github 

ok you update github :)

 

now i have this

Script location: /tmp/user.scripts/tmpScripts/rclone_mount_plugin/script
Note that closing this window will abort the execution of this script
12.06.2019 22:34:25 INFO: Exiting script already running.

Edited by neow
Link to post

This thread has got a lot more action than I or @Kaizac or @slimshizn probably ever hoped for.  There's been a lot of action getting people up and running, so I'm wondering how some of the people in the background are getting on? 

 

How intensively are other people using rclone?  Have you moved all your media?  How are you finding the playback experience?

 

Personally, I don't have any plex content on my unRAID server anymore, except for my photos and I've now got over 300TB of Plex content stored on gdrive, as well as another big crypt with my backups (personal files, VM images etc).  I don't even notice the impact of streaming anymore and when I do have any skips, I actually think they are because of local wi-fi issues rather than rclone.

 

 

Edited by DZMM
Link to post
1 minute ago, DZMM said:

This thread has got a lot more action than I or @Kaizac or @slimshizn probably ever hoped for.  There's been a lot of action getting people up and running, so I'm wondering how some of the people in the background are getting on? 

 

How intensively are other people using rclone?  Have you also moved all your media?  How are you finding the playback experience?

 

Personally, I don't have any plex content on my unRAID server anymore, except for my photos and I've now got over 300TB of Plex content stored on gdrive, as well as another big crypt with my backups (personal files, VM images etc).  I don't even notice the impact of streaming anymore and when I do have any skips, I actually think they are because of local wi-fi issues rather than rclone.

 

 

do you talk to me? i think not :)

 

Link to post
6 minutes ago, neow said:

ok you update github :)

 

now i have this

Script location: /tmp/user.scripts/tmpScripts/rclone_mount_plugin/script
Note that closing this window will abort the execution of this script
12.06.2019 22:34:25 INFO: Exiting script already running.

That's fine.  What's supposed to happen is the /mnt/user/appdata/other/rclone/rclone_mount_running file gets removed when you run the unmount script either at shutdown or array start.  Just manually delete it for now: /mnt/user/appdata/other/rclone/rclone_mount_running

 

I designed the mount script this way so you can run it on a cron job to auto remount if the mount ever fails.  This check makes sure that 2 instances of the script don't run at the same time

 

 

Link to post
6 minutes ago, DZMM said:

That's fine.  What's supposed to happen is the /mnt/user/appdata/other/rclone/rclone_mount_running file gets removed when you run the unmount script either at shutdown or array start.  Just manually delete it for now: /mnt/user/appdata/other/rclone/rclone_mount_running

 

I designed the mount script this way so you can run it on a cron job to auto remount if the mount ever fails.  This check makes sure that 2 instances of the script don't run at the same time

 

 

i don't understand very well what you mean

 

in root mnt user appdata i just have a kde folder with krusader

 

you know i navigate only with krusader i am a big noob 

 

Screen Shot on 2019-06-12 at 22-46-32.png

Edited by neow
Link to post

with the rclone upload plugin i have this

 

 

Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script
Note that closing this window will abort the execution of this script
/tmp/user.scripts/tmpScripts/rclone_upload_plugin/script: line 4: $'\357\273\277': command not found
/tmp/user.scripts/tmpScripts/rclone_upload_plugin/script: line 26: $'\357\273\277': command not found
12.06.2019 22:42:40 INFO: rclone not installed - will try again later.
 

Screen Shot on 2019-06-12 at 22-43-50.png

log 2.txt

Edited by neow
Link to post
2 minutes ago, neow said:

i don't understand very well what you mean

 

in root mnt user appdata i just have a kde folder with krusader

 

restart krusader (this issue keeps coming up) as the file rclone_mount_running is in /mnt/user/appdata/other/rclone 

Link to post
4 minutes ago, DZMM said:

restart krusader (this issue keeps coming up) as the file rclone_mount_running is in /mnt/user/appdata/other/rclone 

i just restarted krusader this change nothing unfortunatly

Screen Shot on 2019-06-12 at 22-51-27.png

Edited by neow
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.