Guide: How To Use Rclone To Mount Cloud Drives And Play Files

[Squid]

29 minutes ago, TheFreemancer said:

I just gave your script a go but Krusader can't find any files. 

 

I'm able to read and write using a smb share but Krusader can't. 

I haven't tried other containers but I believe it would be the same result.

https://forums.unraid.net/topic/57181-real-docker-faq/#comment-564314

 

[TheFreemancer]

1 minute ago, Squid said:

https://forums.unraid.net/topic/57181-real-docker-faq/#comment-564314

 

It worked!

 

But I'm confused what should I use? 

/mnt/user/mount_rclone/google_vfs

or

/mnt/user/mount_unionfs/google_vfs

 

If I add in krusader docker settings it can see both. 

 

On SMB using unionFS I can write to the share. Using mount_rclone I can't. Is that normal or I did something wrong here?

[NewDisplayName]

51 minutes ago, TheFreemancer said:

It worked!

 

But I'm confused what should I use? 

/mnt/user/mount_rclone/google_vfs

or

/mnt/user/mount_unionfs/google_vfs

 

If I add in krusader docker settings it can see both. 

 

On SMB using unionFS I can write to the share. Using mount_rclone I can't. Is that normal or I did something wrong here?

 

 

I've tried to annotate to make editing easy.  Once the script is added you should have a new folder created at /mnt/user/mount_unionfs.  Inside this folder create your media folders i.e. /mnt/user/mount_unionfs/google_vfs/movies and /mnt/user/mount_unionfs/google_vfs/tv_shows.  These are the folders to add to plex, radarr, sonarr etc.

[TheFreemancer]

thanks! Going to use it. 

[DZMM]

3 hours ago, TheFreemancer said:

 

 

On SMB using unionFS I can write to the share. Using mount_rclone I can't. Is that normal or I did something wrong here?

You should avoid trying to write direct to the mount i.e. to /mount_rclone - always try to write to /mount_unionfs where the file(s) will get stored for a future upload in the background via /rclone_upload. 

 

You can also add files direct to /rclone_upload if you want, but it's best to use /mount_unionfs unless you know what you are doing so that you don't mess up sonarr, radarr etc

 

Now to confuse matters - for files that are on gdrive that you want to move within the mount, I find the best way to move them is to move them in /mount_rclone rather than /mount_unionfs as this won't require you to download them just to re-upload, which moving within /mount_unionfs can result in.

[tmoran000]

this is more of a vent so I dont intend anyone to listen but moving to gsuite/Rclone for hosting files and streaming for Plex has been the worst experience I have had in a very long time. You all have been great with your support and the frustration is not with yall. My frustration is the fact that I made the change and it just sucks. between it taking HOURS to scan 1 into plex with less then 20 files. From it constantly giving me the "Cant write to drive" error. To it going moving files at 100+mbps to 140kbs ... I think I am about over using Gsuite specifically with Rclone. So Thank you all I think I am throwing in the towel. 

 

 

Edit*** After talking with a member here who has helped me before... I am going to give it one more go.... 

Edited April 1, 2019 by tmoran000

[NewDisplayName]

4 hours ago, tmoran000 said:

this is more of a vent so I dont intend anyone to listen but moving to gsuite/Rclone for hosting files and streaming for Plex has been the worst experience I have had in a very long time. You all have been great with your support and the frustration is not with yall. My frustration is the fact that I made the change and it just sucks. between it taking HOURS to scan 1 into plex with less then 20 files. From it constantly giving me the "Cant write to drive" error. To it going moving files at 100+mbps to 140kbs ... I think I am about over using Gsuite specifically with Rclone. So Thank you all I think I am throwing in the towel. 

 

Ur decicion, its working flawless for me and others since months... initial setup is a bitch, but when it works, it works.

 

Getting my own client id, sped everything up, but it was okay even before.

Edited April 1, 2019 by nuhll

[tmoran000]

It was working perfect as well for about 2 months. Then out of no where. Poof it breaks and has been a terrible since, Removed , reset and garbage., I wont waste space on this thread for others needing help. I am just sad that I lost 16tb of data and the fact that it seemed like such a good idea... I just wish there was another option then Rclone. I personally thing it is the issue  .  Thanks

[TheFreemancer]

17 hours ago, DZMM said:

You should avoid trying to write direct to the mount i.e. to /mount_rclone - always try to write to /mount_unionfs where the file(s) will get stored for a future upload in the background via /rclone_upload. 

 

You can also add files direct to /rclone_upload if you want, but it's best to use /mount_unionfs unless you know what you are doing so that you don't mess up sonarr, radarr etc

 

Now to confuse matters - for files that are on gdrive that you want to move within the mount, I find the best way to move them is to move them in /mount_rclone rather than /mount_unionfs as this won't require you to download them just to re-upload, which moving within /mount_unionfs can result in.

 

Thanks for clearing it up.

I'm using only the mount and unmount script you provided.

All I wanted was to be able to access the files from outside the array and stream those.

And as a bonus be able to edit and even execute some .exe files. I'm testing streaming old games which are light from the cloud.

 

I'm going to ask the most retarded question but what do you guys use to do the rclone copy command?

 

My server is headless. If I access it with a browser I can open a terminal and output the command.

But if I close the browser on my other computer I can't check the copy process.

 

If there a way to log into the array. Do rclone copy, leave it. Get back to it the next day (maybe from another computer) and still access the same terminal session?

Edited April 1, 2019 by TheFreemancer

[Kaizac]

3 minutes ago, TheFreemancer said:

 

Thanks for clearing it up.

I'm using only the mount and unmount script you provided.

All I wanted was to be able to access the files from outside the array and stream those.

And as a bonus be able to edit and even execute some .exe files. I'm testing streaming old games which are light from the cloud.

 

I'm going to ask the most retarded question but what do you guys use to do the rclone copy command?

 

My server is headless. If I access it with a browser I can open a terminal and output the command.

But if I close the browser on my other computer I can't check the copy process.

 

If there a way to log into the array. Do rclone copy, leave it. Get back to it the next day (maybe from another computer) and still access the same terminal session?

Use User Scripts and run it in the background. Through the log you can see what's happening if you want but you don't have to keep a window open.

[DZMM]

14 minutes ago, TheFreemancer said:

 

Thanks for clearing it up.

I'm using only the mount and unmount script you provided.

All I wanted was to be able to access the files from outside the array and stream those.

And as a bonus be able to edit and even execute some .exe files. I'm testing streaming old games which are light from the cloud.

 

I'm going to ask the most retarded question but what do you guys use to do the rclone copy command?

 

My server is headless. If I access it with a browser I can open a terminal and output the command.

But if I close the browser on my other computer I can't check the copy process.

 

If there a way to log into the array. Do rclone copy, leave it. Get back to it the next day (maybe from another computer) and still access the same terminal session?

Use Screen from nerd pack

[TheFreemancer]

1 minute ago, DZMM said:

Use Screen from nerd pack

I knew it would have an elegant solution!

Thanks DZMM.

[TheFreemancer]

@DZMM. 

Your setup is working without a hitch here and I'm sorry if this has been discussed in any of the 20 pages of this thread but, would you recommend using a cache for media files with your setup? 

 

I think I can understand it more, in your setup you have gdrive, crypt, and I'm wondering about adding a disk cache. 

I don't use Plex, I actually use kodi with embycon, which uses emby library and direct stream all the files so no transcoding is necessary. 

 

Would you recommend this? I really would like to hear from someone way more experienced if that would be a good solution or not. 

I think the amount of API hits to the server would be greatly reduced but it would also complicated even further the entire setup. 

 

Maybe this would be something for setups with +1000 movies and +500 tv shows? 

[DZMM]

1 hour ago, TheFreemancer said:

@DZMM. 

Your setup is working without a hitch here and I'm sorry if this has been discussed in any of the 20 pages of this thread but, would you recommend using a cache for media files with your setup? 

 

I think I can understand it more, in your setup you have gdrive, crypt, and I'm wondering about adding a disk cache. 

I don't use Plex, I actually use kodi with embycon, which uses emby library and direct stream all the files so no transcoding is necessary. 

 

Would you recommend this? I really would like to hear from someone way more experienced if that would be a good solution or not. 

I think the amount of API hits to the server would be greatly reduced but it would also complicated even further the entire setup. 

 

Maybe this would be something for setups with +1000 movies and +500 tv shows? 

I'm 99% certain that at least for Plex no disks are used to play files - everything is streamed using RAM.  I think even the rclone dir_cache is stored in RAM.  I'm not sure how Kodi works i.e. is the file downloaded before playing - I don't think so.

 

The thread has kind of got long, which is good - at first I was worried no-one was interested but the collaboration has helped improve setups, including mine.  I've never had an API ban and my library is reasonably big - there are users in this thread with libraries x5 the size of mine and over a Petabyte.  I haven't had any problems or even stuttering for a couple of months, and all my content is in a team drive now:

 

root@Highlander:~# rclone size tdrive_vfs:
Total objects: 53947
Total size: 248.142 TBytes (272835090178653 Bytes)

 

 

 

Edited April 4, 2019 by DZMM

[jonathansayeb]

Hi all, been going through the tuto many times but couldn't get it to work after 2 nights trying to figure it out. Followed space invader tuto as well and for some reasons my Google drive content doesn't show in neither windows 10 via smb nor krusader. I try to keep things as simple as possible and just have an easy way to upload our massive library of footages and pictures to Google drive directly from our Unraid server.
And thanks for your help.

Sent from my LYA-L29 using Tapatalk

[NewDisplayName]

5 hours ago, jonathansayeb said:

Hi all, been going through the tuto many times but couldn't get it to work after 2 nights trying to figure it out. Followed space invader tuto as well and for some reasons my Google drive content doesn't show in neither windows 10 via smb nor krusader. I try to keep things as simple as possible and just have an easy way to upload our massive library of footages and pictures to Google drive directly from our Unraid server.
And thanks for your help.

Sent from my LYA-L29 using Tapatalk
 

We now magically know whats the problem?! U didnt asked ANY questions. Or told us WHERE ure stuck.

 

Maybe youre looking in the wrong directory.

 

Besides that, were supporting here the tutorial on the first page, do it step after step and if you have problems ask specific things so we can help you.

Edited April 5, 2019 by nuhll

[jonathansayeb]

Sorry, it must have been the frustration to get it to work .
I've created all my Google drive mounts and team drives.
I can mount them via script properly.
Once mounted I can see each mounts in both krusader and windows 10 but they are all empty.
I can copy files in the mount but when I check in Google drive the copied files don't appear.
On unraid terminal I can see all the content of the remotes I've setup by using
rclone ls remotename:

My question is how can I see the content of my remotes in krusader and/or smb shares?

Thank you.

Sent from my LYA-L29 using Tapatalk

[Kaizac]

2 minutes ago, jonathansayeb said:

Sorry, it must have been the frustration to get it to work .
I've created all my Google drive mounts and team drives.
I can mount them via script properly.
Once mounted I can see each mounts in both krusader and windows 10 but they are all empty.
I can copy files in the mount but when I check in Google drive the copied files don't appear.
On unraid terminal I can see all the content of the remotes I've setup by using
rclone ls remotename:

My question is how can I see the content of my remotes in krusader and/or smb shares?

Thank you.

Sent from my LYA-L29 using Tapatalk
 

Make sure you restart Krusader before you test. Krusader often only sees the old situation and not the new one, thus you won't see the mount working. 

When you go to your mount folder (mount_rclone) from the tutorial you should see 1 PiB in size to know it worked.

 

If you don't see that, your rclone config and mounting was wrong. To help you with that we need more info about your config and mounting script.

[NewDisplayName]

mount fs makes the directorys and files work just like normal local files and directories (when its setup correctly)

Edited April 5, 2019 by nuhll

[jonathansayeb]

Make sure you restart Krusader before you test. Krusader often only sees the old situation and not the new one, thus you won't see the mount working. 
When you go to your mount folder (mount_rclone) from the tutorial you should see 1 PiB in size to know it worked.
 
If you don't see that, your rclone config and mounting was wrong. To help you with that we need more info about your config and mounting script.

Restarted the server and reinstalled krusader docker and everything looks great now! 1PB ouch Thank you all.

Sent from my LYA-L29 using Tapatalk

[TheWolf]

Looking for some quick clarification before I try this, but am I essentially just supposed to download the plugins, then copy and paste the scripts from github to user scripts, and everything should work, more or less? I already have shares setup for Movies/Shows and the like, so I'm a little fuzzy on how to move everything over to where rclone can pick it up without breaking the downloads and media libraries I already have going.

[NewDisplayName]

I cant fix that undeleteable file problem.

 

18.04.2019 04:40:01 INFO: starting unionfs cleanup.
rm: cannot remove '/mnt/user/mount_rclone/google_vfs/Filme/1922 (2017)/1922 (2017).mkv': No such file or directory
rm: cannot remove '/mnt/user/rclone_upload/google_vfs/Filme/1922 (2017)/1922 (2017).mkv': No such file or directory
rm: cannot remove '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied

 

root@Unraid-Server:~# rm -r /mnt/user/mount_rclone/google_vfs/Filme/1922 (2017)/1922 (2017).mkv
-bash: syntax error near unexpected token `('
root@Unraid-Server:~# rm -r /mnt/user/mount_rclone/google_vfs/Filme/1922\ \(2017\)/
root@Unraid-Server:~# rm -r /mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922\ \(2017\)/
rm: cannot remove '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied
root@Unraid-Server:~# rm -r /mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922\ \(2017\)/1922\ \(2017\).mkv_HIDDEN~ 
rm: cannot remove '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied
root@Unraid-Server:~# chmod 777 /mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922\ \(2017\)/1922\ \(2017\).mkv_HIDDEN~ 
chmod: changing permissions of '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied

 

any idea?

[DZMM]

18 minutes ago, nuhll said:

I cant fix that undeleteable file problem.

 

18.04.2019 04:40:01 INFO: starting unionfs cleanup.
rm: cannot remove '/mnt/user/mount_rclone/google_vfs/Filme/1922 (2017)/1922 (2017).mkv': No such file or directory
rm: cannot remove '/mnt/user/rclone_upload/google_vfs/Filme/1922 (2017)/1922 (2017).mkv': No such file or directory
rm: cannot remove '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied

 

root@Unraid-Server:~# rm -r /mnt/user/mount_rclone/google_vfs/Filme/1922 (2017)/1922 (2017).mkv
-bash: syntax error near unexpected token `('
root@Unraid-Server:~# rm -r /mnt/user/mount_rclone/google_vfs/Filme/1922\ \(2017\)/
root@Unraid-Server:~# rm -r /mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922\ \(2017\)/
rm: cannot remove '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied
root@Unraid-Server:~# rm -r /mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922\ \(2017\)/1922\ \(2017\).mkv_HIDDEN~ 
rm: cannot remove '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied
root@Unraid-Server:~# chmod 777 /mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922\ \(2017\)/1922\ \(2017\).mkv_HIDDEN~ 
chmod: changing permissions of '/mnt/user/mount_unionfs/google_vfs/.unionfs/Filme/1922 (2017)/1922 (2017).mkv_HIDDEN~': Permission denied

 

any idea?

remove manually?

[NewDisplayName]

I posted you the output from me tryin to manually remov eit

 

edit:
i think you mean via smb? 
 

I cant see the directory in mount rclone, in mount unionfs i found it and deleted it, still same output from cleanscript

Edited April 19, 2019 by nuhll

[DZMM]

hmm have you tried unmounting and removing from rclone_upload or directly from the mount?