Guide: How To Use Rclone To Mount Cloud Drives And Play Files

[DZMM]

yes - when the mounts are active - files should be added to /mnt/user/mount_unionfs/google_vfs

 

When the mounts aren't mounted, those paths should be empty - any files you've previously added to your upload queue will be in /mnt/user/rclone_upload

[NewDisplayName]

SO.

/mnt/user/mount_unionfs/google_vfs

 

as directory for plex and radarr and so on?

 

/mnt/user/rclone_upload

 

there new files should be added?

 

 

What you think about TS/discord/Teamviewer? Maybe we can make the tutorial easier to understand...  (or find the problem)

 

 

root@Unraid-Server:~# rm -r /mnt/user/mount_rclone/google_vfs
 

root@Unraid-Server:~# rclone mount --rc-addr=192.168.86.2:5572 --allow-other --buffer-size 64M --dir-cache-time 72h --drive-chunk-size 64M --fast-list --log-level INFO --vfs-read-chunk-size 64M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs
 

root@Unraid-Server:~# rm -r /mnt/user/mount_rclone/google_vfs

 

root@Unraid-Server:~# rclone mount --rc-addr=192.168.86.2:5572 --allow-other --buffer-size 64M --dir-cache-time 72h --drive-chunk-size 64M --fast-list --log-level INFO --vfs-read-chunk-size 64M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs
2018/11/11 02:43:01 Fatal error: Can not open: /mnt/user/mount_rclone/google_vfs: open /mnt/user/mount_rclone/google_vfs: no such file or directory

 

root@Unraid-Server:~# mkdir /mnt/user/mount_rclone/google_vfs
mkdir: cannot create directory ‘/mnt/user/mount_rclone/google_vfs’: File exists

Edited November 11, 2018 by nuhll

[DZMM]

11 hours ago, DZMM said:

if you add files to :

 

- mount_rclone: the file will get transferred immediately.  rclone isn't too smart about this, so if the transfer fails the file can get lost

- rclone_upload: the file will get moved using the rclone move script.  This is more intelligent and will retry files if there's a problem

- mount_unionfs: the file gets moved to the rclone_upload folder and gets moved as above

When you unmount the unionfs folder i.e. where it merges files from the cloud /mount_rclone and local files /rclone_upload it should be empty.  If the folder isn't empty pre-mount the mount can fail

[NewDisplayName]

11 hours ago, DZMM said:

When you unmount the unionfs folder i.e. where it merges files from the cloud /mount_rclone and local files /rclone_upload it should be empty.  If the folder isn't empty pre-mount the mount can fail

Ill try tomorrow from scratch, but the scripts should really count for it and if it needs to be empty maybe move it to tmp and then back... while mounting?!

[DZMM]

1 hour ago, nuhll said:

Ill try tomorrow from scratch, but the scripts should really count for it and if it needs to be empty maybe move it to tmp and then back... while mounting?!

The scripts do, but occasionally something odd happens - the scripts are exactly the same as what myself and a few others have been using without problems for a while so I'm trying to cover all potential problems to help you find what's going wrong

[NewDisplayName]

On 11/11/2018 at 11:12 PM, DZMM said:

The scripts do, but occasionally something odd happens - the scripts are exactly the same as what myself and a few others have been using without problems for a while so I'm trying to cover all potential problems to help you find what's going wrong

I think the problem is/was that i was stopping the upload script, BUT that could happen all time.

 

U say your script already moves the files, but if that were true, why should i check if there are files? (i guess you dont know what i mean?)

 

I mean if mounting only works if there are no files, put a mv to a temp folder before mounting and then mv it back (or mv it to upload?)... so mount always works no matter what happens.

 

Edited November 14, 2018 by nuhll

[NewDisplayName]

I never created these directories ... ?

 

root@Unraid-Server:~# vi /mnt/user/mount_unionfs/google_vfs/
movies/   tv_shows/

 

(im german, so i wont use english names for my folders)

 

Where does they come from!?!?? They always get created when running the mount script. 

 

Edit: i found out line 25 "mkdir -p /mnt/user/mount_unionfs/google_vfs/{movies,tv_shows}"

 

Wouldnt it be better to not do that? Bc not everyone uses "movies" and "tv shows"?
 

Could you please explain me which directories needs to be empty when mounting.

 

I should put new files in /mnt/user/rclone_upload (but can i also link radarr, sonarr to this directory?)

 

Even when i delete the directorys i get errors mounting. I guess its the check of mountcheck which fails. 

 

Coulnt we meet in TS or discord or something to talk (or chat) about this, maybe its a very easy fix which i dont see, i did everything from scratch and still cant mount.

 

Latest error i get is 

 

14.11.2018 18:40:09 INFO: mounting rclone vfs.
14.11.2018 18:40:19 INFO: Check rclone vfs mount success.
fusermount: failed to unmount /mnt/user/mount_unionfs/google_vfs: Invalid argument
fuse: mountpoint is not empty
fuse: if you are sure this is safe, use the 'nonempty' mount option
14.11.2018 18:40:19 CRITICAL: unionfs Remount failed.

 

That mount failed before is probably due bc of to low sizes?! (was using 64mb for all values) Thats the error i get when i use your values.

 

I already ordered new RAM so this will be np in future i hope.

Edited November 14, 2018 by nuhll

[DZMM]

Sorry for slow reply, I've been rebuilding my PC so no internet etc

21 hours ago, nuhll said:

Edit: i found out line 25 "mkdir -p /mnt/user/mount_unionfs/google_vfs/{movies,tv_shows}"

 

Wouldnt it be better to not do that? Bc not everyone uses "movies" and "tv shows"?

 

I was trying to make the script as 'off the shelf' as possible and after seeing some of the confusion you've had with which folders to use, I added it in.  Stupid idea as this is the cause of your latest error i.e. the unionfs folder isn't empty - fixed on github - just change to: 

mkdir -p /mnt/user/mount_unionfs/google_vfs

 

21 hours ago, nuhll said:

Could you please explain me which directories needs to be empty when mounting.

See above - After you've unmounted, on rare I've seen there are residual files in the /mount_unionfs/ folder.  This hasn't happened for a long, long time so whatever was causing it seems to have gone away - I again mentioned this because you were having strange problems, so I thought this was worth checking.

 

21 hours ago, nuhll said:

I should put new files in /mnt/user/rclone_upload (but can i also link radarr, sonarr to this directory?)

Your dockers should be pointed at the sub-folders within /mount_unionfs/ - that's the whole point of the setup, that dockers can't distinguish between files that are in the cloud (mount_rclone) or files that are local and will be moved to the cloud /rclone_upload

 

 

[Kaizac]

On 11/14/2018 at 4:22 PM, nuhll said:

Text

 

 

Silly question, did you reboot your Unraid box after every change to the scripts? And are you running the scripts "In the background"? I've noticed myself that just unmounting and remounting the scripts doesn't work without rebooting it. After the reboot download the logs and see what the mounting scripts give for error.

 

When you get the error your fuse folders are not empty you should start up Krusader. Go to your mount_unionfs folder and clear it all. You can just move the files with Krusader to your rclone_upload folders. Only folders which can be populated (thus not empty) are your rclone_upload folders (your folder where your local folders are stored waiting to be uploaded) and your mount itself (thus your cloud storage). 

Your unionfs folder is not a real directory and just a merge of the local and cloud folders and thus should be empty.

Edited November 22, 2018 by Kaizac

[NewDisplayName]

There are no silly questions. 

 

Also, it was already found out why i got the directory not empty error, it was the change he did to the script to help ppl understand what directorys are for what.

 

 

I found a workaround for my "directory problem". Ill just add another directory (the virtual) to the same plex library. Question is only if i add a movie to the library and it get later moved to the new folder, will plex understand this, or is this then a new movie for it? Im asking bc i dont want to show wrong movies as new and also i dont want to reassign the movies when they get moved after 2 years.

 

Couldnt test anymore, ill try again when im on vacation, also ive upgraded my mainboard to double my ram from 16 to 32 gb, but i cant get it working, so that takes my free time up atm.

 

Anyway, i will report!

Quote

 

I was trying to make the script as 'off the shelf' as possible and after seeing some of the confusion you've had with which folders to use, I added it in.  Stupid idea as this is the cause of your latest error i.e. the unionfs folder isn't empty - fixed on github - just change to: 

mkdir -p /mnt/user/mount_unionfs/google_vfs
 

 

Edited November 22, 2018 by nuhll

[DZMM]

14 minutes ago, nuhll said:

 

I found a workaround for my "directory problem". Ill just add another directory (the virtual) to the same plex library. Question is only if i add a movie to the library and it get later moved to the new folder, will plex understand this, or is this then a new movie for it? Im asking bc i dont want to show wrong movies as new and also i dont want to reassign the movies when they get moved after 2 years.

 

the unionfs folder once setup acts like a normal folder where you can move, delete, add files etc.  Plex sees this folder just like any folder, so if you move a movies in or out of the unionfs folder, whether or not Plex says it is missing, or automatically removes it from the collection will be determined by your plex settings not rclone or unionfs.

[NewDisplayName]

I mean it will be 2 years in

 

/normal/folder/movie

 

then moved to unionfs folder.. 

 

/unionfs/movie

Edited November 22, 2018 by nuhll

[DZMM]

23 minutes ago, nuhll said:

I mean it will be 2 years in

 

/normal/folder/movie

 

then moved to unionfs folder.. 

 

/unionfs/movie

and depending on how you've got plex setup, it will either (i) delete the reference to the /normal/folder/movie or (ii) say /normal/folder/movie is missing and delete it when you empty trash.

 

I prefer the 2nd way, because if I've made any changes to the metadata (labels, posters etc) they are preserved whereas the first way ditches all custom edits and redownloads Plex's metadata.  

 

Edit:  the 2nd way is also advisable because if the mount ever fails, plex won't delete the items from its libraries, and then re-add them

Edited November 22, 2018 by DZMM

[NewDisplayName]

Where exactly i can change these options?

 

[DZMM]

"empty trash automatically after every scan" in Plex library settings

[NewDisplayName]

I cant find that exact option, the best i found i would translate into remove old packets weekly.

 

Its the second on the "sheduled things" tab.

[DZMM]

7 minutes ago, nuhll said:

I cant find that exact option, the best i found i would translate into remove old packets weekly.

 

Its the second on the "sheduled things" tab.

[NewDisplayName]

I found some time to test again.

 

After some unmounts and deleting files mounting worked, expect the RC part.

 

It said it cant connect to localhost. even i have the following in the config:

rclone mount --rc-addr=192.168.86.2:5572 --allow-other --buffer-size 1G --dir-cache-time 72h --drive-chunk-size 256M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs &
 

 

Is this wrong?

 

Whats now the correct way, unmounting? Then remount should work?

 

 

And thanks for the info about plex, i found the option. If it works i have to test.

Edited November 29, 2018 by nuhll

[DZMM]

5 minutes ago, nuhll said:

After some unmounts and deleting files mounting worked, expect the RC part.

 

rc doesn't like it if you keep unmounting and mounting - it tends to only work the first mount.  Don't worry about the errors that's fine. 

 

The purpose of the rc command is to pre-populate the directory cache at mount to ensure navigating the mount is as fast as possible from the outset.  The dir cache stills gets populated without it, just not as fast.

[NewDisplayName]

Can i use "cache: Yes" on "mount_rclone" and "mount_unionfs"?

 

Im still unsure how to transfer my library. If i just move the files, sonarr and radarr will think the files are missing and would reload them. So only chance is to merge the whole diretory, but uploading 10tb would take a long time... 

 

How you did the process?

Edited November 29, 2018 by nuhll

[DZMM]

10 minutes ago, nuhll said:

Can i use "cache: Yes" on "mount_rclone" and "mount_unionfs"?

 

it doesn't matter as no real files exist on your array for those folders - just virtual files.

 

11 minutes ago, nuhll said:

Im still unsure how to transfer my library. If i just move the files, sonarr and radarr will think the files are missing and would reload them. So only chance is to merge the whole diretory, but uploading 10tb would take a long time... 

 

How you did the process?

Just add the current folders to your unionfs mount and then move them manually behind the scenes to your rclone_upload folder where they'll get uploaded over time e.g.

 

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO:/mnt/user/OLD_MEDIA_LOCATION=RO /mnt/user/mount_unionfs/google_vfs

Then add the unionfs folders to sonarr etc.  You just have to make sure that the filepaths matchup e.g.

 

- /mnt/user/mount_rclone/google_vfs/TV Show 1

- /mnt/user/Old/Media_location/TV Show 1

[NewDisplayName]

Its not working. I dont get it.

 

Mount worked, besides the RC part.

 

I put a file inside rclone_upload/google_vfs/Filme/film.mkv

 

I press upload script:
Script location: /tmp/user.scripts/tmpScripts/rclone upload/script
Note that closing this window will abort the execution of this script
/tmp/user.scripts/tmpScripts/rclone upload/script: line 4: $'\357\273\277': command not found
/tmp/user.scripts/tmpScripts/rclone upload/script: line 26: $'\357\273\277': command not found
29.11.2018 18:32:19 INFO: rclone not installed - will try again later.

 

 

(After i saw the erorr about command not found i copied a fresh copy from your github, still the same..)

Just to be sure, how long i need to wait when i press the mount script? Is there anythign what needs more time?

Edited November 29, 2018 by nuhll

[DZMM]

25 minutes ago, nuhll said:

29.11.2018 18:32:19 INFO: rclone not installed - will try again later.

It's saying rclone isn't installed - check plugins

[Kaizac]

@DZMM I've changed directories for some series in Sonarr. So these should move from one folder on the Gdrive to another. It seems so far that I can't just move them, so I would essentially be a download and re-upload. Is that correct or is there a way to transfer files easily?

[DZMM]

4 minutes ago, Kaizac said:

@DZMM I've changed directories for some series in Sonarr. So these should move from one folder on the Gdrive to another. It seems so far that I can't just move them, so I would essentially be a download and re-upload. Is that correct or is there a way to transfer files easily?

What I find is that if you move files in the mount by moving them in the unionfs mount, unionfs downloads the file and reuploads it. 

 

What I do if I'm doing a big move like that, is I move the files manually within the rclone mount as they don't get downloaded then