Guide: How To Use Rclone To Mount Cloud Drives And Play Files


DZMM

Recommended Posts

10 hours ago, testdasi said:

 

Something is making too many API calls to cause yours to be blocked.  You need to check what your various dockers are doing. From previous posts, it looks like Bazarr and Emby / Plex subtitle searches may be the main contributor.


I run Plex and Sonarr refreshes a few times today already + calculate how much space I'm using (a lot of API calls to count things) and I'm nowhere close to the limit. Even the per 100s limit of 1000, I only get to 20% on the worst day. So your dockers must be doing something very drastic to cause API ban. You might want to separate that docker on its own client_id.

Once banned, there's nothing you can do but to wait till your quota is reset. Usually reset time is midnight US Pacific (where Google HQ is). 

(You can see when it's reset and how many API calls you have done from your API dashboard - https://console.developers.google.com/apis/dashboard then click on quota)

 

That is assuming you have set up your own API + OAUTH client_id + share the team drive with the appropriate account.

Hi I have my own client id but i think its maybe the way i put my files.

 

For movies exemple i only got one main folder. Tv series got there own folder 

 

So maybe i need to create one folder per movies ?

 

whats your setting in plex for scan ?

 

image.png.2ef0bbbd4f79ea6bd6f2e039a3a266c8.png

Link to comment

I think it's a perfect storm kind of situation.

  • My French is terrible but I'm guessing you set it up to scan on partial change? When you are uploading files to gdrive, every little change will cause a scan of the folder.
  • You have all the movies in the same folder, that probably means Plex will rescan the entire folder, including files that were not changed (it doesn't know what has been changed, just that something was changed so it has to scan to know).

You might want to disable automatic scanning and do it manually while you reorganise your library.

 

 

Link to comment
27 minutes ago, testdasi said:

I think it's a perfect storm kind of situation.

  • My French is terrible but I'm guessing you set it up to scan on partial change? When you are uploading files to gdrive, every little change will cause a scan of the folder.
  • You have all the movies in the same folder, that probably means Plex will rescan the entire folder, including files that were not changed (it doesn't know what has been changed, just that something was changed so it has to scan to know).

You might want to disable automatic scanning and do it manually while you reorganise your library.

 

 

Yes its in french sorry.

 

So how can i rescan manually ?

 

Should i make one folder per movies inside the main one 

 

Movies

  • folder movie 1
  • folder movie 2

thx a lot

Link to comment

You untick that partial scan option.

On the main library page, hover your mouse over Libraries, there should be a button that when you click shows the various options, among which is rescan plex library (or something like that). Click on that to manually rescan.

 

It is good practice to have each movie in its own folder under a main Movies folder

e.g. 

Movies\Movie 1

Movies\Movie 2

etc.

 

I don't use Radarr but I think it does all the organisation for you so that's a faster option if you know how to use it. I organise things manually but have always done that for years.

Link to comment
You untick that partial scan option.

On the main library page, hover your mouse over Libraries, there should be a button that when you click shows the various options, among which is rescan plex library (or something like that). Click on that to manually rescan.

 

It is good practice to have each movie in its own folder under a main Movies folder

e.g. 

Movies\Movie 1

Movies\Movie 2

etc.

 

I don't use Radarr but I think it does all the organisation for you so that's a faster option if you know how to use it. I organise things manually but have always done that for years.

Great thx i will try that

 

Envoyé de mon Pixel 2 XL en utilisant Tapatalk

 

 

I found a little software that creat a folder for each files you have automatic.

 

File2folder ot seem to be working really good because i got like 7000 movies

Link to comment
1 hour ago, francrouge said:

Great thx i will try that

 

Envoyé de mon Pixel 2 XL en utilisant Tapatalk

 

 

I found a little software that creat a folder for each files you have automatic.

 

File2folder ot seem to be working really good because i got like 7000 movies

Please note that it may or may not resolve your issue until you find out exactly what caused the high API calls. You might want to follow other users on this topic who have multiple client_id for different purposes. Then if one API is banned because of accidental overload, you can switch to something else.

  • Like 1
Link to comment
Please note that it may or may not resolve your issue until you find out exactly what caused the high API calls. You might want to follow other users on this topic who have multiple client_id for different purposes. Then if one API is banned because of accidental overload, you can switch to something else.
Mm i didnt know that i will check thx again

Envoyé de mon Pixel 2 XL en utilisant Tapatalk

Link to comment
2 hours ago, francrouge said:

Should i make one folder per movies inside the main one 

 

Movies

  • folder movie 1
  • folder movie 2

thx a lot

Yes!!  If you organise your files in one big folder, when Plex is told there is a change it will scan all files in that folder.  Having a folder per movie is more efficient as Plex will only scan that folder for changes

  • Like 1
Link to comment

Has anyone seen this behaviour?

 

I know if Sonarr tries to do something to a file on a unionfs mount (and because the upload location is RW and the rclone mount is RO) it will make a copy of the file from rclone mount to upload location with whatever it tries to change (e.g. a rename or a date change).

The problem is there is one particular episode (and only that one!) for which the copy and the original are identical in every way (e.g. the filename, dates, the data itself, etc.). So I have no idea what Sonarr is trying to change.

So you say just upload it to update the file? I did! And once it's done, Sonarr would do the exact same thing again.

 

I think this behaviour has always been there, I didn't notice it previously because the write was done live immediately to storage before I moved to the gdrive model.

 

The fact that it's a Doctor Who episode makes it even spookier. I'm considering just sodding it and delete it.

Edited by testdasi
Link to comment
4 minutes ago, testdasi said:

Has anyone seen this behaviour?

 

I know if Sonarr tries to do something to a file on a unionfs mount (and because the upload location is RW and the rclone mount is RO) it will make a copy of the file from rclone mount to upload location with whatever it tries to change (e.g. a rename or a date change).

The problem is there is one particular episode (and only that one!) for which the copy and the original are identical in every way (e.g. the filename, dates, the data itself, etc.). So I have no idea what Sonarr is trying to change.

So you say just upload it to update the file? I did! And once it's done, Sonarr would do the exact same thing again.

 

I think this behaviour has always been there, I didn't notice it previously because the write was done live immediately to storage before I moved to the gdrive model.

 

The fact that it's a Doctor Who episode makes it even spookier. I'm considering just sodding it and delete it.

Hmm not sure what's going on there.  Maybe set the episode to unmonitored in sonarr?

Link to comment
On 6/12/2019 at 4:35 PM, DZMM said:

This thread has got a lot more action than I or @Kaizac or @slimshizn probably ever hoped for.  There's been a lot of action getting people up and running, so I'm wondering how some of the people in the background are getting on? 

 

How intensively are other people using rclone?  Have you moved all your media?  How are you finding the playback experience?

 

Personally, I don't have any plex content on my unRAID server anymore, except for my photos and I've now got over 300TB of Plex content stored on gdrive, as well as another big crypt with my backups (personal files, VM images etc).  I don't even notice the impact of streaming anymore and when I do have any skips, I actually think they are because of local wi-fi issues rather than rclone.

 

 

It has been quite busy here with 29 pages now this is something I'd never imagine. Currently I'm using roughly 200TB stored on gdrive, 50TB at home, which will probably be cleared once I switch to a new remote server and shut down for the summer for upgrades, more power efficient parts and such. 

Link to comment
3 hours ago, slimshizn said:

It has been quite busy here with 29 pages now this is something I'd never imagine. Currently I'm using roughly 200TB stored on gdrive, 50TB at home, which will probably be cleared once I switch to a new remote server and shut down for the summer for upgrades, more power efficient parts and such. 

Damn how can you have 200tb is it all media ? (just curious)

Link to comment

Thanks for the guide, seem to have got everything working. When I try to move a file straight to the rclone_upload folder for upload, it moves instantly, but when I move a file into the mount_unionfs, it takes a couple of minutes for a 10GB movie. Why is this and how can I speed it up? Any ideas? :)

Link to comment

 

8 hours ago, guyturner797 said:

Thanks for the guide, seem to have got everything working. When I try to move a file straight to the rclone_upload folder for upload, it moves instantly, but when I move a file into the mount_unionfs, it takes a couple of minutes for a 10GB movie. Why is this and how can I speed it up? Any ideas? :)

 

What is your unionfs line in your mount script?

That sounds like you set your mount_rclone as the RW location so when you copy file over, it tries to upload to gdrive live during the transfer.

rclone_upload is instant because it's local until you run the upload script.

Link to comment
unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

Pretty sure i've set it up correctly? I was expecting it to be instant but when using krusader it was slow to copy to the unionfs folder.

Edited by guyturner797
Link to comment
2 hours ago, guyturner797 said:

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

Pretty sure i've set it up correctly? I was expecting it to be instant but when using krusader it was slow to copy to the unionfs folder.

Oops sorry I missed the "instant" keyword. It is not instant because Unraid will try to copy the file to the unionfs location and then delete the original. Unionfs isn't smart enough to realise the RW location is on the same media as your original.

Link to comment
2 minutes ago, francrouge said:

Hi guys 

 

quick question  How do you manage your're download with sonar or rssfeed etc.

 

Do you keep a copie of the file offline and on google 

 

thx

 

Not sure if I understand you properly. You point Sonarr to your unionfs folder so it doesn't matter where you store your file.

Link to comment
8 minutes ago, Kaizac said:

Not sure if I understand you properly. You point Sonarr to your unionfs folder so it doesn't matter where you store your file.

I mean like i download a files with sonarr in unraid from there i need to upload them with the script upload ?

 

Is there another way ?

 

 

I'm trying to figure out out to download + post process + upload files automaticly via rssfeed also that why

Edited by francrouge
Link to comment
2 minutes ago, francrouge said:

I mean like i download a files with sonarr in unraid from there i need to upload them with the script upload ?

 

Is there another way ?

You could use mount_rclone as your RW folder and it will download directly to your Gdrive. However this will slowed by your upload speed. And it will probably also cause problems while direct writing to the mount. Rclone copy/move/etc. is intended to solve this issue by doing file checks.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.