Jump to content
DZMM

Guide: How To Use Rclone To Mount Cloud Drives And Play Files

1060 posts in this topic Last Reply

Recommended Posts

What are the setting I should deactive to limit API call ? On Sonarr and Radarr ? Thanks

 

(On plex I removed the setting to scan library on change, or partial scans, and put a scan every 24h)

Edited by yendi

Share this post


Link to post
3 minutes ago, yendi said:

What are the setting I should deactive to limit API call ? On Sonarr and Radarr ? Thanks

I don't think you can. Maybe @DZMM

knows an rclone setting to limit calls. He is better with the commands.

Share this post


Link to post
58 minutes ago, yendi said:

What are the setting I should deactive to limit API call ? On Sonarr and Radarr ? Thanks

 

(On plex I removed the setting to scan library on change, or partial scans, and put a scan every 24h)

It sounds like you are using the rclone client_ID - it's strongly advisable to create your own:

 

https://rclone.org/drive/#making-your-own-client-id

Share this post


Link to post

[gdrive]
type = drive
clientXXXXXXXXXXXXhr.apps.googleusercontent.com
client_secret = ycDVPT_WXXXXXXXXXX
scope = drive
token = {"access_token":"ya29.XXXXXX_6InM3eF6zcbSs39LJ4qYRbYgBU-XXXXXXXXXXX_3u5fRjPNAyu9FPoh6BHVsIBc16OXdLIVBjUfHSuoEZ6D0NIcY","token_type":"Bearer","refresh_token":"1/XXXXXXXXXXXXXXXX","expiry":"2019-09-15T11:04:49.004526138+02:00"}

[gdrive_media_vfs]
type = crypt
remote = gdrive:crypt
filename_encryption = standard
directory_name_encryption = true
password = XXXXXXXXX-a68Inf_HY_qUAy_
password2 = CXXXXXXXX

 

 

Im on mobile so difficult to clean the passWord and API but I added a bunch of XXX 
 

there is no way to add a verbose mode to the RClone mount? To be able to identify if it’s api calls?

Edited by yendi

Share this post


Link to post

So it's not a API call issue as I have no problem uploading without any error. When I reach API limit I had a message in the upload saying API Rate limit exceed or something similar.

2019/09/15 19:34:29 DEBUG : gumkl37i63l70uc0ufklrjq3ic/jqdod4b7jtrggrjc7perjig6venga6g4jj8tl6dsvgr22e8r45qg/uf7cq92dioivmjp6m1lmvdnui69pattb328dev1376fp5s6fisrlckb4v8up65e6p696qm4hscs0pb0tgmp1vh88jo7fl4k84u5gc0ht7clpnc4gmru2kfuq0bnrhl4u: Sending chunk 15569256448 length 536870912
2019/09/15 19:34:39 INFO :
Transferred: 29.516G / 5.521 TBytes, 1%, 8.177 MBytes/s, ETA 1w1d3h38m21s
Errors: 0
Checks: 1446 / 1448, 100%
Transferred: 48 / 2562, 2%
Elapsed time: 1h1m36.4s
Checking:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s
Transferring:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s

2019/09/15 19:34:29 DEBUG : gumkl37i63l70uc0ufklrjq3ic/jqdod4b7jtrggrjc7perjig6venga6g4jj8tl6dsvgr22e8r45qg/uf7cq92dioivmjp6m1lmvdnui69pattb328dev1376fp5s6fisrlckb4v8up65e6p696qm4hscs0pb0tgmp1vh88jo7fl4k84u5gc0ht7clpnc4gmru2kfuq0bnrhl4u: Sending chunk 15569256448 length 536870912
2019/09/15 19:34:39 INFO :
Transferred: 29.516G / 5.521 TBytes, 1%, 8.177 MBytes/s, ETA 1w1d3h38m21s
Errors: 0
Checks: 1446 / 1448, 100%
Transferred: 48 / 2562, 2%
Elapsed time: 1h1m36.4s
Checking:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s
Transferring:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s

2019/09/15 19:34:46 DEBUG : gumkl37i63l70uc0ufklrjq3ic/ldkif04thouaso6v4mutn223f1t3p6sf6bt9ug9s8r9ivf36v6l0/f5bc12htqdevl3o3osg28mpp6ktobisoetrpcjvpmk2t5egv3ooj2ncmna68p1mm4mfl7aq3d6jrcuheui5m6qi20mju5o9ulqs6t2u1jsqv6g83dubm11vgil4als6g: Sending chunk 16106127360 length 536870912

 

Here is what I have on plex when I try to start a remote movie: image.thumb.png.a026e907e97e42cfccddd0798fca5b2c.png

 

I tried to create a new plex docker to do some tests (plex inc provider this time instead of linuxserver), here is my findings:

 

I created 2 libraries: One pointing on rclone_upload (so all the pending uploads) and an other on the mount_rclone (so all the already uploaded files).

Plex can parse both folders, local and remote and find the correct info from the media (Resolution, subtitles available etc) but is still unable to play remote content... 

 

I dont understand how it can get metadata and file info and not be able to open the media...

 

Crazy thing is that I am able _sometimes_ to open a remote file, I would say 10% of the time.

Edited by yendi

Share this post


Link to post
1 hour ago, yendi said:

So it's not a API call issue as I have no problem uploading without any error. When I reach API limit I had a message in the upload saying API Rate limit exceed or something similar.


2019/09/15 19:34:29 DEBUG : gumkl37i63l70uc0ufklrjq3ic/jqdod4b7jtrggrjc7perjig6venga6g4jj8tl6dsvgr22e8r45qg/uf7cq92dioivmjp6m1lmvdnui69pattb328dev1376fp5s6fisrlckb4v8up65e6p696qm4hscs0pb0tgmp1vh88jo7fl4k84u5gc0ht7clpnc4gmru2kfuq0bnrhl4u: Sending chunk 15569256448 length 536870912
2019/09/15 19:34:39 INFO :
Transferred: 29.516G / 5.521 TBytes, 1%, 8.177 MBytes/s, ETA 1w1d3h38m21s
Errors: 0
Checks: 1446 / 1448, 100%
Transferred: 48 / 2562, 2%
Elapsed time: 1h1m36.4s
Checking:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s
Transferring:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s

2019/09/15 19:34:29 DEBUG : gumkl37i63l70uc0ufklrjq3ic/jqdod4b7jtrggrjc7perjig6venga6g4jj8tl6dsvgr22e8r45qg/uf7cq92dioivmjp6m1lmvdnui69pattb328dev1376fp5s6fisrlckb4v8up65e6p696qm4hscs0pb0tgmp1vh88jo7fl4k84u5gc0ht7clpnc4gmru2kfuq0bnrhl4u: Sending chunk 15569256448 length 536870912
2019/09/15 19:34:39 INFO :
Transferred: 29.516G / 5.521 TBytes, 1%, 8.177 MBytes/s, ETA 1w1d3h38m21s
Errors: 0
Checks: 1446 / 1448, 100%
Transferred: 48 / 2562, 2%
Elapsed time: 1h1m36.4s
Checking:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s
Transferring:
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 18% /80.073G, 4.128M/s, 4h29m9s
* 4k/xxxxxxxxx…ueHD.7.1.Atmos-FGT.mkv: 30% /47.306G, 4.076M/s, 2h17m11s

2019/09/15 19:34:46 DEBUG : gumkl37i63l70uc0ufklrjq3ic/ldkif04thouaso6v4mutn223f1t3p6sf6bt9ug9s8r9ivf36v6l0/f5bc12htqdevl3o3osg28mpp6ktobisoetrpcjvpmk2t5egv3ooj2ncmna68p1mm4mfl7aq3d6jrcuheui5m6qi20mju5o9ulqs6t2u1jsqv6g83dubm11vgil4als6g: Sending chunk 16106127360 length 536870912

 

Here is what I have on plex when I try to start a remote movie: image.thumb.png.a026e907e97e42cfccddd0798fca5b2c.png

 

I tried to create a new plex docker to do some tests (plex inc provider this time instead of linuxserver), here is my findings:

 

I created 2 libraries: One pointing on rclone_upload (so all the pending uploads) and an other on the mount_rclone (so all the already uploaded files).

Plex can parse both folders, local and remote and find the correct info from the media (Resolution, subtitles available etc) but is still unable to play remote content... 

 

I dont understand how it can get metadata and file info and not be able to open the media...

 

Crazy thing is that I am able _sometimes_ to open a remote file, I would say 10% of the time.

What are your Plex scheduled task settings, and radarr/sonarr? Something is hammering the mount?

Share this post


Link to post

I deactivated plex scans, and Sonarr/Radarr cannot be change and are set to 12h.

But as explained, it does not seems to be an API limit as I have no issue uploading...

I just mounted the share on windows directly with rclone windows and I do not seems to have any issues here, really odd.

Share this post


Link to post

Not sure what's going on - have you tried the rclone forums?  Post back if you find a solution

Share this post


Link to post
5 hours ago, yendi said:

I deactivated plex scans, and Sonarr/Radarr cannot be change and are set to 12h.

But as explained, it does not seems to be an API limit as I have no issue uploading...

I just mounted the share on windows directly with rclone windows and I do not seems to have any issues here, really odd.

Upload and download Api hits are seperate. So your mount can still upload but not download. You can get banned for several reasons, with the current state or rclone my experience is that a single filed being opened too many times is often the issue. Plex can be the cause of this.

 

You say direct mounting in windows works. What exactly works? Just opening the file through your windows explorer? I'm very sure if that works, Plex will also work now and your ban has just been lifted. This often happens around midnight CET.

Share this post


Link to post

I have pretty much sonarr/radarr/plex standard, just deactivated the preview creation in plex (took too long)

 

Never had any ban or issues to retrieve remote files... so far. I have around 3000+ movies and around 150 + series.

 

How big is your database?

Share this post


Link to post
46 minutes ago, nuhll said:

I have pretty much sonarr/radarr/plex standard, just deactivated the preview creation in plex (took too long)

 

Never had any ban or issues to retrieve remote files... so far. I have around 3000+ movies and around 150 + series.

 

How big is your database?

I have about the same size of library, but I think I have idenfied the issue:

As explained I have also the problem if I open a file on windows, from an UnRaid rclone share. I tried to mount the same share directly from unraid Windows and have no issues.

 

Then to troubleshoot a bit better I installed a new Plex server on UnRaid and HAD the issue, I then installed a Plex server on Windows pointing on the rclone windows, and no issues.

 

As my server is connected via ethernet I remove this from the equation, and I assume that there is some kind of issues on my unraid system/docker engine. I had few unclean shutdown because the docker engine was stuck so I am asking myself if it did not create some side effects issues, dunno...

I have backed up everything that is not a media and will be format and clean reinstall unraid today (start from scratch and reinstall everything. I am thinking on keeping the appdata from all dockers except Plex, install them one by one if the issue is gone to immediately identify it if the issue reappears after reinstalling a docker), hopefully this will solve my issue.

Edited by yendi

Share this post


Link to post

Didnt had a single issue (after i created my own api id).

 

Only one thing i can think of, how you have your movies organzied?

 

movies/moviename/files?

or movies/moviename.file ?

Edited by nuhll

Share this post


Link to post
6 minutes ago, nuhll said:

Didnt had a single issue (after i created my own api id).

 

Only one thing i can think of, how you have your movies organzied?

 

movies/moviename/files?

or movies/moviename.file ?

I have a very standard setup: 

Radarr 1080p

Radar UHD

Sonarr

Plex

Ombi

Tauttuli

 

Organization:

 

Root

-- 4k
---- Movie 4K 1 (YEAR)
------ File
---- Movie 4K 2 (YEAR)
------ File

-- Movies
---- Movie 1080p 1 (YEAR)
------ File
---- Movie 1080p 2 (YEAR)
------ File

--TV_Shows
etc...

--Other

 

What makes me think it comes from UnRAID is that with the smae MOUNT command on a windows PC, same network and a brand new Plex server Windows I have no issue whatsover...

Edited by yendi

Share this post


Link to post
2 minutes ago, nuhll said:

So each movie is in a seperate folder?


 

Yep

Share this post


Link to post

Nope, I added 8gigs few days back. I have 20Gigs of DDR4 with a usage of ~30%

Share this post


Link to post

@yendican you post a screenshot of your Plex docker settings? Did you put the right folders as r/w slave? And what folders did you add to your Plex library? Did you disable all the analyzing stuff?

Share this post


Link to post
1 hour ago, Kaizac said:

@yendican you post a screenshot of your Plex docker settings? Did you put the right folders as r/w slave? And what folders did you add to your Plex library? Did you disable all the analyzing stuff?

image.thumb.png.284ce4c44fb5d7d4e571131ef9f43127.png

 

Regarding the R/W I restored the "Safe Docker Permissions" but did not touch anything else. Maybe it could be the folder where the temporary movie is downloaded that has RW issues? I mean Rclone must be storing chunks somewhere when playing a media, maybe he is not able to do so ? Do you guys know where it could be ?

Share this post


Link to post
9 minutes ago, yendi said:

image.thumb.png.284ce4c44fb5d7d4e571131ef9f43127.png

 

Regarding the R/W I restored the "Safe Docker Permissions" but did not touch anything else. Maybe it could be the folder where the temporary movie is downloaded that has RW issues? I mean Rclone must be storing chunks somewhere when playing a media, maybe he is not able to do so ? Do you guys know where it could be ?

Rclone stores it in your RAM, so that shouldn't be the R/W issue. I don't understand why you have that /dev/dri in your Plex template? I don't have it, so it's something you've added manually?

Also why didn't you fill in the links to Movies and such? They should be pointing to your unionfs/Movies folder with r/w access.

Share this post


Link to post
Just now, Kaizac said:

Rclone stores it in your RAM, so that shouldn't be the R/W issue. I don't understand why you have that /dev/dri in your Plex template? I don't have it, so it's something you've added manually?

Also why didn't you fill in the links to Movies and such? They should be pointing to your unionfs/Movies folder with r/w access.

The dev/dri is for hardware acceleration. Regarding the /Movies and /TV i use an other variable the /rclone one that you can see at the bottom of the screenshot...

Share this post


Link to post
Just now, yendi said:

The dev/dri is for hardware acceleration. Regarding the /Movies and /TV i use an other variable the /rclone one that you can see at the bottom of the screenshot...

Try running Plex vanilla. So without the hardware acceleration and see if it runs then. Are you sure your Plex library is pointing to the right level of folder? When you go into your Plex library and you open a movie (not play it) do you see that everything is filled in correctly? And the linking is correct to the right folder and file?

Share this post


Link to post
7 minutes ago, Kaizac said:

Try running Plex vanilla. So without the hardware acceleration and see if it runs then. Are you sure your Plex library is pointing to the right level of folder? When you go into your Plex library and you open a movie (not play it) do you see that everything is filled in correctly? And the linking is correct to the right folder and file?

For the folder mapping, yes I am sure it is mapped correctly and plex SEE the file (it can analyse it and see subs included).

Regarding Hardware transcoding, no luck, I just tried to deactivate and some media are playing, other not (remote). Crazy thing is that the same media is playing remotely with no issue and an other one is not. But I am able to open all media on a windows computer so it is not media related...

Share this post


Link to post
2 minutes ago, yendi said:

For the folder mapping, yes I am sure it is mapped correctly and plex SEE the file (it can analyse it and see subs included).

Regarding Hardware transcoding, no luck, I just tried to deactivate and some media are playing, other not (remote). Crazy thing is that the same media is playing remotely with no issue and an other one is not. But I am able to open all media on a windows computer so it is not media related...

Ok, so where do we stand now? Opening files through windows explorer all works? They all play? And only Plex now gives problems with sometimes playing and sometimes not?

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.