DZMM

Thanks for posting the thread. I hadn't noticed any problems with playback and none of my users had mentioned anything, but my upload job was getting 429 errors. Adding the user agent to my upload script seems to have got it going again but it seems a bit slower - about 75% of the previous speed. It's a bit worrying if they are targetting the rclone user agent.....

Manually remove this file /mnt/user/appdata/other/rclone/rclone_upload It should get removed at array start/end depending on how you've setup your unmount script

You either need to put the files in the right subfolders in your upload folder, or your scripts aren't quite setup properly - post your upload script

The scripts don't move files to the upload folder - you have to do that, or you've setup radarr to add new files to the mount. The min-age setting tells rclone not to move files from the upload folder to the mount that's younger than 30 mins.

The touch command creates a file - you need to run the next commands to copy it to the mounts: rclone copy mountcheck gdrive_media_vfs: -vv --no-traverse rclone copy mountcheck tdrive_media_vfs: -vv --no-traverse

Playback will be awful if you stream using a normal mount and you'll probably suffer from API bans

Can you post your rclone config (remember to take out any personal info e.g. passwords) and mount script please

not sure what's happening, but try taking: mkdir -p /mnt/user/mount_rclone/google_vfs out of your mount script

Just configure tdrive to not be a crypt and make it a direct mount of your google drive.

Yes, although I'd do every 5 minutes personally and also run the unmount at array start so it runs if you have an unclean shutdown

sorry, @francrouge you were having the problem I just fixed. If the mount was successful it wasn't deleting the /mnt/user/appdata/other/rclone/rclone_mount_running file allowing a manual mount to be run, until the unmount script runs at array stop or start. I've removed this check altogether as it's not needed as the script automatically remounts only if the mountcheck file isn't there, which is the real check that needs doing. Here's your scripts: #!/bin/bash ####### Start rclone mounts ########## # create directories for rclone mount and unionfs mount mkdir -p /mnt/user/appdata/other/rclone mkdir -p /mnt/user/mount_rclone/google_vfs mkdir -p /mnt/user/mount_unionfs/google_vfs mkdir -p /mnt/user/rclone_upload/google_vfs ####### Start rclone gdrive mount ########## # check if gdrive mount already created if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check rclone vfs already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: mounting rclone vfs." rclone mount --allow-other --buffer-size 512M --dir-cache-time 72h --drive-chunk-size 512M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs & # check if mount successful # slight pause to give mount time to finalise sleep 5 if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check rclone gdrive vfs mount success." else echo "$(date "+%d.%m.%Y %T") CRITICAL: rclone gdrive vfs mount failed - please check for problems." exit fi fi ####### End rclone gdrive mount ########## ####### Start unionfs mount ########## if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs already mounted." else unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs mounted." else echo "$(date "+%d.%m.%Y %T") CRITICAL: unionfs Remount failed." exit fi fi ####### End Mount unionfs ########## ############### starting dockers that need unionfs mount ###################### # only start dockers once if [[ -f "/mnt/user/appdata/other/rclone/dockers_started" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: dockers already started" else touch /mnt/user/appdata/other/rclone/dockers_started echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start PlexMediaServer docker start tautulli docker start binhex-sonarr docker start binhex-sickchill docker start binhex-jackett docker start binhex-delugevpn docker start binhex-krusader fi ############### end dockers that need unionfs mount ###################### exit #!/bin/bash # unmount to be safe fusermount -uz /mnt/user/mount_unionfs/google_vfs fusermount -uz /mnt/user/mount_rclone/google_vfs # Remove other dummy files if [[ -f "/mnt/user/appdata/other/rclone/rclone_upload" ]]; then echo "upload file present - removing dummy file" rm /mnt/user/appdata/other/rclone/rclone_upload else echo "rclone upload already exited properly" fi if [[ -f "/mnt/user/appdata/other/rclone/rclone_cleanup" ]]; then echo "cleanup file present - removing dummy file" rm /mnt/user/appdata/other/rclone/rclone_cleanup else echo "cleanup already exited properly" fi if [[ -f "/mnt/user/appdata/other/rclone/dockers_started" ]]; then echo "removing dummy docker check file" rm /mnt/user/appdata/other/rclone/dockers_started else echo "docker run once already removed" fi exit

I've fixed some errors in the mount and unmount scripts which might explain why some users were having problems re-mounting

I'm going to try and do a review of the github content this weekend as I haven't updated it for over 4 months now.

Post your full script

Yes

My script should automatically remount if you set it up to run on a cron job say every 5 mins - that's what I use. The problem with Plex is if it scans while the mount is down, if your settings are 'wrong' it will assume the files no longer exist and will remove them from the library. To avoid this make sure your Plex library settings are set to not automatically remove missing content - Plex will then add a trash sign next to content it can no longer see, and will fix itself if that content becomes available again. Doing this does mean you need to manually empty trash every now and then when you know it's safe to do so i.e. when the mount is active.

I'm not sure what's going on there or if this will solve your problems: https://rclone.org/drive/#making-your-own-client-id You might need to post on the rclone forums

Afraid so. If you're a rclone user it looks rclone union will get beefed up in 1.48 so I'll be able to ditch unionfs

With only 4GB you might need to go even smaller if you've got concurrent usage as I think the buffer is per stream. I see you also removed --vfs-read-chunk-size-limit off from your mount script. This defaults to off so removing won't help - if you're still having memory problems I'd set this again to something like 256MB of 512MB, but you're going to struggle to play remux files IMHO if you don't have enough ram to buffer the stream. I don't know what size chunks my setup requests, but I have no issues. If you're happy with only being able to play files once they've been uploaded then this is fine. However, if you have a lot of files that are pending upload and you want to Plex/Sonarr/Radarr etc to be able to see them, then this is where the unionfs and rclone_upload folders come into play - e.g. unless you make your setup a lot more complicated you can only upload 750GB/day. There are ways around this e.g. my setup allows me to upload at my full 1G line speed 247, but it took me a few hours to setup, but it means I don't have files waiting to be uploaded for more than a few mins. I still need the unionfs setup though so that Plex doesn't have to rescan thinking the file has changed when it moves, when it hasn't etc etc Check out the new scripts on github as it accounts for this e.g. if it tries to mount and there's no internet connection then it tries again a few mins later. This check is useful if your mount ever drops as it will automatically remount.

just stop radarr, sonarr, plex etc - anything that's accessing the mount before you unmount. Shouldn't have any problems

hmm have you tried unmounting and removing from rclone_upload or directly from the mount?

remove manually?

pfsense traffic shaping does shape downstream - it does this by dropping packets, so that's why setting your bandwidth settings is important.

But from the official documentation I can see Why have you enabled the Quick flag? Ignore my first version as it didn't work properly. It should do. I use floating rules as it was easier for me to have all in one place, and also you can adjust the order then.

I'm 99% certain that at least for Plex no disks are used to play files - everything is streamed using RAM. I think even the rclone dir_cache is stored in RAM. I'm not sure how Kodi works i.e. is the file downloaded before playing - I don't think so. The thread has kind of got long, which is good - at first I was worried no-one was interested but the collaboration has helped improve setups, including mine. I've never had an API ban and my library is reasonably big - there are users in this thread with libraries x5 the size of mine and over a Petabyte. I haven't had any problems or even stuttering for a couple of months, and all my content is in a team drive now: root@Highlander:~# rclone size tdrive_vfs: Total objects: 53947 Total size: 248.142 TBytes (272835090178653 Bytes)