DZMM

Thanks for replying so quickly. I tried editing line 35 based on the link, but it's still not working for me: # make sure that your dns has a cname set for organizr server { listen 443 ssl; listen [::]:443 ssl; server_name www.*; include /config/nginx/ssl.conf; client_max_body_size 0; # enable for ldap auth, fill in ldap details in ldap.conf #include /config/nginx/ldap.conf; location / { # enable the next two lines for http auth #auth_basic "Restricted"; #auth_basic_user_file /config/nginx/.htpasswd; # enable the next two lines for ldap auth #auth_request /auth; #error_page 401 =200 /login; include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; proxy_pass http://192.168.50.17:80; } location ~ /auth-([0-9]+) { # This is used for Organizr V2 internal; include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; # proxy_pass http://192.168.50.17:80/api/?v1/auth&group=$1; proxy_pass http://192.168.50.17:80/api/v2/auth?group=$1; proxy_set_header Content-Length ""; } }

Thanks, but I'm not sure what to do to get access to my sites. This is what I currently have for radarr and the others are similar. Can you tell me what to change please: location ^~ /radarr { auth_request /auth-0; # enable the next two lines for http auth #auth_basic "Restricted"; #auth_basic_user_file /config/nginx/.htpasswd; # enable the next two lines for ldap auth, also customize and enable ldap.conf in the default conf #auth_request /auth; #error_page 401 =200 /login; include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; # set $upstream_radarr radarr; proxy_pass http://192.168.50.93:7878; } location ^~ /radarr/api { auth_request /auth-0; include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; # set $upstream_radarr radarr; proxy_pass http://192.168.50.93:7878; }

Ahh I understand what you are saying now. I'm on the move so I can't check easily, but I think the upload script has delete-empty-src-dirs set to on, which explains this behaviour. Edit: I just checked - it is on All of this is controlled in your share settings in unraid

Nope, I added this to the script to try and help first-timers. Do you do this before or after the mount script? It's a bad idea to do b4 the script runs, as you might get mounting issues - "mountpoint isn't empty" errors. Once rclone and mergerfs are mounted, it's 100% safe to 'create' folders in mergerfs (in reality, the folder is created in /local until uploaded to gdrive) and this is what you should do. That's the whole point - radarr/sonarr/manual rips etc being added to mergerfs that are accessible to plex regardless of what stage they are at - still residing locally or moved to gdrive. Thanks for the beer just now - if only I could go somewhere to buy one right now!

Thanks for the Swedish Beer donation via Paypal whoever sent it!

I do - just a bit of paranoia that if something went wrong with the upload, then the streaming mount wouldn't be impacted. Probably overkill as I think I've only had 1 or 2 API bans in over 2 years and none since I started doing this.

Edit the script if you want to, or let the script create the directory. create another instance of the upload script and choose copy not move or sync Yes - create more instances of the mount script and disable the mergerfs mount if you don't need. If you want the other drives in your mergerfs mount, add the extra rclone mount locations as extra local folder locations in the mount script that creates the mergerfs mount.

of course not It supports whatever backends rclone supports. What the streaming experience is like for non-google storage? I don't know - check the rclone forums

You don't have to encrypt your files if you don't want to. Just create an unecrpyted rclone remote. This is very easy to do - if you need help doing this there are other threads (although you can probably work out what you need to do in this thread) as this thread is for support of my scripts In my scripts, RcloneMountShare="/mnt/wherever_you_want/mount_rclone" - doesn't matter as these aren't actually stored anywhere LocalFilesShare="/mnt/ua_hdd_or_whatever_you_called_it/local" - for the files that are pending upload to gdrive MergerfsMountShare="/mnt/wherever_you_want/mount_mergerfs" - doesn't matter as these aren't actually stored anywhere I've just checked my readme, and once you've worked out how to setup your remotes, which isn't covered but shows what they should look like afterwards, all the information you need is there https://github.com/BinsonBuzz/unraid_rclone_mount/blob/latest---mergerfs-support/README.md

In your logs: failed when making oauth client: error opening service account credentials file: open /mnt/user/appdata/other/rclone/service/sa_gdrive.json: no such file or directory

@live4ever all looks ok. Look in /mnt/user/mount_rclone and /mnt/user/local and you should see the source of the weird files - maybe you did a duff upload somewhere. Either way - if you don't need them (unlikely), just delete and they should go away.

There's your problem - your remotes aren't setup properly or don't exist: - superplex_media_vfs - superplex If you'd read your logs you would have seen this.

No - post your mount script and rclone config

you don't have a remote called gdrive_upload_vfs in your rclone config.

I would: 1. create another version of the mount script to mount your unencrypted media in another location e.g. /mnt/user/mount_rclone/2nd_remote and remember to chose not to create a mergerfs mount 2. then in your main mount script set LocalFilesShare2 to that location i.e. LocalFilesShare2="/mnt/user/mount_rclone/2nd_remote" Looking at your last post with your mount script it looks like you're using an old version or you've cut bits out as you don't have a LocalFilesSharex section. Make sure you are using the latest from https://github.com/BinsonBuzz/unraid_rclone_mount

you won't get API bans if you use these scripts. Teamdrives don't need client ID and secrets if setup correctly. I know this has become a long thread, but if you read the early posts and use the search a lot of the questions you are asking are easy to find answers to.

@animeking script looks fine and you should be seeing files from /mnt/user/local/gdrive_vfs and /mnt/user/mount_rclone/gdrive_vfs in /mnt/user/mount_mergerfs/gdrive_vfs. To test , add something manually to /mnt/user/local/gdrive_vfs and see if it appears in the mergerfs folder. Then run the upload script and see if it moves from /mnt/user/local/gdrive_vfs to /mnt/user/mount_rclone/gdrive_vfs - or just drag/copy it to (I'd do a small file) /mnt/user/mount_rclone/gdrive_vfs and see if still shows up in /mnt/user/mount_mergerfs/gdrive_vfs If not, post your mount logs please after running please.

Post your mount script and rclone config please - remember to blank out your passwords etc. Can you use the Code button on the forum to post as it's easier to read and reply

Best thing to do is move the files within gdrive so you don't hit the 750GB/day limit. I think the answer your config question is yes...if you've setup the SAs, you've done the hard bit.

The mount script checks if there's internet connectivity. Add a cron job so the script runs if the test fails. ####### Checking have connectivity ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online" ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online" else echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity. Will try again on next run" rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi

I just spotted this in my upload logs: 2020/10/16 08:37:03 INFO : Transferred: 1.344T / 9.679 TBytes, 14%, 13.292 MBytes/s, ETA 1w14h39m29s Checks: 286 / 290, 99% Deleted: 143 Renamed: 143 Transferred: 143 / 2190, 7% Elapsed time: 29h27m1.0s I wonder if Google have removed/increased the 750GB/day transfer limit? Edit: I think it's still 750GB/day - my elapsed time is high so I must have fluked transferring less than 750GB over 24 hours

my cache isn't big enough for a permanent vfs cache. choose your own preferred location put to ignore - I left it in as an example - in my real script all 6 are populated with various tdrives.

Here's my updated mount script. I had problems adding @watchmeexplode5's pull that creates a single config file. Adds support for vfs local caching and also changed the mount defaults, which for me, has significantly improved playback and skipping: Edit: I just spotted I've hardcoded the cache size at 400GB - I'll make this configurable when I have time to sort out github. #!/bin/bash ###################### #### Mount Script #### ###################### ### Version 0.96.8 ### ###################### ####### EDIT ONLY THESE SETTINGS ####### # INSTRUCTIONS # 1. Change the name of the rclone remote and shares to match your setup # 2. NOTE: enter RcloneRemoteName WITHOUT ':' # 3. Optional: include custom command and bind mount settings # 4. Optional: include extra folders in mergerfs mount # REQUIRED SETTINGS RcloneRemoteName="tdrive_vfs" RcloneMountShare="/mnt/user/mount_rclone" LocalFilesShare="/mnt/user/local" MergerfsMountShare="/mnt/user/mount_mergerfs" DockerStart="duplicati lazylibrarian LDAPforPlex letsencrypt nzbget ombi organizrv2 plex qbittorrentvpn radarr radarr-uhd radarr-collections sonarr sonarr-uhd tautulli" MountFolders=\{"downloads/complete,downloads/seeds,documentaries/kids,documentaries/adults,movies_adults_gd,movies_kids_gd,tv_adults_gd,tv_kids_gd,uhd/tv_adults_gd,uhd/tv_kids_gd,uhd/documentaries/kids,uhd/documentaries/adults"\} # OPTIONAL SETTINGS # Add extra paths to mergerfs mount in addition to LocalFilesShare LocalFilesShare2="/mnt/user/mount_rclone/gdrive_media_vfs" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder. Enter 'ignore' to disable LocalFilesShare3="" LocalFilesShare4="" LocalFilesShare5="" LocalFilesShare6="" # Add extra commands or filters Command1="--rc" Command2="" Command3="" Command4="" Command5="" Command6="" Command7="" Command8="" CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address NetworkAdapter="eth0" # choose your network adapter. eth0 recommended VirtualIPNumber="2" # creates eth0:x e.g. eth0:1. I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them ####### END SETTINGS ####### ############################################################################### ##### DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING ####### ############################################################################### ####### Preparing mount location variables ####### LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" ####### create directories for rclone mount and mergerfs mounts ####### mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName #for script files mkdir -p /mnt/user0/mount_rclone/cache/$RcloneRemoteName #for cache files if [[ $LocalFileShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested." else echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders." eval mkdir -p $LocalFilesLocation/"$MountFolders" fi mkdir -p $RcloneMountLocation mkdir -p $MergerFSMountLocation ####### Check if script is already running ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}" echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running." if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running." exit else echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding." touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running fi ####### Create Rclone Mount ####### # Check If Rclone Mount Already Created if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote." # Creating mountcheck file in case it doesn't already exist echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote." touch mountcheck rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse # Check bind option if [[ $CreateBindMount == 'Y' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" else echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}" ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber fi echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}" else RCloneMountIP="" echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}" fi # create rclone mount rclone mount \ --allow-other \ --dir-cache-time 720h \ --log-level INFO \ --poll-interval 15s \ --cache-dir=/mnt/user0/mount_rclone/cache/$RcloneRemoteName \ --vfs-cache-mode full \ --vfs-cache-max-size 400G \ --vfs-cache-max-age 336h \ --bind=$RCloneMountIP \ $RcloneRemoteName: $RcloneMountLocation & # Check if Mount Successful echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds" # slight pause to give mount time to finalise sleep 10 echo "$(date "+%d.%m.%Y %T") INFO: continuing..." if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems." docker stop $DockerStart find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi ####### Start MergerFS Mount ####### if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested." else if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place." else # check if mergerfs already installed if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount" else # Build mergerfs binary echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now." mkdir -p /mnt/user/appdata/other/rclone/mergerfs docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin # check if mergerfs install successful echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds" sleep 10 if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount." else echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully. Please check for errors. Exiting." docker stop $DockerStart find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi # Create mergerfs mount echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount." # Extra Mergerfs folders if [[ $LocalFilesShare2 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare2=":$LocalFilesShare2" else LocalFilesShare2="" fi if [[ $LocalFilesShare3 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare3=":$LocalFilesShare3" else LocalFilesShare3="" fi if [[ $LocalFilesShare4 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare4=":$LocalFilesShare4" else LocalFilesShare4="" fi if [[ $LocalFilesShare5 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare5} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare5=":$LocalFilesShare5" else LocalFilesShare5="" fi if [[ $LocalFilesShare6 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare6} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare6=":$LocalFilesShare6" else LocalFilesShare6="" fi # mergerfs mount command mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4$LocalFilesShare5$LocalFilesShare6 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true # check if mergerfs mount successful echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created." if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed." docker stop $DockerStart find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi fi ####### Starting Dockers That Need Mergerfs Mount To Work Properly ####### echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start $DockerStart rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running echo "$(date "+%d.%m.%Y %T") INFO: ${RcloneRemoteName} Script complete" exit

prefer cache is a preference - go with wherever you want files storing Correct. Advice is to run using user scripts on a schedule e.g. every 2-5 mins, so if for whatever reason the mount fails, it will stop the dockers to try and avoid any meltdowns since rclone 1.53 (I think) full works better - before it was a bad idea. I've updated my local scripts, but I had a few problems updating github so that's a bit behind (it also takes advantage of the new cache feature, which makes a BIG difference). I'll try again to update github again this weekend. Correct - rclone added a nice feature a few releases back that stopped a transfer job when the 750GB limit is hit. SAs allow a different user or quota to be used on the next run, so if you set a cron job, you can do multiple amounts of 750GB/day e.g. 16 SAs allows you to max out a 1Gbps upload 24/7. yeah, you won't get the full benefit of mergerfs. I would honestly just do this for all your dockers: /user --> /mnt/user and then within radarr add as your media locations /user/gDrive/ShareName/Movies etc etc I just find it easier to do this as all dockers will always match up, and I think the convention of mapping /downloads --> xxxxxx /media --> xxxxx etc, just makes live more complicated. Moving is easy: 1. add the new /user --> /mnt/user mapping 2. add it /user/whatever as a new media location in radarr 3. go to Movie Editor, create a custom filter e.g. show movies with /ShareName/Movies etc in path 4. select all and move them to /user/whatever within Radarr

Thanks. I was too scared to click on the upgrade link before.... I went for it this time and it's spitting out £15.30/mth in the UK - strangely no discount for annual. I'm paying £7.82/mth at the moment - double the price, but still great value for over 600TB of storage!! I'm not upgrading either yet - unless I can find a coupon for Enterprise Standard. It's a good sign that Google aren't shutting down the facility - there are bigger things behind this change, rather than trying to address users like us. I still believe we are a drop in the ocean Vs some institutions e.g. research universities.