DZMM Posted April 29, 2022 Author Share Posted April 29, 2022 (edited) On 4/28/2022 at 6:25 PM, southloven said: I just got a 2nd mount up and running and it hit me I May have a problem with restarts. If docker containers have to be started after the mounts are established how to you make sure both mounts have completed before starting the dockers? Has any one addressed? or is this not a problem? put the mounts in one script and start the dockers only when the 2nd mount is successful! Edit: hardcode the dockers you need to stop in the relevant section for the first mount if it fails Here's a snippet from my combined mount script that mounts about 7 team drives in sequence, but only the last mount creates the mergerfs mount. The key line is the hardcoded docker stop section that stops the dockers if any of the mounts fail: ################################################################ ###################### mount uhd ############################## ################################################################ # REQUIRED SETTINGS RcloneRemoteName="tdrive_uhd_vfs" RcloneMountShare="/mnt/user/mount_rclone" LocalFilesShare="ignore" MergerfsMountShare="ignore" DockerStart="swag" MountFolders=\{"uhd/tv_adults_gd,uhd/tv_kids_gd,uhd/documentaries/kids,uhd/documentaries/adults"\} # OPTIONAL SETTINGS # Add extra paths to mergerfs mount in addition to LocalFilesShare LocalFilesShare2="ignore" LocalFilesShare3="ignore" LocalFilesShare4="ignore" # Add extra commands or filters Command1="" Command2="" Command3="" Command4="" Command5="" Command6="" Command7="" Command8="" CreateBindMount="N" RCloneMountIP="192.168.1.76" NetworkAdapter="eth0" VirtualIPNumber="6" ####### END SETTINGS ####### # fusermount -uz /mnt/user/mount_rclone/$RcloneRemoteName ####### Preparing mount location variables ####### LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" ####### create directories for rclone mount and mergerfs mounts ####### mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName #for script files mkdir -p /mnt/user/mount_rclone/cache/$RcloneRemoteName #for cache files if [[ $LocalFilesShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested." else echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders." eval mkdir -p $LocalFilesLocation/"$MountFolders" fi mkdir -p $RcloneMountLocation mkdir -p $MergerFSMountLocation ####### Check if script is already running ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}" echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running." if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running." exit else echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding." touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running fi ####### Create Rclone Mount ####### # Check If Rclone Mount Already Created if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote." # Creating mountcheck file in case it doesn't already exist echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote." touch mountcheck rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse # Check bind option if [[ $CreateBindMount == 'Y' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" else echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}" ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber fi echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}" else RCloneMountIP="" echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}" fi # create rclone mount rclone mount \ $Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \ --allow-other \ --umask 000 \ --dir-cache-time 5000h \ --attr-timeout 5000h \ --log-level INFO \ --poll-interval 10s \ --cache-dir=/mnt/user/mount_rclone/cache/$RcloneRemoteName \ --drive-pacer-min-sleep 10ms \ --drive-pacer-burst 1000 \ --vfs-cache-mode full \ --vfs-read-chunk-size 256M \ --vfs-cache-max-size 400G \ --vfs-cache-max-age 96h \ --vfs-read-ahead 1G \ --bind=$RCloneMountIP \ $RcloneRemoteName: $RcloneMountLocation & # Check if Mount Successful echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds" # slight pause to give mount time to finalise sleep 10 echo "$(date "+%d.%m.%Y %T") INFO: continuing..." if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems." docker stop qbittorrentvpn readarr plex radarr_new radarr-uhd sonarr sonarr_new sonarr-uhd find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi ####### Start MergerFS Mount ####### if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested." else if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place." else # check if mergerfs already installed if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount" else # Build mergerfs binary echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now." mkdir -p /mnt/user/appdata/other/rclone/mergerfs docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin # check if mergerfs install successful echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds" sleep 10 if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount." else echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully. Please check for errors. Exiting." docker stop qbittorrentvpn readarr plex radarr_new radarr-uhd sonarr sonarr_new sonarr-uhd find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi # Create mergerfs mount echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount." # Extra Mergerfs folders if [[ $LocalFilesShare2 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare2=":$LocalFilesShare2" else LocalFilesShare2="" fi if [[ $LocalFilesShare3 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare3=":$LocalFilesShare3" else LocalFilesShare3="" fi if [[ $LocalFilesShare4 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare4=":$LocalFilesShare4" else LocalFilesShare4="" fi # mergerfs mount command mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true # check if mergerfs mount successful echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created." if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed." docker stop qbittorrentvpn readarr plex radarr_new radarr-uhd sonarr sonarr_new sonarr-uhd find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi fi ####### Starting Dockers That Need Mergerfs Mount To Work Properly ####### if [[ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ]] || [[ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ]] ; then echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers." else echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start $DockerStart fi rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running echo "$(date "+%d.%m.%Y %T") INFO: ${RcloneRemoteName} Script complete" ################################################################ ###################### mount tdrive ######################## ################################################################ # # fusermount -uz /mnt/user/mount_rclone/tdrive_vfs # # fusermount -uz /mnt/user/mount_mergerfs/tdrive_vfs # REQUIRED SETTINGS RcloneRemoteName="tdrive_vfs" RcloneMountShare="/mnt/user/mount_rclone" LocalFilesShare="/mnt/user/local" MergerfsMountShare="/mnt/user/mount_mergerfs" DockerStart="qbittorrentvpn readarr plex radarr_new radarr-uhd sonarr sonarr_new sonarr-uhd" MountFolders=\{"anime/movies,downloads/complete,documentaries/kids,documentaries/adults,movies_adults_gd,movies_kids,tv_adults,uhd/tv_adults_gd,uhd/tv_kids_gd,uhd/documentaries/kids,uhd/documentaries/adults"\} # OPTIONAL SETTINGS # Add extra paths to mergerfs mount in addition to LocalFilesShare LocalFilesShare2="/mnt/user/mount_rclone/gdrive_media_vfs" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder. Enter 'ignore' to disable LocalFilesShare3="/mnt/user/mount_rclone/tdrive_uhd_vfs" LocalFilesShare4="/mnt/user/mount_rclone/tdrive_tv_adults_vfs" LocalFilesShare5="/mnt/user/mount_rclone/tdrive_tv_kids_vfs" LocalFilesShare6="/mnt/user/mount_rclone/tdrive_movies_adults_vfs" # Add extra commands or filters Command1="" Command2="" Command3="" Command4="" Command5="" Command6="" Command7="" Command8="" CreateBindMount="N" RCloneMountIP="192.168.1.77" NetworkAdapter="eth0" VirtualIPNumber="7" ####### END SETTINGS ####### # fusermount -uz /mnt/user/mount_rclone/$RcloneRemoteName ####### Preparing mount location variables ####### LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" ####### create directories for rclone mount and mergerfs mounts ####### mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName #for script files mkdir -p /mnt/user/mount_rclone/cache/$RcloneRemoteName #for cache files if [[ $LocalFilesShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested." else echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders." eval mkdir -p $LocalFilesLocation/"$MountFolders" fi mkdir -p $RcloneMountLocation mkdir -p $MergerFSMountLocation ####### Check if script is already running ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}" echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running." if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running." exit else echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding." touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running fi ####### Create Rclone Mount ####### # Check If Rclone Mount Already Created if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote." # Creating mountcheck file in case it doesn't already exist echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote." touch mountcheck rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse # Check bind option if [[ $CreateBindMount == 'Y' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" else echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}" ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber fi echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}" else RCloneMountIP="" echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}" fi # create rclone mount rclone mount \ $Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \ --allow-other \ --umask 000 \ --dir-cache-time 5000h \ --attr-timeout 5000h \ --log-level INFO \ --poll-interval 10s \ --cache-dir=/mnt/user/mount_rclone/cache/$RcloneRemoteName \ --drive-pacer-min-sleep 10ms \ --drive-pacer-burst 1000 \ --vfs-cache-mode full \ --vfs-cache-max-size 400G \ --vfs-cache-max-age 96h \ --vfs-read-ahead 1G \ --bind=$RCloneMountIP \ $RcloneRemoteName: $RcloneMountLocation & # Check if Mount Successful echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds" # slight pause to give mount time to finalise sleep 10 echo "$(date "+%d.%m.%Y %T") INFO: continuing..." if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems." docker stop $DockerStart fusermount -uz /mnt/user/mount_mergerfs/tdrive_vfs find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi ####### Start MergerFS Mount ####### if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested." else if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place." else # check if mergerfs already installed if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount" else # Build mergerfs binary echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now." mkdir -p /mnt/user/appdata/other/rclone/mergerfs docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin # check if mergerfs install successful echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds" sleep 10 if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount." else echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully. Please check for errors. Exiting." docker stop $DockerStart fusermount -uz /mnt/user/mount_mergerfs/tdrive_vfs find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi # Create mergerfs mount echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount." # Extra Mergerfs folders if [[ $LocalFilesShare2 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare2=":$LocalFilesShare2" else LocalFilesShare2="" fi if [[ $LocalFilesShare3 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare3=":$LocalFilesShare3" else LocalFilesShare3="" fi if [[ $LocalFilesShare4 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare4=":$LocalFilesShare4" else LocalFilesShare4="" fi if [[ $LocalFilesShare5 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare5} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare5=":$LocalFilesShare5" else LocalFilesShare5="" fi if [[ $LocalFilesShare6 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare6} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare6=":$LocalFilesShare6" else LocalFilesShare6="" fi # mergerfs mount command mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4$LocalFilesShare5$LocalFilesShare6 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true # check if mergerfs mount successful echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created." if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed." docker stop $DockerStart fusermount -uz /mnt/user/mount_mergerfs/tdrive_vfs find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi fi ####### Starting Dockers That Need Mergerfs Mount To Work Properly ####### if [[ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ]] || [[ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ]] ; then echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers." else echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start $DockerStart fi rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running echo "$(date "+%d.%m.%Y %T") INFO: ${RcloneRemoteName} Script complete" Edited April 29, 2022 by DZMM 1 Quote Link to comment
sol Posted May 4, 2022 Share Posted May 4, 2022 New email from Google. Anyone else get this? Anyone figured out how to proceed? (edited out some identifying content) Our records indicate you have OAuth clients that used the OAuth OOB flow in the past. Hello Google OAuth Developer, We are writing to inform you that OAuth out-of-band (OOB) flow will be deprecated on October 3, 2022, to protect users from phishing and app impersonation attacks. What do I need to know? Starting October 3, 2022, we will block OOB requests to Google’s OAuth 2.0 authorization endpoint for existing clients. Apps using OOB in testing mode will not be affected. However, we strongly recommend you to migrate them to safer methods as these apps will be immediately blocked when switching to in production status. Note: New OOB usage has already been disallowed since February 28, 2022. Below are key dates for compliance September 5, 2022: A user-facing warning message may be displayed to non-compliant OAuth requests October 3, 2022: The OOB flow is blocked for all clients and users will see the error page. Please check out our recent blog post about Making Google OAuth interactions safer for more information. What do I need to do? Migrate your app(s) to an appropriate alternative method by following these instructions: Determine your app(s) client type from your Google Cloud project by following the client links below. Migrate your app(s) to a more secure alternative method by following the instructions in the blog post above for your client type. If necessary, you may request a one-time extension for migrating your app until January 31, 2023. Keep in mind that all OOB authorization requests will be blocked on February 1, 2023. The following OAuth client(s) will be blocked on Oct 3, 2022. OAuth client list: Project ID: rcloneclientid-247*** Client: 211984046708-hahav9pt2t2v6mc6*********apps.googleusercontent.com Thanks for choosing Google OAuth. — The Google OAuth Developer Team Quote Link to comment
axeman Posted May 4, 2022 Share Posted May 4, 2022 3 hours ago, sol said: New email from Google. Anyone else get this? Anyone figured out how to proceed? (edited out some identifying content) Our records indicate you have OAuth clients that used the OAuth OOB flow in the past. Hello Google OAuth Developer, We are writing to inform you that OAuth out-of-band (OOB) flow will be deprecated on October 3, 2022, to protect users from phishing and app impersonation attacks. What do I need to know? Starting October 3, 2022, we will block OOB requests to Google’s OAuth 2.0 authorization endpoint for existing clients. Apps using OOB in testing mode will not be affected. However, we strongly recommend you to migrate them to safer methods as these apps will be immediately blocked when switching to in production status. Note: New OOB usage has already been disallowed since February 28, 2022. Below are key dates for compliance September 5, 2022: A user-facing warning message may be displayed to non-compliant OAuth requests October 3, 2022: The OOB flow is blocked for all clients and users will see the error page. Please check out our recent blog post about Making Google OAuth interactions safer for more information. What do I need to do? Migrate your app(s) to an appropriate alternative method by following these instructions: Determine your app(s) client type from your Google Cloud project by following the client links below. Migrate your app(s) to a more secure alternative method by following the instructions in the blog post above for your client type. If necessary, you may request a one-time extension for migrating your app until January 31, 2023. Keep in mind that all OOB authorization requests will be blocked on February 1, 2023. The following OAuth client(s) will be blocked on Oct 3, 2022. OAuth client list: Project ID: rcloneclientid-247*** Client: 211984046708-hahav9pt2t2v6mc6*********apps.googleusercontent.com Thanks for choosing Google OAuth. — The Google OAuth Developer Team Post #5 https://forum.rclone.org/t/google-oauth-migration-for-rclone/30545/5 That's from THE lead on rclone. seems it might not be a big deal. Quote Link to comment
Bolagnaise Posted May 4, 2022 Share Posted May 4, 2022 As long as you are running rclone 1.58 or newer then rclone will continue to work normally. Essentially what is happening is google is deprecating the copy paste method for the client ID and secret from the admin console but only for ios,android and some other devices. The method rclone uses in 1.58 to re-authenticate to google is different. TLDR: Ensure your rclone plugin is up to date and everything will continue to work fine. Quote Link to comment
Bolagnaise Posted May 6, 2022 Share Posted May 6, 2022 @DZMM I @ you in this thread but for awareness of everyone else using RC, seems to be a permissions error with rc builds as LT updated how the permissions are managed from 6.9.2 Quote Link to comment
Bolagnaise Posted May 6, 2022 Share Posted May 6, 2022 (edited) adding the below to you mount script should resolve the permissions issue if your updating to 6.10 RC7, i will test in approx 12 hours if this fixes the permissions issue with docker containers and plex. Quote --uid 98 \ --gid 99 \ if you run the below from cli it should confirm what USER nobody is set too Quote getent group | grep no Edited May 6, 2022 by Bolagnaise 1 1 Quote Link to comment
Kaizac Posted May 6, 2022 Share Posted May 6, 2022 4 hours ago, Bolagnaise said: adding the below to you mount script should resolve the permissions issue if your updating to 6.10 RC7, i will test in approx 12 hours if this fixes the permissions issue with docker containers and plex. What is the reason for you to use uid 98 and gid 99 instead of the uid 99 and gid 99? Quote Link to comment
MadMatt337 Posted May 6, 2022 Share Posted May 6, 2022 (edited) 11 hours ago, Bolagnaise said: @DZMM I @ you in this thread but for awareness of everyone else using RC, seems to be a permissions error with rc builds as LT updated how the permissions are managed from 6.9.2 Odd, I have been using the RC's right from 6.10 RC2 on and have never had a permission issue, no modifications to the base script. I did not upgrade from 6.9.2 or anything lower though, I did a fresh install of 6.10 RC2 and have been updating since then. That being said I did not let the script make any of my folders within the mergerfs folder, I made them myself after the script had everything mounted, maybe that is the difference in my case? Edited May 6, 2022 by MadMatt337 Quote Link to comment
Bolagnaise Posted May 6, 2022 Share Posted May 6, 2022 10 hours ago, Kaizac said: What is the reason for you to use uid 98 and gid 99 instead of the uid 99 and gid 99? I tested 99/99, it did not work for me and the file permissions continued If you run Quote getent group | grep no in command line, you will see that Unraid uses 'USER NOBODY' on ID 98, and "USERGROUP NOBODY" on ID 99. At least for me it was I woke up early and updated my server to RC7 and now it works fine, before none of my dockers utilizing the gdrive folders would work and threw file permission errors in the log. Quote Link to comment
DZMM Posted May 6, 2022 Author Share Posted May 6, 2022 54 minutes ago, Bolagnaise said: I tested 99/99, it did not work for me and the file permissions continued If you run in command line, you will see that Unraid uses 'USER NOBODY' on ID 98, and "USERGROUP NOBODY" on ID 99. At least for me it was I woke up early and updated my server to RC7 and now it works fine, before none of my dockers utilizing the gdrive folders would work and threw file permission errors in the log. I'm on rc6 and I don't think I'm having any problems?? Quote Link to comment
Bolagnaise Posted May 6, 2022 Share Posted May 6, 2022 23 minutes ago, DZMM said: I'm on rc6 and I don't think I'm having any problems?? Yeah im not sure why this error has crept up, but i think it might have something to do with the patch notes from rc2 where it states Quote Fixed issue in User Share file system where permissions were not being honored. This seems to have introduced a permissions issue for some people using rclone as there where no issues in rc1. Quote Link to comment
veritas2884 Posted May 19, 2022 Share Posted May 19, 2022 On 5/4/2022 at 6:03 PM, Bolagnaise said: As long as you are running rclone 1.58 or newer then rclone will continue to work normally. Essentially what is happening is google is deprecating the copy paste method for the client ID and secret from the admin console but only for ios,android and some other devices. The method rclone uses in 1.58 to re-authenticate to google is different. TLDR: Ensure your rclone plugin is up to date and everything will continue to work fine. The rclone plug-in I am running in unRAID is by Waseh. It is up to date with a 2022.1.20 last update. However, reading through the change logs it seems like the last change log that mentions a version update is from 2017 mentioning a 1.3x rclone version. Is there a way to verify what version of rclone the plugin is running so I don’t get snagged by the oath issue? Quote Link to comment
axeman Posted May 21, 2022 Share Posted May 21, 2022 Anyone have a quick/dirty script to move the metadata from the g drive folders to local array? I foolishly didn't set the filters - and am now thinking it might be better to have all metadata local ... I imagine faster loading, but more importantly, avoiding the 400K objects in team drives. Quote Link to comment
hinduguy Posted May 22, 2022 Share Posted May 22, 2022 Anyone getting this error after updating to Version: 6.10.1 21.05.2022 19:55:07 INFO: Creating local folders. 21.05.2022 19:55:07 INFO: Creating MergerFS folders. 21.05.2022 19:55:07 INFO: *** Starting mount of remote gdrive 21.05.2022 19:55:07 INFO: Checking if this script is already running. 21.05.2022 19:55:07 INFO: Script not running - proceeding. 21.05.2022 19:55:07 INFO: *** Checking if online 21.05.2022 19:55:08 PASSED: *** Internet online 21.05.2022 19:55:08 INFO: Success gdrive remote is already mounted. 21.05.2022 19:55:08 INFO: Mergerfs already installed, proceeding to create mergerfs mount 21.05.2022 19:55:08 INFO: Creating gdrive mergerfs mount. mv: cannot move '/mnt/user/mount_mergerfs/gdrive' to '/mnt/user/local/gdrive/gdrive': File exists fuse: mountpoint is not empty fuse: if you are sure this is safe, use the 'nonempty' mount option 21.05.2022 19:55:08 INFO: Checking if gdrive mergerfs mount created. 21.05.2022 19:55:08 CRITICAL: gdrive mergerfs mount failed. Stopping dockers. plex Error response from daemon: No such container: nzbget Error response from daemon: No such container: sonarr Error response from daemon: No such container: radarr Error response from daemon: No such container: ombi Script Finished May 21, 2022 19:55.08 Script #!/bin/bash ###################### #### Mount Script #### ###################### ## Version 0.96.9.3 ## ###################### ####### EDIT ONLY THESE SETTINGS ####### # INSTRUCTIONS # 1. Change the name of the rclone remote and shares to match your setup # 2. NOTE: enter RcloneRemoteName WITHOUT ':' # 3. Optional: include custom command and bind mount settings # 4. Optional: include extra folders in mergerfs mount # REQUIRED SETTINGS RcloneRemoteName="gdrive" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash e.g. /mnt/user/mount_rclone RcloneMountDirCacheTime="720h" # rclone dir cache time LocalFilesShare="/mnt/user/local" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable RcloneCacheShare="/mnt/user0/mount_rclone" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone RcloneCacheMaxSize="400G" # Maximum size of rclone cache RcloneCacheMaxAge="336h" # Maximum age of cache files MergerfsMountShare="/mnt/user/mount_mergerfs" # location without trailing slash e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable DockerStart="nzbget plex sonarr radarr ombi" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount # Note: Again - remember to NOT use ':' in your remote name above # OPTIONAL SETTINGS # Add extra paths to mergerfs mount in addition to LocalFilesShare LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder. Enter 'ignore' to disable LocalFilesShare3="ignore" LocalFilesShare4="ignore" # Add extra commands or filters Command1="--rc" Command2="" Command3="" Command4="" Command5="" Command6="" Command7="" Command8="" CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address NetworkAdapter="eth0" # choose your network adapter. eth0 recommended VirtualIPNumber="2" # creates eth0:x e.g. eth0:1. I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them ####### END SETTINGS ####### ############################################################################### ##### DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING ####### ############################################################################### ####### Preparing mount location variables ####### RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location for rclone mount LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" # Location for local files to be merged with rclone mount MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" # Rclone data folder location ####### create directories for rclone mount and mergerfs mounts ####### mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName # for script files mkdir -p $RcloneCacheShare/cache/$RcloneRemoteName # for cache files if [[ $LocalFilesShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested." LocalFilesLocation="/tmp/$RcloneRemoteName" eval mkdir -p $LocalFilesLocation else echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders." eval mkdir -p $LocalFilesLocation/"$MountFolders" fi mkdir -p $RcloneMountLocation if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating MergerFS folders as requested." else echo "$(date "+%d.%m.%Y %T") INFO: Creating MergerFS folders." mkdir -p $MergerFSMountLocation fi ####### Check if script is already running ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}" echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running." if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running." exit else echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding." touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running fi ####### Checking have connectivity ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online" ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online" else echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity. Will try again on next run" rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi ####### Create Rclone Mount ####### # Check If Rclone Mount Already Created if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote." # Creating mountcheck file in case it doesn't already exist echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote." touch mountcheck rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse # Check bind option if [[ $CreateBindMount == 'Y' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" else echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}" ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber fi echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}" else RCloneMountIP="" echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}" fi # create rclone mount rclone mount \ $Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \ --allow-other \ --umask 000 \ --dir-cache-time $RcloneMountDirCacheTime \ --attr-timeout $RcloneMountDirCacheTime \ --log-level INFO \ --poll-interval 10s \ --cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \ --drive-pacer-min-sleep 10ms \ --drive-pacer-burst 1000 \ --vfs-cache-mode full \ --vfs-cache-max-size $RcloneCacheMaxSize \ --vfs-cache-max-age $RcloneCacheMaxAge \ --vfs-read-ahead 1G \ --bind=$RCloneMountIP \ $RcloneRemoteName: $RcloneMountLocation & # Check if Mount Successful echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds" # slight pause to give mount time to finalise sleep 5 echo "$(date "+%d.%m.%Y %T") INFO: continuing..." if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems. Stopping dockers" docker stop $DockerStart rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi fi ####### Start MergerFS Mount ####### if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested." else if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place." else # check if mergerfs already installed if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount" else # Build mergerfs binary echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now." mkdir -p /mnt/user/appdata/other/rclone/mergerfs docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin # check if mergerfs install successful echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds" sleep 5 if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount." else echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully. Please check for errors. Exiting." rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi fi # Create mergerfs mount echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount." # Extra Mergerfs folders if [[ $LocalFilesShare2 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare2=":$LocalFilesShare2" else LocalFilesShare2="" fi if [[ $LocalFilesShare3 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare3=":$LocalFilesShare3" else LocalFilesShare3="" fi if [[ $LocalFilesShare4 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare4=":$LocalFilesShare4" else LocalFilesShare4="" fi # make sure mergerfs mount point is empty mv $MergerFSMountLocation $LocalFilesLocation mkdir -p $MergerFSMountLocation # mergerfs mount command mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true # check if mergerfs mount successful echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created." if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed. Stopping dockers." docker stop $DockerStart rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi fi fi ####### Starting Dockers That Need Mergerfs Mount To Work Properly ####### # only start dockers once if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: dockers already started." else # Check CA Appdata plugin not backing up or restoring if [ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ] || [ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ] ; then echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers." else touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start $DockerStart fi fi rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running echo "$(date "+%d.%m.%Y %T") INFO: Script complete" exit Quote Link to comment
DZMM Posted May 22, 2022 Author Share Posted May 22, 2022 4 hours ago, hinduguy said: 21.05.2022 19:55:08 INFO: Creating gdrive mergerfs mount. mv: cannot move '/mnt/user/mount_mergerfs/gdrive' to '/mnt/user/local/gdrive/gdrive': File exists fuse: mountpoint is not empty Manually move whatever files are already in the mountpoint and then run the script again Quote Link to comment
hinduguy Posted May 22, 2022 Share Posted May 22, 2022 5 hours ago, DZMM said: Manually move whatever files are already in the mountpoint and then run the script again Thank you that did the trick, Thank you @DZMM for putting this guide and script together Quote Link to comment
DZMM Posted May 26, 2022 Author Share Posted May 26, 2022 Did anyone else get the new email about the Workspace migration and G Suite legacy continuing to be free? I'm so happy as it was going to cost me a small fortune as my family all use and a massive amount of time to help them all move!! Quote Link to comment
DZMM Posted May 26, 2022 Author Share Posted May 26, 2022 6 minutes ago, DZMM said: Did anyone else get the new email about the Workspace migration and G Suite legacy continuing to be free? I'm so happy as it was going to cost me a small fortune as my family all use and a massive amount of time to help them all move!! my other account that has my storage on and just 1 user got moved to Enterprise Standard for £15.30/mth when the offer period ends which I can live with as the effort to move so much data and get everything working again will be massive: Quote Link to comment
axeman Posted May 26, 2022 Share Posted May 26, 2022 2 hours ago, DZMM said: my other account that has my storage on and just 1 user got moved to Enterprise Standard for £15.30/mth when the offer period ends which I can live with as the effort to move so much data and get everything working again will be massive: Funny I started uploading a lot recently, as I'm moving to an area that doesn't offer symetrical Gig service... One day after I started uploading, I got the email that I was moved. I thought the increase in usage triggered it. Quote Link to comment
BRiT Posted May 26, 2022 Share Posted May 26, 2022 6 hours ago, DZMM said: Did anyone else get the new email about the Workspace migration and G Suite legacy continuing to be free? I'm so happy as it was going to cost me a small fortune as my family all use and a massive amount of time to help them all move!! I migrated over to ZohoMail a few months ago. I plan on staying there despite Google continuing their free offers. Zoho's Free offering allows for upto 5 accounts. You can still send emails via SMTP. The only possible downside is the free accounts do not offer imap/pop3 services. However their Android App for mail is pretty solid and not much different from the GMAIL app. The setup guide and migration guide were easy to follow and documented everything that had to be done. The pay tiers are pretty reasonable too, plans from $1 User/Month to $4 User/Month for Mail. They also have a workplace plan that includes online apps like word processing and spreadsheets. Quote Link to comment
Roudy Posted May 26, 2022 Share Posted May 26, 2022 7 hours ago, DZMM said: my other account that has my storage on and just 1 user got moved to Enterprise Standard for £15.30/mth when the offer period ends which I can live with as the effort to move so much data and get everything working again will be massive: Did you notice they updated services summary to 5TB for 5 or more users instead of unlimited? I feel like restrictions will be coming in the future... Quote Link to comment
Roudy Posted May 26, 2022 Share Posted May 26, 2022 It also says to contact Google Support every 90 days to request more storage. Quote Link to comment
DZMM Posted May 26, 2022 Author Share Posted May 26, 2022 3 hours ago, Roudy said: It also says to contact Google Support every 90 days to request more storage. I'm trying to find that page - where do I go? Thanks Quote Link to comment
DZMM Posted May 26, 2022 Author Share Posted May 26, 2022 2 minutes ago, DZMM said: I'm trying to find that page - where do I go? Thanks Found it - good value for £15.30/pm! 2 Quote Link to comment
Roudy Posted May 27, 2022 Share Posted May 27, 2022 23 hours ago, DZMM said: Found it - good value for £15.30/pm! Wow! That's taking advantage of the service haha! I hope the unlimited sticks. If they switch to the 5TB for every user, it's going to kind of suck. May have to look at establishing a library and share the encryption keys amongst people who pay to have access. I have something like that set up for a remote server at the moment. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.