live4ever

Try installing google-auth-oauthlib: pip install google-auth-oauthlib

So after I upgraded my main uploading unRAID server from 6.9 series to 6.11 I noticed how permissions could mess up using the scripts from GitHub. My mount script is a bit different - I create a “backup” folder acting as sort of Dropbox for uploads to gdrive at: /mnt/user/local/gdrive_vfs/backup This folder would get deleted after rclone_upload completes and then get recreated with rclone_mount (with root:root). In DZMM rclone_mount the MountFolders bit: MountFolders=\{"downloads/complete,downloads/intermediate,downloads/seeds,movies,tv"\} # comma separated list of folders to create within the mount In DZMM rclone_upload: Command1="--exclude downloads/**" So the paths from MountFolders: /mnt/user/local/gdrive_vfs/movies /mnt/user/local/gdrive_vfs/tv will get deleted (after uploading) - then the rclone_mount script will create them again with root:root (and dockers trying to use them will give permission errors. Is there a way for the rclone_mount script to create the empty MountFolders with nobody:user permissions?

The above was on unRAID 6.10.3 - another server I have on 6.9.3 doesn't have "dig" (not in nerd tools either) so the GES script won't run (in case anyone tries on the 6.9.x release). *********************************************** ******** Google Endpoint Scanner (GES) ******** *********************************************** [GES] [INFO] The application started on Thu Jul 7 10:22:21 EDT 2022. [GES] [WARNING] Hosts backup file not found. Backing it up. [GES] [ERROR] The command 'dig' is not installed or not reachable in this user's $PATH. *********************************************** * Finished Google Endpoint Scanner (GES) * Message: Install dig or make sure its executable is reachable, then try again. *********************************************** Script Finished Jul 07, 2022 10:22.21

Some test from Canada this morning: ******** Google Endpoint Scanner (GES) ******** *********************************************** [GES] [INFO] The application started on Thu Jul 7 10:02:04 EDT 2022. [GES] [WARNING] Hosts backup file not found. Backing it up. [GES] [WARNING] Did not find a permanent blacklist at /root/blacklisted_google_ips. Creating a new one. [GES] [INFO] Please wait. Downloading the test file from 142.251.41.42... [GES] [INFO] Parsing connection with 142.251.41.42. [GES] [INFO] 1.538 MiB/s. Below criterion endpoint. Blacklisting IP '142.251.41.42'. [GES] [INFO] Please wait. Downloading the test file from 142.251.32.74... [GES] [INFO] Parsing connection with 142.251.32.74. [GES] [INFO] 8.334 MiB/s. Above criterion endpoint. Whitelisting IP '142.251.32.74'. [GES] [INFO] Please wait. Downloading the test file from 172.217.1.10... [GES] [INFO] Parsing connection with 172.217.1.10. [GES] [INFO] 1.564 MiB/s. Below criterion endpoint. Blacklisting IP '172.217.1.10'. [GES] [INFO] Please wait. Downloading the test file from 172.217.165.10... [GES] [INFO] Parsing connection with 172.217.165.10. [GES] [INFO] 1.545 MiB/s. Below criterion endpoint. Blacklisting IP '172.217.165.10'. [GES] [INFO] Please wait. Downloading the test file from 142.251.41.74... [GES] [INFO] Parsing connection with 142.251.41.74. [GES] [INFO] 10.32 MiB/s. Above criterion endpoint. Whitelisting IP '142.251.41.74'. [GES] [INFO] Please wait. Downloading the test file from 142.251.33.170... [GES] [INFO] Parsing connection with 142.251.33.170. [GES] [INFO] 1.566 MiB/s. Below criterion endpoint. Blacklisting IP '142.251.33.170'. [GES] [INFO] The fastest IP is 142.251.41.74. Putting into the hosts file. *********************************************** * Finished Google Endpoint Scanner (GES) * Message: Reached EOF without errors *********************************************** Script Finished Jul 07, 2022 10:04.31 Full logs for this script are available at /tmp/user.scripts/tmpScripts/Gdrive_api/log.txt Script Starting Jul 07, 2022 10:11.25 Full logs for this script are available at /tmp/user.scripts/tmpScripts/Gdrive_api/log.txt *********************************************** ******** Google Endpoint Scanner (GES) ******** *********************************************** [GES] [INFO] The application started on Thu Jul 7 10:11:25 EDT 2022. [GES] [INFO] Hosts backup file found. Restoring it. [GES] [INFO] Found permanent blacklist. Parsing it. [GES] [INFO] Please wait. Downloading the test file from 142.251.32.74... [GES] [INFO] Parsing connection with 142.251.32.74. [GES] [INFO] 33.19 MiB/s. Above criterion endpoint. Whitelisting IP '142.251.32.74'. [GES] [INFO] Please wait. Downloading the test file from 142.251.41.74... [GES] [INFO] Parsing connection with 142.251.41.74. [GES] [INFO] 39.40 MiB/s. Above criterion endpoint. Whitelisting IP '142.251.41.74'. [GES] [INFO] The fastest IP is 142.251.41.74. Putting into the hosts file. *********************************************** * Finished Google Endpoint Scanner (GES) * Message: Reached EOF without errors *********************************************** Script Finished Jul 07, 2022 10:11.39 So initially it was the 50MB file then the 200MB file (at 10:11.25) - been having some buffering the past couple weeks.

Scroll down to the bottom of page “On a different network” here: https://support.plex.tv/articles/200288586-installation/

I’ve got two unRAIDs: 6.9 b22 47d uptime 6.9 b35 33d uptime That have been still working fine here.

@M1kep_ Thanks, the command: root@Tower:~# find /mnt/user/appdata/other/rclone/remotes -name mount_running* /mnt/user/appdata/other/rclone/remotes/gdrive_media_vfs/mount_running find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete allowed the rclone_mount script to start and download/install the mergerfs software.

@DZMM After a server reboot I can’t seem to get the the rclone_mount script (same one in my last post) to work all I’m seeing is: ————- Script Starting Nov 14, 2020 23:51.50 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount/log.txt 14.11.2020 23:51:50 INFO: Creating local folders. 14.11.2020 23:51:50 INFO: *** Starting mount of remote gdrive_media_vfs 14.11.2020 23:51:50 INFO: Checking if this script is already running. 14.11.2020 23:51:50 INFO: Exiting script as already running. Script Finished Nov 14, 2020 23:51.50 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount/log.txt _______ Thanks

@DZMMI do it after the mount script is up and running - and I did another test last night created two directories with the following paths: /mnt/user/mount_mergerfs/gdrive_media_vfs/test/video.mkv /mnt/user/mount_mergerfs/gdrive_media_vfs/test2/ The directory ../test/video.mkv (with a file within it) was there in this morning while the ../test2/ directory (without any sub-folders or files) was deleted/removed. Not a big deal really though, was just a little concerned that (empty) folders I created in mergerfs were disappearing. Also, I noticed that the mount script created shares/folders only on /mnt/disk14 and /mnt/disk15 (the least full disks of my array) - can I manually create the: /mnt/local/gdrive_media_vfs share on disks1-13? Or how do I get the mergerfs share to appear on all disks (to prevent moving files between disks before upload)? Thanks again

@DZMM had to reboot unRAID server to get it to clear up - unmounting with fusermount wasn't enough. Quick question, does the mount script have to be used to create the subfolders within the mergerfs folder? For example my mount script does: MountFolders=\{"downloads/complete,downloads/incomplete,backup"\} and if I manually create a folder like /mnt/user/mount_mergerfs/gdrive_media_vfs/test/video.mkv the /test/video.mkv disappears (I guess when the mount script runs after the next 10min?) Thanks again

Using rclone 1.53.1 Here's the rclone_mount script: #!/bin/bash ###################### #### Mount Script #### ###################### ### Version 0.96.8 ### ###################### ####### EDIT ONLY THESE SETTINGS ####### # INSTRUCTIONS # 1. Change the name of the rclone remote and shares to match your setup # 2. NOTE: enter RcloneRemoteName WITHOUT ':' # 3. Optional: include custom command and bind mount settings # 4. Optional: include extra folders in mergerfs mount # REQUIRED SETTINGS RcloneRemoteName="gdrive_media_vfs" RcloneMountShare="/mnt/user/mount_rclone" LocalFilesShare="/mnt/user/local" MergerfsMountShare="/mnt/user/mount_mergerfs" DockerStart="radarr sonarr" MountFolders=\{"downloads/complete,downloads/incomplete,backup"\} # OPTIONAL SETTINGS # Add extra paths to mergerfs mount in addition to LocalFilesShare LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder. Enter 'ignore' to disable LocalFilesShare3="" LocalFilesShare4="" LocalFilesShare5="" LocalFilesShare6="" # Add extra commands or filters Command1="--rc" Command2="" Command3="" Command4="" Command5="" Command6="" Command7="" Command8="" CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address NetworkAdapter="eth0" # choose your network adapter. eth0 recommended VirtualIPNumber="2" # creates eth0:x e.g. eth0:1. I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them ####### END SETTINGS ####### ############################################################################### ##### DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING ####### ############################################################################### ####### Preparing mount location variables ####### LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" ####### create directories for rclone mount and mergerfs mounts ####### mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName #for script files mkdir -p /mnt/user0/mount_rclone/cache/$RcloneRemoteName #for cache files if [[ $LocalFileShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested." else echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders." eval mkdir -p $LocalFilesLocation/"$MountFolders" fi mkdir -p $RcloneMountLocation mkdir -p $MergerFSMountLocation ####### Check if script is already running ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}" echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running." if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running." exit else echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding." touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running fi ####### Create Rclone Mount ####### # Check If Rclone Mount Already Created if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote." # Creating mountcheck file in case it doesn't already exist echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote." touch mountcheck rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse # Check bind option if [[ $CreateBindMount == 'Y' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" else echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}" ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber fi echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}" else RCloneMountIP="" echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}" fi # create rclone mount rclone mount \ --allow-other \ --dir-cache-time 720h \ --log-level INFO \ --poll-interval 15s \ --cache-dir=/mnt/user0/mount_rclone/cache/$RcloneRemoteName \ --vfs-cache-mode full \ --vfs-cache-max-size 100G \ --vfs-cache-max-age 336h \ --bind=$RCloneMountIP \ $RcloneRemoteName: $RcloneMountLocation & # Check if Mount Successful echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds" # slight pause to give mount time to finalise sleep 10 echo "$(date "+%d.%m.%Y %T") INFO: continuing..." if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems." docker stop $DockerStart find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi ####### Start MergerFS Mount ####### if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested." else if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place." else # check if mergerfs already installed if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount" else # Build mergerfs binary echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now." mkdir -p /mnt/user/appdata/other/rclone/mergerfs docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin # check if mergerfs install successful echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds" sleep 10 if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount." else echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully. Please check for errors. Exiting." docker stop $DockerStart find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi # Create mergerfs mount echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount." # Extra Mergerfs folders if [[ $LocalFilesShare2 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare2=":$LocalFilesShare2" else LocalFilesShare2="" fi if [[ $LocalFilesShare3 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare3=":$LocalFilesShare3" else LocalFilesShare3="" fi if [[ $LocalFilesShare4 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare4=":$LocalFilesShare4" else LocalFilesShare4="" fi if [[ $LocalFilesShare5 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare5} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare5=":$LocalFilesShare5" else LocalFilesShare5="" fi if [[ $LocalFilesShare6 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare6} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare6=":$LocalFilesShare6" else LocalFilesShare6="" fi # mergerfs mount command mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4$LocalFilesShare5$LocalFilesShare6 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true # check if mergerfs mount successful echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created." if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed." docker stop $DockerStart find /mnt/user/appdata/other/rclone/remotes -name mount_running* -delete rm /mnt/user/appdata/other/scripts/running/fast_check exit fi fi fi ####### Starting Dockers That Need Mergerfs Mount To Work Properly ####### echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start $DockerStart rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running echo "$(date "+%d.%m.%Y %T") INFO: ${RcloneRemoteName} Script complete" exit Here is rclone.conf [gdrive] type = drive client_id = xxxx client_secret = xxxx scope = drive token = {"access_token":"xxxxx","token_type":"Bearer","refresh_token":"xxxx","expiry":"2020-10-23T11:50:00.72593565-04:00"} [gdrive_media_vfs] type = crypt remote = gdrive:crypt filename_encryption = standard directory_name_encryption = true password = xxxx password2 = xxxx

Just set this up using the new mount script (posted a few pages back by @DZMM) seems to working and uploading fine. Created two folders (via mount script backup and downloads) in gdrive_media_vfs crypt remote, and question is should I be seeing all these other folders in /mnt/user/mount_mergerfs/gdrive_media_vfs (attached images)? Thanks

I can't seem to get the ADDITIONAL_PORTS variable to work. Tried with Telly port 6077 and ChannelsDVR port 8089 Neither of these are accessible at unRAID.lan.ip:6077 or 8089

Any updates to this? Still getting: docker exec -it -u abc tvheadend /usr/bin/tv_grab_zz_sdjson_sqlite --configure Can't locate LWP/UserAgent/Determined.pm in @INC (you may need to install the LWP::UserAgent::Determined module) (@INC contains: /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl) at /usr/bin/tv_grab_zz_sdjson_sqlite line 4927. BEGIN failed--compilation aborted at /usr/bin/tv_grab_zz_sdjson_sqlite line 4927. Using the newest update from the weekend (4.2.7-34~g036b9cbab). Thanks again.

Is anyone using the sqlite SchedulesDirect grabber? Linux 4.14.49-unRAID. Last login: Mon Nov 5 07:26:05 -0500 2018 on /dev/pts/3. root@Tower:~# docker exec -it -u abc tvheadend /usr/bin/tv_grab_zz_sdjson_sqlite --configure Can't locate LWP/UserAgent/Determined.pm in @INC (you may need to install the LWP::UserAgent::Determined module) (@INC contains: /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl) at /usr/bin/tv_grab_zz_sdjson_sqlite line 4927. BEGIN failed--compilation aborted at /usr/bin/tv_grab_zz_sdjson_sqlite line 4927. Maybe the docker image doesn't have the perl bits in it? Thanks for all the ls-io dockers!

You just need to fill in something for the "Container Path:" field . . . like "/movies"

Thanks for posting this (disable fast startup). My Win10 VM was hanging with a black screen & cursor on first start after shutdown, thought it might be trying to boot off the install iso but it was still doing it after detaching from the VM. Thanks

How can I add two new disks to my already existing shares? I have disk6 and disk7 but they don't show in the menu to be included in the previously created shares. They also can't be used in any new shares I want to create.

I'm running unRAID version 5.0.4 and just installed unMENU from the google code page. I can access the main unMENU page on 8080 but all of the menu links at the go to "page not found". Thanks for your help. edit: never mind saw you have to access via name and not IP http://code.google.com/p/unraid-unmenu/issues/detail?id=3