May 14, 20206 yr 20 minutes ago, DZMM said: Yes it does if done correctly. If you're making changes to the mount outside of rclone that will be the source of your problem. Or, your settings aren't right - best advice I can give is use my scripts. Sorry I think I may not have explained my situation properly. I've not been using mergerfs. And without mergerfs, the partial scan doesn't work correct? And now I'm intending to use mergerfs to solve this problem. However, I don't have write access to the drive. It's a shared drive and I only have read access. If the owner makes changes directly to the gdrive, like adding folders etc, will the changes be reflected in the mergerfs folder? So basically I'm not going to be having a local folder that uploads to the drive. I just want to use mergerfs for the updates if that makes sense. I wish to use your scripts except for the local folder part where it uploads to the drive. Will it still work or have I understood this wrongly? Edited May 14, 20206 yr by blizz
May 14, 20206 yr On my latest reboot I get this from the second mountscript: And the mergerfs directory is empty. Script Starting May 14, 2020 11:30.01 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount_sh/log.txt 14.05.2020 11:30:01 INFO: Creating local folders. 14.05.2020 11:30:01 INFO: *** Starting mount of remote googleSh_crypt 14.05.2020 11:30:01 INFO: Checking if this script is already running. 14.05.2020 11:30:01 INFO: Script not running - proceeding. 14.05.2020 11:30:01 INFO: Mount not running. Will now mount googleSh_crypt remote. 14.05.2020 11:30:01 INFO: Recreating mountcheck file for googleSh_crypt remote. 2020/05/14 11:30:01 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "mountcheck" "googleSh_crypt:" "-vv" "--no-traverse"] 2020/05/14 11:30:01 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf" 2020/05/14 11:30:03 DEBUG : mountcheck: Modification times differ by -15m0.133405247s: 2020-05-14 11:30:01.832405247 +0200 CEST, 2020-05-14 09:15:01.699 +0000 UTC 2020/05/14 11:30:05 INFO : mountcheck: Copied (replaced existing) 2020/05/14 11:30:05 INFO : Transferred: 32 / 32 Bytes, 100%, 19 Bytes/s, ETA 0s Transferred: 1 / 1, 100% Elapsed time: 1.6s 2020/05/14 11:30:05 DEBUG : 11 go routines active 2020/05/14 11:30:05 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "mountcheck" "googleSh_crypt:" "-vv" "--no-traverse"] 14.05.2020 11:30:05 INFO: *** Creating mount for remote googleSh_crypt 14.05.2020 11:30:05 INFO: sleeping for 5 seconds 14.05.2020 11:30:10 INFO: continuing... 14.05.2020 11:30:10 INFO: Successful mount of googleSh_crypt mount. 14.05.2020 11:30:10 INFO: Mergerfs already installed, proceeding to create mergerfs mount 14.05.2020 11:30:10 INFO: Creating googleSh_crypt mergerfs mount. 14.05.2020 11:30:10 INFO: Checking if googleSh_crypt mergerfs mount created. 14.05.2020 11:30:10 INFO: Check successful, googleSh_crypt mergerfs mount created. 14.05.2020 11:30:10 INFO: Starting dockers. sonarr 14.05.2020 11:30:10 INFO: Script complete Script Finished May 14, 2020 11:30.10 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount_sh/log.txt 14.05.2020 11:34:57 INFO: Creating local folders. 14.05.2020 11:34:57 INFO: *** Starting mount of remote googleSh_crypt 14.05.2020 11:34:57 INFO: Checking if this script is already running. 14.05.2020 11:34:57 INFO: Script not running - proceeding. 14.05.2020 11:34:57 INFO: Mount not running. Will now mount googleSh_crypt remote. 14.05.2020 11:34:57 INFO: Recreating mountcheck file for googleSh_crypt remote. 2020/05/14 11:34:57 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "mountcheck" "googleSh_crypt:" "-vv" "--no-traverse"] 2020/05/14 11:34:57 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf" 2020/05/14 11:34:58 DEBUG : mountcheck: Modification times differ by -4m55.337592306s: 2020-05-14 11:34:57.169592306 +0200 CEST, 2020-05-14 09:30:01.832 +0000 UTC 2020/05/14 11:35:00 INFO : mountcheck: Copied (replaced existing) 2020/05/14 11:35:00 INFO : Transferred: 32 / 32 Bytes, 100%, 20 Bytes/s, ETA 0s Transferred: 1 / 1, 100% Elapsed time: 1.5s 2020/05/14 11:35:00 DEBUG : 11 go routines active 2020/05/14 11:35:00 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "mountcheck" "googleSh_crypt:" "-vv" "--no-traverse"] 14.05.2020 11:35:00 INFO: *** Creating mount for remote googleSh_crypt 14.05.2020 11:35:00 INFO: sleeping for 5 seconds 2020/05/14 11:35:01 Fatal error: Directory is not empty: /mnt/user/mount_rclone/googleSh_crypt If you want to mount it anyway use: --allow-non-empty option 14.05.2020 11:35:05 INFO: continuing... 14.05.2020 11:35:05 INFO: Successful mount of googleSh_crypt mount. 14.05.2020 11:35:05 INFO: Check successful, googleSh_crypt mergerfs mount in place. 14.05.2020 11:35:05 INFO: dockers already started. 14.05.2020 11:35:05 INFO: Script complete Script Finished May 14, 2020 11:35.05 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount_sh/log.txt Edited May 14, 20206 yr by Bjur
May 14, 20206 yr it places a mountcheck file in /mnt/user/rclone_mount/googleSh_crypt and if I delete it and run the script again the same error occurs.
May 14, 20206 yr Author 4 hours ago, blizz said: Sorry I think I may not have explained my situation properly. I've not been using mergerfs. And without mergerfs, the partial scan doesn't work correct? And now I'm intending to use mergerfs to solve this problem. However, I don't have write access to the drive. It's a shared drive and I only have read access. If the owner makes changes directly to the gdrive, like adding folders etc, will the changes be reflected in the mergerfs folder? So basically I'm not going to be having a local folder that uploads to the drive. I just want to use mergerfs for the updates if that makes sense. I wish to use your scripts except for the local folder part where it uploads to the drive. Will it still work or have I understood this wrongly? Mergerfs just shows what rclone knows about. If changes are made direct to gdrive, rclone won't know about these until it polls the remote to see if there are new files - that's why your scans are 'failing'. If you're mounting a remote where changes are made outside of the rclone i.e. your mate is loading files and not rclone, then you need your mate to install plex autoscan or something similar, so that it tells plex/rclone to re-scan the changed directory.
May 14, 20206 yr Author 3 hours ago, Bjur said: 2020/05/14 11:35:01 Fatal error: Directory is not empty: /mnt/user/mount_rclone/googleSh_crypt If you want to mount it anyway use: --allow-non-empty option there's something in /mnt/user/mount_rclone/googleSh_crypt prior to the mount that you need to check and delete if appropriate.
May 14, 20206 yr 6 minutes ago, DZMM said: Mergerfs just shows what rclone knows about. If changes are made direct to gdrive, rclone won't know about these until it polls the remote to see if there are new files - that's why your scans are 'failing'. If you're mounting a remote where changes are made outside of the rclone i.e. your mate is loading files and not rclone, then you need your mate to install plex autoscan or something similar, so that it tells plex/rclone to re-scan the changed directory. Hey mate! Thanks for the reply again. Rclone does work properly ; my poll interval is set to 5m so the changes show up in the rclone folder. It was just that Plex was unable to see the changes unless I did a manual scan. And I just set up mergerfs and it works! The changes get reflected in the mergerfs folder, even when it is added directly to the remote by the owner. The problem now is still Plex is not seeing it with partial scan, even though it is in mergerfs. I gather from your last reply that I might need to install the plex autoscan plugin to solve this. My only doubt is why even though the file appears in mergerfs, which is "local", it is not appearing in Plex when it should. By right, for local files you don't need plex autoscan, correct?
May 14, 20206 yr 16 minutes ago, DZMM said: there's something in /mnt/user/mount_rclone/googleSh_crypt prior to the mount that you need to check and delete if appropriate. Thanks. If using Krusader nothing is there even when enabling show hidden files. When browsing in shell it showed mountcheck file. Even when deleting it wouldn't mount correctly and show folders/mountcheck in mergerfs. I think I got it working now, but will report back if not. Thanks.
May 14, 20206 yr @watchmeexplode5 Would it make more sense to me to make a Download folder to /local/download instead of /local/REMOTE/downloads. Limit the local folder share to only 1 disk and the completed folders as: /local/googleFi_crypt/movies or /local/googleSh_crypt/series It should then be easy to move the completed files quickly and still have a separate download folder before the remotes. Does that makes sense?
May 14, 20206 yr @Bjur Shouldn't be any issue to have a single download and that location is fine. In your case with the two mounts, I think it makes sense to have a single download like you listed. The scripts (if you leave it default) will auto create /local/REMOTE/downloads folder but feel free to ignore it or edit it to something like this: googleFi_crypt: MountFolders=\{"movies"\} # comma separated list of folders to create within the mount googleSh_crypt: MountFolders=\{"series"\} # comma separated list of folders to create within the mount Edited May 14, 20206 yr by watchmeexplode5
May 14, 20206 yr Author 4 hours ago, blizz said: My only doubt is why even though the file appears in mergerfs, which is "local", it is not appearing in Plex when it should. By right, for local files you don't need plex autoscan, correct? Not sure how Plex scans for changes in a directory. My setup has sonarr/radarr etc telling plex something has changed which works perfectly.
May 14, 20206 yr Author 3 hours ago, Bjur said: Thanks. If using Krusader nothing is there even when enabling show hidden files. When browsing in shell it showed mountcheck file. Even when deleting it wouldn't mount correctly and show folders/mountcheck in mergerfs. I think I got it working now, but will report back if not. Thanks. Krusader has caused problems for other people - it needs restarting after a new mount. I get confused with your filepaths, but if the mount isn't working then the mountcheck file shouldn't be there and needs deleting...the mountcheck file should be on the remote and it's presence is used to check the mount has completed successfully.
May 15, 20206 yr hey @DZMM finally got around to testing out your script, but I seem to be hitting a wall. I've gotten the script to run successfully, and it seems at first glance to be working. Problem is, as soon as I try to touch any of the data - aka mount anything, move anything, copy anything, etc - the mouths disconnect and are going away. Any tips on where I can start to look to debug? I know not a lot of detail, hoping to start with debug so I can see what I can find first before bogging you down with lots of info to troubleshoot. I'll add an edit - I thought it might be relative to the whole mapping scenario you highlighted, but another issue I've noticed before I could even get to that - after a short while the mounts actually disappear. So whether I do anything or not, they seem to disappear after a short time as well. I'm really scratching my head. Edited May 15, 20206 yr by 1activegeek
May 15, 20206 yr Author @1activegeek post your rclone config and your mount script please. When you say the mount goes missing, where do you mean? In dockers, file explorer, shell etc?
May 15, 20206 yr @watchmeexplode5: Thanks. So I think I will create a manual download folder in /mnt/local/Downloads /mnt/local/Downloads/temp /mnt/local/Downloads/finished and then let Filebot move them to: /mnt/local/googleFi_crypt /mnt/local/googleSh_crypt What will happen when you move files to local folder while an upload is running. Will that be fine or will it mess the upload up?
May 15, 20206 yr @Bjur Files will only be upload once the min age time (set in upload script) has been hit. Default is 15 minutes. So no issue moving files into the upload location while upload is running.
May 15, 20206 yr Okay have you tried it. Just read earlier today on Rclone it was not a good idea, so I just want to make sure before I try it out. I don't won't to mess with the upload I'm running since it will take a long time to finish. Thanks for the help. Edited May 15, 20206 yr by Bjur
May 15, 20206 yr Author 15 minutes ago, Bjur said: Okay have you tried it. Just read earlier today on Rclone it was not a good idea, so I just want to make sure before I try it out. I don't won't to mess with the upload I'm running since it will take a long time to finish. Thanks for the help. I think you're confusing uploading files directly via the mount (which we don't) which is bad if you don't know what you are doing , and via rclone upload (which we do) which is 100% safe. If it didn't work, this whole 69 page thread would be a waste of time! Edited May 15, 20206 yr by DZMM
May 15, 20206 yr No I have my files in /mnt/local/remote. I'm running the upload script. While it is running I want to add new files to my /mnt/local/remote while it is running
May 15, 20206 yr Author 44 minutes ago, Bjur said: Will that work. Yes, otherwise this whole thread would be pointless.
May 15, 20206 yr 16 hours ago, DZMM said: @1activegeek post your rclone config and your mount script please. When you say the mount goes missing, where do you mean? In dockers, file explorer, shell etc? [1a_mediaz] type = drive scope = drive team_drive = ~~REDACTED~~ service_account_file = /boot/config/plugins/rclone-beta/sa/1.json server_side_across_configs = TRUE Sorry for the lack of details. So I mean as in after I mount it, I can LS and see the contents. A few moments will pass, then if I try and LS on the directory - the folders are gone and hence the "disappear". rclone_mount.sh
May 15, 20206 yr Author 40 minutes ago, 1activegeek said: [1a_mediaz] type = drive scope = drive team_drive = ~~REDACTED~~ service_account_file = /boot/config/plugins/rclone-beta/sa/1.json server_side_across_configs = TRUE Sorry for the lack of details. So I mean as in after I mount it, I can LS and see the contents. A few moments will pass, then if I try and LS on the directory - the folders are gone and hence the "disappear". rclone_mount.sh 9.52 kB · 1 download 1a_mediaz is where your files are, unencrypted?
May 16, 20206 yr 1 hour ago, DZMM said: 1a_mediaz is where your files are, unencrypted? Yes, correct. That's just one mount, I have some others I'll work on and/or work on adjusting to point at crypt volume - but for now yes that's the example I'm working with to make sure its working as expected first. Did I miss something about this being reliant on crypt mount? I figured it should work either way as long as the destination is my intended destination.
May 16, 20206 yr Ok, so after some consolidated and focused testing - I think I'm ok. I think I had a mixup of signals, way too much chaos at work, and extreme late nights trying to play with tech and not remembering what I did. The short of my issue I believe - and to help in case others do similar - is that I was switching between rclone variants, and testing the new shortcuts function, and shifting between running the script in the UR UI vs running in background. The initial tests I think had an issue because of the rclone vs rclone-beta because I was testing the new shortcuts. Then I tried shifting to the beta, and tried simply running the script - a few times I left the window open, but after time I eventually closed the window. This is what I believe led to the "miraculous disappearance" scenario I was left with. I had originally been doing it with the background, but when I hit an issue initially, it was annoying finding the process to kill so I just ran it via the GUI And left it there. That switch is where I believe I messed it up. Right now, I've got it up and running using rclone-beta with shortcuts enabled (that 1a_mediaz is a pool from multiple TD locations under one mount). Next I'll venture into the territory with the merger_fs stuff, but at the moment that's not a requirement for me. I just want to be sure I've got a solid mount/unmount for my core rclone location. I'll adjust the script for my need to add other mounts beyond the one. This is awesome, thank you for the work and effort. Sorry for the firedrill over nothing!!
May 16, 20206 yr Question: Every time the scripts is run I get the following error on multiple Movies/TV Quote Duplicate directory found in destination - ignoring I'm not even trying to upload the one listed but somehow they show on the log. Why is it doing this?
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.