BLKMGK Posted August 13, 2017 Share Posted August 13, 2017 Been beating my head against the wall for awhile on a config issue. I've been using this plugin and rclone for months now with great success, first on ACD and now on GSuite. My commandline is as follows: rm /mnt/rclone-bak.log # remove the old backup mv /mnt/rclone.log /mnt/rclone-bak.log # create the new backup rclone sync --log-file /mnt/rclone.log --max-size=40G --copy-links --transfers=4 --exclude '/FOO/*.*' --bwlimit=10M --verbose /mnt/user/ google_crypt:/ --checkers=10 -v This removes a logfile from the previous run 2 prior, copies over the existing current logfile, then kicks off rclone in verbose mode with output piped to a file. I can tail these files to monitor status or just open them (hint: do not place them in a directory being backed up). The bolded portion however is a real problem... I have recently moved data to a new share we'll call FOO. Foo is about 60gig of 50Kb files, it's not something I require to be backed up and the tiny file sizes push rclone into spasms so my backup never completes! I'm in a 403 tarpit... How the bloody hell do I skip the FOO directory? I have tried each of the following to no result: --exclude '/FOO/*.*' --exclude "/FOO/*.*" --exclude='/FOO/*.*' --exclude '/FOO/*.*' --exclude="/FOO/*.*" --exclude "FOO/*.*" --exclude="FOO/*.*" --exclude "/mnt/user/FOO/*.*" I have probably tried a few other combinations too! I have been told that the path should be relative to the one given to rclone for backup so I think I've got this right not giving it the full path. I have tried with and without a leading slash, I have tried dual and single quotes - I'm pretty sure single is correct. For bonus points there's another share I'd ALSO like to skip but it's much less critical as it's simply a working directory. Quote Link to comment
BLKMGK Posted August 22, 2017 Share Posted August 22, 2017 Thanks for updating the plugin to the latest Rclone! Still trying to exclude a dir - argh! Quote Link to comment
TODDLT Posted August 31, 2017 Share Posted August 31, 2017 I decided to try and use RClone to backup my appdata, which contains the Plex database. A large portion of it is copying but also it looks like it doesn't like some file types and has a large number of the following errors: Any advice on why or what to do with this? I see there is an appdata backup that is part of community applications and may just try to go that route. Quote Link to comment
bobbintb Posted September 1, 2017 Share Posted September 1, 2017 16 hours ago, TODDLT said: I decided to try and use RClone to backup my appdata, which contains the Plex database. A large portion of it is copying but also it looks like it doesn't like some file types and has a large number of the following errors: Any advice on why or what to do with this? I see there is an appdata backup that is part of community applications and may just try to go that route. The Plex data must be using symlinks. Did you use -L or --copy-links like the error suggests? Quote Link to comment
TODDLT Posted September 1, 2017 Share Posted September 1, 2017 6 hours ago, bobbintb said: The Plex data must be using symlinks. Did you use -L or --copy-links like the error suggests? Not knowing what "symlinks" are or being familiar with the -L or --copy-links , I didn't want to go add swtiches into the script without checking first. I'll see if I can find explanations of the switches on the Rclone page this evening but any help with symlinks or what this really means is appreciated. Quote Link to comment
Squid Posted September 1, 2017 Share Posted September 1, 2017 (edited) 5 minutes ago, TODDLT said: Not knowing what "symlinks" are Symlink / hardlinks in English (vastly simplified) Imagine a file named "x" that requires 100Meg to store. Such and such program wants to store a copy of that file under the filename "y" Without symlinks / hardlinks, the copy would take up another 100Meg. With links, the copy doesn't take up any extra space, as the "y" filename simply points to the exact same data that "x" does. Plex makes very extensive use of them, and any backup of Plex's appdata without handling symlinks / hardlinks is pointless. Either add the switches, or use CA's appdata backup. Edited September 1, 2017 by Squid Quote Link to comment
TODDLT Posted September 1, 2017 Share Posted September 1, 2017 (edited) 4 minutes ago, Squid said: Symlink / hardlinks in English (vastly simplified) Imagine a file named "x" that requires 100Meg to store. Such and such program wants to store a copy of that file under the filename "y" Without symlinks / hardlinks, the copy would take up another 100Meg. With links, the copy doesn't take up any extra space, as the "y" filename simply points to the exact same data that "x" does. Plex makes very extensive use of them, and any backup of Plex's appdata without handling symlinks / hardlinks is pointless. Either add the switches, or use CA's appdata backup. So Plex, rather than having an extremely large database hog of a file, is saving all the image files used for the cover art etc in individual files and then referencing those locations in the database. correct?? Do I need to add both switches as they are written? or is this an either / or situation? I looked at the CA backup and if I can handle this in a single line for Rclone I'll do it. If not I'll go investigate the settings I need to learn in CA Backup. Edited September 1, 2017 by TODDLT Quote Link to comment
Squid Posted September 1, 2017 Share Posted September 1, 2017 Just now, TODDLT said: So Plex, rather than having an extremely large database hog of a file, is saving all the image files used for the cover art etc in individual files and then referencing those locations in the database. correct?? Yeah. Multiple files all referencing the exact same picture. 1 minute ago, TODDLT said: Do I need to add both switches as they are written? or is this an either / or situation? Honestly, I would assume just add the switches. (don't use rclone - I did the appdata backup plg, which may or may not suit your purposes depending upon the destination) Quote Link to comment
TODDLT Posted September 1, 2017 Share Posted September 1, 2017 9 minutes ago, Squid said: Yeah. Multiple files all referencing the exact same picture. Honestly, I would assume just add the switches. (don't use rclone - I did the appdata backup plg, which may or may not suit your purposes depending upon the destination) Thanks much Quote Link to comment
bobbintb Posted September 5, 2017 Share Posted September 5, 2017 My plugin doesn't seem to update. It keep saying there is an update, I update, and it still says there is an update. Quote Link to comment
Waseh Posted September 5, 2017 Author Share Posted September 5, 2017 I added a check for rclone running (or rather being unable to copy the new file) so that might be the reason - The update should stop with an error telling this. Quote Link to comment
GilbN Posted September 24, 2017 Share Posted September 24, 2017 I keep getting this error when I try to update the plugin plugin: updating: rclone.plg plugin: downloading: http://beta.rclone.org/rclone-beta-latest-linux-amd64.zip ... done +============================================================================== | Skipping package rclone-2016.11.19-bundle (already installed) +============================================================================== Archive: /boot/config/plugins/rclone-beta/install/rclone-beta-latest.zip creating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/ inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/rclone inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/README.txt inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/README.html inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/rclone.1 inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/git-log.txt Copy failed - rclone running plugin: run failed: /bin/bash retval: 1 Quote Link to comment
starbix Posted September 28, 2017 Share Posted September 28, 2017 I get the same error as @GilbN I'm running 6.4 rc9f Quote Link to comment
Waseh Posted September 28, 2017 Author Share Posted September 28, 2017 I should probably change that error text. What is happening is the copy command to copy the rclone file is failing. This is usually because the rclone file is being used in some process. What used to happen before i added this was that the update went through without any errors but rclone wouldnt actually be updated. 1 Quote Link to comment
GilbN Posted October 2, 2017 Share Posted October 2, 2017 On 9/28/2017 at 7:49 PM, Waseh said: I should probably change that error text. What is happening is the copy command to copy the rclone file is failing. This is usually because the rclone file is being used in some process. What used to happen before i added this was that the update went through without any errors but rclone wouldnt actually be updated. I'm an idiot. How can I stop it and update? Quote Link to comment
tooviral Posted October 14, 2017 Share Posted October 14, 2017 On 9/24/2017 at 8:43 AM, GilbN said: I keep getting this error when I try to update the plugin plugin: updating: rclone.plg plugin: downloading: http://beta.rclone.org/rclone-beta-latest-linux-amd64.zip ... done +============================================================================== | Skipping package rclone-2016.11.19-bundle (already installed) +============================================================================== Archive: /boot/config/plugins/rclone-beta/install/rclone-beta-latest.zip creating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/ inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/rclone inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/README.txt inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/README.html inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/rclone.1 inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.37-157-g5a3a56ab#U03b2-linux-amd64/git-log.txt Copy failed - rclone running plugin: run failed: /bin/bash retval: 1 Same error here. Have tried "killall -9 rclone" but dosent work. Prob because of mount points Quote Link to comment
cthrumplu Posted October 30, 2017 Share Posted October 30, 2017 On 10/14/2017 at 6:02 PM, tooviral said: Same error here. Have tried "killall -9 rclone" but dosent work. Prob because of mount points I'd been seeing this error for a while, just took a look to actually fix it. After searching processes there was also an rcloneorig process that needed to be killed before I could update. root@unraid:~# ps aux | grep rclone root 5954 0.0 0.0 9640 1852 pts/1 S+ 05:09 0:00 grep rclone root 10526 0.0 0.0 39764 16280 ? Sl Oct10 1:23 rcloneorig --config /boot/config/plugins/rclone-beta/.rclone.conf mount --max-read-ahead 1024k --allow-other securedb: /mnt/disks/vault root@unraid:~# kill -9 10526 Quote Link to comment
ajgriglak Posted November 2, 2017 Share Posted November 2, 2017 Hey guys. Looks like this docker hasn't been updated in quite a while and there hasn't been any activity in the thread. If I'm coming in fresh now, is this still a viable Unraid <> Google Drive sync tool? Is there anything better / newer? Quote Link to comment
bobbintb Posted November 2, 2017 Share Posted November 2, 2017 45 minutes ago, ajgriglak said: Hey guys. Looks like this docker hasn't been updated in quite a while and there hasn't been any activity in the thread. If I'm coming in fresh now, is this still a viable Unraid <> Google Drive sync tool? Is there anything better / newer? To be clear, it's the plugin version, not a docker. Just in case you specifically wanted that one. I don't recall but does this need frequent updating. Does the plugin just pull the latest stable release? Quote Link to comment
ajgriglak Posted November 2, 2017 Share Posted November 2, 2017 Yes, sorry - plugin, not docker. Quote Link to comment
Waseh Posted November 2, 2017 Author Share Posted November 2, 2017 The plugin always pulls the latest version. If there's a new version you can reinstall the plugin or restart the server to pull the latest. I'll probably add an update button at some point. 1 Quote Link to comment
Exos Posted November 27, 2017 Share Posted November 27, 2017 Hello. I am attempting to use rclone to backup my unraid nas to B2. I have been following the instructions here. Unfortunately, when I use rclone to create a mount point in /mnt/disks that should be linked to my B2 account, the command never creates the mount point. I have tried many ways. I execute the script with 'user scripts' plugin, running in background. The logs in 'user scripts' show no errors. However, never is there a folder created in /mnt/disks. Has anyone had this issue? Thanks. Quote Link to comment
d2dyno Posted December 18, 2017 Share Posted December 18, 2017 It seems the config editor in the unRAID webui only allows a certain amount of characters. I had a few profiles/remotes on there, and one of the passwords got cut off, thus changing the encryption key! Quote Link to comment
westonthenet Posted December 27, 2017 Share Posted December 27, 2017 I used RClone some time ago, before the issue with Amazon Cloud Drive and encrypted secrets in the code. I've just re-installed the plugin on my new build but I'm really struggling to setup RClone again. I don't know how to get my Amazon Cloud Drive client_id and client_secret or use a a third party ouath proxy as suggested in the documentation here https://rclone.org/amazonclouddrive/ as the wizard setup just worked in the past. Is there an updated setup walk-through to help novices like me set this up, or can anyone help guide me through the initial setup please? Quote Link to comment
disruptorx Posted January 20, 2018 Share Posted January 20, 2018 what do I do to make rclone survive a reboot ? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.