rragu Posted August 31, 2020 Share Posted August 31, 2020 On 8/29/2020 at 5:25 AM, mgutt said: 1.) Check the destination documentation of rclone. Every destination has its own special parameters and behaviour like Google Drive has its own, too: https://rclone.org/drive/ 2.) Do not copy/sync to the mount of the destination like /mnt/disks/gdrive. Use only the rclone alias like gdrive:subfolder. rclone does not work properly if the target is a local path. Example: If you use the mount path and your destination is a webdav server it does not presere the file modification time. 3.) If it does not preserve the file modifcation time it must use checksums to verify that the destination file is the same (or not) before being able to skip (or overwrite it). This means it downloads and calculates the checksums (high cpu usage) and uploads at the same time (rclone uses 8 parallel running --checkers for this). 4.) There are minor performance differences between sync and copy as they act a little bit different, but finally this should not influence the transfer speed of files: sync = deletes already existing files from destination that are not present on source copy = ignores existing files on destination Conclusion: Check if your transfer preserves the timestamps. If yes, then disable the checksum check through --ignore-checksum (if you are ok with that) and test different --checkers and --transfers (4 is the default) values until you reach the best performance. For example I use "--checkers 2 --transfers 1" for my Nextcloud destination as there was no benefit uploading multiple files and finally it does not raise the total transfer speed if I have two parallel uploads instead of one. My command as an example: rclone sync /mnt/user/Software owncube:software --create-empty-src-dirs --ignore-checksum --bwlimit 3M --checkers 2 --transfers 1 -vv --stats 10s Thanks! I lowered the checkers to 2 and transfers to 1. Combined with a chunk-size of 256M, I get the same ~80MBps with half the CPU utilization as before, even without --ignore checksum Quote Link to comment
Waseh Posted September 15, 2020 Author Share Posted September 15, 2020 (edited) Hi Guys So I’ve finally started using rclone mount again after several years with only doing weekly scripted backups. This prompted me to give the plugin a bit of an overhaul to fix some of my own long-standing nuisances with the plugin. Gone are the default scripts and the script editor (You should really use UserScripts for all your script needs). The config file can still be edited. The settings page has been overhauled and i intend to deprecate the beta branch of the plugin for this new unified model. You are now able to change branch at will in the settings page as well as update rclone if a new version is available. Current version as well as newest version will now be shown in the settings page. If you want to check it out you can install/update the plugin manually by inserting THIS URL in the plugins/install plugin settings menu. It's a drop in replacement for the stable branch. Whenever this version has proven itself without any major bugs i will merge it with the master branch and deprecate the beta branch. @Stupifier @DZMM if you get the chance please test out the new version and see if there's any glaring bugs I’ve missed. Edit: A picture to show the new settings page Edited September 15, 2020 by Waseh 2 Quote Link to comment
Stupifier Posted September 15, 2020 Share Posted September 15, 2020 @Waseh Awesome. I should be able to check this out in the next 4-6 hours....sorry busy at work. I'll uninstall the beta branch plugin first...then install using your URL...then I'll play around and report back. Quote Link to comment
Waseh Posted September 15, 2020 Author Share Posted September 15, 2020 For some reason rclone is not returning the version number for betas right now. I will add an error message instead of just a blank space in the next release. Quote Link to comment
Stupifier Posted September 16, 2020 Share Posted September 16, 2020 (edited) @Waseh Just did an install.....really basic look over. Everything looks good so far. I did NOT experience your beta version number issue though. First image shows the update to beta rclone popup. The next image shows a version check on stable branch, then another version check once flipped over to beta branch. All I've done so far is copy over my .rclone.conf file and run a few simple rclone copy and rclone lsd commands and rclone mount commands. Ran into zero issues. Edited September 16, 2020 by Stupifier Quote Link to comment
KeyBoardDabbler Posted September 17, 2020 Share Posted September 17, 2020 My logs are being filled every second with the below line. Quote Sep 15 22:37:40 HTPC-unRAID emhttpd: error: share_luks_status, 6256: Operation not supported (95): getxattr: /mnt/user/mount_mergerfs Not to sure where to go next. cheers htpc-unraid-diagnostics-20200916-1522.zip Quote Link to comment
mgutt Posted September 17, 2020 Share Posted September 17, 2020 1 hour ago, KeyBoardDabbler said: Not to sure where to go next. Which rclone command are you using? Does this help? It's related to Google drive and happens because it's possible to have the same filename two times: https://forum.rclone.org/t/rclone-mount-google-drive-shows-empty-directory-which-is-not-empty/3838/6 Quote Link to comment
KeyBoardDabbler Posted September 17, 2020 Share Posted September 17, 2020 52 minutes ago, mgutt said: Does this help? It's related to Google drive and happens because it's possible to have the same filename two times: https://forum.rclone.org/t/rclone-mount-google-drive-shows-empty-directory-which-is-not-empty/3838/6 I start to see the error once the mergerfs command finishes mounting i think, I also have been recently moving alot of files around so i will check to see if i have any dupes as per the link you provided. I will report back. thanks Quote Link to comment
DZMM Posted September 17, 2020 Share Posted September 17, 2020 @Waseh I've just updated and spotted a few things. - I'm currently on 1.53.0 and when I tried to update to 1.53.1 it didn't work and asked if rclone was running - The config editor seems better as before I couldn't use it as it wouldn't show my whole file. However, even though it shows my file I can't apply any edits as it says please shorten this text to 4096 characters or less (you are currently using 5648 characters) Quote Link to comment
Waseh Posted September 17, 2020 Author Share Posted September 17, 2020 @DZMM with the risk of sounding like a broken record, is rclone running? (Including a rclone mount 😉) I'll bump up the max chars on the editor Quote Link to comment
Stupifier Posted September 17, 2020 Share Posted September 17, 2020 I never tried or used the config editor stuff....I dunno....just seemed like it was more straight forward to edit the .rclone.config file directly or as rclone devs intend by using the 'rclone config' command in a terminal. But yes....I did not attempt to update while rclone processes were active. Verified using htop command. Given that, on Ubuntu, rclone is capable of updating seemlessly while rclone mounts are active. Don't ask me how...but I know for a fact it does. I'm sure the same applies to other modern OS's as well. Quote Link to comment
Waseh Posted September 17, 2020 Author Share Posted September 17, 2020 I mostly use the editor to add new config flags as they get released in new versions as well as easy backup. Quote Link to comment
DZMM Posted September 18, 2020 Share Posted September 18, 2020 On 9/17/2020 at 11:29 AM, Waseh said: @DZMM with the risk of sounding like a broken record, is rclone running? (Including a rclone mount 😉) I'll bump up the max chars on the editor yes it works fine. I just did a reboot and the plugin disappeared - I don't know if it was because of my failed upgrade to 1.53.1 Quote Link to comment
Waseh Posted September 18, 2020 Author Share Posted September 18, 2020 @DZMM Yea actually the update script might actually ruin the ability for plugin to survive a reboot (if no internet connection is present at boot) I'll fix that for the release. I'm pushing the update to the main branch in an hour or so it will be live for everyone Quote Link to comment
Waseh Posted September 18, 2020 Author Share Posted September 18, 2020 (edited) @DZMM Scratch that, the update script shouldn't affect survivability after a reboot. I'll investigate, could just be because the hackish way i did the test version. Edited September 18, 2020 by Waseh Quote Link to comment
Waseh Posted September 18, 2020 Author Share Posted September 18, 2020 (edited) @DZMM I can't seem to reproduce it. Oh and my question about rclone running was not a question about if it was working but related to the fact that if the rclone binary probably was busy. With that being said i have updated the way the binary is replaced so we shouldn't run into the same problem again. Edited September 18, 2020 by Waseh Quote Link to comment
Waseh Posted September 19, 2020 Author Share Posted September 19, 2020 The update is now live. If you installed the test version you should be able to update directly to the new live version by checking for plugin updates. 1 Quote Link to comment
Nickjones818 Posted September 19, 2020 Share Posted September 19, 2020 Im curious if there is any head-way (or keeping track of progress) on Bi-directional Sync functionality. I say this not even sure if its being pursued at the moment, so forgive me if i've mis-assumed. I remember a few months back reading some discussion on this functionality, but havent seen anything lately. Quote Link to comment
Waseh Posted September 20, 2020 Author Share Posted September 20, 2020 @Nickjones818 what exactly do you mean with bi-directional sync? 😄 This plugin only provides a way to install and keep rclone up to date. All functions are provided by rclone itself and the scripts of the community such as @DZMM rclone mount scripts. Quote Link to comment
diffdrumdave Posted September 20, 2020 Share Posted September 20, 2020 It does not give me the option to update. The Fix Common Problems plugin tells me that it has been deprecated and I should change to the new one. Quote Link to comment
Waseh Posted September 20, 2020 Author Share Posted September 20, 2020 (edited) That's because you are on the beta branch. As mentioned that one is getting deprecated while the old stable branch now provides the ability to install both stable and beta. So to sum up that is as expected 😄 Edited September 20, 2020 by Waseh Quote Link to comment
Waseh Posted September 20, 2020 Author Share Posted September 20, 2020 (edited) The beta branch plugin has now been updated with a migration script. Users of the beta branch will now be moved to the new plugin automatically when you check for updates and then update the plugin. @diffdrumdave Edited September 20, 2020 by Waseh Quote Link to comment
diffdrumdave Posted September 21, 2020 Share Posted September 21, 2020 Thank you. Quote Link to comment
Lilarcor Posted September 22, 2020 Share Posted September 22, 2020 For the latest version, will it re-download rclone after each rebooting ? I experienced the issue when using the version in 2019. Quote Link to comment
Waseh Posted September 22, 2020 Author Share Posted September 22, 2020 @Lilarcor If there is an Internet connection it will download the latest version on reboot. If no connection is available it will install the last fetched version. This is the intended behavior Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.