[Plugin] rclone

[Waseh]

@testdasi
You are completely right. Pushed an update with a fix.
I didn't even notice that i had hard coded paths for each branch in the rclone wrapper since the very first version 😅

Edited October 23, 2019 by Waseh

[mbc0]

That update is not working on my rc3? 

 

plugin: updating: rclone.plg

+==============================================================================
| Skipping package rclone-2019.10.13-bundle (already installed)
+==============================================================================

Downloading rclone
Downloading certs
Archive: /boot/config/plugins/rclone/install/rclone-current.zip
creating: /boot/config/plugins/rclone/install/rclone-v1.49.5-linux-amd64/
inflating: /boot/config/plugins/rclone/install/rclone-v1.49.5-linux-amd64/README.html
inflating: /boot/config/plugins/rclone/install/rclone-v1.49.5-linux-amd64/rclone.1
extracting: /boot/config/plugins/rclone/install/rclone-v1.49.5-linux-amd64/git-log.txt
inflating: /boot/config/plugins/rclone/install/rclone-v1.49.5-linux-amd64/README.txt
inflating: /boot/config/plugins/rclone/install/rclone-v1.49.5-linux-amd64/rclone
Copy failed - is rclone running?
plugin: run failed: /bin/bash retval: 1

 

[testdasi]

14 minutes ago, mbc0 said:

That update is not working on my rc3? 

...

Copy failed - is rclone running?
...

 

Sounds like a clue?

...
Copy failed - is rclone running?
...

You need to unmount everything (and kill all rclone processes if there's any still running) before you can update the plugin. I tend to do it after a restart.

[mbc0]

ah, ok, normally I just click update for all my plugins and they just do the biz! 

 

Do you know if this update resolves the nonempty issue I am having?

 

[Waseh]

It doesn't. Your problem will only be solved if rclone gets patched by the maintainer or if you start mounting to an empty folder

[mbc0]

1 hour ago, Waseh said:

It doesn't. Your problem will only be solved if rclone gets patched by the maintainer or if you start mounting to an empty folder

ok, thank you

[adoucette]

I See that rclone now offers a web GUI https://rclone.org/gui/

Is it possible to use this with the Unraid rclone plugin? I have tried installing the rclone plugin on Unraid, opening the command prompt from the Unraid web UI, running the command at the rclone page above. Here's the output:

Linux 4.19.56-Unraid.
Last login: Wed Oct 23 18:41:04 -0400 2019 on /dev/pts/0.
root@SERVER:~# rclone rcd --rc-web-gui --rc-web-gui-update
2019/10/23 18:45:58 NOTICE: A new release for gui is present at https://github.com/rclone/rclone-webui-react/releases/download/v0.0.8/currentbuild.zip
2019/10/23 18:45:58 NOTICE: Downloading webgui binary. Please wait. [Size: 4646512, Path :  /root/.cache/rclone/webgui/v0.0.8.zip]
2019/10/23 18:45:59 NOTICE: Unzipping
2019/10/23 18:45:59 NOTICE: Serving remote control on http://127.0.0.1:5572/

Then, if I point a browser to that port at my Unraid server's IP, I get a page not found error.

[testdasi]

2 minutes ago, adoucette said:

I See that rclone now offers a web GUI https://rclone.org/gui/

Is it possible to use this with the Unraid rclone plugin? I have tried installing the rclone plugin on Unraid, opening the command prompt from the Unraid web UI, running the command at the rclone page above. Here's the output:

Linux 4.19.56-Unraid.
Last login: Wed Oct 23 18:41:04 -0400 2019 on /dev/pts/0.
root@SERVER:~# rclone rcd --rc-web-gui --rc-web-gui-update
2019/10/23 18:45:58 NOTICE: A new release for gui is present at https://github.com/rclone/rclone-webui-react/releases/download/v0.0.8/currentbuild.zip
2019/10/23 18:45:58 NOTICE: Downloading webgui binary. Please wait. [Size: 4646512, Path :  /root/.cache/rclone/webgui/v0.0.8.zip]
2019/10/23 18:45:59 NOTICE: Unzipping
2019/10/23 18:45:59 NOTICE: Serving remote control on http://127.0.0.1:5572/

Then, if I point a browser to that port at my Unraid server's IP, I get a page not found error.

Go back on the previous page for the script from nuhll.

[ur6969]

I have been able to make Rclone work with B2 and am using encryption.  Before I start to upload my backups I wanted to check and see if there was a way to download the encrypted files and decrypt.

 

For instance, if my Unraid server was destroyed I would like to be able to download the files from B2 and then decrypt them.  Is this possible using Windows or OSX?  I obviously have the encryption key and salt saved in multiple places.

 

[adoucette]

2 hours ago, testdasi said:

Go back on the previous page for the script from nuhll.

Thank you.

To make it easier for others, here is the post testdasi was referring to:

 

To run a web gui of rclone on unraid, here is the command to run (after the rclone plugin is installed, of course):

rclone rcd --rc-web-gui --rc-user=test --rc-pass=test --rc-addr 192.168.1.10:12345

Replace "192.168.1.10" with your Unraid server's local IP. Replace "12345" with the [unused] port you would like the rclone server to use, replace the rc-user and rc-pass variables with the usernames and passwords you want to use.

Then point your web browser to the address you specified and log in with the credentials you specified. Nice.

Edited October 24, 2019 by adoucette

[NewDisplayName]

Thanks for this plugin, i saw a few updates over the last weeks... sadly there is no changelog and the name of the update doesnt seem to belong with the official naming... so, i would ask if you could give us a changelog? (or make the naming the same like original?)

 

e.g. which version belong to 2019.10.23? 

v1.49.5 - 2019-10-05?

 

Edited October 24, 2019 by nuhll

[Waseh]

There is a changelog both present in unraid webgui and on GitHub embedded in the .plg, so I'm not sure how you missed it.

 

Edited October 24, 2019 by Waseh

[NewDisplayName]

Sorry, i cant see it. Is that the correct way to view the changelog?

 

[Waseh]

That would be the place.

Mine looks like this.

[NewDisplayName]

Hmm, any idea why that wont work? I just checked other plugins, some work, some not, where does that data get pulled from? Maybe its piholed? (but i dont see any blocks, so no idea)

Edited October 24, 2019 by nuhll

[NewDisplayName]

Okay, i found something out while looking at the source code.

The text you showed is there, but its invisible...??? If i click and drag, i can see it... using latest google chrome..  wtf

 

the code seems to be the same also, (same css, same layout "ul" "li" "h2" and so on...??)

 

I can see... 

<!DOCTYPE HTML>

<html lang="en">

<head>

<meta name="robots" content="noindex, nofollow">

<meta http-equiv="Content-Security-Policy" content="block-all-mixed-content">

<link type="text/css" rel="stylesheet" href="/webGui/styles/default-fonts.css?v=1571655817">

<link type="text/css" rel="stylesheet" href="/webGui/styles/default-popup.css?v=1571655817">

</head>

<body style="margin:14px 10px">

<h2>dynamix.system.stats</h2>

 

<h3>2019.01.31c</h3>

 

<ul>

<li>fixed memory graph</li>

<li>updated font-awesome icons</li>

<li>fixed bar color regression</li>

<li>added 'autov' function for .css and .js loading</li>

 

I cant see...

 

!DOCTYPE HTML>

<html lang="en">

<head>

<meta name="robots" content="noindex, nofollow">

<meta http-equiv="Content-Security-Policy" content="block-all-mixed-content">

<link type="text/css" rel="stylesheet" href="/webGui/styles/default-fonts.css?v=1571655817">

<link type="text/css" rel="stylesheet" href="/webGui/styles/default-popup.css?v=1571655817">

</head>

<body style="margin:14px 10px">

<h2>rclone</h2>

 

<h3>2019.10.23</h3>

 

<ul>

<li>Make rclone wrapper branch agnostic</li>

</ul>

Edited October 24, 2019 by nuhll

[ur6969]

On 10/24/2019 at 12:28 AM, ur6969 said:

I have been able to make Rclone work with B2 and am using encryption.  Before I start to upload my backups I wanted to check and see if there was a way to download the encrypted files and decrypt.

 

For instance, if my Unraid server was destroyed I would like to be able to download the files from B2 and then decrypt them.  Is this possible using Windows or OSX?  I obviously have the encryption key and salt saved in multiple places.

 

FWIW I was able to make this happen.  I knew it was possible just surprised myself in actually making it work.

 

I am not looking to mount the drive or use it in real time as a sync folder, just wanted to be able to download my B2 backup should my Unraid server die or get destroyed.  I did NOT want to have a B2 encrypted backup with no way to pull it down other than a new Unraid server and it looks like I am setup to go with B2 as a backup solution.

 

ETA:  I am having trouble getting multiple folders to sync.  Does everything need to be under one folder or can I pick and choose multiple folders to sync from different locations or shares?

Edited October 27, 2019 by ur6969

[Halvliter]

It might be an error on my part, but after the latest update my config file is empty:

My mount script log says this:

2019/10/29 17:31:45 NOTICE: Config file "/boot/config/plugins/rclone/.rclone.conf" not found - using defaults
2019/10/29 17:31:45 Failed to create file system for "disk1:media": didn't find section in config file

 

and when I SSH into the server I see this:
 

2019/10/29 17:39:47 NOTICE: Config file "/boot/config/plugins/rclone/.rclone.conf" not found - using defaults

No remotes found - make a new one

 

Are there any backup config files or did I update the script without reading the fine print?

[Stupifier]

4 hours ago, Halvliter said:

It might be an error on my part, but after the latest update my config file is empty:

My mount script log says this:

2019/10/29 17:31:45 NOTICE: Config file "/boot/config/plugins/rclone/.rclone.conf" not found - using defaults
2019/10/29 17:31:45 Failed to create file system for "disk1:media": didn't find section in config file

 

and when I SSH into the server I see this:
 

2019/10/29 17:39:47 NOTICE: Config file "/boot/config/plugins/rclone/.rclone.conf" not found - using defaults

No remotes found - make a new one

 

Are there any backup config files or did I update the script without reading the fine print?

This command (below) will show you where rclone is looking for your config file....maybe rclone is looking for your config file in a location you didn't know about.
 

rclone config file

For example, the output of that command on my unraid box is:
 

/boot/config/plugins/rclone-beta/.rclone.conf

 

Edited October 29, 2019 by Stupifier

[Waseh]

@Halvliter are you using the stable or the beta branch? I made a mistake in the wrapper on the beta branch at one point so it pointed to the wrong directory. It's fixed in the latest release though.

[Halvliter]

27 minutes ago, Stupifier said:

This command (below) will show you where rclone is looking for your config file....maybe rclone is looking for your config file in a location you didn't know about.
 

rclone config file

For example, the output of that command on my unraid box is:
 

/boot/config/plugins/rclone-beta/.rclone.conf

 

 

Configuration file doesn't exist, but rclone will use this path:

/boot/config/plugins/rclone/.rclone.conf

 

13 minutes ago, Waseh said:

@Halvliter are you using the stable or the beta branch? I made a mistake in the wrapper on the beta branch at one point so it pointed to the wrong directory. It's fixed in the latest release though.

I'm using the beta-branch. I updated a couple of days ago (maybe a week?) and after that I had some problems with booting after a clean shutdown (That problem is solved here: https://forums.unraid.net/topic/84652-solved-only-blinking-cursor-after-clean-shutdown/)

 

When I did the setup of rclone I used Spaceinvaderone's guide, and I've looked for the original config file, but the only one I can find is the empty one.

 

It might be easier to just do a clean config?

 

[Waseh]

Check if you are on the latest version of the plugin. Even then try to uninstall and then install it again to see if that resolves the issue

[Halvliter]

19 hours ago, Waseh said:

Check if you are on the latest version of the plugin. Even then try to uninstall and then install it again to see if that resolves the issue

That did the trick! I uninstalled by ticking the box and "Remove", and then installing "Rclone-beta". Thank you very much!

[theDrell]

I am having a weird problem.  So I have RClone Beta setup on Unraid 6.7.2.  It was working great.  I went and restarted my machine and when rclone tries to mount the drive I get  

Failed to create file system for "gdrive:": couldn't find root directory ID: googleapi: Error 404: File not found:

I downloaded my Rclone.conf file, and ran it on my windows machine. Everything mounts just fine using the same file.

My mount script looks like this:

 

mkdir -p /mnt/disks/gdrive

rclone mount --max-read-ahead 1024k --allow-other gdrive: /mnt/disks/gdrive &

 

Under my rclone-beta config I have the following, with secrets and tokens removed

 

[gdrive]
type = drive
client_id =CODE.apps.googleusercontent.com
client_secret = SECRET
scope = drive.file
token = {"access_token":"TOKEN","token_type":"Bearer","refresh_token":"1//SOMETHING","expiry":"2019-10-31T10:12:03.849383085-05:00"}

 

 

Anyone have any idea?
 

Edited November 1, 2019 by theDrell
add more detail

[Halvliter]

Something is off with my network after I decided to upload my library using rclone. It's laggy and sometimes totally unresponcive. I checked the logs and noticed this:

 

2019/11/01 10:13:03 INFO :
Transferred: 465.178G / 996.936 GBytes, 47%, 4.478 MBytes/s, ETA 1d9h46m42s
Errors: 24 (retrying may help)
Checks: 2947 / 2947, 100%
Transferred: 363 / 458, 79%
Elapsed time: 29h32m57.2s
Transferring:
* Filmer/Clo***********.mkv: 27% /49.695G, 655.219k/s, 15h59m21s
* Filmer/Ene***********.mkv: 65% /15.468G, 124/s, 12957h36m16s
* Filmer/Ferd***********.mkv: 90% /5.465G, 782.069k/s, 11m15s
* Filmer/Fin***********.mkv: 18% /17.577G, 6/s, 625091h40m4s

2019/11/01 10:13:23 INFO :
Transferred: 643.361G / 1007.445 GBytes, 64%, 4.583 MBytes/s, ETA 22h35m55s
Errors: 50 (retrying may help)
Checks: 2734 / 2734, 100%
Transferred: 628 / 686, 92%
Elapsed time: 39h55m59.4s
Transferring:
* Filmer/Fin***********.mkv: 64% /17.577G, 811.294k/s, 2h13m36s
* Filmer/Goo***********.mkv: 93% /5.172G, 1014.824k/s, 5m40s
* Filmer/Hai***********.mkv: 44% /8.748G, 1.350M/s, 1h1m15s
* Filmer/Hosti***********.mkv: 45% /9.838G, 1.189M/s, 1h17m26s

2019/11/01 10:13:24 INFO :
Transferred: 59.923G / 548.503 GBytes, 11%, 3.072 MBytes/s, ETA 1d21h14m4s
Errors: 21 (retrying may help)
Checks: 3067 / 3067, 100%
Transferred: 8 / 75, 11%
Elapsed time: 5h32m52.3s
Transferring:
* Filmer/Find***********.mkv: 33% /18.877G, 822.903k/s, 4h26m10s
* Filmer/Gho***********.mkv: 17% /7.650G, 1.622M/s, 1h6m40s
* Filmer/Ha***********.mkv: 43% /8.748G, 1.003M/s, 1h24m45s
* Filmer/Hon***********.mkv: 28% /16.219G, 1.075M/s, 3h5m2s

Normally rclone only uploads 4 files at a time. 

Edited November 1, 2019 by Halvliter