[Support] Rclone (beta)


Recommended Posts

I am missing something, but I can't figure out how to authorize amazon cloud drive. Following the remote_setup instructions at http://rclone.org/remote_setup/, when I follow these instructions "If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth", I get a "Hmm, we can't reach this page." error.  I know I am missing something, but reading through this forum and the instructions, I can't figure it out.  My default gateway is 192.168.29.1, not sure if that has anything to do with it.  I tried altering the address in the instructions, but nothing has worked.  Thanks in advance for your help.

 

Use the other option. The one that says something like "use this option if running headless or the first option doesn't work". Unraid is headless.

Link to comment

On the headless box

...

Remote config

Use auto config?

* Say Y if not sure

* Say N if you are working on a remote or headless machine

y) Yes

n) No

y/n> n

For this to work, you will need rclone available on a machine that has a web browser available.

Execute the following on your machine:

rclone authorize "amazon cloud drive"

Then paste the result below:

result>

Then on your main desktop machine

rclone authorize "amazon cloud drive"

If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth

Log in and authorize rclone for access

Waiting for code...

Got code

Paste the following into your remote machine --->

SECRET_TOKEN

<---End paste

 

Those instructions are for a headless box.  I will try option two.  Was curious what I was doing wrong with the first set of instructions.  Thanks.

Link to comment
  • 3 weeks later...

Hi everyone. I've set up Rclone and gotten the authorization token for OneDrive.

 

I've attached my config below:

 

index.php?action=dlattach;topic=52033.0;attach=39492;image

 

Right now there is nothing in the data path, it's a folder I just created on unRAID.

 

The sync destination "remote:To To Read/"  refers to the "To To Read" folder in the root of my OneDrive.

 

1) When I run Rclone, what will happen? Will Rclone copy the files from sync destination to data path? Or will it erase everything at sync destination because my data path is currently empty?

 

2) I don't see a log option when I click on the Rclone docker. How do I see what's going on and if this is working?

 

The background to what I'm trying to do: I have a OneDrive folder on my laptop and would like to stop using that. I want to have the OneDrive files reside on my unRAID so they don't occupy space on the laptop.

 

Screen_Shot_2016-12-22_at_3_25.02_AM.png.ea9ee7f88bd07a7238e3ba8c09a76e9a.png

Link to comment

Thanks for that.

 

The Rclone docker config seems to be set up such that the remote is the destination.

 

The default Rclone command is 

 

rclone sync /data $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH

 

Would I be able to flip that around to

 

rclone sync $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH /data

 

So the empty folder on my unRAID will fill up with the files from OneDrive?

 

EDIT:

 

Ok so simply flipping the command doesn't work, guess that would've been too easy.

 

Executing => rclone sync  /data
2016/12/23 00:16:00 Local file system at /data: Waiting for checks to finish
2016/12/23 00:16:00 Local file system at /data: Waiting for transfers to finish
2016/12/23 00:16:00 Local file system at /data: not deleting files as there were IO errors
2016/12/23 00:16:00 Attempt 1/3 failed with 0 errors and: directory not found

2016/12/23 00:16:00 Local file system at /data: Waiting for checks to finish
2016/12/23 00:16:00 Local file system at /data: Waiting for transfers to finish
2016/12/23 00:16:00 Local file system at /data: not deleting files as there were IO errors
2016/12/23 00:16:00 Attempt 2/3 failed with 0 errors and: directory not found

2016/12/23 00:16:00 Local file system at /data: Waiting for checks to finish
2016/12/23 00:16:00 Local file system at /data: Waiting for transfers to finish
2016/12/23 00:16:00 Local file system at /data: not deleting files as there were IO errors
2016/12/23 00:16:00 Attempt 3/3 failed with 0 errors and: directory not found

2016/12/23 00:16:00 Failed to sync: directory not found

 

Does anyone know if Rclone can sync from OneDrive to unRAID instead of the default unRAID to OneDrive?

 

I tried a custom command like

 

rsync sync remote:/Test/ /data/

 

which didn't work too.

Link to comment
  • 1 month later...

I've merged the dev branch into master, so that the latest tag on docker hub contains the newest version.

 

I feel it is quiet stable, as I've been backing up million of files during the past few months. I'm using it for offsite backup of docker application data.

 

If somebody is missing a feature let me know.

Link to comment

Thanks for that.

 

The Rclone docker config seems to be set up such that the remote is the destination.

 

The default Rclone command is 

 

rclone sync /data $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH

 

Would I be able to flip that around to

 

rclone sync $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH /data

 

So the empty folder on my unRAID will fill up with the files from OneDrive?

 

EDIT:

 

Ok so simply flipping the command doesn't work, guess that would've been too easy.

 

Executing => rclone sync  /data
2016/12/23 00:16:00 Local file system at /data: Waiting for checks to finish
2016/12/23 00:16:00 Local file system at /data: Waiting for transfers to finish
2016/12/23 00:16:00 Local file system at /data: not deleting files as there were IO errors
2016/12/23 00:16:00 Attempt 1/3 failed with 0 errors and: directory not found

2016/12/23 00:16:00 Local file system at /data: Waiting for checks to finish
2016/12/23 00:16:00 Local file system at /data: Waiting for transfers to finish
2016/12/23 00:16:00 Local file system at /data: not deleting files as there were IO errors
2016/12/23 00:16:00 Attempt 2/3 failed with 0 errors and: directory not found

2016/12/23 00:16:00 Local file system at /data: Waiting for checks to finish
2016/12/23 00:16:00 Local file system at /data: Waiting for transfers to finish
2016/12/23 00:16:00 Local file system at /data: not deleting files as there were IO errors
2016/12/23 00:16:00 Attempt 3/3 failed with 0 errors and: directory not found

2016/12/23 00:16:00 Failed to sync: directory not found

 

Does anyone know if Rclone can sync from OneDrive to unRAID instead of the default unRAID to OneDrive?

 

I tried a custom command like

 

rsync sync remote:/Test/ /data/

 

which didn't work too.

 

Did you ever figure this out? By reading http://rclone.org/commands/rclone_sync/ it seems like it should be possible? Did you perhaps try to use copy rather than sync?

Link to comment

Does can the docker version encrypt files with the builtin crypt of rclone or do we have to use the plugin if we want to encrypt?

 

Well, yeah. Crypt is a feature of rclone. It has nothing to do with docker. All docker does is package the program in a container. The docker version might use an older version of rclone before the crypt feature was added, if that's what you are asking.

 

The only real difference between the docker and the plugin versions of rclone, as far as usage goes, is that because of the way docker works you cannot share a mount with another docker. So for instance, if you are planning on using a rclone docker encrypted mount with Plex, you will need the rclone plugin.

Link to comment

Does can the docker version encrypt files with the builtin crypt of rclone or do we have to use the plugin if we want to encrypt?

 

Well, yeah. Crypt is a feature of rclone. It has nothing to do with docker. All docker does is package the program in a container. The docker version might use an older version of rclone before the crypt feature was added, if that's what you are asking.

 

The only real difference between the docker and the plugin versions of rclone, as far as usage goes, is that because of the way docker works you cannot share a mount with another docker. So for instance, if you are planning on using a rclone docker encrypted mount with Plex, you will need the rclone plugin.

 

Yes, i've uploaded several TB's of encrypted data with the docker.

 

However I'm using the plugin for mounting the crypted volume back to my server. So the docker can be used in collaboration with the plugin if you which to use the encrypted volumes in other docker containers like Plex for instace.

Link to comment

 

Did you ever figure this out? By reading http://rclone.org/commands/rclone_sync/ it seems like it should be possible? Did you perhaps try to use copy rather than sync?

 

I made some progress.

 

Running

rclone -v sync remote:/Test/ data

in the docker's advance setting:

 

Executing => rclone -v sync remote:/Test/ data
2017/02/02 23:35:00 rclone: Version "v1.35" starting with parameters ["rclone" "-v" "sync" "remote:/Test/" "data"]

2017/02/02 23:35:01 Local file system at /config/data: Modify window is 1s
2017/02/02 23:35:01 One drive root 'Test': Reading ""
2017/02/02 23:35:01 One drive root 'Test': Finished reading ""
2017/02/02 23:35:01 Local file system at /config/data: Waiting for checks to finish
2017/02/02 23:35:01 Local file system at /config/data: Waiting for transfers to finish
2017/02/02 23:35:02 unraid-banner.png: Copied (new)
2017/02/02 23:35:02 Go routines at exit 7
2017/02/02 23:35:02 Waiting for deletions to finish
2017/02/02 23:35:02
Transferred: 87.364 kBytes (77.966 kBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 1.1s

 

Rclone creates the folder "data" in Rclone's config folder and the "unraid-banner.png" is copied.

Running

rclone -v sync remote:/Test/ /data

in the docker's advance setting, I get "read-only file system" error:

 

Executing => rclone -v sync remote:/Test/ /data
2017/02/02 23:41:00 rclone: Version "v1.35" starting with parameters ["rclone" "-v" "sync" "remote:/Test/" "/data"]

2017/02/02 23:41:01 Local file system at /data: Modify window is 1s
2017/02/02 23:41:01 One drive root 'Test': Reading ""
2017/02/02 23:41:01 One drive root 'Test': Finished reading ""
2017/02/02 23:41:01 Local file system at /data: Waiting for checks to finish
2017/02/02 23:41:01 Local file system at /data: Waiting for transfers to finish
2017/02/02 23:41:01 unraid-banner.png: Failed to copy: open /data/unraid-banner.png: read-only file system

2017/02/02 23:41:01 Local file system at /data: not deleting files as there were IO errors
2017/02/02 23:41:01 Attempt 1/3 failed with 1 errors and: not deleting files as there were IO errors

 

Running

rclone sync $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH /data

gets me "directory not found error" because the $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH variable is not being passed to the command:

 

Executing => rclone sync  /data
2017/02/02 23:44:00 Local file system at /data: Waiting for checks to finish
2017/02/02 23:44:00 Local file system at /data: Waiting for transfers to finish
2017/02/02 23:44:00 Local file system at /data: not deleting files as there were IO errors
2017/02/02 23:44:00 Attempt 1/3 failed with 0 errors and: error listing source: Local file system at /config/:: directory not found

 

Running the default command (manually putting it into the text box)

rclone sync /data $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH

gets me this, not sure what it is:

 

Executing => rclone sync /data 
2017/02/02 23:50:00 Local file system at /config/:: Waiting for checks to finish
2017/02/02 23:50:00 Local file system at /config/:: Waiting for transfers to finish
2017/02/02 23:50:00 Waiting for deletions to finish
2017/02/02 23:50:00
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 0s

 

It seems to think /data = /config/ even though I had set it up the data path to be /mnt/user/Comics/Test/ ?

 

index.php?action=dlattach;topic=52033.0;attach=40836;image

 

And $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH always appear as :/ in the actual command.

 

I hope this helps, I'm very confused.

 

The first test kinda worked except it was creating the folder and syncing to the docker's config folder.

Screen_Shot_2017-02-02_at_6_57.22_PM.png.aedcc421808b0fa3b86dc0d18959c49a.png

Link to comment

Ok I solved it, it was a silly permissions problem except I didn't know where to look.

 

I had to edit the settings for the Data Path. Access Mode was set to "Read Only" by default, and I changed it to "Read/Write":

 

index.php?action=dlattach;topic=52033.0;attach=40890;image

 

But I had to run it this way: rclone -v sync remote:/Test/ /data which is different from the default format of rclone sync /data $SYNC_DESTINATION:/$SYNC_DESTINATION_SUBPATH

Screen_Shot_2017-02-04_at_2_14.29_PM.png.8c28cce7bb3cd6e0ab95e7687894f47a.png

Link to comment

@vurt Great it worked out - and thanks for posting the solution.

 

I will consider changing the default permission of the /data path to RW.

 

I realize it's probably set to Read Only so nothing goes wrong with the data that Rclone is meant to be backing up to the cloud, and in my case I'm doing the reverse of that. But glad it works both ways!

 

Confused as to why I have to run the command that way instead of the default format though.

Link to comment

Does can the docker version encrypt files with the builtin crypt of rclone or do we have to use the plugin if we want to encrypt?

 

Well, yeah. Crypt is a feature of rclone. It has nothing to do with docker. All docker does is package the program in a container. The docker version might use an older version of rclone before the crypt feature was added, if that's what you are asking.

 

The only real difference between the docker and the plugin versions of rclone, as far as usage goes, is that because of the way docker works you cannot share a mount with another docker. So for instance, if you are planning on using a rclone docker encrypted mount with Plex, you will need the rclone plugin.

 

Cool thanks. That plex example is what I was looking at. I just using it to backup so I can stick with the docker. Thanks.

Link to comment
  • 1 month later...
  • 3 weeks later...
On 1/29/2017 at 7:04 AM, bobbintb said:

The only real difference between the docker and the plugin versions of rclone, as far as usage goes, is that because of the way docker works you cannot share a mount with another docker. So for instance, if you are planning on using a rclone docker encrypted mount with Plex, you will need the rclone plugin.

 

@bobbintb / @hackztor I've created a seperate docker which now supports sharing the FUSE mount to the host and other docker containers: Rclone-mount

Link to comment

I'm having some issues when I try to define multiple sync folders.

In the sync command I'm defining the variable like "rclone sync /folder1 remoteB2:bucket1 && rclone sync /folder2 remoteB2:bucket2" but when I apply the changes on docker menu I get the following warning on the UI before stopping/removing/etc. the docker:

 

Warning: SimpleXMLElement::addChild(): unterminated entity reference & rclone sync /folder1 remoteB2:bucket1 in /usr/local/emhttp/plugins/dynamix.docker.manager/include/CreateDocker.php on line 211

 

Any ideas?

If we check in the command itself, the variable is empty ("SYNC_COMMAND"=""). 

 

 

Link to comment
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.