[Support] binhex - Rclone


Recommended Posts

On 2/18/2022 at 5:16 PM, Keichi said:

Hello,

 

I am trying to do a simple thing : copy my Google Drive in /mnt/user/GoogleDrive.

But i missed something...

 

Here the details if you can help, thanks.

 

I successfully installed this Docker with that :

695503179_Capturedecran2022-02-18a17_08_20.thumb.png.e717ebb1aa8adc2777a81d8b464b90a7.png

I can navigate from the docker to my /media path

1068609709_Capturedecran2022-02-18a17_10_24.png.f8b8d14e6193b65daeb6b731b9feb115.png

 

I can see my Google Drive on the WebGUI :

362191687_Capturedecran2022-02-18a17_11_17.thumb.png.d4f1ff81e99c304313a1f4504b11e312.png

 

But in the log, i have :

119205086_Capturedecran2022-02-18a17_15_32.png.32ebea18dab65191a78cd429e4c17a8b.png

 

I try a lot of things for RCLONE_MEDIA_SHARES, but i think i don't understand what it is.

 

Thank you for your assitance,

 

K.

 

 

Hello, no one for help ?

 

Thanks

Link to comment
  • 3 weeks later...

Hi, was something changed about a month ago that effected the parsing of variables, specifically the RCLONE_USER_FLAGS variable?

 

I used to use the below string as one of my user flags in order to BACKUP files from the SYNC into the correct folder based on the current rclone_media_shares_item share being run e.g. BACKUPS/PERSONAL, BACKUPS/PROFESSIONAL, etc.. This was running correctly up until about a month ago when it then created a new folder at "BACKUPS/${rclone_media_shares_item}" where it then started comining all my backup files from all shares into that same folder.

 

RCLONE_USER_FLAGS="other flags... --backup-dir BACKUPS/${rclone_media_shares_item}"

 

I've looked at your repository and can't seem to be able to find what the hell changed.

 

Would appreciated any help.

Link to comment
  • 2 months later...
  • 4 weeks later...

Hello, thanks for this docker. I've just finished setting it up, and wanted to share a possible workaround for the issue mentioned here: https://github.com/binhex/arch-rclone/issues/1

 

Note: while I am familiar with docker/linux/unraid, I am by no means an expert - there is likely a better way to do what I am doing, so don't hesitate to improve on this approach, or tell me why it's a bad idea.

 

In short, I set my docker up to not copy or sync anything in the start.sh script, and instead utilize a user script to do my syncing.

 

After setting up the docker as normal, I changed the following parameters in the container:

  • RCLONE_MEDIA_SHARES is now set to a single sub-folder in /media as it causes an error if empty, but the actual folder doesn't matter as I don't sync it this way
  • RCLONE_SLEEP_PERIOD is set to 30d, but any long duration should work
  • RCLONE_USER_FLAGS is set to "--dry-run". This is what prevents the start script from doing any actual syncing
  • RCLONE_POST_CHECK and ENABLE_WEBUI are both set to "no", as I don't use these

 

With this configuration, I created a user script with the following:

#!/bin/bash

docker exec binhex-rclone sh -c "if [ ! -f /home/nobody/.config/rclone/rclone.conf ]; then ln -s /config/rclone/config/rclone.conf /home/nobody/.config/rclone/rclone.conf; fi"

docker exec binhex-rclone sh -c "rclone sync -P /media/folder remote:Pictures"

 

The first docker exec line creates a link for the config file so that the second line can run (only if the desired config file does not already exist). The second docker exec line calls the docker container and runs a sync command that syncs the folder "folder" from the media path (replace with your appropriate folder name) to the remote destination "remote:Pictures". For me, this creates the contents of /media/folder in my Pictures folder on my remote destination, and not in a "media" folder.

 

So, if I have a folder called "holiday" inside of /media/folder and I run this script, it will create a corresponding folder on onedrive (my remote) inside the Pictures folder there. Meaning:

/media/folder/holiday becomes onedrive:Pictures/holiday

 

I only just got this working, but my intent is to set the script to run on a schedule (daily).

  • Like 1
  • Thanks 1
Link to comment
  • 5 weeks later...

I've got 2 questions/issues.

 

Firstly, While the UI is loading, it constantly prompts for credentials and complains about not being able to connect to rclone. I checked the UI log and its stating the following:

 

2022/09/27 09:02:58 INFO  : Using --user admin --pass XXXX as authenticated user
2022/09/27 09:02:58 NOTICE: Serving remote control on http://0.0.0.0:5572/
2022/09/27 09:02:58 NOTICE: Web GUI is not automatically opening browser. Navigate to http://admin:[email protected]:5572/?login_token=TOKENREMOVED to use.

 

XXXX is not my password obviously, but my password is showing in the line below stating "Web GUI is not automatically opening browser." if I copy that URL it logs in and works just fine, but why is it not working directly from the Unraid UI?

 

Secondly, I have a few shares with spaces in the name, for Example /mnt/user/appdata backup. But I am having issues getting the docker to see these as Media Shares. I've tried the following:

 

  • /media/appdata backup
  • /media/'appdata backup'
  • '/media/'appdata backup''
  • '/media/appdata backup'
  • /media/'appdata\ backup'
  • /media/appdata\ backup

All yield the same result, folder not found. How do I deal with handling a folder space in the path name for this docker?

 

Thanks!

Link to comment
  • 4 weeks later...
  • 2 weeks later...

Trying to Sync from OneDrive to unRAID share named OneDrive. The issue I can't seem to deteremine the correct paths to sync data from OneDrive to the unRAID OneDrive share. The data always syncs to the root / of the container when I login to the console. Below is the settings:

 

I have tried every combination of paths I can think of, only to sync to the root of the container from the console rather to the OneDrive share located at /mnt/user/OneDrive. Any suggestions would be appreciated.

 

image.png.6038db19f6e7f84f61b59b60b9b8e4a8.png

Link to comment
10 hours ago, Govnah said:

Trying to Sync from OneDrive to unRAID share named OneDrive. The issue I can't seem to deteremine the correct paths to sync data from OneDrive to the unRAID OneDrive share. The data always syncs to the root / of the container when I login to the console. Below is the settings:

 

I have tried every combination of paths I can think of, only to sync to the root of the container from the console rather to the OneDrive share located at /mnt/user/OneDrive. Any suggestions would be appreciated.

 

image.png.6038db19f6e7f84f61b59b60b9b8e4a8.png

set the first path like so:-

image.png.3c1c70644b7185b1c0950d97ca309dd8.png 

 

then set RCLONE_MEDIA_SHARES as follows:-

image.png.342dfc6b6c9c448f087cac31c39ae518.png

 

also please ensure that RCLONE_REMOTE_NAME does indeed match the remote name defined in your rclone config file, including CaSe

 

Link to comment

Thank you binhex for this amazing docker.

I have a question concerning a persistent configuration, which I was not able to solve by reading this thread.

When I edit the config with "rclone config --config /config/rclone/config/rclone.config", the content of this file is of course not transfered to the /home/nobody/.config/rclone/config/rclone.config. 

I replace the file in the /home/nobody folder by "cp /config/rclone/config/rclone.config /home/nobody/.config/rclone/config/" but I don't think this is how this docker is intended to be used. 

How can I tell rclone, that it should use the config located at the /config/rclone/config folder and NOT that from the /home/nobody/.conf/rclone/config

Here you can see my config:

 

docker run

  -d

  --name='binhex-rclone'

  --net='bridge'

  -e TZ="Europe/Berlin"

  -e HOST_OS="Unraid"

  -e HOST_HOSTNAME="Tilda"

  -e HOST_CONTAINERNAME="binhex-rclone"

  -e 'RCLONE_CONFIG_PATH'='/config/rclone/config/rclone.conf'

  -e 'RCLONE_MEDIA_SHARES'='/media/photo_florian,/media/backups'

  -e 'RCLONE_REMOTE_NAME'='nextcloud'

  -e 'RCLONE_SLEEP_PERIOD'='24h'

  -e 'RCLONE_USER_FLAGS'=''

  -e 'RCLONE_OPERATION'='copy'

  -e 'RCLONE_DIRECTION'='localtoremote'

  -e 'RCLONE_POST_CHECK'='yes'

  -e 'ENABLE_WEBUI'='yes'

  -e 'WEBUI_USER'='Florian'

  -e 'WEBUI_PASS'='supersafepassword'

  -e 'UMASK'='000'

  -e 'PUID'='99'

  -e 'PGID'='100'

  -l net.unraid.docker.managed=dockerman

  -l net.unraid.docker.webui='http://[IP]:[PORT:5572]/'

  -l net.unraid.docker.icon='https://raw.githubusercontent.com/binhex/docker-templates/master/binhex/images/rclone-icon.png'

  -p '53682:53682/tcp'

  -p '5572:5572/tcp'

  -v '/mnt/user':'/media':'rw'

  -v '/mnt/user/appdata/binhex-rclone':'/config':'rw' 'binhex/arch-rclone'

8639103e43462abab43ab1d8a7f5b043e079a1ae55d4db864f307ea99f36795e

The command finished successfully!


Thank you very much for your help!

Maginos

Link to comment
11 hours ago, Maginos said:

When I edit the config with "rclone config --config /config/rclone/config/rclone.config", the content of this file is of course not transfered to the /home/nobody/.config/rclone/config/rclone.config. 

it doesnt need to be transferred, the rclone cli is set to use whatever path is specified for RCLONE_CONFIG_PATH which in your case is /config/rclone/config/rclone.conf

Link to comment
19 hours ago, animeking said:

I don't know what i'm doing wrong as it cannot find my config path. I have read all the issues and tried them all but none work for me: 

ok you need to go to unraid webui/docker/ left click rclone and then click at the bottom ' Show more settings ...' this will then show you the host path for /config, so its probably somethng like /mnt/user/appdata/binhex-rclone so then go to windows explorer and look for a share named appdata, if its not shown then enable smb for the share.

Link to comment
8 minutes ago, binhex said:

it doesnt need to be transferred, the rclone cli is set to use whatever path is specified for RCLONE_CONFIG_PATH which in your case is /config/rclone/config/rclone.conf


My problem is the following:

After downloading the rclone docker, I run "rclone config --config /config/rclone/config/rclone.config" and set up my web services.

When I try to access these webservices for example with "rclone lsd nextcloud:/" I get the error message, that there is no config at /home/nobody/...

After I copy the config from the /config/... folder to /home/nobody/... I can access the webservices. So this is the reason, why I do the copy.

 

 

4 minutes ago, binhex said:

ok you need to go to unraid webui/docker/ left click rclone and then click at the bottom ' Show more settings ...' this will then show you the host path for /config, so its probably somethng like /mnt/user/appdata/binhex-rclone so then go to windows explorer and look for a share named appdata, if its not shown then enable smb for the share.

 

I already saw this, and yes, the file is created. But as I said above, as long as I don't copy the config, rclone doesn't work.

Additionally: When I execute "rclone config file", the file from the /home/nobody/... folder is displayed. 

I hope, you can now better understand my problem.

Link to comment
34 minutes ago, Maginos said:

When I try to access these webservices for example with "rclone lsd nextcloud:/" I get the error message, that there is no config at /home/nobody/...

yeah this is where you are going wrong, this image is not meant to be driven from the rclone cli, once the rclone config has been generated then all config should be done through the env vars e.g. 'RCLONE_OPERATION', 'RCLONE_DIRECTION' etc . If you do want to do custom operations with rclone then i would recommend using the rclone plugin instead - https://forums.unraid.net/topic/51633-plugin-rclone/
 

 

if however you do want to proceed with using the rclone cli inside of the container (no support) then you can do what you want by simply using the config option flag and passing it to rclone, e.g.:-

 

--config=/config/rclone/config/rclone.conf

  • Upvote 1
Link to comment
5 minutes ago, binhex said:

yeah this is where you are going wrong, this image is not meant to be driven from the rclone cli, once the rclone config has been generated then all config should be done through the env vars e.g. 'RCLONE_OPERATION', 'RCLONE_DIRECTION' etc . If you do want to do custom operations with rclone then i would recommend using the rclone plugin instead - https://forums.unraid.net/topic/51633-plugin-rclone/
 

 

if however you do want to proceed with using the rclone cli inside of the container (no support) then you can do what you want by simply using the config option flag and passing it to rclone, e.g.:-

 

--config=/config/rclone/config/rclone.conf

Ok thank you, I will have a look at the plugin and the --config option and will decide, what fits my needs better.

Thank you for your help!

Link to comment
  • 4 weeks later...

Revisting from an unsuccessful effort, I am attempting to "remotetolocal (OneDrive to unRAID). I have a share on unRAD for OneDrive located at /mnt/user/OneDrive.

 

I have tried every possible combination I can think related to the following:

 

image.png.da183e7f1dfcf00dc67e221af14ac7fa.png

 

 

Log:

2022-12-02 18:44:13,542 DEBG 'start-script' stdout output:
[info] Running rclone copy from remote 'remote:/media/OneDrive' to local share '/media/OneDrive'...

2022-12-02 18:44:14,953 DEBG 'start-script' stdout output:
[info] rclone copy finished

 

Any help would be appreciated.

 

Thanks

 

Edited by Govnah
Link to comment
Revisting from an unsuccessful effort, I am attempting to "remotetolocal (OneDrive to unRAID). I have a share on unRAD for OneDrive located at /mnt/user/OneDrive.
 
I have tried every possible combination I can think related to the following:
 
image.png.da183e7f1dfcf00dc67e221af14ac7fa.png
 
 
Log:
2022-12-02 18:44:13,542 DEBG 'start-script' stdout output:
[info] Running rclone copy from remote 'remote:/media/OneDrive' to local share '/media/OneDrive'...
2022-12-02 18:44:14,953 DEBG 'start-script' stdout output:
[info] rclone copy finished
 
Any help would be appreciated.
 
Thanks
 
Does '/media/OneDrive/' exist on the remote (OneDrive)?

Sent from my 22021211RG using Tapatalk

Link to comment

It does not, here is my directory structure in OneDrive/remote:

 

My Files from left; 

  Pictures

  Apps

  Documents

  Public

  Music

  etc...

 

So what I am attempting is to sync/remotetolocal from the above OneDrive root (including all the directories listed) to the root of /mnt/user/OneDrive.

 

So the question, if I wanted to sync the entire OneDrive (remote) to /mnt/user/OneDrive (local) what would be the paths for "Container Path" and "Rclone_Media-Shares"? All OneDrive directories are in the root of OneDrive, I don't have a "media" directory w/ all the directories underneath that. I think I understand the logic, but just can't seem to come w/ the correct variables to achieve what I am trying to accomplish, just simply synching OneDrive to /mnt/user/OneDrive.

 

Thanks

 

 

 

Edited by Govnah
Link to comment
  • 2 months later...

I'm trying to connect this to an existing backblaze b2 bucket and failing somewhere. In my template I have:

  • Host path 2 = /mnt/user/data-media/
  • RCLONE_MEDIA_SHARES = /media/<folder-with-same-name-as-b2-bucket>
  • RCLONE_REMOTE_NAME = b2:<existing-bucket-name>
  • RCLONE_OPERATION = copy
  • RCLONE_DIRECTION = both

I went through the initial config in the console. It said no remotes found, do I want to create a new one? I said, yes and entered info for my existing b2 bucket. I entered the bucket's applicationKeyId and Application Key, as mentioned here.

 

I can access the UI, and in the Explorer I can see my existing bucket and its contents. I tried deleting a file through the explorer, but it remains in the bucket. I tried adding a file to the /media/<folder-with-same-name-as-b2-bucket> on Unraid, but it doesn't get copied to the bucket.

 

Both the local-to-remote and remote-to-local reports say, "Failed to create file system for "b2:<bucket-name>/media/<bucket-name>": didn't find section in config file"

 

The rclone.conf file exists in the right place. It only contains the details for my b2 bucket.

 

rclone ls <bucket-name>: returns "NOTICE: Config file "/home/nobody/.config/rclone/rclone.conf" not found - using defaults"

 

Can anyone see where I'm scewing this up?

 

 

Link to comment
  • 2 months later...

Are the backend fields editable when using the webGUI? I'd like to change the number of transfers from 4 to 1 but while it looks like I can type a number into the field, the field is not saved.  Is there somewhere else I'm supposed to change this? It's working great for me otherwise.

 

Thanks

 

*Edit* Seems like I figured it out.  You need to add the --transfers flag to the RCLONE USER FLAG section

Edited by BorisBee
Link to comment
  • 1 month later...

Thank you binhex for this amazing container. For me this will solve I assume many wishes regarding my cloud backup!

One question while I'm playing with my setup: In the GUI you can set bandwith limits. However I tried to define this in the docker environment, but this didn't work. How can I set the limits static / before starting the container?

This didn't work unfortunately:
image.png.a95fae7b94c4fe050168e5ed1affc162.png

Link to comment
6 hours ago, nitrosont said:

Thank you binhex for this amazing container. For me this will solve I assume many wishes regarding my cloud backup!

One question while I'm playing with my setup: In the GUI you can set bandwith limits. However I tried to define this in the docker environment, but this didn't work. How can I set the limits static / before starting the container?

This didn't work unfortunately:
image.png.a95fae7b94c4fe050168e5ed1affc162.png

you cant just specify random env vars and expect them to work, i wish they did :-). 

 

you can instead try specifying the bandwidth option for key RCLONE_USER_FLAGS by setting the value to '--bwlimit 100M'

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.