Jump to content
Waseh

[Plugin] rclone

487 posts in this topic Last Reply

Recommended Posts

16 minutes ago, Waseh said:

If the curl command doesnt solve it then the fix probably won't work either though since that's the fix im doing 😅
I just updated the plugin with a longer timeout.

Thanks for the quick reply. Strange thing is that it returns that --connect-timeout is an unknown command. Even though the info says it's a valid one. Do you get the same result?

 

EDIT: make a totally fresh install, deleted plugin and all folders. Then installed the beta and got the following result:

Verifying package rclone-2018.08.25-bundle.txz.
Installing package rclone-2018.08.25-bundle.txz:
PACKAGE DESCRIPTION:
Package rclone-2018.08.25-bundle.txz installed.
Downloading rclone
Downloading certs
Download failed - No existing archive found - Try again later
plugin: run failed: /bin/bash retval: 1

EDIT: found the issue, same as before. Can't ping 1.1.1.1 somehow. Really annoying. Will look into my router settings again. Sorry

Edited by Kaizac

Share this post


Link to post

That doesn't really make sense though since that should result in the "No internet - Skipping download and using existing archives" error.
Are you sure you can reach the rclone webpage?

Share this post


Link to post
3 minutes ago, Waseh said:

That doesn't really make sense though since that should result in the "No internet - Skipping download and using existing archives" error.
Are you sure you can reach the rclone webpage?

Yeah no problem reaching. I can ping 8.8.8.8 and 1.0.0.1 fine but not 8.4.4.8 and 1.1.1.1. Very strange, currently looking into it but seems very weird.

Share this post


Link to post

Yea but reaching 8.8.8.8 is fine. The plugin checks 8.8.8.8 first and only tries 1.1.1.1 if 8.8.8.8 fails.

Share this post


Link to post
1 minute ago, Waseh said:

Yea but reaching 8.8.8.8 is fine. The plugin checks 8.8.8.8 first and only tries 1.1.1.1 if 8.8.8.8 fails.

Disabled my pi-hole and used unbound on Pfsense box. Still can't ping 8.4.4.8 but now I got this and it just flew through the install.

 


 

Verifying package rclone-2018.08.25-bundle.txz.
Installing package rclone-2018.08.25-bundle.txz:
PACKAGE DESCRIPTION:
Package rclone-2018.08.25-bundle.txz installed.
Downloading rclone
Downloading certs
Archive: /boot/config/plugins/rclone-beta/install/rclone-beta-latest.zip
creating: /boot/config/plugins/rclone-beta/install/rclone-v1.45-185-gf3874707-beta-linux-amd64/
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.45-185-gf3874707-beta-linux-amd64/rclone.1
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.45-185-gf3874707-beta-linux-amd64/rclone
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.45-185-gf3874707-beta-linux-amd64/README.txt
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.45-185-gf3874707-beta-linux-amd64/README.html
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.45-185-gf3874707-beta-linux-amd64/git-log.txt

-----------------------------------------------------------
rclone-beta has been installed.
-----------------------------------------------------------

plugin: installed

Updating Support Links

rclone-beta --> http://lime-technology.com/forum/index.php?topic=53365.0

Still no idea why Im having this problem but at least I got it installed!

Share this post


Link to post

Yea i'm still going with your pihole blocking the rclone download. For the record i also use a pihole instance as my local dns resolver which works without any problems.
Reviewing the code that's pretty much the only way to get that exact error without any other errors.

Edited by Waseh

Share this post


Link to post
plugin: updating: rclone.plg

+==============================================================================
| Skipping package rclone-2018.08.25-bundle (already installed)
+==============================================================================

Downloading rclone
Downloading certs
Archive: /boot/config/plugins/rclone-beta/install/rclone-beta-latest.zip
creating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/rclone.1
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/rclone
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/README.txt
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/README.html
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/git-log.txt
Copy failed - is rclone running?
plugin: run failed: /bin/bash retval: 1

Just got the same error...

Share this post


Link to post

So was rclone running? I mean the error is pretty self explanatory. 

Share this post


Link to post

I have a strange issue, I have rclone setup for OneDrive as well as Dropbox. Both appear to work correctly, I can SSH to my server and see files in both. I have a sabnzbd script which processes certain downloads and moves them into a folder in DropBox which works fine. As soon as I change the location to the same named folder in OneDrive is fails with permission denied.

 

Not too sure how to go about diagnosing it. Looking at the detailed ls output they are both showing drwxrwxrwx

Share this post


Link to post

I want to throw a HUGE Thank you to Kaizac. He is a magic man! Had me up and running in no time and knows exactly what he is doing. Great work my friend! 

Share this post


Link to post

I was having troubles installing this. I got this error:

 

plugin: installing: https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg
plugin: downloading https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg
plugin: downloading: https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg ... done

+==============================================================================
| Installing new package /boot/config/plugins/rclone-beta/install/rclone-2018.08.25-bundle.txz
+==============================================================================

Verifying package rclone-2018.08.25-bundle.txz.
Installing package rclone-2018.08.25-bundle.txz:
PACKAGE DESCRIPTION:
Package rclone-2018.08.25-bundle.txz installed.
Downloading rclone
Downloading certs
Download failed - No existing archive found - Try again later
plugin: run failed: /bin/bash retval: 1

Updating Support Links

Looking at the plugin script, the failing line was this:

curl --connect-timeout 5 --retry 3 --retry-delay 2 --retry-max-time 30 -o /boot/config/plugins/&name;/install/rclone-&rcloneversion;.zip &rcloneurl;

Which translates to this:

curl --connect-timeout 5 --retry 3 --retry-delay 2 --retry-max-time 30 -o /boot/config/plugins/rclone-beta/install/rclone-beta-latest.zip https://beta.rclone.org/rclone-beta-latest-linux-amd64.zip

Running that gave me a resolve timeout error:

root@Tower:/boot/config/plugins/rclone-beta/install$ curl --connect-timeout 5 --retry 3 --retry-delay 2 --retry-max-time 30 -o rclone-beta-latest.zip https://beta.rclone.org/rclone-beta-latest-linux-amd64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
Warning: Transient problem: timeout Will retry in 2 seconds. 3 retries left.
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left.
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
Warning: Transient problem: timeout Will retry in 2 seconds. 1 retries left.
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
curl: (28) Resolving timed out after 5000 milliseconds

So I just bumped the timeout to 15 and the command ran fine:

root@Tower:/boot/config/plugins/rclone-beta/install$ curl --connect-timeout 15 --retry 3 --retry-delay 2 --retry-max-time 30 -o rclone-beta-latest.zip https://beta.rclone.org/rclone-beta-latest-linux-amd64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 9758k  100 9758k    0     0  1409k      0  0:00:06  0:00:06 --:--:-- 2184k
root@Tower:/boot/config/plugins/rclone-beta/install$ ls
rclone-2018.08.25-bundle.txz*  rclone-beta-latest.zip*

After that I was able to do the normal plugin install no problem.

 

@Waseh - I think this would be good to update in the plugin script.

Edited by JibbsIsMe

Share this post


Link to post
On 2/9/2019 at 8:20 PM, remati said:

plugin: updating: rclone.plg

+==============================================================================
| Skipping package rclone-2018.08.25-bundle (already installed)
+==============================================================================

Downloading rclone
Downloading certs
Archive: /boot/config/plugins/rclone-beta/install/rclone-beta-latest.zip
creating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/rclone.1
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/rclone
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/README.txt
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/README.html
inflating: /boot/config/plugins/rclone-beta/install/rclone-v1.46.0-006-gef5e1909-beta-linux-amd64/git-log.txt
Copy failed - is rclone running?
plugin: run failed: /bin/bash retval: 1

Just got the same error...

Did you get this figured out? I've got the same error even though rclone isn't running.

Share this post


Link to post
2 minutes ago, Tortoise Knight said:

Did you get this figured out? I've got the same error even though rclone isn't running.

The binary is busy as the copy is failing. So something has the binary locked. 

Share this post


Link to post
6 minutes ago, Waseh said:

The binary is busy as the copy is failing. So something has the binary locked. 

I'm sorry, I'm pretty much Linux illiterate so I wouldn't know how to check this or fix it.

Share this post


Link to post
9 hours ago, Tortoise Knight said:

I'm sorry, I'm pretty much Linux illiterate so I wouldn't know how to check this or fix it.

Did you have a mount running? If so your rclone is busy. Best way to make sure is just a reboot of your server with no mounts on. So if you have scripts for mounts, disable those.

Share this post


Link to post
6 hours ago, Kaizac said:

Did you have a mount running? If so your rclone is busy. Best way to make sure is just a reboot of your server with no mounts on. So if you have scripts for mounts, disable those.

This guy knows his stuff. Finally got it it updated. Thanks!

Share this post


Link to post

So I have been running Rclone for a couple of weeks after getting some help setting it up and I have a few questions about it. 

 

First, When I am transferring, Every so often I get the following error and if I hit retry it just pops back up however if I wait a while and hit retry it will pick back up. my thought is that my uploads are storing on a cache drive of 750gb on google and I have to wait for some files to move off before I can put more onto it, well atleast thats what it seems because of needing to wait some time before I can put some more to google.

1170953112_ScreenShot2019-03-08at11_55_22AM.thumb.png.ff5ecd07e86fdda8e8e8cb55917caad7.png

 

 

Second: 

When someone is streaming to plex from the Rclone mount, I would assume that this file is temp stored on the server while It is being streamed, Now does this create a cache folder on the array and then clears when it is done streaming. Or does it hit ram. I ask because it appears that when someone is streaming my ram usage is raised but when done it lowers. Now when I was getting help I do not recall any caching set up and I know he had mentioned something about it but I have not done any configurations about it. Can anyone answer either of these questions? 

Share this post


Link to post
4 minutes ago, tmoran000 said:

So I have been running Rclone for a couple of weeks after getting some help setting it up and I have a few questions about it. 

 

First, When I am transferring, Every so often I get the following error and if I hit retry it just pops back up however if I wait a while and hit retry it will pick back up. my thought is that my uploads are storing on a cache drive of 750gb on google and I have to wait for some files to move off before I can put more onto it, well atleast thats what it seems because of needing to wait some time before I can put some more to google.

1170953112_ScreenShot2019-03-08at11_55_22AM.thumb.png.ff5ecd07e86fdda8e8e8cb55917caad7.png

 

 

Second: 

When someone is streaming to plex from the Rclone mount, I would assume that this file is temp stored on the server while It is being streamed, Now does this create a cache folder on the array and then clears when it is done streaming. Or does it hit ram. I ask because it appears that when someone is streaming my ram usage is raised but when done it lowers. Now when I was getting help I do not recall any caching set up and I know he had mentioned something about it but I have not done any configurations about it. Can anyone answer either of these questions? 

The error you get is probably because you maxxed your 750gb daily upload to gdrive. You are writing directly to the Gdrive so your local drives are not hit or bottlenecking you.

 

With your setup you are not using rclone cache. When you stream however it is putting your buffer in your RAM. Also depending on your Plex setup it might be transcoding to RAM aswell. So if you have enough RAM there is no problem. If you are running short you can play with the mount settings to put less in buffer.

 

 

Share this post


Link to post

Thank you. With Rclone cache, does that let you choose where it gets cached to? i.e. if I add a 120 SSD Mount to unraid, can I choose that as the cache for the streaming so that it does not hit ram or any other location on the array?

Share this post


Link to post
3 minutes ago, tmoran000 said:

Thank you. With Rclone cache, does that let you choose where it gets cached to? i.e. if I add a 120 SSD Mount to unraid, can I choose that as the cache for the streaming so that it does not hit ram or any other location on the array?

You don't want rclone cache since you use VFS. Rclone cache is just a temporary storage folder from which it gets uploaded to your gdrive. VFS is superior.

 

What you can do though is use an SSD to use for your Plex to use as transcoder location. But if you are mostly direct streaming this won't help. So first check what kind of streams you are having.

 

And like I said, you're probably better of reducing the buffer commands in your mount command. You can look the seperate commands up and see whether lowering them will help your RAM usage.

Share this post


Link to post
48 minutes ago, Kaizac said:

You don't want rclone cache since you use VFS. Rclone cache is just a temporary storage folder from which it gets uploaded to your gdrive. VFS is superior.

 

What you can do though is use an SSD to use for your Plex to use as transcoder location. But if you are mostly direct streaming this won't help. So first check what kind of streams you are having.

 

And like I said, you're probably better of reducing the buffer commands in your mount command. You can look the seperate commands up and see whether lowering them will help your RAM usage.

Thank you, I remember you mentioning it but now that I am live I just couldnt remember what you said. I have enough ram for now but I typically have 10-15 people streaming at any one time so I just didnt want to run into any issues. Maybe Ill just add some more ram to give me some over head and see how it goes, Otherwise I will do what you said with lowering the Buffer size. Plex already has a dedicated 500gb ssd for metadata and transcoding .

Share this post


Link to post
On 3/7/2019 at 1:52 PM, Tortoise Knight said:

Did you get this figured out? I've got the same error even though rclone isn't running.

Yes, rclone was not running but still gave me that error. I had to stop the array, reboot and then update. Seems to have solved that issue.

Share this post


Link to post

I get this error when running rclone after sshing into my unRAID box.

 

# rclone
/usr/sbin/rclone: line 21: rcloneorig: command not found

 

Is there a way to uninstall/reinstall or force reinstall?

Share this post


Link to post

Hey, does someone know how to make the mount point writable from a smb share?

 

I'm monting like this:

rclone mount --allow-other --dir-cache-time 96h --drive-chunk-size 32M --log-level INFO --log-file /var/log/rclone.log --timeout 1h --dir-perms 0777 --file-perms 0777 --umask 002 --rc secure: /mnt/disks/gdrive &

 

And I've tried these smb settings: 

 

[secure-cloud]
      path = /mnt/disks/gdrive
      comment =
      browseable = yes
      # Public
      public = yes
      writeable = yes
      vfs objects =

[gdrive]
   comment = share
   path = /mnt/disks/gdrive
   browseable = yes
   read only = no
   writeable = yes
   create mask = 0771
   directory mask = 0771
   valid users = unraid

 

And I keep getting no permission error. 

 

Any ideas? I don't plan to upload like this, I'm aware of rclone copy but sometimes I need to edit my files in the cloud. 

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.