RClone not saved on reboot

[Jules]

Hi,

 

I'm building a server and I'm starting simple at the moment, trailing unRaid as the OS.

 

I'm trying to set it up and see how I go before purchasing.

 

So far I was moderately successful in setting up all the Dockers and plugins, etc.

 

I got to the stage where I got rclone using this awesome guide. The problem that I have now is that every time I reboot the server, the rclone's settings are kept but the plugin itself is gone. I have to reinstall the plugin every time and run it as per normal as the settings are already there.

 

I want to mention that when I installed rclone, I've also installed krusader and user scripts, both of which are still installed and working fine. I've reinstalled rclone and keeps uninstalling on reboot. Tried it about 5 times last night, with no success.

 

Any help would be greatly appreciated.

 

[DZMM]

Is your server connected to the internet when it's starting?  The plugin needs to download rclone at every boot.  This causes me a problem as my server is also my router (pfsense vm) so i install the plugin a few mins after boot via a script, and uninstall it at shutdown via a script.

 

[Jules]

Thank you for your reply DZMM.

 

I didn't know that plugins are installed at every reboot.

 

I'm tipping that dockers are permanently installed?

 

With saying that, how come

• Auto Update Applications
• Backup/Restore Appdata
• Cleanup Appdata
• Fix Common Problems
• User Scripts

are not uninstalled on reboot?

I'm sure there must be a way for rclone and other similar plugins to stay installed. I'm worried that with time, I will forget what I've installed and will miss them on one of the reboots.

[pwm]

Note that lots of binary files are unpacked from the USB drive and placed in your RAM file system.

And the USB drive has a directory tree with scripted plugins that are also copied to RAM.

 

But unRAID doesn't have a way to patch the files on the USB drive to include additional system-level applications that will on next boot be copied to the RAM file system.

So instead, a script has to install additional binary tools into the RAM file system after the system has booted.

And have to repeat it on every boot, since the RAM file system is regenerated on every boot.

[Jules]

@pwm, thank you for your reply. It helps me better understand the issue.

 

Would anyone be kind enough to provide a link to a script or a script for automation the installation of the plugin on startup?

 

I'm not very familiar with unRaid, but if this was say debian based, I would have worked my way around it. Not sure what's the unRaid equivalent to "apt-get install".

[Squid]

The problem is that if the program itself (not the plugin per se) insists on downloading the binary, then internet access at time of plugin install is required.  99.9% of the plugins out there for unRaid do not suffer this fate, even if they have binary dependencies, as they are all saved on the flash drive.

 

But, to answer your particular question

 

https://lime-technology.com/forums/topic/72240-solved-is-there-a-way-to-installuninstall-plugins-from-script/?tab=comments#comment-664116

 

[Jules]

15 minutes ago, Squid said:

The problem is that if the program itself (not the plugin per se) insists on downloading the binary, then internet access at time of plugin install is required.  99.9% of the plugins out there for unRaid do not suffer this fate, even if they have binary dependencies, as they are all saved on the flash drive.

 

But, to answer your particular question

 

https://lime-technology.com/forums/topic/72240-solved-is-there-a-way-to-installuninstall-plugins-from-script/?tab=comments#comment-664116

 

 

Wow @Squid that was fast answer. Thank you. Your answer(link) will alleviate part of the problem.

 

Looking around, found out that unRaid based on Slackware. So that answered part of my question.

 

Having installed last night 3 other plugins, nerd pack, button and cpu temp, after rebooting, observed that those are still there, whilst the rclone is gone. I'm stumped by this behavior. Looking through the logs, it doesn't appear to be anything out of place. No errors, no warnings, nothing.

 

Ill try and use your solution for now and see how that goes.

 

What happens if on start there is no internet available?

[Squid]

3 minutes ago, Jules said:

What happens if on start there is no internet available?

Then it won't download.  Hence the 5 minute delay after array start to reinstall the plugin.

 

I looked at the .plg the other day, and it *appears* that it should save anything downloaded, but since I'm not the author, I'm obviously not in control of it.

 

 

[DZMM]

#!/bin/bash

#######  Check if script already running  ##########

if [[ -f "/mnt/user/software/rclone_install_running" ]]; then

exit
else

touch /mnt/user/software/rclone_install_running

fi


plugin install https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg

rm /mnt/user/software/rclone_install_running

exit

Shortened version of what I run every 5 mins after startup - I do this as my longer version also mounts drives, so they get remounted if something goes wrong.

 

Uninstall script runs at array stop:

 

#!/bin/bash

plugin remove rclone.plg

if [[ -f "/mnt/user/software/rclone_install_running" ]]; then
rm /mnt/user/software/rclone_install_running
echo "install running - removing dummy file"
else
echo "Passed: install already exited properly"
fi

exit

 

Edited June 22, 2018 by DZMM

[Jules]

4 hours ago, DZMM said:

#!/bin/bash

#######  Check if script already running  ##########

if [[ -f "/mnt/user/software/rclone_install_running" ]]; then

exit
else

touch /mnt/user/software/rclone_install_running

fi


plugin install https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg

rm /mnt/user/software/rclone_install_running

exit

Shortened version of what I run every 5 mins after startup - I do this as my longer version also mounts drives, so they get remounted if something goes wrong.

 

Uninstall script runs at array stop:

 

#!/bin/bash

plugin remove rclone.plg

if [[ -f "/mnt/user/software/rclone_install_running" ]]; then
rm /mnt/user/software/rclone_install_running
echo "install running - removing dummy file"
else
echo "Passed: install already exited properly"
fi

exit

 

Legend!

[DZMM]

Just realised that I cut too much out of my script as you don't have the bit that checks if rclone is already installed - just the bit that checks if the script is already running.  I check if rclone is already running by seeing if my rclone cache mount is there:

 

#!/bin/bash

#######  Check if script already running  ##########

if [[ -f "/mnt/user/software/rclone_install_running" ]]; then

exit
else

touch /mnt/user/software/rclone_install_running

fi

#######  Check if rclone already installed/mounted  ##########

if [[ -f "/mnt/disks/google_media/tv_adults_gd/mountcheck" ]] && [[ -f "/mnt/disks/google_media/tv_kids_gd/mountcheck" ]] && [[ -f "/mnt/disks/google_media/movies_adults_gd/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check rclone mounted success."
else

plugin install https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg

#######  Mount rclone   ###########  

rclone mount --allow-other --dir-cache-time=71h --cache-db-path=/tmp/rclone --cache-chunk-size=10M --cache-total-chunk-size=10G --cache-info-age=72h --cache-db-purge --cache-workers=6 --cache-chunk-no-memory --cache-tmp-upload-path=/mnt/user/rclone_upload --cache-tmp-wait-time=75m --buffer-size 0M --log-level INFO gdrive_media: /mnt/disks/google_media &

fi

#######  Mount Fusion   ##########

# check if mounted

if [[ -f "/mnt/disks/fusion/tv_adults_fuse/mountcheck" ]] && [[ -f "/mnt/disks/fusion/tv_kids_fuse/mountcheck" ]] && [[ -f "/mnt/disks/fusion/movies_adults_fuse/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, Fusion Series and Movies fuse mounted."
exit

else

sleep 15

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/tv_adults_upload=RW:/mnt/disks/google_media/tv_adults_gd=RO /mnt/disks/fusion/tv_adults_fuse
unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/tv_kids_upload=RW:/mnt/disks/google_media/tv_kids_gd=RO /mnt/disks/fusion/tv_kids_fuse
unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/movies_adults_upload=RW:/mnt/disks/google_media/movies_adults_gd=RO /mnt/disks/fusion/movies_adults_fuse

if [[ -f "/mnt/disks/fusion/tv_adults_fuse/mountcheck" ]] && [[ -f "/mnt/disks/fusion/tv_kids_fuse/mountcheck" ]] && [[ -f "/mnt/disks/fusion/movies_adults_fuse/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Remount of Series and Movies successful."
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: Remount failed."
fi

fi

rm /mnt/user/software/rclone_install_running

exit

I have a seperate rclone move script that moves files added to the local *_upload directories to the *_gd rclone equivalents every 30 mins. 

Edited June 22, 2018 by DZMM

[Squid]

Got around to debugging the .plg file

 

From the changelog :

Quote

2017.09.23
- Always pull newest version on install

 

Whether the side effect of a no-install at boot up time is intended or not, this is how the plugin file is designed.  There are a couple of ways to work around this and still have the plugin install if @Waseh contacts me.

 

In the meantime, a comment has been placed on rclone and rclone-beta in CA and FCP

 

[Nyghthawk]

Yup. Hopefully. As the beta.rclone.org is DOWN, so therefor the rclone plugin is DOWN after a reboot.

[Jules]

mmm that's interesting and not so encouraging.

 

So if the NAS is on LAN, doing its thing, there's no rclone option to backup/sync local computers or other nas servers within the LAN?

[Nyghthawk]

Just now, Jules said:

mmm that's interesting and not so encouraging.

 

So if the NAS is on LAN, doing its thing, there's no rclone option to backup/sync local computers or other nas servers within the LAN?

I removed the plugin since it was giving me an error, and never encountered this before, and when I go to install it again i get

 

plugin: installing: https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg
plugin: downloading https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg
plugin: downloading: https://raw.githubusercontent.com/Waseh/rclone-unraid/beta/plugin/rclone.plg ... done
plugin: downloading: http://beta.rclone.org/rclone-beta-latest-linux-amd64.zip ... failed (Network failure)
plugin: wget: http://beta.rclone.org/rclone-beta-latest-linux-amd64.zip download failure (Network failure)

 

all it does. So I went tot he beta.rclone.org and sure enough. Cannot connect, its down. I might try and see if there is a local way of doing this. So that maybe weekly, I can grab the latest version store it locally and have the plugin "install" from local source which will always be there regardless of rclones site status. I probably just added hours and hours of headaches by removing the plugin and not letting it do its thing. 

[pwm]

People should really think twice about using installation models that automatically grabs new software from remote servers.

 

If the remote server gets hacked, then anyone who uses auto-fetch will get burned. Servers hosting projects with auto-update are prioritized targets for hackers, since one good weekend can give them tens of thousands or more owned machines.

 

A good model for handling plugins is to only allow manual updates - and to check reviews of the relevant version before fetching.

 

In this case, it would be good to modify the plugin to retrieve and install local files.

[pimogo]

On 6/22/2018 at 4:49 AM, DZMM said:

####### Check if script already running ##########

 

if [[ -f "/mnt/user/software/rclone_install_running" ]]; then

 

So i got the script working, now onto the next bit..

 

i'm trying to check if rclone is running but i get a directory not found error.. which is understandable considering i never scripted the installation of directory above.. i.e. rclone_install_running.. how did you create this directory?

 

thanks again for all your help.

 

p.s. whats the best method to start script 5 mins after start of array. i have a VM Pfsense box that needs to become active before...

 

thanks!

[DZMM]

16 minutes ago, pimogo said:

 

So i got the script working, now onto the next bit..

 

i'm trying to check if rclone is running but i get a directory not found error.. which is understandable considering i never scripted the installation of directory above.. i.e. rclone_install_running.. how did you create this directory?

 

thanks again for all your help.

 

p.s. whats the best method to start script 5 mins after start of array. i have a VM Pfsense box that needs to become active before...

 

thanks!

 

you can set a custom schedule via the user scripts plugin or put 'sleep 300' or 'sleep 5m' in your script to add a 5 min pause 

Edited August 18, 2018 by DZMM

[vw-kombi]

Been running this on my new unraid system for  1 month.  Rebooted two days ago and only now found out that this does not stay thee after re-boot.

I can see it on the plugin description now.

I was following spaceinvaders guide - he made no mention of that in the vid.

 

So I re-installed the plugin and I can see it in settings, but the rclone command in the ssh window is still saying command not found......

 

Maybe something wrong with the plugin installer as when I retry, I get this :

 

plugin: installing: https://raw.githubusercontent.com/Waseh/rclone-unraid/master/plugin/rclone.plg
plugin: downloading https://raw.githubusercontent.com/Waseh/rclone-unraid/master/plugin/rclone.plg
plugin: downloading: https://raw.githubusercontent.com/Waseh/rclone-unraid/master/plugin/rclone.plg ... done
plugin: downloading: http://downloads.rclone.org/rclone-current-linux-amd64.zip ...

 

I am off on holiday tomorrow early so this is the worst time for this to happen - as if someone comes in to steal my stuff, they will take the computer also - and no video's of them doing this will have been uploaded to gdrive.

 

Is this the only way to sync some video footage to Gdrive ?  realtime is best (like I had on windows) but every 5 minutes if not ?

 

While it has been rock solid since I set it up, this failure now has left me wanting to investigate other solutions.