stop useless backups


Recommended Posts

Hi, since latest patch ive installed and enabled the my server thingy, but ive noticed the following

 

Quote

Aug 20 12:45:36 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 13:19:16 Unraid-Server emhttpd: read SMART /dev/sdi
Aug 20 13:45:43 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 14:21:17 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 20 14:26:53 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 20 14:38:56 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 20 14:45:50 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 15:26:55 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 20 15:36:48 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 20 15:45:56 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 16:15:00 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 20 16:36:49 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 20 16:46:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 16:47:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 16:48:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 17:16:31 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 20 17:16:31 Unraid-Server emhttpd: spinning down /dev/sdk
Aug 20 17:16:48 Unraid-Server emhttpd: read SMART /dev/sdk
Aug 20 17:17:12 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 20 17:45:10 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 18:18:45 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 20 18:45:16 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

 

So it seems to backup around 9 times a day? Why? No option or anything was changed. What about a check to see IF ANYTHING has changed before backup? Or just backup/upload once a day?

Edited by nuhll
Link to comment
45 minutes ago, nuhll said:

What about a check to see IF ANYTHING has changed before backup?

 

Indeed, that is what it does.  The system detected that a file on the flashdrive changed, so it was backed up.

Link to comment
Posted (edited)
4 minutes ago, ljm42 said:

 

Indeed, that is what it does.  The system detected that a file on the flashdrive changed, so it was backed up.

So i wonder what does it see as a change? I didnt changed anything or even logged into unraid the whole day. Only used VPN...?

Edited by nuhll
Link to comment
Posted (edited)

so what now? my server keeps making backups over and over, this doenst seem usefull... since you say this shouldnt happen, thats probably a bug? Where do i need to start to look?

 

Quote

Aug 21 18:46:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 18:47:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 19:33:54 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 21 19:45:10 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 19:47:36 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 21 19:47:36 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 21 20:45:17 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 21:00:28 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 21 21:00:28 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 21 21:11:33 Unraid-Server emhttpd: read SMART /dev/sdi
Aug 21 21:35:57 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 21 21:45:24 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 21:47:41 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 21 22:11:37 Unraid-Server emhttpd: cmd: /usr/local/emhttp/plugins/user.scripts/showLog.php test1
Aug 21 22:35:07 Unraid-Server emhttpd: read SMART /dev/sdf
Aug 21 22:35:14 Unraid-Server emhttpd: spinning down /dev/sdi
Aug 21 22:45:31 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 22:52:00 Unraid-Server emhttpd: shcmd (4179136): /usr/local/sbin/mover &> /dev/null &
Aug 21 22:58:41 Unraid-Server emhttpd: read SMART /dev/sdg
Aug 21 23:03:44 Unraid-Server emhttpd: read SMART /dev/sdd
Aug 21 23:45:38 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 23:50:31 Unraid-Server emhttpd: read SMART /dev/sdi
Aug 21 23:54:26 Unraid-Server emhttpd: read SMART /dev/sde
Aug 22 00:45:45 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

 

Ive uploaded a diagnostic, maybe its some of my cron jobs, but i dont think what exactly could be the problem.

unraid-server-diagnostics-20210822-0105.zip

Edited by nuhll
Link to comment

Flash backup uses git, so if you are familiar with git you can see what the changes were.

 

Open a web terminal and type:
  cd /boot
then:
  git show
to see what changed in the last commit.

 

If you type:
  git log
you'll see a list of commits. Grab the commit id and you can view any one of them:
  git show c2d9e4112dd9449b0eb98b0cd3775655cfd8a457

  • Like 1
Link to comment
Posted (edited)

Thanks.

 

Its the rclone config, it changes the access token from time to time...

 

and now? :/

 

can we have a static config for like backup once a week/month?

Edited by nuhll
Link to comment

This solution is designed to make a backup when files change. If you would you like to backup on a schedule you'll need a different solution.

 

But if you tell me the path to the file that is constantly changing, I'll tell you how to prevent it from being backed up.

Link to comment
Posted (edited)

This solution is not (good) working for a greater part of your community. Im asking you for a different solution, manual backups and or simple shedules instead of automatic doenst sound so crazy to implement...

 

Rclone is used by many users.

 

b/config/plugins/rclone/.rclone.conf

 

Quote

/usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 14:10:16 Unraid-Server emhttpd: spinning down /dev/sde
Aug 23 14:12:20 Unraid-Server emhttpd: spinning down /dev/sdi
Aug 23 14:46:06 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 14:47:06 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 15:10:00 Unraid-Server emhttpd: spinning down /dev/sdd
Aug 23 15:45:13 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 16:45:20 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 17:45:26 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 18:45:33 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 18:57:42 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 23 19:03:53 Unraid-Server emhttpd: read SMART /dev/sdd
Aug 23 19:04:02 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 23 19:05:36 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 19:45:40 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

Edited by nuhll
Link to comment

To prevent this file:
  /config/plugins/rclone/.rclone.conf
from triggering backups, open a webterminal and type:

echo ".rclone.conf" > /boot/config/plugins/rclone/.gitignore

 

The problem of course is that now your config file is not being backed up.

 

The real fix would be to update the plugin so that it does not write continuously to the flash drive.

 

@Waseh can you think of any options for the rclone plugin so that this data that changes regularly is not stored in the main config file? Ideally it would be stored in /tmp so there are not continuous writes to the flash drive. If it has to be stored on the flash, then adding a .gitignore file to the plugin will prevent it from triggering flash backup updates.

Link to comment

Had a chat with Waseh, super helpful.  They don't have access to this area of the forum currently, but we talked over DM and I summarized it over here:
  https://forums.unraid.net/topic/51633-plugin-rclone/page/37/?tab=comments#comment-1028139

 

 

Basically, the plugin has been updated to prevent this config file from triggering backups. That means the config file will no longer be backed up, so you'll want to grab a separate copy of it for disaster recovery purposes. Since the config file contains passwords and the flash backup isn't yet encrypted, probably best it isn't included in the flash backup at this point anyway.

Link to comment
  • 2 weeks later...

I dont know what to do.


You said its patched, but it still happens...? My plugin say its up to date. Is anything manual needed?

 

Date:   Thu Sep 2 20:46:06 2021 +0200

    Config change

diff --git a/config/plugins/rclone/.rclone.conf b/config/plugins/rclone/.rclone.conf
index 3e1d0d7..fe5739c 100644
--- a/config/plugins/rclone/.rclone.conf
+++ b/config/plugins/rclone/.rclone.conf

Edited by nuhll
Link to comment

Why would it be too bad to just set it to once a week or something like this? Or do you want to add things to ignore list the rest of your life? xD

 

Date:   Fri Sep 3 08:48:08 2021 +0200

    Config change

diff --git a/config/plugins/rclone/.rclone.conf b/config/plugins/rclone/.rclone.conf
index bebd7f0..1d177e0 100644
--- a/config/plugins/rclone/.rclone.conf
+++ b/config/plugins/rclone/.rclone.conf

Edited by nuhll
Link to comment

OK .gitignore will stop a file from being added, but if it is already in the repo it doesn't prevent it from being tracked. So we need to remove it from the repo. Please run this:
 

git -C /boot rm --cached /boot/config/plugins/rclone/.rclone.conf

 

Link to comment

root@Unraid-Server:/boot# git -C /boot rm --cached /boot/config/plugins/rclone/.rclone.conf
rm 'config/plugins/rclone/.rclone.conf'

 

was a little bit shocked bc it looked like it did rm 'config/plugins/rclone/.rclone.conf' but its still there...

will report back if it works.

Edited by nuhll
Link to comment

root@Unraid-Server:/boot# git show
commit 1b19d5968a7a5f7e04208edabd244530a4fbad7a (HEAD -> master, origin/master)
Author: gitbot <gitbot@unraid.net>
Date:   Sun Sep 5 11:46:07 2021 +0200

    Config change

diff --git a/config/plugins/rclone/.rclone.conf b/config/plugins/rclone/.rclone.conf
index 13515a6..96e316d 100644
--- a/config/plugins/rclone/.rclone.conf
+++ b/config/plugins/rclone/.rclone.conf

Edited by nuhll
Link to comment

did i do something wrong?

 

Quote

Sep 7 16:45:11 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 17:45:17 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 18:40:05 Unraid-Server kernel: veth9bb0f53: renamed from eth0
Sep 7 18:40:05 Unraid-Server kernel: docker0: port 7(veth295f12e) entered disabled state
Sep 7 18:40:05 Unraid-Server avahi-daemon[14701]: Interface veth295f12e.IPv6 no longer relevant for mDNS.
Sep 7 18:40:05 Unraid-Server avahi-daemon[14701]: Leaving mDNS multicast group on interface veth295f12e.IPv6 with address fe80::9023:67ff:fef1:1a56.
Sep 7 18:40:05 Unraid-Server kernel: docker0: port 7(veth295f12e) entered disabled state
Sep 7 18:40:05 Unraid-Server kernel: device veth295f12e left promiscuous mode
Sep 7 18:40:05 Unraid-Server kernel: docker0: port 7(veth295f12e) entered disabled state
Sep 7 18:40:05 Unraid-Server avahi-daemon[14701]: Withdrawing address record for fe80::9023:67ff:fef1:1a56 on veth295f12e.
Sep 7 18:42:16 Unraid-Server emhttpd: spinning down /dev/sdi
Sep 7 18:44:04 Unraid-Server emhttpd: spinning down /dev/sdk
Sep 7 18:44:31 Unraid-Server emhttpd: spinning down /dev/sdg
Sep 7 18:45:24 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 19:45:31 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 19:56:43 Unraid-Server emhttpd: spinning down /dev/sde
Sep 7 20:45:38 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 21:26:30 Unraid-Server webGUI: Successful login user root from 192.168.86.102
Sep 7 21:45:44 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 22:45:51 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

Edited by nuhll
Link to comment

Alright, I think I've got this figured out. The system is not happy that we are running commands manually but let's try this:

 

/etc/rc.d/rc.flash_backup stop
git -C /boot rm --cached /boot/config/plugins/rclone/.rclone.conf
git -C /boot commit -m "remove .rclone.conf"
git -C /boot push
/etc/rc.d/rc.flash_backup start

 

This should prevent the .rclone.conf file from being tracked.

Link to comment