stop useless backups


Recommended Posts

Hi, since latest patch ive installed and enabled the my server thingy, but ive noticed the following

 

Quote

Aug 20 12:45:36 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 13:19:16 Unraid-Server emhttpd: read SMART /dev/sdi
Aug 20 13:45:43 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 14:21:17 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 20 14:26:53 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 20 14:38:56 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 20 14:45:50 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 15:26:55 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 20 15:36:48 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 20 15:45:56 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 16:15:00 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 20 16:36:49 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 20 16:46:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 16:47:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 16:48:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 17:16:31 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 20 17:16:31 Unraid-Server emhttpd: spinning down /dev/sdk
Aug 20 17:16:48 Unraid-Server emhttpd: read SMART /dev/sdk
Aug 20 17:17:12 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 20 17:45:10 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 20 18:18:45 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 20 18:45:16 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

 

So it seems to backup around 9 times a day? Why? No option or anything was changed. What about a check to see IF ANYTHING has changed before backup? Or just backup/upload once a day?

Edited by nuhll
Link to comment
4 minutes ago, ljm42 said:

 

Indeed, that is what it does.  The system detected that a file on the flashdrive changed, so it was backed up.

So i wonder what does it see as a change? I didnt changed anything or even logged into unraid the whole day. Only used VPN...?

Edited by nuhll
Link to comment

so what now? my server keeps making backups over and over, this doenst seem usefull... since you say this shouldnt happen, thats probably a bug? Where do i need to start to look?

 

Quote

Aug 21 18:46:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 18:47:03 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 19:33:54 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 21 19:45:10 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 19:47:36 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 21 19:47:36 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 21 20:45:17 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 21:00:28 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 21 21:00:28 Unraid-Server emhttpd: spinning down /dev/sdh
Aug 21 21:11:33 Unraid-Server emhttpd: read SMART /dev/sdi
Aug 21 21:35:57 Unraid-Server emhttpd: read SMART /dev/sdh
Aug 21 21:45:24 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 21:47:41 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 21 22:11:37 Unraid-Server emhttpd: cmd: /usr/local/emhttp/plugins/user.scripts/showLog.php test1
Aug 21 22:35:07 Unraid-Server emhttpd: read SMART /dev/sdf
Aug 21 22:35:14 Unraid-Server emhttpd: spinning down /dev/sdi
Aug 21 22:45:31 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 22:52:00 Unraid-Server emhttpd: shcmd (4179136): /usr/local/sbin/mover &> /dev/null &
Aug 21 22:58:41 Unraid-Server emhttpd: read SMART /dev/sdg
Aug 21 23:03:44 Unraid-Server emhttpd: read SMART /dev/sdd
Aug 21 23:45:38 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 21 23:50:31 Unraid-Server emhttpd: read SMART /dev/sdi
Aug 21 23:54:26 Unraid-Server emhttpd: read SMART /dev/sde
Aug 22 00:45:45 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

 

Ive uploaded a diagnostic, maybe its some of my cron jobs, but i dont think what exactly could be the problem.

unraid-server-diagnostics-20210822-0105.zip

Edited by nuhll
Link to comment

Flash backup uses git, so if you are familiar with git you can see what the changes were.

 

Open a web terminal and type:
  cd /boot
then:
  git show
to see what changed in the last commit.

 

If you type:
  git log
you'll see a list of commits. Grab the commit id and you can view any one of them:
  git show c2d9e4112dd9449b0eb98b0cd3775655cfd8a457

  • Like 1
Link to comment

This solution is designed to make a backup when files change. If you would you like to backup on a schedule you'll need a different solution.

 

But if you tell me the path to the file that is constantly changing, I'll tell you how to prevent it from being backed up.

Link to comment

This solution is not (good) working for a greater part of your community. Im asking you for a different solution, manual backups and or simple shedules instead of automatic doenst sound so crazy to implement...

 

Rclone is used by many users.

 

b/config/plugins/rclone/.rclone.conf

 

Quote

/usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 14:10:16 Unraid-Server emhttpd: spinning down /dev/sde
Aug 23 14:12:20 Unraid-Server emhttpd: spinning down /dev/sdi
Aug 23 14:46:06 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 14:47:06 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 15:10:00 Unraid-Server emhttpd: spinning down /dev/sdd
Aug 23 15:45:13 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 16:45:20 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 17:45:26 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 18:45:33 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 18:57:42 Unraid-Server emhttpd: spinning down /dev/sdj
Aug 23 19:03:53 Unraid-Server emhttpd: read SMART /dev/sdd
Aug 23 19:04:02 Unraid-Server emhttpd: read SMART /dev/sdj
Aug 23 19:05:36 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 23 19:45:40 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

Edited by nuhll
Link to comment

To prevent this file:
  /config/plugins/rclone/.rclone.conf
from triggering backups, open a webterminal and type:

echo ".rclone.conf" > /boot/config/plugins/rclone/.gitignore

 

The problem of course is that now your config file is not being backed up.

 

The real fix would be to update the plugin so that it does not write continuously to the flash drive.

 

@Waseh can you think of any options for the rclone plugin so that this data that changes regularly is not stored in the main config file? Ideally it would be stored in /tmp so there are not continuous writes to the flash drive. If it has to be stored on the flash, then adding a .gitignore file to the plugin will prevent it from triggering flash backup updates.

Link to comment

Had a chat with Waseh, super helpful.  They don't have access to this area of the forum currently, but we talked over DM and I summarized it over here:
  https://forums.unraid.net/topic/51633-plugin-rclone/page/37/?tab=comments#comment-1028139

 

 

Basically, the plugin has been updated to prevent this config file from triggering backups. That means the config file will no longer be backed up, so you'll want to grab a separate copy of it for disaster recovery purposes. Since the config file contains passwords and the flash backup isn't yet encrypted, probably best it isn't included in the flash backup at this point anyway.

Link to comment
  • 2 weeks later...

I dont know what to do.


You said its patched, but it still happens...? My plugin say its up to date. Is anything manual needed?

 

Date:   Thu Sep 2 20:46:06 2021 +0200

    Config change

diff --git a/config/plugins/rclone/.rclone.conf b/config/plugins/rclone/.rclone.conf
index 3e1d0d7..fe5739c 100644
--- a/config/plugins/rclone/.rclone.conf
+++ b/config/plugins/rclone/.rclone.conf

Edited by nuhll
Link to comment

Why would it be too bad to just set it to once a week or something like this? Or do you want to add things to ignore list the rest of your life? xD

 

Date:   Fri Sep 3 08:48:08 2021 +0200

    Config change

diff --git a/config/plugins/rclone/.rclone.conf b/config/plugins/rclone/.rclone.conf
index bebd7f0..1d177e0 100644
--- a/config/plugins/rclone/.rclone.conf
+++ b/config/plugins/rclone/.rclone.conf

Edited by nuhll
Link to comment

root@Unraid-Server:/boot# git -C /boot rm --cached /boot/config/plugins/rclone/.rclone.conf
rm 'config/plugins/rclone/.rclone.conf'

 

was a little bit shocked bc it looked like it did rm 'config/plugins/rclone/.rclone.conf' but its still there...

will report back if it works.

Edited by nuhll
Link to comment

root@Unraid-Server:/boot# git show
commit 1b19d5968a7a5f7e04208edabd244530a4fbad7a (HEAD -> master, origin/master)
Author: gitbot <gitbot@unraid.net>
Date:   Sun Sep 5 11:46:07 2021 +0200

    Config change

diff --git a/config/plugins/rclone/.rclone.conf b/config/plugins/rclone/.rclone.conf
index 13515a6..96e316d 100644
--- a/config/plugins/rclone/.rclone.conf
+++ b/config/plugins/rclone/.rclone.conf

Edited by nuhll
Link to comment

did i do something wrong?

 

Quote

Sep 7 16:45:11 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 17:45:17 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 18:40:05 Unraid-Server kernel: veth9bb0f53: renamed from eth0
Sep 7 18:40:05 Unraid-Server kernel: docker0: port 7(veth295f12e) entered disabled state
Sep 7 18:40:05 Unraid-Server avahi-daemon[14701]: Interface veth295f12e.IPv6 no longer relevant for mDNS.
Sep 7 18:40:05 Unraid-Server avahi-daemon[14701]: Leaving mDNS multicast group on interface veth295f12e.IPv6 with address fe80::9023:67ff:fef1:1a56.
Sep 7 18:40:05 Unraid-Server kernel: docker0: port 7(veth295f12e) entered disabled state
Sep 7 18:40:05 Unraid-Server kernel: device veth295f12e left promiscuous mode
Sep 7 18:40:05 Unraid-Server kernel: docker0: port 7(veth295f12e) entered disabled state
Sep 7 18:40:05 Unraid-Server avahi-daemon[14701]: Withdrawing address record for fe80::9023:67ff:fef1:1a56 on veth295f12e.
Sep 7 18:42:16 Unraid-Server emhttpd: spinning down /dev/sdi
Sep 7 18:44:04 Unraid-Server emhttpd: spinning down /dev/sdk
Sep 7 18:44:31 Unraid-Server emhttpd: spinning down /dev/sdg
Sep 7 18:45:24 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 19:45:31 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 19:56:43 Unraid-Server emhttpd: spinning down /dev/sde
Sep 7 20:45:38 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 21:26:30 Unraid-Server webGUI: Successful login user root from 192.168.86.102
Sep 7 21:45:44 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Sep 7 22:45:51 Unraid-Server flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

Edited by nuhll
Link to comment

Alright, I think I've got this figured out. The system is not happy that we are running commands manually but let's try this:

 

/etc/rc.d/rc.flash_backup stop
git -C /boot rm --cached /boot/config/plugins/rclone/.rclone.conf
git -C /boot commit -m "remove .rclone.conf"
git -C /boot push
/etc/rc.d/rc.flash_backup start

 

This should prevent the .rclone.conf file from being tracked.

Link to comment
  • 1 month later...
On 9/8/2021 at 9:15 PM, ljm42 said:

Alright, I think I've got this figured out. The system is not happy that we are running commands manually but let's try this:

 

/etc/rc.d/rc.flash_backup stop
git -C /boot rm --cached /boot/config/plugins/rclone/.rclone.conf
git -C /boot commit -m "remove .rclone.conf"
git -C /boot push
/etc/rc.d/rc.flash_backup start

 

This should prevent the .rclone.conf file from being tracked.

 

My system was suffering from this condition. I could not execute this fix quoted above due to file locks. I was able to resolve the problem by de-activating the flash backup in the MyServers plug in, rebooting, and re-activating the flash backup. I hope this fix sticks. 🙂

  • Like 2
Link to comment
  • 2 weeks later...
On 10/17/2021 at 9:16 PM, kp74508 said:

 

My system was suffering from this condition. I could not execute this fix quoted above due to file locks. I was able to resolve the problem by de-activating the flash backup in the MyServers plug in, rebooting, and re-activating the flash backup. I hope this fix sticks. 🙂

This stooped it for me, thank you.

Link to comment
On 10/18/2021 at 11:54 PM, Andiroo2 said:

I prefer a backup on any file change, even if that’s 10x per day.  If my flash drive dies, I want to know that the latest config is available to restore.  Not one from a week ago before I made some change (and you KNOW that would happen right before the drive dies…).

It creates unneccessary writes/reads and uploads which "kills your hardware sooner" im not asking to remove the automatic part, just a option to use another way like once a week or something, usually you dont change your unraid config every hour...?

 

Anyway, i tried to activate it again (after restart and update to latest version), but it just circles forever. 

 

Nov 7 00:21:48 Unraid-Server nginx: 2021/11/07 00:21:48 [error] 10875#10875: *63798 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.86.102, server: ....unraid.net, request: "POST /plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "....unraid.net:444", referrer: "https://...unraid.net:444/Settings/ManagementAccess"

 

edit: nvm, it finished after the 3. try while posting this i still would like to have that file backuped, and would prefer a "once a week" or so option... 

Edited by nuhll
Link to comment