Jump to content
Squid

[Plugin] CA Appdata Backup / Restore

354 posts in this topic Last Reply

Recommended Posts

A plugin designed to manually or on a schedule create backups of your Docker appdata (and Flash Drive and VM XML's)

 

Formerly part of Community Applications, this module is now packaged separately.

 

This plugin will automatically be installed with new installs of CA, but is now able to be uninstalled separately from CA, or installed separately (if CA is not installed)

 

Untitled.thumb.png.f2007b4d772df0b6e4a2e9c1d724dc92.png

 

To install this plugin separately, either go to CA's CA Modules section and install it from there.

 

If for some reason, you do not have CA installed on your system, then paste the following url into the Install Plugins section:

 

https://raw.githubusercontent.com/Squidly271/ca.backup/master/plugins/ca.backup.plg
 
Edited by Squid
  • Upvote 3

Share this post


Link to post

Changed my backup destination to a network share (UA mounted). Now I'm getting errors on backup-

2016/11/15 21:35:57 [4364] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]

 

Couldn't attach the full log. Wayyyyyy too big.

Share this post


Link to post

Changed my backup destination to a network share (UA mounted). Now I'm getting errors on backup-

2016/11/15 21:35:57 [4364] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]

 

Couldn't attach the full log. Wayyyyyy too big.

Nothing I can do about that.  Perhaps the destination filesystem doesn't support hardlinks, etc.  Either way, rsync is returning errors, so as far as ca is concerned, the backup failed.  You could try playing with the rsync options (google rsync man page), but no guarantees

 

EDIT:  If you load the log, then isolate the lines that have rsync in them, that's going to be the failure lines.  Would really only need a sampling of them for myself (or other people) to help out

Share this post


Link to post

Changed my backup destination to a network share (UA mounted). Now I'm getting errors on backup-

2016/11/15 21:35:57 [4364] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]

 

Couldn't attach the full log. Wayyyyyy too big.

Nothing I can do about that.  Perhaps the destination filesystem doesn't support hardlinks, etc.  Either way, rsync is returning errors, so as far as ca is concerned, the backup failed.  You could try playing with the rsync options (google rsync man page), but no guarantees

 

I suspect you're right. The destination disk is formatted HFS+. Google shows quite a few problems getting hardlinks preserved with HFS+, some workarounds, but they're way over my head. May just try a USB connected drive. What would be the best format?

 

 

 

EDIT:  If you load the log, then isolate the lines that have rsync in them, that's going to be the failure lines.  Would really only need a sampling of them for myself (or other people) to help out

 

I'll see what I can dig out of the log.

Share this post


Link to post

I suspect you're right. The destination disk is formatted HFS+. Google shows quite a few problems getting hardlinks preserved with HFS+, some workarounds, but they're way over my head. May just try a USB connected drive. What would be the best format?

All my array drives are XFS with no problems as a destination.  But UD adds another layer of complexity to it.  Best advice is try and see.  8)

Share this post


Link to post

Changed my backup destination to a network share (UA mounted). Now I'm getting errors on backup-

2016/11/15 21:35:57 [4364] rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]

 

Couldn't attach the full log. Wayyyyyy too big.

Nothing I can do about that.  Perhaps the destination filesystem doesn't support hardlinks, etc.  Either way, rsync is returning errors, so as far as ca is concerned, the backup failed.  You could try playing with the rsync options (google rsync man page), but no guarantees

 

EDIT:  If you load the log, then isolate the lines that have rsync in them, that's going to be the failure lines.  Would really only need a sampling of them for myself (or other people) to help out

Here's the excerpts from the log. Looks like a huge number of errors with symlinks.

Exported_Results.txt.zip

Share this post


Link to post

gummy.jpg

 

Added: Ability to have CA Backup automatically call CA Autoupdate to check for updates to (and install if selected within Auto Update Settings) docker applications after a backup is done.

Added: Ability to leave selected Docker Applications still running throughout the backup procedure.

 

The ability to not stop certain docker applications, you get to by hitting "Show Advanced Settings" on the main Backup Settings page.  Any app which you check off will not get stopped by the backup system.  This is an advanced setting, because in a perfect world you should also set that particular docker app's appdata folder to be excluded from the backup so that any files which happen to be open for writing etc at the time of backup are not going to mess up with the integrity of the backup.

 

* Side Note:  This feature actually now turns CA Appdata Backup / Restore into a general purpose backup / restore system for any particular share on your server as it is now possible to not have it stop any docker apps at all when running.  Not what its designed for, but it will now work outside of the box.

 

If CA Auto Update is also installed (and presumably up to date), then you also have the ability to have the backup system check for and then automatically install any updates available for the docker apps which you have selected under Auto Update Settings (Docker Section).  There's a new option of the Backup Settings page dealing with this. 

Presumably, since your appdata backup runs on a schedule, you could safely disable the schedule for docker auto updates.

 

* Thanks to CHBMB I can release this a week ahead of schedule  ;D

Edited by Squid

Share this post


Link to post

* Thanks to CHBMB I can release this a week ahead of schedule  ;D

 

Nice work mate, can I delete that container now then?

Share this post


Link to post

* Thanks to CHBMB I can release this a week ahead of schedule  ;D

 

Nice work mate, can I delete that container now then?

Sure

Share this post


Link to post

Hi...

 

I just updated the plugin to ver 2016.12.05 and when I go into it at the top is the following error...

 

Warning: array_diff(): Argument #2 is not an array in /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(292) : eval()'d code on line 103 Warning: natsort() expects parameter 1 to be array, null given in /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(292) : eval()'d code on line 104 Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(292) : eval()'d code on line 192

 

 

Yet it seems to run. 

 

 

 

 

Share this post


Link to post

Do you have docker enabled but no apps? I never saw that but will look at it tonight

 

Sent from my LG-D852 using Tapatalk

 

 

Share this post


Link to post

Oh no, I have apps!  That is why this plug-in is so outstanding!  It is/was greatly needed to protect these files vs rebuilding.

Share this post


Link to post

Oh no, I have apps!  That is why this plug-in is so outstanding!  It is/was greatly needed to protect these files vs rebuilding.

Quick look at the code and I think your array is stopped.

Share this post


Link to post

Nope...Sorry...Surely NOT stopped.  In fact I ran a test backup to see if it all still ran and it did.

Share this post


Link to post

Recently I replaced the Parity drive with a larger one and then replaced drive 5 with a larger drive.  But that is it array wise.

Share this post


Link to post

To save some time, can you post an actual screenshot showing that error, and diagnostics after the error appears.

 

I *think* somethings isn't quite right on your server, hence why I thought the array wasn't started.  (But I could be wrong  ;) )

Share this post


Link to post

BTW....I am running 6.1.9 of UnRAID.  So not sure if it is something with that and your code is looking for the newer version.

Share this post


Link to post

BTW....I am running 6.1.9 of UnRAID.  So not sure if it is something with that and your code is looking for the newer version.

Check for updates.  Was actually an error in there for a month or two.

Share this post


Link to post

All nice and clean. :)  I can even select a drive again to store to. (Not that I needed to change it, but realized I could not but now can.)

 

All good it seems.

 

THANKS!

Share this post


Link to post

I've just had a really bad failure with this plugin. I pressed the restore button (not fully intending to do one) and it prompts to continue or not, good. However I selected no (or cancel, I can't recall the exact working), but I made sure to select to not proceed. It went ahead anyway and started deleting appdata files, after few retries, the abort finally worked. But now it's fucked most of my dockers.

Share this post


Link to post

I've just had a really bad failure with this plugin. I pressed the restore button (not fully intending to do one) and it prompts to continue or not, good. However I selected no (or cancel, I can't recall the exact working), but I made sure to select to not proceed. It went ahead anyway and started deleting appdata files, after few retries, the abort finally worked. But now it's fucked most of my dockers.

Pretty sure that you hit OK instead of cancel.

Share this post


Link to post

I've just had a really bad failure with this plugin. I pressed the restore button (not fully intending to do one) and it prompts to continue or not, good. However I selected no (or cancel, I can't recall the exact working), but I made sure to select to not proceed. It went ahead anyway and started deleting appdata files, after few retries, the abort finally worked. But now it's fucked most of my dockers.

Pretty sure that you hit OK instead of cancel.

I haven't looked, but is it possible the dialog is a little ambiguous? If I hit NO on a proceed, followed by an OK on a dialog to stop the process, I expect the OK to confirm my previous NO. Wording and placement of confirmation second chance dialogs is tricky.

Share this post


Link to post

I've just had a really bad failure with this plugin. I pressed the restore button (not fully intending to do one) and it prompts to continue or not, good. However I selected no (or cancel, I can't recall the exact working), but I made sure to select to not proceed. It went ahead anyway and started deleting appdata files, after few retries, the abort finally worked. But now it's fucked most of my dockers.

Pretty sure that you hit OK instead of cancel.

 

Even if I did, there was nothing to restore, it shouldn't have gone forward and started deleting all my docker data for which there was no backups.

Share this post


Link to post
On 12/13/2016 at 8:59 AM, jonathanm said:

I haven't looked, but is it possible the dialog is a little ambiguous? If I hit NO on a proceed, followed by an OK on a dialog to stop the process, I expect the OK to confirm my previous NO. Wording and placement of confirmation second chance dialogs is tricky.

Not a double confirm.  In retrospect, the only change to it that I could have had from the get go is that clicking outside the popup results in an automatic cancel.

 

Untitled11_zpsuidp2ffu.png

Edited by Squid

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now