Jump to content
Squid

[Plugin] CA Appdata Backup / Restore v2

356 posts in this topic Last Reply

Recommended Posts

I made a manual backup to app data and flash drive to a external HDD, i set use compression to YES, but when i looked on my HDD the Flash drive is not compress, is compression not working if i use a external HDD ? 

Share this post


Link to post
46 minutes ago, Danuel said:

I made a manual backup to app data and flash drive to a external HDD, i set use compression to YES, but when i looked on my HDD the Flash drive is not compress, is compression not working if i use a external HDD ? 

 

Share this post


Link to post
4 hours ago, trurl said:

 

then something changed because when i start with Unraid 30 days ago and made the first backup was archived .. :D, but glad to know now that is not from my end 

Edited by Danuel

Share this post


Link to post
Posted (edited)

Hey @Squid

 

Nice work on this plugin!

I was going to roll something similar as a script before I found this. Nice Job.

 

Bug:

Docker img (libvirt) is not backed up if its not in the right location (which I cant find documented anywhere) or named correctly

I use  /mnt/cache/docker.img as I setup docker ages before I found this plugin.

 a field to allow customisation of the libvirt source image would be ideal

 

Comment on functionality:

The "Delete backups if they are this many days old" doesn't make a lot of sense when used with anything other than a daily schedule.

Eample scenario

Initial Setup

Set "Delete after"  to 5

Set "Schedule" to Daily

Outcome: 5 backups kept

 

Changed my mind and I want weekly backups now

Set "Schedule" to weekly

Forgot to calculate how many days I want (7/14) to keep relative to the schedule type so it remains at 5

Outcome: the weekly backup is erased after 5 days leaving 2 days before the next scheduled backup takes place.

 

Worse again

Changed the "Schedule" to monthly and forgot to adjust the days to keep (31/62) so it stays at 5.

Outcone: a Backup is created at the specified monthly interval and is then sebsequently erased after 5 days leaving 3/4 of the month with no recovery option.

 

A possibly better and simpler solution to this issue would be to change the "Delete after" to "Backups to keep"

in this way it would not matter about the schedule at all as you always know there are X daily/weekly/monthly backups at any given time.

 

I did actually run into this exact scenario (the weekly one) and thought you should know.

 

Keep up the great work

 

Edited by Loophole

Share this post


Link to post
32 minutes ago, Loophole said:

Docker img (libvirt) is not backed up if its not in the right location (which I cant find documented anywhere) or named correctly

I use  /mnt/cache/docker.img as I setup docker ages before I found this plugin.

 a field to allow customisation of the libvirt source image would be ideal

 

Docker img is NOT (libvirt)

 

Libvirt is for the VMs

 

There is no good reason to backup docker image since the containers can just be redownloaded. The only thing that matters is the appdata and the templates. The templates are the settings you made such as mappings when you installed a docker. These are on flash so make sure you are backing up flash. In fact it's probably better if you just recreate docker image instead of even attempting to back it up. You can always reinstall your containers exactly as they were using the Previous Apps feature on the Apps page, as long as you have the templates and appdata.

Share this post


Link to post
3 minutes ago, trurl said:

Docker img is NOT (libvirt)

 

Libvirt is for the VMs

 

There is no good reason to backup docker image since the containers can just be redownloaded. The only thing that matters is the appdata and the templates. The templates are the settings you made such as mappings when you installed a docker. These are on flash so make sure you are backing up flash. In fact it's probably better if you just recreate docker image instead of even attempting to back it up. You can always reinstall your containers exactly as they were using the Previous Apps feature on the Apps page, as long as you have the templates and appdata.

Oh I see
Looks like I was confusing them (Docker & VM in general)

Yep definitely backing up the flash, checked that before


thanks for the clarification

Share this post


Link to post
Posted (edited)

So i need to flash my raid controller to jbod mode and so i'm trying to make a simple backup of my appdata for my rebuild

 

I first tried to backup using your tool to windows smb shares that i have mounted with the unassigned devices tool - i had previously had issues torrenting to the smb shares (be they drivepool or normal hdd) as without fail some torrents on recheck would be missing part of their data

 

A similar thing is happening with this backup tool - i have done both compressed and not compressed - the compressed tar.gz open as corrupt and the regular uncompressed tar say they are the right size and then show only one folder from appdata

Backing up to an unraid share works perfectly fine and is a similar size to the broken tar's on the windows smb share

 

I feel like this is something i am missing but i would appreciate any help @Squid

 

EDIT:  Also the verify runs on the backups and finds no problems - so thats also strange

Edited by mcrommert

Share this post


Link to post

Not sure.  You could try the backup to the SMB, then change the SOURCE to another share on your array and then run the restore operation and see what winds up in there.

Share this post


Link to post
Posted (edited)

As always, thanks for your addons, makes management 1000% better.....

 

Request:

 

Backup - detect extra docker networks and backup their info

Restore - recreate extra docker networks before restore

 

I had just corrupted my cache drive some how and did a restore  (went well) - but some dockers were not working, took me a bit to figure out they were on a custom network for my reverse proxies stuff.

 

Had to manually create network, edit each docker to change to network to something else and back to custom network so it would re save and restart docker to use custom network (just restarting would not make it use custom network).

 

Thanks for the consideration of this request.

 

Myk

 

Edited by MyKroFt

Share this post


Link to post

This plugin critically fails to backup poste, WITHOUT USER NOTIFICATION

 

Unfortunately,  this has gutted my email system as I deleted the appdata folder for it after a change didn't work and was going to restore from backups.

 

As far as I can tell, files and folders with mail group and mem user are what's missed:
image.png.5fa49c4abbb760c623b447d8ddcac55d.png

Share this post


Link to post

Thankfully that was a horribly false alert. It seems I made the mistake of copying another folder that had those files missing instead.
I was from an instance that broke that I could never narrow down exactly what cause it.

Checking the backups, I can confirm the correct folder was backed up safely, sorry for the trouble.

Now I can relax with it back up >.<

Share this post


Link to post

Is there anything special I have to do to get Start Scripts to work or should a script that works as a UserScript just work?

 

Its says: 
Executing custom start script

 

But its not running or at least not doing what it should.

 

The script syncs the backup to google drive and works fine when I just start it from user scripts.

 

Share this post


Link to post
2 hours ago, TeKo said:

Is there anything special I have to do to get Start Scripts to work or should a script that works as a UserScript just work?

 

Its says: 
Executing custom start script

 

But its not running or at least not doing what it should.

 

The script syncs the backup to google drive and works fine when I just start it from user scripts.

 

User scripts is far more forgiving on scripts than backup.

 

Notable differences:

  • The "interpreter line" ( ie: #!/bin/bash ) has to be present in backup
  • Line endings must be linux style, and not Windows style (ie: use Notepad++ to edit the script and set EOL to be unix/linux)

Share this post


Link to post
2 hours ago, Squid said:

User scripts is far more forgiving on scripts than backup.

 

Notable differences:

  • The "interpreter line" ( ie: #!/bin/bash ) has to be present in backup
  • Line endings must be linux style, and not Windows style (ie: use Notepad++ to edit the script and set EOL to be unix/linux)

Thanks, looks like the EOLs were the problem.

Share this post


Link to post
On 3/30/2019 at 12:00 AM, MyKroFt said:

As always, thanks for your addons, makes management 1000% better.....

 

Request:

 

Backup - detect extra docker networks and backup their info

Restore - recreate extra docker networks before restore

 

I had just corrupted my cache drive some how and did a restore  (went well) - but some dockers were not working, took me a bit to figure out they were on a custom network for my reverse proxies stuff.

 

Had to manually create network, edit each docker to change to network to something else and back to custom network so it would re save and restart docker to use custom network (just restarting would not make it use custom network).

 

Thanks for the consideration of this request.

 

Myk

 

Any consideration on this?

 

Thanks

Myk

Share this post


Link to post

Would the "Path To Custom Start Script:" field allow for a script to be run before all docker containers are started back up?  My goal is to run a backup, then ryn an rsync of all appdata to a remote server which needs to take place while also the containers are offline.

Share this post


Link to post

No.  Stop script is run prior to stopping, and start script is run after starting.  My suggestion would be to do a user script or something to replicate what this plugin does to rsync the appdata share appropriately

Share this post


Link to post
On 4/8/2019 at 3:49 AM, MyKroFt said:

Backup - detect extra docker networks and backup their info

Restore - recreate extra docker networks before restore

All you should need to do for this is manually make a copy of the docker.img file.

Share this post


Link to post
8 hours ago, Squid said:

All you should need to do for this is manually make a copy of the docker.img file.

Is there a possibility to add a before/after path to custom script option?

Share this post


Link to post
16 hours ago, MyKroFt said:

Is there a possibility to add a before/after path to custom script option?

Not following what you mean by this.

Share this post


Link to post

you have a path to custom stop/start scripts - am assuming that is a way to only shutdown certain dockers via script.

 

what about a before/after - that can be run and let the backup program manage stopping/starting the dockers itself

 

if I am reading correctly what the custom stop/start are to be used for?

Share this post


Link to post
13 minutes ago, MyKroFt said:

am assuming that is a way to only shutdown certain dockers via script.

Hit Advanced settings.  You can leave any particular container running the entire time

Share this post


Link to post

Hi Guy's after having a few issues I just need some clarification, All my docker containers are stored on an SSD / the cache drive  like this 

 

/mnt/user/appdata/letsencrypt/

/mnt/user/appdata/organizr/

 

CA Backups is set to /mnt/user/CA Backups/appdatabackup

/mnt/user/CA Backups/flashdrivevbackup/

/mnt/user/CA Backups/libvirt backup/ 

 

So unless I am wrong the backup is backed up to the SSD / Cache? 

So if said SSD / cache was to fail / die id be FUBAR and lose all my dockers etc so do I manually move the backups or do I set them to a folder on the share / array 

TIA 

 

please forgive my terminology its been a really long 2 days and I haven't the time to investigate  

Regards 

 

L33 

 

 

Share this post


Link to post

Simply set the CA backup share to use cache : no

Sent via telekinesis

Share this post


Link to post

So I a having occasional issues where all my dockers fail to restart after the backup. Do you have any idea what could be the cause?

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.