[Plugin] CA Appdata Backup / Restore v2


Squid

Recommended Posts

  • 3 weeks later...

Hey @Squid

 

Nice work on this plugin!

I was going to roll something similar as a script before I found this. Nice Job.

 

Bug:

Docker img (libvirt) is not backed up if its not in the right location (which I cant find documented anywhere) or named correctly

I use  /mnt/cache/docker.img as I setup docker ages before I found this plugin.

 a field to allow customisation of the libvirt source image would be ideal

 

Comment on functionality:

The "Delete backups if they are this many days old" doesn't make a lot of sense when used with anything other than a daily schedule.

Eample scenario

Initial Setup

Set "Delete after"  to 5

Set "Schedule" to Daily

Outcome: 5 backups kept

 

Changed my mind and I want weekly backups now

Set "Schedule" to weekly

Forgot to calculate how many days I want (7/14) to keep relative to the schedule type so it remains at 5

Outcome: the weekly backup is erased after 5 days leaving 2 days before the next scheduled backup takes place.

 

Worse again

Changed the "Schedule" to monthly and forgot to adjust the days to keep (31/62) so it stays at 5.

Outcone: a Backup is created at the specified monthly interval and is then sebsequently erased after 5 days leaving 3/4 of the month with no recovery option.

 

A possibly better and simpler solution to this issue would be to change the "Delete after" to "Backups to keep"

in this way it would not matter about the schedule at all as you always know there are X daily/weekly/monthly backups at any given time.

 

I did actually run into this exact scenario (the weekly one) and thought you should know.

 

Keep up the great work

 

Edited by Loophole
  • Upvote 1
Link to comment
32 minutes ago, Loophole said:

Docker img (libvirt) is not backed up if its not in the right location (which I cant find documented anywhere) or named correctly

I use  /mnt/cache/docker.img as I setup docker ages before I found this plugin.

 a field to allow customisation of the libvirt source image would be ideal

 

Docker img is NOT (libvirt)

 

Libvirt is for the VMs

 

There is no good reason to backup docker image since the containers can just be redownloaded. The only thing that matters is the appdata and the templates. The templates are the settings you made such as mappings when you installed a docker. These are on flash so make sure you are backing up flash. In fact it's probably better if you just recreate docker image instead of even attempting to back it up. You can always reinstall your containers exactly as they were using the Previous Apps feature on the Apps page, as long as you have the templates and appdata.

Link to comment
3 minutes ago, trurl said:

Docker img is NOT (libvirt)

 

Libvirt is for the VMs

 

There is no good reason to backup docker image since the containers can just be redownloaded. The only thing that matters is the appdata and the templates. The templates are the settings you made such as mappings when you installed a docker. These are on flash so make sure you are backing up flash. In fact it's probably better if you just recreate docker image instead of even attempting to back it up. You can always reinstall your containers exactly as they were using the Previous Apps feature on the Apps page, as long as you have the templates and appdata.

Oh I see
Looks like I was confusing them (Docker & VM in general)

Yep definitely backing up the flash, checked that before


thanks for the clarification

Link to comment
  • 2 weeks later...

So i need to flash my raid controller to jbod mode and so i'm trying to make a simple backup of my appdata for my rebuild

 

I first tried to backup using your tool to windows smb shares that i have mounted with the unassigned devices tool - i had previously had issues torrenting to the smb shares (be they drivepool or normal hdd) as without fail some torrents on recheck would be missing part of their data

 

A similar thing is happening with this backup tool - i have done both compressed and not compressed - the compressed tar.gz open as corrupt and the regular uncompressed tar say they are the right size and then show only one folder from appdata

Backing up to an unraid share works perfectly fine and is a similar size to the broken tar's on the windows smb share

 

I feel like this is something i am missing but i would appreciate any help @Squid

 

EDIT:  Also the verify runs on the backups and finds no problems - so thats also strange

Edited by mcrommert
Link to comment

As always, thanks for your addons, makes management 1000% better.....

 

Request:

 

Backup - detect extra docker networks and backup their info

Restore - recreate extra docker networks before restore

 

I had just corrupted my cache drive some how and did a restore  (went well) - but some dockers were not working, took me a bit to figure out they were on a custom network for my reverse proxies stuff.

 

Had to manually create network, edit each docker to change to network to something else and back to custom network so it would re save and restart docker to use custom network (just restarting would not make it use custom network).

 

Thanks for the consideration of this request.

 

Myk

 

Edited by MyKroFt
Link to comment

This plugin critically fails to backup poste, WITHOUT USER NOTIFICATION

 

Unfortunately,  this has gutted my email system as I deleted the appdata folder for it after a change didn't work and was going to restore from backups.

 

As far as I can tell, files and folders with mail group and mem user are what's missed:
image.png.5fa49c4abbb760c623b447d8ddcac55d.png

Link to comment

Thankfully that was a horribly false alert. It seems I made the mistake of copying another folder that had those files missing instead.
I was from an instance that broke that I could never narrow down exactly what cause it.

Checking the backups, I can confirm the correct folder was backed up safely, sorry for the trouble.

Now I can relax with it back up >.<

Link to comment

Is there anything special I have to do to get Start Scripts to work or should a script that works as a UserScript just work?

 

Its says: 
Executing custom start script

 

But its not running or at least not doing what it should.

 

The script syncs the backup to google drive and works fine when I just start it from user scripts.

 

Link to comment
2 hours ago, TeKo said:

Is there anything special I have to do to get Start Scripts to work or should a script that works as a UserScript just work?

 

Its says: 
Executing custom start script

 

But its not running or at least not doing what it should.

 

The script syncs the backup to google drive and works fine when I just start it from user scripts.

 

User scripts is far more forgiving on scripts than backup.

 

Notable differences:

  • The "interpreter line" ( ie: #!/bin/bash ) has to be present in backup
  • Line endings must be linux style, and not Windows style (ie: use Notepad++ to edit the script and set EOL to be unix/linux)
Link to comment
2 hours ago, Squid said:

User scripts is far more forgiving on scripts than backup.

 

Notable differences:

  • The "interpreter line" ( ie: #!/bin/bash ) has to be present in backup
  • Line endings must be linux style, and not Windows style (ie: use Notepad++ to edit the script and set EOL to be unix/linux)

Thanks, looks like the EOLs were the problem.

Link to comment
On 3/30/2019 at 12:00 AM, MyKroFt said:

As always, thanks for your addons, makes management 1000% better.....

 

Request:

 

Backup - detect extra docker networks and backup their info

Restore - recreate extra docker networks before restore

 

I had just corrupted my cache drive some how and did a restore  (went well) - but some dockers were not working, took me a bit to figure out they were on a custom network for my reverse proxies stuff.

 

Had to manually create network, edit each docker to change to network to something else and back to custom network so it would re save and restart docker to use custom network (just restarting would not make it use custom network).

 

Thanks for the consideration of this request.

 

Myk

 

Any consideration on this?

 

Thanks

Myk

Link to comment

No.  Stop script is run prior to stopping, and start script is run after starting.  My suggestion would be to do a user script or something to replicate what this plugin does to rsync the appdata share appropriately

Link to comment
On 4/8/2019 at 3:49 AM, MyKroFt said:

Backup - detect extra docker networks and backup their info

Restore - recreate extra docker networks before restore

All you should need to do for this is manually make a copy of the docker.img file.

Link to comment

you have a path to custom stop/start scripts - am assuming that is a way to only shutdown certain dockers via script.

 

what about a before/after - that can be run and let the backup program manage stopping/starting the dockers itself

 

if I am reading correctly what the custom stop/start are to be used for?

Link to comment
  • 4 weeks later...

Hi Guy's after having a few issues I just need some clarification, All my docker containers are stored on an SSD / the cache drive  like this 

 

/mnt/user/appdata/letsencrypt/

/mnt/user/appdata/organizr/

 

CA Backups is set to /mnt/user/CA Backups/appdatabackup

/mnt/user/CA Backups/flashdrivevbackup/

/mnt/user/CA Backups/libvirt backup/ 

 

So unless I am wrong the backup is backed up to the SSD / Cache? 

So if said SSD / cache was to fail / die id be FUBAR and lose all my dockers etc so do I manually move the backups or do I set them to a folder on the share / array 

TIA 

 

please forgive my terminology its been a really long 2 days and I haven't the time to investigate  

Regards 

 

L33 

 

 

Link to comment
  • Squid locked this topic
Guest
This topic is now closed to further replies.