[Plugin] CA Appdata Backup / Restore v2


Squid

Recommended Posts

8 hours ago, Squid said:

 

Probably because your date/time is out to lunch on the server.  Set it properly in the BIOS, and also replace the battery.

That would make sense because when I created the unraid server, that motherboard has issues with the battery/RTC.  Does that mean that NTP is not up and running when the server starts writing to the unraid thumb drive and hence the date/time is being pulled from the RTC?

Link to comment

I still have the V1, i didn't realize there was an update.  

I am having issues restoring from backup.  My server had a cache drive problem and crashed mid way through a recovery and it pretty much wiped out a bunch of dockers, my vm's etc...

I do have a backup that I'm trying to deploy from last monday and I see it in the location it should be and the restore tab says

"No backup settings have already been defined. You must set those settings before you are able to restore any backups".   It says 'no backup sets found', however I can see one right there and I pulled it off the server to confirm validity and its all good.  

Is there a way to force CA backup to recognize the backup set so I can continue on?

Link to comment
14 minutes ago, ledfortr said:

 

The backup is saved as a .tar, do I need to untar it then copy? I ended up copying the whole tar file over to appdata.

here is where everything is

my appdata is /mnt/user/appdata

the backup file is /mnt/user/appdata/CA_backup.tar

What command would I type to untar and replace the appdata path?  

Link to comment

Hi, not sure if it is related to this plugin, but this AM I noticed that none of my dockers are running.

Actually only one is running, postfix, but that does not use any appdata storage.

I looked at the log, and it looks like backup ran, last reported verifying the backup, but I'm not sure why the containers were not restarted after the backup.

Feb  2 02:00:01 Server-1 Plugin Auto Update: Checking for available plugin updates
Feb  2 02:00:03 Server-1 Plugin Auto Update: community.applications.plg version 2020.02.01 does not meet age requirements to update
Feb  2 02:00:04 Server-1 Plugin Auto Update: Community Applications Plugin Auto Update finished
Feb  2 03:00:01 Server-1 CA Backup/Restore: #######################################
Feb  2 03:00:01 Server-1 CA Backup/Restore: Community Applications appData Backup
Feb  2 03:00:01 Server-1 CA Backup/Restore: Applications will be unavailable during
Feb  2 03:00:02 Server-1 CA Backup/Restore: this process.  They will automatically
Feb  2 03:00:02 Server-1 CA Backup/Restore: be restarted upon completion.
Feb  2 03:00:02 Server-1 CA Backup/Restore: #######################################
Feb  2 03:00:02 Server-1 CA Backup/Restore: Stopping Duplicacy
Feb  2 03:00:02 Server-1 CA Backup/Restore: docker stop -t 60 Duplicacy
Feb  2 03:00:02 Server-1 CA Backup/Restore: Stopping nginx
Feb  2 03:00:06 Server-1 kernel: veth8a1bd58: renamed from eth0
Feb  2 03:00:06 Server-1 CA Backup/Restore: docker stop -t 60 nginx
Feb  2 03:00:06 Server-1 CA Backup/Restore: Stopping plex
Feb  2 03:00:10 Server-1 kernel: vethf123b74: renamed from eth0
Feb  2 03:00:10 Server-1 CA Backup/Restore: docker stop -t 60 plex
Feb  2 03:00:10 Server-1 CA Backup/Restore: postfix set to not be stopped by ca backup's advanced settings.  Skipping
Feb  2 03:00:10 Server-1 CA Backup/Restore: Stopping radarr
Feb  2 03:00:14 Server-1 kernel: veth3440e0e: renamed from eth0
Feb  2 03:00:14 Server-1 CA Backup/Restore: docker stop -t 60 radarr
Feb  2 03:00:14 Server-1 CA Backup/Restore: Stopping sabnzbd
Feb  2 03:00:18 Server-1 kernel: vethd3efd80: renamed from eth0
Feb  2 03:00:18 Server-1 CA Backup/Restore: docker stop -t 60 sabnzbd
Feb  2 03:00:18 Server-1 CA Backup/Restore: Stopping sonarr
Feb  2 03:00:22 Server-1 kernel: veth323bd38: renamed from eth0
Feb  2 03:00:22 Server-1 CA Backup/Restore: docker stop -t 60 sonarr
Feb  2 03:00:22 Server-1 CA Backup/Restore: Stopping vouch-proxy
Feb  2 03:00:23 Server-1 kernel: veth21f55b3: renamed from eth0
Feb  2 03:00:23 Server-1 CA Backup/Restore: docker stop -t 60 vouch-proxy
Feb  2 03:00:23 Server-1 CA Backup/Restore: Backing up USB Flash drive config folder to 
Feb  2 03:00:23 Server-1 CA Backup/Restore: Using command: /usr/bin/rsync  -avXHq --delete  --log-file="/var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log" /boot/ "/mnt/user/backup/Unraid/USB" > /dev/null 2>&1
Feb  2 03:00:23 Server-1 CA Backup/Restore: Changing permissions on backup
Feb  2 03:00:23 Server-1 CA Backup/Restore: Backing up libvirt.img to /mnt/user/backup/Unraid/libvirt/
Feb  2 03:00:23 Server-1 CA Backup/Restore: Using Command: /usr/bin/rsync  -avXHq --delete  --log-file="/var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log" "/mnt/user/system/libvirt/libvirt.img" "/mnt/user/backup/Unraid/libvirt/" > /dev/null 2>&1
Feb  2 03:00:27 Server-1 CA Backup/Restore: Backing Up appData from /mnt/user/appdata/ to /mnt/user/backup/Unraid/appdata/2020-02-02@03.00
Feb  2 03:00:27 Server-1 CA Backup/Restore: Using command: cd '/mnt/user/appdata/' && /usr/bin/tar -cvaf '/mnt/user/backup/Unraid/appdata/2020-02-02@03.00/CA_backup.tar.gz' --exclude 'docker.img'  * >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress
Feb  2 04:00:01 Server-1 Docker Auto Update: Community Applications Docker Autoupdate running
Feb  2 04:00:01 Server-1 Docker Auto Update: Checking for available updates
Feb  2 04:00:10 Server-1 Docker Auto Update: Installing Updates for code-server nginx sabnzbd
Feb  2 04:00:40 Server-1 Docker Auto Update: Community Applications Docker Autoupdate finished
Feb  2 04:40:01 Server-1 apcupsd[10670]: apcupsd exiting, signal 15
Feb  2 04:40:01 Server-1 apcupsd[10670]: apcupsd shutdown succeeded
Feb  2 04:40:04 Server-1 apcupsd[13274]: apcupsd 3.14.14 (31 May 2016) slackware startup succeeded
Feb  2 04:40:04 Server-1 apcupsd[13274]: NIS server startup succeeded
Feb  2 06:18:13 Server-1 CA Backup/Restore: Backup Complete
Feb  2 06:18:13 Server-1 CA Backup/Restore: Verifying backup
Feb  2 06:18:13 Server-1 CA Backup/Restore: Using command: cd '/mnt/user/appdata/' && /usr/bin/tar --diff -C '/mnt/user/appdata/' -af '/mnt/user/backup/Unraid/appdata/2020-02-02@03.00/CA_backup.tar.gz' > /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress

The CA backup status tab says verifying, but it has been more than 2 hours of verifying.

Is it really verifying, if so, should the containers not be restarted after backup, not after verify, else they will be offline much longer than needed?

Any ideas how to find out if verify is really running, or something went wrong?

Edited by ptr727
Link to comment
15 minutes ago, ptr727 said:

but it has been more than 2 hours of verifying.

The backup took 3:18, you can expect the verify to take that long.

 

16 minutes ago, ptr727 said:

should the containers not be restarted after backup, not after verify,

It would be a somewhat safe assumption that when starting back up, any given app is going to wind up making changes to the appdata due to its start procedure (ie: rescanning media, etc).  If you start the apps before verification, then any verification is going to fail.

 

Turn off verification.  

Link to comment
11 minutes ago, Squid said:

The backup took 3:18, you can expect the verify to take that long.

 

It would be a somewhat safe assumption that when starting back up, any given app is going to wind up making changes to the appdata due to its start procedure (ie: rescanning media, etc).  If you start the apps before verification, then any verification is going to fail.

 

Turn off verification.  

Ah, so verify is comparing the files, not just verifying integrity, got it.

I bet it is the Plex metadata that is taking forever, is there an ability to exclude the metadata from backup (it can be redownloaded), maybe a path exclusion?

Link to comment
8 minutes ago, Squid said:

image.png.ff24ab4f4e2337f816007b87516f324b.png

Ah, I was about to say but it only allows the top level to be excluded, not child folders, then I noticed the edit box, and I can add my own path instead of using the GUI.

Would be cool if the GUI allowed sub-folder selection, but I'll wait for restore to complete, then try to exclude the Plex metadata folder.

Thx

Link to comment
  • 2 weeks later...
1 hour ago, IamSpartacus said:

@Squid How hard would it be to implement multiple different schedules into this plugin?  So that, for example, one could backup all containers once a week but only the "production" containers every day.

Quite a bit (or more truthfully a metric tonne)

 

OTOH though it can be done via some playing around with settings.

 

Set up the options for backup #1.  Make a backup of the settings from the flash drive

Set up the options for backup #2.  Make a backup of the settings from the flash drive

Disable the backup from running on a schedule.

 

Via user scripts you set 2 scripts with whatever schedule you want, and that script will restore the appropriate settings file onto the flash drive and then execute the backup script.

 

Where there's a will, there's a way (and time to spare at work letting me think about it)

Link to comment
8 hours ago, Squid said:

Quite a bit (or more truthfully a metric tonne)

 

OTOH though it can be done via some playing around with settings.

 

Set up the options for backup #1.  Make a backup of the settings from the flash drive

Set up the options for backup #2.  Make a backup of the settings from the flash drive

Disable the backup from running on a schedule.

 

Via user scripts you set 2 scripts with whatever schedule you want, and that script will restore the appropriate settings file onto the flash drive and then execute the backup script.

 

Where there's a will, there's a way (and time to spare at work letting me think about it)

 

Thanks for the suggestion, that's actually a super workable solution.

Edited by IamSpartacus
Link to comment

Is it possible to trigger the mover immediately after CA Backup completes?  

I have the pluggin creating its data on a share that's set to "use cache=yes" and I'd rather not leave my backup sitting on the cache drive until the scheduled mover.  I guess I could create a new share that doesn't use the cache but then the backup will take significantly longer. 

 

Edited by rcrh
Link to comment
4 minutes ago, rcrh said:

Is it possible to trigger the mover immediately after CA Backup completes?  

I have the pluggin creating its data on a share that's set to "use cache=yes" and I'd rather not leave my backup sitting on the cache drive until the scheduled mover.  I guess I could create a new share that doesn't use the cache but then the backup will take significantly longer. 

 

image.thumb.png.a3fca751b8f5bebec2f818ff8bde0ecd.png

 

And enter this in

/usr/local/emhttp/plugins/dynamix/scripts/mover

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.