[Plugin] CA Appdata Backup / Restore v2


Squid

Recommended Posts

What is the protocol for restoring individual files/folders. Can you browse the .tar via unraid? If so, can someone help with commands to navigate it. My tar is ~50GB and I need to restore a folder that's probably under 10MB. Winrar doesn't seem to like trying to open it via SMB on my Windows 10 computer. Not sure what the best way to do this is.

Link to comment

1 of the docker settings got messed up and failed to update etc so it was removed when i readd it all settings on the docker are gone.

 

I did a restore of yesterday and checked but still all settings of that docker are gone. i even tried the backup of the day before yesterday. Some problem..

 

How can i get those settings back, are they in a file or so.  the files it self in the docker are still there just not the settings when you edit the image.

Link to comment
21 minutes ago, Squid said:

You're talking about reinstalling the app itself.  It sounds like you reinstalled from a regular search in CA (which uses default settings on the app) vs the Previous Apps section (which uses the last saved template)

i did use the previous apps section.

 

i found the file with some settings and did it manually

Link to comment

Hi all,

I just started using this plugin again, and my first back up ran Monday night. It all seems to have worked, I'm seeing a dated TAR file for the app data as expected, however, my USB back up seems to just be a copy of the data - no TAR and no date-version. 

 

Is this expected behaviour? I was expecting TARs for both the "Use Compression: Yes" setting?

 

Thanks,

 

whiteatom

Link to comment

Yup.  That's correct.  USB backup was just tacked onto the plugin as a feature req (realistically it changes very infrequently), and was done prior to the GUI even supporting any method of backup of the USB.

 

The USB backup will be removed completely in the future.  Waiting on a feature from LT being worked on...

Link to comment
3 minutes ago, Squid said:

Yup.  That's correct.  USB backup was just tacked onto the plugin as a feature req (realistically it changes very infrequently), and was done prior to the GUI even supporting any method of backup of the USB.

 

The USB backup will be removed completely in the future.  Waiting on a feature from LT being worked on...

Ok sounds good.. is LT adding a USB backup tool? I think mine is on the way out, (very slow boot), so I'd just like to make sure I have a backup of the data!

Link to comment
17 hours ago, Squid said:

Is the USB mounted?  Can you access it over the network?  What's the contents within the appdatabackup folder on it?

The USB is mounted and I can access it over the network-share

grafik.png.c5db87311738843e94c19a772fecfbde.png

This is the content and the schedule for backup was successful but I don't get the option to recover

When I am getting in the folder there are folders for every date and in there is a file like "CA_backup.tar.gz"

Link to comment

I'll have to look at it.  Not sure why it won't work.  What you might try doing though is create a new share on the array, and copy the backup folder into that share.

 

Then on the Backup settings, change the destination to the array share and folder.  Reload the tab and see if it picks up the backup set on the restore tab

  • Like 1
Link to comment
23 hours ago, Squid said:

I'll have to look at it.  Not sure why it won't work.  What you might try doing though is create a new share on the array, and copy the backup folder into that share.

 

Then on the Backup settings, change the destination to the array share and folder.  Reload the tab and see if it picks up the backup set on the restore tab

Now it is working!

Thank you

Link to comment

How does the line "Delete backups if they are this many days old:" work exactly? If I left that at it's default of 1, and set up a scheduled backup frequency of weekly, is that saying that the backup would be deleted a day later after it's creation, and I would go 6 days without a backup being saved anywhere until the next scheduled backup occurred? Does that apply to the appdata backup, USB backup, and libvirt backup?

 

I noticed that only the appdata backup is held in a dated/timestamped folder. I take that to mean that every backup will overwrite the USB and libvirt folders? Is it just the appdata backup that is "kept" until the deletion date is reached?

 

Does anyone have any ideas on how to best copy off the backup folders to the cloud (Google Drive, etc.) on a recurring basis? I'm thinking something that uses the "Path to Custom Start (or Pre-Start?) Script" field, so that it always makes a copy to the cloud once a new set of backups have been placed on my UnRaid 'backup' share? My goal is to ensure I can reach these backups, even if there is a more catastrophic failure of my server that makes reaching the 'backup' share difficult/impossible.

Edited by Magic815
Link to comment
On 11/5/2017 at 5:05 AM, Squid said:

securityIcon-WithShadow.png

 

CA Appdata Backup / Restore (v2)

 

Due to some fundamental problems with XFS / BTRFS, the original version of Appdata Backup / Restore became unworkable and caused lockups for many users.  Development has ceased on the original version, and is now switched over to this replacement.

 

Fundamentally they are both the same and accomplish the same goals (namely, backing up your Appdata share and USB / libvirt), but this version is significantly faster at the job. 

 

This version uses tar instead of rsync (and offers optional compression of the archive - roughly 50% if not including any downloads in the archive - which you really shouldn't be anyways). 

 

Because of using tar, there is no longer any incremental backups.  Rather, every backup goes into its own separate dated subfolder.  Old backups can optionally be deleted after a successful backup.  Even without incremental backups, the speed increase afforded by tar means that there should be no real difference in the end.  (ie:  A full backup using my appdata on the old plugin takes ~1.5 hours.  This plugin can do the same thing uncompressed in about 10 minutes, and compressed in 20 minutes.  The optional verification of the archive takes a similar amount of time.  An incremental backup on the old plugin using my appdata averaged around 35 minutes).

 

The options for separate destination for USB / VM libvirt backups is changed so that if there is no destination set for those backup's then they will not be backed up.

 

Additionally, unlike the original plugin, no cache drive is necessary, and the appdata source can be stored on any device in your system (ie: unassigned devices).  The destination as usual can go to any mount point within your system.  Unfortunately because of no more incremental backups, this version may no longer be suitable for ultimately backing up offsite to a cloud service (ie: via rclone)

 

You can find this within the Apps tab (Search for Appdata Backup).

 

The original v1 plugin should be uninstalled if migrating to this version.

 

 

Love the plugin, use it all the time. however do you take requests?

 

If so here are my requests:

     1) Multiple Backup Settings. I want to backup some of my appdata daily, while there are others I want done weekly. Would be nice to be able to do so. Currently I exclude the weekly ones and back them up manually.

    2) Selective restore. I have had it that one os the db from an app goes bad and I need to restore it, but there is no way to just restore the one I need, its all or nothing. Currently I have to extract the backup, then copy only the files I need.

 

Thanks again for your hard work.

 

 

 

 

Link to comment
1 hour ago, tuxbass said:

When the plugin starts the containers back up - are the order & wait times still respected?
Every morning one of the services has gone bad, likely due to a dependent container not having started by the expected time.

No issue with system reboot though.

Not currently.  On my todo list

  • Thanks 1
Link to comment

I know it's going to be an unpopular request, but could we have "Use tar?" option before "Use Comporession?", latter being disabled & defaulting to 'no' if user opts out of tarring.

There are use-cases when simply copying the files as-is can be preferred.

Edited by tuxbass
Link to comment
15 hours ago, tuxbass said:

I know it's going to be an unpopular request, but could we have "Use tar?" option before "Use Comporession?", latter being disabled & defaulting to 'no' if user opts out of tarring.

There are use-cases when simply copying the files as-is can be preferred.

That's what v1 of the plugin did.  Worked for some people, but a fair proportion of users would wind up having their servers basically lockup while XFS / BTRFS chewed through the massive amount of changes caused by copying potentially a million files at a time.  You can still get it by enabling Show Deprecated Apps within CA's Settings

Link to comment

Situation: I lost my cache array due to an issue running RAM out of spec.

 

I managed to restore all docker containers using a backup.

 

I then lost the USB drive without a backup, but managed to recreate parts of the USB using the files located in a diagnostics file the day before. I lost some customizations including plugins, but drive assignments and share were there.

 

I reinstalled the plugin, and pointed it at the old backup, and.... well now it claims the backups are corrupted with:


gzip: stdin: unexpected end of file
Zoneminder/data/events/2/2020-08-19/2267/00122-capture.jpg
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Error is not recoverable: exiting now
Backup/Restore Complete. tar Return Value:
Restore finished. Ideally you should now restart your server
Backup / Restore Completed

 

Now I used those backups, that are hosted not on the unraid server to restore just the day previous. Nothing should have been accessing them. Yet now all 3 previous days are corrupted.

 

When I pull down the tar and try and open it it does seem to be corrupted.

 

Any idea what happened ? 

 

 

Link to comment
  • Squid locked this topic
Guest
This topic is now closed to further replies.