Jump to content
Squid

[Plugin] CA Appdata Backup / Restore v2

283 posts in this topic Last Reply

Recommended Posts

Any plans on adding an option to be able to back up vdisk images? Would be great to be able to back up my vm's since I run them on my cache drive also. @Squid

Share this post


Link to post
7 hours ago, jrdnlc said:

Any plans on adding an option to be able to back up vdisk images? Would be great to be able to back up my vm's since I run them on my cache drive also. @Squid

There is that script available.  My problem with VM backups is that for various reasons I've gone back to dedicated hardware for all but one LibreElec VM, and there's a ton of variables to account for - shutdowns, what to do if it doesn't shutdown in a set time period, what about if an update on Win10 winds up being installed, etc etc.  

 

I think that by far the better solution is for unRaid (or a script) to support snapshots which eliminates all of the various issues.

Share this post


Link to post
6 hours ago, Squid said:

There is that script available.  My problem with VM backups is that for various reasons I've gone back to dedicated hardware for all but one LibreElec VM, and there's a ton of variables to account for - shutdowns, what to do if it doesn't shutdown in a set time period, what about if an update on Win10 winds up being installed, etc etc.  

 

I think that by far the better solution is for unRaid (or a script) to support snapshots which eliminates all of the various issues.

 

You can always set it to auto skip the vdisk images backup if the VM is still running to prevent having to implement all those variables. 

 

For some reason I thought the plugin backed them up but then I realized it didn't when I formatted my cache and lost all my vm's. Kind of pointless in having a backup of my libvirt.img only now. @Squid

Edited by jrdnlc

Share this post


Link to post
On 1/5/2018 at 8:59 AM, aptalca said:

Feature request: Instead of one giant tarball, could this app use separate tarballs for each folder in appdata? That way it would be much easier to restore a specific app's data (manually) or even pull a specific file since most of them could be opened with untar guis. 

 

Plex is the major culprit with it gargantuan folder. 

 

I'm going to sign on for this feature request as well! I saw someone else commenting about extracting individual app from the tar, but people (mostly me, myself, and I) are much more likely to fail during a non gui procedure.  

 

Part of the reason for the request is that I know exactly how well the plugin works.  I managed (unsurprisingly) to pooch my Nextcloud container during the last update, and the restore option on the plugin fixed literally every issue I had.  If I could have only restored the Nextcloud and associated DB containers, life would have been that much more perfect.

 

Thanks again @Squid!

Share this post


Link to post
There is that script available.  My problem with VM backups is that for various reasons I've gone back to dedicated hardware for all but one LibreElec VM, and there's a ton of variables to account for - shutdowns, what to do if it doesn't shutdown in a set time period, what about if an update on Win10 winds up being installed, etc etc.  
 
I think that by far the better solution is for unRaid (or a script) to support snapshots which eliminates all of the various issues.
Snapshots were originally include in the VM manager. It's still there in VMMachines.page but disabled. There were issues with snapshots and libvirt-php. I don't remember exactly what though. I think they might have disappeared but would still hold space in qcow.

Share this post


Link to post

Sorry if this is a really n00b question - but how would I go about restoring just one of my apps? One of my MariaDB db's seems to have gotten corrupted, but not sure how to go about restoring just that app's data, rather than everything

 

Thanks!

 

Edit:

ok - looks like I just need to un-tar the most recent backup and copy the relevant file across - not too onerous-a-job, but would +1 either app-specific tar files or app-specific restore functionality - would cut the restore time down dramatically!

Edited by extrobe

Share this post


Link to post

If you're going to implement per app settings I have a few requests. 

1. I currently have configs in two different directories because I created my won appdata directory before UnRaid had a default. It would be nice if the app would allow me to specify the config directory for each app instead of just pointing it to one root appdata directory. (I know I should probably just work on getting all appdata into one root directory)
 

2. Once you start creating separate backups for each app it would awesome to only shutdown the container that is being backed up instead of shutting down all containers then backing up. This would prevent all containers from being down for hours because some containers (i.e. Plex) take forever to backup.

Share this post


Link to post

I also wonder if there's something better that can be done ref Plex - we all know it's a pain when it comes to backing up, and this accounts for the majority of the backup size.

 

The cache directory doesn't actually need backing up though according to plex documentation. Doesn't help with the masses of metadata files, but would cut the backup size significantly.

 

I'm sure tailoring the plugin to have app-specific behaviours isn't going to be particularly welcome from a development point of view, but Plex feels like a worthy exception.

 

 

Share this post


Link to post

Any plans to also compress the Flash / libvirt backups in the same manner as appdata backup?

Share this post


Link to post

Hi Guys,

 

I cannot seem to get this to properly back up.

 

Please see below my config page. I have tried setting the source both as /mnt/cache/appdata and /mnt/user/appdata. When I start a manual backup, it shuts down all my dockers properly. It then properly backs up my thumb drive. Then after, it simply re-starts all the dockers. It never backs up appdata.

 

The folder is created in my destination, and even creates one with current date and time. But it is an empty folder.

 

Please help. I would love to get this running.

 

Thanks,

 

H.

 

gKn31FT.png

Share this post


Link to post

I just started using this and backed up everything (it made a folder called "2018-03-18@12.19" and inside that a "CA_backup.rar" file). However, if I go to the restore tab under "Select a backup set to restore", it says "No backup sets found".

 

Is this because I only have one backup? 

Edited by tyrindor

Share this post


Link to post

It means that there's either no tar inside the folder, or on the backup settings the destination is wrong.

Share this post


Link to post

Backup settings/directory were never changed since I backed up. It should be reading from the same directory it exported too.

 

What do you mean by "tar"? I don't see anything like that in the folder it created.

Edited by tyrindor

Share this post


Link to post

I ran it again, and now that I have 2 backups in the same folder I can select which backup. Seems when you only have 1 backup it shows "No backup found", probably just a display issue.

 

Great tool this will save me a lot of time.

Edited by tyrindor

Share this post


Link to post
7 minutes ago, tyrindor said:

I ran it again, and now that I have 2 backups in the same folder I can select which backup. Seems when you only have 1 backup it shows "No backup found", probably just a display issue.

 

Great tool this will save me a lot of time.

It works fine with only one backup.  What you may be seeing is that with the unRaid GUI as a whole, all the separate tabs within any page are actually all rendered at the same time.  So you went and did a backup, then immediately went to Restore.  Since the restore page was rendered at the same time as when you loaded the backup page, it showed no sets available (because you hadn't already created one)

Share this post


Link to post

Thanks for this, but it would be great if there was an option to create the full file/folder structure like the old version instead of a tar. Changes on my server backup are also backed up online, so this version could be expensive... O.o

Share this post


Link to post
1 hour ago, Rick Gillyon said:

Thanks for this, but it would be great if there was an option to create the full file/folder structure like the old version instead of a tar. Changes on my server backup are also backed up online, so this version could be expensive... O.o

You would have to install the old version instead (enable show deprecated within CA's settings).  Due to many users having problems with the rsync used in the old version (basically rsync would crash the system trying to figure out potentially a million files), the v2 will NOT ever have that in there.

Share this post


Link to post
1 minute ago, Squid said:

You would have to install the old version instead (enable show deprecated within CA's settings).  Due to many users having problems with the rsync used in the old version (basically rsync would crash the system trying to figure out potentially a million files), the v2 will NOT ever have that in there.

Thanks, I reinstalled v1 already. My only worry is if it stops working in a future unRAID build, and isn't being maintained any longer.

Share this post


Link to post
23 minutes ago, Squid said:

Due to many users having problems with the rsync used in the old version (basically rsync would crash the system trying to figure out potentially a million files)

I've never had this issue before, but I think this is what happened last night. When I woke up I saw in the log it was still attempting to do the backup 7 hrs after it started (saw this form the timestamp), just line after line of emby files was showing up in the log. Docker service was not back up, and I couldn't get it to start. Did a reboot and after the second time of disabling/enabling the docker service, it came back up. According to my e-mail the backup had succeeded, but that's clearly not what happened. I should also mention that this is my first backup with 6.5.0.

 

Is this the same issue others have been reporting? If so maybe I should just install the v2. Of course, I forgot to download diagnostic before I rebooted..     

Share this post


Link to post

Sounds similar if the docker service didn't restart.  Probably lucky though that the server didn't actually crash

Share this post


Link to post

Ok, then I'll just go ahead and take a look at the v2. Thanks for the awesome work you do btw! :)

Share this post


Link to post

went to upgrade from v1 to v2.. wish I would have had a screen shot but instead I'm using the old settings .json file.

 

I thought I had it setup correctly.. told it to do a backup and I see it deleting my .kodi folder :(

 

was using:

    "source": "/mnt/user/appdata/",
    "destination": "/mnt/user",
    "destinationShare": "CommunityApplicationsAppdataBackup",
    "excluded": ".kodi/userdata/Thumbnails",

    "backupFlash": "appdata",
    "usbDestination": "",
    "backupXML": "appdata",
    "xmlDestination": "",

 

cant seem to make v2 do the same thing v1 was doing

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.