[Plugin] CA Appdata Backup / Restore v2

[Vaggeto 1]

Amazing tool, thank you so much!

 

I have a slightly irregular situation. I changed flash drives when building a new unRAID system. I wanted to start from scratch and have all of my data moved over but I did want to keep my older docker setup. I was trying to use this app though and it doesn't recognize any backups even if I point to the folder where the backup was just taken.

I already have all of my appdata folder on the new system and the docker image since I moved the old cache drive over, but the dockers don't show because they are not configured on the flash drive.

Does anyone know which files I need to get from my backup to move over to the new flash drive from the old one? these files?: \flash\config\plugins\dockerMan\templates-user or is it something more substantial?

 

I'd prefer not to manually reinstall these dockers and have to reconfigure them if possible.

[bnevets27 24]

Except that it doesn't delete any backups until a successful backup is completed.  But yeah, retain x number of backups will partially alleviate this if you catch it prior to the threshold being reached

True but the successful backup that it completed is unfortunately not a backup of a full working system. And in this case basically blank. I was lucky I did a manual for copy at the start so it wasn't a complete loss. I usually also keep (by renaming the folder) a permanent backup that CA backup/restore creates once in a while copy in case I don't catch something that's gone awry within the deletion period, so I can go further back if necessary.

 

But yes with a delete backups every 60 days and say a minimum of 2 backups kept it would definitely keep a good backup long enough.

 

In that same right, keep every nth backup indefinitely would be nice too but I can understand not wanting to put in too many options.

 

Thanks squid for the great app!

 

[Vaggeto 1]

23 hours ago, Vaggeto said:

Amazing tool, thank you so much!

 

I have a slightly irregular situation. I changed flash drives when building a new unRAID system. I wanted to start from scratch and have all of my data moved over but I did want to keep my older docker setup. I was trying to use this app though and it doesn't recognize any backups even if I point to the folder where the backup was just taken.

I already have all of my appdata folder on the new system and the docker image since I moved the old cache drive over, but the dockers don't show because they are not configured on the flash drive.

Does anyone know which files I need to get from my backup to move over to the new flash drive from the old one? these files?: \flash\config\plugins\dockerMan\templates-user or is it something more substantial?

 

I'd prefer not to manually reinstall these dockers and have to reconfigure them if possible.

As I mentioned here I copied these over and to DockerMan on the flash drive and then finally when I went to "previous apps" in CA they were there to reinstall and had both their settings and actual data. So this is solved. I still can't figure out why backup/restore doesn't see any backups though.

[Nilok3 0]

I apologize if this question has been asked before, but is there a log file for this plugin saved somewhere? Verifying the backups seems to take way longer than I think it should and I'm curious if it's getting stuck somewhere.

[DZMM 236]

Is there a way to check in a script if an appdata backup or restore is occuring?  I have another script running that restarts dockers if they aren't running - I just realised I need it to NOT restart dockers, IF the appdata script is running.

 

Thanks in advance.

[Squid 2986]

9 hours ago, DZMM said:

Is there a way to check in a script if an appdata backup or restore is occuring?  I have another script running that restarts dockers if they aren't running - I just realised I need it to NOT restart dockers, IF the appdata script is running.

 

Thanks in advance.

If either /tmp/ca.backup2/tempFiles/backupInProgress or /tmp/ca.backup2/tempFiles/restoreInProgress exists then a backup / restore is running

[DZMM 236]

On 11/12/2020 at 6:00 PM, Squid said:

If either /tmp/ca.backup2/tempFiles/backupInProgress or /tmp/ca.backup2/tempFiles/restoreInProgress exists then a backup / restore is running

Thanks - perfect.

[khager 0]

Steps to restore?

 

My cache drive crashed. I replaced it. Now I want to restore my dockers from backup.

Is there a step-by-step? The wiki didn't make sense to me - please help me understand.

Seems like I should

1) reinstall my dockers from the "Previous apps" page

2) run the restore process using CA Appdata Backup/Restore plugin

3) maybe restart

4) done, right?

 

That's not what it seemed like the wiki was telling me to do. 

Can someone please point me in the right direction for the order of tasks?

 

Thanks,

Kyle

[Squid 2986]

From the Appdata Plugin's Restore page

Quote

General Procedure On How To Restore Your Appdata

Replace your cache drive if necessary

Recreate your appdata share (and set it to be cache only)

Recreate your docker.img file if necessary (ideally, this should be performed regardless)

Select a backup set (if using dated backup sets) and hit 'Restore'

Sit back and have a coffee

Ideally restart your server

Re-add your previously installed docker applications via CA's Previous Apps section. No adjustment of the templates should be necessary

 

[khager 0]

That's what I read but I was confused by restoring the backup before adding the previously-installed docker apps.

 

And - how do you "recreate your docker.img file"? I thought that got created when you installed the docker apps (?)

 

...told you I was confused...

[khager 0]

OK - skip the "how do you create docker.img" question. It was already created.

I followed your instructions and, by doing so, I learned a few things. I still don't fully understand what's part of the docker and what's part of the backup.

 

In any case, I'm up and running so thanks for posting those instructions (again <sheepish>).

Edited November 21, 2020 by khager

[itimpi 890]

6 minutes ago, khager said:

OK - skip the "how do you create docker.img" question. It was already created.

I followed your instructions and, by doing so, I learned a few things. I still don't fully understand what's part of the docker and what's part of the backup.

 

In any case, I'm back up and running so thanks for posting those instructions (again).

The docker.img file is created (if not already present) when the docker service starts as something to contain the binaries for apps.  Think of the docker container as being the binary files for an app and the backup contains all the apps working file.    

[Squid 2986]

The backup does not include the docker.img (binary files) as it's very easily recreated and the apps reinstalled and it would automatically add 20+Gig to the backup set for no benefit

[Froberg 21]

Got this running now. Is it possible to have it just keep a copy of the flashdrive backup in .zip format instead? 

I like that it's at least ON a disk this way, but having a few copies wouldn't hurt, either.  Or maybe it's unnecessary and I'm making it overly complicated. 

[trurl 1568]

7 hours ago, Froberg said:

having a few copies wouldn't hurt

You can easily download a zipped backup of flash at any time from Main - Boot Device - Flash - Flash Backup.

[Froberg 21]

33 minutes ago, trurl said:

You can easily download a zipped backup of flash at any time from Main - Boot Device - Flash - Flash Backup.

I know. But this is automated

[Thx And Bye 9]

The plugin works really well for the task but is it intended behavior that only one thread is used for compression? I think on a multicore CPU there could be substantial performance improvements if more cores are used for this task. 

If there is no better method available on Unraid by default, maybe make it optional if a better compression program is detected (e.g. pigz installed via NerdPack)? 

 

I think anyone can profit from this as multi-core CPUs are the norm nowadays.

[Squid 2986]

It uses the built-in tar.   IIRC, installing pigz via NerdPack causes compatibility problems...

[Thx And Bye 9]

3 hours ago, Squid said:

It uses the built-in tar.   IIRC, installing pigz via NerdPack causes compatibility problems...

Compatibility problems specifically with CA Backup or with the tar on Unraid? Shouldn't it be as easy as to replace the -z flag of tar with -I pigz when pigz is installed?

Also if pigz doesn't work, wouldn't a other compression that supports multicore (like pbzip2) possible?

Edited November 28, 2020 by Thx And Bye

[Squid 2986]

If it doesn't come with Unraid, I'm not using it in the plugin.

[craigr 22]

Want to say thanks again for CA Backup / Restore Appdata as it just saved me a lot of time.  I changed the location of my appdata folder to a different hard drive and forgot one setting in PLEX and lost my whole config last night.  Thankfully I slept on it and remembered, "hey, I have a backup."

 

This app just saved me a few hours and it has saved me a few hours a few other times too

 

Thank you,

craigr

[amiskell 1]

Is there a way to troubleshoot why the backups aren't getting automatically executed? I have the plugin configured to backup everything on Sunday morning at 3am, but nothing appears to be happening during that time. I get no backup files in the defined backup locations or anything.

 

I also tried a manual backup and it says it completed successfully, but still nothing is generated in any of the defined locations (appdata, vm or usbflash).

Edited December 5, 2020 by amiskell

[Squid 2986]

A screenshot of the settings, and your diagnostics covering that time period would be needed

[amiskell 1]

2 hours ago, Squid said:

A screenshot of the settings, and your diagnostics covering that time period would be needed

scarif-diagnostics-20201205-1408.zip

[Squid 2986]

The diagnostics don't cover Sundary the 29th.  Post new diagnostics Monday AM