Jump to content
Squid

[Plugin] CA Appdata Backup / Restore

354 posts in this topic Last Reply

Recommended Posts

A couple questions:

 

Would it be too much to request rclone support? I was going to just copy the logic and translate it for rclone but it looks more involved than I expected. Rclone is similar to rsync but works on cloud services and also has an encryption option. I thought about just initiating rclone to sync after rsync but that loses the "attempt faster sync" feature and will use a lot of bandwidth and time re-uploading files unnecessarily. I know there can be issues with integrating other plugins but I thought it was worth asking. Or maybe you can suggest an alternative way to do it with existing tools and features.

 

I see you can exclude folders but is there any way to exclude subfolders? I'm asking because Plex is really bloated and takes up a lot of space. I really only need it to backup the database and not tons and tons of cached image files and such. Alternatively, I can just exclude Plex altogether and use a custom script to copy just the DB files but I'd rather not complicate things if not necessary.

Any mounted path is supported.  I believe that rclone does give you a mount point within /mnt/disks, but since I don't have any use for the plugin, I can't tell you much.  But I wouldn't be surprised if you run into hardlink issues with it.  Best thing would be to try and set it as a destination and see what happens.

 

If you don't use the dated backup options, then every backup is only the changed files, so your sync will also only reflect the changed files.

 

To exclude subfolders, manually type them in

 

Ahh, good point. Yes, rclone does to mount points, I just never use them for anything other than browsing to make sure the data is there. I think last I checked it did not work so well and I just generally prefer to do them in-program then through mount point as it seems more reliable but I think that just might save me a big headache. I'll give that a go.

 

As far as hardlinks go, I hope I won't have much issue. I think Plex is the only one that really uses them, maybe I'm wrong, and I'll probably just back up the databases and exclude all the other bloat that can easily be re-downloaded. But a followup question on that. I haven't used rsync much but I read the manpage for it. I just want to make sure I understand. If I use -H (as CA backup does) and my source directory has 2 files that are hard linked, and my destination is on the same filesystem, does rsync copy that file to the destination, preserve the hardlinks, and end up with 2 hardlinked files in the source and 2 identical hardlinked files in the destination, or are all 4 files hard linked? I'm assuming all 4 are hardlinked but I wanted to make sure. I'm sure I will lose hardlinks if I store in the cloud but I'm sure I can fix that on restore.

Share this post


Link to post

Hi,

 

I created a appdata backup folder on my array and pointed to this folder in the settings. I manually ran a backup which saved the  backup data to the created folder but it also created a backup of the appdata in my appdata folder.  I have no idea why it did this.  Is this normal behaviour?  I can't see a setting where it would also create a backup in my appdata folder in addition to the folder I created in my user share on my array.

 

Thanks in advance,

 

Ryan.

Share this post


Link to post

Post the backup options.json file from the config/plugins/CA.backup folder on the flash drive

 

Sent from my LG-D852 using Tapatalk

 

 

Share this post


Link to post

There's nothing wrong in the xml (and I didn't expect there to be)

 

Are you sure that it's a backup of appdata being placed into your appdata share and not simple the backup of the USB and VM xml's?

 

Post your diagnostics as it outright includes the command that the backup runs in it.

Share this post


Link to post

Those are set as backup to the same destination as appdata.  I'll post the diagnostics shortly.

Share this post


Link to post

Question for you.

 

I use CA Backup to backup to a user share.  I then need to be able to backup the user share to an external hard drive.  This is probably more of a Crashplan issue, but when CA Backup copies the Crashplan files it picks up their permissions as well, listed below.  These prevent me from pulling the files down to my windows box.  Any ideas on how to approach this?  Thanks.

[code]
drwx------ 1 root   root   4096 Dec 21 18:39 adb/
-rw-r--r-- 1 nobody users  3365 Feb  7  2016 custom_sample.properties
-rw-r--r-- 1 nobody users  1119 Feb  7  2016 default.service.xml
-rw-rw-rw- 1 nobody users 24077 Jan 21 10:42 my.service.xml
-rw-rw-rw- 1 root   root   4826 Mar 25  2016 service.log.xml
-rw-rw-rw- 1 root   root     76 Feb  7  2016 service.login
-rw------- 1 nobody users  9024 Jan 21 18:43 service.model
-rw-r--r-- 1 nobody users  1384 Feb  7  2016 ui.log.xml
-rw-r--r-- 1 nobody users   278 Feb  7  2016 ui.properties
-rw-rw-rw- 1 nobody root     97 Dec  3 18:07 ui_nobody.properties
-rw-rw-rw- 1 root   root     50 Dec 21 18:39 ui_root.properties
-rw-rw-rw- 1 root   root      0 Feb  9  2016 upgradeui.log.properties
-rw-r--r-- 1 nobody users  1335 Feb  7  2016 upgradeui.log.xml
-rw-r--r-- 1 nobody users    44 Feb  7  2016 upgradeui.properties

 

Share this post


Link to post

Question for you.

 

I use CA Backup to backup to a user share.  I then need to be able to backup the user share to an external hard drive.  This is probably more of a Crashplan issue, but when CA Backup copies the Crashplan files it picks up their permissions as well, listed below.  These prevent me from pulling the files down to my windows box.  Any ideas on how to approach this?  Thanks.

[code]
drwx------ 1 root   root   4096 Dec 21 18:39 adb/
-rw-r--r-- 1 nobody users  3365 Feb  7  2016 custom_sample.properties
-rw-r--r-- 1 nobody users  1119 Feb  7  2016 default.service.xml
-rw-rw-rw- 1 nobody users 24077 Jan 21 10:42 my.service.xml
-rw-rw-rw- 1 root   root   4826 Mar 25  2016 service.log.xml
-rw-rw-rw- 1 root   root     76 Feb  7  2016 service.login
-rw------- 1 nobody users  9024 Jan 21 18:43 service.model
-rw-r--r-- 1 nobody users  1384 Feb  7  2016 ui.log.xml
-rw-r--r-- 1 nobody users   278 Feb  7  2016 ui.properties
-rw-rw-rw- 1 nobody root     97 Dec  3 18:07 ui_nobody.properties
-rw-rw-rw- 1 root   root     50 Dec 21 18:39 ui_root.properties
-rw-rw-rw- 1 root   root      0 Feb  9  2016 upgradeui.log.properties
-rw-r--r-- 1 nobody users  1335 Feb  7  2016 upgradeui.log.xml
-rw-r--r-- 1 nobody users    44 Feb  7  2016 upgradeui.properties

Run the new permissions tool built-in to unRaid (FCP's docker safe new perms will always ignore backup locations)

 

This is not an issue with the backup, nor will I change it to change the permissions, as at that point it is no longer a backup, and in the event of a restore there may be issues with the apps.

Share this post


Link to post

Thanks, I know it's not an issue with the backup.  I'll run the new perms and that should at least let me pull down the files on an ad-hoc basis.  I'm sure the permissions will come back with the next backup, though.

 

I'd like to fully automate pulling down the backed-up files to the windows box, but I'm not sure if there's a good approach given that CA Backup is doing what it's supposed to - faithfully backing up Crashplan's permissions along with the files.

Share this post


Link to post

Diagnostics attached.  It does seem to be only the usb and vm_xml.  It's possible that my settings are wrong.

The anonymizer has messed up the line.  Can you give me the output of this command:

 

cat /var/log/syslog | grep -i "Using Command:"

 

Share this post


Link to post

Jan 22 12:29:24 Tower root: Using command: /usr/bin/rsync -avXHq --delete --excl        ude "/mnt/cache/docker.img"  --log-file="/var/lib/docker/unraid/ca.backup.datast        ore/appdata_backup.log" "/mnt/user/appdata/" "/mnt/user/Documents/appdatabackup"          > /dev/null 2>&1

Share this post


Link to post

I missed in your reply that it seems to be only the USB and XML backups being placed into appdata.

 

Yes, that is normal when they are set to same destination.  It backs it them up to appdata and then backs up the appdata.  Far, far simpler....  Personally, I've set them to a separate destination...

Share this post


Link to post

Is it possible to restore a single file from the flash backup?

What single file did you have in mind? Most of the files on flash are just text files that save settings made in the webUI. You can open them up and read them. In fact, I highly recommend it. Helps make things a little less mysterious.

Share this post


Link to post

smb.conf

Had to downgrade my server from 6.2.4 to 6.1.9 because of problems with my SuperMicro motherboard that prevented booting. Now I've resolved the problems and back on 6.2.4, but somewhere in the process I lost the file I had modified and I'm back to the vanilla settings. I know the flash backup is still on my backup disk. Was hoping I could just pull the file from there.

Share this post


Link to post

Yeah, copy it from the backup destination onto the flash manually...

Share this post


Link to post

I would like to understand if I have done something wrong with excluding folders. This is the specific line of BackupOptions.json:

"excluded": "/mnt/user/system/docker/appdata/emby/cache, /mnt/user/system/docker/appdata/emby/metadata",

 

However all Emby metadata as well as the Emby cache is being saved every day. Anything I need to correct in the definition above?

 

Thanks a lot.

Try a relative folder.  ie: only type in emby/cache and emby/metadata

Thanks a lot Squid, that was doing the trick... and I had to remove the space between the two items.

Share this post


Link to post

Im getting a weird issue here an am running out of ideas. I ran my first backup last week and the backup completed successfully however my linuxserver/mariadb container refused to function and nothing that used that container would work either. I ended up having to remove the databases and then move the backed up files back to the appdata folder and then mariadb started functioning normally again. I assumed it was something to do with the backup so i excluded it at that point. Last night the backup ran again and mariadb was not backed up as intended, however the same issue persists where the container wont function and nothing can connect to it. I assume it has something to do with the way this plugin is stopping containers or updating them or something. Any ideas?

Share this post


Link to post

No real clue.  The back does standard docker stop and docker start commands.  You can tell it to keep any container running thru the advanced section

 

Sent from my LG-D852 using Tapatalk

 

 

Share this post


Link to post

No real clue.  The back does standard docker stop and docker start commands.  You can tell it to keep any container running thru the advanced section

 

Sent from my LG-D852 using Tapatalk

 

Ok, will try that and see what happens.

Share this post


Link to post

Hey all. First off, great plugin! It works wonders to backup my Docker implementation each night.

 

I've got two quick questions -

 

[*]Is there a way to add a secondary backup source/dest? I'd love to backup another unrelated share to an external server mounted via the Unassigned Devices Plugin

[*]Is there a way to have the notifications be sent via the system default? Right now, I use "Agent" for all notifications to get info sent via Pushover to my devices. But for some reason, CA Backup only presents browser toast notifications.

 

Thanks for the assistance, and keep up the wonderful work!

Share this post


Link to post

Hey all. First off, great plugin! It works wonders to backup my Docker implementation each night.

 

I've got two quick questions -

 

[list type=decim

  • Is there a way to add a secondary backup source/dest? I'd love to backup another unrelated share to an external server mounted via the Unassigned Devices Plugin
  • Is there a way to have the notifications be sent via the system default? Right now, I use "Agent" for all notifications to get info sent via Pushover to my devices. But for some reason, CA Backup only presents browser toast notifications.

 

Thanks for the assistance, and keep up the wonderful work!

1 - Not currently.  I have however begun initial planning on a new plugin (or significantly expanding the capabilities of this one) that will have multiple sources, multiple destinations, multiple schedules, etc etc etc.

2 - The plugin uses the built-in notification system.  You should be able to enable Agents on the Notifications from Settings Notifications, and have it work.  I don't use any agents at all so not sure about that though.

Share this post


Link to post

Awesome plug man, has saved me quite a few times from myself accidentally deleting stuff from appdata.
 Couple of questions:
 

  1. is stopping dockers really necessary? Never seen the 'file open error' when copying appdata manually;
  2. how did you find out which files need to be backed up for vms? (it's not just the xml templates);
  3. as an extension to 1st - what logic is applied on the docker restart? Every time backup job finishes, it also starts dnsmasq after starting the dockers (and other services?), so the dns docker fails to start up since port 53 is occupied (my issue thread)

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now