[Plugin] Appdata.Backup

[KluthR]

Appdata.Backup

Support Thread

This is the support thread for appdata.backup (formerly known as ca.backup2).

 

This plugin primary takes care of your appdata backup! It allows you to configure backup settings for each of your docker containers. Flash and VM meta backup is integrated as well!

 

If you encounter any issues, post it here with the debug log file attached!

For your beta feedback, please post here instead!

 

Why do I need to reinstall the plugin (again)?

After my adoption of the ca.backup2 plugin (which needed a manual reinstall from every user the 1st time), I decided to make a complete overhaul of the whole plugin. The main reason was the PHP8 incompatibility. This new plugin is the result. Because it is actually a seperate new plugin, you have to switch to it manually again. This will be the last time

 

Hints

The plugin send me here to check for some Plex exclusions.

Yes, thats right. You dont need to save Plex cache (and some other) directories. But its up to you to exclude them or not. Currently you are advised to set the following exclusions for plex:

• Each following paths need to be prefixed with your correct appdata path!
  • Library/Application Support/Plex Media Server/Cache
  • Library/Application Support/Plex Media Server/Media
  • Library/Application Support/Plex Media Server/Metadata

 

The plugin sent me here for help with the grouping function

The grouping function allows you to group cotainers. The backup process stops all those containers, backup them and starts them back up before going to the next single container. This works for the mode "Stop, backup, start for each container" only. If the plugin reaches a group, it stops all containers in it, backup them and start them back up. The backup method "Stop all, backup, start all" has no effect here because every single container will be stopped nefore starting backup.

Edited November 28 by KluthR

[KluthR]

Known issues

• Sometimes, users backing up Plex, are seeing backup issues for some "Application Support subfolders"

Edited August 16 by KluthR

[KluthR]

ToDo / Roadmap

• backup without packing files into an archive
  • https://forums.unraid.net/topic/132721-plugin-ca-appdata-backup-restore-v25/page/18/#comment-1251220
• Use Dockerman default order when no order was set before
• Per container option „No backup“ which includes the container to the stop/start process without backing them up (for container dependencies)
• Container groups to manage dependencies
  • https://forums.unraid.net/topic/137710-plugin-appdatabackup/?do=findComment&comment=1276020
• Add option to adjust compression level
• Add option to set cpu limit for multicore compression
• Anonymous debug sharing with the dev
• Anonymous statistics (Opt-In)

Edited November 3 by KluthR

[ConnerVT]

I'm not ready to fire this up and try it yet.  But have been following the threads for some time now.

 

Thanks for all of your time and effort bringing this up to date!

[Kazino43]

The log seems fine until the end after the back up of the flash drive: 

 

[warning] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum.

 

[KluthR]

Please attach the debug log.

[Kazino43]

Ah, sorry, didn‘t save the edit.

 

Edited April 19 by Kazino43

[KluthR]

Your icloudpd container is the issue:

 

It has the following volume mappings:

(
    [0] => /mnt/user/appdata/icloudpd:/config:rw
    [1] => /mnt/user/iCloud_Daniel:/home/user/iCloud:rw,slave
    [2] => /mnt/user/appdata/icloudpd/root:/root:rw
)

 

You can see, that /root is inside the mapping for /config. So your unraid path for config is /mnt/user/appdata/icloudpd and your /root path points to /mnt/user/appdata/icloudpd/root.

 

This causes a verification error during backup (at tars side). While it is working, it would make more sense to seperate those mapping into /mnt/user/appdata/icloudpd/root and /mnt/user/appdata/icloudpd/config.

 

This is the 2nd time I see such issue. Maybe the plugin should remove mappings within another mapping automagically. The final backup contents would be the same - but without verification errors.

[Kazino43]

Just wanted to confirm that seperating those two paths did clear the error on a new backup run. Thanks. I guess for now we just have to remember that. Question is how many user are really having this sort of path mapping. For me at least it was the original way from CA. Might be that many others just change such a mapping directly instead of copying the template 1:1.

Edited April 11 by Kazino43

[KluthR]

44 minutes ago, Kazino43 said:

For me at least it was the original way from CA

The Grafana Unraid Stack template has the same issue. I dont know if its wrong practice to do such mappings, but it feels bad. It works, though. Maybe the plugin should detect those and writes a warning.

[keywal]

I'm afraid i'm going to be that user that will be of little help with my issue

 

I have an issue here that results in a blank page with 'settings' as a tab in the top left.

The rest of the page is blank...

The only way i can regain access is remove the plugin and reinstall it.

This doesnt last long though before the screen 'refreshes' and goes blank.

 

- Multiple browsers tried on MacOS 13.3 - Chrome, Firefox and Safari.

- Unable to get log file as i remove and reinstall; there is nothing there.

- Unraid Version: 6.12.0-rc2 

 

Also having an issue when i do have control where im trying to restore a back up - Doesnt matter how i fill it in, it tells me that it doesnt appear to be valid.

The back up was made by the previous version of the tool. (ca.backup2)

Let me see if i can trigger that error and see if it has anything in the debug logs for you.

 

[KluthR]

Ooops. Thats interesting.

 

If you see the issue again: Please open Tools -> PHP Options, set "Error reporting level" to "All categories" and check the "Show errors on screen" checkbox. Hit apply and open the page again. Im more than excited to see what the output will be...

 

(After you reboot the next time, those PHP settings will be reset).

 

You also not on latest RC, but that should not be the issue here.

Edited April 11 by KluthR

[keywal]

PHP Fatal error:  Uncaught TypeError: Cannot assign string to property unraid\plugins\AppdataBackup\ABSettings::$containerOrder of type array in /usr/local/emhttp/plugins/appdata.backup/include/ABSettings.php:82


Stack trace:
#0 /usr/local/emhttp/plugins/appdata.backup/pages/content/settings.php(121): unraid\plugins\AppdataBackup\ABSettings->__construct()
#1 /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(706) : eval()'d code(2): include('/usr/local/emht...')
#2 /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(706): eval()
#3 /usr/local/emhttp/plugins/dynamix/template.php(82): require_once('/usr/local/emht...')
#4 {main}
  thrown in /usr/local/emhttp/plugins/appdata.backup/include/ABSettings.php on line 82

 

Also this is the latest RC i can see - Maybe you are special?  

 

EDIT: Sorry there was more to that which could be useful. 

Edited April 11 by keywal

[KluthR]

Oops. Looking o to that soon. 
 

37 minutes ago, keywal said:

Also this is the latest RC i can see - Maybe you are special?

Maybe. Iam on 2.10 now. But we developer have other builds. I thought those were the same now. 

[KluthR]

1 hour ago, keywal said:

I have an issue here that results in a blank page

Do you have any docker containers in your setup yet?

 

1 hour ago, keywal said:

Also having an issue when i do have control where im trying to restore a back up - Doesnt matter how i fill it in, it tells me that it doesnt appear to be valid.

The back up was made by the previous version of the tool. (ca.backup2)

Unfortunately, those arent valid backups anymore for this plugin version.

[KluthR]

@keywalPlease update, the issue should be gone.

[keywal]

Awesome - thank you for your quick action!

I will poke it later and see if its working now.

 

[halorrr]

I'm not sure what it was trying to include but I just ran this and had to abort it because two of the apps it backed up (bazarr and krusader) ended up with ~800GB in size for their backups:

 

 

I'm running version 2023.04.11

 

Thanks

 

Edited April 15 by halorrr

[anibis2]

9 minutes ago, halorrr said:

I'm not sure what it was trying to include but I just ran this and had to abort it because two of the apps it backed up (bazarr and krusader) ended up with ~800GB in size for their backups:

 

 

I'm running version 2023.04.11

 

Thanks

I'm having the same issue with Plex. The .tar for my 32GB Plex appdata folder was up to ~250GB before I stopped the backup. I tried excluding my media folders thinking it was grabbing those, but that didn't affect anything. I also tried to exclude my Grafana db's, but nah. Good thing we're in beta.

 

Also, I run Unraid in dark mode, and the coloring is off when you drive to the individual docker settings. 

 

Edited April 15 by darcon

[halorrr]

For what it is worth here is the section of the debug log for bazarr and krusader for that run:

 

[14.04.2023 14:08:07][info] Stopping bazarr... 
[14.04.2023 14:08:12][info] done! (took 5 seconds)
[14.04.2023 14:08:12][debug] Backup bazarr - Container Volumeinfo: Array
(
    [0] => /mnt/user/Movies/:/movies:rw
    [1] => /mnt/user/TV Shows/:/tv:rw
    [2] => /mnt/user/Reality TV/:/realityTV:rw
    [3] => /mnt/user/zTV/:/zTV:rw
    [4] => /mnt/user/Anime/:/anime:rw
    [5] => /mnt/user/appdata/bazarr:/config:rw
)

[14.04.2023 14:08:12][debug] Final volumes: /mnt/user/Movies, /mnt/user/TV Shows, /mnt/user/Reality TV, /mnt/user/zTV, /mnt/user/Anime, /mnt/user/appdata/bazarr
[14.04.2023 14:08:12][debug] Target archive: /mnt/user/Backups/Appdata Backup/ab_20230414_140800/bazarr.tar
[14.04.2023 14:08:12][debug] Generated tar command: -c -P -f '/mnt/user/Backups/Appdata Backup/ab_20230414_140800/bazarr.tar' '/mnt/user/Movies' '/mnt/user/TV Shows' '/mnt/user/Reality TV' '/mnt/user/zTV' '/mnt/user/Anime' '/mnt/user/appdata/bazarr'
[14.04.2023 14:08:12][info] Backing up bazarr...
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/Authelia.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/bazarr.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/binhex-krusader.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/data.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/binhex-qbittorrentvpn.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/calibre.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/calibre-web.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/chromedp.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/ddclient.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/heimdall.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/homeassistant.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/mariadb.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/jdownloader2/cfg.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/jdownloader2/logs.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/ombi.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/overseerr.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/plex.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/plex-eros.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/prowlarr.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/qbittorrent.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/qbittorrent-unlimited.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/radarr.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/sonarr.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/stash/generated.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/stash/binary-data.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/stash/config.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/stash/metadata.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/stash/cache.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/swag.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/recipes/data.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/recipes/media.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/tautulli.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/plex/Library/Application Support/Plex Media Server/Logs.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/tautulli.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/tdarr/server.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/tdarr/configs.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/tdarr/logs.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/trackerautologin/config.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/Transmission_VPN.
[14.04.2023 14:09:41][debug] unraid\plugins\AppdataBackup\ABHelper::isVolumeWithinAppdata: /mnt/user/appdata IS within /mnt/user/appdata/Transmission_VPN.
[14.04.2023 17:05:57][debug] Tar out: tar: /mnt/user/Backups/Appdata Backup/ab_20230414_140800/bazarr.tar: Cannot write: No space left on device; tar: Error is not recoverable: exiting now
[14.04.2023 17:05:57][error] tar creation failed! More output available inside debuglog, maybe.
[14.04.2023 17:05:57][info] Starting bazarr... (try #1)
[14.04.2023 17:06:00][info] Stopping binhex-krusader... 
[14.04.2023 17:06:01][info] done! (took 1 seconds)
[14.04.2023 17:06:01][debug] Backup binhex-krusader - Container Volumeinfo: Array
(
    [0] => /mnt/user/appdata/binhex-krusader:/config:rw
    [1] => /mnt/:/UNRAID:rw
    [2] => /boot/:/FLASH:rw
    [3] => /mnt/disks/:/UNASSIGNED:rw,slave
    [4] => /root/:/ROOT:rw
)

[14.04.2023 17:06:01][debug] Final volumes: /mnt/user/appdata/binhex-krusader, /mnt, /boot, /mnt/disks, /root
[14.04.2023 17:06:01][debug] Target archive: /mnt/user/Backups/Appdata Backup/ab_20230414_140800/binhex-krusader.tar
[14.04.2023 17:06:01][debug] Generated tar command: -c -P -f '/mnt/user/Backups/Appdata Backup/ab_20230414_140800/binhex-krusader.tar' '/mnt/user/appdata/binhex-krusader' '/mnt' '/boot' '/mnt/disks' '/root'
[14.04.2023 17:06:01][info] Backing up binhex-krusader...
[14.04.2023 20:07:34][debug] Tar out: tar: /mnt/user/Backups/Appdata Backup/ab_20230414_140800/binhex-krusader.tar: Wrote only 2048 of 10240 bytes; tar: Error is not recoverable: exiting now
[14.04.2023 20:07:34][error] tar creation failed! More output available inside debuglog, maybe.
[14.04.2023 20:07:34][info] Starting binhex-krusader... (try #1)

 

[KluthR]

Oops. Thank for reporting both issues, I check this asap!

 

Could you please share your complete debug log together with your config (/boot/config/plugins/appdata.backup/config.json)?

Edited April 15 by KluthR

[KluthR]

As I can see, the debug log excerpt does not showing sanitizng infos. So: Do you have "Backup external volumes" enabled? This would backup TV Shows etc as well.

Edited April 15 by KluthR

[anibis2]

44 minutes ago, KluthR said:

As I can see, the debug log excerpt does not showing sanitizng infos. So: Do you have "Backup external volumes" enabled? This would backup TV Shows etc as well.

 

 

I do not see a setting for "Backup external volumes". I do remember seeing it on the old Appdata Backup plugin. I did try disabling the flash drive / VM meta backup, but there was no change. 

 

Edited April 15 by darcon

[isvein]

Is there a way to duplicate this plugin or setup multiple schedules? 

I have some dockers i want to backup say ones a month and some I want to backup daily. 🤔

[KluthR]

1 hour ago, darcon said:

I do remember seeing it on the old Appdata Backup plugin

No, that cant be the case, since its a completely new feature

 

Open the per container settings by clicking on its name. There you will find it.

Not explicitly enabling it means, it should be off. Im curious what its current state is.