[Plugin] Appdata.Backup

[Hardy]

Great plugin, thank you for your efforts!

 

One feature request i have would be to be able to group updates of certain containers.

I managed to corrupt my Nextcloud installation because i shutdown the database (because of an update) one time.

Since then i stop Nextcloud before updating the database container.

 

I could stop them all together with your plugin already, but the Nextcloud instance takes some time and the rest could stay up meanwhile.

 

So if we could make a backup group and shut them down in a certain order and start them together the plugin would be perfect for me.

 

Thank you again

Edited August 23, 2023 by Hardy

[nearcatch]

Feature request described below. TLDR: a way to provide exclusions to extra files

 

I have a folder in my appdata folder called "mergerfs" where my jellyfin nfos and images are stored. This is also where jellyscrub stores thumbnails for scrubbing using "trickplay" folders in each media item's directory under "mergerfs". These trickplay folders greatly bloat the size of the "mergerfs" folder and I would like to exclude them, but otherwise backup everything in mergerfs.

[KluthR]

On 8/22/2023 at 4:04 PM, urbanracer34 said:

How can I set the backup destination to a share?

You just have to set the absolute path as destination.

 

On 8/22/2023 at 4:18 PM, Trinistan said:

Getting an error when attempting to backup my appdata.

You try to backup jellyfin within jfa-go. Please exclude the jellyfin share.

 

On 8/23/2023 at 10:13 AM, helvete said:

I'm getting a warning in the logs and the folders are creating with a -failed

"Tar said: tar: Removing leading `/' from member names; mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Mod time differs; mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Size differs". Something is changing the mentioned files. Some other container which has the tdarr volume attached?

[Ancient Wizard]

On 8/2/2023 at 12:47 AM, LoyalScotsman said:

So when i first set up this it was failing so i now have a selection of failed backups that i cannot delete, i have tried to delete them via kurseder but it refuses to delete any advice on how to delete the failed one because taking up unnessaray space

 

I have the same problem. Only 2 dockers are installed. Bin Hex Krusader and Bin Hex Plex Pass. The failed files are not dropping off and I cannot manually clear them. Older successful files may be dropping off but I cannot say for certain. A solution would be appreciated.

[helvete]

6 hours ago, KluthR said:

"Tar said: tar: Removing leading `/' from member names; mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Mod time differs; mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Size differs". Something is changing the mentioned files. Some other container which has the tdarr volume attached?

Possibly, so I  think best to close all apps back up and then restart

Also any idea on why my plex appdata folder is not backing up, its really the one I  wouldn't want to lose.

[KluthR]

1 hour ago, Ancient Wizard said:

I have the same problem.

Will looking into this the next days

 

32 minutes ago, helvete said:

Also any idea on why my plex appdata folder is not backing up

Please share the debug log.

[KluthR]

11 hours ago, Ancient Wizard said:

I have the same problem. Only 2 dockers are installed. Bin Hex Krusader and Bin Hex Plex Pass. The failed files are not dropping off

Could you provide a "ls -la" from inside your backup destination within the Unraid terminal?

[helvete]

16 hours ago, KluthR said:

 

 

Please share the debug log.

Here you go - thank

backup.debug.log

[MadMatt337]

Not sure if it is just myself as I don't see reference to it in previous posts but I have noticed my dockers are no longer automatically updating after backup. As far as I can tell this started happening after the 2023.08.16 update. The logs are showing that the dockers are all up to date but in this particular log I have 5 different dockers that have updates available.

backup.log

[EthanBezz]

5 hours ago, MadMatt337 said:

Not sure if it is just myself as I don't see reference to it in previous posts but I have noticed my dockers are no longer automatically updating after backup.

I can confirm it's not just you, my containers have also not been updating. My logs, like yours, also say that there's no update available for containers which do have updates.

Edited August 25, 2023 by EthanBezz

[Mathew R.]

On 8/8/2023 at 10:53 PM, alkon_rojo said:

Hi to everyone, and first Thanks so much for this awesome plugin.

 

I'm facing one problem regarding the task of updating a docker during the run, when the same image docker is being use over two dockers:

 

[09.08.2023 04:59:52][Main][info] Auto-Update for 'bazarr' is enabled - checking for update...
[09.08.2023 04:59:53][Main][info] Update available! Installing...
[09.08.2023 05:00:10][Main][info] Update finished (hopefully).
[09.08.2023 05:00:10][Main][info] Auto-Update for 'ApacheGuacamole' is enabled - checking for update...
[09.08.2023 05:00:12][Main][info] No update available.
[09.08.2023 05:00:12][Main][info] Auto-Update for 'bazarr-vo' is enabled - checking for update...
[09.08.2023 05:00:15][Main][info] No update available.
[09.08.2023 05:00:15][Main][info] Auto-Update for 'sonarr-vo' is enabled - checking for update...
[09.08.2023 05:00:18][Main][info] No update available.
[09.08.2023 05:00:18][Main][warning] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum.
[09.08.2023 05:00:18][Main][info] DONE! Thanks for using this plugin and have a safe day ;)
[09.08.2023 05:00:18][Main][info] ❤️

 

The problem is for example in this logs the dockers are pulling over the same docker image is "Bazarr" so I have two docker "bazarr" and "bazarr-vo" and when it has a new image version only gets update the first that it finds it, the next docker sharing the same docker image finds that there's no new image to pull, but when it start it doesn't start updated. So this problema is the same with all the dockers that are sharing the same docker image, so only gets update the first that finds the new docker image to be update, the rest never gets update.

 

I was facing several this problem several days ago, but in the community I didn't find anyone with the problem.

 

Thanks to all and best regards.

 

5 hours ago, MadMatt337 said:

Not sure if it is just myself as I don't see reference to it in previous posts but I have noticed my dockers are no longer automatically updating after backup. As far as I can tell this started happening after the 2023.08.16 update. The logs are showing that the dockers are all up to date but in this particular log I have 5 different dockers that have updates available.

backup.log 14.76 kB · 2 downloads

 

2 minutes ago, EthanBezz said:

I can confirm it's not just you, my containers have also not been updating. My logs, like yours, also say that there's no update available for containers which do have updates.

 

I as well can also confirm this issue has been present for a little while now.

[jbrown705]

I am having an issue with getting any backup to complete.  I have been trying for about 2 weeks without success.  I keep getting messages about "tar creation failed ..." and usually something about a file within a docker changing before it was read, then a the end it tell me at the end: "[23.08.2023 05:15:44][Main][warning] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum."

 

I have also uploaded my logs.  Here is my log ID: c1b978de-a3a4-4d79-82e4-3dd8a46290ca

 

  

[Kilrah]

On 8/26/2023 at 5:30 AM, jbrown705 said:

and usually something about a file within a docker changing before it was read

Either allow the container to stop or set the plugin to ignore errors for those containers, knowing the backup for those won't be consistent.

Edited August 27, 2023 by Kilrah

[sonic6]

On 7/25/2023 at 8:34 AM, KluthR said:

The stop mechanism is exactly the same for all options.

 

Have to test this later then. Currently Iam not at home.

so i tested the backup again with your changes to the "stop, backup start for each container" and it worked good for me. thank you for that changes.

is it right, that there is no log from the last backup periode? the shedule runs last night, but the log is emty or not existing.

[MarianKoniuszko]

Hello, everytime Appdata Backup plugin do the job my docker service stop working. Some docker apps works like emby and *arrs but other don't work.

 

[ArveVM]

Version: 6.12.3, AppdataBackup 2023.08.16 

- is there an issue with the docker-update functionality ??

i have a few dockers that is not updating,, sqlitebrowser as an example:

 

Appdata backup has run after this,, finding no update:

 

in Appdata-settings i have the "update containers after backup" set to YES , and this is the specific:

 

Sometimes i just click the updates when doing other stuff,, so not really sure if it has worked at all after updating to 6.12?
Is there something else I need to configure or any other checks to do??

 

[rapt]

4 hours ago, ArveVM said:

Version: 6.12.3, AppdataBackup 2023.08.16 

- is there an issue with the docker-update functionality ??

i have a few dockers that is not updating,, sqlitebrowser as an example:

 

Appdata backup has run after this,, finding no update:

 

in Appdata-settings i have the "update containers after backup" set to YES , and this is the specific:

 

Sometimes i just click the updates when doing other stuff,, so not really sure if it has worked at all after updating to 6.12?
Is there something else I need to configure or any other checks to do??

 

I have the same issue, i am not completely sure when it started but updates were working for sure a few weeks ago.

[randomusername]

On 8/11/2023 at 8:47 PM, KluthR said:

Feels wrong.

 

could you open up the archive and see what root (with one layer deeper) are inside?

plex.tar.gz/mnt/user/appdata/plex

[KluthR]

On 8/25/2023 at 1:41 PM, helvete said:

Here you go - thank

You mapped the binhex-plex /config to "/mnt/cache/appdata/", thats why. Prove me wrong, but that seems not right. The /mnt/cache/appdata ist also listed as source directory, si its being skipped. Any other mappings are treated as external volume mappiung as discarded, resulting into no folder being backed up.

 

On 8/26/2023 at 1:41 AM, Mathew R. said:

I as well can also confirm this issue has been present for a little while now.

While writing this, Iam on my way to find the bug.

 

On 8/26/2023 at 5:30 AM, jbrown705 said:

I am having an issue with getting any backup to complete.

Seems, that the mentioned container is accessing data from Home-Assistant which, I guess, is running? If so, please exclude the HA mapping from that metioned container.

Sidenote: Future update will implement a is-a-mapping-mapped-for-more-than-one-container? feature. That will detect multiple mappings. What the plugin does then is still open, but I feel the need for such detection.

 

13 hours ago, sonic6 said:

is it right, that there is no log from the last backup periode? the shedule runs last night, but the log is emty or not existing.

If something is cleaning up files inside /tmp, it could lead to this behavior. The log file is saved within /tmp and is being cleared usually after a reboot only.

 

13 hours ago, MarianKoniuszko said:

Hello, everytime Appdata Backup plugin do the job my docker service stop working

The plugin does NOT touch the docker service itself in any way! Please share a debug lof for further inspection.

 

8 hours ago, ArveVM said:

is there an issue with the docker-update functionality

Yep, see above. Im on it.

 

 

3 hours ago, randomusername said:

plex.tar.gz/mnt/user/appdata/plex

Seems right then. Hmm.

 

Edited August 28, 2023 by KluthR

[KluthR]

2023.08.28 should fix the docker auto update issue.

[TheControlled]

Quote

[28.08.2023 12:28:33][ℹ️][Main] 👋 WELCOME TO APPDATA.BACKUP!!
[28.08.2023 12:28:33][ℹ️][Main] Backing up to: /mnt/stockpile/backups/ab_20230828_122833
[28.08.2023 12:28:33][ℹ️][Main] Saving container XML files...
[28.08.2023 12:28:33][ℹ️][Main] Method: Stop/Backup/Start
[28.08.2023 12:28:33][ℹ️][Plex] Stopping Plex... done! (took 9 seconds)
[28.08.2023 12:28:42][ℹ️][Plex] Should NOT backup external volumes, sanitizing them...
[28.08.2023 12:28:42][ℹ️][Plex] Plex does not have any volume to back up! Skipping
[28.08.2023 12:28:42][ℹ️][Plex] Starting Plex... (try #1) done!

Hello! I'm trying to get this app to work but the above keeps happening. It happens with qBitTorrent too.

 

Thanks!

 

d8f3925c-6e39-4577-b996-071baf38aef8

newflesh-diagnostics-20230828-1240.zip

[KluthR]

18 minutes ago, TheControlled said:

Hello! I'm trying to get this app to work but the above keeps happening.

Easy one: add /mmt/user/appdata as allowed sources as well (and remove the docker.img path).

 

please read the hints inside the help block for the sources field.

[TheControlled]

TIL what help blocks are.

 

That solved it. Thank you soooooooo much. Simple, but made my day! I just recently lost all my appdata and it was devastating losing all my Plex data. Now I can breathe easy. Thanks again.

[samsausages]

Hi, I think I might have found a bug related to how the plugin handles the "Extra Files" section and Unriads new "Exclusive Shares"
I have a share called share1 that is an exclusive share under /mnt/user/share1
I added that path to the "extra files" section long ago and it always worked as expected.  But now I'm on 6.12 with Exclusive Shares enabled and when I went to restore from a backup today I found out that the files are no longer being backed up.  Looks like because of the symbolic link.
Containers that use exclusive shares are backing up correctly, this seems to be limited to the "Extra Files" section.

I'm assuming I can work around that by using /mnt/poolname/share1 but I wanted to report the behavior as it looks like files for people using exclusive shares probably aren't backing up as expected and no error/warning is triggered.
 

Edited August 28, 2023 by samsausages

[KluthR]

Good point, have to test that.