[Plugin] Appdata.Backup

[sumsh]

Is there a way to set a limit on the number of flash drive backups kept? I’ll eventually run out of disk space, if not. 

[Blasman]

Thanks for this plugin. It's been great! I have a question regarding custom scripts.

 

Would it be possible to be able to run a custom script after dockers are stopped but before anything else happens? Currently, 'pre-backup' custom scripts run before dockers are stopped, and 'post-backup' custom scripts run after backups are done and dockers containers are updated.

 

The reason being is that I wrote some custom ZFS scripts (snapshots, etc) that would ideally work best at the time dockers are stopped and before the update dockers feature happens. I know that 'update dockers' is an option, but I like having my dockers update using the plugin and would like to keep using that feature if possible. Thanks!

[ConnerVT]

18 hours ago, benweiner said:

Do you know how to overcome this error? 

 

You are stopping Plex, but do you possibly have some other program which may be updating the Plex database?  If so, you can/should stop all of the related Dockers together.

[droopie]

This plugin hasn't been working for me. It never completes the backup and just stops. Is there an alternative? I got this code d68f7d91-3109-432a-8bf3-762e0ca9e535

[KluthR]

4 hours ago, droopie said:

This plugin hasn't been working for me. It never completes the backup and just stops

Maybe you should reading the log then? The error comes from tar which fails to backup due to "mod time differs" error. The path in question (/mnt/cache/appdata/plex) is being accessed by (another?) Plex. The container "plex" however, is stopped.

 

So, some other process is modifying the files during backup. Thats the issue.

[KluthR]

14 hours ago, sumsh said:

Is there a way to set a limit on the number of flash drive backups kept?

One per backup. So, changing number of to-keep-backups also changes the to-keep- flash backups.

[KluthR]

11 hours ago, Blasman said:

Would it be possible to be able to run a custom script after dockers are stopped but before anything else happens?

That would only apply to "Stop-All" method then. But: No, not possible currently. preBackup is being triggered before determining the method. postBackup after dockers are started again. Your request would require some changes in the code.

[KluthR]

19 hours ago, ABEIDO said:

Is there a way to make it not see it as Plex?

No. And it does not harm that it does so. A simple "plex" in any name at any position triggers this UI-only information.

[KluthR]

On 8/1/2024 at 5:14 AM, benweiner said:

Do you know how to overcome this error?

Seems that somewthing is still working with the files. The debug log is missing the lsof section for tar creation (it shows what is accessing the paths). This section is only being written if verification fails, not if creation fails. I should change this. I dont see any other container using the path though.

[ABEIDO]

2 hours ago, KluthR said:

No. And it does not harm that it does so. A simple "plex" in any name at any position triggers this UI-only information.

Ok thanks for info, so its only the name it reacts to.

[sumsh]

8 hours ago, KluthR said:

One per backup. So, changing number of to-keep-backups also changes the to-keep- flash backups.

This hasn’t been my experience. It’s been creating a flash backup for every app data backup, but not removing old ones, so I have a ton of flash backups. 

[Kilrah]

The flash backup is in the same folder as the appdata backup and so is removed when the whole folder is. If the whole folder isn't removed it's a different subject and is probably because backups are marked as failed.

Edited August 2 by Kilrah

[benweiner]

20 hours ago, KluthR said:

Seems that somewthing is still working with the files. The debug log is missing the lsof section for tar creation (it shows what is accessing the paths). This section is only being written if verification fails, not if creation fails. I should change this. I dont see any other container using the path though.

I've just made it so everything is off until the backup is done and it all works now! Thanks to you and to @ConnerVT for the help. (And you, @KluthR, for the fabulous plugin)

[dcuellar44]

Got a couple of questions that I was hoping to have answered before I set this up.

 

I have a cache pool of two nvme's setup in single mode so that I can use the full space of each drive. Naturally, I want to be able to back up the cache now in case one of the drives fails. Can this plugin do that or is it truly only meant to backup the docker containers? I'm asking because I was recommended the CA Backup Plugin and was told that could do it but that, as we know, has been deprecated.

 

I should mention that I will have the appdata share stored on the cache, not the array. It will live in the cache permanently and not be moved. 

[Kilrah]

This backups the appdata of your containers, not the entire cache, although you could probably add extra folders or use it to run your own script for whatever you want to do with the rest.

[dcuellar44]

6 hours ago, Kilrah said:

This backups the appdata of your containers, not the entire cache, although you could probably add extra folders or use it to run your own script for whatever you want to do with the rest.

Ok. I wonder what else is in cache then. I'm pretty sure it's just any newly downloaded items, right?

 

Also, a follow-up question for anyone... When this creates the backups does it keep one backup and replace it each time? Does it only update changes? I have over 800GB of data on cache from appdata. I imagine this would take forever if it was just replacing the backup each time. 

[larrykim2]

On 6/6/2023 at 12:27 PM, rogueosb said:

Would it be possible to add an option to pass gzip arguments such as --rsyncable (for smaller rsync deltas)? I'm guessing using something like the following for tar:

-I 'gzip ARGS'

 

Can probably achieve this with a post-script anyway, but would be nice to have this integrated.

Last year, rogueosb suggested --rsyncable to make it easy to backup the backups, but I didn't find any follow up on this. Can you take another look? I appreciate it!

[dcuellar44]

1 minute ago, larrykim2 said:

No, it creates a new backup. I wonder how you can have so much appdata, I have 20 containers but only 3 GB appdata. "Cache" is a drive, "appdata" is a share. Don't you have other shares on your cache drive? 

Yeah. I have other shares on the cache drive. My appdata contains my client's websites. They are photographers and they have tons of photos they sell on their e-commerce sites. All of that data is stored in MariaDB containers.

[larrykim2]

1 minute ago, dcuellar44 said:

Yeah. I have other shares on the cache drive. My appdata contains my client's websites. They are photographers and they have tons of photos they sell on their e-commerce sites. All of that data is stored in MariaDB containers.

I see, I suggest you map the mariadb volume to another share in the cache so you can use other tools to back it up. This plugin is not designed to backup this much data.

[dcuellar44]

1 minute ago, larrykim2 said:

I see, I suggest you map the mariadb volume to another share in the cache so you can use other tools to back it up. This plugin is not designed to backup this much data.

Got it. Thanks!

[Dro]

There are so many pages to look through. I tried to check the last few but didn't see it.. Is it a known issue on 7 /beta2 that this plugin does not see unassigned drives? Everything else sees my unassigned drive just fine besides the plugin so i know its no an unassigned drive issue. Thanks! I usually send my appdata backups to unassigned drives.. Assuming this is an issue with the 7 beta, which is fine.. Just need to know so i can stop troubleshooting it.

[Mysteic]

I have been reading in here and changing configs, and reading the help info in the plugin for hours and I just give up. I only care about backing up plex app data and it either fails if I include mnt/cache/.... or doesn't back anything up with "there isn't any volume" when I only include mnt/user/.....

 

I attached the debug.log but at this point I might just delete the plugin and move on.

backup.debug.log

[KluthR]

4 hours ago, Dro said:

Is it a known issue on 7 /beta2 that this plugin does not see unassigned drives?

I dont think that this is an issue with the plugin, since it just sees the filesystem. I dont know how Unassigned drives are mounted on the current beta but, if its a simple path like now, it should work. I didnt had the time to setup my dev environment to beta yet.

 

2 hours ago, Mysteic said:

I only care about backing up plex app data

The log shows, that verification fails due to files which shrank. Is anything outside plex accessing these files? I dont see anything on the docker side which uses the same path.

[Kilrah]

Indeed doesn't list all contents of /mnt on 7.0.0b2, might be at unraid level though?

 

[Dro]

7 hours ago, KluthR said:

I dont think that this is an issue with the plugin, since it just sees the filesystem. I dont know how Unassigned drives are mounted on the current beta but, if its a simple path like now, it should work. I didnt had the time to setup my dev environment to beta yet.

 

 

Thank you, yea for some reason, when going into the plugin, choosing my backup destination the "disks" folder where the external unassigned drive is found is not showing up.. It does work on previous version.. so i am not sure where the issue is. I am able to see that unassigned drive from other things like SMB, unraid gui itself, can browse to it and use it ok.. Not sure why its not showing up in the app.. I'm sure its something related to the beta, which is fine as its beta..