[Plugin] Appdata.Backup

[zoggy]

How would one exclude a folder/file from the flash backup?

As the nvidia plugin driver is ~250M which I would not want to include  "config/plugins/nvidia-driver/packages/*"

[KluthR]

4 hours ago, zoggy said:

but with this app, does the exclude path get constructed from the container root folder or none.?

Always full paths. Use the file selector to see example.

[KluthR]

4 hours ago, zoggy said:

How would one exclude a folder/file from the flash backup?

Currently no way. Unraids flash backup is the used method and it does not allow exclusions.

[zoggy]

30 minutes ago, KluthR said:

Always full paths. Use the file selector to see example.

 

ok took me a sec but I see, by default it only backs up the "internal volume", but then down below you can exclude things from within that via the selector

At first glance i didnt realize you could drill down, but I see now. Thanks!

 

Dunno how hard it would be to have that tree auto expand to the 'internal volume' path, but prob not worth the hassle

 

 

 

 

[KluthR]

23 hours ago, Masterwishx said:

is anyway the containers start order can be readed from 'userprefs.cfg' from Dockerman ?

Currently the new plugin has its own order setting. I dont know yet if your request would be a useful setting maybe?

 

20 hours ago, muffinman1604 said:

Not sure what happened

Seems your are such Plex user without these common Plex exclusion list. Dont know. But obe of the Plex Library files shrank during backup which lead to the failure. Search for [error] inside the debug log and youll see.

 

11 hours ago, Pete0 said:

Everything else I have left pretty much as it was by default.

Your two sources seem odd. The one is inside the other. Are you sure those appdata paths are right? Then please share the debug log. The plugin sanitized all other container volumes resulting in nothing to back up. Please also note the info help block from „sources“ field.

[KluthR]

34 minutes ago, zoggy said:

Dunno how hard it would be to have that tree auto expand to the 'internal volume' path, but prob not worth the hassle

I will try. This is a unraid tool and I already tried that but maybe not hard enough 😂

[Proxykn]

Hey, first thank you for your work on this app but i seem to have a problem and i can't find out why the backup failes.

i get and error whit the backup of tdarr and i dont know whats wrong with it all the others seem fine.

ab.debug.log

[Masterwishx]

1 hour ago, KluthR said:

Currently the new plugin has its own order setting. I dont know yet if your request would be a useful setting maybe?

 

It's order is A-Z, in v2.5 was right order not A-Z. 

In my case I have about 100 containers, 57 running. 

Also I'm using Docker Folder, sure I can set order manually. Or edit json...

 

The problem is some of containers needs to be started befor others like Databases. 

 

 

[Kilrah]

One suggestion, check for volumes to back up before stopping and just skip everything if there are none. Currently it stops container, maybe there's no volume to back up, then starts it. On containers that don't correctly respond to the shutdown signal it waits until timeout for no reason.

 

[16.06.2023 17:24:42][leantime][info] Stopping leantime... 
[16.06.2023 17:25:13][leantime][info] done! (took 31 seconds)
[16.06.2023 17:25:13][leantime][info] Should NOT backup external volumes, sanitizing them...
[16.06.2023 17:25:13][leantime][info] leantime does not have any volume to back up! Skipping
[16.06.2023 17:25:13][leantime][info] Starting leantime... (try #1)

 

[Pete0]

14 hours ago, KluthR said:

Your two sources seem odd. The one is inside the other. Are you sure those appdata paths are right? Then please share the debug log. The plugin sanitized all other container volumes resulting in nothing to back up. Please also note the info help block from „sources“ field.

By default (after first installation) the plugin has set these two paths:

/mnt/user/appdata

/mnt/cache/appdata

 

I removed the second one because they point to the same directory. But one through the unRAID filesystem layer (I forgot the name)

I left /mnt/user/appdata as this is the path where all docker container are stored - or is that wrong?

 

I added the jellyfin-conf directory to as this is not a docker container directory but a directory created by jellyfin docker. It contains all configuration. I wonder why it only works for two docker container directories. Every other directory is in the same location.

 

I attached a debug log. Thank you. ab.debug.log

 

Edit: I may have found my problem. For some docker container I changed data dir (where possible) from /mnt/user/appdata/ to /mnt/cache/appdata as they are always on the cache drive and this results in better performance.

 

I have not changed the configuration back yet. But my question would be:

Is it ok to leave the source configuration (as it was in the beginning) to:

/mnt/user/appdata

/mnt/cache/appdata

 

Does it not cause issues if they point to actually the same location?

Edited June 16, 2023 by Pete0

[Revan335]

26 minutes ago, Pete0 said:

I left /mnt/user/appdata as this is the path where all docker container are stored - or is that wrong?

appdata is the Path that store any Data/Configs ... from the Docker Containers.

The Docker Containers himself are stored under /mnt/system/docker.

When you are used the default Paths.

Edited June 16, 2023 by Revan335

[Revan335]

29 minutes ago, Pete0 said:

/mnt/cache/appdata

 

I removed the second one because they point to the same directory.

cache is only the Path for your Cache Pool without the Fuse Filesystem Layer.

user is Cache + Array with the Fuse Filesystem Layer.

[zoggy]

13 hours ago, Masterwishx said:

 

It's order is A-Z, in v2.5 was right order not A-Z. 

In my case I have about 100 containers, 57 running. 

Also I'm using Docker Folder, sure I can set order manually. Or edit json...

 

The problem is some of containers needs to be started befor others like Databases. 

 

 

 

the old ca backup app used to just inherit how the dockers were sorted in unraid docker gui.

it would show them in "unraid docker gui" reverse order, which is how it would stop them.

then when it starts back up it does the reverse of that (aka, the normal "unraid docker page gui" order)

 

 

 

 

 

 

 

Edited June 16, 2023 by zoggy

[Kilrah]

2 hours ago, Pete0 said:

I have not changed the configuration back yet. But my question would be:

Is it ok to leave the source configuration (as it was in the beginning) to:

/mnt/user/appdata

/mnt/cache/appdata

 

Does it not cause issues if they point to actually the same location?

That's how it should be since some containers will refer to /mnt/user/appdata and others to /mnt/cache/appdata, the plugin just considers that if the template has a path defined in any of the paths defined here it'll be considered an "internal" volume while any that are not are "external" and thus only backed up optionally if configured so in the container-specific settings. You would want to add any other appdata locations if you e.g. had a separate pool named differently to hold your appdata.

 

That's all to differentiate and by default only backup the settings, if a container has an appdata path that's 100MB and a media path that's 10TB... you don't want the 10TB backed up by default and the plugin needs a way to differentiate them, it does so with those paths.

Edited June 16, 2023 by Kilrah

[KluthR]

13 hours ago, Kilrah said:

One suggestion, check for volumes to back up before stopping

Thats already on my list, yea 👍

 

8 hours ago, Pete0 said:

I removed the second one because they point to the same directory.

Thats not why the paths is set there. The real paths are read from the container config. Please read the info help block from that source path setting. It decides which volumes get back up if „skip external volumes“ is on.

 

 

[KluthR]

21 hours ago, Masterwishx said:

It's order is A-Z

„Its“ (you mean the plugin?) order is configureable. Default could be A-Z, I dont remember. Default could be picked up by Dockerman, yes. But this is just important for onboarding.

[KluthR]

22 hours ago, Proxykn said:

i get and error whit the backup of tdarr

The tdarr log changes. Some other container that might access it?

 

mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Mod time differs; mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Size differs

 

[Proxykn]

37 minutes ago, KluthR said:

The tdarr log changes. Some other container that might access it?

 

mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Mod time differs; mnt/user/appdata/tdarr/logs/Tdarr_Node_Log.txt: Size differs

 

 

I changed setting to stop all containers and then do backup and no errors now.

my guess is the tdarr node doing something with the main tdarr when closing and doing backup one by one.

all seems to be working fine now, thank you!

[KluthR]

Check if they share the same volume maybe. That would cause such errors. If yes, exclude it from the secondary container (which just read from it). I assume that is how they work, just a shot in the dark.

 

If your environment can go with stopping all containers first, then this is ok.

Edited June 17, 2023 by KluthR

[Proxykn]

Yes, they share a transcode temp volume on both containers.

The old backup app turned off all apps when doing backup and i don't mind it so i can keep doing it this way🙂

[Masterwishx]

8 hours ago, zoggy said:

the old ca backup app used to just inherit how the dockers were sorted in unraid docker gui.

it would show them in "unraid docker gui" reverse order, which is how it would stop them.

then when it starts back up it does the reverse of that (aka, the normal "unraid docker page gui" order)

 

 

So Why here not same like in old one ...

[Masterwishx]

1 hour ago, KluthR said:

„Its“ (you mean the plugin?) order is configureable

Sorry, english not my first lang, 

 

1 hour ago, KluthR said:

Default could be picked up by Dockerman, yes

How to do it ? i cant find setting for this...

[KluthR]

1 minute ago, Masterwishx said:

So Why here not same like in old one ...

Because one could set own order. But I agree: the default order should read from dockermans order. I will look into that after my vacation.

[KluthR]

6 minutes ago, Proxykn said:

Yes, they share a transcode temp volume on both containers.

The old backup app turned off all apps when doing backup and i don't mind it so i can keep doing it this way🙂

If you exclude the temp share from both, you should be able to do stop, backup, start method.

[Jaycedk]

Hi there.

First of thanks for making this plugin.

I keep getting backup failed, but can't seem to figure out why.

Can I get someone to look at this log ?

 

Thanks i advance

backup.debug.log