[Plugin] Appdata.Backup

[cralich]

49 minutes ago, KluthR said:

This is known and already fixed in future versions. Containers get their update eight after backup before starting again.

 

Thank you for the work you have done and are doing. This plugin is great!

[KluthR]

Thanks! The next update is almost ready and more to come after that. Maybe incremental backups would be nice too? 😉

[M8trix]

App data backup keeps failing nightly.  There is an error in the log from last night about Tdarr that is below.  Log is attached.

 

Version: 2023.10.16 

Your debug log ID:  095b94b8-e954-4034-a431-cbd156bd7d78

 

Quote

[13.12.2023 00:00:03][ℹ️][Main] 👋 WELCOME TO APPDATA.BACKUP!!
[13.12.2023 00:00:03][ℹ️][Main] Backing up from: /mnt/cache_nvme/appdata, /mnt/cache_nvme/appdata/ApacheGuacamole, /mnt/cache_nvme/appdata/Backblaze_Personal_Backup, /mnt/cache_nvme/appdata/binhex-delugevpn, /mnt/cache_nvme/appdata/binhex-krusader, /mnt/cache_nvme/appdata/binhex-minecraftbedrockserver, /mnt/cache_nvme/appdata/binhex-privoxyvpn, /mnt/cache_nvme/appdata/binhex-prowlarr, /mnt/cache_nvme/appdata/binhex-radarr, /mnt/cache_nvme/appdata/binhex-sabnzbdvpn, /mnt/cache_nvme/appdata/binhex-sonarr, /mnt/cache_nvme/appdata/clamav, /mnt/cache_nvme/appdata/cloudflared, /mnt/cache_nvme/appdata/code-server, /mnt/cache_nvme/appdata/crafty-4, /mnt/cache_nvme/appdata/dashy, /mnt/cache_nvme/appdata/diskspeed, /mnt/cache_nvme/appdata/duplicati, /mnt/cache_nvme/appdata/mariadb, /mnt/cache_nvme/appdata/minecraftbedrockedition, /mnt/cache_nvme/appdata/nextcloud, /mnt/cache_nvme/appdata/Nginx-Proxy-Manager-Official, /mnt/cache_nvme/appdata/onlyoffice, /mnt/cache_nvme/appdata/onlyofficeds, /mnt/cache_nvme/appdata/overseerr, /mnt/cache_nvme/appdata/pialert, /mnt/cache_nvme/appdata/pihole, /mnt/cache_nvme/appdata/plex, /mnt/cache_nvme/appdata/Plex-Meta-Manager, /mnt/cache_nvme/appdata/postgresql14, /mnt/cache_nvme/appdata/qbittorrent, /mnt/cache_nvme/appdata/redis, /mnt/cache_nvme/appdata/syncthing, /mnt/cache_nvme/appdata/tailscale, /mnt/cache_nvme/appdata/tautulli, /mnt/cache_nvme/appdata/tdarr
[13.12.2023 00:00:03][ℹ️][Main] Backing up to: /mnt/user0/backups/ca/ab_20231213_000003
[13.12.2023 00:00:03][ℹ️][Main] Selected containers: ApacheGuacamole, Backblaze_Personal_Backup, ClamAV, DiskSpeed, Nginx-Proxy-Manager-Official, OnlyOfficeDocumentServer, Plex-Meta-Manager, PuTTY, Tailscale, adminer, binhex-krusader, binhex-prowlarr, binhex-qbittorrentvpn, binhex-radarr, binhex-sabnzbdvpn, binhex-sonarr, cloudflared, code-server, dashy, dockersocket, duplicati, flaresolverr, homarr, maintainerr, mariadb, netdata, nextcloud, office-document-server-ee, overseerr, pihole, plex, postgres, redis, scrutiny, syncthing, tautulli, tdarr, tdarr_node
[13.12.2023 00:00:03][ℹ️][Main] Saving container XML files...
[13.12.2023 00:00:04][ℹ️][Main] Method: Stop/Backup/Start

...

[13.12.2023 00:02:15][ℹ️][tdarr] Stopping tdarr... done! (took 5 seconds)
[13.12.2023 00:02:20][ℹ️][tdarr] Should NOT backup external volumes, sanitizing them...
[13.12.2023 00:02:20][ℹ️][tdarr] Calculated volumes to back up: /mnt/cache_nvme/appdata/tdarr/logs, /mnt/cache_nvme/appdata/tdarr/server, /mnt/cache_nvme/appdata/tdarr/configs
[13.12.2023 00:02:20][ℹ️][tdarr] Backing up tdarr...
[13.12.2023 00:03:52][ℹ️][tdarr] Backup created without issues
[13.12.2023 00:03:52][ℹ️][tdarr] Verifying backup...
[13.12.2023 00:04:39][❌][tdarr] tar verification failed! Tar said: tar: Removing leading `/' from member names; mnt/cache_nvme/appdata/tdarr/logs/Tdarr_Node_Log.txt.1: Mod time differs; mnt/cache_nvme/appdata/tdarr/logs/Tdarr_Node_Log.txt.1: Size differs
[13.12.2023 00:04:45][ℹ️][tdarr] Starting tdarr... (try #1) done!

...

[13.12.2023 00:07:43][ℹ️][Main] Docker update check finished!
[13.12.2023 00:07:43][⚠️][Main] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum.
[13.12.2023 00:07:43][ℹ️][Main] DONE! Thanks for using this plugin and have a safe day
[13.12.2023 00:07:43][ℹ️][Main] ❤️

 

Log.txt

[KluthR]

Classic. Tdarr uses volume by plex or vice versa but one of it writes to it.

 

I really need to add such detection….

[KluthR]

Oh, sorry: To resolve it for you, please exclude the volume in tdarr which is commonly used with the other container.

[KluthR]

@mdoherty66 Looks odd. Whats the command?

Edited December 15, 2023 by KluthR

[DennisT]

I have just setup an Unraid server and am using this plugin to backup the Appdata and Docker configs. I use the 'Stop all containers, backup, start all' method for the backup and have MariaDB which is used by several apps, so i want that container to stop after all others. The order shown in the app starts with MariaDB and then the others in no particular order
(see attached image), but when the backup starts the sorted order seems to be incorrect as the MariaDB container is placed somewhere in the middle and not in the order as shown in the app. As i understand this order is used in both backup modes. Or am i wrong? I want to stop everything and backup to make sure all configs and data in MariaDB are in sync, so backing up each container individually is not an option for me. Apps will likely also break as they are running and loose connection to the database.

 

My debug log is: e2b38684-2098-46e9-b4fe-f2efde1ab44c

 

The particular lines that i am confused about (and unable to determine cause by looking through the source code) are:

 

[15.12.2023 10:19:41][debug][Main] Sorted Stop : Dropbox, phpMyAdmin, NginxProxyManager, MariaDB, Jackett, Radarr, SpeedtestTracker, Transmission
[15.12.2023 10:19:41][debug][Main] Sorted Start: MariaDB, NginxProxyManager, phpMyAdmin, Dropbox, Jackett, Radarr, SpeedtestTracker, Transmission

 

Edited December 15, 2023 by DennisT
Removed duplicate log lines

[MaximusFSU]

Hey there,

 

I recently upgraded to unraid 6.12.4 and ever since I upgraded to app backup from the older deprecated version, it crashes itself and the plex docker when it tries to back up.  All other dockers seem unaffected, since it completes fine when I skip plex.   Ran a test this morning to grab fresh logs.  Cancelled after it had hung for a couple hours.

 

debug log: 288826d3-020b-4e8c-9806-d0a448f14be1

 

 

[15.12.2023 09:57:28][ℹ️][Plex-Media-Server] Stopping Plex-Media-Server... Error while stopping container! Code: Server error - trying 'docker stop' method
[15.12.2023 09:58:41][❌][Plex-Media-Server] docker stop variant was unsuccessful as well! Docker said: 
[15.12.2023 09:58:41][ℹ️][Plex-Media-Server] Should NOT backup external volumes, sanitizing them...
[15.12.2023 09:58:41][ℹ️][Plex-Media-Server] Calculated volumes to back up: /mnt/user/appdata/Plex-Media-Server
[15.12.2023 09:58:41][ℹ️][Plex-Media-Server] Backing up Plex-Media-Server...
[15.12.2023 11:46:27][❌][Plex-Media-Server] tar creation failed! Tar said: 
[15.12.2023 11:46:27][⚠️][Main] Backup cancelled! Executing final things. You will be left behind with the current state!
[15.12.2023 11:46:27][ℹ️][Main] DONE! Thanks for using this plugin and have a safe day
 

[KluthR]

@DennisT
 

The order displayed is the Start oder, not the stop order. Stop is the reverse of Start. And As I see the orders seems right.

 

the reason for the rest being always on the end: they have no order because they were created after saving the plugin settings last time. In that case the unknown containers are sorted at the end of the start and stop list. I really dont know if thats a bug or a feature because I would adjust the order if needed. And I know my containers.

 

So: All is working as intended so far. Hit Save one time and the Transmission would stop first.

 

_Maybe_ it would be more clear to order for stop and reverse Start instead vice versa like now. 

Edited December 15, 2023 by KluthR

[KluthR]

3 minutes ago, MaximusFSU said:

recently upgraded to unraid 6.12.4 and ever since I upgraded to app backup from the older deprecated version, it crashes itself and the plex docker when it tries to back up.

It dont crash. And neither Plex does. The plugin is just unable to srop PMS. There are few known issues from users and in all cases the mentioned „docker stop“ method is working. But not in your case.

 

are you able to stop PMS yourself?

[DennisT]

@KluthRThanks for your answer! Hitting Save did indeed fix the issue. I understand that the order shown is for starting and stopping is reverse, so that works properly now. Now i only need to remember to save each time i add a container. 

[KluthR]

28 minutes ago, DennisT said:

Now i only need to remember to save each time i add a container. 

Thats maybe improvable because thats not what I want users to do. It should not be always necessary, only if some important nee containers were added (which should stop in order).

[MaximusFSU]

1 hour ago, KluthR said:

It dont crash. And neither Plex does. The plugin is just unable to srop PMS. There are few known issues from users and in all cases the mentioned „docker stop“ method is working. But not in your case.

 

are you able to stop PMS yourself?

 

Yep.  Works perfectly when stopping the docker from the dashboard.

[KluthR]

Hmm. The thing is: the plugin does it the exact same way. like: Unraid and the plugin telling docker „Hey, please stop PMS!“ and docker returns to the plugin then „Server error“ (most likely PMS did not stopped in time then).

 

is PMS logging something for this time the backup ran?

[vurt]

Hi there, recently updated to the latest Appdata.Backup so I migrated from the previous one.

 

I'm getting this new error:

tar creation failed! Tar said: tar: /mnt/user/appdata/downloads/incomplete: file changed as we read it

 

This is the debug log ID that was sent: 030ae8ae-4efa-40f0-957a-cc7ab4d3e48c hope that works, never sent a debug log like this before!

[vurt]

Bump.

[KluthR]

1 minute ago, vurt said:

Bump.

There will be an answer when there is free time for it. Sorry. No need to „bump“.

[KluthR]

On 12/17/2023 at 11:25 PM, vurt said:

Hi there, recently updated to the latest Appdata.Backup so I migrated from the previous one.

 

I'm getting this new error:

Classic. a volume ist used by more than one container. During backup the data inside that folder got changed, thus the error.

 

Solution for now: Exclude this folder for any container except that one(s) who is really writing to it. The next bigger update brings grouping option for containers, maybe you put all those in one group then.

 

I also plan to implement a detection for those cases (volume x in use by more than one container) and display a small warning.

[vurt]

5 hours ago, KluthR said:

Classic. a volume ist used by more than one container. During backup the data inside that folder got changed, thus the error.

 

Solution for now: Exclude this folder for any container except that one(s) who is really writing to it. The next bigger update brings grouping option for containers, maybe you put all those in one group then.

 

I also plan to implement a detection for those cases (volume x in use by more than one container) and display a small warning.

 

Thank you, that makes sense but wasn't an error previously. The appdata/downloads folder is used by Sabnzbd, Radarr, Sonarr, and Hydra.

[KluthR]

1 hour ago, vurt said:

Thank you, that makes sense but wasn't an error previously.

Nah, thats not entirely correct. The previous version aka 2.0 (not 2.5) has a broken error detection. Guess how many people thought the plugin update to my version caused backup issues like that 😅

 

https://github.com/Squidly271/ca.backup2/issues/5

Edited December 19, 2023 by KluthR

[mdoherty66]

On 12/14/2023 at 10:25 PM, KluthR said:

@mdoherty66 Looks odd. Whats the command?

Figured it out, for anyone else reading this that might have a similar issue, the fix was saving the script properly as a Unix Script in Notepad++. It was saving in Windows format which put some weird characters at the beginning, causing the script to fail.

[KluthR]

Oh, yes. Had this issue several times as well.

[vurt]

On 12/19/2023 at 2:23 PM, KluthR said:

Nah, thats not entirely correct. The previous version aka 2.0 (not 2.5) has a broken error detection. Guess how many people thought the plugin update to my version caused backup issues like that 😅

 

https://github.com/Squidly271/ca.backup2/issues/5

 

Oh does this mean I probably always had this error but the previous version wasn't reporting it?

 

Can I click on /mnt/user/appdata/downloads and it'll be excluded from backup? I think this is the folder that's causing the error. It doesn't require backing up anyway.

[KluthR]

12 hours ago, vurt said:

Oh does this mean I probably always had this error but the previous version wasn't reporting it?

Exactly

12 hours ago, vurt said:

Can I click on /mnt/user/appdata/downloads and it'll be excluded from backup?

Not yet. You have to add that complete path to the per-container settings exclusion list.

[scissabob]

Hi,

I'm Marco and I'm new here
Maybe somebody can help me.

I created a Stack with 4 Containers... everything is running fine. 

But when I try to backup everything, I got this message:

[21.12.2023 10:32:42][❌][teslalogger-webserver-1] tar verification failed! Tar said: tar: Removing leading `/' from member names; mnt/user/appdata/TeslaLogger/TeslaLogger/bin/nohup.out: Mod time differs; mnt/user/appdata/TeslaLogger/TeslaLogger/bin/nohup.out: Size differs

Does anybody know, what I can do to fix this?

thx in advance