Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
Message added by KluthR,

[Plugin] Appdata.Backup

Featured Replies

1 hour ago, KluthR said:

No, not really at all. I see, that plex backup was running from 11:08:05 to 12:30:57. A "file changed as we read it" occured and the lsof (which file is open within given path) took from 12:30:57 to 12:43:55.

 

While writing I realize, that the cotainer listing (from dockerMan) took from 05:00:05 to 09:55:21. Thats nothing special and loaded during backup initialization. Seems like Unraid had an issue.

Yeah, the Plex issue was because I didn't realise the backup was still running and started the container before spotting that it was still backing up and shut it back down again, I was expecting that one. If it was just a one-off I wouldn't've thought much of it. But then this happened last week for about the same 5 hourish duration (the one I don't have a complete log of).

[20.01.2025 05:03:37][ℹ️][sonarrRET] Stopping sonarrRET... done! (took 13 seconds)
[20.01.2025 05:03:50][ℹ️][sonarrRET] Should NOT backup external volumes, sanitizing them...
[20.01.2025 05:03:50][ℹ️][sonarrRET] Calculated volumes to back up: /mnt/user/appdata/sonarrRET
[20.01.2025 05:03:50][ℹ️][sonarrRET] Backing up sonarrRET...
[20.01.2025 10:19:56][ℹ️][sonarrRET] Backup created without issues (took 05:16:06 (hours:mins:secs))
[20.01.2025 10:19:56][ℹ️][sonarrRET] Verifying backup...
[20.01.2025 10:20:44][ℹ️][sonarrRET] Verification ended without issues (took 00:00:48 (hours:mins:secs))
[20.01.2025 10:20:44][ℹ️][sonarrRET] Starting sonarrRET... (try #1) done!

That folder is about 1.3gb and never normally takes just a couple of minutes. I fully appreciate that there's only so much information that be gleaned from these logs but I still felt I should bring it up. :) I'll keep an eye out for any further issues in the meantime.

  • Replies 2.2k
  • Views 364.9k
  • Created
  • Last Reply

Top Posters In This Topic

Most Popular Posts

  • Feature freeze I have less and less time for a complete care of this plugin. You already noticed this with the fact, that announced features were not implemented yet. Another reason is, that I will mo

  • The new update is coming It been a while since the last stable update. There were some betas (never got feedback though) but I had other work to do the last weeks. I tested the major changes agai

  • 2023.08.28 should fix the docker auto update issue.

Posted Images

DOH!  I think I figured out what happened.  🫢

 

As I wrote previously, I have a number of maintenance tasks I run.  Several bash scripts and some plugins, all that send Telegram notifications when run.  All of the server tasks are spread out so not to overlap (and at times where they have low impact on media server users).

 

But I also have another task - a backup of my desktop workstation which runs daily.  Macrium Reflect, which runs local on that workstation.  It only sends notifications on failure.

 

Macrium does a full backup on the first Monday, weekly differential every Monday, and an incremental daily.  The daily incremental is usually very fast (10 minutes or less).  The day before Appdata Backup ran, I copied a large amount of data to the workstation (+1 TB).  I usually move this to a drive on the workstation that isn't backed up, but there wasn't enough free space.  So it ended up on a drive that is included in the backup.

 

When Appdata Backup ran, the server was busy with Macrium writing a ton of data directly to the parity protected array, while trying to verify the 16GB / 80000+ files of the Plex appdata backup file.  I'm pretty confident that the array drives were thrashing around madly for a few hours.  The Macrium backup completed without any error, so I did not receive any notification from it.

 

My apologies for sounding the alarm.  At least this is something you can add to your troubleshooting toolkit as a possibility if someone asks about something similar.  Thank you for your time and support of this very useful plugin.

On 12/10/2024 at 1:22 PM, Kilrah said:

That's the goal, for example that 2nd destination can be somewhere that's accessible if the server is down. 

What if I don’t want the backup flash drive to be duplicated? There should be an option to choose whether to do it or not, right?

If you want only one copy it'll have to be in the appdata backup's folder.

There is no option to have it only in another folder. An extra backup copy never hurts and neither do a few hundred megs, so why bother. 

3 minutes ago, Kilrah said:

If you want only one copy it'll have to be in the appdata backup's folder.

I've already tried that, but the flash drive backup still gets duplicated. One flash drive backup is in /mnt/user/backup/appdata/, and another flash drive backup is in /mnt/user/backup/appdata/ab_20250128_170634/

12 minutes ago, Kilrah said:

If you want only one copy it'll have to be in the appdata backup's folder.

 

i've just tried not to write destination folder in the "Copy the flash backup to a custom destination" field and the flash drive backup doesn't get duplicated. 
thx

Is there an option to 'Skip All Container Backups' instead of setting each container to Yes to skip the container?

 

Or

 

Is there an option to just back up the Flash Drive without backing up the containers?

 

Having either options would be great if I just want to backup the flashdrive

24 minutes ago, Kloudz said:

Is there an option to just back up the Flash Drive without backing up the containers?

 

Having either options would be great if I just want to backup the flashdrive

You do not even need the plugin for that. There is a script that comes with Unraid (that the app actually uses) that you can just run yourself. 

/usr/local/emhttp/webGui/scripts/flash_backup

It should generate the backup zip file in Unraid's root directory if there is enough space.

58 minutes ago, Blasman said:

You do not even need the plugin for that. There is a script that comes with Unraid (that the app actually uses) that you can just run yourself. 

/usr/local/emhttp/webGui/scripts/flash_backup

It should generate the backup zip file in Unraid's root directory if there is enough space.

I was trying to avoid creating a script of my own. I figured someone out there created a plugin by now where you can just do everything in the GUI and this plugin is very close.

 

But thank you for the suggestion, I guess scripting is the way to go

On 1/26/2025 at 3:56 AM, KluthR said:

I believe that all of your errors come from empty folders at their original locations. Is the error always the same?

 

I cannot see why photoprism is that big. The log states it only backups the path "/mnt/user/appdata/photoprism" which seems legit. How big is it?

Hi, I... Do you know why my backup is not working? If so, can you tell me how to do it? Please help.

2702fd6f-be8a-4a64-a049-f97266f6fcd4

keep geting traily "/" again..

 

[29.01.2025 14:51:55][ℹ️][Main] 👋 WELCOME TO APPDATA.BACKUP!! :D
[29.01.2025 14:51:55][ℹ️][Main] Backing up from: /mnt/cache, /mnt/cache/appdata, /mnt/user, /mnt/user/appdata
[29.01.2025 14:51:55][ℹ️][Main] Backing up to: /mnt/vm-zfs/Backups/AppdataBackupPlugin/ab_20250129_145155
[29.01.2025 14:51:55][ℹ️][Main] Selected containers: ClamAV-clamdscan, MakeMKV-Ripper, PostgreSQL_Immich, Redis_Immich, Twingate-Connector, immich, plex, sftp
[29.01.2025 14:51:55][ℹ️][Main] Saving container XML files...
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'ClamAV-clamdscan' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'immich' is enabled and update is available! Schedule update after backup...
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'MakeMKV-Ripper' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'plex' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'PostgreSQL_Immich' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'Redis_Immich' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'sftp' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Auto-Update for 'Twingate-Connector' is enabled but no update is available.
[29.01.2025 14:51:57][ℹ️][Main] Method: Stop/Backup/Start
[29.01.2025 14:51:57][ℹ️][Twingate-Connector] No stopping needed for Twingate-Connector: Not started!
[29.01.2025 14:51:57][ℹ️][Twingate-Connector] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:51:57][⚠️][Twingate-Connector] Twingate-Connector does not have any volume to back up! Skipping. Please consider ignoring this container.
[29.01.2025 14:51:57][ℹ️][Twingate-Connector] Starting Twingate-Connector is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:51:57][ℹ️][ClamAV-clamdscan] No stopping needed for ClamAV-clamdscan: Not started!
[29.01.2025 14:51:57][ℹ️][ClamAV-clamdscan] Removing container mapping "/mnt/user" because it is a source path (exact match)!
[29.01.2025 14:51:57][ℹ️][ClamAV-clamdscan] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:51:57][ℹ️][ClamAV-clamdscan] Calculated volumes to back up: /mnt/user/appdata/CAVclamdscan/db, /mnt/user/appdata/CAVclamdscan/log, /mnt/user/appdata/CAVclamdscan/etc
[29.01.2025 14:51:57][ℹ️][ClamAV-clamdscan] Backing up ClamAV-clamdscan...
[29.01.2025 14:52:04][ℹ️][ClamAV-clamdscan] Backup created without issues (took 00:00:07 (hours:mins:secs))
[29.01.2025 14:52:04][ℹ️][ClamAV-clamdscan] Verifying backup...
[29.01.2025 14:52:05][ℹ️][ClamAV-clamdscan] Verification ended without issues (took 00:00:01 (hours:mins:secs))
[29.01.2025 14:52:05][ℹ️][ClamAV-clamdscan] Starting ClamAV-clamdscan is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:52:05][ℹ️][sftp] No stopping needed for sftp: Not started!
[29.01.2025 14:52:05][ℹ️][sftp] Removing container mapping "/mnt/user" because it is a source path (exact match)!
[29.01.2025 14:52:05][ℹ️][sftp] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:52:05][ℹ️][sftp] Calculated volumes to back up: /mnt/user/appdata/sftp
[29.01.2025 14:52:05][ℹ️][sftp] Backing up sftp...
[29.01.2025 14:52:05][ℹ️][sftp] Backup created without issues (took 00:00:00 (hours:mins:secs))
[29.01.2025 14:52:05][ℹ️][sftp] Verifying backup...
[29.01.2025 14:52:05][ℹ️][sftp] Verification ended without issues (took 00:00:00 (hours:mins:secs))
[29.01.2025 14:52:05][ℹ️][sftp] Starting sftp is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:52:05][ℹ️][MakeMKV-Ripper] No stopping needed for MakeMKV-Ripper: Not started!
[29.01.2025 14:52:05][ℹ️][MakeMKV-Ripper] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:52:05][ℹ️][MakeMKV-Ripper] Calculated volumes to back up: /mnt/user/appdata/MakeMKV/config, /mnt/user/appdata/MakeMKV/storage, /mnt/user/Dockers/MakeMKV/outputdata
[29.01.2025 14:52:05][ℹ️][MakeMKV-Ripper] Backing up MakeMKV-Ripper...
[29.01.2025 14:52:06][ℹ️][MakeMKV-Ripper] Backup created without issues (took 00:00:01 (hours:mins:secs))
[29.01.2025 14:52:06][ℹ️][MakeMKV-Ripper] Verifying backup...
[29.01.2025 14:52:06][][MakeMKV-Ripper] tar verification failed! Tar said: tar: Removing leading `/' from member names; tar: /mnt/user/Dockers/MakeMKV/outputdata: Not found in archive; tar: Exiting with failure status due to previous errors
[29.01.2025 14:52:06][ℹ️][MakeMKV-Ripper] Starting MakeMKV-Ripper is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:52:06][ℹ️][plex] No stopping needed for plex: Not started!
[29.01.2025 14:52:06][ℹ️][plex] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:52:06][ℹ️][plex] Calculated volumes to back up: /mnt/user/Dockers/Plex/Data, /mnt/cache/appdata/Plex/config, /mnt/cache/appdata/Plex/temptranscode
[29.01.2025 14:52:06][ℹ️][plex] Backing up plex...
[29.01.2025 14:55:10][ℹ️][plex] Backup created without issues (took 00:03:04 (hours:mins:secs))
[29.01.2025 14:55:10][ℹ️][plex] Verifying backup...
[29.01.2025 14:55:46][][plex] tar verification failed! Tar said: tar: Removing leading `/' from member names; tar: /mnt/user/Dockers/Plex/Data: Not found in archive; tar: Exiting with failure status due to previous errors
[29.01.2025 14:56:41][ℹ️][plex] Starting plex is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:56:41][ℹ️][immich] No stopping needed for immich: Not started!
[29.01.2025 14:56:41][ℹ️][immich] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:56:41][ℹ️][immich] Calculated volumes to back up: /mnt/user/Dockers/immich/photos, /mnt/user/Dockers/immich/config, /mnt/user/Dockers/immich/libaries
[29.01.2025 14:56:41][ℹ️][immich] Backing up immich...
[29.01.2025 14:56:41][ℹ️][immich] Backup created without issues (took 00:00:00 (hours:mins:secs))
[29.01.2025 14:56:41][ℹ️][immich] Verifying backup...
[29.01.2025 14:56:41][][immich] tar verification failed! Tar said: tar: /mnt/user/Dockers/immich/photos: Not found in archive; tar: /mnt/user/Dockers/immich/config: Not found in archive; tar: /mnt/user/Dockers/immich/libaries: Not found in archive; tar: Exiting with failure status due to previous errors
[29.01.2025 14:57:44][ℹ️][immich] Installing planned update for immich...
[29.01.2025 14:58:33][ℹ️][immich] Starting immich is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:58:33][ℹ️][PostgreSQL_Immich] No stopping needed for PostgreSQL_Immich: Not started!
[29.01.2025 14:58:33][ℹ️][PostgreSQL_Immich] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:58:33][ℹ️][PostgreSQL_Immich] Calculated volumes to back up: /mnt/user/appdata/PostgreSQL_Immich
[29.01.2025 14:58:33][ℹ️][PostgreSQL_Immich] Backing up PostgreSQL_Immich...
[29.01.2025 14:58:41][ℹ️][PostgreSQL_Immich] Backup created without issues (took 00:00:08 (hours:mins:secs))
[29.01.2025 14:58:41][ℹ️][PostgreSQL_Immich] Verifying backup...
[29.01.2025 14:58:44][ℹ️][PostgreSQL_Immich] Verification ended without issues (took 00:00:03 (hours:mins:secs))
[29.01.2025 14:58:44][ℹ️][PostgreSQL_Immich] Starting PostgreSQL_Immich is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:58:44][ℹ️][Redis_Immich] No stopping needed for Redis_Immich: Not started!
[29.01.2025 14:58:44][ℹ️][Redis_Immich] Should NOT backup external volumes, sanitizing them...
[29.01.2025 14:58:44][⚠️][Redis_Immich] Redis_Immich does not have any volume to back up! Skipping. Please consider ignoring this container.
[29.01.2025 14:58:44][ℹ️][Redis_Immich] Starting Redis_Immich is being ignored, because it was not started before (or should not be started).
[29.01.2025 14:58:44][ℹ️][Main] Backing up the flash drive.
[29.01.2025 14:59:33][ℹ️][Main] Flash backup created!
[29.01.2025 14:59:33][ℹ️][Main] Copying the flash backup to '/mnt/vm-zfs/Backups/AppdataBackupPlugin/UnraidFlashDriveBackup/' as well...
[29.01.2025 14:59:36][ℹ️][Main] Backing up extra files...
[29.01.2025 15:14:06][][Main] tar creation failed! Tar said: tar: /mnt/cache/system/docker-xfs.img: file changed as we read it
[29.01.2025 15:14:06][⚠️][Main] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum.
[29.01.2025 15:14:06][ℹ️][Main] DONE! Thanks for using this plugin and have a safe day ;)
[29.01.2025 15:14:06][ℹ️][Main] ❤️


something int he tar option for the plugin...

all dockers are off when i start a manlu backup..

appdata sources:
 

/mnt/cache
/mnt/cache/appdata
/mnt/user
/mnt/user/appdata

image.png.0c5e1ed89ebe5faf1e1aea964e1b89a8.png

 

image.thumb.png.1c1164187bd2e7adaf499824b1f96155.png

I've remode the unraid instance...

include extras:

/mnt/cache
/mnt/cache/appdata
/mnt/cache/appdata/CAVclamdscan
/mnt/cache/appdata/MakeMKV
/mnt/cache/appdata/NetProbe
/mnt/cache/appdata/Plex
/mnt/cache/appdata/PostgreSQL_Immich
/mnt/cache/appdata/sftp

 

global exclusiong:
 

/mnt/vm-zfs
/mnt/vm-zfs/Dockers
/mnt/user/Dockers
/mnt/user/Dockers/immich
/mnt/user/Dockers/MakeMKV
/mnt/user/Dockers/Plex
/mnt/vm-zfs/Users
/mnt/user/Users
/mnt/user/Users/Will
/mnt/user/Users/Logan
/mnt/user/Users/Chris
/mnt/user/Users/Brandon
/mnt/user/Users/Elliott


as I only what the data in /mnt/user/appdata "/mnt/user/cache" backed up...

not sure why tar is failing with the error pathing trailing "/"

1 hour ago, bmartino1 said:

not sure why tar is failing with the error pathing trailing "/"

That's not the error, what follows is.

 

Quote

tar: /mnt/user/Dockers/Plex/Data: Not found in archive

typically means the folder was empty

 

1 hour ago, bmartino1 said:
tar creation failed! Tar said: tar: /mnt/cache/system/docker-xfs.img: file changed as we read it

The Docker image shouldn't be backed up. It being included from the system share doesn't match with your lists.

 

 

29 minutes ago, Kilrah said:

That's not the error, what follows is.

 

typically means the folder was empty

 

The Docker image shouldn't be backed up. It being included from the system share doesn't match with your lists.

 

 

its not empty...

 

Quote

root@Docker:/mnt/user/Dockers# ls
MakeMKV/  Plex/  containers@  immich/
root@Docker:/mnt/user/Dockers# 

root@Docker:/mnt/user/Dockers# cd Plex/
root@Docker:/mnt/user/Dockers/Plex# ls
1for1/  Data/
root@Docker:/mnt/user/Dockers/Plex# 

root@Docker:/mnt/user/Dockers/Plex/Data# ls
Movies/  Personal\ Family\ Home\ Videos/


However, I don't want the plugin backing up that folder as it contine all the movie files... I only want the appdata config files for pelx...

regardless this was working before, now it not..

Edited by bmartino1

Hey @KluthR, what do you think about a feature to sync Groups with groups from the FolderView plugin? :)

  • Author

Dont know what FolderView plugin is. Could you explain what it does?

Question: I tried to do a manual backup today and after it took too long I aborted it. According to the log the backup is still running however there has not been anything written to the backup directory. 

 

Is the a process I need to kill a process so it won't be listed as running?

 

 

Thanks

 

Edited by mwoods98

How does restore work? I've clicked through and done the steps but where / how to access the restored container etc.

 

I've just had a bad update of zigbee2mqqt & want to 'roll back'

 

 

Screenshot 2025-02-02 at 09.32.01.png

Edited by dopeytree

  • Author

Looking good. Since the restore only does data part, I would recommend:

 

* delete container

* delete the container appdata

* restore (like you shown)

* create container from template

 

should fire up the backed up version.

Still downloads the latest version when you add the container..

 

Screenshot 2025-02-02 at 10.11.00.png

Edited by dopeytree

  • Author

The image? Yes, of course. The image has nothing to do with the appdata. If you want to pin a specific image version from that tool, you have to work with image tags. Like imagegenius/zigbee2mqtt:1.2.3

 

I ran into a problem last night when the Docker Engine did not start after Appdata Backup. Posted this into the Docker Engine subforum and now here as it was suggested to post here, too.

Quote

 

Hello all,

 

I am using the Appdata Backup plugin for quite a while now and never had problems. That changed last night.

 

In the morning I noticed that all containers were not running. I checked the GUI and did not see any Docker containers at all. But Docker was running. I stopped it through settings ("Enable Docker" to No, then Apply) and then enabled it again. On the Docker page I saw "Status: Running" but still no containers. Retried, no change. In syslog I saw something that no memory is left so I stopped two VMs and then also with 20% memory usage I still got the same error. So I rebooted the Unraid machine and everything came back online. The problem is that I also had to bring down the VMs (one of them is OPNsense and another Pi-hole, so no Internet...) and also rebooting and having everything back running takes some time. So that's quite a deal.

 

As everything went offline at 3pm it looked to me already from the beginning of my investigation that it must be the weekly Appdata Backup which starts at 3am on Monday mornings. And yes, the log tells me that as well. The backup itself was okay, but then not all containers were started and it took a long time (one time over one hour) from one container start to the next which failed. I attached the log here. I also got a mail from Appdata Backup that something failed and that I need to check debug log. As I already rebooted (had to get out of the house to work and the services back online...) I do not see any logs anymore and I am also not sure if debug logging is enabled for Appdata Backup, I guess, it's not. I did not find debug in the plugin settings.

 

Anyway, I installed some more Docker containers last week, so something changed here. And I also have lots of data inside these containers for backup, e.g. 5GB+ of zstd compressed MariaDB data.

 

Do I need to adjust some settings? I never did for the Docker Engine. But is that really Docker as all containers get stopped before backup (I set it that way) and started again after backup?

 

Any ideas?

 

I'm glad to bring more data, I would also run a backup again and check debug first before rebooting. At the moment I hope for something simple, maybe a user problem with some setting I should have changed because of more containers, load or whatever.

 

Thanks.

 

 

backup.log

  • Author

No idea. I would restart the whole system and see if the issue still persists. For me it is no issue with Appdata backup.

4 hours ago, KluthR said:

No idea. I would restart the whole system and see if the issue still persists. For me it is no issue with Appdata backup.

 

I recreated the Docker image file today. When I had the problem, I restarted the machine because the Docker system did not work anymore. After restarting the machine everything was fine again.

  

So I still have the issue with the container group order not getting saved, however this time I have some debug info, @KluthR. 😀

I used the new "?debugForm" feature to obtain telemetry, here's what saving the page returned for my MariaDB group:

containerGroupOrder%5BMariaDB%5D=containerGroupOrder%5BMariaDB%5D%5B%5D%3Dmariadb%26containerGroupOrder%5BMariaDB%5D%5B%5D%3DDirectus%26containerGroupOrder%5BMariaDB%5D%5B%5D%3Dlychee

Sadly the order stays like this: Directus, lychee, mariadb, so precisely the opposite of what's needed.

Edited by Glassed Silver

  • Author

The order string seems wrong, as it always talks about mariadb.

 

Do you have 6.12.15 ot 7.0.0? If yes, those versions fixed the javascript debug console logging. Please hit F12 in Chrome or Firefox and open the tab "Console" and reload the page and try some ordering, maybe something is displayed there.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.