[Plugin] CA Appdata Backup / Restore v2.5


KluthR

Recommended Posts

The socket error could be fixed by putting it to the exclusion list. The main issue is the file ./radarr/MediaCover/129/fanart-360.jpg: which seems to be changed during backup. I dont know why, because all dockers seem to be stopped. Yet another user with this issue. I dont get whats causing it.

 

Could you do me a favor? Please open /usr/local/emhttp/plugins/ca.backup2/scripts/backup.php, go to line 317. 317 should be:

backupLog("tar verify failed!");

 

DIRECTLY AFTER that line, paste:

 

            $debug = "/tmp/ca.backup.debug.".uniqid();
            file_put_contents($debug, json_encode($dockerClient->getDockerContainers(), JSON_PRETTY_PRINT));
            $output = $outval = null;
            exec("docker ps -a", $output, $outval);
            file_put_contents($debug, json_encode($output, JSON_PRETTY_PRINT), FILE_APPEND);
            backupLog("DEBUG: dockerInfo written to /tmp/");

 

Save and let the baclup run again (or do it manually). The log should post the DEBUB: line. If so, please attach (or send via PM) all /tmp/ca.backup-debug* files. Maybe I see, if some docker things are going on.

Link to comment

Okay, not sure if I'm missing something but here's what I've seen after the backups ran last night:

  1. The error logs from the backups are much longer now, way more issues than before. Lots of size/mod time/content differs messages per container
  2. I've navigated to /tmp/ but there's no ca.backup-debug files, despite me adding the code you specified to the backup.php script
  3. When I went to double-check the backup.php script I no longer see ca.backup2 in the path /usr/local/emhttp/plugins/
Link to comment

Idk what happened at your end tonight, but this debug thing is working on my side:

 

grafik.png.1717437d4e713d4e2fb545192bf6e743.png

 

I dont event know what caused a removal of that folder, sorry.

 

47 minutes ago, IT_Feldman said:

way more issues than before. Lots of size/mod time/content differs messages per container

This is being logged everytime a error is detected + overhead of this debug adaption.

 

Link to comment
On 1/11/2023 at 5:43 AM, KluthR said:

I'll add an option to let the user completely ignore any errors for now - later this day. In the upcoming options, users have more options for that. Backups while containers are running are no valid backups for me.

It would be possible (next versions) to only include a pattern to the backup. Use case would be: If a container creates backups itself, AppData Backup could only pick those up while not stopping this container. Backup and verification would be successful because those files are static at that moment.

 

We will see.

 

@All Plex users: Please share your Plex exclusion list to create a list of useful exclusions for the next big upfates :)

My backup, including Plex, is 102GB. I don't exclude any files/folders. 

I have over 1700 movies in Plex. There's no way I would want to go through an update every poster or some other metadata just to save GB's of space on my server. 

The whole point of a backup is to backup data, so that you can restore easily and quickly. 

 

Link to comment

First, thank you KluthR for taking over this plugin.

 

I would love for you to change this setting: "Delete backups if they are this many days old:".

For me, this setting is the reverse of what I would like to have. I would prefer to "Keep this many backups:". 

 

The reason for this is in this forum post

I had several disk failures and it took me over a month to fix everything, but by that time, my appdata backups had been deleted because I was only keeping 15 days of backups.  

 

Even if you just added "Keep this many backups" without removing the current setting, that would be awesome!

  • Like 1
  • Upvote 1
Link to comment
15 hours ago, jeffreywhunter said:

I'm looking for wisdom on using appdata backup/restore v3 to backup Plex.  My appdata directory is 61gb with 412gb free.  I've read that some do not backup plex but use other methods.  What's the best practice?

 

Thanks!

I use this plugin to backup my Linuxserver Plex Docker to a separate backups share I have on the same unRAID server. I've spent too much time on my Plex metadata to not backup. 

 

I also take screenshots of the Plex docker container for reference. This helps if I have to re-install the docker container since some of the settings won't re-populate correctly when re-installing the container from the Previous Apps tab in CA. 

Link to comment
On 2/15/2023 at 3:42 AM, FQs19 said:

My backup, including Plex, is 102GB. I don't exclude any files/folders. 

I have over 1700 movies in Plex. There's no way I would want to go through an update every poster or some other metadata just to save GB's of space on my server. 

The whole point of a backup is to backup data, so that you can restore easily and quickly. 

 

My Plex folder alone, not including other AppData is approximately 300GB - this is because of the *.bif files used for video preview thumbnails. I don't see the need to back them up, because they will be regenerated as a scheduled task if needed, and not including them in backups would save me about 270GB of disk space, and 40mins of time on each backup. There is literally no need to back them up - they aren't user generated content and my system can replace them in about three days if I ever had to restore.

 

@KluthR - The ability to exclude certain filetypes would be an *extremely* useful addition. Even moreso if you can exclude certain filetypes on a per-folder or per-tree basis!

 

Edit: Spelling.

Edited by Dalarielus
  • Like 2
Link to comment
9 minutes ago, Dalarielus said:

My Plex folder alone, not including other AppData is approximately 300GB - this is because of the *.bif files used for video preview thumbnails. I don't see the need to back them up, because they will be regenerated as a scheduled task if needed, and not including them in backups would save me about 270GB of disk space, and 40mins of time on each backup. There is literally no need to back them up - they aren't user generated content and my system can replace them in about three days if I ever had to restore.

 

@KluthR - The ability to exclude certain filetypes would be an *extremely* useful addition. Even moreso if you can exclude certain filetypes on a per-folder or per-tree basis!

 

Edit: Spelling.

 

Since my backup wasn't that large, I've never dug into what files aren't necessary. 

 

Of course, being able to exclude folders from backing up would be a great addition. 

Link to comment
6 minutes ago, FQs19 said:

Of course, being able to exclude folders from backing up would be a great addition.

Thats already possible?

 

17 minutes ago, Dalarielus said:

The ability to exclude certain filetypes would be an *extremely* useful addition

I would bet, if you declare something like /mnt/cache/appdata/myfolder/*.ext that would ignore all .ext files.

We will see if there are some additions needed after next release.

Link to comment
1 minute ago, KluthR said:

Thats already possible?

 

I would bet, if you declare something like /mnt/cache/appdata/myfolder/*.ext that would ignore all .ext files.

We will see if there are some additions needed after next release.

 

Sorry. I did know that folders could be excluded. I just don't know Plex's folder structure. Maybe @Dalarielus isn't able to exclude a folder because it has multiple different file types and some of those files are needed in the backup. 

 

I'll let Dalarielus continue this discussion. 

Link to comment
4 hours ago, KluthR said:

I would bet, if you declare something like /mnt/cache/appdata/myfolder/*.ext that would ignore all .ext files.

We will see if there are some additions needed after next release.

I wonder if that's recursive. The way Plex stores it's metadata is odd, or at least not very human friendly.

 

I have a folder called /appdata/binhex-plexpass/Plex Media Server/Media/localhost/ which contains a serial numbered folder for each object in my Plex library, each one containing several files - including the bif file that I don't want to backup.

 

Obviously it wouldn't be possible to exclude each numbered folder individually. My only worry is that declaring `/appdata/binhex-plexpass/Plex\ Media\ Server/Media/localhost/*.bif` as an exclusion would only exclude bif files located in /.../localhost and not anything deeper. 

 

Hmm. Time to experiment!

 

Edit: Escaped my spaces :)

Edited by Dalarielus
Escaped my spaces :)
Link to comment

Hi, 

 

Can anyone please help me out with what could be causing the input/output error I am getting with CA Backup on my Nextcloud container? 

The container is stopped and I do not even have calendar installed on Nextcloud

 

[20.02.2023 03:42:10] Backing Up: nextcloud
/usr/bin/tar: nextcloud/www/nextcloud/apps/calendar/: Cannot savedir: Input/output error
/usr/bin/tar: Exiting with failure status due to previous errors
[20.02.2023 03:43:21] tar creation/extraction failed!
[20.02.2023 03:43:21] Verifying Backup nextcloud
[20.02.2023 03:44:26] Backing Up: unifi-controller
[20.02.2023 03:44:46] Verifying Backup unifi-controller
[20.02.2023 03:44:51] Backing Up: zigbee2mqtt
[20.02.2023 03:44:51] Verifying Backup zigbee2mqtt
[20.02.2023 03:44:51] done
[20.02.2023 03:44:51] Starting gsdock... (try #1) done!
[20.02.2023 03:44:55] Starting mariadb... (try #1) done!
[20.02.2023 03:44:59] Starting nextcloud... (try #1) done!
[20.02.2023 03:45:04] Starting unifi-controller... (try #1) done!
[20.02.2023 03:45:12] A error occurred somewhere. Not deleting old backup sets of appdata
[20.02.2023 03:45:12] Backup / Restore Completed

Link to comment

I found one issue, but I dont know if thats your issue: Your usbBackup Destination is set to /mnt/user/Backups/. Thats the same dir for the destination.

 

Since the USB Backup makes an 1:1 copy of it, it also deletes EVERY other data (including your backups). So your backups are currently broken. Please create a dir names "USBBackup" or similar INSIDE /mnt/user/Backups/ and let the USB backup path point to it. Then try again, please. You can safely clear /mnt/user/Backups/ before - its broken anyway.

 

That is one thing, which will be fixed in the new version. Got less time this week, but I try to finish it asap.

Edited by KluthR
Link to comment
20 minutes ago, mbc0 said:

my backups were ok and not broken

I dont think so. You can see that this rsync always deletes the backup set. So you only have one working backup. Now it should grow to the „keep x backups“ setting.

 

is it always this calendar app thats failing? Could you look into it whats inside?

 

since your nextcloud container is stopped, there is nothing accessing it. Weird.

 

some posts ago I asked an user to add some diag code, could you do the same? Dont have the post link, Iam at my mobile phone.

Link to comment

Hope I'm not posting an issue that's already been brought up, tried to search/skim the thread. I put the following into the "Excluded Folders" field in the backup setup.

 

/mnt/user/appdata/plex/Library/Application Support/Plex Media Server/Cache/, /mnt/user/appdata/plex/Library/Application Support/Plex Media Server/Media/, /mnt/user/appdata/plex/Library/Application Support/Plex Media Server/Metadata/, /mnt/user/appdata/sonarr/MediaCover/

 

I thought this would ignore just those child folders. However in actuality what happened is the entire parent folder was ignored. So I got no backup of Plex or Sonarr in my backup folder. Did I configure this improperly? Seems like others have no issues with this setup.

 

Pre post updated – lol haven't even hit submit yet but I'm going to post this question in case I'm wrong here.. I'm assuming these paths are supposed to be relative to the appdata folder? So I can drop /mnt/user/appdata/? Seems weird that would cause the entire folder to be ignored though. Anyway, thanks in advance for any help! I just had to restore and realized I didn't have a Plex backup, but I did have many other critical folders backed up safely and Plex can rebuild itself. Love the plugin!

Link to comment
  • KluthR changed the title to [Plugin] CA Appdata Backup / Restore v2.5

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.