[Plugin] CA Appdata Backup / Restore - Deprecated


Squid

Recommended Posts

1 hour ago, deusxanime said:
From what I've seen on Plex forums, tar'ing is a recommended way to back it up, because the issue you are running into with millions of files. You only end up with a single file to delete/transfer which can greatly improve times. By doing that though you don't have the option of doing incremental syncs, but the trade-off seems like it would probably be worth it rather than hanging your entire NAS! 
 
Not sure if it only makes sense to restrict that to Plex though, or make it universal for all the appdata backups (maybe one tar for each folder in there?).

Alpha version already working more or less. Sometime in the weekend is the release. Upshot is an insane speed increase. My full backup time is 10min from 1.5 hours. With compression it's 20 min.

Downside is that initially it doesn't appear you can do an incremental backup with deleting files no longer present without running through it twice. So no more incrementals, along with the fact that with a single file its not really suitable to upload to the cloud offsite

Edited by Squid
Link to comment
Quote

NOTE: All files within your chosen destinations will be deleted when the backup runs. Only select a share/folder created SPECIFICALLY for appdata (and USB) backup purposes

 

This just got me thinking.  I've been looking for a way to automate my VM backups

At the moment I have to manually delete the old backups as they start taking up a lot of space after a while.  If I use your plugin to run my VM backup script at Stop and set the VM backup location as 

backup/Community_Applications_Appdata_Backup/VM_backup

would the following occur when the dated sub folders option is off?

  1. CA_backup runs and deletes old files in backup/Community_Applications_Appdata_Backup/ including old files in backup/Community_Applications_Appdata_Backup/VM_backup
  2. appdata backup job runs and stores backup in backup/Community_Applications_Appdata_Backup/ and when finished starts my vm backup script
  3. vm backup script recreates vm backups in backup/Community_Applications_Appdata_Backup/VM_backup
  4. CA_backup finishes until next scheduled run

If so, then I'll be a very happy bunny as I can do all my backups in one go and only have one instance that manually refreshes!

Link to comment

Won't work the way you have it, but this will:

 

Set your vm backup to go to your appdata share (not to the backup location) and is run as the docker stop script, then appdata will handle everything else, deleting old versions (if using dated backup sets)

Link to comment

This version of the plugin is now deprecated due to some flaws with regards to filesystems handling mass deletions.  If you have no problems with this version of the plugin, there is no need to change to the v2 version here, but there will be no further development on this version.

 

If you do happen to switch over, then this v1 version will need to be uninstalled.

Link to comment
  • 2 weeks later...
On 05/11/2017 at 8:54 PM, Squid said:

This version of the plugin is now deprecated due to some flaws with regards to filesystems handling mass deletions.  If you have no problems with this version of the plugin, there is no need to change to the v2 version here, but there will be no further development on this version.

 

If you do happen to switch over, then this v1 version will need to be uninstalled.

 

I've never had a problem with the plugin but I think the reason for that might be the fact that while my Plex database has a lot of files it doesn't have literally millions. I don't have Generate video thumbnail previews enabled in the Plex Settings -> Server -> Library page, though I do have Generate chapter thumbnails enabled as a scheduled task. I recommend that anyone not needing the video preview thumbnails make sure they are switched off.

Link to comment
  • 1 month later...

Thanks a lot for this plugin, Squid.

 

1. I set it months ago to compress, and it does,
however on the plugin settings it says "No" for "Use Compression?".

 

2. I also have a Plex library with about 1M files. Krusader fails to open the 21GB tar file,

so I have doubts whether in time of need I will be able to restore from it.

Can the plugin offer an option to restore to a different folder?

This option can be used for recovery drill, and also good for comparing what files have changed between backups.

Link to comment
  • 1 month later...
  • 4 weeks later...

So forgive me if this is mentioned somewhere, but my searches on this forum don't always seem to come up properly. Aka sometimes I see no hits when something exists. I wanted to see if there could be any clarity provided on the "custom start script" and "custom stop script" options. I'm trying to understand 2 things about these:

  1. Are these for custom scripts that run the backups? Like to override the scripts that are actually being used? Or are these intended to allow a user running a custom script at start/stop.
  2. Are the start/stop referencing the start/stop of the backup? Or the start/stop of the docker containers to process the backups? 
Link to comment
12 minutes ago, 1activegeek said:
  • Are these for custom scripts that run the backups? Like to override the scripts that are actually being used? Or are these intended to allow a user running a custom script at start/stop.
  • Are the start/stop referencing the start/stop of the backup? Or the start/stop of the docker containers to process the backups? 

Run in addition to the backup

 

Stop scripts run before the containers are stopped.  Start scripts are run after the containers are restarted

Link to comment
9 minutes ago, Squid said:

Run in addition to the backup

 

Stop scripts run before the containers are stopped.  Start scripts are run after the containers are restarted

 

Aha, ok cool. Glad I asked. I was thinking it was like an override. I would assume this means I should be able to easily link to scripts setup in the UserScripts plugin, just pointing to /boot/config/plugins/user.scripts/scripts/pause_statuscake/script for example? 

 

 

Link to comment
2 minutes ago, 1activegeek said:

 

Aha, ok cool. Glad I asked. I was thinking it was like an override. I would assume this means I should be able to easily link to scripts setup in the UserScripts plugin, just pointing to /boot/config/plugins/user.scripts/scripts/pause_statuscake/script for example? 

 

 

Sure.  Except that user scripts will run a script that has CRLF line endings by design, but the scripts accepted by Backup have to be standard LF only

Link to comment
19 minutes ago, Squid said:

Sure.  Except that user scripts will run a script that has CRLF line endings by design, but the scripts accepted by Backup have to be standard LF only

Ok so to be sure I have no issue, I'll want to just re-create those scripts ensuring only LF endings. So I don't want to link to this directly. 

 

Thanks for the direction, and the great plugins! 

Link to comment

I messed up a VM definition today and went to restore its XML from the CA Backup backup and lo and behold, the copy there is from last year! libvert.img is backing fine, as is the USB stick, but the XML for my one and only VM is old.

 

So I went to look at the CA Backup/Restore settings and I can't even find a setting to backup the XML. I could have sworn it was there when I set it up last year (as also hinted at by post #1.)

 

Where is this setting?

 

Why isn't it backing up my XML along with the other backups?

 

Thanks!

Link to comment
  • 1 month later...
  • 1 month later...

I am having some problems lately when a mount goes offline that appdata backup and restore is set to backup too.

 

Using unassinged devices to mount an smb share at /mnt/disks/myshare/
Using this as a target for appdata backup and restore.

 

If, for any reason, that mount goes offline, backup will now fill up my ram inside the /mnt/disks/myshare/ folder.

Is there a way to precheck if the remote share is correctly mounted and throwing an error instead of filling up a folder in ram?

Link to comment
  • 2 months later...
16 hours ago, mbc0 said:

Hi,

 

Can anyone see why I should get 2 backups within 45 mins of each other every 3 days please? I cannot work this out? I just want a backup every 3 days.

 

image.png.47e30fa621dfb9fffcf3df38830264db.png

 

image.png.698dbab8483b50929a346ec9e2017f29.png

 

In your custom cron entry, you have a "*" character in the first slot which makes it so it is set to try to run at every minute during the 3:00 AM hour.

 

* * * * * *
| | | | | | 
| | | | | +-- Year              (range: 1900-3000)
| | | | +---- Day of the Week   (range: 1-7, 1 standing for Monday)
| | | +------ Month of the Year (range: 1-12)
| | +-------- Day of the Month  (range: 1-31)
| +---------- Hour              (range: 0-23)
+------------ Minute            (range: 0-59)

 

So it kicks off at 3:00 AM and runs and then if it completes before 4:00 AM (so any 3:XX time) it tries to run again. If you want it to only run at 3:00 AM you need this for your custom cron entry:

 

0 3 */3 * *

 

  • Like 2
Link to comment
11 hours ago, deusxanime said:

 

In your custom cron entry, you have a "*" character in the first slot which makes it so it is set to try to run at every minute during the 3:00 AM hour.

 


* * * * * *
| | | | | | 
| | | | | +-- Year              (range: 1900-3000)
| | | | +---- Day of the Week   (range: 1-7, 1 standing for Monday)
| | | +------ Month of the Year (range: 1-12)
| | +-------- Day of the Month  (range: 1-31)
| +---------- Hour              (range: 0-23)
+------------ Minute            (range: 0-59)

 

So it kicks off at 3:00 AM and runs and then if it completes before 4:00 AM (so any 3:XX time) it tries to run again. If you want it to only run at 3:00 AM you need this for your custom cron entry:

 


0 3 */3 * *

 

You are a star! Thank you so much!!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.