[Plugin] CA Appdata Backup / Restore v2


Squid

Recommended Posts

6 hours ago, norp90 said:

As of last night, on 6.10.3, fix common problems is alerting me that:

The plugin ca.backup2.plg is not known to Community Applications and is possibly incompatible with your server.

What's changed and what now?

 

5 hours ago, CiscoCoreX said:

Same here.

 

4 hours ago, alexbn71 said:

Me too

 

2 hours ago, Hitcher said:

Following for the same reason.

 

In a nutshell, GitHub is a piece of shit.  (If some one can come up with a better term that has all of the same connotations then please let me know)

 

A change has been now implemented in the application feed to account for it being a P.O.S., and you won't be seeing this kind of stuff again

  • Like 1
  • Thanks 7
  • Haha 7
  • Confused 1
Link to comment

Tried searching but can't find the answer I'm looking for. 

 

Managing a friends server and he has his Plex appdata in a separate pool by itself. Was wondering if CA Backup would grab his Cache and plexCache simultaneously or what is the best way to accomplish this? Can "appdata source" be comma separated for multiple locations?

Link to comment
6 minutes ago, rmeaux said:

Managing a friends server and he has his Plex appdata in a separate pool by itself. Was wondering if CA Backup would grab his Cache and plexCache simultaneously or what is the best way to accomplish this? Can "appdata source" be comma separated for multiple locations?

There is a way to make this work by setting things up with the containers so there is only one appdata user share, but different containers use appdata on different pools.

 

For example, say you have most containers using appdata on cache, and plex is using appdata on plexpool.

 

If the other containers specify /mnt/cache/appdata for their appdata, and plex container specifies /mnt/plexpool/appdata for its appdata, then the appdata would be on separate pools, but CA Backup could still get them all with the single path /mnt/user/appdata.

  • Thanks 1
Link to comment
1 minute ago, trurl said:

There is a way to make this work by setting things up with the containers so there is only one appdata user share, but different containers use appdata on different pools.

Yep, this is how I have mine set up...

  • An SSD with a cache pool named "Docker"
  • An SSD with a cache pool named "Plex"
  • An "appdata" User share on the Docker pool.  All docker appdata is in here except Plex
  • There is also an "appdata" folder on the Plex drive.  All Plex appdata is in here
  • "/mnt/user/appdata/" as the Appdata Share (Source) in Backup & Restore sees the folders on both Docker and Plex
  • All docker templates point to /mnt/user/appdata/<docker folder>, but I guess they could point to /mnt/docker/appdata (no idea)

Downsides..

  • Occasionally, a "PlexMediaServer" folder appears on my docker drive - have to move it to the Plex drive (no problems with service, and happens very rarely)
  • Fix Common Problems plugin issues a warning that appdata files/folders exist on the Plex pool.  I've just ignored that thinking there's currently no way around it (unless there is and I don't know)
Link to comment
3 minutes ago, Cessquill said:

All docker templates point to /mnt/user/appdata/<docker folder>

This doesn't follow the instructions I gave, and would explain why you have those "downsides".

 

You need each template to specify the path to the exact pool you want its appdata on

16 minutes ago, trurl said:

other containers specify /mnt/cache/appdata for their appdata, and plex container specifies /mnt/plexpool/appdata for its appdata

 

Link to comment
1 hour ago, trurl said:

This doesn't follow the instructions I gave, and would explain why you have those "downsides".

 

You need each template to specify the path to the exact pool you want its appdata on

 

Absolutely, yes. Thought of that as I was typing - apologies.  Fix Common Problems would still warn against a share across multiple pools though, yes?

 

EDIT: I've now gone and set all dockers to point to /mnt/docker/.appdata...
Turns out I had already pointed Plex to /mnt/plex/appdata, so I'd obviously started at some point.

Also to note, stopped the docker service and set the default docker location to /mnt/docker/appdata

Edited by Cessquill
Link to comment
  • 2 weeks later...
On 9/8/2022 at 5:33 PM, Cessquill said:

Absolutely, yes. Thought of that as I was typing - apologies.  Fix Common Problems would still warn against a share across multiple pools though, yes?

 

EDIT: I've now gone and set all dockers to point to /mnt/docker/.appdata...
Turns out I had already pointed Plex to /mnt/plex/appdata, so I'd obviously started at some point.

Also to note, stopped the docker service and set the default docker location to /mnt/docker/appdata

Sorry for hijacking this thread, but what's the benefit of splitting the Plex appdata from the general Appdata folder?

Link to comment
On 9/18/2022 at 3:21 AM, gshlomi said:

Sorry for hijacking this thread, but what's the benefit of splitting the Plex appdata from the general Appdata folder?

For myself and my friend, we generate thumbnails which gets rather large as your library grows. That's the main reason. Plex can have a 1TB pool for itself.

 

If I exclude my plex app data from my CA Backup I drop from 510Gb to about 27Gb with ~30 dockers. I now exclude three sub folders in my plex app data to basically obtain the same results but it's still in it's own pool now. 

 

I also moved my nextCloud data folder to the new pool so it's served from a never spun down SSD and somewhat protected with the pool's RAID. I then syncthing my nextCloud folder to my friends server and he does the same to mine for an offsite. (send only/receive only setup)

Edited by rmeaux
  • Thanks 1
Link to comment
  • 2 weeks later...

Hello all

 

I am using the "docker.io/library/postgres:14.5-alpine" image and the "dpage/pgadmin4:latest" image. Both images use their own users and so the rights of the created file in the appdata folder are also different from other images.

 

My problem is that these file permissions are gone after each backup and the containers either do not start at all or only with errors. Why are the file permissions changed during backup and how can I disable this?

 

Thanks Marzel

Link to comment

Hi All.

 

I've noticed that my backups since updating Unraid have almost doubled in time to make.  While I diagnose, I am just throwing this out there in case something external has changed that I am not aware of.

 

Has anyone else seen this behavior?

 

I'll do some manual 'tar' stuff to further see what is up...

 

TY.

 

Kev.

Link to comment

With several dockers running (Plex, mariaDB, Photoprism, nextcloud, etc) the appdata tar backup file is getting to about 250Gb and that is for about 3.6million files.

While I know how to just extract the folder needed in case let's say plex appdata files are getting corrupted, just for the tar command to go trough the 3.6million files it can take hours. 

Would it be possible during the backup process to create tarballs for individual appdata folders rather than the whole appdata. 

  • Upvote 1
Link to comment

No knock to tar or the plug-in, but with my archive approaching extremely large sizes, the streaming storage method of tar makes for very long access to its internals via MC and the the like.

 

I've ended up moving my backups to restic.  While the initial backup (with new built in compression!) took about 14 hours, subsequent backups now take minutes.  I can also now opt to run backup scrips to segment backups, so I perform a Plex /appdata backup separately from all the other /appdata contents (which run an hour later).

 

Restic (and Borg) are pretty awesome and perhaps I will fork the existing plug-in to use restic if all parties are good with it.

 

Kev.

  • Like 2
Link to comment

I dont know if this feature asked before, but it would be nice if every docker appdata folder have its own tar backup file instead of one big tar file?

as example "Emby.tar.gz", "DiskSpeed.tar.gz",... instead of "CA_backup.tar.gz"

Edited by ichz
  • Like 1
  • Upvote 1
Link to comment
18 minutes ago, ichz said:

I dont know if this feature asked before

Three posts above yours.

 

On 9/18/2022 at 9:21 AM, gshlomi said:

Sorry for hijacking this thread, but what's the benefit of splitting the Plex appdata from the general Appdata folder?

With me, it was making sure no other disk activity on my main AppData drive slowed down anything that Plex was doing.  I've currently got a lot of read/write on that drive which is slowing my dockers down (trying to track down the culprit).

Link to comment
22 minutes ago, DanielPT said:

Hi :)

 

Thanks for this nice plugin!

 

Can somebody tell me if the CA backup tool still brakes the binhex - Qbittorrent VPN docker?

 

Regards!

Why would it break it?

 

The plugin stops the containers, does a backup, and restarts the containers...

Link to comment
10 minutes ago, Squid said:

Why would it break it?

 

The plugin stops the containers, does a backup, and restarts the containers...

 

I just read that binhex tells that it breaks the docker. I will try to find it :)

[Support] binhex - qBittorrentVPN - Page 81 - Docker Containers - Unraid

here you can see it.

Edited by DanielPT
Link to comment
8 hours ago, DanielPT said:

 

I just read that binhex tells that it breaks the docker. I will try to find it :)

[Support] binhex - qBittorrentVPN - Page 81 - Docker Containers - Unraid

here you can see it.

You forgot the link, but if you're referring to this then that's something else. Pause/resume is not the same thing as stop/start. I back that container data up every night and it has never "broke".

  • Like 1
Link to comment
  • Squid locked this topic
Guest
This topic is now closed to further replies.