[Plugin] CA Appdata Backup / Restore v2

[trurl 1519]

4 minutes ago, doogalbeez said:

My disk was having 150 read errors

Did it have a red X next to it?

[doogalbeez 0]

4 minutes ago, trurl said:

Are you sure that drive was the source of the noise? Typically there will be other symptoms. Were there? Do you have Notifications setup to alert you immediately by email or other agent when Unraid detects a problem?

 

You might try opening it in 7zip on your PC.

I assumed it was, it's pretty old. Looking at the attributes, it was in "pre-fail" and highlighted. I never removed it from the computer yet, its sitting unassigned.

[doogalbeez 0]

9 minutes ago, trurl said:

Did it have a red X next to it?

No. The parity checks were taking much longer, i figured it was part of the clicking/problem hard drive.

[trurl 1519]

The correct way to replace a disk is to simply assign the new drive to the slot of the drive your want to replace, and the replacement drive will be rebuilt from the parity calculation so it has the identical contents as the original, replaced drive.

 

 

[doogalbeez 0]

Just wanted to thank those who helped me and my dumb mistakes. I was able to restore the dockers using the "add container". No real data was lost. Also I had to unzip the backup, then using terminal, to copy over the data to the appdata folder. The failing could've been, the restore was backing up to a subfolder within the appdata, or who knows... nevertheless its working again, and i'll follow through updated guides and not be a dumbass

[DZMM 236]

I keep my Plex appdata on a UD drive i.e. Plex is stored at /mnt/disks/nvme/appdata/plex and all the others at /mnt/cache/appdata.

 

Is there a way for the plugin to backup both locations?

 

Thanks

[trurl 1519]

6 hours ago, DZMM said:

I keep my Plex appdata on a UD drive i.e. Plex is stored at /mnt/disks/nvme/appdata/plex and all the others at /mnt/cache/appdata.

 

Is there a way for the plugin to backup both locations?

 

Thanks

User Scripts is probably the best approach for handling all the possible variety people would need for all the possible configurations they might make.

[_rogue 3]

I switched to this plugin this past weekend. I used to tar my backups manually anyways to it's nice to have it integrated. Question though, can we get an option where the dockers are updated and restarted before the verification. I just like to minimize downtime for my services as much as possible. Thanks

[lotekjunky 0]

I just found this app and LOVE it. Great job!  I also just searched through this whole thread for information on how to make the backup password protected / encrypted. My whole system is using luks, but my backups are out in the clear. I want to store them in my gdrive, but I don't want to put them in the cloud without a little bit of protection.

 

Any ideas on this? I thought about wrapping the .tar.gz (I'm using compression, holy cow... it dropped my backup from 5GB to 900MB) in a passworded archive, but that seems archaic.

[notadoktor 0]

Hello. I've been using this plugin for some time without issue (running Unraid 6.7.2). A couple days ago I replaced my CPU (i5-3470 -> i7-3770) and when I wake up in the morning all my dockers are not running (not sure what changing the CPU has to do with anything but it seems coincide with when issue started). I run this plugin every morning at 3:55am. When I try to start the dockers I get "Execution Error - Server Error". I can temporarily fix the issue by restarting the system but it reappears every morning.

 

As another data point, I am using a AOC-SAS2LP-MV8. I'm aware of the issues folks are experiencing, I personally haven't seen any issues (that I'm aware of) so I've just continued to run it. All my disks are attached to it (including the SSD cache drive). My cache drive is a Crucial BX500 240GB (2 months old).

 

Any help would be greatly appreciated.

tower-diagnostics-20191010-1257.zip

[Squid 2894]

8 minutes ago, notadoktor said:

"Execution Error - Server Error".

If you view your docker run command https://forums.unraid.net/topic/57181-real-docker-faq/#comment-564345  the error will be listed after it for you to see and fix

[DavidSpek 4]

For the past 2 weeks or so I have also been having the problem that the CA Backup/Restore is stopping my dockers but not restarting them. However, in the morning I am just able to properly start all my dockers manually without any problem. This has been happening almost every night, but not always strangely enough. The Backup is run at 5 am and the auto update runs at 6 am. This is an example of the system log for one of the containers but it is nearly the same for all of them (except the port X (vetheXXXXXX) entered disabled state. 

 

Oct 10 05:00:01 Tower CA Backup/Restore: Stopping antennas
Oct 10 05:01:01 Tower kernel: veth1ff6810: renamed from eth0
Oct 10 05:01:01 Tower kernel: docker0: port 9(vethe0d1822) entered disabled state
Oct 10 05:01:02 Tower avahi-daemon[9942]: Interface vethe0d1822.IPv6 no longer relevant for mDNS.
Oct 10 05:01:02 Tower avahi-daemon[9942]: Leaving mDNS multicast group on interface vethe0d1822.IPv6 with address fe80::e42f:30ff:fee0:8bb1.
Oct 10 05:01:02 Tower kernel: docker0: port 9(vethe0d1822) entered disabled state
Oct 10 05:01:02 Tower kernel: device vethe0d1822 left promiscuous mode
Oct 10 05:01:02 Tower kernel: docker0: port 9(vethe0d1822) entered disabled state
Oct 10 05:01:02 Tower avahi-daemon[9942]: Withdrawing address record for fe80::e42f:30ff:fee0:8bb1 on vethe0d1822.
Oct 10 05:01:03 Tower CA Backup/Restore: docker stop -t 60 antennas

 

After this auto update of the dockers runs without any error that I can see and this is the output from me starting the dockers manually. 

 

Oct 10 07:34:13 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered blocking state
Oct 10 07:34:13 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered disabled state
Oct 10 07:34:13 Tower kernel: device vethf1af6d1 entered promiscuous mode
Oct 10 07:34:13 Tower kernel: IPv6: ADDRCONF(NETDEV_UP): vethf1af6d1: link is not ready
Oct 10 07:34:13 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered blocking state
Oct 10 07:34:13 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered forwarding state
Oct 10 07:34:13 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered disabled state
Oct 10 07:34:14 Tower kernel: eth0: renamed from veth38fb9c8
Oct 10 07:34:14 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethf1af6d1: link becomes ready
Oct 10 07:34:14 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered blocking state
Oct 10 07:34:14 Tower kernel: br-a91e46b94306: port 1(vethf1af6d1) entered forwarding state
Oct 10 07:34:14 Tower kernel: br-a91e46b94306: port 2(vetha16a16d) entered blocking state
Oct 10 07:34:14 Tower kernel: br-a91e46b94306: port 2(vetha16a16d) entered disabled state
Oct 10 07:34:14 Tower kernel: device vetha16a16d entered promiscuous mode
Oct 10 07:34:14 Tower kernel: IPv6: ADDRCONF(NETDEV_UP): vetha16a16d: link is not ready
Oct 10 07:34:14 Tower kernel: br-a91e46b94306: port 2(vetha16a16d) entered blocking state
Oct 10 07:34:14 Tower kernel: br-a91e46b94306: port 2(vetha16a16d) entered forwarding state
Oct 10 07:34:15 Tower kernel: br-a91e46b94306: port 2(vetha16a16d) entered disabled state
Oct 10 07:34:15 Tower avahi-daemon[9942]: Joining mDNS multicast group on interface vethf1af6d1.IPv6 with address fe80::245e:1aff:fec9:5b64.
Oct 10 07:34:15 Tower avahi-daemon[9942]: New relevant interface vethf1af6d1.IPv6 for mDNS.
Oct 10 07:34:15 Tower avahi-daemon[9942]: Registering new address record for fe80::245e:1aff:fec9:5b64 on vethf1af6d1.*.
Oct 10 07:34:15 Tower kernel: eth0: renamed from veth68dd4be
Oct 10 07:34:15 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vetha16a16d: link becomes ready

Please let me know if other information is needed and thank you all for the help. 

tower-diagnostics-20191010-1637.zip

[Squid 2894]

There's an overlap on the scheduling which may have something to do with it.

 

The Backup starts at 5am and it is finished in and around 8am daily.  The auto update runs at 6am.   Even there though, on Oct 8, 9, 10 it certainly appeared that the apps were all restarted.

[Coolsaber57 4]

Is there any chance of adding an option to backup vdisks along with the libvirt file?  Or is there a better tool to use for that?

 

I do have rclone, but am unsure if I need to take the VM offline to use that tool.

[DavidSpek 4]

45 minutes ago, Squid said:

There's an overlap on the scheduling which may have something to do with it.

 

The Backup starts at 5am and it is finished in and around 8am daily.  The auto update runs at 6am.   Even there though, on Oct 8, 9, 10 it certainly appeared that the apps were all restarted.

I honestly had no idea the backups were taking 3 hours. Could it be that I am manually starting the docker containers before the backup is complete, as I usually check around 7:30 am? Regarding the scheduling, should I turn off the auto-update schedule and let the backup app schedule the update, or schedule the auto-update at a different point in time? Also, is there a way to reduce the time it takes to make a backup? Sorry for the large amount of questions, but I am guessing this information is probably also applicable for others. Again, thank you for the help. 

[Squid 2894]

1 minute ago, DavidSpek said:

or schedule the auto-update at a different point in time?

That's what I do

1 minute ago, DavidSpek said:

I honestly had no idea the backups were taking 3 hours.

You've also got verification turned on.  I personally can't be bothered to do that as it takes twice as long.

2 minutes ago, DavidSpek said:

Also, is there a way to reduce the time it takes to make a backup?

See above

 

Another way to speed things up (but not really) is under advanced options you can choose to leave certain apps still running.  I've never had any problems with doing this, but there is a slight chance that if an app modifies multiple files at a certain point and the backup has already grabbed file #1, but not file #2 (now modified) that the backup will not technically be correct.  YMMV

[nuhll 94]

16 hours ago, Squid said:

You've also got verification turned on.  I personally can't be bothered to do that as it takes twice as long.

 

 

Hey, a suggestion, what about including that important information into the app? I didnt know why it took so long always...  disbaled it now... 

 

edit:
While were at it, could you implement different versions of backups for flashdrive and libvirtio?

Edited October 11, 2019 by nuhll

[Squid 2894]

4 hours ago, nuhll said:

backups for flashdrive

I'm anticipating that the flash drive backup routine will be deprecated (finally) this weekend, so no further work on it will ever happen

4 hours ago, nuhll said:

libvirtio

If / when I revisit the plugin, it's already on the grocery list.

[nuhll 94]

Oh, okay, thanks.

 

Why deprecated (finally)?

Will it no longer get backuped and if so, why?

[Squid 2894]

1 hour ago, nuhll said:

Will it no longer get backuped and if so, why?

Better solution from LT.

[nuhll 94]

19 hours ago, Squid said:

Better solution from LT.

Which is not public atm?

[Squid 2894]

8 minutes ago, nuhll said:

Which is not public atm?

Why its not deprecated yet

[nuhll 94]

8 minutes ago, Squid said:

Why its not deprecated yet

Sorry, you must know, english it not my first language so im bad at reading between lines... (you know what i mean?)

 

So there will come a better solution instead the actual solution for backing up the flashdrive? Is that correct?

Edited October 12, 2019 by nuhll

[Squid 2894]

Yes there will be a better solution

[Dazog 91]

I got this error trying to run the backup/restore plugin on 6.8 RC1

 

Oct 16 08:23:13 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:23:13 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:23:13 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:23:13 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:23:13 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:25:50 unRAID nginx: 2019/10/16 08:25:50 [crit] 7268#7268: *499895 pwrite() "/var/lib/nginx/client_body/0000001182" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/disks?buffer_length=2 HTTP/1.1", host: "localhost" Oct 16 08:26:03 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:03 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token Oct 16 08:26:05 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: wrong csrf_token Oct 16 08:26:06 unRAID root: error: /plugins/dynamix.plugin.manager/scripts/PluginAPI.php: wrong csrf_token Oct 16 08:26:06 unRAID root: error: /webGui/include/Notify.php: wrong csrf_token Oct 16 08:26:06 unRAID root: error: /webGui/include/Notify.php: wrong csrf_token Oct 16 08:26:12 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:26:14 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:26:22 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:26:23 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:26:32 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token Oct 16 08:26:33 unRAID root: error: /plugins/ca.backup2/include/backupExec.php: uninitialized csrf_token Oct 16 08:26:34 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:35 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:36 unRAID root: error: /webGui/include/DeviceList.php: wrong csrf_token Oct 16 08:26:36 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:37 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:38 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:39 unRAID root: error: /webGui/include/DeviceList.php: wrong csrf_token Oct 16 08:26:39 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:40 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:42 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:42 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /webGui/include/DeviceList.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token Oct 16 08:26:43 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token Oct 16 08:26:44 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: uninitialized csrf_token Oct 16 08:26:44 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:45 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:46 unRAID root: error: /webGui/include/DeviceList.php: wrong csrf_token Oct 16 08:26:46 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:46 unRAID root: error: /webGui/include/DeviceList.php: wrong csrf_token Oct 16 08:26:46 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:47 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:47 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:48 unRAID root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 16 08:26:52 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:52 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:52 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token Oct 16 08:26:53 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:53 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:53 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token Oct 16 08:26:54 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:54 unRAID root: error: /webGui/include/Notify.php: uninitialized csrf_token Oct 16 08:26:54 unRAID root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: uninitialized csrf_token