[Plugin] CA Appdata Backup / Restore - Deprecated

[Squid]

2 minutes ago, thaddeussmith said:

why does it make the webgui, smb, etc completely unresponsive? i'm able to SSH, but even it is laggy.

During a backup its reading / writing from the cache drive to the array, which depending upon your hardware may impact performance.  Your system is probably laggy during SMB transfers by themselves.

 

If watching the progress during a backup, then it's checking the status and updating the screen every (IIRC) 2 seconds.

[thaddeussmith]

1 minute ago, Squid said:

Your system is probably laggy during SMB transfers by themselves.

 

Not at all - I have ample compute and bonded 1G NIC's. I can't login to the webgui to check anything or manage the array, let alone cancel the tasks and restart my docker containers.

[Squid]

Diagnostics during a backup.

 

SSH, diagnostics

[thaddeussmith]

attempting now.. so far it's just hanging on that command

[thaddeussmith]

forced a reboot of the server. looks like the backup succeeded and it started to hang when deleting the previous backup. I presume the issue is the massive amount of file in Plex directory.. I'll exclude Plex and see if the next run is any better.

[shooga]

I've been having trouble on and off with this too. It seems to work fine when I do a manual backup, but sometimes the automated backups seem to hang and cause the server to become non-responsive. I'm also backing up my Plex directory, but that's one of the main things I want to back up, so I don't want to exclude it.

 

@thaddeussmithAre you just looking at the system log? Does that persist past the forced reboot? I'm unable to access the server when mine hangs and couldn't get to the log file.

[thaddeussmith]

Does what persist past the reboot? 

[shooga]

The log file info that you used to troubleshoot your issue.

[thaddeussmith]

Derp, sorry. I just checked the status tab of the plug-in's settings section. The referenced directory wasn't yet removed like the last line stated.

 

[Squid]

You guys might want to read this starting here:

 

Net result is that I have zero clue why a couple of people have problems with this as both the copy and the removing of old sets fails / hangs as its simply a standard bash command

 

 

[thaddeussmith]

Already using xfs across the board. I deleted both backup directories manually using rm -rf and they cleared in a couple of minutes each - no gui hang ups. Likewise I never have issues with manual rsync.

 

Good to know the issue is a big ¯\_(ツ)_/¯

[shooga]

Thanks @Squid.  I understand your logic here and it makes sense. I am using xfs on my array and btrfs on my cache pool.

 

I will try changing the backup setting to use a single disk. I am also trying to space out my scheduled mover and backup times, just to make sure they are not running at the same time.

 

I guess it's worth mentioning that I was just deleting some old backup data and when I used rm -r on the plex directory it took ~10 minutes and my GUI was unresponsive for part of that. It did come back though. The hang I see when backing up seems to last indefinitely, but the behavior is similar.

Edited June 4, 2017 by shooga

[Squid]

1 hour ago, shooga said:

Thanks @Squid.  I understand your logic here and it makes sense. I am using xfs on my array and btrfs on my cache pool.

 

I will try changing the backup setting to use a single disk. I am also trying to space out my scheduled mover and backup times, just to make sure they are not running at the same time.

 

I guess it's worth mentioning that I was just deleting some old backup data and when I used rm -r on the plex directory it took ~10 minutes and my GUI was unresponsive for part of that. It did come back though. The hang I see when backing up seems to last indefinitely, but the behavior is similar.

There's also something you can set in the Tips & Tweaks plugin.  (Something about vfs.cache_pressure IIRC)  You'd have to search...

[HellDiverUK]

Oddity with this plugin doing a backup under 6.4.0rc2.  Says it's going to do it's thing, then does feck all.

 

Log:

Jun 6 17:07:36 WHS root: #######################################
Jun 6 17:07:36 WHS root: Community Applications appData Backup
Jun 6 17:07:36 WHS root: Applications will be unavailable during
Jun 6 17:07:36 WHS root: this process. They will automatically
Jun 6 17:07:36 WHS root: be restarted upon completion.
Jun 6 17:07:36 WHS root: #######################################
Jun 6 17:18:05 WHS root: Community Applications rsync process running. Killing

I waited for it to do something for 11 minutes, then hit the Abort button.  It's still doing feck all.

 

It worked great until the upgrade to 6.4, backup took about 90 seconds to run before, including time waiting for the destination drive spinning up.

 

Ideas?

Edited June 6, 2017 by HellDiverUK

[Squid]

A previous backup (scheduled?) was still running (or the program thought it was running for some reason).  A reboot should fix you up.

[1812]

I had a cache drive bite it today. Thankfully I had this great backup already setup. I followed the steps listed and got all the dockers back within minutes. But I'm not sure how to restore my vm data which is backed up in appdata. The images are still in place becase they were on unassigned disks. I could recreate the xml, but is here an easier way I'm just overlooking? 

 

[JorgeB]

3 minutes ago, 1812 said:

I could recreate the xml, but is here an easier way I'm just overlooking? 

 

Do you backup the libvirt.img? If yes just restore that.

[1812]

5 minutes ago, johnnie.black said:

 

Do you backup the libvirt.img? If yes just restore that.

If it's not including in in this backup/restore program, then no. Unless it is stored somewhere in the array, then I have a backup via crash plan. 

 

Edit-

 

if it's the img located in the system file, it was not a part of my crash plan backup for some reason....

Edited June 6, 2017 by 1812

[Squid]

Just go to your VM XML backup over the network, open up the xml's, then add a custom VM, and paste the contents of the xml in there.  You'll lose the icon though.

 

XML backups are something just tacked onto system, and if I ever make it far enough down my whiteboard todo list, then I'd integrate danioj's vm backup script entirely... 

[JorgeB]

2 minutes ago, 1812 said:

If it's not including in in this backup/restore program, then no. Unless it is stored somewhere in the array, then I have a backup via crash plan. 

 

Settings -> VM Manger -> (Advanced mode enable) - Libvirt storage location:

[JorgeB]

1 minute ago, Squid said:

XML backups are something just tacked onto system, and if I ever make it far enough down my whiteboard todo list, then I'd integrate danioj's vm backup script entirely...

 

Why not backup the libvirt.img? All you need to restore VMs is that and the vdisks.

Edited June 6, 2017 by johnnie.black

[Squid]

At the time it was made I didn't run a VM, so merely added what someone here asked me to add.

 

But now that I'm shamed into it, I can feel an update coming up...

[1812]

3 minutes ago, Squid said:

At the time it was made I didn't run a VM, so merely added what someone here asked me to add.

 

But now that I'm shamed into it, I can feel an update coming up...

 

Awesome! Great things come out of my failures!

[1812]

Had to use krusader to move the nvram back, but all is good now. Looking forward to the automated version in the future!

[Squid]

By design.  When set to same destination its first copied to to appdata.  That way it can be part of backup sets, etc.  Far simpler coding.