MickMorley Posted March 28, 2017 Share Posted March 28, 2017 4 hours ago, Squid said: Well that blew away the only theory I had... Here's what's going on: The rsync started and was carrying on no problems. Then, up in the middle of it it just decided to quit. This is probably because something somewhere killed off CA's backup script. Except all of this happened without any errors or notifications or anything being logged anywhere. There have been intermittent reports of mover crashing (also uses rsync), but AFAIK no one has ever adequately explained why or how. This is a built-in command to linux, and is not an add-on at all. But the messed up thing is that if you run it manually, it runs to completion. My suggestion on how to fix you're probably not going to like: Easter is coming... Take the server with you on Easter Sunday to your local church and get it blessed along with all of your food for the dinner.... It's possessed. IE: I have no clue what's going on, nor is there anything I can do about it. The cause is not within CA Appdata Backup. EDIT: Not quite true. I have a feeling that php is getting Bus Errors causing complete abnormal exits (may be a bug in php7) (does anything appear on the locally attached monitor?), but either way it's way out of my purvue. Thanks for the input and taking a look. Hopefully some explanation surfaces in the near future. Aside from a full Ubuntu server, unRaid is the only solution that fits my needs. I'll see if it stops on Emby again next week and if so I may remove that docker. I'll see if I can get further logs and information on this, maybe through Fix Common Problems and/or the local monitor. I haven't had a monitor attached for any of these incidents. For me it needs to be plugged in when I start the server or it won't recognize it later. It's funny that you mention bringing it to church, I actually help out with the tech team at church and recently convinced the sound bar tech to go unRaid. Quote Link to comment
fc0712 Posted March 30, 2017 Share Posted March 30, 2017 Hey I'm using Google drive rclone to backup my media. I'm looking into also backing up my app data folder. Should I then just do a backup through this tool to a local folder on my server, and then make an rclone script to sync that folder with my cloud provider (google drive) or is there a better way? Quote Link to comment
bobbintb Posted March 30, 2017 Share Posted March 30, 2017 2 minutes ago, fc0712 said: Hey I'm using Google drive rclone to backup my media. I'm looking into also backing up my app data folder. Should I then just do a backup through this tool to a local folder on my server, and then make an rclone script to sync that folder with my cloud provider (google drive) or is there a better way? There are many ways to do it, just depends on what you need/want. I would probably use this to backup to another folder and then use rclone to backup that folder. The reason being is the backup app is capable of stopping dockers cleanly before backup and other things that rclone cannot. Quote Link to comment
Squid Posted March 30, 2017 Author Share Posted March 30, 2017 9 hours ago, fc0712 said: Hey I'm using Google drive rclone to backup my media. I'm looking into also backing up my app data folder. Should I then just do a backup through this tool to a local folder on my server, and then make an rclone script to sync that folder with my cloud provider (google drive) or is there a better way? Problem with appdata backups either directly through a rclone script or with this plugin to a folder mounted with the rclone plugin is symlinks / hardlinks. Most (if not all) docker apps makes extensive use of them, and they *may* be incompatible. While CA Appdata Backup does support destinations of folders mounted with the rclone plugin, if it returns errors (and it probably will) then the backup may or may not be of any use to you in the event of a cache drive failure. I only recommend (and support) backups with a destination on the local array (or to an Unassigned Device mounted via the UD plugin). See here for another user's attempt: Quote Link to comment
fc0712 Posted March 31, 2017 Share Posted March 31, 2017 On 30/3/2017 at 6:03 PM, Squid said: Problem with appdata backups either directly through a rclone script or with this plugin to a folder mounted with the rclone plugin is symlinks / hardlinks. Most (if not all) docker apps makes extensive use of them, and they *may* be incompatible. While CA Appdata Backup does support destinations of folders mounted with the rclone plugin, if it returns errors (and it probably will) then the backup may or may not be of any use to you in the event of a cache drive failure. I only recommend (and support) backups with a destination on the local array (or to an Unassigned Device mounted via the UD plugin). See here for another user's attempt: Would it work if I just do a backup to a local folder on my unraid server and then set up a rclone sync script to sync that folder to my google drive? Quote Link to comment
Squid Posted March 31, 2017 Author Share Posted March 31, 2017 Would it work if I just do a backup to a local folder on my unraid server and then set up a rclone sync script to sync that folder to my google drive?The local copy would definitely work on a restore. Ymmv on the Google versionSent from my LG-D852 using Tapatalk Quote Link to comment
Ockingshay Posted April 1, 2017 Share Posted April 1, 2017 I do a weekly backup that includes the flash drive and just had the thought that if my USB dies i won't be able to access that backup as it's saved to a share. What's a good way of after completing just the flash backup it's sent to something like dropbox/another LAN? I would like it to only keep the latest backup, otherwise it could start to get quite large over time. what do other people do? cheers! Quote Link to comment
wgstarks Posted April 1, 2017 Share Posted April 1, 2017 54 minutes ago, Ockingshay said: I do a weekly backup that includes the flash drive and just had the thought that if my USB dies i won't be able to access that backup as it's saved to a share. What's a good way of after completing just the flash backup it's sent to something like dropbox/another LAN? I would like it to only keep the latest backup, otherwise it could start to get quite large over time. what do other people do? cheers! I backup to a external drive mounted via Unassigned Devices. You could also use OwnCloud or something similar to backup to the cloud or another machine probably. Might want to test your backups to insure that hardlinks still work. 1 Quote Link to comment
Squid Posted April 1, 2017 Author Share Posted April 1, 2017 3 hours ago, Ockingshay said: I do a weekly backup that includes the flash drive and just had the thought that if my USB dies i won't be able to access that backup as it's saved to a share. What's a good way of after completing just the flash backup it's sent to something like dropbox/another LAN? I would like it to only keep the latest backup, otherwise it could start to get quite large over time. what do other people do? cheers! Its seemingly an issue, but it really isn't. If the flash dies, make a new trial usb. Boot it up. And assign the disks At that point, all of your shares are now accessible. Copy the backup from the array to the new flash. Reboot and and your back in business like nothing happened. Or, as @wgstarks said, set the backup destination for USB to something else accessible. (A UD assigned disk, a UD mounted SMB share on a different computer, etc) 1 Quote Link to comment
Squid Posted April 1, 2017 Author Share Posted April 1, 2017 On 3/28/2017 at 6:27 PM, MickMorley said: Thanks for the input and taking a look. Hopefully some explanation surfaces in the near future. Aside from a full Ubuntu server, unRaid is the only solution that fits my needs. I'll see if it stops on Emby again next week and if so I may remove that docker. I'll see if I can get further logs and information on this, maybe through Fix Common Problems and/or the local monitor. I haven't had a monitor attached for any of these incidents. For me it needs to be plugged in when I start the server or it won't recognize it later. It's funny that you mention bringing it to church, I actually help out with the tech team at church and recently convinced the sound bar tech to go unRaid. Try this: No guarantees Quote Link to comment
Ockingshay Posted April 1, 2017 Share Posted April 1, 2017 2 hours ago, Squid said: Its seemingly an issue, but it really isn't. If the flash dies, make a new trial usb. Boot it up. And assign the disks At that point, all of your shares are now accessible. Copy the backup from the array to the new flash. Reboot and and your back in business like nothing happened. Or, as @wgstarks said, set the backup destination for USB to something else accessible. (A UD assigned disk, a UD mounted SMB share on a different computer, etc) Thanks squid, that's what I needed to know. Quote Link to comment
DZMM Posted April 3, 2017 Share Posted April 3, 2017 I'm getting a warning that errors occurred after every daily backup. Is this the problem: 2017/04/03 04:37:41 [17206] rsync warning: some files vanished before they could be transferred (code 24) at main.c(1178) [sender=3.1.2] Thanks backup.zip Quote Link to comment
MickMorley Posted April 3, 2017 Share Posted April 3, 2017 On 3/28/2017 at 2:00 PM, Squid said: Well that blew away the only theory I had... Here's what's going on: The rsync started and was carrying on no problems. Then, up in the middle of it it just decided to quit. This is probably because something somewhere killed off CA's backup script. Except all of this happened without any errors or notifications or anything being logged anywhere. There have been intermittent reports of mover crashing (also uses rsync), but AFAIK no one has ever adequately explained why or how. This is a built-in command to linux, and is not an add-on at all. But the messed up thing is that if you run it manually, it runs to completion. My suggestion on how to fix you're probably not going to like: Easter is coming... Take the server with you on Easter Sunday to your local church and get it blessed along with all of your food for the dinner.... It's possessed. IE: I have no clue what's going on, nor is there anything I can do about it. The cause is not within CA Appdata Backup. EDIT: Not quite true. I have a feeling that php is getting Bus Errors causing complete abnormal exits (may be a bug in php7) (does anything appear on the locally attached monitor?), but either way it's way out of my purvue. Hi Squid, Today I actually got the notification for a successful CA Backup, however, when I tried to log in this morning the WEB UI was frozen. I couldn't SSH in and CA Backup was the only thing that runs each Monday morning. I have attached the log. What does CA Backup do after a successful run that would cause the system to freeze? backup_20170403.7z Quote Link to comment
Holbert Posted April 4, 2017 Share Posted April 4, 2017 7 hours ago, MickMorley said: Hi Squid, Today I actually got the notification for a successful CA Backup, however, when I tried to log in this morning the WEB UI was frozen. I couldn't SSH in and CA Backup was the only thing that runs each Monday morning. I have attached the log. What does CA Backup do after a successful run that would cause the system to freeze? backup_20170403.7z Same happened to me as well last night. Sorry @Squid I have been busy the past week so haven't had a change to look into this further. I have mine set to delete backups older than 9 days and I still have backups from 3/13. Could it be hanging up on that step? I'm going to try and run a backup tonight with the troubleshooting turned on. Quote Link to comment
Squid Posted April 4, 2017 Author Share Posted April 4, 2017 17 hours ago, DZMM said: I'm getting a warning that errors occurred after every daily backup. Is this the problem: 2017/04/03 04:37:41 [17206] rsync warning: some files vanished before they could be transferred (code 24) at main.c(1178) [sender=3.1.2] Thanks backup.zip Based upon the error I'd say that some files in the source were deleted prior to being backed up. Do you have some apps set to stay running during the backup? Quote Link to comment
Holbert Posted April 4, 2017 Share Posted April 4, 2017 2 hours ago, Holbert said: Same happened to me as well last night. Sorry @Squid I have been busy the past week so haven't had a change to look into this further. I have mine set to delete backups older than 9 days and I still have backups from 3/13. Could it be hanging up on that step? I'm going to try and run a backup tonight with the troubleshooting turned on. @Squid I was somewhat able to replicate my issue. Backup completed it appears, but during deletion of old backup the webgui basically will not load. After it finished deleting the two old backups it was usable until I tried to stop the array to restart the server. If completely froze and then kicked it out of my SSH session. Not sure if this is a result of troubleshooting mode or what. I've attached my log from the troubleshooting mode. Also not sure if this is something you could do, but would it be possible to have the deletion of the backups occur before the dockers are restarted? I was thinking it might help to have more CPU + disk bandwidth + RAM for the deleting of old backups? FCPsyslog_tail.txt Quote Link to comment
shooga Posted April 4, 2017 Share Posted April 4, 2017 @Squid I'm having problems with the Backup and Auto-Update plugins too. After using them for many months with no issues, I've had two consecutive weeks where my server has become totally unresponsive (requiring a long power button press) when doing my weekly backup and update. WebGUI won't load, can't SSH in. I've disabled both for now. What's the best way to troubleshoot this? Quote Link to comment
DZMM Posted April 4, 2017 Share Posted April 4, 2017 18 hours ago, Squid said: Based upon the error I'd say that some files in the source were deleted prior to being backed up. Do you have some apps set to stay running during the backup? Doesn't it automatically close and backup one app at a time? I think then what's happening is something like plexpy is deleting a Plex file while Plex is being backed up, or maybe sonarr is removing a completed deluge torrent?? Quote Link to comment
Krzaku Posted April 9, 2017 Share Posted April 9, 2017 (edited) @Squid Could you add an option to create a tar backup instead of the whole directory structure? Edited April 9, 2017 by Krzaku 1 Quote Link to comment
aptalca Posted April 13, 2017 Share Posted April 13, 2017 Squid, quick question for confirmation. . . I have been using the backup plugin for a long time and it was always set to backup to a share, on a specific disk, disk 3. Now disk 3 is getting full, and I'm thinking about changing the target location to "user" instead. Anything I should worry about? The share in question does not use a cache disk, and the same location is available through the /mnt/user path. It should just pickup all the existing files and append, right? Thanks Quote Link to comment
Squid Posted April 13, 2017 Author Share Posted April 13, 2017 3 minutes ago, aptalca said: Squid, quick question for confirmation. . . I have been using the backup plugin for a long time and it was always set to backup to a share, on a specific disk, disk 3. Now disk 3 is getting full, and I'm thinking about changing the target location to "user" instead. Anything I should worry about? The share in question does not use a cache disk, and the same location is available through the /mnt/user path. It should just pickup all the existing files and append, right? Thanks Switching it to user, it will still pick up everything and do a delta of the existing backup(s), and save the files according to the share's include / exclude / split levels Switching it to another disk in the plugin itself, and it won't know about the existing backup(s) Quote Link to comment
aptalca Posted April 13, 2017 Share Posted April 13, 2017 2 minutes ago, Squid said: Switching it to user, it will still pick up everything and do a delta of the existing backup(s), and save the files according to the share's include / exclude / split levels Switching it to another disk in the plugin itself, and it won't know about the existing backup(s) That's what I thought. Thanks so much. I guess switching to another disk would require manually moving the existing files to the new disk first. Quote Link to comment
Squid Posted April 13, 2017 Author Share Posted April 13, 2017 30 minutes ago, aptalca said: That's what I thought. Thanks so much. I guess switching to another disk would require manually moving the existing files to the new disk first. pretty much. But if you're using dated backup sets, then the existing sets will eventually get deleted on their own accord. Non dated sets, then only new files will get written to the replacement disk (when using user) Quote Link to comment
johnieutah Posted April 19, 2017 Share Posted April 19, 2017 I started playing around with this last night, but had trouble finding out exactly how to use this plugin to simply back-up only the USB boot drive . It keeps asking for a source to be defined. If I may offer some critique, the UI for this plugin is a bit of dogs dinner I'm not sure what is possible and what is not, for example the above - is it even possible? Is it not possible to separate input fields and labels up a little or have a simple/expert mode? Or better, a step-through... (I'm waiting for the hate from the power users ) Quote Link to comment
Squid Posted April 19, 2017 Author Share Posted April 19, 2017 (edited) 1 hour ago, johnieutah said: I started playing around with this last night, but had trouble finding out exactly how to use this plugin to simply back-up only the USB boot drive . Nope you can't. Hence the name. It was designed purely for appdata. Flash drive and VM XML's got added in per user request and were both single line codes. (That being said, TBH I never considered that anyone would want it purely for USB) 1 hour ago, johnieutah said: Is it not possible to separate input fields and labels up a little Not exactly sure what you mean by that... 1 hour ago, johnieutah said: I'm not sure what is possible and what is not Pressing Help and following the link shows exactly what all the available options are, 1 hour ago, johnieutah said: If I may offer some critique, the UI for this plugin is a bit of dogs dinner Its a backup utility. Pretty much nothing to it other than a ton of settings. Set once and forget about it. That being said, from looking at what my wife buys for dog food, the dogs eat better than I do (Filet Mignon every night). But, yeah HTML is not my forte, and in terms of function vs style, I'd rather have function... 1 hour ago, johnieutah said: simply back-up only the USB boot drive If that's all you want and on a schedule, then use the user.scripts plugin. The script will be: cp /boot/* /mnt/user/DestinationShare -R -v Edited April 19, 2017 by Squid Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.