RinxKninks Posted December 30, 2022 Share Posted December 30, 2022 (edited) 1 hour ago, Solverz said: Please just say what you exactly want to do without using prolonged sentences and fancy words. Your post is like a riddle that we'd need to break down to understand what you are talking about. Plain & simple: I started using borgmatic writing to rotating removable hard drives, each containing another repositorie. With the same "run" of borgmatic I'd like to copy data already being deduplicated etc. to this target disks, that I'd like not to process by the borgmatic backup process again. So I think about rsnapshot. But the borgmatic docker doesn't have access to the rsnapshot bin on the unraid system - which I understand, nor can I just copy and use rsnapshot into the borgmatic docker. So do you have an idea on how to use rsnapshot as a after_backup hook in the borgmatic/config? Edited December 30, 2022 by RinxKninks Quote Link to comment
Solverz Posted December 30, 2022 Share Posted December 30, 2022 (edited) 13 minutes ago, RinxKninks said: Plain & simple: I started using borgmatic writing to rotating removable hard drives, each containing another repositorie. With the same "run" of borgmatic I'd like to copy data already being deduplicated etc. to this target disks, that I'd like not to process by the borgmatic backup process again. So I think about rsnapshot. But the borgmatic docker doesn't have access to the rsnapshot bin on the unraid system - which I understand, nor can I just copy and use rsnapshot into the borgmatic docker. So do you have an idea on how to use rsnapshot as a after_backup hook in the borgmatic/config? So you have two repos, each on a separate external hard drive? Got that. Not sure what you mean on the rest? Quote copy data already being deduplicated etc. What do you mean here? Edited December 30, 2022 by Solverz Quote Link to comment
Solverz Posted December 30, 2022 Share Posted December 30, 2022 1 hour ago, RinxKninks said: Plain & simple: I started using borgmatic writing to rotating removable hard drives, each containing another repositorie. With the same "run" of borgmatic I'd like to copy data already being deduplicated etc. to this target disks, that I'd like not to process by the borgmatic backup process again. So I think about rsnapshot. But the borgmatic docker doesn't have access to the rsnapshot bin on the unraid system - which I understand, nor can I just copy and use rsnapshot into the borgmatic docker. So do you have an idea on how to use rsnapshot as a after_backup hook in the borgmatic/config? Borgbackup, backs up to a repo. You can choose what you want to be backed up to this repo. So what what do you need to use rsnapshot for that borgbackup is not doing already. Quote Link to comment
RinxKninks Posted December 30, 2022 Share Posted December 30, 2022 2 hours ago, Solverz said: Borgbackup, backs up to a repo. You can choose what you want to be backed up to this repo. So what what do you need to use rsnapshot for that borgbackup is not doing already. OK - got it. Quote Link to comment
Revan335 Posted January 1, 2023 Share Posted January 1, 2023 (edited) What's the URL for a Client (Vorta for example) Connection to the Repository? No GUI or Webui? Edited January 1, 2023 by Revan335 Quote Link to comment
sdub Posted January 1, 2023 Author Share Posted January 1, 2023 (edited) The Borgmatic image does not have ssh built in, so you’d need install Borgmatic using nerd tools plug-in, then you could use “ssh://user@host/path/to/repo” no built in gui because the config files are pretty straight forward to configure, and something intended to be set and forget. Edited January 1, 2023 by sdub Quote Link to comment
Greygoose Posted January 1, 2023 Share Posted January 1, 2023 No gui needed, as sdub said once the configs are setup its leave alone and let it do its thing. The most i need to do is break-lock if i restart or stop the docker during the backup process. 1 Quote Link to comment
Revan335 Posted January 2, 2023 Share Posted January 2, 2023 Its only worked its the SSH Service and Default SSH Port enabled and not working with a different for example the rsync Docker Container? Quote Link to comment
sdub Posted January 2, 2023 Author Share Posted January 2, 2023 17 minutes ago, Revan335 said: Its only worked its the SSH Service and Default SSH Port enabled and not working with a different for example the rsync Docker Container? that won’t work since that container doesn’t have the borg binary. you could try something like the nold360/borgserver container, which is basically just borg and ssh. Just feed in the path to your borg repo. Quote Link to comment
Revan335 Posted January 2, 2023 Share Posted January 2, 2023 8 hours ago, sdub said: that won’t work since that container doesn’t have the borg binary. you could try something like the nold360/borgserver container, which is basically just borg and ssh. Just feed in the path to your borg repo. Many Thanks! I tried! The Vorta Docker doesn't work to, correct? Quote Link to comment
Revan335 Posted January 2, 2023 Share Posted January 2, 2023 13 hours ago, sdub said: you could try something like the nold360/borgserver container, which is basically just borg and ssh. Just feed in the path to your borg repo. OK, new Message. Error: Repository path not allowed. ssh://borg@Unraidname:Port/Borg/backup/Clientname The Permissions of 1000:1000 are correct. ~# ls -la /mnt/user/Borg/ total 0 drwxrwxrwx 1 nobody users 26 Jan 2 13:59 ./ drwxrwxrwx 1 nobody users 172 Jan 2 14:47 ../ drwxr-xr-x 1 1000 1000 10 Jan 2 14:52 backup/ drwxr-xr-x 1 1000 1000 22 Jan 2 14:46 sshkeys/ Quote Link to comment
Revan335 Posted January 4, 2023 Share Posted January 4, 2023 Anybody a Idea why the Repository path not allowed? Quote Link to comment
Revan335 Posted January 8, 2023 Share Posted January 8, 2023 Found the crucial hint in the Ubuntuusers Wiki. // is important for the path. Either this is not in the docs of Vorta/Pika or Borg sites or I must have over seen it despite multiple checks. ssh://borg@Unraidname:Port//backup/Clientname Test it now further. Quote Link to comment
nicjames2378 Posted January 8, 2023 Share Posted January 8, 2023 (edited) FIXED: This issue was caused by the mounted /root/.ssh path on the borgmatic end being protected. After making a new path within appdata and copying the necessary files from UnRaid's /root/.ssh, everything works without unexpected errors. I've left the below errors in case it helps someone else in the future. I've got this working using b3vis/borgmatic:latest-msmtp on the client being backed up, and using nold360/borgserver:1.2.2 on the remote server. The only issue I have encountered (once it was all setup correctly, that is) is that each emailed report from the msmtp comes through as "Failed" due to an error. I know this isn't meant to be a borg support forum, but I've only encountered this while using this container so perhaps someone here knows what's going on? I assumed that since the issues seems to be due to "Operation not permitted", it was a permissions problem. However, I can copy the file via command line when I docker exec -it into the local bormatic container with no problems, so that does not seem to be the case? Remote: hostfile_replace_entries: link /root/.ssh/known_hosts to /root/.ssh/known_hosts.old: Operation not permitted Remote: update_known_hosts: hostfile_replace_entries failed for /root/.ssh/known_hosts: Operation not permitted ssh://[email protected]:PORT/backup/SERVERNAME: Error running actions for repository Command 'borg check --glob-archives backup-* ssh://[email protected]:PORT/backup/SERVERNAME' returned non-zero exit status 2. Error during prune/create/check. /etc/borgmatic.d/config.yaml: Error running configuration file summary: /etc/borgmatic.d/config.yaml: Error running configuration file ssh://[email protected]:PORT/backup/SERVERNAME: Error running actions for repository Remote: hostfile_replace_entries: link /root/.ssh/known_hosts to /root/.ssh/known_hosts.old: Operation not permitted Remote: update_known_hosts: hostfile_replace_entries failed for /root/.ssh/known_hosts: Operation not permitted Traceback (most recent call last): File "/usr/lib/python3/dist-packages/borg/remote.py", line 240, in serve res = f(**args) File "/usr/lib/python3/dist-packages/borg/repository.py", line 1039, in check objects = list(self.io.iter_objects(segment)) File "/usr/lib/python3/dist-packages/borg/repository.py", line 1512, in iter_objects size, tag, key, data = self._read(fd, self.header_fmt, header, segment, offset, File "/usr/lib/python3/dist-packages/borg/repository.py", line 1606, in _read data = fd.read(length) OSError: [Errno 5] Input/output error Borg server: Platform: Linux 1e2c94ae5f5a 5.19.17-Unraid #2 SMP PREEMPT_DYNAMIC Wed Nov 2 11:54:15 PDT 2022 x86_64 Borg server: Linux: Unknown Linux Borg server: Borg: 1.2.3 Python: CPython 3.10.9 msgpack: 1.0.3 fuse: None [pyfuse3,llfuse] Borg server: PID: 343 CWD: /backup/SERVERNAME Borg server: sys.argv: ['/usr/bin/borg', 'serve', '--restrict-to-path', '/backup/SERVERNAME'] Borg server: SSH_ORIGINAL_COMMAND: 'borg serve' Platform: Linux 990a970c11d1 5.15.46-Unraid #1 SMP Fri Jun 10 11:08:41 PDT 2022 x86_64 Linux: Unknown Linux Borg: 1.2.2 Python: CPython 3.10.8 msgpack: 1.0.4 fuse: llfuse 1.4.2 [pyfuse3,llfuse] PID: 108 CWD: / sys.argv: ['/usr/local/bin/borg', 'check', '--glob-archives', 'backup-*', 'ssh://[email protected]:PORT/backup/SERVERNAME'] SSH_ORIGINAL_COMMAND: None Command 'borg check --glob-archives backup-* ssh://[email protected]:PORT/backup/SERVERNAME' returned non-zero exit status 2. Edited January 20, 2023 by nicjames2378 Found solution to my problem. Quote Link to comment
DizkoDan Posted February 17, 2023 Share Posted February 17, 2023 First, thanks for this. I've been using it for a long time, and it's been pretty much rock solid. Just wanted to make a note here, cuz I hadn't seen it anywhere. They apparently moved the image for this from dockerhub to google a few weeks ago. If you look at your containers in unraid, you'll notice an error for this one not being found. Just update the Repository to the following, and you can grab 1.76. http://ghcr.io/borgmatic-collective/borgmatic Quote Link to comment
sdub Posted February 17, 2023 Author Share Posted February 17, 2023 (edited) 1 hour ago, DizkoDan said: First, thanks for this. I've been using it for a long time, and it's been pretty much rock solid. Just wanted to make a note here, cuz I hadn't seen it anywhere. They apparently moved the image for this from dockerhub to google a few weeks ago. If you look at your containers in unraid, you'll notice an error for this one not being found. Just update the Repository to the following, and you can grab 1.76. http://ghcr.io/borgmatic-collective/borgmatic Thanks... it looks like they're still pushing the updates to dockerhub (at least for now), but I updated the template as suggested. (I think you meant to say they moved from dockerhub to github) I'm not sure if template updates affect existing containers. To move existing containers to the new address, open the borgmatic container settings, go to advanced and change the following lines: Repository: ghcr.io/borgmatic-collective/borgmatic Docker Hub URL: https://ghcr.io/borgmatic-collective/borgmatic Icon URL: https://github.com/borgmatic-collective/borgmatic/raw/master/docs/static/borgmatic.png Edited February 17, 2023 by sdub 1 Quote Link to comment
DizkoDan Posted February 17, 2023 Share Posted February 17, 2023 Correct, I mean github, used to dealing with google repo, so missed the h there. I most definitely was not getting the updated version. Unraid changed to showing repo not found or something similar at some point in the past couple of weeks. I checked the log output of the previous run, and it was still showing borgmatic 1.7.5 for the version. 1 Quote Link to comment
kilonde Posted February 25, 2023 Share Posted February 25, 2023 I am confused how to update my repo path in the configuration to avoid this error: Remote repository paths without ssh:// syntax are deprecated. My repo configuration is like this: [email protected]:/mnt/disk/borg_backup/repo and the ssh command looks like this: ssh_command: ssh -i /root/.ssh/Pi Also, is it advised to keep the same version of borg on client and the server? Quote Link to comment
witten Posted March 9, 2023 Share Posted March 9, 2023 (edited) On 2/25/2023 at 2:20 PM, kilonde said: I am confused how to update my repo path in the configuration to avoid this error: Remote repository paths without ssh:// syntax are deprecated. My repo configuration is like this: [email protected]:/mnt/disk/borg_backup/repo and the ssh command looks like this: ssh_command: ssh -i /root/.ssh/Pi "[email protected]:/mnt/disk/borg_backup/repo" would be rewritten as "ssh://[email protected]/mnt/disk/borg_backup/repo". Note both the "ssh://" and the removed ":" after the IP. Newer versions of borgmatic will show you this as part of the deprecation warning: test.yaml: Remote repository paths without ssh:// syntax are deprecated. Interpreting "[email protected]:/mnt/disk/borg_backup/repo" as "ssh://[email protected]/mnt/disk/borg_backup/repo" On 2/25/2023 at 2:20 PM, kilonde said: Also, is it advised to keep the same version of borg on client and the server? It doesn't have to be exactly the same, but it's good to stay within the same major version if you can. (E.g. 1.x and 1.x.) There are some exceptions to that, like there were some major changes in Borg 1.2. Edited March 9, 2023 by witten 1 Quote Link to comment
kilonde Posted March 25, 2023 Share Posted March 25, 2023 On 3/9/2023 at 1:30 AM, witten said: "[email protected]:/mnt/disk/borg_backup/repo" would be rewritten as "ssh://[email protected]/mnt/disk/borg_backup/repo". Note both the "ssh://" and the removed ":" after the IP. Newer versions of borgmatic will show you this as part of the deprecation warning: test.yaml: Remote repository paths without ssh:// syntax are deprecated. Interpreting "[email protected]:/mnt/disk/borg_backup/repo" as "ssh://[email protected]/mnt/disk/borg_backup/repo" It doesn't have to be exactly the same, but it's good to stay within the same major version if you can. (E.g. 1.x and 1.x.) There are some exceptions to that, like there were some major changes in Borg 1.2. Thanks for your help, this has worked! 1 Quote Link to comment
bastl Posted April 12, 2023 Share Posted April 12, 2023 First of all, thanks for this container. I have a question @sdub In your second post of this thread you showed your config as an example which I used to test it on my system. I have an issue with it. As long as I keep the prefix line in the retention block retention: keep_hourly: 2 keep_daily: 7 keep_weekly: 4 keep_monthly: 12 keep_yearly: 10 prefix: 'backup-' I get the following error during sheduled backups: /mnt/borg-repository: Error running actions for repository Command 'borg prune --keep-hourly 2 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --glob-archives backup-* --glob-archives backup-* --list /mnt/borg-repository' returned non-zero exit status 2. Error during prune/create/check. /etc/borgmatic.d/config.yaml: An error occurred borg prune: error: argument -a/--glob-archives: There can be only one. Command 'borg prune --keep-hourly 2 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --glob-archives backup-* --glob-archives backup-* --list /mnt/borg-repository' returned non-zero exit status 2. The argument "--glob-archives backup-*" is added 2 times somehow. Why do you add it at the retention section in your yaml file? Is it really needed or is this line save to remove? Quote Link to comment
Gaurhoth Posted April 12, 2023 Share Posted April 12, 2023 8 hours ago, bastl said: The argument "--glob-archives backup-*" is added 2 times somehow. A recent update changed the way prefix option works. Generally, you can replace multiple prefixes (for example if you had same "prefix:" under retention and consistency) with archive_name_format under storage. Example: [old way] storage: archive_name_format: 'homepage-{now}' retention: prefix: 'homepage-' consistency: prefix: 'homepage-' Can be replaced with: [new way] storage: archive_name_format: 'homepage-{now}' If you need different 'prefix' values for some reason you can look at the match_archives option under storage. This change is documented at https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#code-35 1 Quote Link to comment
witten Posted April 14, 2023 Share Posted April 14, 2023 While Gaurhoth is correct that you can update your configuration to drop the deprecated "prefix" option with recent versions of borgmatic, the fact that "prefix" started breaking as described above was actually a bug. It has been fixed however in borgmatic 1.7.12. Quote Link to comment
darkside40 Posted April 15, 2023 Share Posted April 15, 2023 Was something recently changed in this Image? Using it for over a year but in the past couple of weeks it seems that the cron runs quite random. Normally it should rum at 18:00 every day, but now it sometimes simply runs the script when the server comes from sleep etc. Anyone also experienced this? Quote Link to comment
witten Posted April 15, 2023 Share Posted April 15, 2023 3 hours ago, darkside40 said: Was something recently changed in this Image? Using it for over a year but in the past couple of weeks it seems that the cron runs quite random. Normally it should rum at 18:00 every day, but now it sometimes simply runs the script when the server comes from sleep etc. Anyone also experienced this? The docker-borgmatic image this is based on switched to supercronic a while back. But presumably if that introduced the changes you're seeing, you would've seen it well before now. You might consider filing a ticket describing your problem at: https://github.com/borgmatic-collective/docker-borgmatic Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.