sdub
-
Posts
88 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by sdub
-
-
No I don’t. I use the database backup functions for things like MariaDB and Postgres and the built-in database backup function for file based databases like Plex. For everything else the risk is realistically minimal. Especially with scheduled backups and a multitude of retained versions.
In the off-chance the daily backup happens to be bad, there are other daily, weekly, monthly Archives I can draw from. Losing a day or two of data isn’t a big concern of mine.
-
19 minutes ago, mkono87 said:
Trying to initilize the local repo, I recieve a bunch of traceback errors, yet I still see files created in the folder.
9b90dcb83a73:/# borg init --encryption=none /mnt/borg-repository/ Local Exception Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5324, in main exit_code = archiver.run(args) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5255, in run return set_ec(func(args)) ^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 183, in wrapper return method(self, args, repository=repository, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 290, in do_init with Cache(repository, key, manifest, warn_if_unencrypted=False): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 387, in __new__ return local() ^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 378, in local return LocalCache(repository=repository, key=key, manifest=manifest, path=path, sync=sync, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 478, in __init__ self.path = cache_dir(repository, path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 229, in cache_dir return path or os.path.join(get_cache_dir(), repository.id_str) ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 100, in get_cache_dir with SaveFile(cache_tag_fn, binary=True) as fd: File "/usr/local/lib/python3.11/site-packages/borg/platform/base.py", line 230, in __enter__ self.tmp_fd, self.tmp_fname = mkstemp_mode(prefix=self.tmp_prefix, suffix='.tmp', dir=self.dir, mode=0o666) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 408, in mkstemp_mode return _mkstemp_inner(dir, prefix, suffix, flags, output_type, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 364, in _mkstemp_inner fd = _os.open(file, flags, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/borg/CACHEDIR.TAG-uryrraek.tmp' Platform: Linux 9b90dcb83a73 6.1.49-Unraid #1 SMP PREEMPT_DYNAMIC Wed Aug 30 09:42:35 PDT 2023 x86_64 Linux: Unknown Linux Borg: 1.2.6 Python: CPython 3.11.5 msgpack: 1.0.5 fuse: llfuse 1.5.0 [pyfuse3,llfuse] PID: 65 CWD: / sys.argv: ['/usr/local/bin/borg', 'init', '--encryption=none', '/mnt/borg-repository/'] SSH_ORIGINAL_COMMAND: NoneLooks like the root issue is in that '/root/.cache/borg' folder. If you open a shell into borg and browse to that folder, does everyhing appear normal? Could be that folder mapping is screwed up in your docker config.
-
On 9/5/2023 at 2:56 PM, Daniel15 said:
Is there a way to have two separate configs? I want to back up into two different repos: One for my SSDs and one for larger data on my hard drives.
I can bind mount multiple things under /mnt/user, but it'd all still go to the same repo.
Edit: I just realised that it can take multiple config files, they just have to have different names. I'll try it out.
you can also specify the config file with the borgmatic -c /path/to/config syntax. I personally have a local.yaml and remote.yaml that both include a common.yaml for common options. I call the local and remote backups through separate lines in my crontab-
1
-
-
Your repo should be “/mnt/borg-repository” not “/mnt/user/disks/easystore264D”. You have mounted “/mnt/user” as a read-only path, so you can’t write to the repo via that path.
-
1 hour ago, Killabee44 said:
Hi all,
Can someone please give me a hand in figuring this error out? I take it to mean that borg can't get a lock? I saw a "read only file system" error so I tried changing the permissions to the share:
chmod -R 775 /mnt/user/disks/easystore264DBut still get the error:
summary: /etc/borgmatic.d/config.yml: Loading configuration file /etc/borgmatic.d/config.yml: An error occurred /mnt/user/disks/easystore264D: Error running actions for repository ... ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5144, in run return set_ec(func(args)) ^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 168, in wrapper with repository: File "/usr/local/lib/python3.11/site-packages/borg/repository.py", line 200, in __enter__ self.open(self.path, bool(self.exclusive), lock_wait=self.lock_wait, lock=self.do_lock) File "/usr/local/lib/python3.11/site-packages/borg/repository.py", line 432, in open self.lock = Lock(os.path.join(path, 'lock'), exclusive, timeout=lock_wait).acquire() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/borg/locking.py", line 384, in acquire self._wait_for_readers_finishing(remove, sleep) File "/usr/local/lib/python3.11/site-packages/borg/locking.py", line 397, in _wait_for_readers_finishing self._lock.acquire() File "/usr/local/lib/python3.11/site-packages/borg/locking.py", line 137, in acquire raise LockFailed(self.path, str(err)) from None borg.locking.LockFailed: Failed to create/acquire the lock /mnt/user/disks/easystore264D/lock.exclusive ([Errno 30] Read-only file system: '/mnt/user/disks/easystore264D/lock.exclusive.ofdect2c.tmp'). Platform: Linux 60166bcbb7d8 6.1.34-Unraid #1 SMP PREEMPT_DYNAMIC Fri Jun 16 11:48:38 PDT 2023 x86_64 Linux: Unknown Linux Borg: 1.2.4 Python: CPython 3.11.2 msgpack: 1.0.5 fuse: llfuse 1.4.4 [pyfuse3,llfuse] PID: 67 CWD: / sys.argv: ['/usr/local/bin/borg', 'create', '--patterns-from', '/tmp/tmpvyvh0ei6', '--exclude-if-present', '.nobackup', '--exclude-if-present', '.NOBACKUP', '--compression', 'none', '--one-file-system', '--read-special', '--files-cache', 'mtime,size', '--list', '--filter', 'AMEx-', '/mnt/user/disks/easystore264D::backup-{now}', '--stats', '--debug', '--show-rc'] SSH_ORIGINAL_COMMAND: None terminating with error status, rc 2 Command 'borg create --patterns-from /tmp/tmpvyvh0ei6 --exclude-if-present .nobackup --exclude-if-present .NOBACKUP --compression none --one-file-system --read-special --files-cache mtime,size --list --filter AMEx- /mnt/user/disks/easystore264D::backup-{now} --stats --debug --show-rc' returned non-zero exit status 2. Need some help? https://torsion.org/borgmatic/#issuesThanks!
It looks to me like maybe you are mounting the share to Borgmatic as read-only… not that it’s read only in the host filesystem. What’s your Borgmatic docker config look like?
-
2 hours ago, Revan335 said:
What's the Different between yours and this Borg Backup?
To reduce data transmission, Borg requires the borg binary to be present on both the local machine and the remote machine when performing remote backups. That way the remote machine can do the work of comparing what's changed from the cache.
If you don't want to (or can't) install borg to the host OS, you can run nold360's borgserver container on the remote machine. This image has just the plain "borg" binary and an SSH server to allow it to act as a borg "target". No borgmatic, but allows you to connect and perform a backup "to" that remotely running docker container.
If you were running a backup scheme where computer A backs up to computer B and vice versa, an ideal scenario would be just having the 2 "borgmatic" docker containers talk to each other. Unfortunately, the official borgmatic container lacks an SSH server so you can't. It's been requested to add SSH server support but they've declined to do so to keep the image simple. This CA application uses the "official" borgmatic container published by the borgmatic team.
The alternative to using the "borgserver" container in Unraid is to use the Nerdpack to install borg directly to the host Unraid OS, since it's already running the requred SSH server.
If you were just using vanilla borg and not borgmatic, you certainly could run the "borgserver" container on both the local and remote systems and need nothing else.
Hope that clarifies.
-
1
-
-
OK, @Greygoose and @stridemat, I updated the CA template to point to a static location for the borgmatic icon. Future pulls should get this, but I'm not sure if it will automatically update for you.
To manually fix, go to advanced view in the Borgmatic docker container config. Change "Icon URL" to
https://raw.githubusercontent.com/Sdub76/unraid_docker_templates/main/images/borgmatic.png
This is a static copy that I have in the CA repo, so it shouldn't change unless github changes it's static URLs.
-
2
-
-
3 hours ago, Johnny4233 said:
hi, thank you for this.
I managed to get this up. Works good for me.
only one thing is left: I can extract the backups to an specific unraid location. but if I try to mount from a repo, the files are accessible in the docker container shell, but I can not see it in unraid file system.
device dev/fuse in docker template is set, and the folder is mounted to the container.
anything I forgot?
regards
Jan
I’m afraid I’ve never gotten this to work in the docker container. I’ve tried passing the fuse device in, but I can’t access the share externally. I may have the permissions wrong or maybe it’s something else.
what I’ve always done is use this image to create the backup image then use the Borgmatic binary on the host system (either Unraid or some other host OS that can access the borg repo on the LAN) to browse and extract the files I need. Borg and llfuse can be installed from the Nerdpack in Unraid.
if someone else has been able to expose a mounted repo from outside the docker container I’d love to hear how.
-
1
-
-
On 5/24/2023 at 3:52 PM, stridemat said:
Im having the same. Any ideas?
Sorry I missed this issue. I’ll take a look and get back to you… for most issues it’s more effective to ask in the Borgmatic GitHub support page.
last year they moved the Borgmatic repo from b3vis’s repo to the main “Borgmatic-collective” repo. I updated the template but maybe something else changed.
that specific error at first glance looks like the link to the icon just changed. Is it causing Borgmatic to not update/work for you?
-
I haven’t updated the CA container in some time just because it hasn’t been necessary. The docker container it pulls from is still very actively developed and widely used, so yes it’s still active. If needed I’ll update the CA template.
-
1
-
-
ps -ax-
1
-
-
Borg creates a lock to make sure it has exclusive access to the backup archive and cache to avoid corruption. When a backup gets interrupted, that lock is sometimes left in place, blocking subsequent backups. Whenever you see an error about not being able to acquire a lock, the first thing you need to do is break the lock if you are sure that no borg process is running against that repo.
From inside the borg container (docker exec -it borgmatic /bin/sh), run following command and try again.
borg break-lock /mnt/borg-repository-
1
-
-
2 hours ago, DeanA said:
I have had Borgmatic running weekly backups for about a year thanks to all of the setup tips in this forum. Over the last couple of weeks I had noticed that my backups have stopped running regulary. When I checked in more detail, seems like this started back in Feb. Also, at about the same time, I have been unable to update Borgmatic version without using "force update". I am a real novice at this stuff, so any suggestions on how to trouble shoot would be most welcome. Thanks in advance.
It's going to be hard to help debug this without some specifics from the error logs. Does it simply not run, or does it occasionally fail? If it fails, how does it fail? If the folks on this thread can't help, maybe a support ticket over at Borgmatic docker github could help. Remember I only maintain the docker template in the CA Appstore. @witten is the actual developer of the docker image.
The automatic docker updates in Unraid are handled by the CA APPLICATION AUTO UPDATE plugin. As long as your repository is set to "ghcr.io/borgmatic-collective/borgmatic" it should work. Note that this location changed somewhat recently, but shouldn't explain why "auto-update" vs "forced update" behaves differently. Maybe consult that support thread for help.
-
1
-
-
1 hour ago, DizkoDan said:
First, thanks for this. I've been using it for a long time, and it's been pretty much rock solid.
Just wanted to make a note here, cuz I hadn't seen it anywhere. They apparently moved the image for this from dockerhub to google a few weeks ago. If you look at your containers in unraid, you'll notice an error for this one not being found. Just update the Repository to the following, and you can grab 1.76.
http://ghcr.io/borgmatic-collective/borgmaticThanks... it looks like they're still pushing the updates to dockerhub (at least for now), but I updated the template as suggested. (I think you meant to say they moved from dockerhub to github)
I'm not sure if template updates affect existing containers. To move existing containers to the new address, open the borgmatic container settings, go to advanced and change the following lines:
Repository: ghcr.io/borgmatic-collective/borgmatic
Docker Hub URL: https://ghcr.io/borgmatic-collective/borgmatic
Icon URL: https://github.com/borgmatic-collective/borgmatic/raw/master/docs/static/borgmatic.png
-
1
-
-
17 minutes ago, Revan335 said:
Its only worked its the SSH Service and Default SSH Port enabled and not working with a different for example the rsync Docker Container?
that won’t work since that container doesn’t have the borg binary.
you could try something like the nold360/borgserver container, which is basically just borg and ssh. Just feed in the path to your borg repo.
-
The Borgmatic image does not have ssh built in, so you’d need install Borgmatic using nerd tools plug-in, then you could use “ssh://user@host/path/to/repo”
no built in gui because the config files are pretty straight forward to configure, and something intended to be set and forget.
-
3 hours ago, Roi said:
I am seeing this for two days now. The container seems to be updates. So this must be a new problem.
borgmatic 1.7.5 apprise 1.2.0 crond: crond (busybox 1.35.0) started, log level 8 crond: USER root pid 14 cmd borgmatic -c /etc/borgmatic.d/config.yaml 2>&1 Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/borg/archiver.py", line 38, in <module> from . import helpers File "/usr/local/lib/python3.10/site-packages/borg/helpers/__init__.py", line 9, in <module> from .checks import * # NOQA File "/usr/local/lib/python3.10/site-packages/borg/helpers/checks.py", line 4, in <module> from .errors import Error File "/usr/local/lib/python3.10/site-packages/borg/helpers/errors.py", line 3, in <module> import borg.crypto.low_level ImportError: Error loading shared library libcrypto.so.1.1: No such file or directory (needed by /usr/local/lib/python3.10/site-packages/borg/crypto/low_level.cpython-310-x86_64-linux-gnu.so) /etc/borgmatic.d/config.yaml: Error getting local Borg version Command '('borg', '--version')' returned non-zero exit status 2. /etc/borgmatic.d/config.yaml: Error running configuration file summary: /etc/borgmatic.d/config.yaml: Error running configuration file /etc/borgmatic.d/config.yaml: Error getting local Borg version Command '('borg', '--version')' returned non-zero exit status 2. Need some help? https://torsion.org/borgmatic/#issues
Did you read the last few posts before posting?
-
45 minutes ago, SirBilliamson said:
It appears to be an issue with the latest docker image. Temporary fix can be found here - https://github.com/borgmatic-collective/docker-borgmatic/issues/184#issuecomment-1333599796
Yes I’m seeing it now too. Best thing to do in this case is to specify a specific older version until the container is fixed. In the docker config change “Repository:” to read “b3vis/borgmatic:1.7.5”. This will pin it to that specific version. Once it’s fixed change it back to :latest or just leave that part off.
In this specific instance the workaround described at the link seems easy enough but isn’t persistent if the docker container updates itself. Assuming the fix is in the next docker image, it doesn’t really matter.
-
1
-
-
I think that's
On 11/14/2022 at 2:49 AM, stridemat said:
Is the above post from sdub the accepted way of making Unraid the target for backups without altering the base Unraid install?I think the options are:
1. Install a docker container with sshd and borgbackup, like borgserver (here's one that appears to be more frequently updated)
2. Install borgbackup on just about any Linux or FreeBSD VM under Unraid
3. Install the borgbackup binary somewhere on your host system, either manually in /boot/extra or using NerdTools in 6.11+.
Either way, you'll need the associated SSH daemon exposed on a port that's accessible via LAN or VPN if you don't want it open to the internet, which I would never recommend.
I'm not sure any particular method is preferred, though I suppose options 1 or 2 could be marginally more secure and keep your Unraid install closer to "stock" if you're concerned about that. I personally have borgbackup installed through NerdTools
-
15 minutes ago, Howboys said:
Do i need the `/mnt/fuse/:mnt/borg` mount for the container? Fix Common Problems says borg should not exist in /mnt.
These are the mounts I have:
no it’s not necessary. The idea was to have a mount point for mounting archives somewhere you could get to them outside of the container and /mnt seems like the proper place. Either ignore the error, delete the mount point, or pick another mount point that fox common problems doesn’t complain about.
-
1
-
-
Thanks for this… I updated the first post to remove the path statement from the suggested crontab config.
Underlines the importance of monitoring your backups with something like Healthchecks (can be self hosted) so you know if something breaks ASAP!
-
1
-
-
Is there a workflow to request packages be added? I'd love to get "bat" added, unless there's a better alternative already in there for colorizing logs.
https://github.com/sharkdp/bat
Thanks
-
1
-
-
2 hours ago, Hammy Havoc said:
Does it still make sense to use the Unassigned Devices plugin versus creating a pool specifically for Borg now that we can have multiple pools?
3-2-1 backup means 3 copies, 2 types of media, at least one offsite.
When I wrote that my reasoning was just to get as much diversity as possible between your data and the backup in lieu of 2 types of media. Preferably it would be on a different system with a different Os, filesystem, etc. There’s no “right” or “wrong”, just better or worse.
-
1
-
-
The first one is very slow. Not sure there’sa lot you can do about it. I did my initial remote backup locally then copied it to the remote sever in person. Aka sneaker-net.
if you felt like borg network performance was really the issue you could create the repo locally like I did then use a lightweight protocol like rsync to copy the initial repo across. When you change how you access the repo it will warn you once but allow it.
[Support] borgmatic
in Docker Containers
Posted
I am not at my server right now but I can say that my backups are still working and I haven’t touched the config in months.
the Unicode error makes me wonder… did you possibly edit your config file in windows such that it has a Windows text file format instead of UNIX?