[Support] borgmatic


sdub

Recommended Posts

9 hours ago, sdub said:

It looks to me like maybe you are mounting the share to Borgmatic as read-only… not that it’s read only in the host filesystem. What’s your Borgmatic docker config look like?

Here it is:

 

borg_config.PNG.7961ea3d687fd623dea472cf58339995.PNG

 

My config:

location:
    source_directories:
        - /boot
     #   - /mnt/user/appdata
        - /mnt/user/bootbackup
        - /mnt/user/CommunityApplicationsAppdataBackup
        - /mnt/user/nextcloud
    #    - /mnt/user/downloads
    repositories:
        - /mnt/user/disks/easystore264D
     #   - remote.mydomain.net:/mnt/disks/borg_remote/repo
    one_file_system: true
    files_cache: mtime,size
    patterns:
        - '- [Tt]rash'
        - '- [Cc]ache'
    exclude_if_present:
        - .nobackup
        - .NOBACKUP

storage:
    encryption_passphrase: "passphrase"
    compression: none
    ssh_command: ssh -i /root/.ssh/id_rsa
    archive_name_format: 'backup-{now}'

retention:
    keep_hourly: 2
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 12
    keep_yearly: 10
    prefix: 'backup-'

consistency:
    checks:
        - repository
        - archives
    prefix: 'backup-'

hooks:
    before_backup:
        - echo "Starting a backup."
    after_backup:
        - echo "Finished a backup."
    on_error:
        - echo "Error during prune/create/check."
    mysql_databases:
        - name: all
          hostname: 192.168.1.29
          password: "password"
    

 

Edited by Killabee44
Link to comment
  • 2 weeks later...
On 7/9/2023 at 12:31 PM, Killabee44 said:

Here it is:

 

borg_config.PNG.7961ea3d687fd623dea472cf58339995.PNG

 

My config:

location:
    source_directories:
        - /boot
     #   - /mnt/user/appdata
        - /mnt/user/bootbackup
        - /mnt/user/CommunityApplicationsAppdataBackup
        - /mnt/user/nextcloud
    #    - /mnt/user/downloads
    repositories:
        - /mnt/user/disks/easystore264D
     #   - remote.mydomain.net:/mnt/disks/borg_remote/repo
    one_file_system: true
    files_cache: mtime,size
    patterns:
        - '- [Tt]rash'
        - '- [Cc]ache'
    exclude_if_present:
        - .nobackup
        - .NOBACKUP

storage:
    encryption_passphrase: "passphrase"
    compression: none
    ssh_command: ssh -i /root/.ssh/id_rsa
    archive_name_format: 'backup-{now}'

retention:
    keep_hourly: 2
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 12
    keep_yearly: 10
    prefix: 'backup-'

consistency:
    checks:
        - repository
        - archives
    prefix: 'backup-'

hooks:
    before_backup:
        - echo "Starting a backup."
    after_backup:
        - echo "Finished a backup."
    on_error:
        - echo "Error during prune/create/check."
    mysql_databases:
        - name: all
          hostname: 192.168.1.29
          password: "password"
    

 

Anyone?

Link to comment

Your repo should be “/mnt/borg-repository” not “/mnt/user/disks/easystore264D”. You have mounted “/mnt/user” as a read-only path, so you can’t write to the repo via that path. 

Edited by sdub
Link to comment
  • 1 month later...

Is there a way to have two separate configs? I want to back up into two different repos: One for my SSDs and one for larger data on my hard drives.

 

I can bind mount multiple things under /mnt/user, but it'd all still go to the same repo.

 

Edit: I just realised that it can take multiple config files, they just have to have different names. I'll try it out.

Edited by Daniel15
  • Like 1
Link to comment
On 9/5/2023 at 2:56 PM, Daniel15 said:

Is there a way to have two separate configs? I want to back up into two different repos: One for my SSDs and one for larger data on my hard drives.

 

I can bind mount multiple things under /mnt/user, but it'd all still go to the same repo.

 

Edit: I just realised that it can take multiple config files, they just have to have different names. I'll try it out.


you can also specify the config file with the borgmatic -c /path/to/config syntax. I personally have a local.yaml and remote.yaml that both include a common.yaml for common options. I call the local and remote backups through separate lines in my crontab  

  • Thanks 1
Link to comment
  • 4 weeks later...

Trying to initilize the local repo, I recieve a bunch of traceback errors, yet I still see files created in the folder.

9b90dcb83a73:/# borg init --encryption=none /mnt/borg-repository/
Local Exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5324, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5255, in run
    return set_ec(func(args))
                  ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 290, in do_init
    with Cache(repository, key, manifest, warn_if_unencrypted=False):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 387, in __new__
    return local()
           ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 378, in local
    return LocalCache(repository=repository, key=key, manifest=manifest, path=path, sync=sync,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 478, in __init__
    self.path = cache_dir(repository, path)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 229, in cache_dir
    return path or os.path.join(get_cache_dir(), repository.id_str)
                                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 100, in get_cache_dir
    with SaveFile(cache_tag_fn, binary=True) as fd:
  File "/usr/local/lib/python3.11/site-packages/borg/platform/base.py", line 230, in __enter__
    self.tmp_fd, self.tmp_fname = mkstemp_mode(prefix=self.tmp_prefix, suffix='.tmp', dir=self.dir, mode=0o666)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 408, in mkstemp_mode
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type, mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 364, in _mkstemp_inner
    fd = _os.open(file, flags, mode)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/borg/CACHEDIR.TAG-uryrraek.tmp'

Platform: Linux 9b90dcb83a73 6.1.49-Unraid #1 SMP PREEMPT_DYNAMIC Wed Aug 30 09:42:35 PDT 2023 x86_64
Linux: Unknown Linux
Borg: 1.2.6  Python: CPython 3.11.5 msgpack: 1.0.5 fuse: llfuse 1.5.0 [pyfuse3,llfuse]
PID: 65  CWD: /
sys.argv: ['/usr/local/bin/borg', 'init', '--encryption=none', '/mnt/borg-repository/']
SSH_ORIGINAL_COMMAND: None

 

Link to comment
19 minutes ago, mkono87 said:

Trying to initilize the local repo, I recieve a bunch of traceback errors, yet I still see files created in the folder.

9b90dcb83a73:/# borg init --encryption=none /mnt/borg-repository/
Local Exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5324, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5255, in run
    return set_ec(func(args))
                  ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 290, in do_init
    with Cache(repository, key, manifest, warn_if_unencrypted=False):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 387, in __new__
    return local()
           ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 378, in local
    return LocalCache(repository=repository, key=key, manifest=manifest, path=path, sync=sync,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 478, in __init__
    self.path = cache_dir(repository, path)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 229, in cache_dir
    return path or os.path.join(get_cache_dir(), repository.id_str)
                                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 100, in get_cache_dir
    with SaveFile(cache_tag_fn, binary=True) as fd:
  File "/usr/local/lib/python3.11/site-packages/borg/platform/base.py", line 230, in __enter__
    self.tmp_fd, self.tmp_fname = mkstemp_mode(prefix=self.tmp_prefix, suffix='.tmp', dir=self.dir, mode=0o666)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 408, in mkstemp_mode
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type, mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/helpers/fs.py", line 364, in _mkstemp_inner
    fd = _os.open(file, flags, mode)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/borg/CACHEDIR.TAG-uryrraek.tmp'

Platform: Linux 9b90dcb83a73 6.1.49-Unraid #1 SMP PREEMPT_DYNAMIC Wed Aug 30 09:42:35 PDT 2023 x86_64
Linux: Unknown Linux
Borg: 1.2.6  Python: CPython 3.11.5 msgpack: 1.0.5 fuse: llfuse 1.5.0 [pyfuse3,llfuse]
PID: 65  CWD: /
sys.argv: ['/usr/local/bin/borg', 'init', '--encryption=none', '/mnt/borg-repository/']
SSH_ORIGINAL_COMMAND: None

 

Looks like the root issue is in that '/root/.cache/borg' folder.  If you open a shell into borg and browse to that folder, does everyhing appear normal?  Could be that folder mapping is screwed up in your docker config.

Link to comment
54 minutes ago, sdub said:

Looks like the root issue is in that '/root/.cache/borg' folder.  If you open a shell into borg and browse to that folder, does everyhing appear normal?  Could be that folder mapping is screwed up in your docker config.

image.thumb.png.276c1a90fedea810abdf8ab66baa9964.png

 

It mapped in a subfolder of the repo folder. Is that fine? The cache folder on the host wasnt there so I created it but now when I try to init it says there is something already there.

Link to comment

No I don’t.  I use the database backup functions for things like MariaDB and Postgres and the built-in database backup function for file based databases like Plex. For everything else the risk is realistically minimal. Especially with scheduled backups and a multitude of retained versions.

 

In the off-chance the daily backup happens to be bad, there are other daily, weekly, monthly Archives I can draw from. Losing a day or two of data isn’t a big concern of mine.  

Link to comment
  • 1 month later...

I'm trying to use Borg to backup from unRAID to a Synology DS414. I attached the DS414 as an SMB share using Unassigned Devices. (Had some problems, but don't think they're relevant to this issue.) When I try to create the repo using

borg init --encryption=none /mnt/borg-repository

I get an error message:

/ # borg init --encryption=none /mnt/borg-repository
Local Exception
OSError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5324, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 5255, in run
    return set_ec(func(args))
                  ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/archiver.py", line 290, in do_init
    with Cache(repository, key, manifest, warn_if_unencrypted=False):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 387, in __new__
    return local()
           ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 378, in local
    return LocalCache(repository=repository, key=key, manifest=manifest, path=path, sync=sync,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 485, in __init__
    self.create()
  File "/usr/local/lib/python3.11/site-packages/borg/cache.py", line 513, in create
    with open(os.path.join(self.path, 'README'), 'w') as fd:
OSError: [Errno 28] No space left on device

 

From googling around, it seems like the "No space" error might be caused by the Borg cache being full. I had left it at the default of /mnt/disks/borg_backup/cache, and there's plenty of space on the array.

 

If I run df -h I get this:

/ # df -h
Filesystem                Size      Used Available Use% Mounted on
....
tmpfs                     1.0M      1.0M         0 100% /root/.cache/borg
shfs                    931.1G     48.4G    882.6G   5% /root/.config/borg
....

 

What can I do to fix this?

Link to comment
  • 1 month later...
Quote

Files cache set to use "mtime,size" - Very important as unRAID does not have persistent inode values

 

Is this still best practice? I haven't found any explicit documentation about how unRAID uses inodes in the manual. Inodes in general seem to be something that confuses a lot of people; is it dependent on the filesystem you use or are all filesystems in unRAID implemented without persistent inode values? Or is it because of the flexibility of unRAID's array system?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.