bsim Posted November 29, 2023 Share Posted November 29, 2023 I've read extensively on the forums and on google, but haven't found someone with my exact, relatively simple interests... I have multiple large external hard drives (22TBx) and I have extensive scripts (using rsync) that do automatic backups when powered on (unassigned devices script) every month or so. My question is, what drive format should I use for the external drives? ***I would love to turn my current backup (based-on-changed-files) to verify integrity of backed up files based on FS checksums that would refresh files if bitrot occurs. I'm not extremely worried about accessing the drives from outside machines (windows/ntfs), and definitely don't have any computational limitations. I'm thinking BTRFS and ZFS would be my best bet, but: Can rsync backups use the FS checksumming built into these file systems to determine differences? (better option than rsync?) Does FS checksumming in BTRFS/ZFS happen automatically in the background without complications (outside of the unassigned devices gui)? Is scriptable command line FS checking easy for BTRFS/ZFS? (currently using xfs_repair for XFS) Quote Link to comment
JorgeB Posted November 29, 2023 Share Posted November 29, 2023 Btrfs and zfs create checksums based on blocks, not files, to verify them you need to run a scrub, cannot use an external utility Quote Link to comment
foo_fighter Posted November 30, 2023 Share Posted November 30, 2023 (edited) 16 hours ago, JorgeB said: Btrfs and zfs create checksums based on blocks, not files, to verify them you need to run a scrub, cannot use an external utility You could run file integrity / bunker on top of that and check the Xattr or something like snap raid. Rsync has its own -checksum switch but it slows down the process dramatically. If backing up ZFS to ZFS, you should investigate syncoid(part of sanoid) or Znapzend. Here is a similar thread: Edited November 30, 2023 by foo_fighter Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.