Best external offline backup filesystem (checksum/bitrot)

[bsim]

I've read extensively on the forums and on google, but haven't found someone with my exact, relatively simple interests...

 

I have multiple large external hard drives (22TBx) and I have extensive scripts (using rsync) that do automatic backups when powered on (unassigned devices script) every month or so.

 

My question is, what drive format should I use for the external drives?

 

1. ***I would love to turn my current backup (based-on-changed-files) to verify integrity of backed up files based on FS checksums that would refresh files if bitrot occurs.
2. I'm not extremely worried about accessing the drives from outside machines (windows/ntfs), and definitely don't have any computational limitations.

 

I'm thinking BTRFS and ZFS would be my best bet, but:

 

1. Can rsync backups use the FS checksumming built into these file systems to determine differences? (better option than rsync?)
2. Does FS checksumming in BTRFS/ZFS happen automatically in the background without complications (outside of the unassigned devices gui)?
3. Is scriptable command line FS checking easy for BTRFS/ZFS? (currently using xfs_repair for XFS)

 

 

 

[JorgeB]

Btrfs and zfs create checksums based on blocks, not files, to verify them you need to run a scrub, cannot use an external utility

[foo_fighter]

16 hours ago, JorgeB said:

Btrfs and zfs create checksums based on blocks, not files, to verify them you need to run a scrub, cannot use an external utility

You could run file integrity / bunker on top of that and check the Xattr or something like snap raid. 

Rsync has its own -checksum switch but it slows down the process dramatically. 

If backing up ZFS to ZFS, you should investigate syncoid(part of sanoid) or Znapzend.

 

Here is a similar thread: 

 

Edited November 30, 2023 by foo_fighter