File integrity check with parity for data drives and vice versa

[Duckers]

I want to request a feature to do a file integrity check via parity to make sure individual files won't go corrupt to ensure parity not only holds the math to rebuild mechanical/hardware failure, but also acts as a "make sure individual files are fine too". One of my image zips of 116GB i now have to delete because it's completely corrupted and can't be open.

[itimpi]

Parity has no concept of files so cannot do what you are asking for.

 

if you want to be informed if any files on the array disks are corrupt then you need to have them formatted as BTRFS.

[Jaybau]

Ideally I would like the system to check file integrity (checksum), and if invalid, restore the file(s) from parity (or something else better?).

 

Perhaps Unraid cannot do this because parity is not done at the file level (done at the drive level).

 

I use BTRFS via Unraid, but I don't think Unraid uses BTRFS for file restores.  Perhaps future Unraid has decided to use ZFS instead of BTRFS.

 

 

 

[JorgeB]

9 hours ago, Jaybau said:

I use BTRFS via Unraid, but I don't think Unraid uses BTRFS for file restores.  Perhaps future Unraid has decided to use ZFS instead of BTRFS.

btrfs and zfs are the same for this, they can repair data for redundant pools, not for array devices, and zfs will be available on v6.12.

[spamalam]

How does parity get 'corrupted' too when there's bitrot  out of interest?  I'd have expected when a bit flips the parity check would not know what happened but notice something is wrong.

Edited December 21, 2022 by spamalam

[JorgeB]

21 minutes ago, spamalam said:

I'd have expected when a bit flips the parity check would not know what happened but notice something is wrong.

That would be detected as one or more parity sync errors, but no way of knowing which disk is the problem.

[spamalam]

On 12/21/2022 at 1:43 PM, JorgeB said:

That would be detected as one or more parity sync errors, but no way of knowing which disk is the problem.

Ah great, so in theory i can trust my parity sync to say somethings up but we still need something to checksum the files.

 

I guess hang tight until zfs arrives?

[JorgeB]

Correct, you could also use btrfs but since zfs will be available soon might as well wait a little longer.