BTRFS on UNRAID checksum on read when transfering data to Windows PC

[Hellomynameisleo]

If I was to use BTRFS on UNRAID and mount the usershare to my Windows PC as a drive letter, when using the Windows PC to copy any file/folder from the UNRAID mounted usershare to the Windows PC using a simple copy and paste or robocopy mirror command does UNRAID use BTRFS checksum verification to read the source file before copying over?

[JorgeB]

Yes.

[Hellomynameisleo]

18 minutes ago, JorgeB said:

Yes.

Thanks, will it also give an error and abord automatically if detected so it doesn't overide the good copy with the corrupted file?

[JorgeB]

Correct, though sometimes Samba still allows the copy to go through, less likely to happen with the current release.

[Hellomynameisleo]

9 hours ago, JorgeB said:

Correct, though sometimes Samba still allows the copy to go through, less likely to happen with the current release.

That doesn't sound good, is there a solution around this as I don't want my data to be sometimes corrupted. I know there's a data integrity plugin but have heard mostly bad things about it being very unstable/unreliable.

[primeval_god]

10 hours ago, JorgeB said:

Correct, though sometimes Samba still allows the copy to go through, less likely to happen with the current release.

Has there been any change since the last time this issue was updated?

 

[JorgeB]

30 minutes ago, Hellomynameisleo said:

That doesn't sound good, is there a solution around this as I don't want my data to be sometimes corrupted

You can monitor the btrfs stats, an error will be always logged there, so if it's detected but the file still goes though you'll be warned.

 

23 minutes ago, primeval_god said:

Has there been any change since the last time this issue was updated?

It's not as bad now, before a corrupt file would always go though, in the last tests I did last time with v6.11.5 most times you get an i/o error, but if you retry many times it can still go through.

 

[Hellomynameisleo]

44 minutes ago, JorgeB said:

You can monitor the btrfs stats, an error will be always logged there, so if it's detected but the file still goes though you'll be warned.

 

It's not as bad now, before a corrupt file would always go though, in the last tests I did last time with v6.11.5 most times you get an i/o error, but if you retry many times it can still go through.

 

Thats the problem, I want it to be robust and something I can comfortably rely on. What use if it overides the backup with the corrupted file and notifies me, I can't do anything then.

Thanks for the information and help, hope this will change to be 100% reliable in the future.

[JorgeB]

When I have some time I'm going to test with zfs on Unraid v6.12-beta to see if it's the same or not.

[Hellomynameisleo]

On 1/25/2023 at 8:56 AM, JorgeB said:

When I have some time I'm going to test with zfs on Unraid v6.12-beta to see if it's the same or not.

can this be tested also with BTRFS please?

[JorgeB]

With btrfs the file can go through, even when not using FUSE, so likely a Samba issue, you get an i/o when copying locally.

[Hellomynameisleo]

1 hour ago, JorgeB said:

With btrfs the file can go through, even when not using FUSE, so likely a Samba issue, you get an i/o when copying locally.

thanks but is there a work around this? The only thing I can think if is doing a full system scrub on BTRFS UNRAID before transfering but I've got lots of 50 TB of data so this wouldn't be ideal.

Also does this issue also effect ZFS or BTRFS not on UNRAID when using SMB?

Edited January 26, 2023 by Hellomynameisleo

[JorgeB]

So this is the current status:

 

Btrfs using a disk share, transfer stops for a few seconds when an error is detected, sometimes you get an i/o error, sometimes not, if you get an error and click retry it might copy after one or more tries, this is logged:

 

Jan 27 16:47:53 Tower15  smbd[28944]:   smb2_sendfile_send_data: sendfile failed for file test/1.iso (Input/output error) for client ptr=0x559485d0f570,id=0,addr=ipv4:192.168.1.109:53238. Terminating
Jan 27 16:47:53 Tower15 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 426, gen 0
Jan 27 16:47:53 Tower15 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 257 off 476086272 csum 0xc8687774 expected csum 0x1e2ce107 mirror 1

 

Note the smb error with "Terminating" but it still goes through.

 

Btrfs using a user share, transfer is normal without any slowdown, only the btrfs checksum error is logged:

 

Jan 27 16:50:04 Tower15 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 430, gen 0
Jan 27 16:50:04 Tower15 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 257 off 476086272 csum 0xc8687774 expected csum 0x1e2ce107 mirror 1

 

Zfs using disk share or user share, transfer always aborts, at least it did the multiple times I've tried, only the SMB error is logged (but you can see the checksum errors and the affected file name if you run zpool stats -v)

 

Jan 27 16:42:23 Tower15  smbd[25007]:   smb2_sendfile_send_data: sendfile failed for file test/10_x64.iso (Input/output error) for client ptr=0x55d86e6f6d40,id=0,addr=ipv4:192.168.1.109:52866. Terminating

 

 

 

[JorgeB]

20 hours ago, Hellomynameisleo said:

Also does this issue also effect ZFS or BTRFS not on UNRAID when using SMB?

Don't know, never tried, not easy for me to do, but if I have time someday I will test.

[Hellomynameisleo]

2 hours ago, JorgeB said:

So this is the current status:

 

Btrfs using a disk share, transfer stops for a few seconds when an error is detected, sometimes you get an i/o error, sometimes not, if you get an error and click retry it might copy after one or more tries, this is logged:

 

Jan 27 16:47:53 Tower15  smbd[28944]:   smb2_sendfile_send_data: sendfile failed for file test/1.iso (Input/output error) for client ptr=0x559485d0f570,id=0,addr=ipv4:192.168.1.109:53238. Terminating
Jan 27 16:47:53 Tower15 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 426, gen 0
Jan 27 16:47:53 Tower15 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 257 off 476086272 csum 0xc8687774 expected csum 0x1e2ce107 mirror 1

 

Note the smb error with "Terminating" but it still goes through.

 

Btrfs using a user share, transfer is normal without any slowdown, only the btrfs checksum error is logged:

 

Jan 27 16:50:04 Tower15 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 430, gen 0
Jan 27 16:50:04 Tower15 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 257 off 476086272 csum 0xc8687774 expected csum 0x1e2ce107 mirror 1

 

Zfs using disk share or user share, transfer always aborts, at least it did the multiple times I've tried, only the SMB error is logged (but you can see the checksum errors and the affected file name if you run zpool stats -v)

 

Jan 27 16:42:23 Tower15  smbd[25007]:   smb2_sendfile_send_data: sendfile failed for file test/10_x64.iso (Input/output error) for client ptr=0x55d86e6f6d40,id=0,addr=ipv4:192.168.1.109:52866. Terminating

 

 

 

thanks for the test, what if you was to copy a file from the windows PC side instead of operating from unraid instead such as copying and pasting a file from the NAS user/disk share to the windows pc? Would this come out the same result?

[JorgeB]

23 minutes ago, Hellomynameisleo said:

what if you was to copy a file from the windows PC side

This is what I did.

 

Also note regarding your scenario

On 1/24/2023 at 8:20 AM, Hellomynameisleo said:

will it also give an error and abord automatically if detected so it doesn't overide the good copy with the corrupted file?

This will still be a problem even if it was working correctly (or you start using zfs), i.e., it will ask to overwrite the existing file, if you say yes and then a checksum error is detected the copy will abort but the good existing file on Windows would be gone, though don't see a reason to replace existing files with the same ones.

[Hellomynameisleo]

31 minutes ago, JorgeB said:

This is what I did.

 

Also note regarding your scenario

This will still be a problem even if it was working correctly (or you start using zfs), i.e., it will ask to overwrite the existing file, if you say yes and then a checksum error is detected the copy will abort but the good existing file on Windows would be gone, though don't see a reason to replace existing files with the same ones.

Will rclone be a good solution to this and when a corrupted file that stills go through? It has a --backup-dir option when syncing where any file that is going to to be ovewritten or deleted will simply get to put into a seperate backup directory instead. Seems like a good solution for all this

[JorgeB]

That should work, I believe rclone is based on rsync? By default rsync and robocopy won't overwrite existing files with the same size/modification date.

[Hellomynameisleo]

50 minutes ago, JorgeB said:

That should work, I believe rclone is based on rsync? By default rsync and robocopy won't overwrite existing files with the same size/modification date.

It seems similar but also can do cloud based storage syncing. rsync and rclone can have checksum checks option so it checks the hash of the source and destination when syncing

[JorgeB]

17 minutes ago, Hellomynameisleo said:

rsync and rclone can have checksum checks option so it checks the hash of the source and destination when syncing

Yes, but that off by default, it will take a long time to transfer data with checksums enabled.