This is not a bug, and not specific to v6.4, but since it’s the version being developed and also affected I decided to post here.

It’s something I’ve found when trying to demonstrate to a client the advantages of using unRAID with btrfs as a way to maintain data integrity, so I was showing him how it works, and for those that don’t know, a checksum enable filesystem like btrfs or zfs is supposed to log if a file fails checksum (also correct the error if redundancy exists) and if the error is uncorrectable it should give an I/O error when the user tries to copy/access that file, so the user knows something is wrong with it and doesn’t copy or continue to work on a corrupt file and instead restores it from a backup.

 

So, I purposely corrupted a file on an unRAID data disk, copied it to a Windows desktop and to my surprise the checksum error was logged but the copy completed without any error, this is not how it’s supposed to work as it can leave the user with a corrupt file without any noticeable warning (unless the log is periodically checked), and it had worked like it should for me once before when a file was corrupted after a disk got disable during a move operation.

 

After the client left and I cleaned all the egg from my face I investigated the problem, so first I tried with more files and found that if the file was very corrupt, like more than 20 or so checksums errors I would get the error, but files with a couple of errors would always copy, then I tried copying the file locally, and as expected it wasn’t possible, I got the I/O error even on a file with a single checksum mismatch, so it looked like the issue was samba related, finally I started downgrading unRAID, v6.2.4 had the same issue, v6.1.9 worked as it should, so I started upgrading and luckily the first v6.2-beta, beta18, had the issue back, so looking at the release notes this stood out:

 

Quote

- samba:
  - version: 4.3.5
  - enable asynchronous I/O in /etc/samba/smb.conf

 

So I disabled AIO for reads, by adding to the SMB extra settings:

 

aio read size = 0

 

Tested again and problem solved! A single checksum error was enough to always get an I/O error.

Now I’m not yet saying that AIO should be disable on the next release, if it really improves performance the positives may outweigh the negative, also most users are using xfs, but the question is, does AIO really improve read performance in unRAID? My limited tests says no, even when using 10GbE and certainly not with Gigabit, but anyone using btrfs can try for themselves and if no performance difference is noticed I recommend for now leaving AIO for reads disable (it can still be enable for writes).

 

Screens show a read test copying the same files with AIO enable vs disable, source is a Seagate 8TB archive disk on v6.3.5, if anything with AIO disable it seems a little faster:

 

 

That's some great analysis, thank you.  We'll try to figure out the best place to post this issue, either with btrfs developers (probably) or samba developers.  I think you're right, though for the sake of data integrity, we should disable samba async i/o for now.  The reason it was added was to experiment with "multi-channel" feature, cf:

https://blog.chaospixel.com/linux/2016/09/samba-enable-smb-multichannel-support-on-linux.html

12 hours ago, limetech said:

we should disable samba async i/o for now

 

Cool, also think it's best option, specially since AIO doesn't appear to increase performance without smb multichannel, and multichannel is still experimental and not recommend for production.

 

P.S. anyone reading this and using btrfs for data integrity that is worried a corrupted file may have slipped through, can quickly put their mind at ease without the need to scrub all disks by checking the stats of all btrfs filesystems, these stats don’t reset (unless done on purpose by the user) and show the values since that filesystem was first created, e.g.:

root@Tower1:~# btrfs dev stats /mnt/disk1
[/dev/md1].write_io_errs    0
[/dev/md1].read_io_errs     0
[/dev/md1].flush_io_errs    0
[/dev/md1].corruption_errs  0
[/dev/md1].generation_errs  0

All values should be 0 but any checksum mismatch detected will be added to the corruption errors, if there are any errors you can find which files are affected by running a scrub, corrupt file name(s) will be output to the syslog as they are found by the scrub.