Hi, since a couple of days i0m getting a lot of cache disk errors:

 

Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566422, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566423, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566424, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566425, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566426, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566427, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566428, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566429, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566430, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme2n1p1 errs: wr 7566431, rd 3628509, flush 2077, corrupt 3, gen 0
Jul 31 12:51:40 Home-Server kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme2n1p1 (-5)
Jul 31 12:51:40 Home-Server kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme2n1p1 (-5)
Jul 31 12:51:40 Home-Server kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme2n1p1 (-5)
Jul 31 12:51:40 Home-Server kernel: BTRFS error (device nvme0n1p1): error writing primary super block to device 2

 

reading the FAQ here i tried to run the btrfs stats command on the cache:

 

btrfs dev stats /mnt/cache
[/dev/nvme0n1p1].write_io_errs    0
[/dev/nvme0n1p1].read_io_errs     0
[/dev/nvme0n1p1].flush_io_errs    0
[/dev/nvme0n1p1].corruption_errs  0
[/dev/nvme0n1p1].generation_errs  0
[/dev/nvme2n1p1].write_io_errs    7566484
[/dev/nvme2n1p1].read_io_errs     3628509
[/dev/nvme2n1p1].flush_io_errs    2078
[/dev/nvme2n1p1].corruption_errs  3
[/dev/nvme2n1p1].generation_errs  0

 

Since i have two 1tb NVME cache disks (nvme0n1 and nvme2n1) and since nvme2n1 is a bit older i thought that one disk was going bad and i tried to restart the array with just the good drive. on the first restart it worked correctly but after reboot it said that the cache drive was "Unmountable: Unsupported or no file system after reboot". After that adding back the second drive seemed to have solved it... but when i start a VM the log it's flooded again with errors.

 

Now i restarted the array with just nvme0n1 and it looks ok, but i'm sure that if i restart it i'll get the Unmountable error. right now i can't get the diagnostic from unraid... the page is crashing

after the restart if i run the the btrfs stats command on the cache again i still get two drives even if one was removed:
 

btrfs dev stats /mnt/cache
[/dev/nvme0n1p1].write_io_errs    0
[/dev/nvme0n1p1].read_io_errs     0
[/dev/nvme0n1p1].flush_io_errs    0
[/dev/nvme0n1p1].corruption_errs  0
[/dev/nvme0n1p1].generation_errs  0
[/dev/nvme2n1p1].write_io_errs    7566484
[/dev/nvme2n1p1].read_io_errs     3628509
[/dev/nvme2n1p1].flush_io_errs    2078
[/dev/nvme2n1p1].corruption_errs  3
[/dev/nvme2n1p1].generation_errs  0

 

Edited July 31, 20241 yr by Vulneraria

Ok the cache was just rebalancing, now the btrfs stats shows only one drive:
 

btrfs dev stats /mnt/cache
[/dev/nvme0n1p1].write_io_errs    0
[/dev/nvme0n1p1].read_io_errs     0
[/dev/nvme0n1p1].flush_io_errs    0
[/dev/nvme0n1p1].corruption_errs  0
[/dev/nvme0n1p1].generation_errs  0

 

If you didn't yet reboot post the diagnostics.