BTRFS Errors + Docker

[pearce1340]

Hi All,

 

I'm having some issues recently with my cache drives and BTRFS errors. I think (again not 100% certain) its caused by a windows update I was running on my VM 20H2 was the update. I restarted the VM... which is already a challenge as I have a RX5700 card. Then at some point into the installing updates, my dockers start failing and everything crashes!

 

These are the errors I see in my log - and they fill it up!:

 

Jan  9 13:54:49 Unraid kernel: BTRFS warning (device nvme0n1p1): i/o error at logical 630640209920 on dev /dev/nvme1n1p1, physical 154951610368, root 5, inode 497938, offset 79220768768, length 4096, links 1 (path: domains/Windows 10/vdisk1.img)
Jan  9 13:54:52 Unraid kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme1n1p1 (-5)
Jan  9 13:54:52 Unraid kernel: BTRFS error (device nvme0n1p1): error writing primary super block to device 2
Jan  9 13:54:52 Unraid kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme1n1p1 (-5)
Jan  9 13:54:52 Unraid kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme1n1p1 (-5)
Jan  9 13:54:52 Unraid kernel: BTRFS warning (device nvme0n1p1): lost page write due to IO error on /dev/nvme1n1p1 (-5)
Jan  9 13:54:52 Unraid kernel: BTRFS error (device nvme0n1p1): error writing primary super block to device 2
Jan  9 13:54:54 Unraid kernel: btrfs_dev_stat_print_on_error: 8898923 callbacks suppressed
Jan  9 13:54:54 Unraid kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme1n1p1 errs: wr 185867, rd 46640931, flush 6441, corrupt 1321, gen 0
Jan  9 13:54:54 Unraid kernel: BTRFS error (device nvme0n1p1): bdev /dev/nvme1n1p1 errs: wr 185867, rd 46640932, flush 6441, corrupt 1321, gen 0

 

I have performed a scrub on the cache drives and all seem fine?! Anyone got any ideas? I'm not sure windows updates should be able to cause this level of issues on what should be an image file.

 

Full logs are attached as well.

 

Thanks,

Steve

unraid-diagnostics-20210109-1406.zip

[JorgeB]

NVMe device is dropping offline:

 

Jan  9 12:50:22 Unraid kernel: nvme nvme1: Device not ready; aborting reset, CSTS=0x1
Jan  9 12:50:22 Unraid kernel: nvme nvme1: Removing after probe failure status: -19
Jan  9 12:50:53 Unraid kernel: nvme nvme1: Device not ready; aborting reset, CSTS=0x1

 

Look for a BIOS update, this can also sometimes help, some NVMe devices have issues with power states on Linux, try this, on the main GUI page click on flash, scroll down to "Syslinux Configuration", make sure it's set to "menu view" (on the top right) and add this to your default boot option, after "append" and before "initrd=/bzroot"

nvme_core.default_ps_max_latency_us=0

Reboot and see if it makes a difference.

[pearce1340]

Thanks, I'll try that. You were right in it dropping off... it completely disappeared and took a couple of reboots for it to come back, even though it was showing in the BIOS the whole time. I'll also check out the firmware updates... not too keen on this, as I've just got my VMs working perfectly and that took a fair amount of finding the right firmware lol.