[7.1.4] ZFS Pool Degrades When VM Starts – Phison NVMe (vfio-pci) Still Claimed by ZFS

📝 Issue Summary

I’ve hit a reproducible issue in Unraid 7.1.4 where a ZFS mirror degrades only after starting a VM. The culprit appears to be a Phison E16 NVMe (Sabrent Rocket 4.0 1TB) controller that is bound to vfio-pci but is still partially claimed by the kernel/ZFS.

Key points:

• The drive is not part of my intended ZFS pool.

• I bound it in vfio-pci.cfg and added it to syslinux.cfg via vfio-pci.ids=1987:5016.

• lspci -k shows it bound to vfio-pci.

• Despite this, /dev/nvme2n1p1 appears in my ZFS cache pool mirror, and errors are logged when the VM starts I/O.

• If I remove the card physically, the issue disappears.

This looks like a race condition where the nvme driver claims the device before vfio, leaving ZFS to auto-import it.

🔍 Evidence (diagnostics captured)

1. Boot (before VM start) -

   • zpool status: all members ONLINE, no errors.

   • lspci: 09:00.0 (Phison E16) bound to vfio-pci.

   • lsblk: no /dev/nvme2n1 visible.

   • Diagnostics file: NAS-diagnostics-2025-10-01-preVM.zip

2. After starting VM (“Office”) The VM is a Windows 11 booting from a ZVOL on the cache pool

   • Within seconds of VM start, zpool status shows:

     nvme2n1p1  ONLINE   2  2  0

     → read/write errors logged.

   • Pool still ONLINE, apps unaffected, but this is silent corruption risk.

   • Diagnostics file: NAS-diagnostics-2025-10-01-postVM.zip

3. Reproducibility

   • 100% reproducible: every time VM is started, ZFS pool logs errors on the vfio-pci-bound Phison device.

   • With card removed: never happens.

   • With card present but no VM activity: pool remains clean.

Why this matters

This is not a failing device — it’s a working NVMe that I keep as a fallback baremetal boot disk. ZFS should never claim it once bound to vfio-pci, but right now Unraid’s device claim order allows ZFS to import it and mark the pool degraded.

This is potentially very dangerous, because ZFS silently rewires mirrors with a passthrough device, then throws checksum/I/O errors under load.

• Can Unraid adjust the vfio/nvme claim order so vfio binding prevents block device nodes from being created?

• Or provide a supported mechanism to blacklist an NVMe device from ZFS import, while keeping it physically installed for passthrough/baremetal use?

NAS-diagnostics-2025-10-01-preVM.zip.zip NAS-diagnostics-2025-10-01-postVM.zip.zip