I added an NVME drive using a PCIE adapter and it showed up as a drive in unRaid. I then created a new Windows 10 VM and edited the XML to pass through the NVME controller to the VM and installed Windows 10 directly to the NVME drive.
The problem comes when I start the VM after it has been shut down. I have to do it twice, the first time always fails.
There are no errors in the log, but there is a warning on both the failed and successful attempts to start the VM.
Tower kernel: nvme nvme0: failed to set APST feature (-19)
[Start VM Try #1]
Apr 26 16:22:55 Tower kernel: vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Apr 26 16:22:55 Tower kernel: nvme nvme0: failed to set APST feature (-19)
Apr 26 16:22:55 Tower kernel: br0: port 4(vnet2) entered blocking state
Apr 26 16:22:55 Tower kernel: br0: port 4(vnet2) entered disabled state
Apr 26 16:22:55 Tower kernel: device vnet2 entered promiscuous mode
Apr 26 16:22:55 Tower kernel: br0: port 4(vnet2) entered blocking state
Apr 26 16:22:55 Tower kernel: br0: port 4(vnet2) entered forwarding state
Apr 26 16:22:56 Tower avahi-daemon[2852]: Joining mDNS multicast group on interface vnet2.IPv6 with address fe80::fc54:ff:fede:f67d.
Apr 26 16:22:56 Tower avahi-daemon[2852]: New relevant interface vnet2.IPv6 for mDNS.
Apr 26 16:22:56 Tower avahi-daemon[2852]: Registering new address record for fe80::fc54:ff:fede:f67d on vnet2.*.
Apr 26 16:22:56 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x1e@0x258
Apr 26 16:22:56 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900
Apr 26 16:22:56 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19@0x168
Apr 26 16:22:56 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1e@0x190
Apr 26 16:23:03 Tower avahi-daemon[2852]: Interface vnet2.IPv6 no longer relevant for mDNS.
Apr 26 16:23:03 Tower avahi-daemon[2852]: Leaving mDNS multicast group on interface vnet2.IPv6 with address fe80::fc54:ff:fede:f67d.
Apr 26 16:23:03 Tower kernel: br0: port 4(vnet2) entered disabled state
Apr 26 16:23:03 Tower kernel: device vnet2 left promiscuous mode
Apr 26 16:23:03 Tower kernel: br0: port 4(vnet2) entered disabled state
Apr 26 16:23:03 Tower avahi-daemon[2852]: Withdrawing address record for fe80::fc54:ff:fede:f67d on vnet2.
Apr 26 16:23:04 Tower kernel: vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Apr 26 16:23:04 Tower kernel: nvme nvme0: pci function 0000:04:00.0
Apr 26 16:23:04 Tower kernel: nvme 0000:04:00.0: enabling device (0400 -> 0402)
Apr 26 16:23:05 Tower kernel: nvme0n1: p1 p2 p3 p4
[Start VM Try #2]
Apr 26 16:25:07 Tower kernel: vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Apr 26 16:25:07 Tower kernel: nvme nvme0: failed to set APST feature (-19)
Apr 26 16:25:07 Tower kernel: br0: port 4(vnet2) entered blocking state
Apr 26 16:25:07 Tower kernel: br0: port 4(vnet2) entered disabled state
Apr 26 16:25:07 Tower kernel: device vnet2 entered promiscuous mode
Apr 26 16:25:07 Tower kernel: br0: port 4(vnet2) entered blocking state
Apr 26 16:25:07 Tower kernel: br0: port 4(vnet2) entered forwarding state
Apr 26 16:25:08 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x1e@0x258
Apr 26 16:25:08 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900
Apr 26 16:25:08 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19@0x168
Apr 26 16:25:08 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1e@0x190
Apr 26 16:25:09 Tower avahi-daemon[2852]: Joining mDNS multicast group on interface vnet2.IPv6 with address fe80::fc54:ff:fede:f67d.
Apr 26 16:25:09 Tower avahi-daemon[2852]: New relevant interface vnet2.IPv6 for mDNS.
Apr 26 16:25:09 Tower avahi-daemon[2852]: Registering new address record for fe80::fc54:ff:fede:f67d on vnet2.*.