Jayant P Saikia Posted November 24, 2022 Share Posted November 24, 2022 Hi Team, I would like to raise a concern around my VFIO passthrough experience. May be this is a bug which can be addressed. Or this experience helps someone else from this pitfall. I have a pfSense VM with 4 NIC's passthrough to the VM. The VM is set to autostart ofcourse. The four NIC's are listed as below. Now I decided to add a new Asus PCI M.2 X16 card which supports 4 M.2 drives with PCI bifurcation. When I plugged in this device. My system would boot but freeze on startup. The error thrown out is a Filesystem corruption error displayed on the screenshot below. After multiple hours of troubleshooting, I found that the PCI IDS have changed for the NIC's. The four NIC's are now listed as below. And if you see carefully, the old PCI IDS are now occupied by the SATA controllers. I disabled Autostart of array, but could not disable Autostart of VM. Had to pull out the new Asus card, go back to old config. Set the pfSense VM not to autostart. Then plugged back the Asus card in. And finally I was able to start the VM Manager. At this point, if I manually start the VM without editing the passthrough devices, the same BTRFS corruption error happens and everything is frozen. So I hard rebooted the machine and before starting the pfSense VM, I edited the passthrough devices and everything works. Yey !!! Can this be fixed in anyway ? The lockout situation is very bad, any check before starting the VM to see if the correct devices are passed through would be good. Then at least the VM manager starts and the VM config can be edited. My situation was severely bad as it was throwing out the array disks itself. Quote Link to comment
JorgeB Posted November 24, 2022 Share Posted November 24, 2022 12 minutes ago, Jayant P Saikia said: I disabled Autostart of array, but could not disable Autostart of VM If you disable array auto start the VMs won't auto start after boot. Quote Link to comment
Jayant P Saikia Posted November 24, 2022 Author Share Posted November 24, 2022 I did disable array autostart. And I was able to login. Then I disabled VM Manager start. Then I was able to start the array. But as soon as I enable VM Manager the error happend. The only way was to go back to old config without the Asus card and make the VM to not auto start. Quote Link to comment
JorgeB Posted November 24, 2022 Share Posted November 24, 2022 3 minutes ago, Jayant P Saikia said: But as soon as I enable VM Manager the error happend. That should not happen, VMs should never autostart if array autostart is disabled, it's a constant user complain, i.e., "VMs don't start with autostart enable", because they only start at first boot if array auto start is enabled. Quote Link to comment
Jayant P Saikia Posted November 24, 2022 Author Share Posted November 24, 2022 Well that's not what I experienced. Array Autostart Disabled. VM Manager Enable VMs No. pfSense VM Autostart On. When I doVM Manager Enable VMs No to Yes. The pfSense VM Autostarts irrespective Array Autostart was Disabled or Enabled. Quote Link to comment
Jayant P Saikia Posted November 24, 2022 Author Share Posted November 24, 2022 Also, anyway to check [8086:10d3] 07:00.0 with [8086:1d6b] 07:00.0. and find a miss match. Which stops the PCI device reset ? Seems to me that the VM start is only checking "07:00.0". Quote Link to comment
Solution JorgeB Posted November 24, 2022 Solution Share Posted November 24, 2022 38 minutes ago, Jayant P Saikia said: Well that's not what I experienced. You are correct, behavior changed in the current release, going to find out when and report it since like you've experience this is a problem. 32 minutes ago, Jayant P Saikia said: Also, anyway to check [8086:10d3] 07:00.0 with [8086:1d6b] 07:00.0. and find a miss match. Which stops the PCI device reset ? Not really my area but I'll also inquire about that, it would make things easier if that's possible to do. Quote Link to comment
Jayant P Saikia Posted November 25, 2022 Author Share Posted November 25, 2022 Thanks for the update. Will keep this open to see if the PCI device id mapping can also be improved. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.