Nvidia Passthrough Unable to power on device, stuck in D3

[PeteyBoPetey]

Hi Guys,

 

Upgraded to 6.10 from 6.9 with a clean install, transferred super.dat and plus.key to new config folder. VMs worked flawlessly in 6.9. Recreated VMs in 6.10. Dads VM starts sometimes, Erics VM always starts but freezes at Windows sign in and Isabelles VM nothing at all. I get "Unable to power on device, stuck in D3" in the VM logs. A quick google search shows numerous people with the exact same problem resolved with a motherboard bios update. Why would an Unraid upgrade require a bios update? I'm reluctant to 'try' a bios update to see if this solves the problem because it has the potential to brick the motherboard.

 

So far I have:

• Edited the xml of each VM so each GPU is in a slot with functions 0,1,2 and 3
  • Dads Super Computer slot 5
  • Erics Computer slot 6
  • Isabelle Computer slot 9
• Removed the Nvidia header from the vbios with a hex editor and placed in the isos root folder (A few times Unraid said it couldn't find the rom file?)
• Edited syslynix so Unraid wouldn't use the GPU's
• Set PCIe ACS override to Both
• Set VFIO allow unsafe interrupts to Yes

What am I doing wrong?

TIA

 

Peter

Dads Super ComputerXML.txt Dads VM Log.txt Erics VM Log.txt Erics VM XML.txt Isabelle VM Log.txt Isabelle VM XML.txt tower-diagnostics-20220607-1153.zip tower-syslog-20220606-1901.zip syslinux.cfg

[ghost82]

Hi, all I can see is that there is something wrong in your syslinux append line, your actual cmdline looks like this from the logs:

BOOT_IMAGE=/bzimage pcie acs override=multifunction video=efifb:off vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream,multifunction initrd=/bzroot

 

Remove:

 pcie acs override=multifunction

 

and reboot, but not sure it will fix things.

Edited June 7, 2022 by ghost82

[PeteyBoPetey]

OK. Deleted pcie acs override=multifunction and when I boot I get the attached screen

[PeteyBoPetey]

UPDATE:

In the last couple of days I've had trouble expanding the iso directory tree to select the vbios, sometimes it wont boot, I wonder if my Sandisk is on the way out?. Replacement is on my To Do list, so I guess I better Do sooner rather than later?. I live in Australia, everything takes forever to get delivered, so it might be a while before I get my new USB and continue with my trouble shooting.

[ghost82]

4 hours ago, PeteyBoPetey said:

when I boot I get the attached screen

 

4 hours ago, PeteyBoPetey said:

I wonder if my Sandisk is on the way out?

It seems so. Kernel panic refers to the flash drive.