[FIXED] OVMF TPM BIOS, Not finding boot HDD on SATA IOMMU Passthrough (Win 11 VM).

DemoRic

By DemoRic
in Virtualizing Unraid

Recommended Posts

DemoRic Noob

DemoRic

Posted
Posted (edited)

Hello all.  I've been plugging away at my UnRAID machine.  I've got my network share working just fine, Plex is using the iGPU for transcoding, and I've got a SAS card for all the Unraid SATA drives.   I am doing a Pass-through system on a Windows 11 install and have cleared all on-board sata connections for use in my VM, along with a GTX 970.  

 

My issue is when I start the Win 11 VM, it boots to a command prompt and doesn't automatically find and boot to the Sata drives.   I can "exit" and go to the BIOS and manually select the drive. Everything works fine from there.  However I would like to be able to have the boot be automatic without any manual keyboard entries.  Is there a way to create a boot loader or startup script to find and automatically boot from the drive?

 

Using OVMF TPM BIOS, and the SATA drive is identified in the bios as:  UEFI WMD WDBNCE0010PNC  21164X803630, IOMMU Group is 13

image.thumb.png.47f52e27632ac8183b1e3dd5f8c353ba.png

Edited by DemoRic
fixed
Link to comment
SimonF Experienced

SimonF

Posted
Posted
5 hours ago, DemoRic said:

Hello all.  I've been plugging away at my UnRAID machine.  I've got my network share working just fine, Plex is using the iGPU for transcoding, and I've got a SAS card for all the Unraid SATA drives.   I am doing a Pass-through system on a Windows 11 install and have cleared all on-board sata connections for use in my VM, along with a GTX 970.  

 

My issue is when I start the Win 11 VM, it boots to a command prompt and doesn't automatically find and boot to the Sata drives.   I can "exit" and go to the BIOS and manually select the drive. Everything works fine from there.  However I would like to be able to have the boot be automatic without any manual keyboard entries.  Is there a way to create a boot loader or startup script to find and automatically boot from the drive?

 

Using OVMF TPM BIOS, and the SATA drive is identified in the bios as:  UEFI WMD WDBNCE0010PNC  21164X803630, IOMMU Group is 13

image.thumb.png.47f52e27632ac8183b1e3dd5f8c353ba.png

Are you passing through the sata controller? Can you post the XML.

 

Is there more that one drive other wise why dont you use the drive path in the VM?

  • Thanks 1
Link to comment
DemoRic Noob

DemoRic

Posted
  • Author
Posted

I am passing through the:  Intel 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode]

 

I have 2 drives on that sata controller.  While I don't need both I figured that passing through the sata controller directly would be the fastest I/O speeds for the VM as I am wanting to be able to dual-boot the OS and get the most performance out of the hardware as a VM with the ability to just boot directly if needed.  It also has the benefit of isolating the unraid drives from the windows 11 install.

 

I suppose I could go and try to do a passthrough for just the drive.  

  

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>PINBALL</name>
  <uuid>03000200-0400-0500-0006-000700080009</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>17301504</memory>
  <currentMemory unit='KiB'>17301504</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='6'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-7.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/03000200-0400-0500-0006-000700080009_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='3' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11-NoTPM.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.225-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <interface type='bridge'>
      <mac address='52:54:00:da:74:57'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/EVGA-GTX970SSC_VMFix.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52b'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1532'/>
        <product id='0x0214'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

Link to comment
SimonF Experienced

SimonF

Posted
Posted
3 minutes ago, DemoRic said:

I am passing through the:  Intel 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode]

 

I have 2 drives on that sata controller.  While I don't need both I figured that passing through the sata controller directly would be the fastest I/O speeds for the VM as I am wanting to be able to dual-boot the OS and get the most performance out of the hardware as a VM with the ability to just boot directly if needed.  It also has the benefit of isolating the unraid drives from the windows 11 install.

 

I suppose I could go and try to do a passthrough for just the drive.  

  

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>PINBALL</name>
  <uuid>03000200-0400-0500-0006-000700080009</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>17301504</memory>
  <currentMemory unit='KiB'>17301504</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='6'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-7.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/03000200-0400-0500-0006-000700080009_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='3' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11-NoTPM.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.225-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <interface type='bridge'>
      <mac address='52:54:00:da:74:57'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/EVGA-GTX970SSC_VMFix.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52b'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1532'/>
        <product id='0x0214'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

Not sure if this will work, but have you tried setting boot order on the pci card?

Link to comment
DemoRic Noob

DemoRic

Posted
  • Author
Posted (edited)

So the Sata Controller is the onboard sata controller.  I have the boot order set in BIOS but that really only effects the computer startup and not the VM.  I do know that the HDD's are recognized in the OVMF TPM BIOS, I just don't know or see a way to set a boot order for the drives there.  I was thinking of maybe adding a virtual disk just to boot into a boot loader.  I just don't know where to go to create on that works with the VM's BIOS.  

 

The following is what I boot into, and have to "exit"

image.thumb.png.9477e03b20f720a656d14e7fb3907393.png

 

The following is the drive that I manually set:

image.thumb.png.109741d1325963a51df24e8551c1a7bd.png

 

I will add that in the OVMF TPM BIOS, I can go to Boot Maintenance Manager ->  Boot Options -> Change Boot Order, and change the order hit f10 and save and exit but the settings aren't persistent and will be lost as soon as I turn-off / restart the VM itself.

Edited by DemoRic
Link to comment
SimonF Experienced

SimonF

Posted
Posted (edited)
1 hour ago, DemoRic said:

So the Sata Controller is the onboard sata controller.  I have the boot order set in BIOS but that really only effects the computer startup and not the VM.  I do know that the HDD's are recognized in the OVMF TPM BIOS, I just don't know or see a way to set a boot order for the drives there.  I was thinking of maybe adding a virtual disk just to boot into a boot loader.  I just don't know where to go to create on that works with the VM's BIOS.  

 

The following is what I boot into, and have to "exit"

image.thumb.png.9477e03b20f720a656d14e7fb3907393.png

 

The following is the drive that I manually set:

image.thumb.png.109741d1325963a51df24e8551c1a7bd.png

 

I will add that in the OVMF TPM BIOS, I can go to Boot Maintenance Manager ->  Boot Options -> Change Boot Order, and change the order hit f10 and save and exit but the settings aren't persistent and will be lost as soon as I turn-off / restart the VM itself.

I meant the boot order in the vm template for pci devices, but it only allows nvme. So I think you need to use path to drive.

 

/dev/disk/by-id/

 

Edited by SimonF
Link to comment
DemoRic Noob

DemoRic

Posted
  • Author
Posted (edited)

Got it working.  I spent some time looking into coding the startup.sh file as I found I could still manually access the efi bootloader for Win11 at FS1:\efi\boot\bootx64.efi

I then went down the rabbit hole of looking for Clover EFI bootloader and found one that SpaceInvader One had already created.   I used it and everything boots properly. 

 

image.thumb.png.263dbc8eca83cc55a355dd0c10701e51.png

 

  

See Video for modified clover iso.  https://www.youtube.com/watch?v=RrWOEYldTdk

 

Edited by DemoRic
Link to comment
  • DemoRic changed the title to [FIXED] OVMF TPM BIOS, Not finding boot HDD on SATA IOMMU Passthrough (Win 11 VM).
DemoRic Noob

DemoRic

Posted
  • Author
Posted (edited)

UPDATE:  The above fix works great when I am using a Virtual GPU, it recognizes and automatically boots from my HDD. 

When using my GTX 960 as a passthrough.  It hangs on boot .  *sigh*

 

UPDATE2: Removed the boot order values for everything and now it finds the SSD on the Sata Controller and boots just fine  when using the Virtual Graphics and when I change it to the GTX 960.  

 

Don't really understand what the hangup was but, it now works as desired and automatically boots into the Win11 VM.

Edited by DemoRic
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Go to topic listing