VM Bios change

[wagri]

Hello everyone

 

This is the first time I have used the forum, I have always managed to manage on my own, but I will need help.

 

I created a VM with the pci passthroug of an NVME ssd to have a high speed and low latency har disk.

 

My OS is on this ssd but when I launch my vm I always have to go to the bios of the vm and change the boot order so that it can launch this and I can't save the settings, it resets to zero each time the VM reboots.

 

My question is the following how to save a bios parameter of the VM.

 

Thank you  

[ghost82]

Usually boot preferences are saved in the ovmf vars file (assuming you're using ovmf).

In the ovmf bios you define and save the boot order preference.

If for whatever reason it doesn't work, add the boot order line to your passed through nvme controller, something like:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x82' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
      <boot order='1'/>
    </hostdev>

 

Obviously source (82:00.0) and target (08:00.0) addresses are different (most probably) in your case.

[wagri]

thanks i will try that and send you the result  

[wagri]

It's works ! 

Thank you so much.

 

[FaminedWarrior]

Hey all, so I was having a similar issue but when I tried to apply the recommended fix I get the following error. I have attached my XML file, am I missing something?

 

VM creation error

unsupported configuration: per-device boot elements cannot be used together with os/boot elements

 

 

Quote

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>a2e93cc0-a9e8-1ee0-3c78-6bca4aaed78b</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>17301504</memory>
  <currentMemory unit='KiB'>17301504</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='11'/>
    <vcpupin vcpu='4' cpuset='5'/>
    <vcpupin vcpu='5' cpuset='13'/>
    <vcpupin vcpu='6' cpuset='7'/>
    <vcpupin vcpu='7' cpuset='15'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-5.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/a2e93cc0-a9e8-1ee0-3c78-6bca4aaed78b_VARS-pure-efi.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISO/virtio-win-0.1.190-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:6e:d4:b9'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='2'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/VM/VBIOS/MSI.GTX1080Ti.11264.170118.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    <boot order='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0d' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x185c'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

[ghost82]

4 hours ago, FaminedWarrior said:

am I missing something?

Remove this if you want to add boot order line:

<boot dev='hd'/>

 

[FaminedWarrior]

Tried this but now it just reset loops.

 

3 hours ago, ghost82 said:

Remove this if you want to add boot order line:

<boot dev='hd'/>

 

 

[ghost82]

Is your windows disk ok?

It resets where?

How did you apply modifications?

[FaminedWarrior]

Windows seems fine, will still boot to baremetal. KVM is what is resetting constantly. I didn't really modify anything other than the few lines of code within the XML file, I tried to recreate VM a couple times just in case something is corrupting. The other thought I am having is that because all the PCI lanes are used it maybe doing something strange? Motherboard is Crosshair VII and I have 2 GPUs + an HBA + 2 nvme drives.

[ghost82]

You need to attach some data because with a generic description nobody could help. Diagnostics

[FaminedWarrior]

As requested, I appreciate the help.

obsidian-diagnostics-20220514-0839.zip

[ghost82]

Does it boot without passing through the gpu?

----

Once it boots with vnc:

Your gpu is not attached to vfio at boot <-- attach it to vfio at boot

Your gpu definition in the vm xml is not multifunction <-- define it as multifunction

You are passing a vbios file which seems to be a downloaded one, this is not the preferred way <-- dump your own vbios, or make sure the downloaded file is compatible and remove the header with an hex editor

You are using both qxl/vnc and gpu passthrough <-- this is not ideal, boot with vnc only, enable remote desktop, remove qxl/vnc and passthrough only the gpu: if you get a black screen and the vm boots you could access it from another device and see what's wrong inside the vm; if you need remote management you can still access the vm with the remote desktop enabled inside the vm and not with unraid novnc.

Edited May 14, 2022 by ghost82

[FaminedWarrior]

Thanks for taking a look, it does the same loop reset loop without any GPU being passed through. Do you thinking doing a fresh install on the nvme would help?

 

Ideally I would like to only passthrough the GPU but I am having this issue so I added VNC to troubleshoot.

 

15 minutes ago, ghost82 said:

Does it boot without passing through the gpu?

----

Once it boots with vnc:

Your gpu is not attached to vfio at boot <-- attach it to vfio at boot

Your gpu definition in the vm xml is not multifunction <-- define it as multifunction

You are passing a vbios file which seems to be a downloaded one, this is not the preferred way <-- dump your own vbios, or make sure the downloaded file is compatible and remove the header with an hex editor

You are using both qxl/vnc and gpu passthrough <-- this is not ideal, remove qxl/vnc and passthrough only the gpu: if you need remote management enable remote desktop from inside the vm

 

[ghost82]

7 minutes ago, FaminedWarrior said:

Do you thinking doing a fresh install on the nvme would help?

In your logs there's nothing useful, unfortunately.

Was windows installed with uefi or is it a legacy installation?Because you are using ovmf (uefi).

I don't have any nvme, but some could cause issues when passed through...so yes, if I were you, I would create a new vm (without deleting that one), passing through only the nvme, no usb, no gpu (use vnc), nothing, all emulated hardware (and I would avoid also any virtio device, if any).

Then boot from the windows installer and see if at least your nvme is seen and ready to be formatted inside the windows installer.

Before doing this backup all your important data (if any).

Then it's your choice to proceed and try with a new installation.

If and once it will boot add one device at a time, so you can revert back soon and you wont need to debug multiple things at the same time.

 

Use ovmf+q35 machine type instead of i440fx.

Edited May 14, 2022 by ghost82

[FaminedWarrior]

Thanks for the advice, I created a new VM and the installer saw it so I am in the process of data recovery and then I will try a fresh install. The whole situation is rather bizarre. I will update when I have more information.

 

16 minutes ago, ghost82 said:

 

Then boot from the windows installer and see if at least your nvme is seen and ready to be formatted inside the windows installer.

Before doing this backup all your important data (if any).

Then it's your choice to proceed and try with a new installation.

If and once it will boot add one device at a time, so you can revert back soon and you wont need to debug multiple things at the same time.

 

[ghost82]

I edited my post above, in particular:

22 minutes ago, ghost82 said:

Was windows installed with uefi or is it a legacy installation?Because you are using ovmf (uefi).

and for the new vm:

22 minutes ago, ghost82 said:

Use ovmf+q35 machine type instead of i440fx.

 

[FaminedWarrior]

Sorry missed this, nvme install is UEFI.

 

8 minutes ago, ghost82 said:

Was windows installed with uefi or is it a legacy installation?Because you are using ovmf (uefi).

 

 

 

 

[FaminedWarrior]

A format fixed it, it must have been the way the disk was formatted or something. Before formatting Windows had 4 partitions, this time around it only created 3. Everything seems to be running smoothly now. Thank you @ghost82 for the help.

[FaminedWarrior]

For future users, if this happens to you on an Asus Motherboard. If you have nvme_1 in use be sure that sata 1 is not. The same if nvme_2 is used be sure sata 2 is not. I found that I was getting long post times and then did some more digging and found that these ports can cause conflict.

[ghost82]

11 hours ago, FaminedWarrior said:

Before formatting Windows had 4 partitions, this time around it only created 3

mmmm...the 4 layout partitions is that of an uefi installation:

System - MSR - Windows - Recovery

The 3 layout partitions is that of a legacy installation:

System - Windows - Recovery

 

However you wrote that nvme had 4 partitions and was installed with uefi (and this is accordingly to the 4 partitions layout), but the xml inside your diagnostics (Windows 10 vm) has ovmf and this is not in accordance to the 3 partitions layout you got: did you change from ovmf to seabios in the new installation?