Jump to content

  • [7.0.0-beta.1] Can't boot new OVMF VM by pointing it to existing qcow2 file

    Espressomatic

    By Espressomatic

      • Minor

    Summary

     

    When a VM is created using OVMF BIOS and the primary vdisk is pointed at a pre-made qcow2 file, it will fail to boot and dump you to UEFI shell. This is both for qcow2 files made in Unraid and previously working, or those downloaded or migrated from other environments (like Proxmox)

     

    The issue was described in another thread here:

    This matches exactly the issue I'm seeing and reporting in this thread, including creating initial VMs within Unraid instead of importing from other products.

      

    On 4/12/2024 at 4:31 AM, kleinemeise said:

    Another example: I create a new VM and cleanly install Debian. I save the path to the image. Then I create a new VM and manually set the path to the image before. The VM doesn't start afterwards - same error

     

    Description

     

    If you create a VM in Unraid using install media iso with OVMF BIOS (default for many selections like Debian and Fedora), you can then recreate the VM with the same settings, without install media, point it to the verified working vidsk, and it'll no longer work.

     

    Selecting SeaBIOS when creating the VM will correctly boot the vdisk file.

     

     Here's the XML from two VMs. One still working, and the other I just broke with the steps above.  Working one is Fedora, broken one is Debian. XML looks pretty much identical other than ids and "disconnected" on the non-working for guest agent connection.

     

    Steps to reproduce - short method (pre-made vdisk download)

     

    1. Download Fedora Server 40 qcow2 vdisk file here: https://fedoraproject.org/server/download
    2. Copy/save qcow2 file to desired host file path (such as /mnt/users/domains/fedora/vdisk.qcow2)
    3. Create new VM in Unraid, select Fedora as type
    4. Specify Primary vdisk as MANUAL and point to the above qcow2 file
    5. Press Create button at the bottom to create and start the VM
    6. Open VNC VM Console
    7. Note EFI Shell instead of the vdisk booting

     

    Steps to reproduce - long method (all Unraid)

    1. Download install ISO media for Fedora Server 40 and/or Debian 12
    2. Create new VM
    3. Select Fedora
    4. Select the install media ISO from where it's stored
    5. Press Create VM button at bottom of UI
    6. Proceed through setup using the VNC console
    7. Reboot once setup is complete and verify the vdisk is booting
    8. Create a second VM, this time Debian 12
    9. Select install media ISO from where it's stored
    10. Press Create VM button at bottom of UI
    11. Proceed through setup using the VNC console
    12. Reboot once setup is complete and verify the vdisk is booting
    13. Both VMs are working at this point
    14. Stop both of them and remove the install ISO from each - restart and confirm still working
    15. Both VMs are working at this point
    16. Stop the Debian VM and remove/delete it - do NOT delete the vdisk file
    17. Create a new VM, select Debian as type
    18. Specify Primary vdisk as MANUAL and point to the vdisk qcow2 file you were just using
    19. Press Create button at the bottom to create and start the VM
    20. Open VNC VM Console
    21. Note EFI Shell instead of the vdisk booting
    22. Try selecting STOP VM from Unraid
    23. The VM instance will not stop

     

    Compare XML of Fedora and Debian VMs (samples included below)

     

    Fedora working 

    <?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='13'>
  <name>Fedora</name>
  <uuid>ae05b20a-155b-71d3-a0f9-3f7bc1292789</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Fedora" icon="fedora.png" os="fedora" webui="" storage="default"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>1</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-8.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/ae05b20a-155b-71d3-a0f9-3f7bc1292789_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='1' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='utc'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='no'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='rtc' tickpolicy='catchup'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback'/>
      <source file='/mnt/user/vm_hosts/Fedora/vdisk1.qcow2' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:f3:61:90'/>
      <source bridge='br0'/>
      <target dev='vnet12'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/run/libvirt/qemu/channel/13-Fedora/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5901' autoport='yes' websocket='5701' listen='0.0.0.0' sharePolicy='ignore'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <watchdog model='itco' action='reset'>
      <alias name='watchdog0'/>
    </watchdog>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

     

    Debian not working after recreating and pointing to existing qcow2 file 

    <?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='15'>
  <name>Debian</name>
  <uuid>9512924f-3e97-3821-9dc0-9a17cb6ebc48</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Debian" icon="debian.png" os="debian" webui="" storage="default"/>
  </metadata>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>1</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-8.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/9512924f-3e97-3821-9dc0-9a17cb6ebc48_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='1' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='utc'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='no'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='rtc' tickpolicy='catchup'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback'/>
      <source file='/mnt/user/vm_hosts/Debian/vdisk1.qcow2' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:c3:bc:1c'/>
      <source bridge='br0'/>
      <target dev='vnet14'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/2'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/run/libvirt/qemu/channel/15-Debian/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5902' autoport='yes' websocket='5702' listen='0.0.0.0' sharePolicy='ignore'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <watchdog model='itco' action='reset'>
      <alias name='watchdog0'/>
    </watchdog>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

     

    Go to reports Prereleases

    User Feedback

    Recommended Comments

    SimonF

    Issue is that the bootmgr does not know where to find the boot file for grub. Goto boot maint

    image.png

     

    Goto next page, add boot option select drive

     

    image.png

    Select EFI and go down the tree until you find grub

     

    image.png

     

    image.png

    Add description and commit.

     

    Now this should be stored in the NVRAM and boot.

     

    There is no automatic way I can add this via the GUI I know of and would not know the correct loader, but will continue to see if I can find a solution

    • Upvote 1
    Link to comment
    Espressomatic
    3 hours ago, SimonF said:

    Issue is that the bootmgr does not know where to find the boot file for grub. Goto boot maint

     

    I did these steps last night before you posted, and successfully got Debian up and running. ;) It took a while to get there as I was exploring pretty much every entry in the manager to find somewhere to tell it what to load.

     

    Is this expected behavior, as in some kind of limitation, or can it be addressed to make it automatic/transparent? How does it not affect VMs created anew/from scratch? There's nothing in the XML definition that points to the grub file.

     

    EDIT. I wrote the above without carefully reading the end of your post.  I expect this is the answer to that. The info is in the NVRAM file which doesn't exist on the new import.

     

    3 hours ago, SimonF said:

    Now this should be stored in the NVRAM and boot.

     

    Maybe there's some info out there about how other VM UIs handle this. I did some reading last night and I'm pretty sure Proxmox handles it by not allowing an existing vdisk to be specified in the UI. 🤣 10 seconds in Proxmox is 10 seconds of pure hell.

     

    Interestingly, this doesn't happen with my Home Assistant vdisk - but that's got its own set of issues as there's no way to shut it down (that'll be a different discussion).

    Edited by Espressomatic
    Link to comment
    SimonF
    10 minutes ago, Espressomatic said:

    The info is in the NVRAM file which doesn't exist on the new impor

    The installer must update the NVRAM/boot sector to specify the file to load. Creating the new VM will create a standard NVRAM which would not have been modified hence the non boot.

     

    I think it will depend on the disk partition type ie mbr or gpt for boot sector. Hence why HA will boot. Also with regards to the Fedora qcow file I guess it does not have efi within the image hence will only boot legacy(Seabios),

    • Upvote 1
    Link to comment
    Espressomatic

    Is there a way to add this data to the existing NVRAM? Because manually selecting the grub or other boot file doesn't make a persistent change. The next time booting the VM you're left at the EFI shell again, which means that for all intents and purposes, the VM is destroyed and can't ever be used to auto-boot.

     

     

    Link to comment
    SimonF
    7 minutes ago, Espressomatic said:

    Is there a way to add this data to the existing NVRAM? Because manually selecting the grub or other boot file doesn't make a persistent change. The next time booting the VM you're left at the EFI shell again, which means that for all intents and purposes, the VM is destroyed and can't ever be used to auto-boot.

     

     

    Did you go into boot maintenance and add the file as per screen prints. That will save the file to the nvram.

     

    image.png

    • Upvote 1
    Link to comment
    Espressomatic
    3 hours ago, SimonF said:

    Did you go into boot maintenance and add the file as per screen prints. That will save the file to the nvram.

    OK, got it. I was doing something different than I did the first time I tested this out. Instead of Boot Options I immediately selected Boot From File and it boots only that one time.  Oops.

     

    Thanks for clearing it up.

    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×

  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.

    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.

×
×
  • Create New...