after replacing GPU VM problem

[sjaak]

after i switched te GPU from a RX580 to an Vega64 is have problems with the VM's. at first i have create 2 new XML files because i couldn't edit the config (gifs errors like 'PCI adress' or 'header 127' (i dont remember the exact error), so i deleted the xml file and create new ones with the old vdisk. it work great but when i want to switches between the 2 vm's (Ubuntu / windows10) the screen wont get up. i then need to reboot the whole system, with i dont like.

i did have this problem with the RX580 but i did set in de syslinux config

pcie_aspm=off vfio_iommu_type1.allow_unsafe_interrupts=1 

but that wont work anymore.

when shutting down a vm it just goes down without any problem but when starting a vm it just start, but the screen wont come up. i dont see any errors in the syslog of vm log.

i only deleted the xml files, nothing more (i am not sure if it safe)

i attached the latest diagnostics.

 

lmfsv3-diagnostics-20181103-1016.zip

Edited November 3, 2018 by sjaak

[sjaak]

i had the log open when shutting down the vm, this come throug:

vfio-pci 0000:09:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none

vfio_ecap_init: 0000:09:00.0 hiding ecap 0x19@0x270
vfio_ecap_init: 0000:09:00.0 hiding ecap 0x1b@0x2d0

i didn't attach the GPU vbios rom in the xml file, maybe that is the problem?

Edited November 3, 2018 by sjaak

[sjaak]

found out that i use the wrong cores, so i used LStopo, after change the cores to the right one i got an error:

internal error: PCI host devices must use 'pci' address type

now i cant safe the xm file!

this is going to irritate me

Edited November 3, 2018 by sjaak

[bastl]

Your main issue is that the Vega cards have a reset issue. As soon as a VM shuts down the card isn't reset for later use, it stays in an idle state. The only solution in that case is to restart the host os. For some people a passthrough BIOS helped for others don't passthrough the sound device from the Vega card is a workaround. I guess you have to wait for the kernel with a fix for it. The 4.19 might have that already. Search the forum for "AMD GPU reset bug" or "Vega reset issue" and you will find a couple threads about this topic.

[sjaak]

oh, didn't know that :( have to wait for Unraid got updated to the latest kernel.

but it is a bit strange for me, edit an exited XML gifs errors, create a new XML works fine? :S

is the libvirt.img corrupt? can ii save delete it and recreate the xml file with the exiting Vdisk?

i just dump the vbios and added it to the "new" xml files, going for a another reboot...

[bastl]

Every new Unraid comes with new versions of libvirt and qemu. Everyone has it's own bugs. I currently can't switch the disk bus type from sata or ide to SCSI on a existing Q35 machine type VM via the gui. On i440fx it works. I don't think your libvirt.img is corrupt. If you manual edit the xml  be careful what you add and what you remove. Might be a typo that causes your 'internal error'. For better help you can post your XML that causes the issue and some diagnostics so if people wanna look into it have something to start with.

[sjaak]

after creating a new vm a have to do one manual edit because the Keyboard i use need some manual code:

    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <address bus='7' device='5'/>
      </source>
      <alias name='hostdev5'/>
      <address type='usb' bus='0' port='5'/>
    </hostdev>

that work fine, i can safe it and the vm work fine. but after shutting down it gifs errors. but now i think its because of the Vega bug...

(because when i set graphics to VNC i can safe it without problems)

 

edit: i just reboot the host with set the Vbios and the code i use in the syslinux config, i booted to windows and after shutting down i booted Ubuntu without problems, have to say i booted ubuntu immediately after win10 was down. ( i know it isn't fixed...)

Edited November 3, 2018 by sjaak
more info