ghost82

Yes, exact. Basically, Intel: VT --> intel virtualization technology This splits into VT-x and Vt-d VT-x: hardware virtualization assistance VT-d: allows direct passthrough (enables direct i/o for virtualization) vmx: virtualization extension flag in cpu You can run vms if hardware supports VT-x, even with VT-d disabled or not supported; you must have VT-d support for passed through devices. AMD: AMD-V: same as VT-x AMD-VI: same as VT-d svm: same as vmx On top of that if you emulate an intel cpu in qemu, that cpu must support also EPT (extended page tables) and UG (unrestricted guest): if you look at the first image of the first post, the user is having that issue because he was emulating a Penryn cpu (intel core 2 duo) that lacks these features. As far as I now enabling nested virtualization in a linux host, whatever it is, is not recent, same for VT-x/AMD-V, VT-d/AMD-VI and vmx/svm. Nested virtualization for intel was more or less always working for long time, amd was poorly supported by guest oses; linux guests were the firsts to fix this, then came microsoft, apple never did this and will never do for amd, although nested virtualization works for intel (passed through or emulated).

As far as I can see this mb has 2 pcie slots, 1 pcie 5.0x16 and 1 pcie 4.0x16 that should fit. Additionally it has 2 pcie slots x1 (the shorter ones). RTX 1063?You mean GTX 1060? The 1060 is a 3.0x16, the rx 560d is the same, 3.0x16. So you should be able to install that 2 cards in the 2 pcie slots 5.0x16 and 4.0x16 (5.0 and 4.0 are backward compatible with 3.0). Then, depending on the bios/iommu you should be able or not to passthrough that gpus to vms. Obviously, apart that 2 additional x1 slots you wont have any more pcie slot available.

I think this is because you are using cpu host-passthrough, but since it's an amd it doesn't have such feature (vmx). It could work if the cpu is intel emulated, like Skylake-Client-IBRS or something else supporting the vmx flag. What changed for amd nested virtualization compatibility is in windows itself, that added support for this, if I remember well starting when the first previews of windows 11 were available, then they were ported to windows 10 too. Anyway since it now works with amd host-passthrough, windows is able to make it work with svm (vmx equivalent for amd).

It can be 2 things: 1. bios bug: errors come from acpi tables, if the bios is bugged you can have that errors (you have bios 1.1, latest version should be 1.2, if you are confident you could try an update) 2. kernel bug/outdated: it may be fixed one day with a kernel update _CPC is continuous performance control related tocpu: with cpc enabled the system will use for amd cpus amd-pstate function, otherwise it will fall back to acpi legacy P-states. On intel it enables futures like speed shift (p-states) without consulting acpi. Having that errors should not compromise at all your experience.

Sorry to ask, but are you sure that this graphic adapter is the qxl/vnc one? I'm asking for 2 reasons: 1. in your Windows 10 vm there is only the amd gpu defined, no qxl/vnc 2. when you define qxl/vnc virtual gpu it shouldn't appear with the yellow mark on it Every passed through gpu without its drivers will appear as a basic graphic microsoft adapter because the microsoft driver is attached to it: this can work as a basic adapter or not (as in your case); if it doesn't work you should install the drivers of that gpu. Double check the property of that graphic adapter and see what it is and why it's not running properly, what error outputs? Gpu is isolated (OK) and in its iommu groups (OK), it seems not in use by the host (OK), if it's still doesn't work, change the settings in the xml to it becomes a multifunction device, from this: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x28' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x28' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> to this: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x28' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x28' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/> </hostdev> You may need to pass also the vbios, dump it and pass the rom file too. You may need to change (re-create) the machine type, from i440fx to pc-q35 for better passthrough compatibility.

Sure but you need to attach diagnostics, did you install the graphics driver inside the vm?

just to eat your hands...: https://forums.unraid.net/topic/124191-non-native-display-resolution-with-intel-uhd-graphics-630-graphics-passthorugh-max-2048x1080-instead-of-2560x1440/?do=findComment&comment=1132883 but that's on linux guest..

That's gvt-d, direct access, lucky you that works I have no explanation for the resolution, sorry, I would try to add it manually. PS: can you share your xml for that vm, or even better diagnostics?Another user is struggling to make his uhd630 passthrough without success...

yes, or you can give a try with unraid 6.9.2, following steps 1,2 and for step 3 use only this: <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.ua-igpu.x-igd-opregion=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.ua-igpu.x-igd-gms=1'/> </qemu:commandline>

In real there's nothing really difficult, in this mode you need: In a q35 machine, which has additional bus numbers other than 0x00 (pcie-root-ports): 1. primary gpu (qxl+vnc) on bus 0x00, i.e.: <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <audio id='1' type='none'/> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </video> 2. passthrough of the igpu with vbios, source address 00:02.0, target address with an address different than 00:02.0, and maybe in a bus different than 0x00, i.e. (target in this ex is 05:00.0): <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/i915ovmf.rom'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> <alias name='ua-igpu'/> </hostdev> with an alias added to it (ua-igpu for example) 3. Add opregion and gms with qemu arguments to enable video output, pointing to the igpu alias: <qemu:override> <qemu:device alias='ua-igpu'> <qemu:frontend> <qemu:property name='x-igd-opregion' type='bool' value='true'/> <qemu:property name='x-igd-gms' type='unsigned' value='1'/> </qemu:frontend> </qemu:device> </qemu:override> This for libvirt 8.2.0+, for older versions it was the -set argument: <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.ua-igpu.x-igd-opregion=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.ua-igpu.x-igd-gms=1'/> </qemu:commandline> But (3) for libvirt 8.2.0+ as said is not working correctly at the moment and (3) for libvirt <8.2.0 may require disabling device.json in qemu (quite a mess to follow depending on libvirt/qemu versions), with this additional block: <qemu:capabilities> <qemu:del capability='device.json'/> </qemu:capabilities> Of course this is only theory for me..

I don't think so, because when you passthrough a (i)gpu to a guest all should be managed by the guest and the kernel of the host should have no impact. By "passing through the igpu" you mean gvt-d with a monitor connected to the vga output directly or something else?

totally makes sense, never heard of any packet manipulation with emulated network controllers, they should act as real ones.

it seems no output, ich777 porposes a displaylink device for video output: https://forums.unraid.net/topic/108650-plugin-intel-gvt-g/?do=findComment&comment=994459 Official documentation says: So primary card could be emulated with qxl (on bus 0x00), and with upt target address of igpu will be different than 00:02.0 then it adds: adding x-igd-opregion=on was something like this in the xml: <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-igd-opregion=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-igd-gms=1'/> </qemu:commandline> with hostdev0 being an example of the alias of the igpu (so an alias for the passed through igpu must be defined in the xml). But this worked on older version of qemu/libvirt. the -set commandline has been deprecated by libvirt. In newer unraid version which includes libvirt 8.2.0 and qemu 6.2 there should be <qemu:override>, but it seems it doesn't work properly at the moment: https://forums.unraid.net/bug-reports/stable-releases/6100rc7-6102-odd-behaviour-of-libvirt-r1958/ ---> issue n.2

About this: the owner of the repository stated that he also was receiving this error, but the vm boots: You could also try to boot with vnc/qxl, install remote desktop inside windows, stop all, remove vnc/qxl, add igpu on target 00:02.0 (q35/i400fx + ovmf), connect to remote desktop, see what is happening and try to install intel video drivers.

It seems it's becoming challenging and without having tried directly I'm quite shooting in the dark. Some words about what we are doing: your igpu could be passed through in 2 ways: 1. Universal Pass-Through (UPT) The igpu is attached to the vm as a secondary gpu, without any video output; video output could be obtained with x-igd-opregion (experimental) setting in the vm. In this case you need another primary gpu, like qxl video driver. 2. Legacy mode This should be better, because you should have video output and hardware acceleration; target address of the igpu in the vm must be bus 0x00, slot 0x02, function 0x00. What we are trying to do is gvt-d (direct access). The i915ovmf.rom comes from: https://github.com/patmagauran/i915ovmfPkg A note: You have some dmar errors in the log, but accordingly to the wiki: so, it should be good.. On that github you can download the release of the bios: https://github.com/patmagauran/i915ovmfPkg/releases Inside you will find 2 rom files, what I attached before is the NOT "simple" file. In real I don't know what "simple" means but trying also the simple rom worth a try. Unfortunately, I'm out of ideas. You may want to have a read at gvt-g too: https://forums.unraid.net/topic/108650-plugin-intel-gvt-g/#comment-993616

I'm not saying there's nothing to do to make it better, but: 1. the network "issue" can be fixed, you wont have an unbootable system 2. there are ways and ways to ask things, locking caps and telling users to not update is not the correct method 3. one should always backup the flash drive, limetech writes it everytime in its release notes 4. one should be prepared for bugs, even important: how many times you updated a linux or windows machine and find odd things? 5. one should know how to roll back, whatever os is in use So, the issue with network? Easy fix in some minutes, rollback to previous version, the same. Then complain how much you want, use cap locks and tell users to not upgrade because the system is faulty (without even knowing why), most probably you will be ignored. btw, in the release notes, even if not extended there are info about the blacklist of tg3 One writes things (release notes) not because he has to spend some time because he has nothing to do.

There are no vm defined in your latest zip. Please define vms and enable also syslog server: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#comment-781601 so if the system crashes you will have the syslog saved in the logs folder of the flash drive Attach diagnostics again and that syslog

mmm odd...try to create a i440fx machine type with ovmf, then use the same vbios. After that reattach diagnostics.

Try this, for vm "Windows 11 ovmf q35": 1. Replace xml with this: <!-- WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE OVERWRITTEN AND LOST. Changes to this xml configuration should be made using: virsh edit 1cf47542-bffa-f1a7-50dc-5b1d2ab4250a or other application using the libvirt API. --> <domain type='kvm'> <name>Windows 11 ovmf q35</name> <uuid>1cf47542-bffa-f1a7-50dc-5b1d2ab4250a</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/> </metadata> <memory unit='KiB'>5242880</memory> <currentMemory unit='KiB'>5242880</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='2'/> <vcpupin vcpu='1' cpuset='8'/> <vcpupin vcpu='2' cpuset='3'/> <vcpupin vcpu='3' cpuset='9'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='10'/> <vcpupin vcpu='6' cpuset='5'/> <vcpupin vcpu='7' cpuset='11'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-6.2'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader> <nvram>/etc/libvirt/qemu/nvram/1cf47542-bffa-f1a7-50dc-5b1d2ab4250a_VARS-pure-efi-tpm.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv mode='custom'> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='4' threads='2'/> <cache mode='passthrough'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 11 ovmf q35/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win11_EnglishInternational_x64v1.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.217-1.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x18'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x19'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x1a'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x1b'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-to-pci-bridge'> <model name='pcie-pci-bridge'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x1c'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:fb:4d:6f'/> <source bridge='br0'/> <model type='virtio-net'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='3'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <tpm model='tpm-tis'> <backend type='emulator' version='2.0' persistent_state='yes'/> </tpm> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/i915ovmf.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc05a'/> </source> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x05ac'/> <product id='0x0267'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </memballoon> </devices> </domain> 2. Download the attached zip, extract and save i915ovmf.rom into /mnt/user/isos/vbios/i915ovmf.rom Note: there's no vnc, leave only the passed through igpu, attach a monitor to the igpu. Try and pray (optional, but recommended) i915ovmf.rom.zip

Waiting for diagnostics, it will be a lot easier to explain.

Do not bother for that "error", it's a warning, for amd edac_mce_amd is loaded correctly in unraid, but you still receive the mce "error"; mce errors will be correctly logged in the syslog. It's an old bug in mce. May 24 23:27:19 NanoStorage kernel: mce: [Hardware Error]: Machine check events logged May 24 23:27:19 NanoStorage kernel: [Hardware Error]: Corrected error, no action required. May 24 23:27:19 NanoStorage kernel: [Hardware Error]: CPU:1 (19:21:0) MC9_STATUS[-|CE|-|-|-|-|-|-|-]: 0x8000000271c31163 May 24 23:27:19 NanoStorage kernel: [Hardware Error]: IPID: 0x0000000000000000 May 24 23:27:19 NanoStorage kernel: [Hardware Error]: L3 Cache Ext. Error Code: 3, L3M Tag Multi-way-hit Error. May 24 23:27:19 NanoStorage kernel: [Hardware Error]: cache level: L3/GEN, tx: INSN Temperature problem, thermal paste, not enough cooling, dust in fans Cpu failing Motherboard failing

Have you contacted limetech?they have a contact form. Buying 3 lifetime licenses if you use only one it's definitely NOT the way to go!!

Thank you for explaining, now I understand: what he's doing is using a modified bios of an "old" gpu that didn't include efi; older gpus have only legacy bios but for some of them you can modify the vbios and inject also the efi portion, so that they can be used on uefi systems. But this is not your case, the vbios of your igpu already include efi. yes, efifb:off disable the efi framebuffer. If unraid is booted in uefi mode it may attach the efi framebuffer to the primary gpu; if efifb is attached vfio could not reserve bar and you will get black screen or garbage screen, depending on the case. Yes, this is ok, when booting some lines will be shown on the screen, but as soon as it attaches the gpu to vfio the screen will look like if it's frozen. This is your video portion passthrough: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/intel-igpu2.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </hostdev> As I wrote earlier you need also to bind to vfio and pass to the vm the audio portion. I would suggest to use the more modern q35 machine type instead of i440fx, other than that I'm out of ideas. You can try to build a new q35 vm, after that attach diagnostics file to see if we can help more. igpu vbios should be about 65 kB, so it should be good (but if you want to be sure just check the file content with a hex editor).

English is not my native language but: Maybe you missed it.