snocrash

Hi! Onboard VGA vs Cards Initially, I had the onboard VGA hooked up to a monitor for the console display, with an additional monitor set up for the other cards. I would start up the VMs and connect them each to the monitor to see if they worked. That's how I was testing initially. Same result, one of them would work, the others were blank. Now, I disabled the onboard VGA, and used one of the cards in PCI slot7. When UnRaid boots up, I can see the console. If I want to start up a VM and use that card, it actually switches over nice and clean. But again, none of the other cards will work at that point. If I keep the console displayed on one of my cards, I can't start up a VM on any of the others. I believe I've worked through all the cards to make sure they all individually work. I've tried so many combinations of slots and cards and VMs at this point I wonder if I've just confused the system. OVMF vs SEA BIOS If I switch the VMs to OVMF, the results are slightly different. I do actually get a display on two screens simultaneously, but the boot-up hangs with an error message about the XORG SERVER not starting. I don't know the exact syntax of the message, but there is a console that is displayed before OE starts up - and both VMs will hang on that screen with an XORG Server error. This is actually the only time I have ever had multiple displays working at the same time. ROM I did actually try passing the ROM in the XML file. It gave an error on that one - I'll have to go back and capture the error, but I know it failed there. Here is the only thing that appears in the log when I start a VM: 2015-09-22 04:29:03.982+0000: starting up libvirt version: 1.2.18, qemu version: 2.3.0 Domain id=1 is tainted: high-privileges Domain id=1 is tainted: custom-argv Domain id=1 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0) Diagnostics I've attached the zip to this message. Hopefully it sheds some light. Thankful for any help I can get! tower-diagnostics-20150922-0031.zip

**Edit - 09-30-15 - Incompatible video cards? I've solved the issue with the VMs not working reliably. The Video Cards seem to be the culprit. My video cards are MSI branded Radeon HD 6450. When I combine more than one in the system I get unreliable VMs. I can barely get two to work simultaneously. I went through every BIOS setting and combination of settings I could think of and nothing resolved the issue. I bought several cheap video cards to test - a Saphire branded HD 6450, and a Gigabyte branded GT720. In my system currently, I have x1 MSI HD 6450, x1 Sapphire HD 6450, and x1 GT720. With all three of these devices I can create stock OE VMs for all three, launch all three, and so far I have stress-tested the GT720. I don't know what the issue is with my batch of MSI branded 6450s, but this combination seems to work really well so far. My only issue at the moment is that DTS-HD and TrueHD audio have a slight glitch / pop every few seconds when using audio-passthrough on OE on the GT720. I need to investigate and see whats going on there. I did add the sound.conf setting to fix the crazy audio glitch. That worked, and I get clean interface sounds, perfect DTS passthrough, and stable playback. Big thanks to JohnOdon for getting me started on this. I've gotten part of my setup working, but I've hit a roadblock and can't figure out what I've overlooked. I've built my system identically to what JohnOdon has, so I know that it is possible, but something is obviously wrong with mine. Specs I have a SuperMicro X8DTH-iF, which is a dual Xeon CPU, 7-PCIE-slot monster. I have x3 Radeon 6450 video cards, all brand new and identical. All show up as available devices, each in their own IOMMU groups. I've been able to test each individually, in a number of different slots on the board. I have UnRaid6 running and tried making the stock OpenElec VMs. What Works I am able to create and launch the first Open Elec VM, and can get full passthrough and everything works as expected on that VM. I can confirm proper passthrough because I am getting proper video acceleration in OE. Problem When I create the 2nd, or 3rd OE VM, those DONT work. I get a blank screen on each, with no logs or errors that I can find that would suggest something is wrong. This problem is the same with any VM type that I try. I have created multiple instances of Knoppix and the same situation occurs - only one will run and give me video. It seems as if the first card that spins up is the only one that will ever work until I reboot and reconfigure. I can shuffle cards around, but still, the first one to launch is the only one that will work. Config Each VM is set up as a SeaBios / Q35.2.3, each with their own CPU cores, and each with their own config directories. In my BIOS, I've tried just about every option in there, and none seem to improve this - at least I haven't found the right combination yet. I'd love to compare notes on BIOS settings with someone who has this board, if possible, just to see if the issue is there. I've looked at each XML, and they all point to what I believe is the correct video card. The XML looks like this: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>OE2</name> <uuid>62288c93-4dec-0994-b1ab-da08d6c22f73</uuid> <metadata> <vmtemplate name="OpenELEC" icon="openelec.png" os="openelec" openelec="5.95.3_1"/> </metadata> <memory unit='KiB'>524288</memory> <currentMemory unit='KiB'>524288</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <vcpupin vcpu='0' cpuset='12'/> <vcpupin vcpu='1' cpuset='13'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-2.3'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='2' threads='1'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/usr/local/emhttp/OpenELEC-unRAID.x86_64-5.95.3_1.img'/> <target dev='hda' bus='virtio'/> <readonly/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x05' function='0x0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0' multifunction='on'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </controller> <filesystem type='mount' accessmode='passthrough'> <source dir='/mnt/disk1/Share/OE2/'/> <target dir='appconfig'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> </filesystem> <interface type='bridge'> <mac address='52:54:00:77:f7:2f'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/OE2.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=86:00.0,bus=pcie.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=86:00.1,bus=pcie.0'/> </qemu:commandline> </domain> In this example, 86:00.0 and 86:00.1 are the video and audio portion of the card. 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] 04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series] 83:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] 83:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series] 86:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] 86:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series] My IOMMU groups look like this: /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/2/devices/0000:00:03.0 /sys/kernel/iommu_groups/3/devices/0000:00:05.0 /sys/kernel/iommu_groups/4/devices/0000:00:07.0 /sys/kernel/iommu_groups/5/devices/0000:00:09.0 /sys/kernel/iommu_groups/6/devices/0000:00:13.0 /sys/kernel/iommu_groups/7/devices/0000:00:14.0 /sys/kernel/iommu_groups/7/devices/0000:00:14.1 /sys/kernel/iommu_groups/7/devices/0000:00:14.2 /sys/kernel/iommu_groups/7/devices/0000:00:14.3 /sys/kernel/iommu_groups/8/devices/0000:00:16.0 /sys/kernel/iommu_groups/8/devices/0000:00:16.1 /sys/kernel/iommu_groups/8/devices/0000:00:16.2 /sys/kernel/iommu_groups/8/devices/0000:00:16.3 /sys/kernel/iommu_groups/8/devices/0000:00:16.4 /sys/kernel/iommu_groups/8/devices/0000:00:16.5 /sys/kernel/iommu_groups/8/devices/0000:00:16.6 /sys/kernel/iommu_groups/8/devices/0000:00:16.7 /sys/kernel/iommu_groups/9/devices/0000:00:1a.0 /sys/kernel/iommu_groups/9/devices/0000:00:1a.1 /sys/kernel/iommu_groups/9/devices/0000:00:1a.7 /sys/kernel/iommu_groups/10/devices/0000:00:1d.0 /sys/kernel/iommu_groups/10/devices/0000:00:1d.1 /sys/kernel/iommu_groups/10/devices/0000:00:1d.2 /sys/kernel/iommu_groups/10/devices/0000:00:1d.7 /sys/kernel/iommu_groups/11/devices/0000:00:1e.0 /sys/kernel/iommu_groups/11/devices/0000:07:04.0 /sys/kernel/iommu_groups/12/devices/0000:00:1f.0 /sys/kernel/iommu_groups/12/devices/0000:00:1f.2 /sys/kernel/iommu_groups/12/devices/0000:00:1f.3 /sys/kernel/iommu_groups/13/devices/0000:01:00.0 /sys/kernel/iommu_groups/14/devices/0000:01:00.1 /sys/kernel/iommu_groups/15/devices/0000:04:00.0 /sys/kernel/iommu_groups/15/devices/0000:04:00.1 /sys/kernel/iommu_groups/16/devices/0000:80:00.0 /sys/kernel/iommu_groups/17/devices/0000:80:01.0 /sys/kernel/iommu_groups/18/devices/0000:80:03.0 /sys/kernel/iommu_groups/19/devices/0000:80:05.0 /sys/kernel/iommu_groups/20/devices/0000:80:07.0 /sys/kernel/iommu_groups/21/devices/0000:80:09.0 /sys/kernel/iommu_groups/22/devices/0000:80:13.0 /sys/kernel/iommu_groups/23/devices/0000:80:14.0 /sys/kernel/iommu_groups/23/devices/0000:80:14.1 /sys/kernel/iommu_groups/23/devices/0000:80:14.2 /sys/kernel/iommu_groups/23/devices/0000:80:14.3 /sys/kernel/iommu_groups/24/devices/0000:80:16.0 /sys/kernel/iommu_groups/24/devices/0000:80:16.1 /sys/kernel/iommu_groups/24/devices/0000:80:16.2 /sys/kernel/iommu_groups/24/devices/0000:80:16.3 /sys/kernel/iommu_groups/24/devices/0000:80:16.4 /sys/kernel/iommu_groups/24/devices/0000:80:16.5 /sys/kernel/iommu_groups/24/devices/0000:80:16.6 /sys/kernel/iommu_groups/24/devices/0000:80:16.7 /sys/kernel/iommu_groups/25/devices/0000:83:00.0 /sys/kernel/iommu_groups/25/devices/0000:83:00.1 /sys/kernel/iommu_groups/26/devices/0000:86:00.0 /sys/kernel/iommu_groups/26/devices/0000:86:00.1 /sys/kernel/iommu_groups/27/devices/0000:fe:00.0 /sys/kernel/iommu_groups/27/devices/0000:fe:00.1 /sys/kernel/iommu_groups/28/devices/0000:fe:02.0 /sys/kernel/iommu_groups/28/devices/0000:fe:02.1 /sys/kernel/iommu_groups/28/devices/0000:fe:02.2 /sys/kernel/iommu_groups/28/devices/0000:fe:02.3 /sys/kernel/iommu_groups/28/devices/0000:fe:02.4 /sys/kernel/iommu_groups/28/devices/0000:fe:02.5 /sys/kernel/iommu_groups/29/devices/0000:fe:03.0 /sys/kernel/iommu_groups/29/devices/0000:fe:03.1 /sys/kernel/iommu_groups/29/devices/0000:fe:03.2 /sys/kernel/iommu_groups/29/devices/0000:fe:03.4 /sys/kernel/iommu_groups/30/devices/0000:fe:04.0 /sys/kernel/iommu_groups/30/devices/0000:fe:04.1 /sys/kernel/iommu_groups/30/devices/0000:fe:04.2 /sys/kernel/iommu_groups/30/devices/0000:fe:04.3 /sys/kernel/iommu_groups/31/devices/0000:fe:05.0 /sys/kernel/iommu_groups/31/devices/0000:fe:05.1 /sys/kernel/iommu_groups/31/devices/0000:fe:05.2 /sys/kernel/iommu_groups/31/devices/0000:fe:05.3 /sys/kernel/iommu_groups/32/devices/0000:fe:06.0 /sys/kernel/iommu_groups/32/devices/0000:fe:06.1 /sys/kernel/iommu_groups/32/devices/0000:fe:06.2 /sys/kernel/iommu_groups/32/devices/0000:fe:06.3 /sys/kernel/iommu_groups/33/devices/0000:ff:00.0 /sys/kernel/iommu_groups/33/devices/0000:ff:00.1 /sys/kernel/iommu_groups/34/devices/0000:ff:02.0 /sys/kernel/iommu_groups/34/devices/0000:ff:02.1 /sys/kernel/iommu_groups/34/devices/0000:ff:02.2 /sys/kernel/iommu_groups/34/devices/0000:ff:02.3 /sys/kernel/iommu_groups/34/devices/0000:ff:02.4 /sys/kernel/iommu_groups/34/devices/0000:ff:02.5 /sys/kernel/iommu_groups/35/devices/0000:ff:03.0 /sys/kernel/iommu_groups/35/devices/0000:ff:03.1 /sys/kernel/iommu_groups/35/devices/0000:ff:03.2 /sys/kernel/iommu_groups/35/devices/0000:ff:03.4 /sys/kernel/iommu_groups/36/devices/0000:ff:04.0 /sys/kernel/iommu_groups/36/devices/0000:ff:04.1 /sys/kernel/iommu_groups/36/devices/0000:ff:04.2 /sys/kernel/iommu_groups/36/devices/0000:ff:04.3 /sys/kernel/iommu_groups/37/devices/0000:ff:05.0 /sys/kernel/iommu_groups/37/devices/0000:ff:05.1 /sys/kernel/iommu_groups/37/devices/0000:ff:05.2 /sys/kernel/iommu_groups/37/devices/0000:ff:05.3 /sys/kernel/iommu_groups/38/devices/0000:ff:06.0 /sys/kernel/iommu_groups/38/devices/0000:ff:06.1 /sys/kernel/iommu_groups/38/devices/0000:ff:06.2 /sys/kernel/iommu_groups/38/devices/0000:ff:06.3

Are you using this with a GPU pass through setup? Not yet. Building out my new server currently. I've been using it with regular media center devices.

The HDBaseT kits from Monoprice are what you want. Get the box version of the sender unit and the wall plate version of the receiver. It has HDMI, IR and Ethernet. http://www.monoprice.com/Category?c_id=109&cp_id=10914 I have mine on a shielded cat6 run and it's flawless. I think you could get away with regular cat6 too.