joecool169

Did you make the drive bootable and check out the tips here? https://wiki.unraid.net/USB_Flash_Drive_Preparation#If_the_Flash_will_not_boot.2C_read_over_the_following_tips

Take a close look at those cards you listed. Some of them are meant for internal connections and others have external connections, like to plug in to a disk shelf. The Unraid wiki is a good tool and the link posted above has some good info as well. Best to do some research before ordering.

From my experience western digital drives are usually sata inside. (I've only shucked a few) There is a hardware section in the wiki that lists compatibility for SAS cards. I run an LSI hba flashed in it mode. (Easily found on ebay, check "theartofserver" store for one, but there are many other sellers too. I don't run my cache in raid mode, that would depend on what you are going to store on there. See this video: Sent from my SM-N975U1 using Tapatalk

I'm not on threadripper but my daily driver is an unraid server with a 5950x 64GB Ram and a Rtx3090. I spend most of my time in an Arch linux vm. That vm has 2 usb controllers and a sata controller passed through to it as well as the gpu. I also have a windows 10 vm with the same gpu and USB passed through but 2 nvme drives instead of the sata controller. (So I only use 1 vm at a time, spaceinvader script to switch) My motherboard is Msi meg x570 unify and I think the iommu groupings are excellent. It has taken me much time and effort, lots of tinkering to get everything working the way I want. I play most games on high to ultra settings at 144fps, I don't play anything really new or demanding. If I were to have a complaint it's msfs 2020. In a populated area my fps dips down to 20. I'm pretty sure this is an issue with running in a virtual machine because on my old bare metal Intel 8700k I got 35fps+ in the same areas. All in all I am pretty happy with my setup, but it has been frustrating off and on getting everything to function the way I want. Sent from my SM-N975U1 using Tapatalk

I have one last suggestion for you, and I have no idea if this is relevant. My mobo bios allows me to enable or disable resizable bar, which I think is a fairly new technology. I'm on AMD. Mine is disabled. Any chance your mobo has that setting and have you checked it? It might be completely irrelevant or may not even be there, just thought I'd ask because sometimes someone gives me an off suggestion but it leads me to the right place.

As far as I know you only need to dump vbios for vm passthrough. Use spaceinvader's script, it is super easy.

And you have followed the instructions for CHR? Link

Have you tried "video=efifb:off" in the syslinux config?

If this is the wrong forum to ask in I apoligize I'm using IRSSI to try to join #unraid I log into chat.freenode.net and I get "-NickServ(NickServ@services.)- You are now identified for joecool169." But when I /join #unraid I get "#unraid Cannot join channel (+r) - you need to be identified with services - see https://freenode.net/kb/answer/registration" I'm a newbie I know. Am I missing something easy here?

Can you type exit and then find a boot option it likes? Sent from my SM-N975U1 using Tapatalk

I use MSI afterburner and it seems to accurately report my gpu temp. Sent from my SM-N975U1 using Tapatalk

Keep in mind that I am by no means an expert, but I think that Windows is crashing and then the gpu has not reset properly due to the crash. I would be trying to troubleshoot the cause of the crash, gpu temp, cpu temp, windows logs. There are a couple forum threads about switching slots and downgrading mobo bios for similar problems. I googled "internal error: Unknown PCI header type '127' for device" and then sorted to show only results from unraid.net

Are you monitoring temps when the problem occurs?

After much headache trying to figure this out I used spaceinvaders script and dumped my own vbios. This fixed the black screen issue. I can now access my vm bios by pushing esc at startup. I foolishly thought that because the vm somewhat worked and actually used to work with no vbios that this could not be my issue. I also thought that since I grabbed a vbios from tech power up that matched my card perfectly that was good enough. My advice : DUMP YOUR OWN VBIOS NO MATTER WHAT

Is this address your video card or another device?

Are you measuring frequency or usage? My vm seems to display usage ok, but frequency never updates. I get frequency from the unraid terminal.

This article has some nice info, but idk how much applies to gaming or your setup. https://www.kvm-switches-online.com/kvm-over-ip-guide.html I'm curious too so I'm following.

How about if you create a new vm and use the same vdisk? I've had that fix some issues for me. Also you need to specify the vbios file, simply binding the card to vfio does not provide a vbios.

Started a few weeks ago, all my vm's. I get a blank monitor with no signal until I get to the gui. I no longer see the titancore screen or the windows loading screen on startup. If I tell windows to boot into the uefi I just get no signal. When I start Arch Linux I have nothing until the password screen of the gui. The issue is I reinstalled Arch and now I can't really trouble shoot the passthrough because the screen doesn't work as soon as I add the gpu.

So I got this vm to start. I had to change the vdisk bus to sata and now it works perfect. I have no idea why that would have got changed though.

<?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Arch</name> <uuid>d16b5b50-c09b-dbd5-7276-d7545359fb38</uuid> <metadata> <vmtemplate xmlns="unraid" name="Arch" icon="arch.png" os="arch"/> </metadata> <memory unit='KiB'>16777216</memory> <currentMemory unit='KiB'>16777216</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>16</vcpu> <cputune> <vcpupin vcpu='0' cpuset='8'/> <vcpupin vcpu='1' cpuset='24'/> <vcpupin vcpu='2' cpuset='9'/> <vcpupin vcpu='3' cpuset='25'/> <vcpupin vcpu='4' cpuset='10'/> <vcpupin vcpu='5' cpuset='26'/> <vcpupin vcpu='6' cpuset='11'/> <vcpupin vcpu='7' cpuset='27'/> <vcpupin vcpu='8' cpuset='12'/> <vcpupin vcpu='9' cpuset='28'/> <vcpupin vcpu='10' cpuset='13'/> <vcpupin vcpu='11' cpuset='29'/> <vcpupin vcpu='12' cpuset='14'/> <vcpupin vcpu='13' cpuset='30'/> <vcpupin vcpu='14' cpuset='15'/> <vcpupin vcpu='15' cpuset='31'/> <emulatorpin cpuset='7'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-5.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/d16b5b50-c09b-dbd5-7276-d7545359fb38_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='8' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Arch/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x9'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0xa'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0xb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x8'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0xc'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/> </controller> <controller type='pci' index='8' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='8' port='0xd'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/> </controller> <controller type='pci' index='9' model='pcie-to-pci-bridge'> <model name='pcie-pci-bridge'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:f2:26:38'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x2d' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/EVGA.RTX3090.24576.201008.rom'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x2d' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x25' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x30' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> </domain>

Check out squid's solution here:

I experienced issues when I passed my dac through. (focusrite scarlett solo) The audio cut out and did strange things. I also had a web cam and a single usb controller seemed to run out of bandwidth when I used the webcam and the dac together. For me the solution was to pass through two usb controllers and keep those devices on separate controllers.

I don't know if this helps you any, but my vm always has that at the bottom if the log and yet it works fine. I read somewhere here that those aren't really errors.