Skitals

It took some tinkering to get things stable but here are the final results for Time Spy Bare metal: 3dmark: 9 871 graphics: 9 488 cpu: 12 805 q35-4.0.1: 3dmark: 9 760 graphics: 9 487 cpu: 11 671 It is virtually (haha) identical when you consider the kvm is only running on 20 out of 24 cpu threads.

I, too, am looking to scratch that 6.9.0-rc1 itch.

I will update for 6.9 rc1 when it's out. 6.8 final reverted to a 4.x kernel. 6.9rc1 will be identical to 6.8 final with the addition of the 5.x kernel.

Yay! Bring on 6.9.0-rc1

How long is "a period of time" before it crashes your server? I just got a 5700 XT. I had to switch my win10 vm to q35 for the adrenalin drivers to work. I've experienced 2 hard crashes so far. I wasn't sure if it was due to the 5700 XT, or the change to q35. I just updated to the brand new Adrenalin 2020 drivers. The 2020 drivers are still a no-go for i440fx, but it's working the same with Q35.

build 2020.01.21: 6.8.0-rc5 kernel with navi reset v2 vega reset patch* (used in conjunction with this script) k10temp Ryzen patches from 1/18/20 (new) Fix for onboard audio passthrough on x570 *modified to add 0x6863 in addition to 0x687f 6.8.0-rc5-Skitals-2020.01.21.zip

Just ran 3dmark Time Spy, here are my refults: https://www.3dmark.com/3dm/41799540 Gigabyte 5700 XT Gaming OC, Ryzen 3900X (passing all but first 2 cores), 16 out of 32GB ram, Windows installed on a 1TB Gen4 m.2 NVME (passing through nvme controller, can also boot Win10 bare metal). Machine type set to Q35-4.0.1. I think my results are pretty good! Getting better numbers in a VM than reviews for this specific vendor card. Edit: Adrenalin 2020 drivers came out, seeing a nice bump in performance: http://www.3dmark.com/spy/9630130 3dmark score 9252 -> 9761 Graphics score 8986 -> 9533

I just so happened to buy a 5700 XT since my last post. It works fine. It passes fine with vbios. The latest Adenaline drivers don't work with i440 chipset (you will get a black screen at 40% when installing the drivers). I switched to Q35 and it works fine with the drivers. If you already have the amd drivers installed, you will need to use Q35. The navi reset patch v1 seems to work with this hardware. I am on 6.8.0-RC5 and using the custom 5.3.8 kernel here for the reset patch: I have noticed some bugs with HDMI and legacy boot on this card/mobo combo. I'm waiting for a new displayport monitor to come in, so I can't confirm if the issue is present on displayport as well. I get the "no signal" bug if the bios is set to CSM Enabled (legacy boot support) AND the card is in a certain state. The biggest issue is that there is a somewhat documented CSM bug with this bios where it reenables itself. My windows and unraid installs are both UEFI, but for some reason something is triggering CSM to become reenabled. I think its the 5700 XT switching between BIOS and UEFI boot. When the state of the motherboard and the gpu get out of sync, the only way to reset it and get a signal for the bios is the turn off the computer, turn off the power supply, unplug the power supply, unplug the hdmi, then reconnect everything and turn the computer back on. When this happens, the motherboard has always reverted to CSM Enabled. Interestingly, if I DON'T need to get into the bios, I can just let Unraid boot with "no signal" and when I fire up a vm the card/signal comes to life. Occasionally I will get "no signal" when starting a VM, OR it will be super low res. Turning the monitor off and back on fixes this. There is some kind of handshake not happening. This is all frustrating since I have ZERO issues with my 1070 to a display connected with DVI. I'm hoping that all these issues are somehow because my current monitors are ancient and don't support more modern hdmi (let alone displayport) standards. The takeaway here is: Disable CSM in the bios. Use the kernel above with the reset patch. Use Q35 chipset for your Win10 VM. Use UEFI/OVMF for everything (both bare metal and kvm). Use displayport over hdmi. Don't use an ancient display. The first 4 recommendations solved 95% of my problems, hopefully the new monitor with displayport solves the last 5%.

Anyone have any luck passing a 5700 XT in 10.15.1?

Start from scratch. Download the Unraid USB Creator. For version select Next -> 6.8.0 RC5. Click customize. Check Allow UEFI boot. Insert your new unraid usb into the white port on your motherboard labeled BIOS. Boot your computer into the bios. Reset all settings to factory optimized/default settings. Save and reboot back into the bios. Make ONLY these changes: Tweaker -> Advanced CPU Settings -> SVM Mode -> Enable Settings -> Miscellaneous -> IOMMU -> Enable Settings -> AMD CBS -> ACS Enable -> Enable Settings -> AMD CBS -> Enable AER Cap -> Enable Save and reboot. Assuming you don't have any other boot media attached it will boot unraid. From the web gui Main menu, click the name of your Boot Device (flash). Under Syslinux Config -> Unraid OS, add "video=efifb:off" after "append initrd=/bzroot". The line should now read "append initrd=/bzroot video=efifb:off". Reboot unraid. Create a new vm using win10 template. Use the latest win10 install iso and virtio driver iso. Use the Q35 chipset, and the rest default/usual. Get windows installed/working with vnc graphics. Download appropriate vbios from https://www.techpowerup.com/vgabios/. Pass it along with your gpu video/sound. If you haven't seen it already, spaceinvader one has a video specific for the 5700 XT. The long and short is you need to pass a vbios, and you need to use Q35 to get the drivers to install properly. I would try doing everything fresh and in the order I suggested.

Did you read the first post? Did you try the boot flag video=efifb:off as indicated? That is not in the SpaceinvaderOne guides and required for this board, hence this thread. You do not need legacy mode, uefi works fine. This is confirmed working on both Nvidia and amd cards. You should not need to pass a vbios with an amd card.

I haven't tested it, but I believe you have to use either one method or the other. Try removing vfio-pci.ids from your syslinux.cfg. I'm passing the same exact nvme controller via vfio-pci.cfg, so I can confirm that works as well (XPG 1TB PCIe Gen 4 NVMe, in my case. Picked it up on Black Friday for $120. Getting the rated 5000/4400MB/s).

It doesn't work with this board because there are three usb controllers with the same vendor:device id (1022:149c). One of them has the unraid usb plugged in, and it obviously isn't happy when stubbed. Using the vfio-pci.cfg method lets you bind some of them but not others. Try safe mode, or try removing your vfio-pci.ids from syslinux.cfg. I'm not sure you can bind both ways. You should be able to bind everything you need via the vfio-pci.cfg method. The only annoying part is when you add/remove a pcie device the addresses change. I just added another nvme drive to my system and had to update my vfio-pci.cfg and all my VMs.

I see other (old) bug reports about vfio-pci.cfg not loading, but I don't see an immediate solution. What version of unraid are you running, and what does your syslinux.cfg look like? Try booting unraid in safe mode (essentially bypassing any changes you made to the default boot parameters) and see if the binding works.

What version of unraid are you running? That looks like the correct path and filename. What does your syslinux.cfg look like?

Picked up a pulled Dell PERC H310 for $19 shipped on ebay, crossflashed it on my x570 board w/ UEFI using this guide: https://ciamician.wordpress.com/2015/03/06/flashing-your-dell-perc-h310-to-it-firmware-uefi/ Easy peasy!

The Aorus boards are on sale at Microcenter and I was within my 30-day return period so I just upgraded to the x570 Aorus Ultra. The IOMMU groups are identical. In fact, I didn't have to change a single thing to get back up and running. The USB controllers are split the same way, with the same 4 ports integrated in the cpu (the ones we can't pass). That means the rear USB Type-C passes (along with the type-c header)! Going from Aorus Elite -> Ultra nets me: +1 m.2 slot (3 total!) +1 PCIe x16 slot (@x8) +1 USB Type-C port (-1 type A) Realtek® ALC1220 -> Realtek® ALC1220-VB (haven't succesfully passed this) Bluetoooth 4.2 -> 5.0 (passes with the usb controllers) Wi-Fi 802.11a/b/g/n/ac -> Wi-Fi 802.11a/b/g/n/ac/ax (no use for this right now) Some other fun stuff, like 2 temp sensor headers (with included thermistors)

I made a thread on the X570 AORUS here. Single GPU passthrough, two usb controllers passthrough, cpu temps, fan speed/controller, etc. All without ACS overrides. Only thing I haven't gotten to work is onboard audio passthrough.

Note that I can not pass the on board audio even though it's in it's own group. I've always used a (USB) Astro Mixamp so it's no loss to me.

Here they are with F10 bios and above settings. The Fresco Logic FL1100 is a PCIe card.

I've spent a few weeks getting my X570 AORUS Elite WiFi + 3900X + GTX1070 running to my liking so I thought I would share. These settings are also confirmed working on the AORUS Pro WiFi and AORUS Ultra. It will probably be similar for all the X570 AORUS boards. Here are the settings for USB passthrough, single Nvidia GPU passthrough, and more. Using BIOS F10. This is important as your IOMMU groupings can/will change with AGESA updates. UEFI / BIOS Settings: Tweaker -> Advanced CPU Settings -> SVM Mode -> Enable Settings -> Miscellaneous -> IOMMU -> Enable Settings -> AMD CBS -> ACS Enable -> Enable Settings -> AMD CBS -> Enable AER Cap -> Enable USB Passthrough: Leaving PCIe ACS override disabled, you should have ~34 IOMMU groups (give or take depending on how many PCIe devices you have connected) if you look in Tools > System Devices. There should be 3 USB controllers with the same vendor/device ID (1022:149c). Two of them will be lumped together with a PCI bridge and "Non-Essential Instrumentation." Those are the two we want to pass! The more logical option would be the controller isolated in its own group, but I could NOT get that one to pass. The trick is run your Unraid USB off that third controller, and we can pass the other two controllers together. Run your Unraid USB out of the rear white USB port labeled BIOS. That white USB 3.0 port plus the neighboring 3 blue USB 3.0 ports share a controller. Use these other ports for your keyboard and mouse (to be passed through as devices) and your UPS or whatever else you want Unraid to access. Note the addresses of the two USB controllers AND the "Non-Essential Instrumentation" in that IOMMU. In my case they are 07:00.0, 07:00.1, 07:00.3. Create the file /boot/config/vfio-pci.cfg with the following contents: When you reboot, these devices will available in the vm xml gui to passthrough under Other PCI Devices. Pass all 3 of them together! If you do not pass the "Non-Essential Instrumentation" Unraid will throw a warning in the logs that the .1 controller is dependent on it and unavailable to reset. When you passthrough all three together you will get no errors/warnings and everything works. Bonus: Bluetooth on this board is a usb device tied to the .3 controller and is passed through along with the controller! Note: When you add or remove PCIe devices, these addresses can/will change. When you add or remove a PCIe device, check Tools > System Devices to see if the USB addresses have changed and update vfio-pci.cfg accordingly. Single (NVIDIA) GPU Passthrough: For single GPU passthrough, you need to disable graphical output in Unraid. From the Main menu, click the name of your Boot Device (flash). Under Syslinux Config -> Unraid OS, add "video=efifb:off" after "append initrd=/bzroot". The line should now read "append initrd=/bzroot video=efifb:off". When you reboot you will notice there is no video output when unraid boots (you will be left with a freeze frame of the boot menu). Your solo GPU is now ready to pass. For Nvidia you will need the vbios for your card. I dumped my own following this tutorial using second gpu. If you can't dump your own, trying following this tutorial to download/modify a working vbios. Now simply pass your GPU, vbios, and the Sound Card that goes with your GPU from the vm xml gui. Fan Speed Sensors and PWM Controllers: See warning below! You can already see your CPU temp (Tctl) using the k10temp driver with Dynamix System Temperature. If you want to see Fan Speeds on your dashboard, or use the Dynamix Auto Fan Control plugin, we can force the it87 driver to load for the it8628 on this board. To force this we need to set another boot flag, "acpi_enforce_resources=lax". Add this the same way as above, after "video=efifb:off". That line in your syslinux.cfg should now read "append initrd=/bzroot video=efifb:off acpi_enforce_resources=lax". Next, add the following line to /boot/config/go The it87 driver will now load on boot and your fan speeds will be displayed on the Unraid dashboard, and the fan controllers will be available in Dynamix Auto Fan Control. Warning: Setting acpi_enforce_resources to lax is considered risky for reasons explained here.

To answer my own question, I went ahead and picked up a FL1100 usb card. Passed it through to MacOS and my usb audio works fine now. This is the card I went with: https://www.amazon.com/gp/product/B00B6ZCNGM/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

I created the file /boot/config/vfio-pci.cfg that simply contains: Any device you list there is stubbed and available to select under "other pci devices" on the vm xml gui. My understanding is this method supercedes pci-stub and vfio-pci boot flags. Note that if you add or remove pci devices to your system the addresses can/will change. I had to update my vfio-pci.cfg and vm xmls when I pulled my second gpu.

I'm passing through two onboard usb controllers from my x570 aorus elite wifi, it seems to work natively in High Sierra. I can plug in my usb astro mixamp or usb-c pixel buds, and they are recognized as audio devices. But the audio through either is garbled/clicky. I don't know where to begin, do you think it's a usb problem, or an audio problem? My understanding was if you successfully pass through a usb controller, a usb audio device should just work. All these controllers and devices work fine in my win10 vm. Any suggestions are appreciated, and I appreciate the great work!

For my x570 Aorus Elite Wifi in BIOS I had to enable ACS Enable and Enable AER Cap to get decent IOMMU grouping. After that I STILL had to set PCIe ACS override to "both" in Unraid. I was able to pass 2 out of 3 usb controllers on my board, including the front panel gen 2 USB Type C! Initially I only had one controller in it's own IOMMU group. That controller would not/could not pass, it would lock up Unraid when I started the VM. After the above changes, same story for that controller. I switched my Unraid usb to a port on that controller. The other two controllers (now that they are isolated) pass fine! I'm using /boot/config/vfio-pci.cfg to bind those PCI IDs. The USB controllers now show up under "Other PCI Devices" so I can pass them to VMs without editing any XML.