frodr Posted February 26, 2020 Share Posted February 26, 2020 SIO is saying do not brake up IOMMU groups unless you have to. Any chance to get the RTX 2080 working: IOMMU group 73:[10de:1e07] af:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti Rev. A] (rev a1) [10de:10f7] af:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1) [10de:1ad6] af:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1) [10de:1ad7] af:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1) In the video Advanced GPU passthrough SIO show xml editing. My xml says different bus, not virtual slot. source> <address domain='0x0000' bus='0xaf' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0xaf' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </hostdev> This is the error message as is, with braking up IOMMU groups. After ACS override, Multifunction, the kvm start but with black screen. My questions are: 1) Is it possible/worth while setting GPU without braking up IOMMU groups? 2) If not, how to fix the black screen? This is said to happened when Nvidia is primary GPU. But, in my server this GPU is not the only GPU. An Aspeed GPU is in a PCIe slot and set as onboard in bios. IOMMU group 27:[1a03:1150] 01:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 04) [1a03:2000] 02:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41) Cheers, Frode Asus C621E Sage/BMC | dual Xeon Silver 4116 | Samsung 4 x 32GB DDR4-2933 RDIMM | 4 x Corsair Force MP510 (960GB) NvME ssd on Asus Hyper M.2 Card V2 l | 4 x WD and Seagate hdd l Sapphire Pulse Radeon RX 580 8GB as Mac WS l ASUS GeForce RTX 2080 Ti ROG Strix Advance in gaming kvm l Samsung 970 Pro in M.2 port for gaming kvm l Asmedia PCIe USB card WinPro kvm xml.rtf Server System Devices.rtf Quote Link to comment
testdasi Posted February 26, 2020 Share Posted February 26, 2020 The RTX 2080 has FOUR devices (GPU, HDMI audio, 2x USB devices i.e. everything you see in IOMMU group 73) that must be passed through together for it to work, regardless of IOMMU grouping (i.e. regardless of ACS Override). Your xml only includes the GPU and HDMI audio and is missing the other 2 devices. Go to the app store and look for VFIO PCI Cfg plugin and install it. Then Settings -> VFIO PCI Cfg -> tick all 4 devices in group 73 -> build VFI-PCI.cfg -> reboot. The 2 USB devices should now show up in the Other PCI Devices section in your VM template for you to select. Quote Link to comment
frodr Posted February 26, 2020 Author Share Posted February 26, 2020 2 hours ago, testdasi said: The RTX 2080 has FOUR devices (GPU, HDMI audio, 2x USB devices i.e. everything you see in IOMMU group 73) that must be passed through together for it to work, regardless of IOMMU grouping (i.e. regardless of ACS Override). Your xml only includes the GPU and HDMI audio and is missing the other 2 devices. Go to the app store and look for VFIO PCI Cfg plugin and install it. Then Settings -> VFIO PCI Cfg -> tick all 4 devices in group 73 -> build VFI-PCI.cfg -> reboot. The 2 USB devices should now show up in the Other PCI Devices section in your VM template for you to select. Thanks, brilliant little plugin. Disabled ACS override, ticked all devices in Group 73, marked green. Kvm starts, but still black screen. Can it be that the motherboard sees the Nvidia GPU as main GPU? Quote Link to comment
Chess Posted February 26, 2020 Share Posted February 26, 2020 9 minutes ago, frodr said: Thanks, brilliant little plugin. Disabled ACS override, ticked all devices in Group 73, marked green. Kvm starts, but still black screen. Can it be that the motherboard sees the Nvidia GPU as main GPU? Are you passing through the vBIOS to the VM as well? If you are, pull a new vBIOS from the card and try that. I had a lot of issue getting a working vBIOS for my 1070, but once I got it the black screen issue went away. Quote Link to comment
frodr Posted February 26, 2020 Author Share Posted February 26, 2020 Are you passing through the vBIOS to the VM as well? If you are, pull a new vBIOS from the card and try that. I had a lot of issue getting a working vBIOS for my 1070, but once I got it the black screen issue went away.No, I haven't. Do I have to do the "hex editor thing" explained by SIO?Sent from my iPhone using Tapatalk Quote Link to comment
Chess Posted February 26, 2020 Share Posted February 26, 2020 1 minute ago, frodr said: No, I haven't. Do I have to do the "hex editor thing" explained by SIO? Sent from my iPhone using Tapatalk If you have a second video card to boot from you can just dump the vBIOS from the card. If not, yes that is the other way to do it. That is how I did it, but I had a spare PCIe video card to boot from. Quote Link to comment
frodr Posted February 26, 2020 Author Share Posted February 26, 2020 (edited) I downloaded the vbios from Techpowerup, and added a line in the xml as shown in SIO`s video. Still black screen. Then I passthrough the ASPEED graphics, still black screen. VNC was working. // Frode Edited February 26, 2020 by frodr Quote Link to comment
david279 Posted February 26, 2020 Share Posted February 26, 2020 Did you edit the vbios to remove the nvidia header? Quote Link to comment
frodr Posted February 26, 2020 Author Share Posted February 26, 2020 Just now, david279 said: Did you edit the vbios to remove the nvidia header? No, I didn't. Scared of everything that starts with "hex.....". That's next. It's like the first bath in the Summer. Terribly cold at first, and very nice if you stay long enough. Quote Link to comment
Chess Posted February 26, 2020 Share Posted February 26, 2020 4 minutes ago, frodr said: No, I didn't. Scared of everything that starts with "hex.....". That's next. It's like the first bath in the Summer. Terribly cold at first, and very nice if you stay long enough. You can't really break anything here by doing it. Worst case maybe hang unraid, but more than likely all you'll get is the black screen again. Quote Link to comment
frodr Posted February 26, 2020 Author Share Posted February 26, 2020 I have modded the vbios downloaded from Techpowerup. Hopefully the correct version. Still black screen. I tried booting with a AMD card. It starts, but black screen also with this card. That shouldn't happened?? Quote Link to comment
Chess Posted February 26, 2020 Share Posted February 26, 2020 2 minutes ago, frodr said: I have modded the vbios downloaded from Techpowerup. Hopefully the correct version. Still black screen. I tried booting with a AMD card. It starts, but black screen also with this card. That shouldn't happened?? You have an AMD card in and the Nvidia one in too? If so, dump the vBIOS from your Nvidia one with the AMD card in as boot. I never was able to get a techpowerup vBIOS to work after editing. Nvidia seems to always need a vBIOS, unless I'm mistaken. Quote Link to comment
frodr Posted February 26, 2020 Author Share Posted February 26, 2020 You have an AMD card in and the Nvidia one in too? If so, dump the vBIOS from your Nvidia one with the AMD card in as boot. I never was able to get a techpowerup vBIOS to work after editing. Nvidia seems to always need a vBIOS, unless I'm mistaken.But it is black screen with the AMD as well. Sent from my iPhone using Tapatalk Quote Link to comment
Chess Posted February 26, 2020 Share Posted February 26, 2020 Are you booting with both cards? Which card is unraid booting to? If you can run both cards, dump the vBIOS from the Nvidia. This is what I had to do to get the Nvidia to work at all. I've not tried an AMD card, so can't really help there. But I believe they don't need a vBIOS, but some of them have issues called the AMD reset bug. Quote Link to comment
frodr Posted February 27, 2020 Author Share Posted February 27, 2020 I tried the AMD GPU, still black monitor. Setting this GPU back to the Mac kvm I can passthrough, but MacOS do not recognise the GPU. Tried with and without VFIO-PCI Config as well at VNC/passthrough and restarting the server. I VFIO-PCI Config it says the AMD GPU is reset compatible. Anyone knows how to reset an AMD GPU? Quote Link to comment
Chess Posted February 27, 2020 Share Posted February 27, 2020 11 minutes ago, frodr said: I tried the AMD GPU, still black monitor. Setting this GPU back to the Mac kvm I can passthrough, but MacOS do not recognise the GPU. Tried with and without VFIO-PCI Config as well at VNC/passthrough and restarting the server. I VFIO-PCI Config it says the AMD GPU is reset compatible. Anyone knows how to reset an AMD GPU? There was a script running around to reset the card, but I'm not sure that is 100% needed anymore. Still getting an NVidia card to pass through is easier. You just need the right vBIOS. Can you not boot the system with the two cards installed and pull the vBIOS from the NVidia one? Maybe ask if someone from the fourm here can supply a working NVidia vBIOS for a 2080? I bet there are a few of them here using one. Quote Link to comment
frodr Posted February 27, 2020 Author Share Posted February 27, 2020 1 hour ago, Chess said: There was a script running around to reset the card, but I'm not sure that is 100% needed anymore. Still getting an NVidia card to pass through is easier. You just need the right vBIOS. Can you not boot the system with the two cards installed and pull the vBIOS from the NVidia one? Maybe ask if someone from the fourm here can supply a working NVidia vBIOS for a 2080? I bet there are a few of them here using one. Only black screen when booting from onboard or AMD GPU. Quote Link to comment
Chess Posted March 5, 2020 Share Posted March 5, 2020 From here I'm not sure what to suggest. Booting with the on board for unraid and then passing through a second video card to a vm usually works without issue. That you are not getting an error when starting the vm says that unraid is allowing the pass through and that IMMOU is being used. So from here I'd start to look around and see if anyone has the same mainboard and cards ever got this to work. It could be a bug in the BIOS version you are on, but getting a new BIOS for a board that old is unlikely. I don't like doing BIOS down grades, but that is something I might consider if this was my own setup. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.