Donach
-
Posts
13 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by Donach
-
-
1 hour ago, jammsen said:
Would you be so kind to share your iommu groups with 2 gpus in slots?
Im got really weird iommu groups on the ASUS Prime B350 Plus with dual gpus, which i explained here: https://lime-technology.com/forums/topic/69986-problems-on-ryzen-5-with-asus-prime-b350-plus/
Basically they prevent me from using a dual gpu setup with unraid, which is what i want
Did you use ACS Override?
If not, do.
If that is not enough, search my post back in the thread, where I explain how to setup the "multifunction" into unRaid startup. This helped me a lot to split the groups insanely (like really every device having its own group).
-
6 minutes ago, mikeyosm said:
No - any issues I had were related to the NPT bug and now this is resolved. I also had issues with random pops/crackle in audio even with MSI fix until I decided to pass through my nvme drive to the W10 VM, it now performs like it was bare metal. I suspected that the scsi/vfio driver was the cause but cannot be sure.
About the audio crack - I was experiencing this on any Audio device.
I was passing through USB stick with 2jacks - simple fix though:
In the Windows Sound configuration, changing the sound quality to higher (I think it was 48kHz - DVD) resolved the issue (as far as I can recall).
Thanks for the reply - so just getting newest unRaid RC is enough from what I understood... I will definitely try it sometime soon!
-
18 minutes ago, mikeyosm said:
I'm using RC15 on Asus Crosshair VI Hero, 32GB Corsair veng mem, 1700x Ryzen, Samsung 950 pro (in passthrough) and a plethora of HDDs.
I manage to stay stable for days/weeks whilst running a W10 VM with 1050ti/audio/nvme passthrough, Windows server 2012 VM, Plex and various dockers.
Plex transcodes 4k no issues, W10 gaming is very close the bare metal. Very very pleased thus far.
Did you use any dirty-hack to make it working like bare-metal?
I tried to setup double Win 10 VMs before the NPT fix was found, and it was horrible - had lags even at very basic games...Thanks for clarifying this.
-
7 minutes ago, fgdjhakdftere said:
I'll give that a go. Turns out it was the mouse wireless USB that just doesn't seem to work. It is for the MS Sculpt mouse. I tried another wireless mouse and it works fine so must be something with that particular mouse
What driver are you using for the wireless mouse?
Tried to install the Microsoft Mouse and Keyboard center? If yes, try various version or uninstall it completely... -
2 minutes ago, fgdjhakdftere said:
so I really want to run windows vm on my ryzen rig with a asrock ab350 pro4 motherboard.
I can get it to boot and I have windows installed but am really having issues with USB. Keyboard works but my mouse (both wired and wireless tried) are not working or they will work sporadically. I tried to pass through in the edit screen of VM but that is sporadic. I also tried to pass through USB Iommu group but that just kicks out error. Sorry I am really new and not sure if there is anything I can do?
Hello,
Have you tried various USB ports?
I actually never had such issue, passing any USB device worked flawlessly. Be sure to also have the USB adapter installed for the VM and to have correct drivers installed. -
21 hours ago, yanksno1 said:
Man, should have read this thread before I picked up my Ryzen build. I bought a MSI - B350 PC MATE ATX AM4 Motherboard, Ryzen 7 1700 CPU, and a EVGA - GeForce GT 730 4GB Video Card trying to do a Windows 10 VM on unRaid 6. Enabled KVM in the MB bios and that let me try to setup the VM. But when I click to create, I get this error. You guys have any idea's or am I out of luck with this motherboard?
Do you have ACS Override enabled?
If not, do so. See my post (around 10-15th page), where I explain how to enable it for it to work perfectly.
-
37 minutes ago, mikeyosm said:
Have to agree with the above. I too use an RX460 is my second PCIe x16 slot on the Asus Prime Pro, no issues with it at all. What score did you get in Ungine Josecitox?
Hello,
I tried to do this npt=0 on unRAID, however I was not able to do so...
How can you set this setting? Or more precisely, where?
Many thanks.
-
On 4/17/2017 at 3:02 PM, ColeBrodine said:
Just thought I would chime in here. Over the weekend I put together a new Unraid PC with a single (for now) Windows 10 Pro Gaming VM. I have a Gigabyte GA-AX370-Gaming K7 Motherboard with the Ryzen 1800x CPU. I updated to the newest BIOS before installing anything (I believe it was K3?) I paired it with a MSI GeForce GTX 1080 in the primary PCI slot. Passing it through to my VM with no problems. I have just a couple of dockers running at the moment.
I have ACS Override enabled, but it isn't breaking my groups up quite as much as I'd like. I currently have no sound because my motherboard sound card it grouped in the same IOMMU group as some other items and I can't get it to pass through. GPU Audio passthrough seems to be working fine, but I don't have a monitor with speakers to try out out on. It also has most of my USBs controllers in the same IOMMU group, so I can't pass any of those through without passing my unraid USB also. I'm able to passthrough individual USB devices without any problems. I've ordered a USB sound card that I should be able to passthrough and get some audio out.
So far it seems to be stable as long as I don't try to run any of the Dynamix system stats or temperature plugins. Ran all weekend from Friday to Monday morning with no crashes.
Hello ColeBrodine,
as Pauven said, look at my post on page 13 of this thread - my IOMMU grouping was totally off with the ACS Override enabled also, until I added the "multifunction" option. Since then, works like a charm - I'm able to passthrough anything to the VM, even since the CH6 has 3 PCI-Ex16 ports, I can run 3 VMs, each having it's own GPU.
However, I am now struggling with some specific problems - I want to passthrough USB busses, so I can plug and unplug USB devices on the go.
Another thing is that I have encountered issues running Europa Universalis IV - extreme game stuttering, solely on VMs. I did not see this with any other games yet.
-
Hello everybody!
I have been also dreaming about making it possible to run 2 VMs on one machine.
I can now finally say, I was successful with Ryzen 7 + unRAID!I am now able to run 2 or even 3 VMs with dedicated GPUs.
My setup:
Ryzen 7 1700 @stock (planning OC)
ASUS ROG Crosshair Hero VI
16 GB Corsair 3200 MHz DDR4
Samsung 960 EVO 250GB (cache)
Seagate Barracuda 3TB (planning to get one or two more)
Radeon HD5770, R5 230 (guests); Nvidia GTS 450 (host, guest also possible)
- I am using these lowend GPUs which I had lying around, once all tested, I'm going to get pair of RX480/580.
Linux 4.9.19-unRAID x86_64
ACS Override Enabled (downstream,multifunction)
- note that multifunction option was MANDATORY for my setup, otherwise the override was working only for top 2 PCI-Ex16 slots.Syslinux config:
cat /boot/syslinux/syslinux.cfg
default /syslinux/menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append pcie_acs_override=downstream initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest
IOMMU Groups - these actually played out amazingly! I did not expect after lots of hate on the grouping, that it would separate so perfectly, see for yourself!
IOMMU group 0 [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 1 [1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 2 [1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 3 [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 4 [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 5 [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 6 [1022:1453] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 7 [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 8 [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 9 [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 10 [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 11 [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 12 [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 13 [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1460 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1461 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1462 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1463 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1464 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1465 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1466 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1467 IOMMU group 14 [144d:a804] 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 IOMMU group 15 [1022:43b9] 03:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02) IOMMU group 16 [1022:43b5] 03:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02) IOMMU group 17 [1022:43b0] 03:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02) IOMMU group 18 [1022:43b4] 1d:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 19 [1022:43b4] 1d:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 20 [1022:43b4] 1d:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 21 [1022:43b4] 1d:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 22 [1b21:1343] 21:00.0 USB controller: ASMedia Technology Inc. Device 1343 IOMMU group 23 [8086:1539] 23:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 24 [10de:0dc4] 25:00.0 VGA compatible controller: NVIDIA Corporation GF106 [GeForce GTS 450] (rev a1) IOMMU group 25 [10de:0be9] 25:00.1 Audio device: NVIDIA Corporation GF106 High Definition Audio Controller (rev a1) IOMMU group 26 [1002:677b] 26:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos PRO [Radeon HD 7450] IOMMU group 27 [1002:aa98] 26:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM] IOMMU group 28 [1002:68b8] 27:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 5770] IOMMU group 29 [1002:aa58] 27:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] IOMMU group 30 [1022:145a] 28:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 31 [1022:1456] 28:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1456 IOMMU group 32 [1022:145c] 28:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 145c IOMMU group 33 [1022:1455] 29:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 34 [1022:7901] 29:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 35 [1022:1457] 29:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1457
VMs:
Windows 10
8 CPUs (0-7 and 8-15)
Machine i440fx-2.7
BIOS: OVMF
Hyper-V is OFF
Virtio drivers: virtio-win-0.1.126-2.iso
keyboard and mouse passed through
Audio from USB double jack adapter
- I had to set sound quality to at least 48kHz, or there was crackling in the audio (no manufacturer driver available)
or
Audio passed through from MB ([1022:1457] 29:00.3)
With this setup, I am running 2 GPUs with the Radeon cards passed through to VMs. I was, unfortunately, unable to passthrough the Nvidia card due to constant BSOD (System Thread Exception).
However, with newer Nvidia card, or another AMD GPU it is possible to run even 3 separate machines from this setup!
- unRAID does not mind at all that I am actually taking a GPU from it and it assigns it to the VM correctly - it is even able to boot to the VM, the BSOD appears after installing Nvidia drivers.Hope I'll help at least somebody trying to achieve the same I was able to.
Thank you Limetech! I tried to make this happen myself, but I gave up after many unsuccessful days of trying. Your solution worked right away! I'll be happy to buy the unRAID for my purposes :-).
-
2
-
Radeon RX 7900 XT passthrough
in VM Engine (KVM)
Posted · Edited by Donach
Minor corrections and styling update
Hey all wanted to chime in with my experience...
I had several issues over the years with GPU passthrough, eventually I lately settled on having working 3060Ti in 2nd slot (x8) PCI-E, which works flawlessly.
However, my main goal was to allow two machines running at once, yet the first GPU, whatever I used, was having either reset bug, or Error 43...
Now I've tested just today, and my VM resets and shutsdown-powers up flawlessly - using RX 6800.
Here are all the settings used, hope I remember all:
HW:
R9-5900X
AsRock Phantom Gaming 4S (X570 mobo)
64GB RAM (4 sticks)
UEFI:
Enabled 4G, Resize-Bar
Booting from UEFI unRaid on USB stick
Latest AsRock bios (as of 09/2023 or so).
IOMMU, HVM etc on ofc
unRaid:
Version: 6.12.3
Enabled ACS Override = "both"
VFIO allow unsafe interrupts: = "yes"
Syslinux config:
kernel /bzimage
append video=efifb:off isolcpus=3-11,15-23 pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot
Ofc make sure to bind EVERYTHING from the GPU to VFIO - RX 6800 has 4 devices on it: GPU, Sound, USB controller + USB serial bus
(IDs 0d:00:0 -> 0d:00:3 for me...)
VM config:
Win10 - I actually didn't install "from scratch", I had an old image lying around with Win10 working from testing the RTX 3080 in 1st PCI-E slot - so I cannot say if this helped in any way...
Q35v7.1
Bios: OVMF
Hyper-V: YES
USB: 3.0 (qemu XHCI)
Make sure ALL FOUR devices are passed through to VM
I did NOT do any manual edits, I wanna avoid them as much as possible so...
To install the driver, I also used the VNC console, which I've now removed after installing Barrier to the VM.
With this, I'm able to restart and/or shutdown and bring the VM back up, no issues.
Haven't done performance testing yet, I'll compare the two VMs to see if there is any considerable difference (the 2nd VM has the 3060ti in 2nd PCIE slot).