Jump to content

dmarshman

Members
  • Content Count

    7
  • Joined

  • Last visited

Community Reputation

1 Neutral

About dmarshman

  • Rank
    Newbie

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. What you are doing above should work OK. I listed an alternative method [which is helpful if you have identical cards]. If you want to try it - and I'm not sure you need to - then it's a text file that you need to create [in the /boot/config/ directory]. Use the "shell" button in Unraid's UI to open a window, and then type: nano /boot/config/vfio-pci.cfg This will open [or create and open if it doesn't already exist] the file for editing: Add [based on your post above]: BIND: 11.00.0 11.00.1 Then press <control>-X to exit, "y' to save, then <enter>/<return>, and then reboot...
  2. I didn’t have any issues disabling the primary gfx card from being used by Unraid, and passing through both my Radeon VIIs to VMs. I'll list exactly how I achieved it with my setup below: Looks like you are much more advanced with Clover tweaking than I am, and I may need some tips from you. I reply in a separate message for that.. Two steps for identical GPU pass through with neither being used by Unraid]: - #1 - disable GPU for unraid - #2 - make them available to passthrough to VMs #1 - To disable the Unraid video: From the Main menu, click the name of your Boot Device (flash). Under Syslinux Config -> Unraid OS, add "video=efifb:off" after "append initrd=/bzroot". The line should now read "append initrd=/bzroot video=efifb:off". When you reboot you will notice there is no video output when unraid boots (you will be left with a freeze frame of the boot menu). Your primary GPU is now ready to pass through. #2 Then pass through (I use the following method - by IOMMU group - which works flawlessly for Identical graphics cards, but needs to be updated if you add / remove any devices to your system (as the values may be reassigned)). In following file: /boot/config/vfio-pci.cfg add a single line with “BIND:” followed by the IOMMU group(s) for your graphics cards (or any other PCI devices you want to pass through). Example - For my system - with two Radeon VIIs: BIND=13:00.0 13:00.1 10:00.0 10:00.1 [technically, I think the 2nd sub group - with the xx.xx.1 - is redundant/unnecessary, but I include it anyway as it doesn't hurt either]. By doing both the above, I am able to run Windows and Ubuntu* VMs simultaneously with both having a passed through, fully accelerated GPU. (*and theoretically MacOS too, I just personally haven't got it working properly yet).
  3. Wow. I love your repurposed / custom modified Mac case... was it a PowerMac G5 or a Xeon MacPro? {I still have my 2008 Dual Quad Core 2.8 GHz Xeon MacPro… I haven't used it as my daily driver since 2016, and have not even turned it on in a couple of years, but it's still fully intact... maybe a project for another day} For my X570 computer, I upgraded to 64 GiBs of RAM [4 DIMMs running at 2800 GHz], and have two Radeon VIIs with a 1300W PSU, and currently I have it in a Thermaltake Core P3 [open] case. Re: the KVM machines and MacOS / gfx cards, I haven't experienced the issues you've seen, but have been mainly using the computer with Windows 10 on bare metal [i.e. without Unraid] over the past couple of weeks. I have the Radeon VIIs in PCIE Slot #1 and Slot #3. When using Unraid, I've been running a Windows 10 VM alongside a Ubuntu 18.04 LTS VM with no problems using two Radeon VIIs, but to be honest I gave up on MacOS after four days of experiments where I constantly failed to get any type of acceleration on the gfx card, and ended up corrupting/wiping the installation every few hours with failed boots. I think the [Unraid 6.8 RC stream] reversion back to the 4.19.x kernel [from 5.x] made things less functional for me on MacOS, so I decided to wait for Unraid 6.9 [and latest versions of QEMU] before doing any more experiments. Looks like 6.8.1 RC just dropped with updated virtd and QEMU, but still using the 4.xx kernel, so still waiting, but maybe I'll try again this weekend…. [I have my 2018 Macmini for day to day MacOS usage so not in too much of a rush]. What type of Radeon Vega cards do you have - 56, 64 or VII? Please bear in mind that there is a know reset bug on the Vega 56 and 64 [I think now fixed in recent kernels], and also the Radeon VII [which is not yet fixed], which impacts virtual machines, that could be impacting your system. It basically stops a VM from being restarted if it [the VM] is totally shut down... [you have to reboot unraid. SpaceInvaderOne has videos about it...].
  4. PCI Devices and IOMMU Groups IOMMU group 0: [1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 1: [1022:1483] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 2: [1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 3: [1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 4: [1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 5: [1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 6: [1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 7: [1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 8: [1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 9: [1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 10: [1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 11: [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 12: [1022:1484] 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 13: [1022:1484] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 14: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 15: [1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 IOMMU group 16: [1987:5016] 01:00.0 Non-Volatile memory controller: Phison Electronics Corporation Device 5016 (rev 01) IOMMU group 17: [1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57ad IOMMU group 18: [1022:57a3] 03:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 19: [1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 [1022:1485] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:149c] 0a:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] 0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 20: [1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 [1022:7901] 0b:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 21: [1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 [1022:7901] 0c:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 22: [1b21:1184] 04:00.0 PCI bridge: ASMedia Technology Inc. ASM1184e PCIe Switch Port IOMMU group 23: [1b21:1184] 05:01.0 PCI bridge: ASMedia Technology Inc. ASM1184e PCIe Switch Port [8086:2723] 06:00.0 Network controller: Intel Corporation Device 2723 (rev 1a) IOMMU group 24: [1b21:1184] 05:03.0 PCI bridge: ASMedia Technology Inc. ASM1184e PCIe Switch Port IOMMU group 25: [1b21:1184] 05:05.0 PCI bridge: ASMedia Technology Inc. ASM1184e PCIe Switch Port [8086:1539] 08:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 26: [1b21:1184] 05:07.0 PCI bridge: ASMedia Technology Inc. ASM1184e PCIe Switch Port IOMMU group 27: [1002:14a0] 0d:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 14a0 (rev c1) IOMMU group 28: [1002:14a1] 0e:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 14a1 IOMMU group 29: [1002:66af] 0f:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 20 [Radeon VII] (rev c1) IOMMU group 30: [1002:ab20] 0f:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Vega 20 HDMI Audio [Radeon VII] IOMMU group 31: [1022:148a] 10:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function IOMMU group 32: [1022:1485] 11:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP IOMMU group 33: [1022:1486] 11:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP IOMMU group 34: [1022:149c] 11:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 35: [1022:1487] 11:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller IOMMU group 36: [1022:7901] 12:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 37: [1022:7901] 13:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
  5. If you are not already, try using the "Q35-3.1" machine, rather than the "i440fx-3.1" machine to which the "Windows 10" template defaults. That fixed it for me - [ASRock X570 Taichi, Ryzen 9 3950X and Radeon Vega VII] - in conjunction with the boot flag "video=efifb:off" outlined above.
  6. Per the title.. Plan to use this as a workstation / high end gaming rig; running both Windows 10 and MacOS Catalina in UnRAID VMs [as well as Win 10 natively]. I'll add a new thread in the full build section once I get the machine up and running. Couple of days playing around so far: Latest MBoard BIOS - 2.50 [2019/11/13] Running in UEFI Stock clocks, voltages, etc for 3950X & RAM. 32 GiB RAM : G.Skill Ripjaws V 32GB 2 x 16GB DDR4-3200 PC4-25600 CL16 Dual Channel Desktop Memory Kit F4-3200C16D-32G Hoping/Planning to go to 64 GiB RAM and [maybe] 2x Vega VIIs Initial testing: VM#1 : MacOS Catalina - using SpaceInvader's Docker / template, plus some tweaks - 8 cores / 16 threads - 16 GiB RAM - Vega VII passed through in PCI-E slot #1 [PCI-E 3x16] - - no hardware acceleration / Metal / OpenCL yet Cinebench R20 - 4850 [Multicore] Native Win 10: - 1 TB PCI-E 4 4x NVMe Drive - 16 cores / 32 threads - 32 GiB RAM - Vega VII [stock everything] Cinebench R20 - 9454 [Multicore]