unRAID/KVM noob questions on current AMD GPUs and passthrough

[ledon]

Hi everyone,

 

I am new to unRAID and KVM virtualisation (not Linux though) and currently in the trial period, trying to determine whether I can achieve my desired setup with unRAID (which I truly hope).

 

TL;DR: I have a few questions regarding the support for Vega/ Radeon VII GPUs when passing them through to VMs which can be found at the end of the post. The next two paragraphs are a description of my desired setup and references of my own research giving context to my questions.

 

My desired setup includes running Windows (for gaming) and macOS (must be a current version for development and testing purposes) as VMs with GPU passthrough (I also intend to run Linux, but it does not seem to put any constraints on GPU choice). I currently have an RX580 which works fine in this setup, but upgrading the card is on my wishlist for the coming months. From what I found, the GPU card series listed here are the most recent ones suitable for my use case, since macOS is the least compatible of the OSes I want to run. Most importantly Nvidia GPUs seem to be out of the question for current versions of macOS with no Nvidia web drivers on the horizon. So AMD it is?

 

In my research of AMD GPU compatibility I came across the GPU reset bug affecting Vega cards, but the info out there seems somewhat conflicting. On the one hand, reports here in the forum indicate that the bug is present for Vega 64 (as of February 4th 2019) and Radeon VII (as of June 23rd 2019) respectively. In the same thread are reports that RX580 doesn't work which I cannot confirm from my own testing with my RX580 Nitro+, so I am unsure how reliable the info in the thread is. On the other hand, there apparently is a kernel patch for version 4.19 (which is the kernel version in unRAID 6.7.2) and someone seems to have had success using this patch with a Radeon VII (albeit on Ubuntu). Also, independently of the reset bug there seems to be a bug with the AMD drivers, where the latest version that can be installed on Windows VMs is 18.9.3 (current version would be 19.6.3).

 

Long story short: Since upgradability seems important when deciding to purchase an OS, is there anybody who can confirm

1. whether there has been success running Vega or Radeon VII GPUs with VM shutdown/reset working properly in unRAID?
2. whether unRAID is running a kernel with the aforementioned reset bug patch by default?
3. if not 2, then whether there is an easy way to build a custom kernel with the patch? (The official info seems a bit outdated and includes as prerequisite "... a current kernel to match your chosen unRAID release [...] and who knows what else".)
4. whether anyone here was able to install newer drivers for Radeon VII on Windows VMs, since version 18.9.3 predates this card? (Yes, the patch success post indicates there are drivers for Windows 10, but the ones shipping with Windows or via Windows update rendered HDMI audio for my RX580 unusable so any additional info is highly appreciated!)

Thanks in advance to everybody helping an unRAID noob to figure this GPU passthrough stuff out!

 

All the best
ledon

Edited July 6, 2019 by ledon

[bastl]

First of all, welcome to Unraid.

 

Short answer, current state of AMD GPU passthrough is a nightmare especially the Vega cards and the Radeon VII. To be clear, I don't have a AMD GPU to play with but from all I've read here in the forum and the levelone forum you won't find many people with a working setup. If there is a working kernel patch for that issue I bet the Unraid devs will push it to us users even if it`s a "dirty patch" like on the early days of the first gen TR4 chips. Not sure if  @limetech already heared of that "fix-vega-reset.patch" and played with it. Maybe they have time to have a look at it. 

 

For the driver issue with AMD GPU's I remember end of last year at one point the AMD drivers started causing issues (Black Screen). Some people figured out how to setup their VMs to avoid that bug. Not exactly sure if machine type Q35 for the VM or booting Unraid in legacy or UEFI mode or a combination of these fixes it. If you search the forum you will find a couple people reporting a fix.

[ledon]

Hi @bastl,

 

I know this comes very late, but thanks a lot for your reply and the information! For anyone having the same questions here one piece of information I can add now: The RX 580 will be reset properly even after a force shutdown of the VM in newer Unraid versions (6.8.3 as of this writing).

 

For the upgrade I decided to go the Nvidia route for now and use the RX 580 for the macOS machine.