MSI BIOS update broke GPU passthrough

[tw0884]

First time posting so be gentle.

A little bit of background: I have been running unRAID 6.7.0 rc4 for a few weeks and have had numerous dockers, a W10 gaming VM and array running smoothly (I'm really enjoying UnRaid for all of this functionality). I recently updated my MB BIOS from 7B85v12 to 7B85v16 and it seems like everything has fallen apart. My troubles sound very similar to this thread hereby @mattz.

 

Observed behaviour after BIOS update:

CPU pairings had all changed (instead of 0-1,2-3. I now have 0-6, 1-7)

Layout of unRAID tabs changed from vertical to horizontal.

W10 VM will not start. (Existing and newly made)

unRAID becomes very unstable after attempting to start W10 VM and will hang on reboot.

Dockers will run but the docker page is very unresponsive.

New error in VM log: ‘vfio: Unable to power on device, stuck in D3”

 

VM:

I have a 970 m.2 stubbed that I installed Windows 10 on (as a normal W10 VM through unRAID with libvirt location set to none, and 970 selected as other PCI device). Prior to the BIOS update I was able to start the VM through the unRAID interface as well as through the MSI boot options( bare metal).

 

It was a bit of an adventure getting it going but it ended up working with ovmf, Q35 3.1 and legacy USB startup in the BIOS ( turn off Windows 10 compatibility mode). Also had to change the VM XML so the 970 was first to boot (<boot_order=’1’/>).

 

I always got the 'cannot reset devices , depends on group 33’ error for the stubbed rx580 audio but the VM would always start and I simple had to stop/start the audio device from hardware manager inside the VM and I would get sound.

 

I have PCIe ACS Override=both as well as allow_unsafe_interrupts=true ( both previously worked fine)

 

Steps taken to fix problem:

I have created a new VM with the same options after the BIOS update in hopes it would properly assign the CPU's but got both the ‘D3’ and ‘cannot reset device’ errors. And the VM will not start.

 

As of now, the original poster of the issue has not found a resolution and I am stuck wondering if I should (force) a downgrade of the MSI bios or wait for one that works( or stable unRAID that fixes the problem?)

 

Ps:. Found a tool HERE to downgrade MSI bios’ that people have been having luck with, as you cannot simply downgrade in M-flash anymore.

 

I can post what whatever logs you guys need, let me know . Thx.

 

Hardware:

 

unRAID system:unRAID server Basic, version 6.7.0-rc4
 

Motherboard:    MSI B450 GAMING PRO CARBON AC (MS-7B85) BIOS: 7B85v16 (19/03/06)
 

Processor:    AMD Ryzen 5 2600 Six-Core @ 3.4 GHz
HVM:    Enabled
IOMMU: Enabled
GPU - Sapphire rx580

 

unraid-diagnostics-20190327-1826.zip

 

 

Edit: as I stated in the other thread by mattz I have flashed my MSI BIOS back to v12. This has rectified the issues I had with the VM and unRAID as it did for mattz. I wouldn't call this 'solved' as the issue with the BIOS still occurs but this is a way to revert back to a BIOS works. 

Edited March 30, 2019 by tw0884
Add resolution

[nicksphone]

sounds like you IOMMU groups have changed if you did any device pass though there groups might not be the same stopping the vms from starting

[Badboy]

On 3/27/2019 at 2:50 PM, tw0884 said:

First time posting so be gentle.

A little bit of background: I have been running unRAID 6.7.0 rc4 for a few weeks and have had numerous dockers, a W10 gaming VM and array running smoothly (I'm really enjoying UnRaid for all of this functionality). I recently updated my MB BIOS from 7B85v12 to 7B85v16 and it seems like everything has fallen apart. My troubles sound very similar to this thread hereby @mattz.

 

Observed behaviour after BIOS update:

CPU pairings had all changed (instead of 0-1,2-3. I now have 0-6, 1-7)

Layout of unRAID tabs changed from vertical to horizontal.

W10 VM will not start. (Existing and newly made)

unRAID becomes very unstable after attempting to start W10 VM and will hang on reboot.

Dockers will run but the docker page is very unresponsive.

New error in VM log: ‘vfio: Unable to power on device, stuck in D3”

 

VM:

I have a 970 m.2 stubbed that I installed Windows 10 on (as a normal W10 VM through unRAID with libvirt location set to none, and 970 selected as other PCI device). Prior to the BIOS update I was able to start the VM through the unRAID interface as well as through the MSI boot options( bare metal).

 

It was a bit of an adventure getting it going but it ended up working with ovmf, Q35 3.1 and legacy USB startup in the BIOS ( turn off Windows 10 compatibility mode). Also had to change the VM XML so the 970 was first to boot (<boot_order=’1’/>).

 

I always got the 'cannot reset devices , depends on group 33’ error for the stubbed rx580 audio but the VM would always start and I simple had to stop/start the audio device from hardware manager inside the VM and I would get sound.

 

I have PCIe ACS Override=both as well as allow_unsafe_interrupts=true ( both previously worked fine)

 

Steps taken to fix problem:

I have created a new VM with the same options after the BIOS update in hopes it would properly assign the CPU's but got both the ‘D3’ and ‘cannot reset device’ errors. And the VM will not start.

 

As of now, the original poster of the issue has not found a resolution and I am stuck wondering if I should (force) a downgrade of the MSI bios or wait for one that works( or stable unRAID that fixes the problem?)

 

Ps:. Found a tool HERE to downgrade MSI bios’ that people have been having luck with, as you cannot simply downgrade in M-flash anymore.

 

I can post what whatever logs you guys need, let me know . Thx.

 

Hardware:

 

unRAID system:unRAID server Basic, version 6.7.0-rc4
 

Motherboard:    MSI B450 GAMING PRO CARBON AC (MS-7B85) BIOS: 7B85v16 (19/03/06)
 

Processor:    AMD Ryzen 5 2600 Six-Core @ 3.4 GHz
HVM:    Enabled
IOMMU: Enabled
GPU - Sapphire rx580

 

unraid-diagnostics-20190327-1826.zip 125.82 kB · 1 download

 

 

Edit: as I stated in the other thread by mattz I have flashed my MSI BIOS back to v12. This has rectified the issues I had with the VM and unRAID as it did for mattz. I wouldn't call this 'solved' as the issue with the BIOS still occurs but this is a way to revert back to a BIOS works. 

Hi,

I'm curious, is this the same motherboard you have? https://www.msi.com/Motherboard/B450-GAMING-PRO-CARBON-AC/Specification

 

I have been trying to buy it,  I have seen the same model with different ram specs, the one I want supports 128 gigs of ram. The other one with the same model# supports 64 gigs of ram.  I called MSI, guy told me this board is not suitable for a server, and that it doesn't support unbuffered ram. That's not what I'm seeing. LOL.. I'm seeing this board is better then the Asrock x370 killer SLI I have. Can you shed any light on this?  Much appreciated.  Trying to upgrade my board without changing my Ryzen x1700 right now.

Edited July 31, 2020 by Badboy