COMPLETE TECHNICAL REPORT - VFIO GPU PASSTHROUGH FAILURE - VM Engine (KVM)

January 21Jan 21

HARDWARE CONFIGURATION

Motherboard: MSI MPG X870E Carbon WiFi (MS-7E49)

BIOS Version: E7E49AMSI.1A7D
BIOS Build Date: January 07, 2026
Chipset: AMD X870E

CPU: AMD Ryzen 9 9950X3D

16 Cores / 32 Threads
Base Clock: 4.3 GHz
Integrated Graphics: AMD Radeon (iGPU)

RAM: 128GB DDR5-5200

Brand: G.Skill Flare X5
Configuration: 4x 32GB modules

GPU (Passthrough Target):

Model: NVIDIA GeForce RTX 4060 Ti 16GB (MSI)
PCI Address: 0000:01:00.0 (GPU) + 0000:01:00.1 (Audio)
IOMMU Group: 14
Device IDs: 10de:2805 (GPU), 10de:22bd (Audio)

Primary Display: Integrated Graphics (AMD Radeon iGPU)

Operating System:

Unraid Server OS version 7.2.3
Linux Kernel: 6.12.54-Unraid
Bootloader: Limine (NOT Syslinux - critical detail!)

Storage:

VM vDisk 1: /mnt/windowsfestplatte/vdisk1.img (Windows System)
VM vDisk 2: /mnt/gaming/gaming_storage.img (Gaming Storage)
VirtIO Drivers: /mnt/vm/virtio-win-0.1.285-1.iso

OBJECTIVE

Pass through NVIDIA RTX 4060 Ti to a Windows 11 VM using VFIO-PCI for native GPU performance in gaming and GPU-accelerated workloads.

INITIAL PROBLEM: BOOTLOADER DISCOVERY

Issue #1: Kernel parameters not loading

Initial attempts to configure VFIO via /boot/syslinux/syslinux.cfg had NO EFFECT because:

bash

# cat /proc/cmdline[empty output]```**Root Cause:** System boots via **Limine bootloader**, NOT Syslinux
- Limine config location: `/boot/limine.conf`- Evidence: `ls -la /boot/` showed `limine.conf` with recent modification date- This is a critical issue because most Unraid VFIO guides assume Syslinux
**Solution Applied:**
Modified `/boot/limine.conf` instead of syslinux.cfg:
```
/Unraid OS
    protocol: linux
    kernel_path: boot():/bzimage
    module_path: boot():/bzroot
    cmdline: pcie_acs_override=downstream,multifunction amd_iommu=on iommu.passthrough=1 vfio-pci.ids=10de:2805,10de:22bd video=efifb:off
    comment: Unraid OS

Verification:

bash

# cat /proc/cmdlinepcie_acs_override=downstream,multifunction amd_iommu=on iommu.passthrough=1 vfio-pci.ids=10de:2805,10de:22bd video=efifb:off
# dmesg | grep "Command line"[0.000000] Command line: pcie_acs_override=downstream,multifunction amd_iommu=on iommu.passthrough=1 vfio-pci.ids=10de:2805,10de:22bd video=efifb:off
```

✅

Kernel parameters successfully loaded after Limine configuration --- ### **BIOS CONFIGURATION****Critical Settings for VFIO:** **1. Graphics Configuration:** ```Advanced → Integrated Graphics Configuration - Initiate Graphic Adapter: IGD (iGPU) ✅ - Graphics Card Detection: Ignore ```*Purpose: Force iGPU as primary display, leaving NVIDIA available for passthrough* **2. PCI Subsystem Settings:** ```Advanced → PCI Subsystem Settings - Above 4G Decoding: ENABLED ✅ - Re-Size BAR Support: DISABLED ✅ (CRITICAL!) - SR-IOV Support: DISABLED ✅```*Purpose: Re-Size BAR enabled causes "1:1 IOMMU mapping" firmware errors* **3. AMD IOMMU Settings:** ``` Advanced → AMD CBS → NBIO Common Options - IOMMU: Enabled ✅ - ACS Enable: Auto/Enabled

Verification:

bash

# dmesg | grep -i "amd-vi"AMD-Vi: IOMMU performance counters supported
pci 0000:01:00.0: Adding to iommu group 14pci 0000:01:00.1: Adding to iommu group 14# dmesg | grep -i "iommu"iommu: Default domain type: Passthrough (set via kernel command line)

✅ IOMMU enabled and operational

VFIO-PCI BINDING

Issue #2: Duplicate binding causing conflicts

Problem: GPU was being bound to vfio-pci TWICE:

Via kernel parameter: vfio-pci.ids=10de:2805,10de:22bd
Via Unraid config: /boot/config/vfio-pci.cfg containing BIND=0000:01:00.0|10de:2805 0000:01:00.1|10de:22bd

Solution:

bash

# Backup and clear vfio-pci.cfgcp /boot/config/vfio-pci.cfg /boot/config/vfio-pci.cfg.backup
echo "" > /boot/config/vfio-pci.cfg
reboot

Verification:

bash

# lspci -nnk -d 10de:280501:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD106 [GeForce RTX 4060 Ti 16GB] [10de:2805] (rev a1)    Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:5174]    Kernel driver in use: vfio-pci

✅

Kernel modules: nouveau # ls -la /dev/vfio/crw------- 1 root root 245, 0 Jan 20 20:18 14 ✅crw-rw-rw- 1 root root 10, 196 Jan 20 20:18 vfio

✅ GPU successfully bound to vfio-pci

IOMMU GROUP CONFIGURATION

Issue #3: IOMMU Group in wrong mode

Discovery:

bash

# cat /sys/kernel/iommu_groups/14/typeidentity

❌

IOMMU Group 14 was in "identity" mode instead of "DMA" mode required for VFIO.

Solution:

bash

# echo DMA > /sys/kernel/iommu_groups/14/type# cat /sys/kernel/iommu_groups/14/typeDMA

✅

Additional mitigation:

bash

# echo 1 > /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts

VM CONFIGURATION

Multiple VM configurations tested:

Attempt 1: i440fx machine type (initial)

Machine: pc-i440fx-9.2
Result: GPU not available in WebUI dropdown

Attempt 2: Q35 machine type

Machine: pc-q35-9.2
BIOS: OVMF-TPM (UEFI)
vCPUs: 16 (8 cores, 2 threads)
RAM: 16GB
Primary vDisk: VirtIO bus
Result: GPU still not in dropdown, added manually via XML

Attempt 3: Manual XML configuration

Multiple PCI bus configurations attempted:

Configuration A:

xml

<hostdev mode='subsystem' type='pci' managed='yes'>  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/></source>  <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/></hostdev><hostdev mode='subsystem' type='pci' managed='yes'>  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/></source>  <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/></hostdev>

Result: Failed with "Invalid argument"

Configuration B:

xml

<hostdev mode='subsystem' type='pci' managed='yes'>  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/></source>  <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/></hostdev><hostdev mode='subsystem' type='pci' managed='yes'>  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/></source>  <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x1'/></hostdev>

Result: Failed with "Invalid argument"

Configuration C (Final attempt):

xml

<hostdev mode='subsystem' type='pci' managed='yes'>  <driver name='vfio'/>  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/></source>  <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0' multifunction='on'/></hostdev><hostdev mode='subsystem' type='pci' managed='yes'>  <driver name='vfio'/>  <source><address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/></source>  <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x1'/></hostdev>```
Result: Still failed with "Invalid argument"
---
### **PERSISTENT ERROR**
**Every VM start attempt fails with:**
```
error: Failed to start domain 'Windows 11'
error: internal error: QEMU unexpectedly closed the monitor (vm='Windows 11'): 
2026-01-21T12:35:40.272528Z qemu-system-x86_64: -device {"driver":"vfio-pci",
"host":"0000:01:00.0","id":"hostdev0","bus":"pci.6","multifunction":true,
"addr":"0x0"}: vfio 0000:01:00.0: failed to setup container for group 14: 
Failed to set group container: Invalid argument

Kernel log shows:

bash

# dmesg | tail -5vfio-pci 0000:01:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, 
rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.

DIAGNOSTIC DATA

IOMMU Status:

bash

# dmesg | grep -i "iommu mode"[0.587478] iommu: Default domain type: Passthrough (set via kernel command line)# ls /sys/kernel/iommu_groups/ | wc -l54# for d in /sys/kernel/iommu_groups/14/devices/*; do lspci -nns "${d##*/}"; done01:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD106 [GeForce RTX 4060 Ti 16GB] [10de:2805] (rev a1)01:00.1 Audio device [0403]: NVIDIA Corporation AD106M High Definition Audio Controller [10de:22bd] (rev a1)

VFIO Module Status:

bash

# lsmod | grep vfiovfio_pci               16384  0vfio_pci_core          86016  1 vfio_pci
vfio_iommu_type1       49152  0vfio                   65536  3 vfio_pci_core,vfio_iommu_type1,vfio_pci

Container Status:

bash

# ls -la /dev/vfio/crw------- 1 root root 245, 0 Jan 20 20:18 14crw------- 1 root root 245, 1 Jan 20 20:18 15crw-rw-rw- 1 root root 10, 196 Jan 20 20:18 vfio
# lsof /dev/vfio/14[no output - device not in use]

PCI Device Tree:

bash

# lspci -tv-[0000:00]-+-00.0  Advanced Micro Devices, Inc. [AMD] Device
           +-00.2  Advanced Micro Devices, Inc. [AMD] Device
           +-01.0  Advanced Micro Devices, Inc. [AMD] Device
           +-01.1-[01]--+-00.0  NVIDIA Corporation AD106 [GeForce RTX 4060 Ti 16GB]           |            \-00.1  NVIDIA Corporation AD106M High Definition Audio Controller

TROUBLESHOOTING STEPS ATTEMPTED

✅ 1. Verified hardware compatibility

Confirmed IOMMU support in CPU
Verified motherboard supports VT-d/AMD-Vi

✅ 2. BIOS configuration

Enabled IOMMU
Disabled Re-Size BAR (critical!)
Set iGPU as primary display
Enabled Above 4G Decoding

✅ 3. Bootloader configuration

Discovered Limine (not Syslinux)
Configured correct kernel parameters
Verified parameters loaded

✅ 4. Kernel parameters optimized

amd_iommu=on
iommu.passthrough=1 (later changed to iommu=pt)
pcie_acs_override=downstream,multifunction
vfio-pci.ids=10de:2805,10de:22bd
video=efifb:off

✅ 5. VFIO binding

Removed duplicate bindings
Verified vfio-pci driver attachment
Confirmed VFIO devices created

✅ 6. IOMMU group management

Set group type to DMA
Enabled unsafe interrupts
Verified group isolation

✅ 7. VM configuration

Tested multiple machine types (i440fx, Q35)
Tried various PCI bus configurations
Added multifunction attribute
Attempted different slot/bus combinations

✅ 8. GPU unbind/rebind

bash

echo "0000:01:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind
echo "0000:01:00.1" > /sys/bus/pci/drivers/vfio-pci/unbind
sleep 2echo "0000:01:00.0" > /sys/bus/pci/drivers/vfio-pci/bind
echo "0000:01:00.1" > /sys/bus/pci/drivers/vfio-pci/bind
```

❌

**None of these resolved the issue** --- ### **ROOT CAUSE ANALYSIS****Firmware-Level Restriction:** The error message is clear: ``` vfio-pci 0000:01:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.

This indicates the UEFI firmware is enforcing a DMA protection mechanism that conflicts with VFIO's virtualization requirements.

Possible Contributing Factors:

New Hardware Generation
- X870E chipset released Q4 2024
- Ryzen 9 9950X3D released 2024
- VFIO support may not be mature
Re-Size BAR Conflict
- Despite disabling in BIOS, firmware may still enforce related restrictions
- Common issue with newer NVIDIA GPUs
Firmware Bug
- BIOS version E7E49AMSI.1A7D may have incomplete IOMMU implementation
- May require BIOS update from MSI
Kernel/QEMU Version
- Unraid 7.2.3 uses kernel 6.12.54
- X870E support may require newer kernel
Missing BIOS Option
- Hidden or unavailable option controlling IOMMU strictness
- AMD CBS settings may have additional undocumented options

QUESTIONS FOR MSI/AMD SUPPORT

Is VFIO GPU passthrough officially supported on the MPG X870E Carbon WiFi?
Does BIOS version E7E49AMSI.1A7D have known issues with IOMMU/virtualization?
Is there a newer BIOS version that addresses VFIO compatibility?
Are there hidden BIOS settings (AMD CBS) that control the "1:1 IOMMU mapping" firmware requirement?
Does the "Re-Size BAR Support" setting have secondary effects even when disabled?
What is the recommended BIOS configuration for GPU passthrough on this board?
Is this a known limitation of the X870E chipset?

SYSTEM STATE AT TIME OF FAILURE

All verifications passing:

bash

# Kernel parameters

✅

pcie_acs_override: active ✅ amd_iommu: enabled ✅ iommu.passthrough: active ✅ vfio-pci.ids: bound # BIOS settings✅ IOMMU: enabled ✅ Re-Size BAR: disabled ✅ Above 4G: enabled ✅ iGPU primary: active # GPU binding✅ Driver: vfio-pci ✅ IOMMU group: 14 (isolated)✅ Group type: DMA ✅ VFIO device: /dev/vfio/14 exists

Yet VM start fails with firmware restriction.

COMMUNITY NOTES

Discord user "Mainfrezzer" (experienced with Unraid VFIO) suggested:

"Has anyone told you that pcie pass through does not work with i440fx? Create a new template, set it to q35 at the start and go from there"

However, Q35 configuration also failed with identical error.

Discord user "mowmdown" confirmed Limine configuration was correct:

"you will need to insert this line between the module_path: and comment: lines"

HARDWARE COMPARISON

For context, this same RTX 4060 Ti successfully works with:

VFIO on older platforms (X570/B550)
Native Windows installation on this motherboard

This suggests the issue is specifically with:

X870E + VFIO combination
Firmware restrictions on new chipset
Incomplete BIOS implementation

CONCLUSION

Despite extensive troubleshooting and correct configuration at every level (kernel, BIOS, VFIO), GPU passthrough fails due to a firmware-level restriction requesting "1:1 IOMMU mapping" that cannot be satisfied.

This appears to be either:

A BIOS bug requiring an update from MSI
A chipset limitation of X870E
Missing kernel/QEMU support for new hardware generation

Request to MSI Support: Please investigate firmware-level IOMMU handling on the MPG X870E Carbon WiFi and provide guidance or BIOS update to enable VFIO GPU passthrough.

TECHNICAL LOGS AVAILABLE

Complete dmesg output
libvirt/QEMU logs
VM XML configurations (all iterations)
BIOS screenshots
Full hardware specifications

System available for remote diagnosis if needed.

Report compiled: January 21, 2026
Testing duration: ~8 hours intensive troubleshooting
Configuration attempts: 15+ different approaches
Result: Unsuccessful - firmware restriction

Quote

June 7Jun 7

Hello there, I've a fairly similar setup (X870e Carbon Wifi and 9950X3D). GPU passthrough was working well for me but I was on a fairly old bios (I believed its the A63 back in August 2025). I just upgraded to the latest A95 (March 2026) and it broke the GPU passthrough, I've two Nvidia GPUs in my Unraid server, where one is being passthru to an Ubuntu, and the other to a Windows 11.

These are all working well till the latest Bios upgrade. You could try the A63 version to see if you manage to get it to work.

Quote

June 9Jun 9

@tschatscher I've since reverted to the older bios and the GPU passthrough working fine again. Just thought you should know.

Quote

COMPLETE TECHNICAL REPORT - VFIO GPU PASSTHROUGH FAILURE

Featured Replies

HARDWARE CONFIGURATION

OBJECTIVE

INITIAL PROBLEM: BOOTLOADER DISCOVERY

VFIO-PCI BINDING

IOMMU GROUP CONFIGURATION

VM CONFIGURATION

DIAGNOSTIC DATA

TROUBLESHOOTING STEPS ATTEMPTED

QUESTIONS FOR MSI/AMD SUPPORT

SYSTEM STATE AT TIME OF FAILURE

COMMUNITY NOTES

HARDWARE COMPARISON

CONCLUSION

TECHNICAL LOGS AVAILABLE

Join the conversation

Account

Navigation

Search

Configure browser push notifications

Chrome (Android)

Chrome (Desktop)

Safari (iOS 16.4+)

Safari (macOS)

Edge (Android)

Edge (Desktop)

Firefox (Android)

Firefox (Desktop)