VM GPU passthrough resizable BAR support in 6.1 kernel

[KptnKMan]

@Trozmagon what hardware are you running?

Just wondering, because I've noticed some AMD Ryzen systems have trouble.

[Trozmagon]

2 hours ago, KptnKMan said:

@Trozmagon what hardware are you running?

Just wondering, because I've noticed some AMD Ryzen systems have trouble.

 

Currently i9-11900k, MSI z590i Unify, RTX 3090.

 

I did use to have an RTX 3080 which worked as soon as I updated it firmware on it, which worked without issues, no xml modification or anything. I swapped that for an RX 6900 XT which I was never able to get Resizable Bar working at all, I did read somewhere that Resizable Bar doesn't work on AMD cards in Linux (I believe this is what's changed in to the next Kernel release).

[KptnKMan]

That's interesting, I've been following the ReBAR issue for some time now, trying to make it work on my setup.

From what I've seen, Intel CPU/chipset platforms seem to have a much easier time getting this to work, my understanding is that kernel support is a big factor with it.

 

Until now I've been getting the black screen issue on my passthrough 3090 no matter what I've tried until now, I should try to test without the custom kernel and see if the extra VM extensions are the ticket to getting that working.

[SimonF]

On 1/21/2023 at 12:15 PM, Skitals said:

 

The patch allows you to manipulate the bar size through sysfs. See the example how to use it in this reddit post.

 

This is the User Script I created to set the bar size to 16GB. You would obviously need to tweak this to the proper bar size, pice address, folder path, device id, etc.

 

#!/bin/bash
echo -n "0000:0d:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind
echo 14 > /sys/bus/pci/devices/0000\:0d\:00.0/resource1_resize
echo -n "10de 2782" > /sys/bus/pci/drivers/vfio-pci/new_id || echo -n "0000:0d:00.0" > /sys/bus/pci/drivers/vfio-pci/bind


# Bit Sizes
# 1 = 2MB
# 2 = 4MB
# 3 = 8MB
# 4 = 16MB
# 5 = 32MB
# 6 = 64MB
# 7 = 128MB
# 8 = 256MB
# 9 = 512MB
# 10 = 1GB
# 11 = 2GB
# 12 = 4GB
# 13 = 8GB
# 14 = 16GB
# 15 = 32GB

 

As stated the patch and the script works, but it was ultimately unneeded with my 4070 Ti since passthrough works when I have Above 4G and Resize Bar enabled in my bios. With those enabled the 4070 Ti defaults to a 16GB bar size, so there is no need to manually change it. Other setups/GPUs will give a code 43 or not passthrough at all with Resize Bar enabled in the bios, which is where this patch would be helpful to manually resize the bar. In that case I'm not sure what GPU-Z would report. The only way to know if it's making a difference is to benchmark a game known to benefit from resize bar with and without manipulating the bar size.

Do you think it is worth adding into the GUI. I have Z690 with ReBar enabled and all cards are being set to the max where it can be.

 

 

root@computenode:/tmp# lspci -vvvs 08:00.0 | grep BAR
root@computenode:/tmp# lspci -vvvs 0d:00.0 | grep BAR
                Vector table: BAR=0 offset=00003000
                PBA: BAR=0 offset=00002000
root@computenode:/tmp# lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
03:00.0 VGA compatible controller: Intel Corporation DG2 [Arc A770] (rev 08)
08:00.0 VGA compatible controller: NVIDIA Corporation GK106GL [Quadro K4000] (rev a1)
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 24 [Radeon RX 6400/6500 XT/6500M] (rev c7)
root@computenode:/tmp# lspci -vvvs 0c:00.0 | grep BAR
        Capabilities: [200 v1] Physical Resizable BAR
                BAR 0: current size: 4GB, supported: 256MB 512MB 1GB 2GB 4GB
                BAR 2: current size: 256MB, supported: 2MB 4MB 8MB 16MB 32MB 64MB 128MB 256MB
root@computenode:/tmp# lspci -vvvs 08:00.0 | grep BAR
root@computenode:/tmp# lspci -vvvs 00:02.0 | grep BAR
root@computenode:/tmp# lspci -vvvs 03:00.0 |grep BAR
        Capabilities: [420 v1] Physical Resizable BAR
                BAR 2: current size: 16GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB
root@computenode:/tmp# lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
03:00.0 VGA compatible controller: Intel Corporation DG2 [Arc A770] (rev 08)
08:00.0 VGA compatible controller: NVIDIA Corporation GK106GL [Quadro K4000] (rev a1)
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 24 [Radeon RX 6400/6500 XT/6500M] (rev c7)
root@computenode:/tmp# 

 

I can look at having and option for Changing the 64-bit MMIO window

BTW I am on Kernel 6.1

 

Will speak to Libvirt team to see if there is an option to do via XML rather than QEMU parms.

 

[Interstellar]

Having read through this topic it sounds like that Resize BAR sort-of works with AMD as long as you have kernel v6?

 

My RX6600 with Resize BAR off in BIOS currently shows:

 

lspci -vvvs 03:00.0 | grep BAR
Capabilities: [200 v1] Physical Resizable BAR
BAR 0: current size: 256MB, supported: 256MB 512MB 1GB 2GB 4GB 8GB
BAR 2: current size: 2MB, supported: 2MB 4MB 8MB 16MB 32MB 64MB 128MB 256MB

 

When I turn it on I get a Code 43 error in the VM.

 

It'd be nice for it to work, get a few extra FPS, so I assume I need to wait for the kernel to be upgraded then add the xml changes?

 

 

[KptnKMan]

2 hours ago, Interstellar said:

Having read through this topic it sounds like that Resize BAR sort-of works with AMD as long as you have kernel v6?

 

When I turn it on I get a Code 43 error in the VM.

 

It'd be nice for it to work, get a few extra FPS, so I assume I need to wait for the kernel to be upgraded then add the xml changes?

You can download the kernel in this same thread, and follow the checklist in my summarised post, you don't need to wait until a new kernel.

Also, another user reported that you may not need the 6.1 kernel, but I have not had the time to confirm that as yet.

 

But as it is, yes you need the kernel patch and xml changes.

[DeadDevil6210]

On 1/5/2023 at 1:39 AM, Skitals said:

I back ported the resizable bar patch to unraid 6.11.5 (kernel 5.19.17). It applies and builds cleanly with the other unraid patches and unraid .config file. I attached the patch and a precompiled kernel.

 

I can confirm it's working (resourceN_resize property present), but making it useful is another trick.

resize-bar.patch 7.22 kB · 13 downloads unraid-6.11.5-kernel.zip 5.97 MB · 20 downloads

@Skitalscould you maybe apply the patch to thor2002 unraid kernel with linux 6.2 rc8 kernel? It would be awesome for those who have an Intel arc dgpu since those cards need linux 6.1 at least for propper support.

https://github.com/thor2002ro/unraid_kernel/releases

[SimonF]

4 hours ago, DeadDevil6210 said:

@Skitalscould you maybe apply the patch to thor2002 unraid kernel with linux 6.2 rc8 kernel? It would be awesome for those who have an Intel arc dgpu since those cards need linux 6.1 at least for propper support.

https://github.com/thor2002ro/unraid_kernel/releases

You dont need the patch in 6.2 it was back ported to 5.x for Unraid 6.11.5 but 6.2 will support as newer than 6.1

Edited February 18, 2023 by SimonF

[KptnKMan]

On 1/31/2023 at 2:05 AM, KptnKMan said:

Looks like I figured it out, I had left out the steps to add the extra lines to my Win11 VM, which enabled 64GB ReBAR support.

 

So looks like the checklist to enable this:

- Host BIOS Enable ReBAR support

- Host BIOS Enable 4G Decoding

- Enable & Boot Custom Kernel syslinux configuration (near beginning of this thread)

- Boot Unraid in UEFI Mode

- VM must use UEFI BIOS

- VM must have the top line of XML from <domain type='kvm'> to:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

- VM must have added the following (after the </device> line, before the </domain> line):

  <qemu:commandline>
    <qemu:arg value='-fw_cfg'/>
    <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/>
  </qemu:commandline>

 

After that, looks like everything worked for me as well.

I'm just summarising this for anyone looking for a complete idea.
 

I'll be testing performance over the next weeks as well to see if I'm seeing any improvement.

 

This is great, exactly what I've been waiting for!

An update from my side, I've tested this multiple times now.

The custom ReBAR-enabled firmware is definitely necesarry for me, I've tried booting from the standard kernel enough times.

When I boot standard, booting my 3090-attached VM, I get a black screen with nothing but a blinking text cursor on the screen.

I've tried this a few many times now, and can confirm it.

 

On 2/2/2023 at 6:52 PM, Trozmagon said:

 

Just a heads up, the custom kernel wasn't necessary for me with my RTX 3090, just adding the additional xml to my VM config was enough.

What hardware are you using?

Can you list out your servers specs and unraid version?

 

[Trozmagon]

On 5/3/2023 at 1:00 PM, KptnKMan said:

What hardware are you using?

Can you list out your servers specs and unraid version?

 

Sorry mate, I'm just put of town at the moment, I'll post my specs here either tomorrow or maybe Sunday.

[Trozmagon]

On 5/3/2023 at 1:00 PM, KptnKMan said:

An update from my side, I've tested this multiple times now.

The custom ReBAR-enabled firmware is definitely necesarry for me, I've tried booting from the standard kernel enough times.

When I boot standard, booting my 3090-attached VM, I get a black screen with nothing but a blinking text cursor on the screen.

I've tried this a few many times now, and can confirm it.

 

What hardware are you using?

Can you list out your servers specs and unraid version?

 

 

CPU: i9-11900k

MOBO: MSI MEG z590i Unify

RAM: Patriot Viper 4 Blackout Series DDR4 64GB (2 x 32GB) 3600MHz

GPU: Gigabyte RTX 3090 Turbo Edition

PSU: Asus ROG Loki SFX-L 850W

SSD: 2x 970 EVO Plus 2TB (Cache Drives)

HDD Internal: 4x Seagate Barracuda 2tb 2.5"

HDD External: 5x Seagate Barracuda 8tb 3.5"

HDD External Caddy: Terramaster D5 Thunderbolt 3 (RocketRAID 2720)

 

It could be worth noting that my Unraid USB boot drive is passed through from the Thunderbolt 3 dock, this allowed me to free up the internal USB controller on the Motherboard to passthrough to my Windows 11 VM. Also aside from adding some drives and upgrading my PSU recently my hardware has been the same since owning the 3080 and the 6900 XT.

 

[KptnKMan]

7 hours ago, Trozmagon said:

 

CPU: i9-11900k

MOBO: MSI MEG z590i Unify

RAM: Patriot Viper 4 Blackout Series DDR4 64GB (2 x 32GB) 3600MHz

GPU: Gigabyte RTX 3090 Turbo Edition

PSU: Asus ROG Loki SFX-L 850W

SSD: 2x 970 EVO Plus 2TB (Cache Drives)

HDD Internal: 4x Seagate Barracuda 2tb 2.5"

HDD External: 5x Seagate Barracuda 8tb 3.5"

HDD External Caddy: Terramaster D5 Thunderbolt 3 (RocketRAID 2720)

 

It could be worth noting that my Unraid USB boot drive is passed through from the Thunderbolt 3 dock, this allowed me to free up the internal USB controller on the Motherboard to passthrough to my Windows 11 VM. Also aside from adding some drives and upgrading my PSU recently my hardware has been the same since owning the 3080 and the 6900 XT.

 

Hmm, I wonder if this is related to my AMD Ryzen configuration?

I've noticed some differences when using an AMD Ryzen setup, but honestly its been pretty great for 99% of the time I've had it.

This kernel patch allowed me to properly pass the black EFI bootup screen and get in-VM ReBAR support.

Before, I could only pass through my 3090 by booting Legacy BIOS on host.

 

Posting in case it changes in future:

CPU: Ryzen 9 5950x

Mobo: ASUS TUF GAMING X570-PLUS (WI-FI)

Memory: 128GB (4x32GB) Kingston Unbuffered ECC PC4-25600 DDR4-3200

GPU: Gigabyte RTX3090 Turbo 24G

Storage: 1x980 Pro 1TB NVME (Cache+VMs), 1x970 EVO Plus 1TB NVME (VMs), 860 EVO 256GB SSD (VMs), 840 EVO 1TB SSD (VMs), 2x WD Red 8TB (Parity), 6x WD Red 8TB (Storage).

Networking: 10Gbit with Mellanox ConnectX-3 Dual-port NIC, 1Gbit with onboard Realtek L8200A.

Edited May 8, 2023 by KptnKMan

[N3z]

On 1/22/2023 at 2:36 AM, Skitals said:

 

Is the pci address of your GPU really the same as mine? The GPU needs to be unbound. The previous line in my script unbinds it from vfio, but it also can't be bound to an Nvidia driver or anything else. I also believe you need Above 4G Decoding enabled in your bios, but I'm not sure if that would give you the error you are seeing. 

 

Before referencing my script you should really read and understand the reddit post I linked. You need to be able to read and manipulate the bar sizes on your own because the addresses, paths, number of bars etc will be different than mine.

 

Edit: I just looked up your video card and see it has 8GB VRAM. Why are you trying to set the bar size to 16GB? Like I said you really need to understand the reddit post I linked before preceding. 

I am also using to stop Unraid from reserving the my GPU because my motherboard doesn't have built in Graphics

video=efifb:off

I read over the Reddit post and this forum post several times and am trying to figure out the order of operations.

• 6.11.5 it resize patch applied
• Unraid is already in UEFI mode
• Rebar and 4G decoding enabled in the BIOS
• VM XML edited

Where things get fuzzy is in the unbinding, rebar buffer size change, and rebinding the GPU. I assume this will need to be done every time Unraid is booted.

 

I have tested this several times, with the VM off because the GPU can't be bound.

Factoring in the above video=efifb:off setting, does it prevent the unbind command from working?

Just need to think out loud a bit, but I think I'm close to getting this working. I'm running a 4070 Ti and AMD chipset.

[KptnKMan]

Hey @N3z maybe can help, but would also be good if you can post your full specs because it helps understand specific hardware issues.

Stuff like what's in my signature or others have posted here, you should especially indicate what GPU and CPU/platform you're using.

 

Anyway, I see you're on AMD with a 4070Ti.

Here is the checklist that I figured would be important:

On 1/31/2023 at 2:05 AM, KptnKMan said:

So looks like the checklist to enable this:

- Host BIOS Enable ReBAR support

- Host BIOS Enable 4G Decoding

- Enable & Boot Custom Kernel syslinux configuration (near beginning of this thread)

- Boot Unraid in UEFI Mode

- VM must use UEFI BIOS

- VM must have the top line of XML from <domain type='kvm'> to:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

- VM must have added the following (after the </device> line, before the </domain> line):

  <qemu:commandline>
    <qemu:arg value='-fw_cfg'/>
    <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/>
  </qemu:commandline>

 

I highlighted maybe where you need to look for issues, if you have everything else done?

If you're actually booting the new patched kernel (I found that for AMD the new kernel is a must), and everything else is in place then you need to create a userscript to run "At Startup of Array" that does the unbind.

 

At least that's what I do, and seems to work.

[Interstellar]

On 1/31/2023 at 1:05 AM, KptnKMan said:

Looks like I figured it out, I had left out the steps to add the extra lines to my Win11 VM, which enabled 64GB ReBAR support.

 

So looks like the checklist to enable this:

- Host BIOS Enable ReBAR support

- Host BIOS Enable 4G Decoding

- Enable & Boot Custom Kernel syslinux configuration (near beginning of this thread)

- Boot Unraid in UEFI Mode

- VM must use UEFI BIOS

- VM must have the top line of XML from <domain type='kvm'> to:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

- VM must have added the following (after the </device> line, before the </domain> line):

  <qemu:commandline>
    <qemu:arg value='-fw_cfg'/>
    <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/>
  </qemu:commandline>

 

After that, looks like everything worked for me as well.

I'm just summarising this for anyone looking for a complete idea.
 

I'll be testing performance over the next weeks as well to see if I'm seeing any improvement.

 

This is great, exactly what I've been waiting for!

 

 

So this does not, it seems, work for AMD cards because I suspect it sets BAR 2 to a size above 8MB automatically (256MB) and it seems that this causes Error 43 in guests every time.

 

I've had to manually do it, for example for a 8GB card, 8GB/8MB:

 

echo 13 > /sys/bus/pci/devices/0000:03:00.0/resource0_resize 
echo 3 > /sys/bus/pci/devices/0000:03:00.0/resource2_resize 

 

 

GPU-Z now reports as it being enabled, AMD reports it not being enabled but Windows 10 device manager shows "Large Memory Range".

 

Performance... absolutely zero difference on/off, so who knows.

 

[jonathanselye]

How to make this work with unraid 6.12? which natively has 6.1 kernel.

[KptnKMan]

7 minutes ago, jonathanselye said:

How to make this work with unraid 6.12? which natively has 6.1 kernel.

I haven't tested 6.12 yet (waiting for stable release) but my understanding is that with unraid 6.12 being native on kernel 6.1, this should already be implemented and not need to use the custom kernel.

 

You will still need to do all the other modifications that I listed earlier for convenience, to my understanding.

 

If anyone has tested this on the latest RC release, perhaps they can shed more light on if that is the case?

[jonathanselye]

19 minutes ago, KptnKMan said:

I haven't tested 6.12 yet (waiting for stable release) but my understanding is that with unraid 6.12 being native on kernel 6.1, this should already be implemented and not need to use the custom kernel.

 

You will still need to do all the other modifications that I listed earlier for convenience, to my understanding.

 

If anyone has tested this on the latest RC release, perhaps they can shed more light on if that is the case?

actually the stable is already released but unfortunately it is not natively supported(rebar)

Edited June 15, 2023 by jonathanselye

[KptnKMan]

10 minutes ago, jonathanselye said:

actually the stable is already released but unfortunately it is not natively supported(rebar)

Oh damn, it was released yesterday, I can see that now.

I guess I'll get around to testing that at some point.

 

Well, @jonathanselye can you at least tell us more about what you are doing, what you are running, and EXACTLY what you've encountered?

So far you've made very clear that it doesn't work, but we can't mind read so have literally no idea what could be wrong.

[KptnKMan]

Hi @Skitals I took another look at the kernel patch you linked earlier.

As far as I can tell, this patch is present in kernel 6.1 since v6.1-rc1 which is now present in unraid 6.12.

 

So I wanted to ask, on the surface do you know if there would be any obvious reason that this wouldn't be present, or not work in unraid 6.12?

 

Also, I'm yet to test 6.12 so I haven't personally verified any issues of missing ReBar in unraid 6.12 using kernel 6.1.

 

@jonathanselye can you confirm and describe what you've done to test on unraid 6.12, what hardware you use, and what your exact results were?

[jonathanselye]

1 hour ago, KptnKMan said:

Oh damn, it was released yesterday, I can see that now.

I guess I'll get around to testing that at some point.

 

Well, @jonathanselye can you at least tell us more about what you are doing, what you are running, and EXACTLY what you've encountered?

So far you've made very clear that it doesn't work, but we can't mind read so have literally no idea what could be wrong.

 

55 minutes ago, KptnKMan said:

Hi @Skitals I took another look at the kernel patch you linked earlier.

As far as I can tell, this patch is present in kernel 6.1 since v6.1-rc1 which is now present in unraid 6.12.

 

So I wanted to ask, on the surface do you know if there would be any obvious reason that this wouldn't be present, or not work in unraid 6.12?

 

Also, I'm yet to test 6.12 so I haven't personally verified any issues of missing ReBar in unraid 6.12 using kernel 6.1.

 

@jonathanselye can you confirm and describe what you've done to test on unraid 6.12, what hardware you use, and what your exact results were?

Apologies for my lack of information i actually did not apply the patch for 6.12 because i might break some things and i am asking this thread for the go signal if it is safe.

For the hardware:

the gpu is intel arc a770 i am using if in VM for TDarr and ReBar increase its performance for around 10-30% in transcoding so it is nice to have, and as per the page 1 of the thread it is said that it should be coming in in 6.1 so i am wondering if there is just some codes i need to enable or type in the command line so the reply.

 

Thanks for the quick reply!

[KptnKMan]

12 minutes ago, jonathanselye said:

Apologies for my lack of information i actually did not apply the patch for 6.12 because i might break some things and i am asking this thread for the go signal if it is safe.

Yeah, I wouldn't expect that you would need to apply the patch for unraid 6.12, because to my understanding it should already be in the kernel 6.1.

The patch was built on an earlier kernel 5.19.17, with kernel 6.1 additions, so that sounds like a bad idea. I also would not recommend trying to run the custom compiled kernel under unraid 6.12, I suspect you'll have a bad time.

 

I think you do need to make sure that you have followed all the other requirements/steps I summarised in an earlier comment in this thread, that should give you a decent baseline for cmparison.

 

I mean here, as far as I can tell this is still the best summary to make this work: 

 

Interesting that you're using an Intel Arc GPU, I've read a lot that those GPUs definitely benefit from ReBar.

I'd say in theory it should work if you set the appropriate parameters, so check the summary linked above.

 

Are you running an Intel or AMD CPU/Motherboard platform?

In particular, what CPU and motherboard are you running your Arc A770 on?

 

Let us know how you get on.

[jonathanselye]

33 minutes ago, KptnKMan said:

Yeah, I wouldn't expect that you would need to apply the patch for unraid 6.12, because to my understanding it should already be in the kernel 6.1.

The patch was built on an earlier kernel 5.19.17, with kernel 6.1 additions, so that sounds like a bad idea. I also would not recommend trying to run the custom compiled kernel under unraid 6.12, I suspect you'll have a bad time.

 

I think you do need to make sure that you have followed all the other requirements/steps I summarised in an earlier comment in this thread, that should give you a decent baseline for cmparison.

 

I mean here, as far as I can tell this is still the best summary to make this work: 

 

Interesting that you're using an Intel Arc GPU, I've read a lot that those GPUs definitely benefit from ReBar.

I'd say in theory it should work if you set the appropriate parameters, so check the summary linked above.

 

Are you running an Intel or AMD CPU/Motherboard platform?

In particular, what CPU and motherboard are you running your Arc A770 on?

 

Let us know how you get on.

I see i will try your method later and will comeback for results.

Yes it benefits greatly from rebar even in gaming, my mother board is x570 proart cpu is ryzen 5950x everything is already enabled in your instruction in bios level, because when i used proxmox i was able to utilize rebar, there are still some transcoding in the queue so i cant do it now i will comeback later with results.

Thanks!

[KptnKMan]

2 minutes ago, jonathanselye said:

I see i will try your method later and will comeback for results.

Yes it benefits greatly from rebar even in gaming, my mother board is x570 proart cpu is ryzen 5950x everything is already enabled in your instruction in bios level, because when i used proxmox i was able to utilize rebar, there are still some transcoding in the queue so i cant do it now i will comeback later with results.

Thanks!

Cool, another AMD Ryzen user, and damn that's a nice motherboad. I considered it myself but I went for the TUF GAMING X570-PLUS instead.

I've been gathering evidence that AMD platform users need to do a few more hoops than Intel users, so definitely follow everything I did that worked for me.

 

Also, I updated my summary comment with a link and details of the Rebar UserScript that I use to set my ReBar size, I'd recommend you check this out and do the same.

 

Best of luck.

[jonathanselye]

1 hour ago, jonathanselye said:

Cool, another AMD Ryzen user, and damn that's a nice motherboad. I considered it myself but I went for the TUF GAMING X570-PLUS instead.

I've been gathering evidence that AMD platform users need to do a few more hoops than Intel users, so definitely follow everything I did that worked for me.

 

Also, I updated my summary comment with a link and details of the Rebar UserScript that I use to set my ReBar size, I'd recommend you check this out and do the same.

 

Best of luck.

Yes it has all the server feature built in 10gb and rich pcie on a consumer motherboard, found the updated post and thanks for updating it, one question and im sorry for the dumb question how do i get my ideal bar size?, and will it work if i already unbinded the gpu to unraid?[using vfio-pcie]

and i only have </devices> line, is that the same?

UPDATE:

Tried it and i think it is working but the driver is not yet available when i did what you instructed in device manager there is a new device called "NF I2C Slave Device" when i tried updating the driver both online and latest virtio the drivers seems not available yet so i think it is working the driver just isnt available yet.

Edited June 15, 2023 by jonathanselye