Help requested for AMD Ryzen 5 Pro 4650g Passthrough


DrMucki

Recommended Posts

Hi there. I saw many videos from space invader and read all along this Forum but I did not find a solution so far.
I am just testing Unraid 6.8.3. and everything works fine. I set up a windows 10 VM but wanted to improve the video quality, for using this VM for games and video editing purposes.

My question is : Is it possible to passthrough the graphics from the AMD Ryzen APU to the windows VM?

 

I have got a AMD Ryzen 5 Pro 4650g APU with graphics in chip. and wanted to passthrough the graphics to a VM..  there is no further GPU in the server.

 

 

The System Devices shows the card as:

IOMMU group 14:[1002:1636] 05:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Renoir (rev d9)

it is the only device in this group.

Ok so far dos good. I tried to change the graphics from VNC to AMD Renoir (05:00.0) and started the VM.

The machine is not getting online. I cannot connect via Teamviewer or VNC.

the log file (1) is like is at the end...

Ok i do not have a bios file. there is none "on tech power up".

So I tried to dump the bios file with the script provided by space invader.

Installed the script and it gave an error that bios couldn't be dumped and I should install vife config plugin to bind the graphics to the vfio.

I installed the plugin and run the script which binds the card to vfio. I rebooted the system (btw I only use web access, there is no monitor attached to the server)

After retrying dumbing the bios again gave the same error: (Error 1).

I tried to restart the VM with this binding and it resulted in the same problem. The vm started but was not accessible.

 

Any help is welcome.... Or do I have to just buy another gpu??

 

Thank you for your help in Advance!

 

Marc

 

 

 

Error 1:

 

 

"Script location: /tmp/user.scripts/tmpScripts/Dump VBIOS/script
Note that closing this window will abort the execution of this script
You have selected this device to dump the vbios from
05:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Renoir (rev d9)

This does look like a valid GPU to me. Continuing .........


Checking if location to put vbios file exists
Vbios folder already exists

I will try and dump the vbios without disconnecting and reconnecting the GPU
This normally only works if the GPU is NOT the Primary or the only GPU
I will check the vbios at the end. If it seems wrong I will then retry after disconnecting the GPU

Defining temp vm with gpu attached
Domain dumpvbios defined from /tmp/dumpvbios.xml

Starting the temp vm to allow dump
Domain dumpvbios started

Waiting for a few seconds .....

Stopping the temp vm
Domain dumpvbios destroyed

Removing the temp vm
Domain dumpvbios has been undefined


/tmp/user.scripts/tmpScripts/Dump VBIOS/script: line 298: rom: Permission denied

Okay dumping vbios file named AMD-APU-Ryzen4650G.rom to the location /mnt/user/isos/vbios/
cat: rom: No such file or directory

Um.... somethings gone wrong and I couldn't dump the vbios for some reason
Sometimes when this happens all we need to do to fix this is 'stub' or 'bind to the vfio' the gpu and reboot the server

This can be done in Unraid 6.8.3 with the use of the vfio config plugin or if you are on Unraid 6.9 or above it can be done
directly from the gui in Tools/System Devices .....So please do this and run the script again"

 

 

 

 

LOGFILE 1:

 

 

2020-12-30 14:03:05.783+0000: starting up libvirt version: 5.10.0, qemu version: 4.2.0, kernel: 4.19.98-Unraid, hostname: Tower
LC_ALL=C \
PATH=/bin:/sbin:/usr/bin:/usr/sbin \
HOME='/var/lib/libvirt/qemu/domain-32-Windows 10' \
XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-32-Windows 10/.local/share' \
XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-32-Windows 10/.cache' \
XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-32-Windows 10/.config' \
QEMU_AUDIO_DRV=none \
/usr/local/sbin/qemu \
-name 'guest=Windows 10,debug-threads=on' \
-S \
-object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-32-Windows 10/master-key.aes' \
-blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/f0686497-8ef9-51e9-56fa-27f8c5a495d6_VARS-pure-efi.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-i440fx-4.2,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-cpu host,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vendor-id=none \
-m 4096 \
-overcommit mem-lock=off \
-smp 8,sockets=1,cores=8,threads=1 \
-uuid f0686497-8ef9-51e9-56fa-27f8c5a495d6 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-3-format,id=virtio-disk2,bootindex=1,write-cache=on \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Win10_1909_German_x64.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device ide-cd,bus=ide.0,unit=0,drive=libvirt-2-format,id=ide0-0-0,bootindex=2 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.173-2.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-cd,bus=ide.0,unit=1,drive=libvirt-1-format,id=ide0-0-1 \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:db:42:da,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=38,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-device ich9-intel-hda,id=sound0,bus=pci.0,addr=0x9 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-device vfio-pci,host=0000:05:00.0,id=hostdev0,bus=pci.0,addr=0x5 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-12-30 14:03:05.783+0000: Domain id=32 is tainted: high-privileges
2020-12-30 14:03:05.783+0000: Domain id=32 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2020-12-30T14:03:19.211200Z qemu-system-x86_64: warning: guest updated active QH

 

 

Edited by DrMucki
Link to comment
  • 1 month later...

Hi, did you find the solution? I have the same issue - I would like to use GPU of AMD RYZEN 5 PRO 4650G but it doesn't work properly. 

I was able to create Windows 10 VM using integrated GPU when I use seabios but when I update the GPU driver in Windows, then it seems it is not supported - only black screen, I'm able to connect via TeamViewer but I can't fix issue with GPU driver in Win....

Edited by Mato
Link to comment
  • 2 weeks later...

Is there a solution to passthrough the Renoir GPU to a Windows VM by now? When I choose the Renoir GPU in the config I get: "internal error: unknown pci source type 'vendor'".

I really would appreciate to use it in VMs. Also it is not possible to get a BIOS file for an APU I think...

 

After fixing the Iommu Group I get this: internal error: Unknown PCI header type '127' for device '0000:0d:00.0'

A BIOS dump of the internal gpu is not possible.

Edited by TGP
Link to comment

I am about to purchase a Renoir APU (4650G or 4750G) and would also like to pass through the iGPU to a VM, either via unraid or Proxmox VE. Unfortunately I haven't seen anyone do it successfully yet throughout all my research, and if they do get it, it seems to be "unstable", but they don't describe any errors...

Link to comment
On 3/4/2021 at 5:19 AM, TGP said:

Is there a solution to passthrough the Renoir GPU to a Windows VM by now? When I choose the Renoir GPU in the config I get: "internal error: unknown pci source type 'vendor'".

I really would appreciate to use it in VMs. Also it is not possible to get a BIOS file for an APU I think...

 

After fixing the Iommu Group I get this: internal error: Unknown PCI header type '127' for device '0000:0d:00.0'

A BIOS dump of the internal gpu is not possible.

 

Have you checked this post out? That message (type 127) seems to be an AMD thing when the GPU can't reset properly: https://forums.unraid.net/topic/76510-execution-error-internal-error-unknown-pci-header-type-127/

 

EDIT: it may be possible to extract iGPU VBIOS with UEFITool, but I believe this is Windows-only (unsure, just found this online), see https://github.com/LongSoft/UEFITool/issues/26

 

EDIT2: I think I got the Renoir APU VBIOS, I will PM you and if it works, I'll make it public.

Edited by CodingMicrobe
Link to comment
9 hours ago, CodingMicrobe said:

 

Have you checked this post out? That message (type 127) seems to be an AMD thing when the GPU can't reset properly: https://forums.unraid.net/topic/76510-execution-error-internal-error-unknown-pci-header-type-127/

 

EDIT: it may be possible to extract iGPU VBIOS with UEFITool, but I believe this is Windows-only (unsure, just found this online), see https://github.com/LongSoft/UEFITool/issues/26

 

EDIT2: I think I got the Renoir APU VBIOS, I will PM you and if it works, I'll make it public.

I would really appreciate your APU Vbios for the 4650.. can you post it here of send it to me via PN... on which unraid version are you? Did you get it to passthrough for windows VM or even with macinabox?

how did you manage to get it? I tried to use the tool of Spaceinvader one, but that did not work...

Link to comment

 

1 hour ago, DrMucki said:

I would really appreciate your APU Vbios for the 4650.. can you post it here of send it to me via PN... on which unraid version are you? Did you get it to passthrough for windows VM or even with macinabox?

how did you manage to get it? I tried to use the tool of Spaceinvader one, but that did not work...

 

Hi,


Attached is the Renoir VBIOS I extracted from the ASRock DeskMini X300 BIOS file. I am still waiting on my APU to arrive so can't test it yet, but @TGP seems to get an error when using it about failing to allocate BAR space (I attached their error they PM'ed me below). They said they were using ACS override patch to make the iGPU in its own IOMMU group, but I wonder if this may be the cause of their issue. Also came across this post where someone has the same error and resolves it, may be worth looking into (see link below).

Let me know how this VBIOS works! I'd love to get this working with a Windows VM eventually like you!

 

Cheers.

 

2021-03-07T10:19:06.269103Z qemu-system-x86_64: -device vfio-pci,host=0000:0c:00.0,id=hostdev0,bus=pci.0,addr=0x5,romfile=/mnt/cache/Public/ISO/RenoirVBIOS.rom: Failed to mmap 0000:0c:00.0 BAR 0. Performance may be slow

 

 

RenoirVBIOS.rom

Link to comment

Thank you.

I am just trying to get it to work with your bios...

What I did so far...

Set up a Win 10 VM which is working with VNC or with a AMD GPU.

For the AMD GPU and the iGPU Part and their sound part are both bound to VFIO..

First tried the VNC --> Working with all resolutions

tried my GPU with my extracted Vbios... working perfectly

changed to AMD Renoir with your bios... first I got an error in the log file. Started it with no changes a second time...

no Errors are shown.. and yeah Splashtop connects... windows login screen is coming up (ugly resolution, black and white). You can log onto it and it takes ages to show a screen.

Ok its color but the resolution.... and there it comes it very very laggy on keyboard or even. mouse inputs . I was not able to interact with the machine at all....

 

May be you can tell me how you extracted the bios... I will give it a try on mine 4650g...

 

Link to comment
6 minutes ago, DrMucki said:

Thank you.

I am just trying to get it to work with your bios...

What I did so far...

Set up a Win 10 VM which is working with VNC or with a AMD GPU.

For the AMD GPU and the iGPU Part and their sound part are both bound to VFIO..

First tried the VNC --> Working with all resolutions

tried my GPU with my extracted Vbios... working perfectly

changed to AMD Renoir with your bios... first I got an error in the log file. Started it with no changes a second time...

no Errors are shown.. and yeah Splashtop connects... windows login screen is coming up (ugly resolution, black and white). You can log onto it and it takes ages to show a screen.

Ok its color but the resolution.... and there it comes it very very laggy on keyboard or even. mouse inputs . I was not able to interact with the machine at all....

 

May be you can tell me how you extracted the bios... I will give it a try on mine 4650g...

 

 

So it sounds like you got it working, but it was very very poor performance, with poor resolution and lag on keyboard/mouse inputs?

What specific motherboard & BIOS version do you have? I can extract the VBIOS from that version of the BIOS for you. Maybe there is a newer version.

Link to comment
1 hour ago, CodingMicrobe said:

 

So it sounds like you got it working, but it was very very poor performance, with poor resolution and lag on keyboard/mouse inputs?

What specific motherboard & BIOS version do you have? I can extract the VBIOS from that version of the BIOS for you. Maybe there is a newer version.

Yeah that is right.

 

i have a ASUS PRIME A520M-K.

Unraid gives the following Infos: 

ASUSTeK COMPUTER INC. PRIME A520M-K, Version Rev X.0x
American Megatrends Inc., Version 1401
BIOS dated: Fri 04 Dec 2020 12:00:00 AM CET

 

I know that there is a newer bios but I did not install it so far

Link to comment
1 hour ago, CodingMicrobe said:

@DrMucki also why use Splashtop? Can you try just plugging in an HDMI/VGA/DP cable directly to your motherboard to see if you get output from iGPU Windows VM - without Splashtop at all? That can help narrow down if the issue is with Splashtop & AMD iGPUs, or with the passthrough itself.

It is a headless system... there is no Mouse/keyboard or even a monitor.... Just SSH and web....I will give it a try with the monitor within the next days.. I have to completely remove the system. which is within a server rack... but not today anymore and with this I can go the latest bios as well

Link to comment
34 minutes ago, DrMucki said:

It is a headless system... there is no Mouse/keyboard or even a monitor.... Just SSH and web....I will give it a try with the monitor within the next days.. I have to completely remove the system. which is within a server rack... but not today anymore and with this I can go the latest bios as well

Here is the VBIOS extracted from BIOS version 1401 for the ASUS PRIME A520M-K, it is slightly newer than the one I posted above. The one I posted was compiled by AMD March 10 2020, this one was compiled August 25 2020). Hopefully it improves things, because taking the server out of the rack is a hassle I agree...

 

I also attached the *latest* VBIOS (compiled by AMD on November 13 2020), from BIOS version 1804. I haven't found any more recent VBIOS yet.

 

For others who come across this post, I named VBIOS files with the motherboard BIOS&version I extracted them from, but in theory they are universal for all Renoir APUs.

 

So far I have extracted 3 Renoir APU VBIOSs, compiled/released by AMD on these dates:

  • March 10 2020 (from ASRock DeskMini X300, BIOS v1.40; attached above in my other post)
  • August 25 2020 (from ASUS A520MK, BIOS v1401; also found in ASTock B550M-ITX/ac, BIOS v1.50; attached here)
  • November 13 2020 (from ASUS A520MK, BIOS v1804; attached here)

 

A520MK-1401-RenoirVBIOS.ROM

A520MK-1804-RenoirVBIOS.ROM

Edited by CodingMicrobe
  • Like 1
Link to comment

Hi!

 

As i have the same problem as you guys with Win 10 VM and 4650G igpu passthrough i thought i'll share my results with the provided vbios.

I am able to get my vm to run and i can also install the amd radeon drivers for it but it does not matter if i attach the vbios or not. When i check in device manager the gpu is listed but says that it encountered errors and is not running.

I don't know how to resolve that error :/

 

Mainboard: Asrock x570m Pro4

Link to comment
7 minutes ago, limital said:

Hi!

 

As i have the same problem as you guys with Win 10 VM and 4650G igpu passthrough i thought i'll share my results with the provided vbios.

I am able to get my vm to run and i can also install the amd radeon drivers for it but it does not matter if i attach the vbios or not. When i check in device manager the gpu is listed but says that it encountered errors and is not running.

I don't know how to resolve that error :/

 

Mainboard: Asrock x570m Pro4

That’s promising that windows device manager can see it! Do you get any VGA/HDMI/DP output when you connect directly to motherboard (not VNC/RDP)? What error does windows give you?

Link to comment
26 minutes ago, DrMucki said:

Thanks a lot. I will give it a try and will post the results here. How did you manage to extract the vbios? Can you explain it to us here. A step by step how to could be useful, even for those who come will find this post here


Good point, I will try to make a guide within a few days! 

Link to comment
1 hour ago, CodingMicrobe said:

That’s promising that windows device manager can see it! Do you get any VGA/HDMI/DP output when you connect directly to motherboard (not VNC/RDP)? What error does windows give you?

No i don't get any output :( only when connecting via teamviewer. The Error code i get is Error 43...

Link to comment
1 minute ago, limital said:

No i don't get any output :( only when connecting via teamviewer. The Error code i get is Error 43...

 

Might be worth trying to run these commands on the Unraid server immediately upon bootup (via SSH! or userscripts plugin), and THEN starting the VM via web UI from another computer. Very important not to use unraid GUI mode or text mode at all.
 

echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

 

Also are you running the VM with SeaBIOS or OVMF? I think SeaBIOS is preferred for this type of pass-through, since the VBIOSs provided by AMD don't support UEFI firmware (which is what OVMF is).

Link to comment
29 minutes ago, DrMucki said:

sorry for the question... but how to change the bios within Unraid for the vm... it only gives me OVMF...

I believe once you set it for a VM, you cannot easily change it (for Windows VM at least, not sure about other distros of Linux). There is a way to change SeaBIOS to OVMF, but the opposite is much more difficult/impossible - and I think you would just need to make a new VM, at least for testing purposes now.

Edited by CodingMicrobe
Link to comment
On 3/8/2021 at 1:43 AM, CodingMicrobe said:

Here is the VBIOS extracted from BIOS version 1401 for the ASUS PRIME A520M-K, it is slightly newer than the one I posted above. The one I posted was compiled by AMD March 10 2020, this one was compiled August 25 2020). Hopefully it improves things, because taking the server out of the rack is a hassle I agree...

 

I also attached the *latest* VBIOS (compiled by AMD on November 13 2020), from BIOS version 1804. I haven't found any more recent VBIOS yet.

 

For others who come across this post, I named VBIOS files with the motherboard BIOS&version I extracted them from, but in theory they are universal for all Renoir APUs.

 

So far I have extracted 3 Renoir APU VBIOSs, compiled/released by AMD on these dates:

  • March 10 2020 (from ASRock DeskMini X300, BIOS v1.40; attached above in my other post)
  • August 25 2020 (from ASUS A520MK, BIOS v1401; also found in ASTock B550M-ITX/ac, BIOS v1.50; attached here)
  • November 13 2020 (from ASUS A520MK, BIOS v1804; attached here)

 

A520MK-1401-RenoirVBIOS.ROM 53 kB · 2 downloads

A520MK-1804-RenoirVBIOS.ROM 53.5 kB · 5 downloads

hello gays,i am also using amd 4650g apu,and my motherborad is ASUS B550M  PLUS (WIFI),BIOS 1804, i have the same problem,can u extract this version vbios? thanks!

Link to comment
4 hours ago, elovsa said:

hello gays,i am also using amd 4650g apu,and my motherborad is ASUS B550M  PLUS (WIFI),BIOS 1804, i have the same problem,can u extract this version vbios? thanks!

 

Hello,

I had more discussion with other users from other forums and the version of VBIOS (or from which motherboard it came) shouldn't matter.

These are the 3 versions I have extracted, 017.010.000.015, 017.010.000.026, 017.010.000.028. Same as I posted above just renamed properly for their versions.

 

Renoir-017.010.000.015.000000.ROM Renoir-017.010.000.026.000000.ROM Renoir-017.010.000.028.000000.ROM

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.