AMD GPU Passthrough Woes


Recommended Posts

Hi all,

I was able to get my RX470 passed through by adding the lines to my syslinux.cfg, however when i try and use my R9 290x, or R9 295x2 i get different results...

The RX 470 does not work, whilst (or since) the other cards are connected to the system. I can't easily disconnect/move cards as the 2xx cards are water cooled.

 

The 290x machine doesn't even start up.

The 295x2 machine starts, but hangs at the SeaBIOS boot page.

 

 

They're each in seperate IMMOU groups, although the audio portions are a little skew, but i dont think this is the issue...

IOMMU group 20
	[1002:67b9] 0a:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vesuvius [Radeon R9 295X2]
	[1002:aac8] 0a:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X]
IOMMU group 21
	[1002:67b9] 09:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Vesuvius [Radeon R9 295X2]
IOMMU group 22
	[1002:67b0] 06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X]
	[1002:aac8] 06:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X]

libvirt.txt

2017-05-25 11:47:18.695+0000: 10627: warning : qemuDomainObjTaint:3885 : Domain id=6 name='Win 7 Controller' uuid=28cc97b3-3d6f-8dd5-155a-443a9554dc86 is tainted: high-privileges
2017-05-25 11:47:18.695+0000: 10627: warning : qemuDomainObjTaint:3885 : Domain id=6 name='Win 7 Controller' uuid=28cc97b3-3d6f-8dd5-155a-443a9554dc86 is tainted: host-cpu
2017-05-25 11:47:21.953+0000: 10627: error : x86FeatureInData:780 : internal error: unknown CPU feature __kvm_hv_spinlocks
2017-05-25 11:47:21.953+0000: 10627: error : x86FeatureInData:780 : internal error: unknown CPU feature __kvm_hv_vendor_id
2017-05-25 11:47:22.110+0000: 10626: error : qemuMonitorIO:695 : internal error: End of file from qemu monitor

qemu log

2017-05-25 13:48:08.800+0000: starting up libvirt version: 2.4.0, qemu version: 2.7.1, hostname: Yoda
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Ethereum Miner,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-7-Ethereum Miner/master-key.aes' -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -m 2048 -realtime mlock=off -smp 2,sockets=1,cores=1,threads=2 -uuid 721ca320-eb50-fd38-709a-f7d44d388492 -no-user-config -nodefaults -chardev 'socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-7-Ethereum Miner/monitor.sock,server,nowait' -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive 'file=/mnt/user/VMs/Ethereum Miner/vdisk1.img,format=qcow2,if=none,id=drive-ide0-1-0,cache=writeback' -device ide-hd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:78:71:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev 'socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-7-Ethereum Miner/org.qemu.guest_agent.0,server,nowait' -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 0.0.0.0:0,websocket=5700 -k en-us -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x5 -device vfio-pci,host=06:00.0,id=hostdev1,bus=pci.0,addr=0x6 -device vfio-pci,host=09:00.0,id=hostdev2,bus=pci.0,addr=0x8 -device vfio-pci,host=0a:00.0,id=hostdev3,bus=pci.0,addr=0x9 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa -msg timestamp=on
Domain id=7 is tainted: high-privileges
Domain id=7 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)
2017-05-25T13:48:14.769169Z qemu-system-x86_64: VFIO_MAP_DMA: -14
2017-05-25T13:48:14.769211Z qemu-system-x86_64: vfio_dma_map(0x2b0a92b86900, 0xf3000000, 0x200000, 0x2b0aadc11000) = -14 (Bad address)
qemu: hardware error: vfio: DMA mapping failed, unable to continue
CPU #0:
EAX=00000143 EBX=80002804 ECX=00000143 EDX=00000cfc
ESI=00002800 EDI=00000000 EBP=7ffac1c0 ESP=00006d74
EIP=7ffad71a EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000f71a0 00000037
IDT=     000f71de 00000000
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
CPU #1:
EAX=00000000 EBX=00000000 ECX=00000000 EDX=000206c2
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
2017-05-25 13:48:15.970+0000: shutting down, reason=crashed

Syslog (all cards)

May 25 23:48:08 Yoda kernel: vgaarb: device changed decodes: PCI:0000:04:00.0,olddecodes=io+mem,decodes=io+mem:owns=none
May 25 23:48:08 Yoda kernel: vgaarb: device changed decodes: PCI:0000:06:00.0,olddecodes=io+mem,decodes=io+mem:owns=none
May 25 23:48:08 Yoda kernel: vgaarb: device changed decodes: PCI:0000:0a:00.0,olddecodes=io+mem,decodes=io+mem:owns=none
May 25 23:48:08 Yoda kernel: br0: port 2(vnet0) entered blocking state
May 25 23:48:08 Yoda kernel: br0: port 2(vnet0) entered disabled state
May 25 23:48:08 Yoda kernel: device vnet0 entered promiscuous mode
May 25 23:48:08 Yoda kernel: br0: port 2(vnet0) entered blocking state
May 25 23:48:08 Yoda kernel: br0: port 2(vnet0) entered forwarding state
May 25 23:48:09 Yoda kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19@0x270
May 25 23:48:09 Yoda kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b@0x2d0
May 25 23:48:09 Yoda kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1e@0x370
May 25 23:48:09 Yoda kernel: vfio-pci 0000:06:00.0: enabling device (0140 -> 0143)
May 25 23:48:09 Yoda kernel: vfio_ecap_init: 0000:06:00.0 hiding ecap 0x19@0x270
May 25 23:48:09 Yoda kernel: vfio_ecap_init: 0000:06:00.0 hiding ecap 0x1b@0x2d0
May 25 23:48:10 Yoda kernel: vfio_ecap_init: 0000:09:00.0 hiding ecap 0x19@0x270
May 25 23:48:10 Yoda kernel: vfio_ecap_init: 0000:09:00.0 hiding ecap 0x1b@0x2d0
May 25 23:48:10 Yoda kernel: vfio-pci 0000:0a:00.0: enabling device (0140 -> 0143)
May 25 23:48:10 Yoda kernel: vfio_ecap_init: 0000:0a:00.0 hiding ecap 0x19@0x270
May 25 23:48:10 Yoda kernel: vfio_ecap_init: 0000:0a:00.0 hiding ecap 0x1b@0x2d0
May 25 23:48:14 Yoda kernel: br0: port 2(vnet0) entered disabled state
May 25 23:48:14 Yoda kernel: device vnet0 left promiscuous mode
May 25 23:48:14 Yoda kernel: br0: port 2(vnet0) entered disabled state

Not sure else where to look for further info, but it appears as though in either case it is producing the same error, they're just manifesting themselves differently, one posts the other terminates.

 

Can anyone offer some further assistance in troubleshooting this one? I have decent linux knowledge, but never touched anything virtualised or video based, so this is right out of my comfort zone.

 

I actually made the switch from FreeNAS & ESXi on seperate hosts, to a consolidated unRAID box so that i could utilise GPU passthrough and have a couple cards mining away on hardware that was on 24/7 anyway.

 

Cheers,

Nash

Edited by Nasha
RX 470 Update
Link to comment
  • 4 years later...

Hello everyone without success for this attempt to split a bi GPU also started on mac os and identical result supports separate launch on two simultaneous vm but no black screen support and driver error under windows and kext under mac under linux one of the two gpu does not support iommuv2 and I have not yet attempted to make a ssdt for the gpu for the limited to be recognized as a 280x

Capture d’écran 2021-10-29 à 11.39.39.png

Capture d’écran 2021-10-30 à 12.54.01.png

Capture d’écran 2021-10-31 à 09.53.56.png

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.