GPU Passthrough on hp gen8 microserver problems

[SpaceInvaderOne]

Hi Guys... Has anyone sucessfully passed through a graphics card on a gen8 microserver with xeon processor.

 

I have tried both a radeon hd 6450 and a radeon hd4550 and get the same error below. However i have no problem passing through usb devices.

 

internal error: early end of file from monitor: possible problem:

2015-10-13T11:38:36.151441Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted

2015-10-13T11:38:36.151463Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 1

2015-10-13T11:38:36.151469Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to get group 1

2015-10-13T11:38:36.151478Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: Device initialization failed

2015-10-13T11:38:36.151486Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

I have tried modifying my syslinux.cfg but has not worked (I dont really know what i am doing editing this file but used ones i have seen in forums).

I have tried adding my gpu rom file to a share and putting it in the xml.

Has anyone had success with gpu passthough on a gen8. If so which card have you used.

 

I have been trying for a week now and really need help. Thanks

[jonp]

Hi Guys... Has anyone sucessfully passed through a graphics card on a gen8 microserver with xeon processor.

 

I have tried both a radeon hd 6450 and a radeon hd4550 and get the same error below. However i have no problem passing through usb devices.

 

internal error: early end of file from monitor: possible problem:

2015-10-13T11:38:36.151441Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted

2015-10-13T11:38:36.151463Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 1

2015-10-13T11:38:36.151469Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: vfio: failed to get group 1

2015-10-13T11:38:36.151478Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: Device initialization failed

2015-10-13T11:38:36.151486Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=pcie.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

I have tried modifying my syslinux.cfg but has not worked (I dont really know what i am doing editing this file but used ones i have seen in forums).

I have tried adding my gpu rom file to a share and putting it in the xml.

Has anyone had success with gpu passthough on a gen8. If so which card have you used.

 

I have been trying for a week now and really need help. Thanks

 

Have you tried toggling on the PCIe ACS Override setting under Settings -> VM Manager?

[SpaceInvaderOne]

Thanks for advice Jonp, yes I have tried that and it makes no difference

 

Anything else i can try?

[jonp]

When you click on "info" in the top right of the webgui, what does if say next to IOMMU?

[SpaceInvaderOne]

It says enabled.  I have attached some screenshots of systems both withPCIe ACS Override setting on and off. Also the info page in hope you may be able to see a problem there. Many thanks

[SpaceInvaderOne]

Info

[jonp]

Sending you a PM.

[SpaceInvaderOne]

Thankyou Jon sent you pm

[SpaceInvaderOne]

update to my problem.

 

I tried an older nvidea card and managed for that to be passed through and show up in windows device manager (however had black screen on my monitor). However after putting driver in windows error was unable to assign resources to it.

 

I am not interested in using the nvidea card but i was pleased to have actually got a vm to boot with passthrough enabled even though nothing would show onscreen !!

 

However when I put either of my ati radeon cards in (54500 or 6450) I get unraid "error internal error: early end of file from monitor"

 

[viumden]

Same problem for me.

My setup:

hp microserver Gen8

E3 1265L V2

Nvidia GTX750TI with HDMI output

 

Enabled all suggested parameters in this thread, but cannot get the VM started when the GPU is selected for passthrough, the syslog error:

 Tower kernel: vfio-pci 0000:07:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.

 

My lspci:

07:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] (rev a2)

07:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)

 

My modified syslinux.cfg:

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pcie_acs_override=downstream vfio_iommu_typel.allow_unsafe_interrupts=1 initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

# find /sys/kernel/iommu_groups/ -type l

 

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/2/devices/0000:00:06.0
/sys/kernel/iommu_groups/3/devices/0000:00:1a.0
/sys/kernel/iommu_groups/4/devices/0000:00:1c.0
/sys/kernel/iommu_groups/5/devices/0000:00:1c.4
/sys/kernel/iommu_groups/6/devices/0000:00:1c.6
/sys/kernel/iommu_groups/7/devices/0000:00:1c.7
/sys/kernel/iommu_groups/8/devices/0000:00:1d.0
/sys/kernel/iommu_groups/9/devices/0000:00:1e.0
/sys/kernel/iommu_groups/10/devices/0000:00:1f.0
/sys/kernel/iommu_groups/10/devices/0000:00:1f.2
/sys/kernel/iommu_groups/11/devices/0000:07:00.0
/sys/kernel/iommu_groups/11/devices/0000:07:00.1
/sys/kernel/iommu_groups/12/devices/0000:02:00.0
/sys/kernel/iommu_groups/13/devices/0000:03:00.0
/sys/kernel/iommu_groups/13/devices/0000:03:00.1
/sys/kernel/iommu_groups/14/devices/0000:04:00.0
/sys/kernel/iommu_groups/15/devices/0000:01:00.0
/sys/kernel/iommu_groups/15/devices/0000:01:00.1
/sys/kernel/iommu_groups/15/devices/0000:01:00.2
/sys/kernel/iommu_groups/15/devices/0000:01:00.4

 

Also BIOS modification according to this advisory has been applied for PCI Slot 1:

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566

 

Any ideas to go forward?

[SpaceInvaderOne]

I tried for a long time with 4 different graphics cards. Never got gpu passthrough on Gen8 microserver. In end I sold gen 8 and bought different hardware

[viumden]

I tried for a long time with 4 different graphics cards. Never got gpu passthrough on Gen8 microserver. In end I sold gen 8 and bought different hardware

So what is the proven working hardware collection setup now you have bought? Is it with the comparable size, power usage and build quality like MS Gen8?

[SpaceInvaderOne]

I am using the following hardware

Thermaltake Core V21 micro-ATX case

1240v2 xeon cpu

ASRock B75M-GL R2.0

16 gigs ram

Radeon r7 370

Radeon 6450

Drives 4 x 2 tb drives (3 data, 1 parity)

500gb solid state cache

1tb unassigned drive for game data  on windows vms

I choose a micro atx board as it is small but still had 5 sata ports and 2 pci express slots. I choose an Asrock board as i heard they have good vt-d support.

I had upgraded the gen8 microserver processor to a xeon 1240v i kept that cpu.

The case is bigger than the microserver but still a reasonable size.

I also did have a 750ti but changed it to a ati R370 gpu. If you are going to use your nvidea card in your next build, make sure you have built in graphics on the cpu/motherboard else the nvidea will not work in the primary pci express slot. That is why i am using the R370 now

 

I sold the gen8 on ebay and got more than i paid for it!!

 

I am really happy with my setup now. It runs very stable. I have an openelec vm running off my 6450 card, a windows 10 gaming machine off my r370 card, osx el capitan off the 6450. I have a few other vms such as windows xp, window 98, and windows 7 i run though vnc.

 

[DGB]

I have an hp gen8 but would like to upgrade to somthing similar

Do you know how many watts it uses at idle?

[SpaceInvaderOne]

Im not sure what its watt usage is idle.

I wanted a low wattage machine aswell. The original cpu i had in the microserver was 35 watts. I changed it to a 1240 xeon at 69 watts.

So as i am using the same cpu in my new build and same hardrives i figured the wattage would be similar. Aswell I have the server automatically sleep from 12am to 8 am.

As your chip is a xeon 1265l it only uses 45 watts so if you kept that i would think your power consumption would be quite low. Unraid spins down disks not in use. If you have a solid state cache and installed you vms, dockers etc on that then the main disks in the array wouldnt need to spin up often. Anything downloaded would be downloaded to the cache then mover can move it to the array once a day.