Hello Unraid Community,

 

I'm new to Unraid and just built a Server with the following specs:

 

Aorus B450 Pro Wifi (because I wanted a small ITX board in s small case)

AMD Ryzen 7 2700

2x16 GB DDR4 3000 Mhz

GTX 1080

at the moment I have 500GB m.2 as cache and 3 hdd (4tb parity, 1TB and 2TB)

 

Now the Problem is that I want to passthrough the 1080 to a Windows 10 VM but I only have 1 GPU.

I saw some awesome Videos from the Spaceinvader One and tried to use a dump file for the gpu.

Now everytime I start a vm using my GPU it fails.

You can see that the monitor is getting some connections but after few seconds the VM freezes.

Also no more Monitor connections.

 

I have to force stop the VM and then when I want to restart it the following Error occurs:

Execution error

internal error: Unknown PCI header type '127' for device '0000:09:00.0' <-- This is my GPU, it also should be in an own IOMMU group

 

And after that when I want to reboot or power off my Server, the machine stays on and I have to manually turn off the power.

But only when I tried to use my GPU in a VM... strange...

 

Hope someone have some tips

I also attached my System Devices

System Devices.txt

Edited January 12, 20206 yr by Wulferine

Have you updated to a BIOS based on AMD AGESA 1.0.0.4 B, that's a requirement for passthrough.

Oh OK, I just checked the Version of my BIOS and it's the F41.

On the Aorus Website there is a new one F50 with the AMD AGESA 1.0.0.4 B.

I will update the BIOS when I'm at home and let you know if it worked.

 

Thank you for that hint!

The error 127 is your GPU isn't resetting itself properly. Motherboard BIOS could be a problem but I would say the most likely culprit is your vbios file.

How did you obtain the vbios?

I followed the tutorial from Spaceinvader One and used a rom file from the techpowerup website.

Also removed the header with an hex editor. But I think I also tried the VM without any vbios files.

Once I started the VM, the display got a connection but after few seconds the VM freezes.

Maybe that's why it fails to reset.

4 minutes ago, Wulferine said:

and used a rom file from the techpowerup website

Why don't you dump your own vbios? SpaceInvaderOne explain how to do it, it's very simple and you can do it from unraid itself.

Well that's a good point😅

I will do that when I did the BIOS update and it's sill freezing.

Does is also work when I have only one PCIe port? Because I thought I can only dump one from the 2nd card.

Otherwise I can dump one with my PC.

32 minutes ago, Wulferine said:

Does is also work when I have only one PCIe port?

mmm, not 100% sure, I don't remember how I dumped my vbios, maybe I used a second pc to connect to unraid 🤨

Since you need to unbind the gpu I think so.

By this way you are 100% the vbios is correct for your own gpu.

Edited January 13, 20206 yr by ghost82

2 hours ago, Wulferine said:

I followed the tutorial from Spaceinvader One and used a rom file from the techpowerup website.

The vbios has to be specific to your exact specimen (brand + model + revision). So it's very likely that you used something that doesn't match your card. It is not uncommon for some models to not have vbios on TPU.

I have even seen a vbios dumped from 2nd slot not working if the card is in the 1st slot but that seems rare (have only seen 1 report).

 

So given you don't have a 2nd GPU, the only thing you can do is to try to get the right vbios from TPU (if it's available).

I believe SIO has a guide on how to dump it from gpuz (which runs from Windows) so if you can somehow get a Windows installation up and running, you may be able to follow that guide to do it.

Alternatively, you can also dump the vbios if you have another computer as well.

 

Passing through the GTX 1080 as only GPU without vbios is unlikely to work due to error code 43. Even with the right vbios, you might still need some workarounds in the xml but we'll deal with that when we get there.

On 1/13/2020 at 4:47 AM, dodgypast said:

Have you updated to a BIOS based on AMD AGESA 1.0.0.4 B, that's a requirement for passthrough.

Thank you!!!

It really solved the problem with the unknown pci header type.

 

So now I started the VM with the dump file I already had.

It works! But now the next problem: No drivers will be installed.

 

So like you mentioned, I think I have to dump my own vbios.

Anything else I have to look for?

 

Edit: I dumped my own vbios but the results are the same. The device manager still shows error 43.

Here's what I already tried:

- Switched from Q and i in the Machine settings

- Added the multifunction parameter in the xml, also added following lines:

   <kvm>
      <hidden state='on'/>
   </kvm>
   <ioapic driver='kvm'/>

- Started the VM with both VNC and the GPU

 

EDIT 2: I don't know what exactly solved the problem, but it's now working with drivers on Linux and Windows.

Something what I know what I did is, I changed this parameter:

 

<vendor_id state='on' value='28a2c82d201d'/>  <ioapic driver='kvm'/>

Just type in a value with a lenghth of 12 random characters.

 

Thanks for all the replies!

Edited January 16, 20206 yr by Wulferine

I am having a similar issue with my VM crashing my Unraid box. My Ubuntu VM boots fine and works well; if I reboot Ubuntu it crashes Unraid sometimes. A reboot of Unraid will trigger the 127 error; it has to be a full shutdown to get the VM back up. I have the following hardware:

Asrock X570 phantom gaming 4s

ryzen 3700x

xfx rx 550

question: I have the latest bios which is not
AMD AGESA 1.0.0.4 B. Do I have to use this Bios regardless if it’s not the latest one?

 

On 7/13/2020 at 10:18 PM, isisyodin said:

I am having a similar issue with my VM crashing my Unraid box. My Ubuntu VM boots fine and works well; if I reboot Ubuntu it crashes Unraid sometimes. A reboot of Unraid will trigger the 127 error; it has to be a full shutdown to get the VM back up. I have the following hardware:

Asrock X570 phantom gaming 4s

ryzen 3700x

xfx rx 550

question: I have the latest bios which is not
AMD AGESA 1.0.0.4 B. Do I have to use this Bios regardless if it’s not the latest one?

 

I also started having this problem lately. Updating the BIOS didn't help. Through long experimentation, I came to the conclusion that the graphics driver ADM is to blame.
Rolled back to version 20.5.1 - there were no crashes yet.

After nearly two weeks of no problems with my new RTX2060-Super (ASUS), today I get the 127 error. 

Anyone got an idea why it works for such a long period without any problems and suddenly it doesn`t?

I don't even reboot my unRAID server or did anything special since the last times. 

 

BTW: Today - after my server wakes up - my RTX start spinning up the fans.

Or to be precisely - only one of the three. But I don't started my VM. My whole unRAID server was idling. 

All temperatures are very low - no need for any fans. 

 

On 1/13/2020 at 1:54 PM, ghost82 said:

it's very simple and you can do it from unraid itself.

So - if I understand you ( @ghost82 ) right - I can start my VM with my passthroughed GPU and then start to dump my own vbios within the VM? 

I don't have a second PC or anything else to do a dump... so this seems to be my only solution. 

 

 

1 hour ago, Maddeen said:

So - if I understand you ( @ghost82 ) right - I can start my VM with my passthroughed GPU and then start to dump my own vbios within the VM?

No, I followed this video:

 

But without using a second gpu; I simply run unraid and dumped the vbios in unraid as described in the video.

Thanks @ghost82

I've tried it - but I get an error when dumping the ROM. I only get a zero-sized file  

 

root@v1ew-s0urce:/sys/bus/pci/devices/0000:01:00.0# echo 1 > rom
root@v1ew-s0urce:/sys/bus/pci/devices/0000:01:00.0# cat rom > /mnt/user/unraid_backup/rtx2060s.dump
cat: rom: Input/output error

When running dmesg I get the following

pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff

 

Any ideas?

 

/////////// Solution - only useable if you can start a VM with the affected GPU \\\\\\\\\\

• Boot unRAID from other GPU/iGPU
• Run a VM using the card where you need the rom and shut the vm down.
• Unbind the GPU from VFIO with the command >> echo "0000:01x00.0" > /sys/bus/pci/drivers/vfio-pci/unbind
• Follow the guide as SpaceInvaderOne described it. 

 

Edited July 27, 20206 yr by Maddeen
Solution found

10 hours ago, Maddeen said:

I've tried it - but I get an error when dumping the ROM

Thank you for having reported that.

Now I'm not 100% sure if I didn't use a second gpu to dump the rom, or a second pc connected through ssh (it was more than 1 year ago)..it seems so by reading my own posts ...Anyway I'm happy you solved the issue!

Edited July 28, 20206 yr by ghost82