Error 127 - Try to passthrough PCIe 3.0x8 as x4/x4 Bifurcation with 2x M.2 to 5x Sata JMB585 Controller


Recommended Posts

Hi @all,

 

my mainboard is a Gigabyte z590 Aorus Master which has 3x mechanical x16 PCIe Ports. The first and the second one is connected to the CPU and the third to the chipset. For the first both i can configure the bios to these bifurcation modes: Auto, 8x/8x and 8x/4x/4x.

 

I selected 8x/4x/4x and put only in the second slot these PCIe to 2x M.2 Adapter Card in:

image.png.1784d03968ceb0c56d034b1c352887cc.png

Dpofirs Adapterkarte mit 2 X 32 Gbit/s Erweiterungskarten, Dual M.2 NVMe SSD zu PCIE X8 M Key Festplattenkonverter Leser Erweiterungskarte, Unterstützt Full Speed NVME SSD/M.2 PCIE(ph45)

image.thumb.png.48181eddd272b8a6b6b67266ab833e13.png

 

In these PCIe Adapter I would connect 2 of these JMB585 SATA Controller cards with 5x Sata ports each.

image.png.5355ae0b97a054e55e86ccaf5d075141.png

 

Unraid itself detects all fine. Both JMB585 are shown correctly and the connected HDD's will also shown and accesable: (in this picture the HDD's were not shown because they are bounded to VFIO for passing through to a VM)

image.png.1d2941ff04febdd1154592c2d9844b71.png

 

The problem I have is when I passthrough both devices to one and the same vm I get the error: 

internal error: Unknown PCI header type '127' for device '0000:02:00.0'

 

I also tried to play with these arguments in the syslinux config at the boot:

vfio-pci.ids=197b:0585

or

xen-pciback.hide=(02:00.0)(03:00.0) which I had from this thread because of the same Vendor ID of the both JMB585

 

But still the same Error 127...

 

When I go with the mouse in the system devices over the first JMB585 this pops up (maybe one other an other person know what to get out of this information):

image.png.99ab514e43dee00001bfb77a4826b360.png

 

Inside the VM and its template I passthrough the devices like this:

image.png.d4d64acee6a0e7af2f36766cdae7853b.png

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

I think here is all fine and like the way it should be done...

 

//EDIT: The freakiest part of the things: 

Here it should be added that it did not work all the time, and I just left the server on for 1 hour or so. I tried it again and it starts the VM... So slowly I despair of the sense here. Could it be that after an unraid, i.e. host reboot, it takes a while until it stops addressing the PCIe devices and they virtually "go into standby" so that they lose the "already booted status"? I shut down the VM afterwards and could restart it immediately without any problems... buuuut when I did Unraid itself restarted and I get again error 127. After many try I can not reproduce this described scenario...

 

At this point I have to give up and will be happy if you guys can help me! Thanks a lot for your time and reading my problems!!!

Edited by WowaDriver
Link to comment
7 hours ago, WowaDriver said:

Here it should be added that it did not work all the time, and I just left the server on for 1 hour or so. I tried it again and it starts the VM... So slowly I despair of the sense here. Could it be that after an unraid, i.e. host reboot, it takes a while until it stops addressing the PCIe devices and they virtually "go into standby" so that they lose the "already booted status"?

It sounds like a voltage issue to the pcie slot, so I would say...hardware related.

That card should need 3.3V, I think that sometimes it drops, leaving the device in the unknown header type 7f.

You could try to look into your bios settings, I doubt you will find any voltage values to tweak for pcie, but maybe some settings like 'maximum performance', or something like that.

Make also sure your power supplier has enough watts to run the build.

Link to comment
  • 3 months later...
On 3/19/2022 at 2:40 PM, WowaDriver said:

Maybe you are right, but the only power saving settings I found are these both and they are disabled...

 

All three ASPM Settings are manually off

image.png.50b1915a983e8be87406263028dfabe1.png

 

and the LPM Support too

image.png.06f8c688d9c782ff02985459c970053a.png

 

Do you have an other idea?

 

 

Hi!!!
I was looking for something associated with z590 aorus and I found your link! Well, ahaha you're going to laugh, but with what I see, my identification with the problem is EVIDENCE... I'll explain:
I have a gigabyte Z590 Aourus Ultra (almost the same as yours) and I had the geforce 980ti in the first slot and in the second slot dedicated to the second Graphics (here IS THE PROBLEM) the soundblaster z sound card... well it turned out that when I played the minutes the games closed for me! and it was due to a channel dedication problem, that is, this second slot does weird things with devices that are NOT graphics cards, since it hangs from the north chipset, that is, directly from the CPU and instead the other one that remains is pciex4 that hangs directly from the south bridge, my advice FORGET about this second slot and use the third slot to add your device as I did and no more hassle!!!
I await your news ;)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.