I need help in getting both of my 2 GPUs recognized by unRaid.


Recommended Posts

I currently have a RX580 and P2000 installed on my unRaid server. The RX580 shows up and but the P2000 does not until I install the Nvidia drivers. Once the driver plugin is installed, the RX580 disappears and the P2000 shows up. Is there a way to have them both show up as available GPUs?

 

I have IOMMU and HVM enabled in the bios. I have tried all the options in the PCIe ACS override. I have the bios in legacy boot as well.

 

I have tried uninstalling the Nvidia drivers, binding the RX580 in System Devices and then installing the Nvidia driver. That does not work either. Its seems to be one or the other in the in the System Devices, as I have yet to see them both there at the same time. I did notice that they both take the same IOMMU groups.

 

I also did confirm that I can video from both when plugging them into a monitor. I am using my P2000 strictly for Plex and Unmanic. I want to use the RX580 for everything else.

Nvidia IOMMU.png

AMD IOMMU.png

cybertron-diagnostics-20210505-1021-AMD GPU Visible.zip cybertron-diagnostics-20210505-1549 Nvidia GPU Visible.zip

Link to comment

Looking at your kernel logs, neither syslog nor lspci output indicate that the Linux kernel can see both devices at the same time. The output of lspci.txt shows only one GPU or the other -- not both. This means the PCI subsystem itself cannot see both devices, which points to hardware. Even using vfio hardware passthrough, lspci should still show the hardware as present. This also means something changed aside from just installing the drivers, between those boots. Drivers cannot mask hardware from the kernel.

 

Your problem seems to be motherboard device support, physical installation, or some related issue -- from what I can see, no operating system would be able to see both devices.

 

First - Your motherboard does not have the bandwidth to run two GPUs at PCIe 3.0 x16 link speed so you may not see ideal performance, and depending on your configuration, it may be running as low as x4 bandwidth. At best, with that board, you will get PCI3 3.0 x8 link speed but only if you connect your GPUs to physical slots marked PCIE_1 and PCIE_4 AND you're not using a "with Vega Radeon Graphics" or "with Radeon Graphics" variant CPU. Any other ports will run PCIe 2.0 x4 bandwidth or lower regardless of their physical size. If you ARE running a "with xx Graphics" variant CPU, you will only get one PCIE slot to work, as in this case the onboard lane swithc will run x8/x0 (or x4/x0) on PCIE_1/PCIE_4 respectively.

 

Second - Check for BIOS updates, as this may improve compatibility slightly. The above CPU/link width restrictions will still apply, however, as this is the result of using a relatively low-end gaming motherboard with insufficient PCIE lane capacity.

 

What use case do you have for two GPUs? If there's significant need to discuss upgrade paths, we could look into it.

  • Like 1
Link to comment

The two screenshots suggest that both GPUs are being given the same PCIe address - 27:00.0 - which is why they can't co-exist. Can you confirm that you didn't swap the cards round between the two screenshots? X470 ATX-sized motherboards generally support bifurcation of the 16 PCIe lanes normally allocated for GPU use but you might have to enable it in the BIOS. (Sorry, I'm not as familiar with MSI motherboards as I am with Gigabyte ones.) That would mean setting it to On, rather than Auto. I would expect, when bifurcation is working properly, for eight of the lanes to be routed to the primary x16 slot and eight to the secondary one, and for each device to receive a unique address and be in a different IOMMU group. That doesn't seem to be the case with this motherboard. Maybe a different BIOS would fix it.

 

Eight PCIe gen 3 lanes are more than sufficient for each of your cards and the use case you give in your OP is a perfectly valid one. I'm leaning towards the problem being a buggy BIOS.

Link to comment

@John_M  I reviewed specification of their motherboard before I made the post. Its contents are factual for this exact context. Depending on which CPU they're using it either gets x8/x8, x8/x0 or x4/x0 depending on what CPU they have installed. If their CPU allows it, simply loading the board should have allowed it to function if it was going to. No configuration should be required within the UEFI setup.

 

 

Also I didn't mention it but on this board PCIE_6 is a PCIE 2.0 x4 interface despite having an x16 physical connector, I wouldn't suggest using it for your GPU regardless of how much space there is.

 

It is quite correct that the use case specified will run pretty much flawlessly at x8, assuming "everything else" on the other card doesn't require x16 bandwidth which is why I was seeking clarification on the matter. I suppose it's moot since the hardware isn't capable of maxing out an x16 interface anyway..

 

If both GPUs worked in Windows with this board, they should also work in Linux (and thus Unraid) on this board -- there should be no need to use a different one, if that's too much trouble. That still leaves some discrepancies -- both logs have the same physical address (as pointed out by John_M) and the kernel itself shows no indication of any errors or anything else loaded into a PCIe lane that is misbehaving. It might be worth ensuring everything is installed correctly, fully seated and powered, and capturing another diagnostic.zip for us to peruse if you're still having issues.

Link to comment
1 hour ago, codefaux said:

Depending on which CPU they're using

 

It's a 3600, according to the diagnostics, so 24 PCIe ver. 4 lanes, though the motherboard is only qualified for ver. 3. 4 lanes to the X470, 4 to the NVMe, 8 to the primary slot, 8 via a switch chip which can feed either the secondary slot or the remainder of the primary slot.

 

@xnerdist Since you want to pass through the RX 580, you could try putting it in the primary GPU slot and if the P2000 won't work in the secondary slot (physically x16, electrically x8), where it should ideally go, you could put it in the bottom long slot, which is physically x16 but wired for four PCIe ver. 2 lanes, but loses some if you populate the lower x1 slots. That will work if the BIOS allows you to select that one as the primary GPU. It isn't ideal but should be quite usable as your main purpose for it is not to generate complex graphics but to transcode video, where a much lower PCIe bandwidth is required. Nevertheless, it ought to work properly as you had it and I'm still of the belief that it's a motherboard/BIOS problem. Consumer and gaming motherboards are only tested with Windows and each one works around the bugs in the other so "it works with Windows" is no guarantee that it will work with any other operating system.

 

Link to comment

@John_M That may be factually accurate, but specifically I was quoting the breakdown the motherboard manual indicates actually reaches the PCIe slots, not hypothetically edu-guessing based on what lanes should/can be available to a chipset vendor. Implementation-derived specs are always better than application-derived specs because while any vendor can use an X470 chipset, some vendors might choose to implement PCIe lane distribution, weighting, switching, and layout in very different manners, due to differing choices in included/supported peripherals, as well as how widely they wish to implement the ability for the user to map PCIe lanes onto physical slots IE how many different "bank switch sets" can be accomplished per board.

 

I'm not saying you're wrong, I'm only saying that you're making judgements based upon what CAN be done, rather than what the actual manufacturer who implemented/built/sells the finished product based around the application says WAS done.

 

This is the Word Of GOD on what this specific model of motherboard can do, barring solely A) improper documentation or B) updates via UEFI image/etc which changes the capabilities of the board;

image.png.313c144020426a71357fd88a44f5572c.png

 

Other boards USING an X470 can and will have drastically different layouts and implementations within the constraints of what the CPU/APU itself provides, but every X470 does not have the same layout and pretending it does only adds confusion.

 

I'm not trying to argue, I'm telling you for a fact, of the PCIe x16 connectors physically available, this board has only two which can achieve at best x8, one which will never exceed x4 and is also disabled when using PCIe based M.2 storage, and that is it. Your clarification on which CPU is in use is appreciated and seems to preclude the presence of a graphics processor, but I don't have generation-to-model-shorthand memorized so I'd actually have to spend more time to look it up and frankly it's not worth it. The information has been covered.

Link to comment
46 minutes ago, codefaux said:

one which will never exceed x4 and is also disabled when using PCIe based M.2 storage

 

That is explicitly referring to the second M.2 socket (hence M2_2). The first M.2 socket (M2_1) has four lanes direct from the AM4 socket.

 

55 minutes ago, codefaux said:

This is the Word Of GOD

 

Oh, please!

 

Link to comment
2 hours ago, John_M said:

That is explicitly referring to the second M.2 socket (hence M2_2). The first M.2 socket (M2_1) has four lanes direct from the AM4 socket.

 

Fair, I wasn't specific enough in that statement. Yes, that socket is only disabled when using PCIe based M.2 storage in M2_2 -- I left that unsaid since, as you also pointed out, it was already on the screen and I felt no need to handhold the information out any further. Thank you for adding to my specificity, which does not address my original point in that yours was still lacking.

 

 

And perhaps it was overstated, but it's pretty clear that my point was that there is no more accurate definition of a vendor's specific implementation than their own documentation, providing they wrote it correctly and keep it up to date with regard to changes. You can throw attention at my tone and word choice all you wish, that's not the action of a person with strong standing in a conversation. Your statements were generalized based on the chipset's overarching capacity. My overplaying a single phrase might make me overdramatic, but it doesn't make you correct. Generalizations about a chipset's capability don't apply to every version of a board using that chipset -- demonstrated even doubly so in that each brand even has a few tiers of each chipset available as a different model of motherboard. Specificity matters, especially in the context of the computing world. I'm not sure how adding less specific information to what has been an essentially motionless thread or arguing about its merits has helped matters.

 

In any case, it's getting a bit off-topic. If you feel the need to dispute the merits of accuracy further, feel free to send me a message. There's clearly nothing more to be gained here otherwise, I'm clocking out of this thread until OP needs feedback I can provide. Ideally there was enough to be gained here to have made it worth everyone's while. Good luck!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.