Driver Support for Mellanox VPI QSFP+ NIC Port Type Management

[spoon532]

I have the Mellanox ConnectX-2 VPI QSFP+ MHRH2A (dual port) NIC that is detected in lspci:

Quote

InfiniBand: Mellanox Technologies MT25408A0-FCC-GI ConnectX, Dual Port 20Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s In... (rev b0)

 

I would like to use the card in UnRaid as a backbone between multiple UnRaid servers, but after reading the forums and speaking with Jon in Support, there is no Infiniband support for UnRaid. However, I only use ETH networks, so IB is not required. Flashing the firmware to ETH-Only has been unsuccessful.

 

Since this inexpensive card has the ability to switch the port type from Auto, IB, or ETH I should be able to select the port type to force ETH-only using the command below:

Quote

mstconfig –d <device pci> s LINK_TYPE_P1/2=<ETH|IB|VPI>

 

The driver for this, however, is not currently enabled. I would like VPI cards like mine to have the drivers enabled (ie, mlx4 mlx5 etc), even if it only allows for ETH networking to be used. The driver could load, detect the card, and then set the port type automagically to the supported port type. This would allow 20gb+ networking, as well as let the VPI card be seen on the Settings -> Network Settings screen in UnRaid to allow for DHCP, static, RoCE, RDMA, bonding, bridging, just like currently supported 1/10gb cards. I do understand that throughput might be limited to 10gb, but I believe this is a step in the right direction for VPI card support in the UnRaid Community.

I have several of these cards that I can ship anywhere for testing.

[ich777]

 

3 hours ago, spoon532 said:

I have the Mellanox ConnectX-2 VPI QSFP+ MHRH2A

On which Unraid version are you?

If you are on version 6.9.0RC2 this would be possible I think through a plugin.

I have to look into this...

 

Also I have a plugim for Mellanox in the CA APP where mstflint or mstcomfig is integrated and natively available on Unraid if the plugin is installed.

[spoon532]

Hey Ich! Dig your stuff!

I have 6.8.3 on both boxes. I'll search the CA apps for Mellanox.

[ich777]

1 hour ago, spoon532 said:

Hey Ich! Dig your stuff!

I have 6.8.3 on both boxes. I'll search the CA apps for Mellanox.

You need at least to be on Unraid 6.9.0beta35 for the Plugins to show up in the CA App.

[spoon532]

@ich777, bad news:

Quote

# mstconfig query -y
-E- Failed to open device: /sys/bus/pci/devices/0000:01:00.0/config. Device not supported.

Quote

Supported devices:
        4th Generation devices: ConnectX3, ConnectX3-Pro (FW 2.31.5000 and above).
        5th Generation devices: ConnectIB, ConnectX4, ConnectX4-LX, ConnectX5.

 

Since this device is a ConnectX2, it might be too old for this version. Do you happen to know what plugin might be required for the older cards?

[ich777]

2 minutes ago, spoon532 said:

Since this device is a ConnectX2, it might be too old for this version. Do you happen to know what plugin might be required for the older cards?

There is no other Plugin available... I compile the open source version of the Mellanox Firmware Tools and pack it up to use it in my Plugin.

 

From what I've remember I had no problem Flashing the firmware on my ConnectX2.

Now bought a ConnectX3 because the are really cheap now and also much more power efficient.

[threiner]

It seems the card is still not naitve supported??

[Libor]

To use RDMA functions you need at least Connectx-3 and Connectx-3 PRO for RDMAv2 which is the routable version.

Right now I think Im goint to start new thread as Im after NVMe over Fabric support especialy NVMEoRoCE or NVMeoTCP.