February 24, 20179 yr Does anyone have experience using this with unraid? I've got it showing up as a pci device, however it is not showing up as an ethernet device so I don't see an interface. It looks like it can operate in either infiniband or ethernet modes, but swapping between the two is a bit beyond me in unraid at this point. Any tips? 06:00.0 InfiniBand [0c06]: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] [15b3:673c] (rev b0) Subsystem: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] [15b3:0022] Kernel driver in use: mlx4_core Kernel modules: mlx4_core Edited January 12, 20188 yr by Leondre
February 24, 20179 yr The ethernet models work great, link for the mode change if for the connectx-3, are you sure it also works for the connectx-2? From the manual: "The ConnectX-2 adapter card is compliant with the IEEE Std 802.3. The VPI adapter cards listed above, based on this device, provide access to the Ethernet ports by means of QSFP connectors.The Mellanox QSA (QSFP to SFP+) adapter modules can be ordered through Mellanox Technologies authorized dealers." Do you have the adapter? If not probably cheaper to get the the ethernet connectx-2 on ebay, they are available for under 25$.
February 24, 20179 yr Author As far as I can tell from the user manual pertaining to this particular card mode change should be possible. Though even with these instructions I'm pretty lost.
February 24, 20179 yr And you already have the adapters? Edited February 24, 20179 yr by johnnie.black
February 24, 20179 yr Author I do not. The cards function fine when tested between two windows machines just as they are using a qsfp cable, I figured unraid would be able to do the same, is that not the case? Edited February 24, 20179 yr by Leondre
February 24, 20179 yr If they work in windows like that they should also work in Linux if you can get the driver to detect it as ethernet.
October 13, 20178 yr Hi Leondre, Were you able to make this card work? I have a similar issue and your input will be helpful.
April 25, 20188 yr Sorry to resurrect an old thread, did you have any luck getting a ConnectX-2 VPI card working with unRAID @wheelbase or @Leondre ? From my limited linux experience it appears that the mlx4_core module is loaded and used b mlx4_en, however attempting to load mlx4_ib fails iwth error: Quote FATAL: Module mlx4_ib not found in directory /lib/modules/4.14.35-unRAID Searching suggests that I need to compile a new MLNX_OFED kernel module that includes mlx4_ib, which looks like a steep learning curve. I'll try changing the port mode in the first instance as the Mellanox documented instructions are pretty straight forward... Quote Port Type Management/VPI Cards Configuration: Device ports can be individually configured to work as InfiniBand or Ethernet ports. By default, device ports are initialized as InfiniBand ports. If you wish to change the port type, use the mstflint tool after the driver is loaded.5. Install mstflint tools: Zypper install mstflint. 6. Check the PCI address. # lspci | grep Mellanox 00:06.0 Infiniband controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] 7. Use mstconfig to change the link type as desired IB – for InfiniBand, ETH – for Ethernet. # mstconfig –d s LINK_TYPE_P1/2= Example: # mstconfig -d 82:00.1 s LINK_TYPE_P1=ETH 8. Reboot your machine. Edited April 25, 20188 yr by fredhoon
April 28, 20188 yr After doing a bit more digging, the current version of the mlx_core module has dropped support for ConnectX-2 cards, I'm not sure what version unRAID runs however I would assume it to be fairly recent due to the kernel version. It would be a very steep learning curve for me to be able to compile and install a previous version of the OFED driver and possibly even require an older kernel / version of unRAID. I tried a ConnectX-3 card which was detected by unRAID but only provided a single network interface despite being a dual port VPI card. Both ports were in VPI / auto detect mode and despite being directly connected via a QSFP DAC cable to an identical CX-3 card in another machine, the card didn't show a link light. Compiling / resolving dependencies / installing the mft utilities in unRAID (or even mint) was beyond me so I threw the card in a windows machine and changed Port 1 from "Auto / VPI" to "Eth" & did the same for the card at the other end - now I have a 40Gb/s ethernet link from UnRAID to my Win7 Games / backup box! My cache SSDs should be good for 2-300MB/s and Turbo writes for 150MB/s so I can use at least some of the CX-3 bandwidh. Edited April 28, 20188 yr by fredhoon
Archived
This topic is now archived and is closed to further replies.