Uncle08 Posted February 6, 2022 Share Posted February 6, 2022 Hi too all, I do have one question and need please help/advise: I have a 40Gbit Network Card with a 40/4*10 Splitter Cable - working well in my Windows 10 Computer. Now I like to buy a second one. So I tried to bring the Card in my unraid server to life. In Tools/System devices the Card is identified as: 03:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] lspci shows: root@Tower:~# lspci -vs 03:00.0 03:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter Flags: fast devsel, IRQ 27, NUMA node 0, IOMMU group 50 Memory at f0300000 (64-bit, non-prefetchable) [size=1M] Memory at <ignored> (64-bit, prefetchable) Expansion ROM at f0400000 [disabled] [size=1M] Capabilities: [40] Power Management version 3 Capabilities: [48] Vital Product Data Capabilities: [9c] MSI-X: Enable- Count=128 Masked- Capabilities: [60] Express Endpoint, MSI 00 Capabilities: [c0] Vendor Specific Information: Len=18 <?> Capabilities: [100] Alternative Routing-ID Interpretation (ARI) Capabilities: [148] Device Serial Number 9c-dc-71-ff-ff-50-fa-e0 Capabilities: [108] Single Root I/O Virtualization (SR-IOV) Capabilities: [154] Advanced Error Reporting Capabilities: [18c] Secondary PCI Express Kernel modules: mlx4_core The Mellanox Firmware Tools shows any info only, if I remove and reinsert the GBIC for the card to come online during Unraid is runnning. Mellanox Network Card: Temperature:53 °C Info:FW Version: 2.42.5700 FW Release Date: 8.4.2020 Product Version: 02.42.57.00 Rom Info: version_id=8025 type=CLP type=UEFI version=14.11.49 cpu=AMD64 type=PXE version=3.4.754 Device ID: 4103 Description: Node Port1 Port2 Sys image GUIDs: 9cdc71ffff50fae0 9cdc71ffff50fae1 9cdc71ffff50fae2 9cdc71ffff50fae3 MACs: 9cdc7150fae1 9cdc7150fae2 VSD: PSID: HP_1380110017 But the card is not working and shown on Dashboard/Interface.. Any ideas to bring it up ? Thanks for any help. best regards Quote Link to comment
SimonF Posted February 6, 2022 Share Posted February 6, 2022 1 hour ago, Uncle08 said: Hi too all, I do have one question and need please help/advise: I have a 40Gbit Network Card with a 40/4*10 Splitter Cable - working well in my Windows 10 Computer. Now I like to buy a second one. So I tried to bring the Card in my unraid server to life. In Tools/System devices the Card is identified as: 03:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] lspci shows: root@Tower:~# lspci -vs 03:00.0 03:00.0 Network controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter Flags: fast devsel, IRQ 27, NUMA node 0, IOMMU group 50 Memory at f0300000 (64-bit, non-prefetchable) [size=1M] Memory at <ignored> (64-bit, prefetchable) Expansion ROM at f0400000 [disabled] [size=1M] Capabilities: [40] Power Management version 3 Capabilities: [48] Vital Product Data Capabilities: [9c] MSI-X: Enable- Count=128 Masked- Capabilities: [60] Express Endpoint, MSI 00 Capabilities: [c0] Vendor Specific Information: Len=18 <?> Capabilities: [100] Alternative Routing-ID Interpretation (ARI) Capabilities: [148] Device Serial Number 9c-dc-71-ff-ff-50-fa-e0 Capabilities: [108] Single Root I/O Virtualization (SR-IOV) Capabilities: [154] Advanced Error Reporting Capabilities: [18c] Secondary PCI Express Kernel modules: mlx4_core The Mellanox Firmware Tools shows any info only, if I remove and reinsert the GBIC for the card to come online during Unraid is runnning. Mellanox Network Card: Temperature:53 °C Info:FW Version: 2.42.5700 FW Release Date: 8.4.2020 Product Version: 02.42.57.00 Rom Info: version_id=8025 type=CLP type=UEFI version=14.11.49 cpu=AMD64 type=PXE version=3.4.754 Device ID: 4103 Description: Node Port1 Port2 Sys image GUIDs: 9cdc71ffff50fae0 9cdc71ffff50fae1 9cdc71ffff50fae2 9cdc71ffff50fae3 MACs: 9cdc7150fae1 9cdc7150fae2 VSD: PSID: HP_1380110017 But the card is not working and shown on Dashboard/Interface.. Any ideas to bring it up ? Thanks for any help. best regards I have an IBM Card root@unraid:~# lspci -vs 06:00.0 06:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3] Subsystem: Mellanox Technologies MT27500 Family [ConnectX-3] Flags: bus master, fast devsel, latency 0, IRQ 28, NUMA node 0, IOMMU group 27 Memory at cf400000 (64-bit, non-prefetchable) [size=1M] Memory at cd000000 (64-bit, prefetchable) [size=8M] Expansion ROM at cf300000 [disabled] [size=1M] Capabilities: [40] Power Management version 3 Capabilities: [48] Vital Product Data Capabilities: [9c] MSI-X: Enable+ Count=128 Masked- Capabilities: [60] Express Endpoint, MSI 00 Capabilities: [c0] Vendor Specific Information: Len=18 <?> Capabilities: [100] Alternative Routing-ID Interpretation (ARI) Capabilities: [148] Device Serial Number f4-52-14-03-00-8b-46-20 Capabilities: [154] Advanced Error Reporting Capabilities: [18c] Secondary PCI Express Kernel driver in use: mlx4_core Kernel modules: mlx4_core FW Version: 2.36.5000 FW Release Date: 26.1.2016 Product Version: 02.36.50.00 Rom Info: type=PXE version=3.4.719 type=UEFI version=14.9.46 Device ID: 4099 Description: Node Port1 Port2 Sys image GUIDs: f4521403008b4620 f4521403008b4621 f4521403008b4622 f4521403008b4623 MACs: f452148b4621 f452148b4622 VSD: PSID: IBM1090111019 I have not been able to get it to work with 40 - 10Gb fan out. But works fine with 40Gb DAC and 40 to 10Gb converter and 10Gb DAC. I have two cards, would need to check if I flashed both to Mellanox Flash, but had to set ports to ethernet. May need to enable with SR-IOV but I dont think my cards supports it. Have a look at this thred from BVD Quote Link to comment
Uncle08 Posted February 6, 2022 Author Share Posted February 6, 2022 (edited) Thanks for this Info. I bought this: QSFP-4SFP10G-CU1M, 40Gb/s QSFP+ zu 4X SFP+ Direct Attach Breakout-Kabel And in Windows, I got 1 of the 4 10Mbit lines to work with the switch, checked with both ports. As I do only have a 10 Mbit switch, I like to use the ability the 1x40Mbit split to 4 x10Mbit lines and from there to 2 other switches. (1 poe switch for the ip cameras and one for the rest) However, both Ports of the card are working in Ethernet mode, so I am also fine with 2 x 10 Mbit. This worked also in windows 10. But I need unraid to make use of the card. The funny part is, you can buy these network cards for about 20 Euros. So more ideas please 🙂 Edit: I just learned that Mellanox ConnectX-3 Pro will not work with breakout cable. So I have to use the 2 Ports of the card. Won`t be 20 Mbits max speed, but at least 2x10 Mbit shared Edited February 7, 2022 by Uncle08 Quote Link to comment
Uncle08 Posted February 7, 2022 Author Share Posted February 7, 2022 Hi if there is no solution available, my question is: I like a dual Port card. Are there recommendations ? Dual 10Gbit SFP+ ? Dual 40Gbit QSFP ? to work with unraid ? will this one work out of the box ? Mellanox ConnectX-3 CX354A Thanks Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.