10Gbe Mellanox issues


Recommended Posts

I purchased 2 Mellanox 10GBe cards from ebay:

 

Link: https://www.ebay.com/itm/MNPA19-XTR-10GB-MELLANOX-CONNECTX-2-PCIe-X8-10Gbe-SFP-NETWORK-CARD/131634470127?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m2749.l2649

 

I also purchased this cable from ebay: https://www.ebay.com/itm/SFP-H10GB-CU3M-CISCO-SFP-TWINAX-COPPER-CABLE-3M-PASSIVE-DAC-CABLE/282060432334?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m2749.l2649

 

unRaid detects the cards and loads drivers and I have assigned them IP addresses on a separate subnet. My goal is to use 10GBe connection to backup Primary unRaid server to a backup unRaid server.

 

I see blinking lights when cable is plugged in and cards are online, but i am unable to ping between them. DId I purchase the wrong type of cable?

 

Not sure where to look to troubleshoot but appears that there is no link. I can ping each cards own IP from the server its connected to, but cant ping across to the other.

 

Any assistance is appreciated.

 

 

Link to comment

root@JUPITER:~# ethtool eth2
Settings for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
        Link detected: no

 

 

appears no link.

Link to comment

Yes, I use the same NIC but between Windows and unRAID, and since there's no switch sometimes after booting or rebooting no link is detect, I just disable and re-enable the NIC in Windows and the link is detected again, you might need to do the same on unRAID, it should be enough to go to Settings -> Network Settings and click "Link Down" on the correct interface, then "Link Up"

Link to comment

i ran ifup eth2 and then ethtool eth2 again and now appears link is up, but still unable to ping.

 

 for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
        Link detected: yes

Link to comment
8 minutes ago, johnnie.black said:

Yep, link is not staying up, that would explain it, but no clue why it's happening, I never used them between 2 unRAID server, but I would expect that once up the link would stay up, until possibly a reboot or power cycle.

 

 

I've used these same cards between 2 unraid servers for backup. Once its up, it was rock solid. That is, until one card started to die. Then it would drop connection....

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.