I purchased 2 Mellanox 10GBe cards from ebay:

 

Link: https://www.ebay.com/itm/MNPA19-XTR-10GB-MELLANOX-CONNECTX-2-PCIe-X8-10Gbe-SFP-NETWORK-CARD/131634470127?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m2749.l2649

 

I also purchased this cable from ebay: https://www.ebay.com/itm/SFP-H10GB-CU3M-CISCO-SFP-TWINAX-COPPER-CABLE-3M-PASSIVE-DAC-CABLE/282060432334?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m2749.l2649

 

unRaid detects the cards and loads drivers and I have assigned them IP addresses on a separate subnet. My goal is to use 10GBe connection to backup Primary unRaid server to a backup unRaid server.

 

I see blinking lights when cable is plugged in and cards are online, but i am unable to ping between them. DId I purchase the wrong type of cable?

 

Not sure where to look to troubleshoot but appears that there is no link. I can ping each cards own IP from the server its connected to, but cant ping across to the other.

 

Any assistance is appreciated.

 

 

Looks like to correct cable to me, if you run:

 

ethtool ethX

 

Do you get link detected? (replace X with correct number)

root@JUPITER:~# ethtool eth2
Settings for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
        Link detected: no

 

 

appears no link.

Yes, I use the same NIC but between Windows and unRAID, and since there's no switch sometimes after booting or rebooting no link is detect, I just disable and re-enable the NIC in Windows and the link is detected again, you might need to do the same on unRAID, it should be enough to go to Settings -> Network Settings and click "Link Down" on the correct interface, then "Link Up"

i ran ifup eth2 and then ethtool eth2 again and now appears link is up, but still unable to ping.

 

 for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
        Link detected: yes

With link up it should work, make sure the IP addresses are correctly configured.

Here is config. Seems that when both interfaces are up, i run ping command, it fails and then takes link back down.

 

 

weird, seems every now and then it will ping successfully.. Thoughts?

19 minutes ago, unsainted said:

Thoughts?

Not really, maybe if you can try between unRAID and a Windows desktop to see if it's the same.

thanks for the help as always, I will see if i can move my desktop around to get them connected.

 

one other thing of note. I ran tail -f /var/log/syslog on each system..

Yep, link is not staying up, that would explain it, but no clue why it's happening, I never used them between 2 unRAID server, but I would expect that once up the link would stay up, until possibly a reboot or power cycle.

8 minutes ago, johnnie.black said:

Yep, link is not staying up, that would explain it, but no clue why it's happening, I never used them between 2 unRAID server, but I would expect that once up the link would stay up, until possibly a reboot or power cycle.

 

 

I've used these same cards between 2 unraid servers for backup. Once its up, it was rock solid. That is, until one card started to die. Then it would drop connection....