10Gb network card

[alturismo]

Hi, may somehere tested a 10Gb NIC and could give some advices what is a working solution.

 

im gonna replace my router to the Netgear x10 and as it has a 10Gb port im considering using it and use the LAG to the switch.

 

For any tipps thanks ahead

[JorgeB]

ConnectX-2 works great, most Intel should also work.

[uldise]

10 minutes ago, johnnie.black said:

ConnectX-2 works great, most Intel should also work.

@johnnie.black and how about cables to get these to work on about 10m-15m distance?

[JorgeB]

2 minutes ago, uldise said:

@johnnie.black and how about cables to get these to work on about 10m-15m distance?

 

I use 3 meter passive cables, max for passive is 5 or 7m, I believe active cables can go up to 15m.

[alturismo]

and may wich cables are u using ?

 

wich also fits into the netgear x10 ?

[JorgeB]

1 hour ago, alturismo said:

and may wich cables are u using ?

 

SFP+ Direct Attach

 

1 hour ago, alturismo said:

wich also fits into the netgear x10 ?

 

Don't know that router, but it's usually SFP+ or 10GBASE-T.

[alturismo]

ok, may another question about those cables.

 

if i take the connectx-2 card and check for cables, i always see they are marked for brands

 

example for mellanox cable i thought it would fit.

 

https://www.amazon.de/10Gtek-Ubiquiti-Direct-Attach-Kupferkabel/dp/B01DCZCYZI/ref=sr_1_5?ie=UTF8&qid=1495737008&sr=8-5&keywords=sfp%2B%2Bkabel&th=1

 

are the cables also brand specific ? so they only would fit mellanox to mellanox and so on .... ?

 

cause netgear doesnt claim what brand the 10G SFP+ connecor is ...

 

thanks ahead for an advice

Edited May 25, 2017 by alturismo

[JorgeB]

44 minutes ago, alturismo said:

are the cables also brand specific ?

 

No.

 

Since your linking amazon.de I assume you're in Europe, you may want to look at this ebay seller, I've bough from him all my 10GbE NICs and cables, he usually has good deals, eg:

 

http://www.ebay.co.uk/itm/SET-2x-long-Mellanox-MNPA19-XTR-10GbE-3m-Kabel-10Gbps-Ethernet-Bundle-10G-10Gb-/152140377690?hash=item236c45ee5a:g:PLUAAOSwJQdW~V5c

[alturismo]

Great, and thanks alot for the link

Yes, from Germany

Gesendet von meinem SM-G930F mit Tapatalk

[demonmaestro]

I use the ipolex for Intel X520-DA2 10G nics. Haven't had an issue with them yet.

http://amzn.to/2r7mqEb

[alturismo]

Hi,

my new hardware arrived and with the connectx-2 i have a issue while booting, may someone here has an idea

 

card probably wants to network boot, and i cant find a way out ... as soon i plug in the card boot stops here (see attachement)

 

also cant access my bios anymore, says setup loading but stalls on this screen directly after ...

 

tested on 2 different pcie ports ... sme behavior, may some bios setting i have to do before mounting the card,

boot option is only set to USB, all others are disabled ... may it activates itself while new NIC is discovered,

bus as i cannot enter the bios anymore nor the boot menu im stuck ...

 

tried all kind of keys including ctlr + .... nothing happens, only power cycle ... 

 

For any tips thanks ahead

 

Edited May 31, 2017 by alturismo

[JorgeB]

That should appear for 1 or 2 seconds and go away, I believe there's an option to disable it by flashing it, but never tried it since it doesn't bother me.

[alturismo]

Thanks, i contacted the dealer now and he sends me a replacement to check.

Gesendet von meinem SM-G930F mit Tapatalk

[alturismo]

as the new hardware is on its way with 2 choices i hope one will work

 

another mellanox and a brocade 1010, so ...  one should work, @johnie.black, the suggested dealer is really fast and gentle, thanks again.

 

may i ask when there are now 2 NIC´s, better disable the onboard one so setup should be straight forward or is it no big deal to have 2 NIC´s

and assign all traffic to the major 10G one, just keep the regular one as fallback option if there are issues.

also so they dont interupt with the dockers, plugins, ...

 

thanks ahead

[uldise]

1 hour ago, alturismo said:

another mellanox and a brocade 1010,

as for brocade 1010 - any connectivity issues with them? i read somewhere, they support only ACTIVE DACs.

i have 1010 and 1020, and i can interconnect them with Cisco 7m Active DAC.. no other cables for me so far, so i can't test..

[JorgeB]

I don't have a 10GbE switch, I'm using a direct connection from the desktop to the unRAID servers, so all still have Gbit for internet and all other traffic, I edited the Windows hosts file so that 10GbE is always used for the servers that have it.

[uldise]

1 minute ago, johnnie.black said:

I don't have a 10GbE switch, I'm using a direct connection from the desktop to the unRAID servers, so all still have Gbit for internet and all other traffic, I edited the Windows hosts file so that 10GbE is always used for the servers that have it.

me too, i don't have switch too, but trying to interconnect server with my workstation. i'm on esxi and i'm wondering how to configure esxi side to connect to any VMs with 10GBE connection...

[curiouskid]

29 minutes ago, uldise said:

me too, i don't have switch too, but trying to interconnect server with my workstation. i'm on esxi and i'm wondering how to configure esxi side to connect to any VMs with 10GBE connection...

 

Assuming your 10GBE is VmWare compatible, it will be detected as vmnic2 when you boot your ESX host. If not, you would have to hack your way to get that NIC to show up in ESX.

[uldise]

1 minute ago, curiouskid said:

Assuming your 10GBE is VmWare compatible, it will be detected as vmnic2 when you boot your ESX host. If not, you would have to hack your way to get that NIC to show up in ESX.

10GBE nic on esxi side is detected ok after i installed additional drivers..

i'm wondering about needed vswitch configuration to get it to work - all VMs must access internet over 1GBe connection, but use 10Gbe on LAN on connection with my workstation..

[alturismo]

6 hours ago, uldise said:

as for brocade 1010 - any connectivity issues with them? i read somewhere, they support only ACTIVE DACs.

i have 1010 and 1020, and i can interconnect them with Cisco 7m Active DAC.. no other cables for me so far, so i can't test..

 

the cards arrived today, i ll try them tomorrow (currenty @work) and let you know wich cable works (active, passiv),

i have both cables here ...

[alturismo]

ok, replace card mellanox also doest work, same behavior, so most common not compatible with my mainboard.

 

the brocade 1010, at least my system is booting then  and its listed in the system devices, but its not listed in network settings,

so actually also not usable to me ...

 

@uldise, cannot tell if the passive cable work as i cant get it to run properly in unraid, router doesnt show any connectivity, i just see it in system devices indide unraid ...

 

[1657:0014] 02:00.0 Fibre Channel: Brocade Communications Systems, Inc. 1010/1020/1007/1741 10Gbps CNA (rev 01)
[1657:0014] 02:00.1 Ethernet controller: Brocade Communications Systems, Inc. 1010/1020/1007/1741 10Gbps CNA (rev 01)

so hardware found, but not recognized as network interface ... i only have 1 interface eth0 in settings

[uldise]

2 minutes ago, alturismo said:

ok, replace card mellanox also doest work, same behavior, so most common not compatible with my mainboard.

 

the brocade 1010, at least my system is booting then  and its listed in the system devices, but its not listed in network settings,

so actually also not usable to me ...

 

@uldise, cannot tell if the passive cable work as i cant get it to run properly in unraid, router doesnt show any connectivity, i just see it in system devices indide unraid ...

 

[1657:0014] 02:00.0 Fibre Channel: Brocade Communications Systems, Inc. 1010/1020/1007/1741 10Gbps CNA (rev 01)
[1657:0014] 02:00.1 Ethernet controller: Brocade Communications Systems, Inc. 1010/1020/1007/1741 10Gbps CNA (rev 01)

so hardware found, but not recognized as network interface ... i only have 1 interface eth0 in settings

i think, you are missing Brocade drivers.. not sure, how to include them in unRAID, you can ask @limetech to include them..

can you boot some Linux LiveCD? on my Ubuntu 16.04 Brocade 10.10 worked out of the box..

[alturismo]

thanks for the advice, i actually dont think limetech will add me a driver in their software ... , may if there would be xx queries about it.

 

so i ll try a intel now here and hope ...

 

thanks for the tipp, i thought as soon they are listed in hardware devices drivers are loaded, but seems you right, in ubuntu the card ist there.

[JorgeB]

37 minutes ago, alturismo said:

i actually dont think limetech will add me a driver in their software

 

They usually add any NIC driver request for the next release.

[alturismo]

i sended all back now, dont want to let the dealer wait for ages if maybe ...

 

im looking for an x520-Da now, i hope its compatible with my hardware, so  unraid should support it when i understood it correctly,

just my mainboard ... i ll see.

 

thanks anyway