Setting up Mellanox network card

[eeans]

I'm a bit stuck with how to set this up in unRAID.

 

I have two Mellanox ConnectX 2 VPI 10GB network cards, one in my unRAID server and one in my VM host. What I'd like to do is have all traffic between the VM host and unRAID go over the 10GB link between them, this will be for backups from the host to the unRAID server.

 

I can see that unRAID can see the card, under System Devices:

 

01:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

 

But I'm not sure where to start in assigning it an eth and an IP address?

[MSattler]

Did you get any feedback on this elsewhere?

 

Thanks!

[John_M]

Have you read this thread? It might contain some information you can use.

 

[1812]

I essentially do this between unRaid 2 servers.

 

Go to setting tab> network setting.

Make sure the mellanox is not a bonding member.

Scroll down to the interface that is the mellanox card. For me it is eth4 (eth0-3 being my 4 onboard gigabit ports)

Set ip address assignment to static

Set ip to something that isn't the rest of your network (I used 10.0.0.1 on one box, then 10.0.0.2 on the other)

I set MTU to 9000

Click apply.

 

 

Assuming you setup your networking properly on the vm/client side, then just connect to the server ip address via the one you defined.

 

having 10gbe is a bit overkill for the unRaid array due to disk read/write limitations. But when moving things between cache drives, it does provide a nice bump.

 

[MSattler]

I essentially do this between unRaid 2 servers.

 

Go to setting tab> network setting.

Make sure the mellanox is not a bonding member.

Scroll down to the interface that is the mellanox card. For me it is eth4 (eth0-3 being my 4 onboard gigabit ports)

Set ip address assignment to static

Set ip to something that isn't the rest of your network (I used 10.0.0.1 on one box, then 10.0.0.2 on the other)

I set MTU to 9000

Click apply.

 

 

Assuming you setup your networking properly on the vm/client side, then just connect to the server ip address via the one you defined.

 

having 10gbe is a bit overkill for the unRaid array due to disk read/write limitations. But when moving things between cache drives, it does provide a nice bump.

 

I like that.  That way when copying things between unRaid servers I'm not crushing the main interfaces. Granted, drive access may be slowed if they grab something from that drive.

 

I was thinking about grabbing a Quanta LB6M, and trunking the ethernet ports to my main ethernet switches, and switching all my servers over to IPoIB.

[1812]

I essentially do this between unRaid 2 servers.

 

Go to setting tab> network setting.

Make sure the mellanox is not a bonding member.

Scroll down to the interface that is the mellanox card. For me it is eth4 (eth0-3 being my 4 onboard gigabit ports)

Set ip address assignment to static

Set ip to something that isn't the rest of your network (I used 10.0.0.1 on one box, then 10.0.0.2 on the other)

I set MTU to 9000

Click apply.

 

 

Assuming you setup your networking properly on the vm/client side, then just connect to the server ip address via the one you defined.

 

having 10gbe is a bit overkill for the unRaid array due to disk read/write limitations. But when moving things between cache drives, it does provide a nice bump.

 

I like that.  That way when copying things between unRaid servers I'm not crushing the main interfaces. Granted, drive access may be slowed if they grab something from that drive.

 

I was thinking about grabbing a Quanta LB6M, and trunking the ethernet ports to my main ethernet switches, and switching all my servers over to IPoIB.

 

Something like the LB6M is on my plan for future upgrades, but i'm pretty well set for the moment.... we'll just see how long the moment last..