[GUIDE] How to Setup NIC Bonding/LACP Cisco Switch


Tolete

Recommended Posts

if you have multiple ports on your server and have a switch that supports NIC teaming/LACP you can put them to work.

The environment/equipment will vary. I use Cisco, but if you have a switch the supports NIC teaming/LACP the concept should be the similar. 

 

-environment diagram - -

818440613_unraidlacp2.png.3f67f8bd9ea9a7f1aa00c3e790835ec5.png

 

router = pfsense

switch = Cisco

server = Unraid

 

 

1. switch configuration - -

-configure interfaces you want to use on your switch with LACP.

MAINSWITCH#en
MAINSWITCH#enable
MAINSWITCH#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
MAINSWITCH(config)#
MAINSWITCH(config)#
MAINSWITCH(config)#inter
MAINSWITCH(config)#interface ra
MAINSWITCH(config)#interface range gig
MAINSWITCH(config)#interface range gigabitEthernet 0/37-38
MAINSWITCH(config-if-range)#cha
MAINSWITCH(config-if-range)#channel-g
MAINSWITCH(config-if-range)#channel-group 2 mode
MAINSWITCH(config-if-range)#channel-group 2 mode acc
MAINSWITCH(config-if-range)#channel-group 2 mode ac
MAINSWITCH(config-if-range)#channel-group 2 mode active
Creating a port-channel interface Port-channel 2

MAINSWITCH(config-if-range)#shu
MAINSWITCH(config-if-range)#shutdown
MAINSWITCH(config-if-range)#

-This also creates the logical interface on the switch as a 'port-channel #' and shuts it down.

 

2. Next, configure the newly created logical interface 'port-channel #' to mode access

-Also what VLAN the interface has access to (optional). 

 

MAINSWITCH#
MAINSWITCH#en
MAINSWITCH#enable
MAINSWITCH#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
MAINSWITCH(config)#inter
MAINSWITCH(config)#interface port
MAINSWITCH(config)#interface port-c
MAINSWITCH(config)#interface port-channel 2
MAINSWITCH(config-if)#swi
MAINSWITCH(config-if)#switchport mode acc
MAINSWITCH(config-if)#switchport mode access
MAINSWITCH(config-if)#swi
MAINSWITCH(config-if)#switchport acc
MAINSWITCH(config-if)#switchport access vl
MAINSWITCH(config-if)#switchport access vlan 40
MAINSWITCH(config-if)#

 

3. Next, bring all interfaces back online.

 

MAINSWITCH#en
MAINSWITCH#enable
MAINSWITCH#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
MAINSWITCH(config)#inter
MAINSWITCH(config)#interface ra
MAINSWITCH(config)#interface range gig
MAINSWITCH(config)#interface range gigabitEthernet 0/37-38
MAINSWITCH(config-if-range)#no shut
MAINSWITCH(config-if-range)#no shutdown
MAINSWITCH(config-if-range)#

 

4. check running configuration

MAINSWITCH#show  running-config
!
interface Port-channel2
 switchport access vlan 40
 switchport mode access
!
interface GigabitEthernet0/37
 switchport access vlan 40
 switchport mode access
 channel-group 2 mode active
!
interface GigabitEthernet0/38
 switchport access vlan 40
 switchport mode access
 channel-group 2 mode active
!

 

once your switch is configured, over on your server.

Stop the Docker service. Head over to Settings > Network Settings.

Configure the interfaces on your server you would like to bond together.

-if you are not able to select the interface under 'bonding members' ensure they are disabled.

mode4.thumb.png.92f2b7f3a4db9b0b16ee6cb469581c17.png

-mode, should be 4

Apply/Done.

 

over on terminal you can  and get some information on link status, protocol and bandwidth. 

cat /proc/net/bonding/bond0

 

/sbin/ip link

 

ethtool bond0

 

bond0.png.9fb9f9dd9eb708226a0a8259aadca23b.png

 

🖖

 

Edited by Tolete
  • Thanks 1
Link to comment
  • Tolete changed the title to HOW TO SETUP NIC BONDING Cisco Switch
  • 8 months later...
On 12/26/2022 at 2:36 PM, Tolete said:

if you have multiple ports on your server and have a switch that supports NIC teaming/LACP you can put them to work.

The environment/equipment will vary. I use Cisco, but if you have a switch the supports NIC teaming/LACP the concept should be the similar. 

 

-environment diagram - -

818440613_unraidlacp2.png.3f67f8bd9ea9a7f1aa00c3e790835ec5.png

 

router = pfsense

switch = Cisco

server = Unraid

 

 

1. switch configuration - -

-configure interfaces you want to use on your switch with LACP.

MAINSWITCH#en
MAINSWITCH#enable
MAINSWITCH#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
MAINSWITCH(config)#
MAINSWITCH(config)#
MAINSWITCH(config)#inter
MAINSWITCH(config)#interface ra
MAINSWITCH(config)#interface range gig
MAINSWITCH(config)#interface range gigabitEthernet 0/37-38
MAINSWITCH(config-if-range)#cha
MAINSWITCH(config-if-range)#channel-g
MAINSWITCH(config-if-range)#channel-group 2 mode
MAINSWITCH(config-if-range)#channel-group 2 mode acc
MAINSWITCH(config-if-range)#channel-group 2 mode ac
MAINSWITCH(config-if-range)#channel-group 2 mode active
Creating a port-channel interface Port-channel 2

MAINSWITCH(config-if-range)#shu
MAINSWITCH(config-if-range)#shutdown
MAINSWITCH(config-if-range)#

-This also creates the logical interface on the switch as a 'port-channel #' and shuts it down.

 

2. Next, configure the newly created logical interface 'port-channel #' to mode access

-Also what VLAN the interface has access to (optional). 

 

MAINSWITCH#
MAINSWITCH#en
MAINSWITCH#enable
MAINSWITCH#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
MAINSWITCH(config)#inter
MAINSWITCH(config)#interface port
MAINSWITCH(config)#interface port-c
MAINSWITCH(config)#interface port-channel 2
MAINSWITCH(config-if)#swi
MAINSWITCH(config-if)#switchport mode acc
MAINSWITCH(config-if)#switchport mode access
MAINSWITCH(config-if)#swi
MAINSWITCH(config-if)#switchport acc
MAINSWITCH(config-if)#switchport access vl
MAINSWITCH(config-if)#switchport access vlan 40
MAINSWITCH(config-if)#

 

3. Next, bring all interfaces back online.

 

MAINSWITCH#en
MAINSWITCH#enable
MAINSWITCH#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
MAINSWITCH(config)#inter
MAINSWITCH(config)#interface ra
MAINSWITCH(config)#interface range gig
MAINSWITCH(config)#interface range gigabitEthernet 0/37-38
MAINSWITCH(config-if-range)#no shut
MAINSWITCH(config-if-range)#no shutdown
MAINSWITCH(config-if-range)#

 

4. check running configuration

MAINSWITCH#show  running-config
!
interface Port-channel2
 switchport access vlan 40
 switchport mode access
!
interface GigabitEthernet0/37
 switchport access vlan 40
 switchport mode access
 channel-group 2 mode active
!
interface GigabitEthernet0/38
 switchport access vlan 40
 switchport mode access
 channel-group 2 mode active
!

 

once your switch is configured, over on your server.

Stop the Docker service. Head over to Settings > Network Settings.

Configure the interfaces on your server you would like to bond together.

-if you are not able to select the interface under 'bonding members' ensure they are disabled.

mode4.thumb.png.92f2b7f3a4db9b0b16ee6cb469581c17.png

-mode, should be 4

Apply/Done.

 

over on terminal you can  and get some information on link status, protocol and bandwidth. 

cat /proc/net/bonding/bond0

 

/sbin/ip link

 

ethtool bond0

 

bond0.png.9fb9f9dd9eb708226a0a8259aadca23b.png

 

🖖

 

This was an incredible help.  I couldn't figure out why I was having intermittent issues with Unraid seeming to be unavailable for periods of time (no pattern)...  As it turns out, I had my switch port-channel interfaces set to mode auto (should have been active).   

  • Upvote 1
Link to comment
On 9/29/2023 at 9:27 AM, droidlev said:

This was an incredible help.  I couldn't figure out why I was having intermittent issues with Unraid seeming to be unavailable for periods of time (no pattern)...  As it turns out, I had my switch port-channel interfaces set to mode auto (should have been active).   

 Glad it was helpful.

Link to comment
  • 1 month later...
  • Tolete changed the title to [GUIDE] How to Setup NIC Bonding/LACP Cisco Switch
  • 1 month later...

Thank you for this awesome guide! I ended up here even though I have an older HP 1810 Procurve switch.
I first tried without any switch configuration and Mode 0 (balance-rr) in UNRAID but my write-speed in Windows would drop from 112MB/s to about 33MB/s.
I have 2 nics in my box so wanted to give UNRAID some more speed lanes in case multiple clients write at the same time to my cache-ssd

There is probably no need for a complete guide but for unraid and an older HP switch with LACP you need to configure it like so:
image.thumb.png.d6e9d595748b59e718ec1b4deddd6a5e.png

The trick is to modify and disable the Static Capability setting, then go to Trunk Membership and you should be able to select LACP for the ports you want:
image.thumb.png.8001e195445141d12dc09355940a7f77.png
The rest of the guide for the UNRAID side is the same as written in the startpost.
After my LACP came up, I was unable to connect to the webgui. (ssh worked fine)
2 tips that helped:
1) I set BIND_MGT="yes" in  /boot/config/ident.cfg
2) I disabled USE_SSL="no"  in /boot/config/ident.cfg

/etc/rc.d/rc.nginx restart

After that I was able to connect to the webgui using the UNRAID ip.
Then I changed Local TLD from local to my domain (local.domain.tld) after that the webgui was accessible again. (no idea why this happend)

 

Edited by ___niko___
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.