EVPN On Nokia 7450 Notes Part 2
Following from this blog post.
Scenario 2: Single-active Multi-homing
I updated LAG 60 to be active-standby. This required multi-homing option under ESI to change from all-active to single-active. Also LAG configuration need to be different between PE1 and PE2 as stated in the extract below from Nokia guide “LAYER 2 SERVICES AND EVPN GUIDE: VLL, VPLS, PBB, AND EVPN RELEASE 14.0.R4”
I originally performed the POC without changing LAG configurations between PE1 and PE2. I observe CE1 will forward traffic to non DF as both links was active/active. So non DF will drop traffic and connectivity will be lost.
Configuration
LAG
Under LAG, we add system-priority. PE1 will use 65534 and PE2 will use 65535.
A:PE1>config>lag# info
----------------------------------------------
mode access
encap-type dot1q
port 2/2/13
lacp active administrative-key 60 system-id 00:00:00:00:05:60 system-priority 65534
no shutdown
----------------------------------------------
A:PE2>config>lag# info
----------------------------------------------
mode access
encap-type dot1q
port 2/2/13
lacp active administrative-key 60 system-id 00:00:00:00:05:60 system-priority 65535
no shutdown
----------------------------------------------
ESI
Under ESI, multi-homing is change to single-active from all-active on both PE1 and PE2
A:PE1>config>service>system>bgp-evpn# info
----------------------------------------------
route-distinguisher 192.168.58.210:0
ethernet-segment "ESI-60" create
esi 00:11:22:33:44:55:66:77:88:60
es-activation-timer 0
service-carving
mode auto
exit
multi-homing single-active
lag 60
no shutdown
exit
----------------------------------------------
A:PE2>config>service>system>bgp-evpn# info
----------------------------------------------
route-distinguisher 192.168.58.211:0
ethernet-segment "ESI-60" create
esi 00:11:22:33:44:55:66:77:88:60
es-activation-timer 0
service-carving
mode auto
exit
multi-homing single-active
lag 60
no shutdown
exit
----------------------------------------------
Verification
PE1 and PE2
LAG on PE1 will be operationally up and PE2 will be operationally down.
A:PE1# show lag 60 detail =============================================================================== LAG Details =============================================================================== Description : N/A ------------------------------------------------------------------------------- Details ------------------------------------------------------------------------------- Lag-id : 60 Mode : access Adm : up Opr : up Thres. Exceeded Cnt : 18 Port Threshold : 0 Thres. Last Cleared : 10/08/2017 10:24:18 Threshold Action : down Dynamic Cost : false Encap Type : dot1q Configured Address : a0:f3:e4:5f:73:ac Lag-IfIndex : 1342177340 Hardware Address : a0:f3:e4:5f:73:ac Adapt Qos (access) : distribute Hold-time Down : 0.0 sec Port Type : standard Per-Link-Hash : disabled Include-Egr-Hash-Cfg: disabled Forced : - Per FP Ing Queuing : disabled Per FP Egr Queuing : disabled Per FP SAP Instance : disabled LACP : enabled Mode : active LACP Transmit Intvl : fast LACP xmit stdby : enabled Selection Criteria : highest-count Slave-to-partner : disabled MUX control : coupled Subgrp hold time : 0.0 sec Remaining time : 0.0 sec Subgrp selected : 1 Subgrp candidate : - Subgrp count : 1 System Id : 00:00:00:00:05:60 System Priority : 65534 Admin Key : 60 Oper Key : 60 Prtr System Id : 08:96:ad:4e:5b:e4 Prtr System Priority : 32768 Prtr Oper Key : 60 Standby Signaling : lacp Port weight speed : 0 gbps Number/Weight Up : 1 Weight Threshold : 0 Threshold Action : down ------------------------------------------------------------------------------- Port-id Adm Act/Stdby Opr Primary Sub-group Forced Prio ------------------------------------------------------------------------------- 2/2/13 up active up yes 1 - 32768 ------------------------------------------------------------------------------- Port-id Role Exp Def Dist Col Syn Aggr Timeout Activity ------------------------------------------------------------------------------- 2/2/13 actor No No Yes Yes Yes Yes Yes Yes 2/2/13 partner No No Yes Yes Yes Yes No Yes ===============================================================================
A:PE2# show lag 60 detail =============================================================================== LAG Details =============================================================================== Description : N/A ------------------------------------------------------------------------------- Details ------------------------------------------------------------------------------- Lag-id : 60 Mode : access Adm : up Opr : down Thres. Exceeded Cnt : 5 Port Threshold : 0 Thres. Last Cleared : 10/08/2017 10:05:39 Threshold Action : down Dynamic Cost : false Encap Type : dot1q Configured Address : a0:f3:e4:60:3f:ac Lag-IfIndex : 1342177340 Hardware Address : a0:f3:e4:60:3f:ac Adapt Qos (access) : distribute Hold-time Down : 0.0 sec Port Type : standard Per-Link-Hash : disabled Include-Egr-Hash-Cfg: disabled Forced : - Per FP Ing Queuing : disabled Per FP Egr Queuing : disabled Per FP SAP Instance : disabled LACP : enabled Mode : active LACP Transmit Intvl : fast LACP xmit stdby : enabled Selection Criteria : highest-count Slave-to-partner : disabled MUX control : coupled Subgrp hold time : 0.0 sec Remaining time : 0.0 sec Subgrp selected : 1 Subgrp candidate : - Subgrp count : 1 System Id : 00:00:00:00:05:60 System Priority : 65535 Admin Key : 60 Oper Key : 60 Prtr System Id : 08:96:ad:4e:5b:e4 Prtr System Priority : 32768 Prtr Oper Key : 60 Standby Signaling : lacp Port weight speed : 0 gbps Number/Weight Up : 0 Weight Threshold : 0 Threshold Action : down ------------------------------------------------------------------------------- Port-id Adm Act/Stdby Opr Primary Sub-group Forced Prio ------------------------------------------------------------------------------- 2/2/13 up active down yes 1 - 32768 ------------------------------------------------------------------------------- Port-id Role Exp Def Dist Col Syn Aggr Timeout Activity ------------------------------------------------------------------------------- 2/2/13 actor No No No No Yes Yes Yes Yes 2/2/13 partner No No No No No Yes No Yes ===============================================================================
Multi-homing option is single-active on PE1 and PE2.
PE1 is DF and PE2 is Non DF.
A:PE1# show service system bgp-evpn ethernet-segment name "ESI-60" all =============================================================================== Service Ethernet Segment =============================================================================== Name : ESI-60 Admin State : Enabled Oper State : Up ESI : 00:11:22:33:44:55:66:77:88:60 Multi-homing : singleActive Oper Multi-homing : singleActive Source BMAC LSB : Lag Id : 60 ES Activation Timer : 0 secs Exp/Imp Route-Target : target:11:22:33:44:55:66 Svc Carving : auto ES SHG Label : 261928 =============================================================================== =============================================================================== EVI Information =============================================================================== EVI SvcId Actv Timer Rem DF ------------------------------------------------------------------------------- 60 600 0 yes ------------------------------------------------------------------------------- Number of entries: 1 =============================================================================== ------------------------------------------------------------------------------- DF Candidate list ------------------------------------------------------------------------------- EVI DF Address ------------------------------------------------------------------------------- 60 192.168.58.210 ------------------------------------------------------------------------------- Number of entries: 1 ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- <output omitted>
A:PE2# show service system bgp-evpn ethernet-segment name "ESI-60" all =============================================================================== Service Ethernet Segment =============================================================================== Name : ESI-60 Admin State : Enabled Oper State : Down ESI : 00:11:22:33:44:55:66:77:88:60 Multi-homing : singleActive Oper Multi-homing : singleActive Source BMAC LSB : Lag Id : 60 ES Activation Timer : 0 secs Exp/Imp Route-Target : target:11:22:33:44:55:66 Svc Carving : auto ES SHG Label : 262130 =============================================================================== =============================================================================== EVI Information =============================================================================== EVI SvcId Actv Timer Rem DF ------------------------------------------------------------------------------- 60 600 0 no ------------------------------------------------------------------------------- Number of entries: 1 =============================================================================== ------------------------------------------------------------------------------- DF Candidate list ------------------------------------------------------------------------------- EVI DF Address ------------------------------------------------------------------------------- 60 192.168.58.210 ------------------------------------------------------------------------------- Number of entries: 1 ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- <output omitted>
CE1
Port to PE2 will not be active in LAG.
RP/0/RSP0/CPU0:CE1#show bundle bundle-ether 60
Sat Oct 7 15:36:03.517 UTC
Bundle-Ether60
Status: Up
Local links <active/standby/configured>: 1 / 0 / 2
Local bandwidth <effective/available>: 1000000 (1000000) kbps
MAC address (source): 0896.ad4e.5be1 (Chassis pool)
Inter-chassis link: No
Minimum active links / bandwidth: 1 / 1 kbps
Maximum active links: 64
Wait while timer: 2000 ms
Load balancing: Default
LACP: Operational
Flap suppression timer: Off
Cisco extensions: Disabled
mLACP: Not configured
IPv4 BFD: Not configured
Port Device State Port ID B/W, kbps
-------------------- --------------- ----------- -------------- ----------
Gi0/0/0/2 Local Active 0x8000, 0x0002 1000000
Link is Active
Gi0/0/0/3 Local Configured 0x8000, 0x0003 1000000
Partner System ID/Key do not match that of the Selected links
Ping GW1/2 physical and Virtual IP to verify connectivity.
RP/0/RSP0/CPU0:CE1#ping 8.8.8.1 source 8.8.8.60 Sat Oct 7 15:36:31.005 UTC Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 8.8.8.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms RP/0/RSP0/CPU0:CE1#ping 8.8.8.2 source 8.8.8.60 Sat Oct 7 15:36:35.342 UTC Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 8.8.8.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/3 ms RP/0/RSP0/CPU0:CE1ping 8.8.8.3 source 8.8.8.60 Sat Oct 7 15:36:39.386 UTC Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 8.8.8.3, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/3 ms
PE1 and PE2
VPLS forward database:
- PE1 learnt 3 MAC addresses and advertised them to PE2 via EVPN.
- Previous active/active scenario, PE2 learnt CE1’s LAG MAC address by itself.
- PE1 learnt GW2’s MAC address via EVPN from PE2.
A:PE1# show service id 600 fdb detail
===============================================================================
Forwarding Database, Service 600
===============================================================================
ServId MAC Source-Identifier Type Last Change
Age
-------------------------------------------------------------------------------
600 00:00:5e:00:01:51 sap:lag-81:81 L/0 10/04/17 12:12:23
600 08:96:ad:4c:19:4a eMpls: Evpn 10/08/17 10:38:23
192.168.58.211:262132
600 08:96:ad:4e:5b:e1 sap:lag-60:81 L/210 10/08/17 10:38:10
600 08:96:ad:4e:b1:69 sap:lag-81:81 L/210 10/08/17 10:38:10
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================
A:PE2# show service id 600 fdb detail
===============================================================================
Forwarding Database, Service 600
===============================================================================
ServId MAC Source-Identifier Type Last Change
Age
-------------------------------------------------------------------------------
600 00:00:5e:00:01:51 eMpls: Evpn 10/04/17 12:12:30
192.168.58.210:261591
600 08:96:ad:4c:19:4a sap:lag-82:81 L/210 10/08/17 10:37:57
600 08:96:ad:4e:5b:e1 eES: Evpn 10/08/17 10:38:11
00:11:22:33:44:55:66:77:88:60
600 08:96:ad:4e:b1:69 eMpls: Evpn 10/08/17 10:38:11
192.168.58.210:261591
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend: L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================
To test single active fail over. I began a rapid ping between CE1 and GW1/GW2 Virtual IP address. I broke the link between PE1 and CE1. The results are shown below:
RP/0/RSP0/CPU0:CE1#ping 8.8.8.1 source 8.8.8.60 interval 1 count 10000 Sat Oct 7 15:43:13.231 UTC Type escape sequence to abort. Sending 10000, 100-byte ICMP Echos to 8.8.8.1, timeout is 2 seconds: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! <output omitted> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Success rate is 99 percent (9998/10000), round-trip min/avg/max = 1/1/4 ms