Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the limit-login-attempts-reloaded domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /mnt/plesk_vhosts/vhosts/andonguyen.com/httpdocs/wp-includes/functions.php on line 6114
EVPN On Nokia 7450 Notes Part 1 – An Do Nguyen's Blog
EVPN On Nokia 7450 Notes Part 1

EVPN On Nokia 7450 Notes Part 1

Comments 0 Comment

Background

I discovered that Nokia’s road map for MC-LAG does not include active/active. After discussion with Nokia, they suggest EVPN was the best solution to introduce this capability. So after my research  on EVPN it was time to POC it. This post assume basic knowledge of EVPN. If you would like to review EVPN, this blog post is a great starting point.

Scenario 1: All-active Multi-homing

In the above topology, PE1 and PE2 will  be part of an EVPN-VPLS. LAG 60 will be a multi-homed active/active lag to CE1. LAG 81 and 82 will be traditional LAGs to GW1 and GW2.

The building blocks for EVPN in this solution are:

  • OSPF
  • RSVP-TE
  • MPLS
  • BGP
  • LAG
  • EVPN
  • VPLS

Configuration

OSPF, RSVP-TE and MPLS will not be covered as it is very vanilla.

We will start from BGP configuration on PE1. PE2 is very similar. I will bold lines that are important.

*A:PE1>config>router>bgp# info
----------------------------------------------
        family evpn
        graceful-restart
            stale-routes-time 300
        exit
        damping
        group "EVPN"
            local-as 65456
            peer-as 65456
            local-address 192.168.58.210
            neighbor 192.168.58.211
            exit
        exit
        no shutdown
----------------------------------------------

Port Configuration to CE1 from PE1.

A:PE1>config>port# info
----------------------------------------------
        ethernet
            mode access
            encap-type dot1q
            down-when-looped
                keep-alive 30
                no shutdown
            exit
            lldp
                dest-mac nearest-bridge
                    admin-status tx-rx
                    notification
                    tx-tlvs sys-name
                    tx-mgmt-address system
                exit
            exit
            no autonegotiate
        exit
        no shutdown
----------------------------------------------

LAG configuration to CE1:

  • Both PE1 and PE2 have the same system ID and admin key.
*A:PE1>config>lag# info
----------------------------------------------
        mode access
        encap-type dot1q
        port 2/2/13
        lacp active administrative-key 60 system-id 00:00:00:00:05:60
        no shutdown
----------------------------------------------

ESI constructs and flag options for multi-homing:

  • RD uses :0 to inform the box to self derive per EVI.
  • Can not use all zeros in bytes 2-7 for ESI.
  • Service carving enforces the PEs to load balance DF role.
  • Multi-homing can either be all-active or active-standby.
A:PE1>config>service>system>bgp-evpn# info
----------------------------------------------
                route-distinguisher 192.168.58.210:0
                ethernet-segment "ESI-60" create
                    esi 00:11:22:33:44:55:66:77:88:60
                    es-activation-timer 0
                    service-carving
                        mode auto
                    exit
                    multi-homing all-active
                   lag 60
                    no shutdown
                exit
----------------------------------------------

VPLS service construct:

  • Unique EVI per VPLS.
  • Due to ingress-replication-bum-label. PE advertises another label for BUM traffic.
  • MPLS will auto bind tunnels for data plane. We couldn’t get SDP contstruct to work.
A:PE1>config>service>vpls# info
----------------------------------------------
            description "EVPN service 1 "
            bgp
            exit
            bgp-evpn
                evi 60 #needs to be unique per VPLS
                vxlan
                    shutdown
                exit
                mpls
                    ingress-replication-bum-label 
                    auto-bind-tunnel 
                        resolution any
                    exit
                    no shutdown
                exit
            exit
            stp
                shutdown
            exit
            service-name "EVPN service 1"
            sap lag-60:81 create
                description "access_sap"
            exit
            sap lag-81:81 create
            exit
            no shutdown
----------------------------------------------

Verification

PE1

Physical port from PE1 to CE1 is up and in the correct LAG.

A:PE1#  show port 2/2/13

===============================================================================
Ethernet Interface
===============================================================================
Description        : 
Interface          : 2/2/13                     Oper Speed       : 1 Gbps
Link-level         : Ethernet                   Config Speed     : 1 Gbps
Admin State        : up                         Oper Duplex      : full
Oper State         : up - Active in LAG 60      Config Duplex    : full
Physical Link      : Yes                        MTU              : 1518
<output omitted>

LAG from PE1 to CE1 is up.

A:PE1# show lag 60  detail

===============================================================================
LAG Details
===============================================================================
Description        : N/A
-------------------------------------------------------------------------------
Details
-------------------------------------------------------------------------------
Lag-id              : 60                    Mode                 : access
Adm                 : up                    Opr                  : up
Thres. Exceeded Cnt : 16                    Port Threshold       : 0
Thres. Last Cleared : 09/20/2017 10:22:24   Threshold Action     : down
Dynamic Cost        : false                 Encap Type           : dot1q
Configured Address  : a0:f3:e4:5f:73:ac     Lag-IfIndex          : 1342177340
Hardware Address    : a0:f3:e4:5f:73:ac     Adapt Qos (access)   : distribute
Hold-time Down      : 0.0 sec               Port Type            : standard
Per-Link-Hash       : disabled
Include-Egr-Hash-Cfg: disabled              Forced               : -
Per FP Ing Queuing  : disabled              Per FP Egr Queuing   : disabled
Per FP SAP Instance : disabled
LACP                : enabled               Mode                 : active
LACP Transmit Intvl : fast                  LACP xmit stdby      : enabled
Selection Criteria  : highest-count         Slave-to-partner     : disabled
MUX control         : coupled
Subgrp hold time    : 0.0 sec               Remaining time       : 0.0 sec
Subgrp selected     : 1                     Subgrp candidate     : -
Subgrp count        : 1
System Id           : 00:00:00:00:05:60     System Priority      : 32768
Admin Key           : 60                    Oper Key             : 60
Prtr System Id      : 08:96:ad:4e:5b:e4     Prtr System Priority : 32768
Prtr Oper Key       : 60
Standby Signaling   : lacp
Port weight speed   : 0 gbps                Number/Weight Up     : 1
Weight Threshold    : 0                     Threshold Action     : down

-------------------------------------------------------------------------------
Port-id        Adm     Act/Stdby Opr     Primary   Sub-group     Forced  Prio
-------------------------------------------------------------------------------
2/2/13         up      active    up      yes       1             -       32768

-------------------------------------------------------------------------------
Port-id        Role      Exp   Def   Dist  Col   Syn   Aggr  Timeout  Activity
-------------------------------------------------------------------------------
2/2/13         actor     No    No    Yes   Yes   Yes   Yes   Yes      Yes
2/2/13         partner   No    No    Yes   Yes   Yes   Yes   No       Yes
===============================================================================

Transport labels exchanged between PEs:

  • Label for BUM traffic.
  • Label for Unicast traffic.
A:PE1# show service id 600 evpn-mpls

===============================================================================
BGP EVPN-MPLS Dest
===============================================================================
TEP Address     Egr Label     Num. MACs   Mcast           Last Change
                 Transport
-------------------------------------------------------------------------------
192.168.58.211  262131        0           Yes             10/04/2017 12:12:23
                rsvp
192.168.58.211  262132        1           No              10/06/2017 11:12:06
                rsvp
-------------------------------------------------------------------------------
Number of entries : 2
-------------------------------------------------------------------------------
===============================================================================

<output omitted>

VPLS is up.

A:PE1# show service id  600 all

===============================================================================
Service Detailed Information
===============================================================================
Service Id        : 600                 Vpn Id            : 0
Service Type      : VPLS
Name              : EVPN service 1 for MAS
Description       : EVPN service 1 for MAS
Customer Id       : 1                   Creation Origin   : manual
Last Status Change: 09/11/2017 13:55:56
Last Mgmt Change  : 09/12/2017 13:00:16
Etree Mode        : Disabled
Admin State       : Up                  Oper State        : Up
MTU               : 1514                Def. Mesh VC Id   : 600
SAP Count         : 2                   SDP Bind Count    : 0
Snd Flush on Fail : Disabled            Host Conn Verify  : Disabled
SHCV pol IPv4     : None
Propagate MacFlush: Disabled            Per Svc Hashing   : Disabled
Allow IP Intf Bind: Disabled            Fwd-IPv4-Mcast-To*: Disabled
Def. Gateway IP   : None
Def. Gateway MAC  : None
Temp Flood Time   : Disabled            Temp Flood        : Inactive
Temp Flood Chg Cnt: 0
VSD Domain        : 
SPI load-balance  : Disabled
TEID load-balance : Disabled

-------------------------------------------------------------------------------

VPLS BGP RD and RT are auto derived from EVI.

A:PE1# show service id 600 bgp

===============================================================================
BGP Information
===============================================================================
Vsi-Import           : None
Vsi-Export           : None
Route Dist           : None
Oper Route Dist      : 192.168.58.210:60
Oper RD Type         : derivedEvi
Rte-Target Import    : None                 Rte-Target Export: None
Oper RT Imp Origin   : derivedEvi           Oper RT Import   : 65456:60
Oper RT Exp Origin   : derivedEvi           Oper RT Export   : 65456:60
PW-Template Id       : None
-------------------------------------------------------------------------------
===============================================================================

SAP is up in VPLS.

A:PE1# show service id  600 sap

===============================================================================
SAP(Summary), Service 600
===============================================================================
PortId                          SvcId      Ing.  Ing.    Egr.  Egr.   Adm  Opr
                                           QoS   Fltr    QoS   Fltr
-------------------------------------------------------------------------------
lag-60:81                       600        1     none    1     none   Up   Up
lag-81:81                       600        1     none    1     none   Up   Up
-------------------------------------------------------------------------------
Number of SAPs : 2
-------------------------------------------------------------------------------
===============================================================================

VPLS BGP labels allocated and tunnel is activated.

:PE1# show service id  600 bgp-evpn

===============================================================================
BGP EVPN Table
===============================================================================
MAC Advertisement  : Enabled            Unknown MAC Route  : Disabled
CFM MAC Advertise  : Disabled
VXLAN Admin Status : Disabled           Creation Origin    : manual
MAC Dup Detn Moves : 5                  MAC Dup Detn Window: 3
MAC Dup Detn Retry : 9                  Number of Dup MACs : 0
IP Route Advertise*: Disabled

EVI                : 60

-------------------------------------------------------------------------------
Detected Duplicate MAC Addresses             Time Detected
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
===============================================================================
* indicates that the corresponding row element may have been truncated.

===============================================================================
BGP EVPN MPLS Information
===============================================================================
Admin Status       : Enabled
Force Vlan Fwding  : Disabled           Control Word       : Disabled
Split Horizon Group: (Not Specified)
Ingress Rep BUM Lbl: Enabled            Max Ecmp Routes    : 0
Ingress Ucast Lbl  : 261591             Ingress Mcast Lbl  : 261569
===============================================================================

===============================================================================
BGP EVPN MPLS Auto Bind Tunnel Information
===============================================================================
Resolution         : any
Filter Tunnel Types: (Not Specified)
===============================================================================

The DF for this Ethernet Segment has been selected and the multi-homing flags are correct.

A:PE1# show service system bgp-evpn ethernet-segment name "ESI-60"  all

===============================================================================
Service Ethernet Segment
===============================================================================
Name                    : ESI-60
Admin State             : Enabled            Oper State         : Up
ESI                     : 00:11:22:33:44:55:66:77:88:60
Multi-homing            : allActive          Oper Multi-homing  : allActive
Source BMAC LSB         : 
Lag Id                  : 60
ES Activation Timer     : 0 secs
Exp/Imp Route-Target    : target:11:22:33:44:55:66

Svc Carving             : auto
ES SHG Label            : 261578
===============================================================================

===============================================================================
EVI Information
===============================================================================
EVI                 SvcId               Actv Timer Rem      DF
-------------------------------------------------------------------------------
60                  600                 0                   yes
-------------------------------------------------------------------------------
Number of entries: 1
===============================================================================

-------------------------------------------------------------------------------
DF Candidate list
-------------------------------------------------------------------------------
EVI                                     DF Address
-------------------------------------------------------------------------------
60                                      192.168.58.210
60                                      192.168.58.211
-------------------------------------------------------------------------------
Number of entries: 2
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------

===============================================================================
<output omitted>

The 4 BGP route types for EVPN are received from PE2.

A:PE1# show router bgp neighbor 192.168.58.211 received-routes evpn
===============================================================================
 BGP Router ID:192.168.58.210   AS:65456       Local AS:65456
===============================================================================
 Legend -
 Status codes  : u - used, s - suppressed, h - history, d - decayed, * - valid
                 l - leaked, x - stale, > - best, b - backup, p - purge
 Origin codes  : i - IGP, e - EGP, ? - incomplete

===============================================================================
BGP EVPN Auto-Disc Routes
===============================================================================
Flag  Route Dist.         ESI                           NextHop
      Tag                                               Label
-------------------------------------------------------------------------------
u*>i  192.168.58.211:60   00:11:22:33:44:55:66:77:88:60 192.168.58.211
      0                                                 LABEL 262132

u*>i  192.168.58.211:60   00:11:22:33:44:55:66:77:88:60 192.168.58.211
      MAX-ET                                            LABEL 0

u*>i  192.168.58.211:61   00:11:22:33:44:55:66:77:88:61 192.168.58.211
      0                                                 LABEL 262129

u*>i  192.168.58.211:61   00:11:22:33:44:55:66:77:88:61 192.168.58.211
      MAX-ET                                            LABEL 0

-------------------------------------------------------------------------------
Routes : 4
===============================================================================

===============================================================================
BGP EVPN MAC Routes
===============================================================================
Flag  Route Dist.         MacAddr           ESI
      Tag                 Mac Mobility      Ip Address
                                            NextHop
                                            Label1
-------------------------------------------------------------------------------
u*>i  192.168.58.211:60   08:96:ad:4c:19:4a ESI-0
      0                   Seq:0             N/A
                                            192.168.58.211
                                            LABEL 262132

u*>i  192.168.58.211:60   08:96:ad:4e:5b:e1 00:11:22:33:44:55:66:77:88:60
      0                   Seq:0             N/A
                                            192.168.58.211
                                            LABEL 262132

-------------------------------------------------------------------------------
Routes : 2
===============================================================================

===============================================================================
BGP EVPN Inclusive-Mcast Routes
===============================================================================
Flag  Route Dist.         OrigAddr            NextHop
      Tag
-------------------------------------------------------------------------------
u*>i  192.168.58.211:1    192.168.58.211      192.168.58.211
      0

u*>i  192.168.58.211:60   192.168.58.211      192.168.58.211
      0

u*>i  192.168.58.211:61   192.168.58.211      192.168.58.211
      0

u*>i  192.168.58.211:88   192.168.58.211      192.168.58.211
      0

-------------------------------------------------------------------------------
Routes : 4
===============================================================================

===============================================================================
BGP EVPN Eth-Seg Routes
===============================================================================
Flag  Route Dist.         ESI                           NextHop
      OrigAddr
-------------------------------------------------------------------------------
u*>i  192.168.58.211:0    00:11:22:33:44:55:66:77:88:60 192.168.58.211
      192.168.58.211

u*>i  192.168.58.211:0    00:11:22:33:44:55:66:77:88:61 192.168.58.211
      192.168.58.211

-------------------------------------------------------------------------------
Routes : 2
===============================================================================
<output omitted>

CE1

LAG on CE1 has both ports active/active.

RP/0/RSP0/CPU0:CE1#show bundle bundle-ether 61
Thu Oct  5 16:48:05.211 UTC

Bundle-Ether61
  Status:                                    Up
  Local links <active/standby/configured>:   2 / 0 / 2
  Local bandwidth <effective/available>:     2000000 (2000000) kbps
  MAC address (source):                      0222.2222.2222 (Configured)
  Inter-chassis link:                        No
  Minimum active links / bandwidth:          1 / 1 kbps
  Maximum active links:                      64
  Wait while timer:                          2000 ms
  Load balancing:                            Default
  LACP:                                      Operational
    Flap suppression timer:                  Off
    Cisco extensions:                        Disabled
  mLACP:                                     Not configured
  IPv4 BFD:                                  Not configured

  Port                  Device           State        Port ID         B/W, kbps
  --------------------  ---------------  -----------  --------------  ----------
  Gi0/0/0/6             Local            Active       0x8000, 0x0004     1000000
      Link is Active
  Gi0/0/0/7             Local            Active       0x8000, 0x0005     1000000
      Link is Active
RP/0/RSP0/CPU0:RTCSR0000003NAT#

Ping GW1/2 physical and Virtual IP to verify connectivity.

RP/0/RSP0/CPU0:CE1#ping 8.8.8.1 source 8.8.8.60
Thu Oct  5 20:26:40.002 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
RP/0/RSP0/CPU0:CE1T#ping 8.8.8.2 source 8.8.8.60
Thu Oct  5 20:26:43.089 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
RP/0/RSP0/CPU0:CE1#ping 8.8.8.3 source 8.8.8.60
Thu Oct  5 20:26:45.713 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms

PE1 and PE2

VPLS forward database:

  • Since GW1 is master for VRRP. Most traffic will transverse PE1. We can see 2 MAC addresses are learnt on PE1 then advertise to PE2 via EVPN.
  • The MAC address for CE1’s LAG is learnt on both PE1 and PE2.
  • PE1 learnt GW2’s MAC address from PE2 via EVPN.
A:PE1# show service id 600 fdb  detail

===============================================================================
Forwarding Database, Service 600
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age
-------------------------------------------------------------------------------
600       00:00:5e:00:01:51 sap:lag-81:81            L/0      10/04/17 12:12:23
600       08:96:ad:4c:19:4a eMpls:                   Evpn     10/07/17 21:35:14
                            192.168.58.211:262132
600       08:96:ad:4e:5b:e1 sap:lag-60:81            L/9      10/07/17 21:34:53
600       08:96:ad:4e:b1:69 sap:lag-81:81            L/0      10/07/17 21:34:51
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================

A:PE2# show service id 600 fdb  detail

===============================================================================
Forwarding Database, Service 600
===============================================================================
ServId    MAC               Source-Identifier        Type     Last Change
                                                     Age
-------------------------------------------------------------------------------
600       00:00:5e:00:01:51 eMpls:                   Evpn     10/04/17 12:12:30
                            192.168.58.210:261591
600       08:96:ad:4c:19:4a sap:lag-82:81            L/23     10/07/17 21:34:33
600       08:96:ad:4e:5b:e1 sap:lag-60:81            L/0      10/07/17 21:34:30
600       08:96:ad:4e:b1:69 eMpls:                   Evpn     10/07/17 21:34:32
                            192.168.58.210:261591
-------------------------------------------------------------------------------
No. of MAC Entries: 4
-------------------------------------------------------------------------------
Legend:  L=Learned O=Oam P=Protected-MAC C=Conditional S=Static
===============================================================================

Leave a Reply

Your email address will not be published. Required fields are marked *