Internet DRAFT - draft-sajassi-bess-evpn-umr-mobility

draft-sajassi-bess-evpn-umr-mobility







BESS Working Group                                            A. Sajassi
Internet-Draft                                              L. Krattiger
Updates: 9014 (if approved)                             K. Ananthamurthy
Intended status: Standards Track                                   Cisco
Expires: 13 September 2023                                    J. Rabadan
                                                                   Nokia
                                                                J. Drake
                                                                 Juniper
                                                           12 March 2023


          Mobility Procedures in Presence of Unknown MAC Route
                draft-sajassi-bess-evpn-umr-mobility-01

Abstract

   Interconnect Solution for Ethernet VPN [RFC9014] defines Unknown MAC
   Route (UMR) utilization for Data Center Interconnect (DCI) when EVPN
   MPLS or EVPN VXLAN is used as overlay network for such interconnects.
   The introduction of UMR for such scenarios impacts MAC mobility
   procedures that are not discussed in [RFC9014].  This document
   describes additional changes and enhancements needed for MAC mobility
   procedures when UMR is used.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 13 September 2023.



Sajassi, et al.         Expires 13 September 2023               [Page 1]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   3
   3.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   5
   5.  Inter-network MAC Mobility procedures without UMR . . . . . .   5
   6.  Inter-network MAC Mobility Procedures for UMR . . . . . . . .   8
   7.  Duplicate MAC Address Detection . . . . . . . . . . . . . . .  12
   8.  MAC Mobility for Gateway Local Attachment Circuits  . . . . .  13
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
   10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  13
     11.1.  Normative References . . . . . . . . . . . . . . . . . .  13
     11.2.  Informative References . . . . . . . . . . . . . . . . .  13
   Appendix A.  Acknowledgments for This Document (2022) . . . . . .  14
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  14

1.  Introduction

   Ethernet Virtual Private Network (EVPN) solution is becoming
   pervasive for Network Virtualization Overlay (NVO) services in data
   center (DC) and Enterprise applications and as the next generation
   Virtual Private Network (VPN) services in service provider (SP)
   applications.

   The host IP default route and host unknown MAC route within a DC can
   be used in order to ensure that leaf nodes within a DC only learn and
   store host MAC and IP addresses for that DC.  All other host MAC and
   IP addresses from remote DCs are learned and stored in DC GW nodes
   thus alleviating leaf nodes from learning host MAC and IP addresses
   from the remote DCs and potentially improving the scale of MAC and IP
   addresses on leaf nodes by one to two order of magnitude.




Sajassi, et al.         Expires 13 September 2023               [Page 2]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


   Interconnect Solution for Ethernet VPN [RFC9014] defines Unknown MAC
   Route (UMR) utilization for Data Center Interconnect (DCI) when EVPN
   MPLS or EVPN VXLAN is used as overlay network for such interconnects.
   The introduction of UMR for such scenarios impacts MAC mobility
   procedures that are not discussed in [RFC9014].  This document
   describes additional changes and enhancements needed for MAC mobility
   procedures when UMR is used.

   Section 4 ("Requirements") of this document discusses the
   requirements for MAC Mobility when UMR is used.  Section 5
   ("Inter-network MAC Mobility procedures without UMR") discusses MAC
   Mobility for DCI operation without UMR utilization.  Section 5
   discusses MAC Mobility for DCI operation with UMR and the
   modifications and enhancements needed on top of the baseline
   operation discussed in section 4.

2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  Terminology

   EVI:  An EVPN instance spanning the Provider Edge (PE) devices
      participating in that EVPN.  An EVI may be comprised of one BD
      (VLAN-based, VLAN Bundle, or Port-based services) or multiple BDs
      (VLAN-aware Bundle or Port-based VLAN-Aware services).

   MAC-VRF:  A Virtual Routing and Forwarding table for Media Access
      Control (MAC) addresses on a PE.

   Ethernet Segment (ES):  When a customer site (device or network) is
      connected to one or more PEs via a set of Ethernet links, then
      that set of links is referred to as an 'Ethernet segment'.

   Ethernet Segment Identifier (ESI):  A unique non-zero identifier that
      identifies an Ethernet segment is called an 'Ethernet Segment
      Identifier'.

   VID:  VLAN Identifier.

   Ethernet Tag:  Used to represent a BD that is configured on a given
      ES for the purposes of DF election and <EVI, BD> identification
      for frames received from the CE.  Note that any of the following
      may be used to represent a BD: VIDs (including Q-in-Q tags),



Sajassi, et al.         Expires 13 September 2023               [Page 3]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


      configured IDs, VNIs (Virtual Extensible Local Area Network
      (VXLAN) Network Identifiers), normalized VIDs, I-SIDs (Service
      Instance Identifiers), etc., as long as the representation of the
      BDs is configured consistently across the multihomed PEs attached
      to that ES.

   Ethernet Tag ID:  Normalized network wide ID that is used to identify
      a BD within an EVI and carried in EVPN routes.

   MP2MP:  Multipoint to Multipoint.

   MP2P:  Multipoint to Point.

   P2MP:  Point to Multipoint.

   P2P:  Point to Point.

   PE:  Provider Edge device.

   Single-Active Redundancy Mode:  When only a single PE, among all the
      PEs attached to an Ethernet segment, is allowed to forward traffic
      to/from that Ethernet segment for a given VLAN, then the Ethernet
      segment is defined to be operating in Single-Active redundancy
      mode.

   All-Active Redundancy Mode:  When all PEs attached to an Ethernet
      segment are allowed to forward known unicast traffic to/from that
      Ethernet segment for a given VLAN, then the Ethernet segment is
      defined to be operating in All-Active redundancy mode.

   BUM:  Broadcast, unknown unicast, and multicast.

   DF:  Designated Forwarder.

   Backup-DF (BDF):  Backup-Designated Forwarder.

   Non-DF (NDF):  Non-Designated Forwarder.

   AC:  Attachment Circuit.

   NVO:  Network Virtualization Overlay as described in [RFC8365]

   IRB:  Integrated Routing and Bridging interface, with EVPN procedures
      described in [RFC9135]







Sajassi, et al.         Expires 13 September 2023               [Page 4]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


4.  Requirements

   This section lists the requirements for the MAC mobility solution
   when UMR is used.

   *  When an EVPN overlay network (i.e., DC, Enterprise, or SP network)
      is enabled for UMR operation, then all the PEs (leaf nodes) and
      all the gateways (border leaf nodes) in that network MUST support
      UMR operation - e.g., a DC network cannot have some leaf nodes
      supporting UMR operation and some other leaf nodes incapable of
      UMR operation.  This means when upgrading a DC network for UMR
      capability, all leaf nodes (PEs), all border leaf nodes
      (gateways), and all Route Reflectors (RRs) in that network needs
      to be upgraded before turning on the UMR capability.  If desired,
      a physical DC network can be partitioned into two logical networks
      with one supporting UMR operation and the other not supporting it.

   *  UMR MAC mobility procedures for DC networks that are UMR capable
      MUST operate seamlessly with DC networks that are UMR incapable.

   *  UMR operation is optional and a PE device (a leaf node) that
      supports UMR procedure but doesn't receive the UMR route from its
      gateway (its border leaf), SHALL operate per baseline [RFC7432].
      This does not mean that a DC network can partially operate in UMR
      mode but rather it means that a DC network can gradually be
      upgraded for UMR capability and once the entire network is
      upgraded, then it can operate in UMR mode.

5.  Inter-network MAC Mobility procedures without UMR

   In order to better differentiate the enhancements needed to MAC
   Mobility procedures for networks interconnect scenarios with
   utilization of UMR which is missing in [RFC9014], we first start with
   the description of the baseline MAC Mobility procedures (without
   utilization of UMR) in this section and then we describe the changes
   needed on top of this baseline scenario in the next section.  The
   following ladder diagram is used to help in describing baseline MAC
   Mobility operation.













Sajassi, et al.         Expires 13 September 2023               [Page 5]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


      PE11      PE12      GW1               GW2        PE22       PE21

       |         |         |                 |           |          |
       |AD(M1)   |         |                 |           |          |
    1) X-------->|         |AD(M1)           |AD(M1)     |          |
       |---------|-------->|---------------->|-----------|--------->|
       |         |         |                 |---------->|          |
       |         |         |                 |           |          |
       | AD(M1,1)|AD(M1,1) |AD(M1,1)         |AD(M1,1)   |          |
    2) |<--------X-------->|---------------->|-----------|--------->|
       |         |         |                 |---------->|          |
       |WD(M1)   |         |                 |           |          |
       |-------->|         |                 |           |          |
       |---------|-------->|                 |           |          |
       |         |         |                 |           |          |
       |         |         |                 |           |  AD(M1,2)X
       |         | AD(M1,2)|         AD(M1,2)|   AD(M1,2)|<---------|
       |         |<--------|<----------------|<----------|----------|
       |<--------|---------|                 |           |          |
    3) |         |         |                 |           |          |
       |   WD(M1)|WD(M1)   |WD(M1)           |WD(M1)     |          |
       |<--------|-------->|---------------->|-----------|--------->|
       |         |         |                 |---------->|          |
       |         |         |                 |           |          |
       |         |         |                 |           |          |
       |         | AD(M1,3)|         AD(M1,3)|   AD(M1,3)|AD(M1,3)  |
    4) |         |<--------|<----------------|<----------X--------->|
       |<--------|---------|                 |           |    WD(M1)|
       |         |         |                 |           |<---------|
       |         |         |                 |<----------|----------|

       AD(M1,x) = MAC/IP Route Advertisement for MAC M1 with seq# x
       WD(M1)   = MAC/IP Route Withdrawl for MAC M1
       X        = Host M1 being local to that PE


   This diagram depicts MAC Mobility procedures between two overlay
   networks which in turn are connected via a WAN network; where, GW1
   and GW2 sit at the edge of the WAN.  The first network consists of
   PE11, PE12, and GW1.  The second network consists of PE21, PE22, and
   GW2.  EVPN control plane is used both within each network as well as
   between them.

   1.  PE11 learns host M1 for the first time and it advertises it via
       MAC/IP Route Advertisement to other nodes in its DC network
       participating in that EVI.  Since this is the first time that
       PE11 learns of M1, it sends the advertisement without MAC
       Mobility extended community attribute per section 15 of



Sajassi, et al.         Expires 13 September 2023               [Page 6]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


       [RFC7432].  When the local gateway (GW1 of DC1) receives this
       advertisement, it readvertises this MAC address per procedures of
       [RFC9014] without MAC Mobility extended community attribute.  The
       remote gateway (GW2 of DC2) receives this advertisement and it in
       turn readvertises it to its PEs participating in that EVI.  At
       this point, remote PE21 and PE22 know that the next hop for M1 is
       GW2, and GW2 knows that the next hop for M1 is GW1.

   2.  In this step, host M1 makes an intra-DC move within DC1 network
       and moves from PE11 to PE12, the PE12 follows the MAC mobility
       procedures in [RFC7432] and advertises the MAC/IP Advertisement
       route for M1 with a sequence number which is incremented by one
       (in this case seq = 1).  PE11 and GW1 receive this advertisement
       and update their next hop for M1 to point to PE12.  GW1 follows
       the procedures in [RFC9014] and it readvertises this route with
       this new sequence number received from PE12.  Upon receiving this
       route, GW2 updates its sequence number for M1 and in turn it
       readvertises this route to its DC.  PE21 and PE22 receive this
       advertisement and update their sequence numbers for M1 (seq = 1),
       but there is no change to the next hop for M1 and they keep it as
       GW2.

       Furthermore, after verifying that M1 is no longer present
       locally, PE11 sends a withdrawal message for M1 to all local PEs
       and GWs that are participating in that EVI per MAC Mobility
       procedures of [RFC7432].  When PE12 and GW1 receive this
       withdrawal message, they clean up their BGP tables and remove BGP
       EVPN route for M1 received from PE11.  Since BGP table in GW1 has
       at least one BGP EVPN route learned from its DC1 (and host M1 has
       as its next hop one of the PEs in DC1), GW1 does not readvertise
       the withdrawal message for M1 received from PE11.

   3.  In this step, host M1 moves from PE12 in DC1 to PE21 in DC2.
       PE21 upon learning M1 locally, it advertises the MAC/IP
       Advertisement route for M1 with a sequence number which is
       incremented by one (in this case seq = 2).  PE22 and GW2 receive
       this advertisement, recognize the move, and update their next hop
       for M1 to point to PE21.  GW2 follows the procedures in [RFC9014]
       and it readvertises this route with this new sequence number
       received from PE21.  Upon receiving this route, GW1 updates its
       sequence number for M1 and in turn it readvertises this route to
       its DC1 network.  PE11 and PE12 receive this advertisement,
       recognize the move, and update their next hop to point to GW1,
       and their sequence numbers for M1.

       Furthermore, after verifying that M1 is no longer present
       locally, PE12 sends a withdrawal message for M1 to all local PEs
       and GWs that are participating in that EVI.  When PE11 and GW1



Sajassi, et al.         Expires 13 September 2023               [Page 7]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


       receive this withdrawal message, they clean up their BGP tables
       and remove BGP EVPN route for M1 received from PE12.  Since there
       is no more local BGP EVPN routes for M1 in BGP table of GW1
       (i.e., no more routes from its local PEs), it readvertises this
       withdrawal message to other GWs over WAN.  When other GWs over
       WAN (including GW2) receive this withdrawal message, they remove
       the BGP EVPN route for M1 received from GW1.  At this point the
       only BGP EVPN route entry in GW1 is the one received from GW2,
       and for GW2 is the one received from its local PE21.

   4.  This step is similar to that of step 2 and demonstrates what it
       takes place when an intra-DC move happens but this time within
       DC2 where the host M1 moves from PE21 to PE22.  The PE22 follows
       the MAC mobility procedures in [RFC7432] and advertises the MAC/
       IP Advertisement route for M1 with a sequence number which is
       incremented by one (in this case seq = 3).  PE21 and GW2 receive
       this advertisement and update their next hop for M1 to point to
       PE22.  GW2 follows the procedures in [RFC9014] and it
       readvertises this route with this new sequence number received
       from PE22.  Upon receiving this route, GW1 updates its sequence
       number for M1 and in turn it readvertises this route to its DC.
       PE11 and PE12 receive this advertisement and update their
       sequence numbers for M1 (seq = 3), but there is no change to the
       next hop for M1 and they keep it as GW1.

       Furthermore, after verifying that M1 is no longer present
       locally, PE21 sends a withdrawal message for M1 to all local PEs
       and GWs that are participating in that EVI per MAC Mobility
       procedures of [RFC7432].  When PE22 and GW2 receive this
       withdrawal message, they clean up their BGP tables and remove BGP
       EVPN route for M1 received from PE21.  Since BGP table in GW2 has
       at least one BGP EVPN route learned from its DC2 (and host M1 has
       as its next hop one of the PEs in DC2), GW2 does not readvertise
       the withdrawal message for M1 received from PE21.

6.  Inter-network MAC Mobility Procedures for UMR

   This section describes the changes needed to MAC Mobility procedures
   when UMR is utilized in EVPN overlay networks and hosts are allowed
   to move between these networks.  Since advertisement of MAC/IP
   addresses from the local overlay network are not propagated all the
   way to the PEs of the remote overlay network, the baseline MAC
   mobility procedures described in the previous section, cannot be used
   as is and it needs to be modified.  When a host M1 moves from one
   overlay network (e.g., DC1) to anoher one (e.g., DC2), the PE in DC2
   (PE21) that learns the host locally, it learns it for the very first
   time because of UMR operation - i.e., M1 never previously got
   advertised by DC2 GW (GW2).  Therefore, the PE21 advertises EVPN MAC/



Sajassi, et al.         Expires 13 September 2023               [Page 8]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


   IP route for M1 without any sequence number which breaks the baseline
   MAC mobility procedures described in the previous section.  In order
   to accommodate MAC mobility in the presence of UMR, each gateway
   needs to maintain two sequence numbers per host MAC address - one for
   its local overlay network (e.g., its DC network) and another one for
   the interconnect network (e.g., WAN network).  Only gateways need to
   maintain both MAC Mobility sequence numbers.  The PEs that are
   enabled for UMR operation, only need to maintain a single MAC
   Mobility sequence number per MAC address.  These two sequence numbers
   operate independently (i.e., they get incremented independently) so
   that a local MAC move within an overlay network (e.g., DC1) does not
   impact other overlay networks (e.g., other DCs) and the interconnect
   network (e.g., WAN network) - i.e., when the host mobility is
   confined to a DC (aka intra-DC host mobility), then only the intra-DC
   MAC Mobility counter for that DC is incremented upon host move
   without any changes to inter-DC MAC Mobility counter or any other
   intra-DC MAC Mobility counters in other DCs.  However, when the host
   moves from one DC to another, then the inter-DC MAC Mobility counter
   is impacted.

   The solution described in this section optimizes based on convergence
   time and number of BGP EVPN route advertisements - i.e., it tries to
   minimize the convergence time upon a host move and to minimize the
   number of EVPN route advertisements.  Whenever these two factors are
   in conflict, the preference is given to minimizing the convergence
   time.  The following ladder diagram is used to help in describing MAC
   Mobility procedures for UMR operation.
























Sajassi, et al.         Expires 13 September 2023               [Page 9]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


      PE11      PE12      GW1               GW2        PE22       PE21

       |         |         |                 |           |          |
       |AD(M1)   |         |                 |           |          |
    1) X-------->|         |AD(M1)           |           |          |
       |---------|-------->|---------------->|           |          |
       |         |         |                 |           |          |
       |         |         |                 |           |          |
       | AD(M1,1)|AD(M1,1) |                 |           |          |
       |<--------X-------->|                 |           |          |
       |         |         |                 |           |          |
    2) |WD(M1)   |         |                 |           |          |
       |-------->|         |                 |           |          |
       |---------|-------->|                 |           |          |
       |         |         |                 |           |          |
       |         |         |                 |           |    AD(M1)X
       |         | AD(M1,2)|         AD(M1,1)|     AD(M1)|<---------|
       |         |<--------|<----------------|<----------|----------|
       |<--------|---------|                 |           |          |
       |         |         |                 |           |          |
       |   WD(M1)|WD(M1)   |                 |           |          |
    3) |<--------|-------->|WD(M1)           |           |          |
       |         |<--------|---------------->|           |          |
       |<--------|---------|                 |           |          |
       |         |         |                 |   AD(M1,1)|AD(M1,1)  |
       |         |         |                 |<----------X--------->|
       |         |         |                 |           |          |
    4) |         |         |                 |           |    WD(M1)|
       |         |         |                 |           |<---------|
       |         |         |                 |<----------|----------|

       AD(M1,x) = MAC/IP Route Advertisement for MAC M1 with seq# x
       WD(M1)   = MAC/IP Route Withdrawl for MAC M1
       X        = Host M1 being local to that PE


   This diagram depicts MAC Mobility procedures between two overlay
   networks which in turn are connected via a WAN network where UMR
   operation is utilized.  To avoid repeating the text verbatim from
   previous section and put the emphasis on the new procedures, we
   mainly elaborate on the changes relative to the baseline MAC mobility
   procedures described in the previous section.

   When UMR operation is enabled for a given EVI, all gateways
   participating in that EVI for that overlay network, advertise the UMR
   route to their local overlay network.  A PE that is capable of UMR
   processing, upon receiving the UMR route, activates its UMR procedure
   as described below.  When a gateway receives the UMR route from



Sajassi, et al.         Expires 13 September 2023              [Page 10]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


   another gateway for one of its EVI for which UMR operation is
   enabled, it should simply discard it (i.e., not to add it to its BGP
   table and MAC-VRF).

   1.  When the host M1 is learned in DC1 for the first time, the
       baseline MAC Mobility procedure described in step (1) of
       Section 5 is executed in DC1 among PE11, PE12, and GW1.  When the
       remote gateway (GW2 of DC2) receives this advertisement from GW1,
       it processes it just as step (1) of Section 5 and adds it to its
       BGP table and MAC-VRF for that EVI.  However, it does not
       readvertise it into its own DC network because it is configured
       for UMR operation and no remote MAC/IP Advertisement routes
       (routes received from remote GWs) are ever readvertised locally.

   2.  When the host M1 makes an intra-DC move within DC1 network, the
       baseline MAC Mobility procedure described in step (2) of
       Section 5 is executed in DC1 among PE11, PE12, and GW1.  GW1
       realizes that this is an intra-overlay network MAC move (intra-DC
       MAC move) and thus it does not readvertises this route to other
       GWs in the WAN network.  It should be noted that GW1 maintains
       two sequence numbers for M1 and it increments its intra-DC
       sequence number by one (seq = 1); however, it leaves its inter-DC
       sequence number unchanged (seq = 0).

       Generation of the withdrawal message by PE11 and processing of
       this message by other EVPN devices in DC1 (i.e., PE12 and GW1)
       are the same as the ones described in step (2) of Section 5.
       Since after receiving the withdrawal message and cleaning up its
       BGP table, GW1 still has at least one BGP EVPN route from its
       local DC1 (and host M1 has as its next hop one of the PEs in
       DC1), GW1 does not readvertise the withdrawal message for M1
       received from PE11 to remote gateways (e.g., GW2 of DC2).

   3.  When the host M1 moves from DC1 to DC2 and its presence is
       detected locally by the PE21, the PE21 learns M1 for the very
       first time since its gateway (GW2) never advertised MAC/IP route
       for M1 to its local PEs (because of UMR operation).  The PE21
       advertises MAC/IP route for M1 without any sequence number.  All
       PEs and gateways in DC2 upon receiving this advertisement update
       their BGP and MAC-VRF tables.  In addition to this update, the
       GW2 recognizes that there has been a MAC move, increments its
       inter-DC MAC Mobility counter for M1, and it readvertises this
       MAC/IP route along with the updated MAC Mobility extended
       community.

       GW1 receives this MAC/IP advertisement for M1 and it also
       recognizes that M1 has moved to DC2.  GW1 increments its intra-DC
       MAC Mobility sequence number and it readvertises this route along



Sajassi, et al.         Expires 13 September 2023              [Page 11]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


       with the updated MAC Mobility extended community (seq = 2) to its
       local DC for that EVI.  As the result, all the PEs participating
       for that EVI in DC1, receive this MAC/IP advertisement and update
       their BGP and MAC-VRF tables.  They also update the BGP next hop
       to point to GW1 for MAC address M1.  Besides this update, PE12
       recognizes the MAC move and advertises a withdrawal message for
       M1.  Furthermore, it verifies that M1 has actually moved and is
       no longer present locally.

       When the gateways and other PEs in DC1 receive this withdrawal
       message from PE12, they cleanup their BGP tables and remove the
       corresponding M1 entry from their tables.  After this cleanup,
       GW1 realizes that there is no more entry for M1 in its BGP table
       from its local PEs and thus it sends a withdrawal message for M1
       to all its local PEs and remote gateways (e.g., GW2).
       Furthermore, GW1 must reset its intra-DC MAC mobility counter for
       M1 to zero because M1 no longer exist among its local PEs.  When
       the local PEs (PE11 and PE12) receive this withdrawal message,
       they clean up their BGP and MAC-VRF tables for M1.  After the
       cleanup, there should be no entry in BGP and MAC-VRF tables for
       M1 and thus the forwarding for M1 must follow UMR operation -
       i.e., the packet with the destination MAC address of M1 must be
       load balanced to one of the GWs that has advertised UMR route.
       When the remote gateways receive this withdrawal message, they
       clean up their BGP tables for M1 and the only entry in BGP table
       for M1 should be that of the one received from GW2.

   4.  This step demonstrates an intra-DC MAC move for DC2.  The
       procedure for the PEs (PE21 and PE22) and the corresponding
       gateway (GW2) is the same as the one described in step 2 and thus
       no further explanation is needed.

   Redundant gateways are supported by the described procedure.  All the
   redundant gateways attached to a given BD advertise the EVPN MAC/IP
   Advertisement routes with the same Interconnect ESI [RFC9014], and
   all the redundant gateways MUST use the same sequence numbers when
   advertising MAC addresses to either their local overlay network or
   their interconnect network.

7.  Duplicate MAC Address Detection

   Duplicate MAC addresses can occur as described in section 15.1 of
   [RFC7432].  MAC address duplication can happen within the same DC
   network (e.g., DC1) or across different DC networks (e.g., DC1 and
   DC2) where UMR is utilized.  In either case, the procedure is the
   same as the one described in section 15.1 of [RFC7432].  More
   specifically, the timer and the move counter for a given MAC are kept
   only at the PEs - i.e., there is no need to maintain such timer and



Sajassi, et al.         Expires 13 September 2023              [Page 12]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


   move counter for a given MAC unless that MAC is learned locally on
   that GW.

8.  MAC Mobility for Gateway Local Attachment Circuits

   This section describes MAC Mobility procedures for hosts sitting
   behind local Attachment Circuts (ACs) of a gateway and moving to/from
   a local PE, or another gateway in a remote DC, or a remote PE in a
   remote DC.  TBD.

9.  Security Considerations

   Since this document describes how to address MAC mobility issue as
   the result of using UMR for interconnection solutions of [RFC9014],
   and since no new requiremnts with respect to mobility procedures are
   introduced at the edge devices (e.g., PEs or leafs), there is no
   additional security risks beyond the ones described in [RFC7432] and
   [RFC8365].

10.  IANA Considerations

   This document does not introduce any IANA requirements.

11.  References

11.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC9014]  Rabadan, J., Ed., Sathappan, S., Henderickx, W., Sajassi,
              A., and J. Drake, "Interconnect Solution for Ethernet VPN
              (EVPN) Overlay Networks", RFC 9014, DOI 10.17487/RFC9014,
              May 2021, <https://www.rfc-editor.org/info/rfc9014>.

11.2.  Informative References




Sajassi, et al.         Expires 13 September 2023              [Page 13]

Internet-Draft       EVPN Unknown MAC Route Mobility          March 2023


   [RFC8365]  Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
              Uttaro, J., and W. Henderickx, "A Network Virtualization
              Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365,
              DOI 10.17487/RFC8365, March 2018,
              <https://www.rfc-editor.org/info/rfc8365>.

   [RFC9135]  Sajassi, A., Salam, S., Thoria, S., Drake, J., and J.
              Rabadan, "Integrated Routing and Bridging in Ethernet VPN
              (EVPN)", RFC 9135, DOI 10.17487/RFC9135, October 2021,
              <https://www.rfc-editor.org/info/rfc9135>.

Appendix A.  Acknowledgments for This Document (2022)

   TBD.

Authors' Addresses

   Ali Sajassi
   Cisco
   Email: sajassi@cisco.com


   Lukas Krattiger
   Cisco
   Email: lkrattig@cisco.com


   Krishna Ananthamurthy
   Cisco
   Email: kriswamy@cisco.com


   Jorge Rabadan
   Nokia
   Email: jorge.rabadan@nokia.com


   John Drake
   Juniper
   Email: jdrake@juniper.net











Sajassi, et al.         Expires 13 September 2023              [Page 14]