+-------+   +-------+   +------+   +------+
           |       |   |       |   |      |   |      |
        +--+ AGN11 +---+ AGN21 +---+ ABR1 +---+ LSR1 +--> to AGN
       /   |       |  /|       |   |      |   |      |
+----+/    +-------+\/ +-------+   +------+  /+------+
| AN |              /\                     \/     |
+----+\    +-------+  \+-------+   +------+/\ +------+
       \   |       |   |       |   |      |  \|      |
        +--+ AGN12 +---+ AGN22 +---+ ABR2 +---+ LSR2 +--> to AGN
           |       |   |       |   |      |   |      |
           +-------+   +-------+   +------+   +------+

static route     ISIS L1 LDP             ISIS L2 LDP

<-Access-><--Aggregation Domain--><---------Core---------> 

Figure 5: Deployment Scenario #1

As shown in Figure 5, the access nodes (AN) are connected to the aggregation network via aggregation nodes called AGN1x, either to a single AGN1x or redundantly to two AGN1x. Each AGN1x has redundant uplinks to a pair of second-level aggregation nodes called AGN2x.

Each aggregation domain is connected to the core via exactly two border routers (ABR) on the core side. There can be multiple AGN2 pairs per aggregation domain, but only one ABR pair for each aggregation domain. Each of the AGN2 in an AGN2 pair connects to one of the ABRs in the ABR pair responsible for that aggregation domain.

The ABRs on the core side have redundant connections to a pair of LSR routers.

The LSR pair is also connected via a direct link.

The core LSR are connected to other core LSR in a partly meshed topology so that there are disjunct, redundant paths from each LSR to each other LSR.

5.1.3. Hierarchy based on recursive BGP labeled route lookup

Inline with the explanation in section 4.5, LSP hierarchy is key to a scalable seamless MPLS architecture.

The LSP hierarchy in this design is achieved by:

• Forming separate MPLS domains for aggregation and core areas.
• Intra-domain LSP connectivity provided by combination of IS-IS (as the intra-domain link-state routing protocol) and LDP (used for MPLS label distribution for intra-domain LSPs).
• Inter-domain LSP connectivity provided by labeled BGP [RFC3107] (used for MPLS label distribution for inter-domain LSP FECs) and relying on IS-IS and LDP for intra-domain LSP connectivity between the LSR labeled BGP speakers (AGNs and ABRs). Note that the MPLS core notes are not carrying the labeled BGP routes.

The aggregation and core MPLS domains are mapped to IS-IS areas as follows: Aggregation domains are mapped to IS-IS L1 areas. The core is configured as IS-IS L2. The border routers connecting aggregation and core are IS-IS L1L2 and are referred to as ABRs. From a technical and operational point of view these ABRs are part of the core, although they also belong to the respective aggregation domain purely from a routing protocol point of view.

5.1.4. Intra-Area Routing

5.1.4.1. Core

The core uses ISIS L2 to distribute routing information for the loopback addresses of all core nodes. The border routers (ABR) that connect to the aggregation domains are also part of the respective aggregation ISIS L1 area and hence ISIS L1L2.

LDP is used to distribute MPLS label binding information for the loopback addresses of all core nodes.

5.1.4.2. Aggregation

The aggregation domains uses ISIS L1 as intra-domain routing protocol. All AGN loopback addresses are carried in ISIS.

As in the core, the aggregation also uses LDP to distribute MPLS label bindings for the loopback addresses.

5.1.5. Access

Access nodes do not have their own domain or IGP area. Instead, they directly connect to the AGN1 nodes in the aggregation domain. To keep access devices as simple as possible, ANs do not participate in ISIS.

Instead, each AN has two static default routes pointing to each of the AGN1 it is connected to. Appropriate techniques SHOULD be deployed to make sure that a given default route is invalidated when the link to an AGN1 or that node itself fails. Examples of such techniques include monitoring the pysical link state for loss of light/loss of frame, or using Ethernet link OAM or BFD [RFC5881].

The AGN1 MUST have a configured static route to the loopback address of each of the ANs it is connected to, because it cannot learn the AN loopback address in any other way. These static routes have to be monitored and invalidated if necessary using the same techniques as described above for the static default routes on the AN.

The AGN1 redistributes these routes into ISIS for intra-domain reachability of all AN loopback addresses.

LDP is used for MPLS label distribution between AGN1 and AN. In order to keep the AN control plane as lightweight as possible, and to avoid the necessity for the AN to store 100.000 MPLS label bindings for each upstream AGN1 peer, LDP is deployed in downstream-on-demand (DoD) mode, described below.

To allow the label bindings received via LDP DoD to be installed into the LFIB on the AN without having the specific host route to the destination loopback address, but only a default route, use of the LDP Extension for Inter-Area Label Switched Paths [RFC5283] is made.

5.1.5.1. LDP Downstream-on-Demand (DoD)

LDP downstream-on-demand mode is specified in [RFC5036]. In this mode the upstream LSR will explicitly ask the downstream LSR for a label binding for a particular FEC when needed.

The assumption is that a given AN will only have a limited number of services configured to an even more limited number of destinations, or egress LER. Instead of learning and storing all label bindings for all possible loopback addresses within the entire Seamless MPLS network, the AN will use LDP DoD to only request the label bindings for the FECs corresponding to the loopback addresses of those egress nodes to which it has services configured.

More detailed description of LDP DoD use cases for MPLS access and list of required LDP DoD procedures in the context of Seamless MPLS design is included in [I-D.ietf-mpls-ldp-dod].

5.1.6. Inter-Area Routing

The inter-domain MPLS connectivity from the aggregation domains to and across the core domain is realized primarily using BGP with MPLS labels ("labled BGP/SAFI4" [RFC3107]). A very limited amount of route leaking from ISIS L2 into L1 is also used.

All ABR and PE nodes in the core are part of the labeled iBGP mesh, which can be either full mesh or based on route reflectors. These nodes advertise their respective loopback addresses (which are also carried in ISIS L2) into labeled BGP.

Each ABR node has labeled iBGP sessions with all AGN1 nodes inside the aggregation domain that they connect to the core. Since there are two ABR nodes per aggregation domain, this leads to each AGN1 node having an iBGP sessions with each of the two ABR. Note that the use of iBGP implies that the entire seamless MPLS internetwork is just a single AS to which all core and aggregation nodes belong. The AGN1 nodes advertise their own loopback addresses into labeled BGP, in addition to these loopbacks also being in ISIS L1.

Additionally the AGN1 nodes also redistribute all the statically configured routes to the AN loopback addresses into labeled BGP. Note that as stated obove, the AGN1 MUST ask the AN for label bindings for the AN loopback FECs via LDP DoD in order to have a valid labeled route with a non-null label.

This architecture results in carrying all loopbacks of all nodes except pure P nodes (AN, AGN, ABR and core PE) in labeled BGP, e.g. there will be in the order of 100.000 routes in labeled BGP when approaching the stated scalability goal. Note that this only affects the BGP RIB size and does not necessarily imply that any node needs to actually have active forwarding state (LFIB) in the same order of magnitude. In fact, as will be discussed in the scalability analysis, no single node needs to install all labeled BGP routes into the LFIB, but each node only needs a small percentage of the RIB as active forwarding state in the LFIB. And from a RIB point of view, BGP is known to scale to hundreds of thousands of routes.

5.1.7. Labeled iBGP next-hop handling

The ABR nodes run labeled iBGP both to the core mesh as well as to the AGN1 nodes of their respective aggregation domains. Therefore they operate as iBGP route reflectors, reflecting labeled routes from the aggregation into the core and vice versa.

When reflecting routes from the core into the aggregation domain, the ABR SHOULD NOT change the BGP NEXT-HOP addresses (next-hop-unchanged). This is the usual behaviour for iBGP route reflection. In order to make these routes resolvable to the AGN1 nodes inside the aggregation domain, the ABR MUST leak all other ABR and core PE loopback addresses from ISIS L2 into ISIS L1 of the aggregation domain. Note that the number of leaked addresses is limited so that the overall scalability of the seamless MPLS architecture is not impacted. In the worst case all core loopback addresses COULD be leaked into ISIS L1, but even that would not be a scalability problem.

When reflecting routes from the aggregation into the core, the ABR MUST set then BGP NEXT-HOP to its own loopback addresses (next-hop-self). This is not the default behaviour for iBGP route reflection, but requires special configuration on the ABR. Note that this also implies that the ABR MUST allocate a new local MPLS label for each labeled iBGP FEC that it reflects from the aggregation into the core. This special next-hop handling is essential for the scalability of the overall seamless MPLS architecture since it creates the required hierarchy and enables the hiding of all aggregation and access addresses behind the ABRs from an IGP point of view. Leaking of aggregation ISIS L1 loopback addresses into ISIS L2 is not necessary and MUST NOT be allowed.

The resulting hierarchical inter-domain MPLS routing structure is similar to the one described in [RFC4364] section 10c, only that we use one AS with route reflection instead of using multiple ASes.

5.1.8. Network Availability

The seamless mpls architecture guarantees a sub-second loss of connectivity upon any link or node failures. Furthermore, in the vast majority of cases, the loss of connectivity is limited to sub-50msec.

These network availability properties are provided without any degradation on scale and simplicity. This is a key achievement of the design.

In the remainder of this section, we first introduce the different network availability technologies and then review their applicability for each possible failure scenario.

5.1.8.1. IGP Convergence

IGP convergence can be modelled as a linear process with an initial delay and a linear FIB update [ACM01].

The initial delay could conservatively be assumed to be 260msec: 50msec to detect failures with BFD (most failures would be detected faster with loss of light for example or with faster BFD timers), 50msec to throttle the LSP generation, 150msec to throttle the SPF computation (making sure than all the required LSP’s are received even in case of SRLG failures) and 10msec for shortest-path-first tree computation.

Assuming 250usec per update (conservative), this allows for (1000-260)/0.250= 2960 prefixes update within a second following the outage. More precisely, this allows for 2960 important IGP prefixes updates. Important prefixes are automatically classified by the router implementation through simple heuristic (/32 is more important than non-/32).

The number of IGP important routes (loopbacks) in deployment case study 1 is much smaller than 2960, and hence sub-second IGP convergence is conservative.

IGP convergence is a simple technology for the operator provided that the router vendor optimizes the default IGP behavior (no need to tune any arcane knob).

5.1.8.2. Per-Prefix LFA FRR

A per-prefix LFA for a destination D is a precomputed backup IGP nexthop for that destination. This backup IGP nexthop can be link protecting or node protecting [RFC5286].

The analysis of the applicability of Per-Prefix LFA in the deployment model 1 of Seamless MPLS architecture is straightforward thanks to [RFC6571].

In deployment model 1, each aggregation network either follows the triangle or full-mesh topology. Further more, the backbone region implements a dual-plane. As a consequence, the failure of any link or node within an aggregation domain is protected by LFA FRR (sub-50msec) for all impacted IGP prefixes, whether intra-area or inter-area. No uloop may form as a result of these failures [RFC6571].

Per-Prefix LFA FRR is generally assessed as a simple technology for the operator [RFC6571]. It certainly is in the context of deployment case study 1 as the designer enforced triangle and full-mesh topologies in the aggregation network as well as a dual-plane core network.

5.1.8.3. Hierarchical Dataplane and BGP Prefix Independent Convergence

In a hierarchical dataplane, the FIB used by the packet processing engine reflects recursions between the routes. For example, a BGP route B recursing on IGP route I whose best path is via interface O is encoded as a hierarchy of FIB entry B pointing to a FIB entry I pointing to a FIB entry 0.

BGP Prefix Independent Convergence [BGP-PIC] extends the hierarchical dataplane with the concept of a BGP Path-List. A BGP path-list may be abstracted as a set of primary multipath nhops and a backup nhop. When the primary set is empty, packets destined to the BGP destinations are rerouted via the backup nhop.

For complete description of BGP-PIC technology and its applicability please refer to [BGP-PIC] and [ABR-FRR].

Hierarchical data plane and BGP-PIC are very simple technologies to operate. Their applicability to any topology, any routing policy and any BGP unicast address family allows router vendors to enable this behavior by default.

5.1.8.4. BGP Egress Node FRR

BGP egress node FRR is a Fast ReRoute solution and hence relies on local protection and the precomputation and preinstallation of the backup path in the FIB. BGP egress node FRR relies on a transit LSR ( Point of Local Repair, PLR ) adjacent to the failed protected BGP router to detect the failure and re-route the traffic to the backup BGP router. Number of BGP egress node FRR schemes are being investigated: [PE-FRR], [ABR-FRR], [I-D.minto-2547-egress-node-fast-protection], [I-D.bashandy-bgp-edge-node-frr], [I-D.bashandy-idr-bgp-repair-label], [I-D.bashandy-mpls-ldp-bgp-frr], [I-D.bashandy-bgp-frr-mirror-table], [I-D.bashandy-bgp-frr-vector-label], [I-D.bashandy-isis-bgp-edge-node-frr].

Differences between these schemes relate to the way backup and protected BGP routers get associated, how the protected router's BGP state is signalled to the backup BGP router(s) and if any other state is required on protected, backup and PLR routers. The schemes also differ in compatibility with IP-FRR and TE-FRR schemes to enable PLR to switch traffic towards the backup BGP router in case of protected BGP router failure.

In the Seamless MPLS design, BGP egress node FRR schemes can protect against the failures of PE, AGN and ABR nodes with no requirements on ingress routers.

5.1.8.5. Assessing loss of connectivity upon any failure

We select two typical traffic flows and analyze the loss of connectivity (LoC) upon each possible failure in the Seamless MPLS design in the deployment scenario #1.

• Flow F1 starts from an AN1 in a left aggregation region and ends on an AN2 in a right aggregation region. Each AN is dual-homed to two AGN’s.
• Flow F2 starts from a CE1 homed on L3VPN PE1 connected to the core LSRs and ends at CE2 dual-homed to L3VPN PE2 and PE3, both connected to the core LSRs.

Note that due to the symmetric network topology in case study 1, uni-directional flows F1' and F2', associated with F1 and F2 and forwarded in the reversed direction (AN2 to AN1 right-to-left and PE2 to PE1, respectively), take advantage of the same failure restoration mechanisms as F1 and F2.

5.1.8.5.1. AN1-AGN link failure or AGN node failure

F1 is impacted but LoC <50msec is possible assuming fast BFD detection and fast-switchover implementation on the AN. F2 is not impacted.

5.1.8.5.2. Link or node failure within the left aggregation region

F1 is impacted but LoC <50msec thanks to LFA FRR. No uloop will occur during the IGP convergence following the LFA protection. Note: if LFA is not available (other topology then case study one) or if LFA is not enabled, then the LoC would be < second as the number of impacted important IGP route in a seamless architecture is much smaller than 2960.

F2 is not impacted.

5.1.8.5.3. ABR node failure between left region and the core

F1 is impacted but LoC <50msec thanks to LFA FRR. No uloop will occur during the IGP convergence following the LFA protection.

Note: This case is also called “Local ABR failure” as the ABR which fails is the one connected to the aggregation region at the source of flow F1.

Note: remember that the left region receives the routes to all the remote ABR’s and that the labelled BGP routes are reflected from the core to the left region with next-hop unchanged. This ensures that the loss of the (local) ABR between the left region and the core is seen as an IGP route impact and hence can be addressed by LFA.

Note: if LFA is not available (other topology then case study one) or if LFA is not enabled, then the LoC would be < second as the number of impacted important IGP routes in a seamless architecture is much smaller than 2960 routes.

F2 is not impacted.

5.1.8.5.4. Link or node failure within the core region

F1 and F2 are impacted but LoC <50msec thanks to LFA FRR.

This is specific to the particular core topology used in deployment case study 1. The core topology has been optimized [RFC6571] for LFA applicability.

As explained in [RFC6571], another alternative to provide <50msec in this case consists in using an MPLS-TE full-mesh and MPLS-TE FRR. This is required when the designer is not able or does not want to optimize the topology for LFA applicability and he wants to achieve <50msec protection.

Alternatively, simple IGP convergence would ensure a LoC < second as the number of impacted important IGP routes in a seamless architecture is much smaller than 2960 routes.

5.1.8.5.5. PE2 failure

F1 is not impacted.

F2 is impacted and the LoC is sub-300msec thanks to IGP convergence and BGP PIC.

The detection of the primary nhop failure (PE2 down) is performed by a single-area IGP convergence.

In this specific case, the convergence should be much faster than <sec as very few prefixes are impacted upon an edge node failure. Reusing the introduction on IGP convergence presented in an earlier section and assuming 2 important impacted prefixes (two loopbacks per edge node), one would expect that PE2’s failure is detected in 260msec + 2*0.250msec.

If BGP PIC is used on the ingress PE ( PE1 ) then the LoC is the same as for IGP convergence. The LoC for BGP/L3VPN traffic upon PE2 failure is thus expected to be <300msec.

Provided that all the deployment considerations have been met, LoC is sub-50msec with BGP egress node FRR.

5.1.8.5.6. PE2’s PE-CE link failure

F1 is not impacted.

F2 is impacted and the LoC is sub-50msec thanks to local interface failure detection and local forwarding to the backup PE. Forwarding to the backup PE is achieved with hierarchical data plane and local-repair of BGP egress link providing fast re-route to the backup BGP nhop PE.

5.1.8.5.7. ABR node failure between right region and the core

F2 is not impacted.

F1 is impacted. We analyze the LoC for F1 for both BGP PIC and BGP egress node FRR.

LoC is sub-600msec thanks to BGP PIC.

The detection of the primary nhop failure (ABR down) is performed by a multi-area IGP convergence.

First, the two (local) ABR’s between the left and core regions must complete the core IGP convergence. The analysis is similar to the loss of PE2. We would thus expect that the core convergence completes in ~260msec.

Second, the IGP convergence in the left region will cause all AGN1 routers to detect the loss of the remote ABR. This second IGP convergence is very similar to the first one (2 important prefixes to remove) and hence should also complete in ~260msec.

Once an AGN1 has detected the loss of the remote ABR, thanks to the BGP PIC, in-place modification of shared BGP path-list and pre-computation of BGP backup nhop, the AGN1 reroutes flow F1 via the alternate remote ABR in a few msec’s [##BGP-PIC].

As a consequence, the LoC for F1 upon remote ABR failure is thus expected to be <600msec.

Provided that all the deployment considerations have been met, LoC is sub-50msec with BGP egress node FRR.

5.1.8.5.8. Link or node failure within the right aggregation region

F1 is impacted but LoC <50msec thanks to LFA FRR. No uloop will occur during the IGP convergence following the LFA protection.

Note: if LFA is not available (other topology then case study one) or if LFA is not enabled, then the LoC would be < second as the number of impacted important IGP route in a seamless architecture is much smaller than 2960.

F2 is not impacted.

5.1.8.5.9. AGN (connected to AN2) node failure

F1 is impacted but LoC <50msec thanks to LFA FRR. No uloop will occur during the IGP convergence following the LFA protection.

Note: remember that AGN redistributes the static routes to ANs within ISIS. The loss of an AGN on the IGP path to AN2 is thus seen as an IGP route impact and hence LFA FRR is applicable.

Note: if LFA is not available (other topology then case study one) or if LFA is not enabled, then the LoC would be < second as the number of impacted important IGP route in a seamless architecture is much smaller than 2960.

F2 is not impacted.

5.1.8.5.10. AGN-AN2 link failure

F2 is not impacted.

F1 is impacted.

LoC is sub-300msec with IGP convergence as only one prefix needs to be updated.

Sub-50msec could be guaranteed provided that the LFA implementation supports a redistributed static as a native IGP route.

5.1.8.5.11. AN2 failure

F1 is impacted and the LoC lasts until the AN is recovered.

F2 is not impacted.

5.1.8.5.12. Summary - Loss of connectivity upon any failure

The Seamless MPLS architecture illustrated in deployment case study 1 guarantees sub-50msec upon any link or node failures.

5.1.8.6. Network Resiliency and Simplicity

A fundamental aspect of the Seamless MPLS architecture is the requirement for operational simplicity.

In a network with 10k of IGP/BGP nodes and 100k of MPLS-enabled nodes, it is extremely important to provide a simple operational process.

LFA FRR plays a key role in providing simplicity as it is an automated behavior which does not require any configuration or interoperability testing.

More specifically, [RFC6571] plays a key role in the Seamless MPLS architecture as it describes simple design guidelines which determiniscally ensure LFA coverage for any link and node in the aggregation regions of the network. This is key as it provides for a simple <50msec protection for the vast majority of the node and link failures (>90% of the IGP/BGP3107 footprint at least).

If the guidelines cannot be met, then either the designer will rely on (1) augmenting native LFA coverage with remote LFA [I-D.ietf-rtgwg-remote-lfa], or (2) augmenting native LFA coverage with RSVP, or (3) a full-mesh TE FRR model, or (4) IGP convergence. The first option provides an automatic and fairly simple sub-50msec protection as LFA without introducing any additional protocols. The second option provides the same sub-50msec protection as LFA, but introduces additional RSVP LSPs. The thrid option optimizes for sub-50msec protection, but implies a more complex operational model. The fourth option optimizes for simple operation but only provides <1 sec protection. Up to each designer to arbitrate between these three options versus the possibility to engineer the topology for native LFA protection.

A similar choice involves protection against ABR node failure and L3VPN PE node failure. The designer can either use BGP PIC or BGP egress node FRR. Up to each designer to asssess the trade-off between the valuation of sub-50msec instead of sub-1sec versus additional operational considerations related to BGP egress node FRR.

5.1.8.7. Conclusion

The Seamless MPLS architecture illustrated in deployment case study 1 guarantees sub-50msec for majority of link and node failures by using LFA FRR, except ABR and L3PE node failures, and PE-CE link failure.

L3VPN PE-CE link failure can be protected with sub-50msec restoration, by using hierarchical data plane and local-repair fast-reroute to the backup BGP nhop PE.

ABR and L3PE node failure can be protected with sub-50msec restoration, by using BGP egress node FRR.

Alternatively, ABR and L3PE node failure can be protected with sub-1sec restoration using BGP PIC.

5.1.9. BGP Next-Hop Redundancy

An aggregation domain is connected to the core network using two redundant area boarder routers, and MPLS hierarchy is applied on these ABRs. MPLS hierarchy helps scale the FIB but introduces additional complexity for the rerouting in case of ABR failure. Indeed ABR failure requires a BGP converge to update the inner MPLS hierarchy, in addition to the IGP converge to update the outer MPLS hierarchy. This is also expected to take more time as BGP convergence is performed after the IGP convergence and because the number of prefixes to update in the FIB can be significant. This is clearly a drawback, but the architecture allows for two "local protection" solutions which restore the traffic before the BGP convergence takes place.

BGP PIC would be required on all edge LSR involved in the inner (BGP) MPLS hierarchy. Namely all routers except the AN which are not involved in the inner MPLS hierarchy. It involves pre-computing and pre-installing in the FIB the BGP backup path. Such back up path are activated when the IGP advertise the failure of the primary path. For specification see [BGP-PIC1, 2##].

BGP egress node FRR would be required on the egress LSR involved in the inner (BGP) MPLS hierarchy, namely AGN, ABR and L3VPN PEs. For specification see [PE-FRR], [ABR-FRR], [BGP-edge-FRR##].

Both approaches have their pros and cons, and the choice is left to each Service Provider or deployment based on the different requirements. The key point is that the seamless MPLS architecture can handle fast restoration time, even for ABR failures.

5.2. Scalability Analysis

5.2.1. Control and Data Plane State for Deployment Scenario #1

5.2.1.1. Introduction

Let's call:

• #AN the number of Access Node (AN) in the seamless MPLS domain
• #AGN the number of AGgregation Node (AGN) in the seamless MPLS domain
• #Core the number of Core (Core) in the core network
• #Area the number of aggregation routing domains.

Let's take the following assumptions:

• Aggregation equipments are equally spread across aggregation routing domains
• the number of IGP links is three times the number of IGP nodes
• the number of IGP prefixes is five times the number of IGP nodes (links prefixes + 2 loopbacks)
• Each Access Node needs to have up to 1,000 (1k) LSPs. This is driven by the expected AN access line capacity, and a sum of LSPs required for connectivity to PE routers providing edge services as well as a remote ANs. 100 LSPs per AN (10% of total) are FECs which are outside of their routing domain. Those 100 remote FEC are the same for all Access Nodes of a given AGN.

The following sections roughly evaluate the scalability, both in absolute numbers and relatively with the number of Access Node which is the biggest scalability factor.

5.2.1.2. Core Domain

The IGP & LDP core domain are not affected by the number of access nodes:

IGP:

• node : #Core ~ o(1)
• links : 3*#Core ~ o(1)
• IP prefixes : 5*#Core ~ o(1)

LDP FEC:

• #Core ~ o(1)

Core TN FIBs grows linearly with the number of node in the core domain. In other word, they are not affected by AGN and AN nodes:

Core TN:

• IP FIB : 5*#Core ~ o(1)
• MPLS ILM (LFIB) : #Core ~ o(1)

BGP carries all AN routes which is significant. However, all AN routes are only needed in the control plane, possibly in a dedicated BGP Route Reflector (just like for BGP/MPLS VPNs) and not in the forwarding plane. The number of routes (100k) is smaller than the number of number of routes in the Internet (300k and rising) or in major VPN SP (>500k and rising) so the target can be handled with current implementations. In addition, AN routes are internal routes whose churn and instability is smaller and more under control than external routes.

BGP Route Reflector (RR)

• NLRI : #AN ~ o(n)
• path : 2*#AN ~ o(2n)

ABR handles both the core and aggregations routes. They do not depend on the total number of AN nodes, but only on the number of AN in their aggregation domain.

ABR:

• IP FIB : 5*#Core + (5*#AGN + #AN) / #Area ~ o(#AN /#Area)
• MPLS ILM (LFIB) : #Core + (#AGN + #AN) / #Area ~ o(#AN / #Area)

5.2.1.3. Aggregation Domain

In the aggregation domain, IGP & LDP are not affected by the number of access nodes outside of their domain. They are not affected by the total number of AN nodes:

IGP:

• node : #AGN / #Area ~ o(1)
• links : 3*#AGN / #Area ~ o(1)
• IP prefixes : #Core + #Area + (5*#AGN + #AN) / #Area ~ o(#AN *5/ #Area)
• • + 1 loopback per core node + one aggregate per area + 5 prefixes per AGN in the area + 1 prefix per AN in the area.

LDP FEC:

• Core + (#AGN + #AN) / #Area ~ o(#AN / #Area)
• • + 1 loopback per core node + 1 loopback per AGN & AN node in the area.

AGN FIBs grows with the number of node in the core area, in their aggregation area, plus the number of inter domain LSP required by the AN attached to them. They do not depend on the total number of AN nodes. In the BGP control plane, AGN also needs to handle all the AN routes.

AGN:

• IP FIB : #Core + #Area + (5*#AGN + #AN) / #Area ~ o(#AN *5/ #Area)
• MPLS ILM (LFIB) : #Core + (#AGN + #AN) / #Area + 100 ~ o(#AN / #Area)

AN FIBs grows with its connectivity requirement. They do not depend on the number of AN, AGN, SN or any others nodes.

AN:

• IP RIB : 1 ~ o(1)
• MPLS LIB : 1k ~ o(1)
• IP FIB : 1 ~ o(1)
• MPLS ILM (LFIB) : 1k ~ o(1)

5.2.1.4. Summary

AN requirements are kept to a minimum. BGP is not required on ANs and the size of their FIB is driven only by their own connectivity requirements. In the FIB scale analysis described in sections 5.2.1.x, it was assumed that any single AN will need no more than 1,000 LSPs. This assumption is based on the expected AN access line capacity and LSPs required for connectivity to PE routers providing edge services as well as a sparse mesh of connectivity between ANs.

In the core area, IGP and LDP are not affected by the nodes in the aggregation domains. In particular they do not grow with the number of AGNs or ANs.

In the aggregation areas, IGP and LDP are affected by the number of core nodes and the number of AGNs and ANs in their area. They are not affected by the total number of AGNs or ANs in the seamless MPLS domain.

No FIB of any node is required to handle the total number of AGNs or ANs in the Seamless MPLS domain. In other words, the number of AGNs and ANs in the Seamless MPLS domain is not a limiting factor, and the design can be scaled by growing the number of areas. The main limitation is the MPLS connectivity requirements on the AN, i.e. mainly the number of LSP needed per AN. Another limitation may be the number of different LSPs required by ANs attached to specific AGN. However, given the foreseen deployments and current AGN capabilities, this is not expected to be a limiting factor.

In the control plane, BGP will typically handle all AN routes. This is expected to be substantial, but again the target deployment scale are well within the capabilities of current equipment . In addition, if required, additional techniques could be used to improve the scalability, based on the experience gained with scaling BGP/MPLS VPN (e.g. route partitioning between RR planes, route filtering (static or dynamic with ORF or route refresh) between ANs and on AGN to improve AGN scalability.

5.2.1.5. Numerical application for use case #1

As a recap, targets for deployment scenario 1 are:

• Number of Aggregation Domains 100
• Number of Backbone Nodes 1.000
• Number of AGgregation Nodes 10.000
• Number of Access Nodes 100.000

This gives the following scaling numbers for each category of nodes:

• AN IP FIB 1
• AN MPLS ILM (LFIB) 1 000
• AGN IP FIB 2 600
• AGN MPLS ILM (LFIB) 2 200
• ABR IP FIB 7 600
• ABR MPLS ILM (LFIB) 2 100
• TN IP FIB 5 000
• TN MPLS ILM (LFIB) 1 000
• RR BGP NLRI 100 000
• RR BGP paths 200 000

5.2.1.6. Numerical application for use case #2

As a recap, targets for deployment scenario 1 are:

• Number of Aggregation Domains 30
• Number of Backbone Nodes 150
• Number of AGgregation Nodes 1.500
• Number of Access Nodes 40.000

This gives the following scaling numbers for each category of nodes:

• AN IP FIB 1
• AN MPLS ILM (LFIB) 1 000
• AGN IP FIB 1 700
• AGN MPLS ILM (LFIB) 1 800
• ABR IP FIB 3 700
• ABR MPLS ILM (LFIB) 1 600
• TN IP FIB 750
• TN MPLS ILM (LFIB) 150
• RR BGP NLRI 40 000
• RR BGP paths 80 000

6. Acknowledgements

Many people contributed to this document. The authors would like to thank Wim Henderickx, Robert Raszuk, Thomas Beckhaus, Wilfried Maas, Roger Wenner, Kireeti Kompella, Yakov Rekhter, Mark Tinka and Simon DeLord for their suggestions and review.

7. IANA Considerations

This memo does not include any requests to IANA.

8. Security Considerations

The Seamless MPLS Architecture is subject to similar security threats as any MPLS LDP deployment. It is recommended that baseline security measures are considered as described in Security Framework for MPLS and GMPLS networks [RFC5920], in the LDP specification RFC5036 [RFC5036] and in [I-D.ietf-karp-routing-tcp-analysis]including ensuring authenticity and integrity of LDP messages, as well as protection against spoofing and Denial of Service attacks. Some deployments may require increased measures of network security if a subset of Access Nodes are placed in locations with lower levels of physical security e.g. street cabinets ( common practice for VDSL access ). In such cases it is the responsibility of the system designer to take into account the physical security measures ( environmental design, mechanical or electronic access control, intrusion detection ), as well as monitoring and auditing measures (configuration and Operating System changes, reloads, routes advertisements ).

Security aspects specific to the MPLS access network based on LDP DoD in the context of Seamless MPLS design are described in the security section of [I-D.ietf-mpls-ldp-dod].

9. References

9.1. Normative References

[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

9.2. Informative References

, ", ", ", ", ", ", ", "

[RFC5881]	Katz, D. and D. Ward, "Bidirectional Forwarding Detection (BFD) for IPv4 and IPv6 (Single Hop)", RFC 5881, June 2010.
[I-D.bashandy-bgp-edge-node-frr]	Bashandy, A., Pithawala, B. and K. Patel, "Scalable BGP FRR Protection against Edge Node Failure", Internet-Draft draft-bashandy-bgp-edge-node-frr-03, July 2012.
[I-D.bashandy-bgp-frr-mirror-table]	Bashandy, A., Konstantynowicz, M. and N. Kumar, "BGP FRR Protection against Edge Node Failure Using Table Mirroring with Context Labels", Internet-Draft draft-bashandy-bgp-frr-mirror-table-00, October 2012.
[I-D.bashandy-bgp-frr-vector-label]	Bashandy, A., Kumar, N. and M. Konstantynowicz, "BGP FRR Protection against Edge Node Failure Using Vector Labels", Internet-Draft draft-bashandy-bgp-frr-vector-label-00, July 2012.
[I-D.bashandy-idr-bgp-repair-label]	Bashandy, A., Pithawala, B. and J. Heitz, "Scalable, Loop-Free BGP FRR using Repair Label", Internet-Draft draft-bashandy-idr-bgp-repair-label-04, May 2012.
[I-D.bashandy-isis-bgp-edge-node-frr]	Bashandy, A., "IS-IS Extension for BGP FRR Protection against Edge Node Failure", Internet-Draft draft-bashandy-isis-bgp-edge-node-frr-01, September 2012.
[I-D.bashandy-mpls-ldp-bgp-frr]	Bashandy, A. and K. Raza, "LDP Extension for FRR Edge Node Protection in BGP-Free LDP Core", Internet-Draft draft-bashandy-mpls-ldp-bgp-frr-00, March 2012.
[I-D.minto-2547-egress-node-fast-protection]	Jeganathan, J., Gredler, H. and B. Decraene, "2547 egress PE Fast Failure Protection", Internet-Draft draft-minto-2547-egress-node-fast-protection-02, July 2013.
[RFC3107]	Rekhter, Y. and E. Rosen, "Carrying Label Information in BGP-4", RFC 3107, May 2001.
[RFC4364]	Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, February 2006.
[RFC5036]	Andersson, L., Minei, I. and B. Thomas, "LDP Specification", RFC 5036, October 2007.
[RFC5283]	Decraene, B., Le Roux, JL. and I. Minei, "LDP Extension for Inter-Area Label Switched Paths (LSPs)", RFC 5283, July 2008.
[RFC5286]	Atlas, A. and A. Zinin, "Basic Specification for IP Fast Reroute: Loop-Free Alternates", RFC 5286, September 2008.
[RFC7032]	", .
[I-D.ietf-mpls-ldp-dod]	Beckhaus, T., Decraene, B., Tiruveedhula, K., Konstantynowicz, M. and L. Martini, "LDP Downstream-on-Demand in Seamless MPLS", Internet-Draft draft-ietf-mpls-ldp-dod-07, May 2013.
[I-D.ietf-rtgwg-bgp-pic]	", .
[I-D.ietf-rtgwg-remote-lfa]	", .
[PE-FRR]	Le Roux, J.L., Decraene, B. and Z. Ahmad, "Fast Reroute in MPLS L3VPN Networks - Towards CE-to-CE Protection, MPLS 2006 Conference", .
[ABR-FRR]	Rekhter, Y., "Local Protection for LSP tail-end node failure, MPLS World Congress 2009", .
[ACM01]	Archieving sub-second IGP convergence in large IP networks, ACM SIGCOMM Computer Communication Review, v.35 n.3", July 2005.
[BGP-PIC]	BGP PIC, Technical Report", November 2007.
[RFC6571]	", .
[I-D.ietf-karp-routing-tcp-analysis]	", .
[RFC5920]	", .

Authors' Addresses