Internet DRAFT - draft-eckert-msr6-problem-statement
draft-eckert-msr6-problem-statement
MSR6 T. Eckert, Ed.
Internet-Draft Futurewei Technologies USA
Intended status: Informational 24 October 2022
Expires: 27 April 2023
Yet another Problem Statement for IPv6 Multicast Source Routing (MSR6)
draft-eckert-msr6-problem-statement-00
Abstract
This draft summarizes additional personal opinions of the author for
why existing stateless multicast solutions BIER/BIER-TE are not
sufficient to support the operator, architecture and use-case goals
that MSR6 proposes to solve.
This document is complementary to problems outlined in I-D.liu-msr6-
problem-statement and in no way affects any of them. Instead, it
attempts to look more at lower-layer functional challenges of current
stateless source routing multicast solutions (BIER/BIER-TE), as well
as architectural, network and protocol design ecosystem requirements
of operators.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 27 April 2023.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Eckert Expires 27 April 2023 [Page 1]
Internet-Draft msr6-problem-statement-eckert October 2022
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. P1: BIER: Efficiency of delivery for sparse groups in large
networks . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2. P2: BIER-TE: Packet header efficiency of delivery to sparse
groups in large networks . . . . . . . . . . . . . . . . 5
2.3. P3: BIER/BIER-TE: Efficient amount of per-router state
(BIFT) . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.4. P4: More complex, multi-protocol host stack requirements
with BIER . . . . . . . . . . . . . . . . . . . . . . . . 5
2.5. P5: Support for single, IPv6/RFC8200 compliant network
forwarding plane . . . . . . . . . . . . . . . . . . . . 5
2.6. P6: No Support for SRv6 architecture . . . . . . . . . . 5
2.7. P7: Inability to perform per-hop IPv6 forwarding plane
features . . . . . . . . . . . . . . . . . . . . . . . . 6
2.8. P8: Routing convergence speed and complexity . . . . . . 6
3. Explanations . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. E1: BIER: Efficiency of delivery to sparse groups in large
networks . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2. E2: BIER-TE: Packet header efficiency of delivery to sparse
groups in large networks . . . . . . . . . . . . . . . . 7
3.3. E3: BIER/BIER-TE: Efficient amount of per-router state
(BIFT) . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4. E4: More complex, multi-protocol host stack requirements
with BIER . . . . . . . . . . . . . . . . . . . . . . . . 7
3.5. E5: Support for single, IPv6/RFC8200 compliant network
forwarding plane . . . . . . . . . . . . . . . . . . . . 7
3.5.1. Example 1: IPv4 multicast in MPLS/VPN solutions vs.
native MPLS multicast . . . . . . . . . . . . . . . . 8
3.5.2. Example 2: Native IPv6 multicast vs. IPv4
multicast . . . . . . . . . . . . . . . . . . . . . . 9
3.5.3. Example 3: BIERin6 . . . . . . . . . . . . . . . . . 9
3.5.4. Example 4: Routing for multicast . . . . . . . . . . 10
3.6. E6: No Support for SRv6 architecture . . . . . . . . . . 10
3.7. E7: Inability to perform per-hop IPv6 forwarding plane
features . . . . . . . . . . . . . . . . . . . . . . . . 11
3.8. E8: Routing convergence speed and complexity in large
networks . . . . . . . . . . . . . . . . . . . . . . . . 12
4. Candidate MSR6 solution discussion . . . . . . . . . . . . . 12
Eckert Expires 27 April 2023 [Page 2]
Internet-Draft msr6-problem-statement-eckert October 2022
4.1. S1: BIER: Efficiency of delivery for sparse groups in large
networks . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2. S2: BIER-TE: Efficiency of delivery for sparse groups in
large networks . . . . . . . . . . . . . . . . . . . . . 13
4.3. S3: BIER/BIER-TE: Large amount of per-BFR state (BIFT) . 13
4.4. S4: More complex, multi-protocol host stack requirements
with BIER . . . . . . . . . . . . . . . . . . . . . . . . 13
4.5. S5: Support for single, IPv6/RFC8200 compliant network
forwarding plane . . . . . . . . . . . . . . . . . . . . 13
4.6. S6: No Support for SRv6 architecture . . . . . . . . . . 14
4.7. S7: Inability to perform per-hop IPv6 forwarding plane
features . . . . . . . . . . . . . . . . . . . . . . . . 14
4.8. S8: Routing convergence speed and complexity . . . . . . 15
5. Open issues . . . . . . . . . . . . . . . . . . . . . . . . . 15
6. Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . 15
7. Informative References . . . . . . . . . . . . . . . . . . . 15
Appendix A. Other considerations towards MSR6 . . . . . . . . . 17
A.1. Design separation between tree building and other
functions . . . . . . . . . . . . . . . . . . . . . . . . 17
A.2. Reuse of BIER principles and technology components as much
as possible . . . . . . . . . . . . . . . . . . . . . . . 18
A.3. MSR6 as experimental (not standards track) work . . . . . 19
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 19
1. Introduction
This draft summarizes additional personal opinions of the author for
why existing stateless multicast solutions BIER/BIER-TE are not
sufficient to support the operator, architecture and use-case goals
that MSR6 proposes to solve.
This document is complementary to problems outlined in
[I-D.liu-msr6-problem-statement] and in no way affects any of them.
Instead, it attempts to look more at lower-layer functional
challenges of current stateless source routing multicast solutions
(BIER/BIER-TE), as well as architectural, network and protocol design
ecosystem requirements of operators.
This draft is solely kept separate at this time, because the author
felt it would be impossible for him to merge the text without doing
disservice to that pre-existing draft due to time constraints and the
inability to have reviewed and agreed upon this documents amongst the
larger authorship of [I-D.liu-msr6-problem-statement].
This document is structured as follows.
Eckert Expires 27 April 2023 [Page 3]
Internet-Draft msr6-problem-statement-eckert October 2022
The Problems section (Section 2) summaries problems with existing
solutions and numbers them with Pi. The author found that it is very
difficult trying to find the right degree of detail, but hopes that
the presented list is a good compromise between too much detail and
to abstract.
The Explanations section (Section 3) provides for each problem
additional explanation to avoid that readers have to understand the
problems only after reading up on a multitude of RFC and/or having
similar deployment experiences.
On some points, the explanation section also presents what the author
considers to be evidence as to how the IETF has established case
precedents over the past decades how these problems have been solved
in (in the authors opinion) similar cases.
These prior cases have often been controversial, but in the opinion
of the author, they finally resulted in best serving the network
operators and their customers, even at the expense that this produced
more IETF standards than if some entity could have been able to
persuade all customers and operators that there is a single one-fits-
all approach to standards.
Even though the IESG did ask to only produce a problem statement, the
author found that even the problems where best understood by
participants of the IETF that he discussed them with if he also
provided an outline of the proposed solutions, instead of them having
try to guess how hey could be solved. This is what the Section 4
section discusses, and of course it is not meant to taint the
opinions of readers as to how any of the problem should be solved.
Explanations for problem Pi are numbered Ei, Solution considerations
for Pi are numbered Si.
Finally, the document lists open issues and ends with with an
appendix on other (Appendix A) considerations about how MSR6 might be
operated adjacent to BIER in the IETF and why creation of such a WG
is beneficial beyond the mere solving of a specific set of problems.
2. Problems
2.1. P1: BIER: Efficiency of delivery for sparse groups in large
networks
In networks with (a) a large number of BFER (e.g.: 60k) and (b) a
source-routing header size as favored by products (e.g. 256 bit),
BIER will need to send in a stochastic manner as many packets as
ingress replication (no BIER) instead only one packet.
Eckert Expires 27 April 2023 [Page 4]
Internet-Draft msr6-problem-statement-eckert October 2022
2.2. P2: BIER-TE: Packet header efficiency of delivery to sparse groups
in large networks
Problem Section 2.1 is larger in BIER-TE because significant number
of bits in BIER-TE bitstrings need to be allowed to steering of the
traffic ("tree engineering") making bitstrings effectively even
smaller compared to BIER, increasing this Bitstring fragmentation
problem.
2.3. P3: BIER/BIER-TE: Efficient amount of per-router state (BIFT)
BIER/BIER-TE requires in every BIER router forwarding state for every
egress router. In large networks, this can be as much as e.g.:
60,000 entries (or more) or maybe as much as twice as many with BIER-
TE. This is unacceptable for many networks with this number of end-
routers or hosts and mostly sparse multicast trees.
2.4. P4: More complex, multi-protocol host stack requirements with BIER
Controlled networks such as Data-Centers with VM to VM IP multicast,
or IoT networks do use IPv6 multicast. To use end-to-end stateless
multicast with BIER, they would need to implement in addition to IPv6
host stacks BIER BFIR/BFER stacks as well as any BIER specific
mechanisms for the routing underlay to use IPv6 as a flow overlay.
This duplication of host stacks is undesirable for cost of
development, operations code-space as well as hardware implementation
(Smart NICs etc.).
2.5. P5: Support for single, IPv6/RFC8200 compliant network forwarding
plane
BIER is a separate (layer 2.5?) forwarding plane but not an
extensions to IPv6/RFC8200. This is operationally undesirable for
IPv6-only networks.
2.6. P6: No Support for SRv6 architecture
BIER/BIER-TE including BIERin6 do not support the SRv6 architecture,
making it impossible for networks that utilize a common source
routing architecture across unicast and multicast to as easy as
possible share any applicable or future SRv6 functionality between
unicast and multicast and likely also simplify OAM and design.
Eckert Expires 27 April 2023 [Page 5]
Internet-Draft msr6-problem-statement-eckert October 2022
2.7. P7: Inability to perform per-hop IPv6 forwarding plane features
Operators depend on a wide variety of collateral forwarding plane
features, even hop-by-hop or on dedicated intermediate hops which are
not the ingress or egress router of the packet. Policies and/or
Telemetry of these features are tied to IPv6 source and/or
destination address. Important examples include, but are not limited
to:
1. IPfix and related services for traffic monitoring
2. Traffic filtering (including (S,G)
3. QoS ( (S,G) policies)
4. IPsec/ESP ((S,G) SA policies)
2.8. P8: Routing convergence speed and complexity
BIER/BIER-TE do not allow to build lowest outage, fast-converging
network services because of their reliance on a) IGP re-routing and
b) large BIFT forwarding tables.
3. Explanations
3.1. E1: BIER: Efficiency of delivery to sparse groups in large
networks
A single BIER packet with a 256 bitstring can only create copies to
the 256 BFER addressed via this bitstring. In a network with 60k
BFER, at least 235 different Bitstrings need to be defined (256 * 235
> 60k). A message to even a small number of BFER such as 10 could
statistically easily require to send 10 packets, because each of the
BFER is in a different bitstring.
Efficient replication to sparse multicast groups in large networks
was at the core of the original IP multicast protocol PIM Sparse Mode
in the early 1990th. The "Sparse" in this name exactly refers to
such a group with few members in a large network. It can thus be
seen as a day 1 requirement against any scalable IP multicast
protocol, stateful or stateless.
Eckert Expires 27 April 2023 [Page 6]
Internet-Draft msr6-problem-statement-eckert October 2022
3.2. E2: BIER-TE: Packet header efficiency of delivery to sparse groups
in large networks
Note that BIER-TE was designed only shortly after BIER and hence one
primary goal was to re-use as much as possible of the BIER forwarding
plane to make adoption as easy as possible. This excluded a lot of
the more forward looking options being proposed for MSR6 now
(Section 4.1 / Section 4.2).
3.3. E3: BIER/BIER-TE: Efficient amount of per-router state (BIFT)
With "flat bitstrings" as in BIER/BIER-TE each router needs a
forwarding entry similar to unicast forwarding entries. In large
networks, even enterprises, this is easy in the thousands (for edge
routers) and another possible factor ten when running multicast
source-routing end-to-end into hosts - which is feasible with newer
proposed technologies but not flat bitstrings of BIER/BIER-TE (see
Section 4.1 / Section 4.2
3.4. E4: More complex, multi-protocol host stack requirements with BIER
Introducing a new host stack for a new network protocol like BIER
without UDP tunneling requires most commonly OS upgrades. This is
not a feasible requirement for reasonable fast adoption in many
markets such as industrial, IoT or hosts in regulated environments
such as aviation, finance industry, defense and the like. See
Section 4.4 how MSR can avoid this issue.
Whether in the OS or application (when using UDP tunneling),
additional network stacks such as BIER typically incur more code-
space overhead than extensions to an existing stack. This can be an
issue for constrained code space hosts.
3.5. E5: Support for single, IPv6/RFC8200 compliant network forwarding
plane
The IETF has a long and successful history of extending different
type of unicast network designs with multicast support that best
matches and extends those underlying unicast mechanisms - even when
vendors initially promoted less well aligned multicast technologies
that was more readily available and solutions/workarounds could be
built to support initial use-cases of customers that way.
While specific multicast technical detail benefits of adopting to the
new unicast network design can be developed, this often only happens
as a result of deployment experience with the new network technology
and multicast later in the process. Instead, the most core reason
for the IETF to build such new technologies was the operator/market
Eckert Expires 27 April 2023 [Page 7]
Internet-Draft msr6-problem-statement-eckert October 2022
preference and the logic that multicast functions should be available
as a core architectural extension of any unicast network protocol
architecture. This is why IP Multicast was added to IPv6 and MPLS
and why MSR6 now wants to be chartered for adding source-routed IPv6
multicast based on expanding on RFC8200 principles.
Making multicast follow architectural frameworks of unicast and
expanding it has a large set of benefits for operators, and can best
be described as the overall ecosystem complexity of operating (OAM) a
network, but also in the often met expectation that it simplifies the
adoption of future extensions/enhancements across unicast and
multicast without having to re-define them, because the multicast
solution is built from a different architecture.
The following lists a few mayor instances where the author thinks
this principle can be observed. Of course it is always possible to
create a detailed technical difference between any prior examples and
the case for MSR6, but the author thinks that that is missing the
larger point of the requirement:
Operator preferences may or may not be justifiable on selective
detail technical points versus a pre-existing solution, but instead
the IETF has repeatedly shown that it serves operator preferences
when the proposed solutions are technically solid (working), require
interoperability, and have a sufficient critical mass of supporters
and active participants that can bring standardization to a
successful finish.
Existence of prior, competing but alternating solutions have never
been in the way of such additional technologies in the IETF, or else
IETF would help first-comers to establish monopolies over market
spaces where their solutions themselves may look sufficient but is
not preferred by customers for the variety of reasons mentioned here.
3.5.1. Example 1: IPv4 multicast in MPLS/VPN solutions vs. native MPLS
multicast
In the early 2000th, (large) service providers where increasingly
adopting MPLS (with IPv4) as their single network forwarding plane,
primarily to be able to introduce L3VPN services. The only form of
multicast that high-speed hardware forwarding could support in
available products back then was IPv4 multicast (no MPLS), which is
why vendors implemented and standardized in the IETF so-called
"draft-rosen" that provided Multicast VPN (MVPN) for MPLS networks
with L3VPN services.
Eckert Expires 27 April 2023 [Page 8]
Internet-Draft msr6-problem-statement-eckert October 2022
Nevertheless, operators that where successfully deploying this IPv4
multicast based MVPN service continued to ask vendors to give them a
"native MPLS" forwarding plane for their MVPN services. Some aspects
of that solution, such as RSVP-TE/P2MP did provide easy to understand
new functionality, such as traffic engineering, but the most commonly
now used MPLS multicast solution, mLDP, (multicast LDP) provides
significantly the same functionality that PIM can provide. Only the
encapsulation of packets in the forwarding plane is different, but
not the services that are supported across those two variants.
3.5.2. Example 2: Native IPv6 multicast vs. IPv4 multicast
When IPv6 was introduced, it was not even a question to include it in
the standardization of IPv6, even though in practice, most business
critical use cases where equally feasible by sticking to IPv4: Most
customers where using dual-plane networks and many of them continued
to use IPv4 multicast because neither the applications nor hardware
forwarding in routers did support IPv6 multicast.
The difference over today was, that at that time, no vendor stood up
and claimed IPv6 multicast should not be developed in the IETF
because IPv4 multicast was sufficient, and if necessary with end-to-
end tunneling of IPv6 multicast packets across IPv4 multicast it as
well as hop-by-hop tunneling of IPv4 multicast over IPv6 only hops.
But this is exactly what is done today with BIERin6:
3.5.3. Example 3: BIERin6
The BIER-WG is working on [I-D.ietf-bier-bierin6] as the proposed
solution for how to use BIER in IPv6 networks. It does not change
the deployment model of using BIER forwarding on every replicating
hop (ideally for every router), but instead it recommends to:
1. Encapsulate IPv6 multicast packets end-to-end over BIER
2. Encapsulate BIER packets hop-by-hop across IPv6 when there are
one or more intervening routers not supporting BIER
This approach is very similar to how in Example 1 and 2 networks that
wanted to operationally standardize on only MPLS (Example 1) on every
hop or only IPv6 (Example 2) on every hop, where asked by vendors to
instead use IPv4 multicast because it existed and was hence easier to
sell with existing vendor products. Instead of adding Source-routing
to IPv6 multicast, BIERin6 suggests two layers of tunneling to make
BIER fit into an otherwise IPv6 only network, effectively turning the
network into a dual-plane IPv6-unicast-only + BIER-multicast-only
network.
Eckert Expires 27 April 2023 [Page 9]
Internet-Draft msr6-problem-statement-eckert October 2022
3.5.4. Example 4: Routing for multicast
The re-use of unicast technologies for multicast also expands into
the control plane. Initial multicast designs where proposing
multicast specific routing protocols like CBT, MOSPF and DVMRP, one
of the core selling points that made PIM successful in the IETF was
that it was re-using any deployed IP unicast routing protocol. Even
when functionally for IP multicast itself, each of the prior
protocols had arguably benefits (some of which where later also done
with PIM, such as CBT->Bidir-PIM).
While re-use of existing unicast routing protocols may also be seen
as a functional benefit in itself, these functional advantages where
far less when doing the control plane for MPLS multicast.
mLDP was picked instead of PIM, because it allows customers to
troubleshoot signaling from a single (LDP/TCP) basis for both unicast
and multicast forwarding plane label binding and have more comparable
behavior in a variety of corner situations.
Other MVPN signaling where initially continuing to use PIM in draft-
rosen, but where later on demand of customers re-specified into BGP
to suit the operator preference for a single routing protocol BGP for
all services in the network (BESS model). The functionality itself
was not the differentiator over PIM, instead the original ask was to
do all that works well with PIM equally with BGP.
3.6. E6: No Support for SRv6 architecture
BIER/BIER-TE is a separate source-routing multicast architecture that
is based on re-using the single BIER forwarding mechanisms in any
type of network, whatever its preferred unicast architecture is.
Because there was and is no well-established mechanism for MPLS
forwarding planes how to extend MPLS forwarding planes with extension
headers, BIER did have to come up with its own header for MPLS
networks. This makes it fundamentally different from IPv6, where
[RFC8200] defines the overall source-routing mechanism, and SRv6 is
the one instance of it. [RFC6554] is another instance.
Eckert Expires 27 April 2023 [Page 10]
Internet-Draft msr6-problem-statement-eckert October 2022
Note though that an IETF MPLS design team is currently (2021/2022)
working on future common extension header design for MPLS networks,
so it is but a theoretical but still interesting question if in a
time when MPLS had such established standards, MPLS network operators
would have accepted a BIER header not specifically following all
rules for source routing and header construction established by such
pre-existing MPLS rules. This nevertheless is exactly one of the
problems IPv6 network operators would like to see solved by MSR6
because those definitions do exist in [RFC8200] for IPv6 networks.
3.7. E7: Inability to perform per-hop IPv6 forwarding plane features
IPFIX [RFC7011] is a protocol to export traffic flow telemetry. It
is widely implemented and deployed for IP/IPv6 multicast, allowing to
capture per multicast (S,G) traffic flow statistics for network OAM
services.
With BIER and BIERin6, there is no knowledge at the BIER forwarding
layer about multicast (S,G) flows. This only exists on the flow-
overlay layer, which is not present on non-ingress/egress BFRs.
The underlying forwarding functionality to capture traffic statistics
for IPfix flow export has also been used in other common industry
functions, such as MediaTrace mechanisms.
Policies for IPv6 multicast traffic, such as expressed through ACLs
or other router constructs, are used for a variety of functions.
1. Filtering of traffic based on multicast destination group ranges,
includes administrative scoping. For example, large service
providers often have two tiers of service elements, where the PE
are not the last point of policy control, but instead the first P
hop could be. This avoid more complex CsC style additional
layers.
2. Destination address group bits of IPv6 addresses have been used
to indicate target QoS classes for traffic more flexibly and
easier inter-domain than DSCP.
3. IPsec encryption/decryption can be inserted on any multi-hop path
segments of an IP/IPv6 network using the IP multicast (S,G)
address information as part of setting up the IPsec SA policies
(this is not specific to IP multicast but comes for free from
IPsec being designed against IP and thus the address information
that IPv6 multicast also shares).
Eckert Expires 27 April 2023 [Page 11]
Internet-Draft msr6-problem-statement-eckert October 2022
Applying policies at arbitrary points along the network path is also
more important for MSR6 than BIER, because MSR6 is not solely focused
on only P/PE network designs, and the overall design of the BIER
architecture with flow overlay, BIER layer and routing underlay is
very much expecting such a network design: BIER removes the knowledge
about the actual BIER payload (IPv6 in our case) from the BFR
midpoints and tunnels it over BIER between BFIR and BFER. This is
not sufficient for all MSR6 deployment options.
3.8. E8: Routing convergence speed and complexity in large networks
Multicast services have commonly higher service level objectives than
so-called Internet Best-Effort traffic, yet customers are looking for
equally easy to operationalise solutions for multicast in the
network.
With BIER and a large set of destinations (BFER), the BIFT hardware
forwarding tables on every hop in the network need to be updated with
BIFT state for all destinations. For tenths of thousands of
destinations, this becomes as slow as updating the same number of
unicast FIB entries. (it is also a waste of BIFT state when
potentially at any point in time only a small percentage of
destinations actually does need to receive traffic).
This ultimate hardware forwarding goal is preceded with all the well
understood convergence issues of IGPs (micro-loops etc), as well as
the time IGPs take to converge.
The known solution to this problem is to utilize strict explicit
trees to deliver traffic. This takes the IGP in the network out of
the picture for hop-by-hop forwarding, and when senders do utilize
e.g. link-state information to calculate paths for trees, they can
constrain their calculations to only the destination addresses of
interest, but not the whole set of destinations in the network.
Even though BIER-TE can express strict trees, it can not overcome
these issue well, because of the flat bitstring design (derived from
BIER), it is not possible to build strict-steered multicast trees and
simultaneously reduce BIFT state. Instead, attempting to solve this
issue with BIER-TE makes BIFT larger.
4. Candidate MSR6 solution discussion
Eckert Expires 27 April 2023 [Page 12]
Internet-Draft msr6-problem-statement-eckert October 2022
4.1. S1: BIER: Efficiency of delivery for sparse groups in large
networks
[I-D.eckert-msr6-rbs] and various other MSR6 proposal introduce
different stateless multicast source routing encodings than the BIER/
BIER-TE "flat" bitstrings. These all overcome the problem by more
intelligent encoding. [I-D.eckert-bier-cgm2-rbs] also contains
preliminary scalability simulation results comparing the performance
against BIER. See also {#exp}.
4.2. S2: BIER-TE: Efficiency of delivery for sparse groups in large
networks
Solutions to this problem such as [I-D.eckert-msr6-rbs] are even
stronger solutions to Section 2.2 than to Section 2.1 because BIER-TE
suffers more from the problem than BIER. Equally, see also {#exp}.
4.3. S3: BIER/BIER-TE: Large amount of per-BFR state (BIFT)
In MSR6 proposals such as [I-D.eckert-msr6-rbs], each router only
needs as much per-BFR state (BIFT) as that router has neighbors,
which typically is 10..200, therefore order or magnitudes less than
required in BIER/BIER-TE. This applies equally to using such
proposals instead of BIER or BIER-TE.
4.4. S4: More complex, multi-protocol host stack requirements with BIER
IPv6 socket APIs exist today on every host stack for every OS,
especially a large variety of IoT host stacks, but also host stacks
that are old and will not be able to receive OS-level updates, or
that for regulatory and other business reasons can not receive
available OS-level updates (because of avoid need for re-validation).
IPv6 host stacks also have the socket API option to insert from
application level arbitrary IPv6 extension headers. This allows to
implement MSR6 solely in user-land without additional (UDP) tunneling
required.
4.5. S5: Support for single, IPv6/RFC8200 compliant network forwarding
plane
MSR6 proposed to use [RFC8200] derived IPv6 extension headers for
source routing, similar to unicast source routing headers for IPv6 in
[RFC6554] or [RFC8754] with IPv6 destination address rewrite on every
source routing hop. When the source routing header information
indicates only the destinations, this is called by MSR6 Best Effort
(BE) mode. When the source-routing information indicates the
delivery tree, this is called by MSR6 "Tree Engineered" (TE) mode.
Eckert Expires 27 April 2023 [Page 13]
Internet-Draft msr6-problem-statement-eckert October 2022
To support IPv6 multicast without multiple extension header, MSR6
options such as [I-D.eckert-msr6-rbs] propose to include not only the
source-routing information to perform BE/TE functions (as in BIER/
BIER-TE), but also to include the IPv6 multicast destination address.
This is necessary because the IPv6 destination address is not
transparent end-to-end in RFC8200 compliant source-routing.
It is also another significant difference over BIER, where IPv6
multicast can only be supported as a flow overlay. And it is the
reason why at the layer of IPv6 with MSR all IPv6 multicast address
information is directly available, even when the final IPv6
destination address is not in the base but and IPv6 extension header.
4.6. S6: No Support for SRv6 architecture
As described in Section 4.5, MSR6 proposes extension header similar
in spirit and goals to SRH. It should equally create standards
document that allow to apply the SRv6 terminology and concept of SIDs
(Segment IDentifiers) to MSR6 when this is desired by the network
operator.
By mean of using IPv6 addressing in MSR6, it may also be possible to
utilize existing SRv6/SID benefits with multicast (such as inclusion
of programability aspects in the MSR6 IPv6 addresses). This is
subject to further study.
Note that IPv6 multicast did equally explore and benefit from special
encodings into the IPv6 address space, arguably similar to SRv6
adding additional functionality via IPv6 address bits. This includes
specifically "Unicast-Prefix-based IPv6 Multicast Addresses ",
[RFC3306] and even more so "Embedding the Rendezvous Point (RP)
Address in an IPv6 Multicast Address", [RFC3956].
4.7. S7: Inability to perform per-hop IPv6 forwarding plane features
MSR6 enables all collateral IPv6 forwarding plane features along the
path by explicitly exposing intentionally the same source and
destination address as non-source-routed IPv6 multicast packets.
This is unlike BIERin6, where this information is not in the possible
outer IPv6 header (on loose hops), nor the BIER header, but only in
the BIER end-to-end encapsulated payload header if it is IPv6. On
BFR, the IPv6 context (such as VRF) in which this IPv6 header exists
is not known, so any capturing of information via such "deep packet
inspection" mechanism comes at a lot more problems to solve and would
hardly be something that could be used for standardized IETF
mechanisms such as IPfix.
Eckert Expires 27 April 2023 [Page 14]
Internet-Draft msr6-problem-statement-eckert October 2022
The most fundamental privacy rule for network protocol designs is to
not leverage packet data that is not explicitly meant to be exposed
at a particular layer/point in the network, and on an MSR6 hop, the
MSR6 extension header is explicitly exposed, representing the
multicast (S,G) flow information, whereas in BIER this is never the
case, but only on BFIR/BFER in the flow-overlay layer.
With MSR6, there is an additional overhead for routers to examine the
destination IPv6 address from an extension header, but that is the
cost of doing IPv6 business, because with the way [RFC8200] expects
source routing to operate, this can not be further optimized. Of
course, this means that it is highly important to ensure that exactly
this IPv6 multicast destination address has ideally not multiple
options of extension header or positions in it, but that MSR6 will
develop one-fits-all for this component of the solution.
4.8. S8: Routing convergence speed and complexity
MSR6 proposal that attempt to solve this problem are those that allow
to encode in the source-routing header sparse trees for large
networks such that no IGP routing, but only hop-by-hop forwarding is
required. These solutions do likewise also require only small
amounts of BIFT forwarding state per router that only changes when
link-local MSR6 neighbors change {I-D.eckert-msr6-rbs}} is an example
of such proposals.
5. Open issues
Probably many...
This version of the document does likely not well enough highlight in
all places when the solution to a problem could also be had by
enhancements to BIER, but it does so in hopefully the most important
places.
6. Changelog
00 - initial version.
7. Informative References
Eckert Expires 27 April 2023 [Page 15]
Internet-Draft msr6-problem-statement-eckert October 2022
[I-D.eckert-bier-cgm2-rbs]
Eckert, T. T. and B. Xu, "Carrier Grade Minimalist
Multicast (CGM2) using Bit Index Explicit Replication
(BIER) with Recursive BitString Structure (RBS)
Addresses", Work in Progress, Internet-Draft, draft-
eckert-bier-cgm2-rbs-01, 9 February 2022,
<https://www.ietf.org/archive/id/draft-eckert-bier-cgm2-
rbs-01.txt>.
[I-D.eckert-msr6-rbs]
Eckert, T. T., Geng, X., Zheng, X., Meng, R., and F. Li,
"Recursive Bitstring Structure (RBS) for Multicast Source
Routing over IPv6 (MSR6)", Work in Progress, Internet-
Draft, draft-eckert-msr6-rbs-00, 11 July 2022,
<https://www.ietf.org/archive/id/draft-eckert-msr6-rbs-
00.txt>.
[I-D.ietf-bier-bierin6]
Zhang, Z., Zhang, Z. J., Wijnands, I., Mishra, M. P.,
Bidgoli, H., and G. S. Mishra, "Supporting BIER in IPv6
Networks (BIERin6)", Work in Progress, Internet-Draft,
draft-ietf-bier-bierin6-05, 4 September 2022,
<https://www.ietf.org/archive/id/draft-ietf-bier-
bierin6-05.txt>.
[I-D.liu-msr6-problem-statement]
Liu, Y., Jiang, T., Eckert, T. T., Li, Z., Mishra, G. S.,
Qin, Z., Lin, C., and X. Geng, "Problem Satement of IPv6
Multicast Source Routing (MSR6)", Work in Progress,
Internet-Draft, draft-liu-msr6-problem-statement-01, 21
October 2022, <https://www.ietf.org/archive/id/draft-liu-
msr6-problem-statement-01.txt>.
[RFC3306] Haberman, B. and D. Thaler, "Unicast-Prefix-based IPv6
Multicast Addresses", RFC 3306, DOI 10.17487/RFC3306,
August 2002, <https://www.rfc-editor.org/info/rfc3306>.
[RFC3956] Savola, P. and B. Haberman, "Embedding the Rendezvous
Point (RP) Address in an IPv6 Multicast Address",
RFC 3956, DOI 10.17487/RFC3956, November 2004,
<https://www.rfc-editor.org/info/rfc3956>.
[RFC5015] Handley, M., Kouvelas, I., Speakman, T., and L. Vicisano,
"Bidirectional Protocol Independent Multicast (BIDIR-
PIM)", RFC 5015, DOI 10.17487/RFC5015, October 2007,
<https://www.rfc-editor.org/info/rfc5015>.
Eckert Expires 27 April 2023 [Page 16]
Internet-Draft msr6-problem-statement-eckert October 2022
[RFC6554] Hui, J., Vasseur, JP., Culler, D., and V. Manral, "An IPv6
Routing Header for Source Routes with the Routing Protocol
for Low-Power and Lossy Networks (RPL)", RFC 6554,
DOI 10.17487/RFC6554, March 2012,
<https://www.rfc-editor.org/info/rfc6554>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>.
[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", STD 86, RFC 8200,
DOI 10.17487/RFC8200, July 2017,
<https://www.rfc-editor.org/info/rfc8200>.
[RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation
for Bit Index Explicit Replication (BIER) in MPLS and Non-
MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January
2018, <https://www.rfc-editor.org/info/rfc8296>.
[RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J.,
Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header
(SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020,
<https://www.rfc-editor.org/info/rfc8754>.
Appendix A. Other considerations towards MSR6
A.1. Design separation between tree building and other functions
BIER is designed as a common stateless multicast forwarding layer
meant to be applicable to any network (or ever higher layer) protocol
that can be tunneled/encapsulated across it ("flow-overlay").
This incurs the price of designing a flow overlay layer on BFIR/BFER
(ingress/egress BIER edge routers), and not having any functionality
of that encapsulated flow overlay traffic available on the BIER
midpoints (BFR) - unless any such functionality is meticulously
mapped into the BIER packet header, such as MPLS TC and IP/IPv6 DSCP
as specified in [RFC8296].
Eckert Expires 27 April 2023 [Page 17]
Internet-Draft msr6-problem-statement-eckert October 2022
This design maps very well the common P/PE design in service
providers, but it does not necessarily match arbitrary IPv6 networks
in industry, IoT, data-center (interconnect), or even service
provider networks (aggregation, multi-AS,...). Instead, when such
technologies are used, network designs across all features have to be
built to enable P routers to be mostly feature-less.
IPv6 MSR6 instead is intended to allow re-using by default any pre-
source-routing IPv6 feature as much as possible on any hop of the
network. And minimize the requirements of introducing network
protocol P/PE separation designs across all functions in the network
to enable this.
One example for this principle is an early stage in reducing IP
multicast tree state in data centers.
Originally, OAM for IP multicast traffic in DC was performed on PIM-
SM (S,G) state (per state packet/byte counters), but diagnosis of
PIM-SM state is challenging in itself and scales linearly with the
amount of applications running - operators did not like it. Even
worse, the amount of PIM-SM state deployed in the target data centers
was about to exceed performance of routers. This lead to the
development of Bidir-PIM [RFC5015] which reduced PIM state to (*,G),
for example in typical data-centers from 20,000 states to 200 states
at the time.
But Bidir-PIM made it impossible to have per (S,G) counters anymore
on every hop through the multicast tree state. Instead, IPfix for IP
multicast was developed, which could be configured for arbitrary
subsets of multicast flows (based on (S,G) ACL policies), thus also
not being tied to the scaling problems of the IP multicast tree
building, and by virtue of operating completely independent of IP
multicast it also had no tie-in with IP multicast protocols and can
therefore also help to diagnose them.
A.2. Reuse of BIER principles and technology components as much as
possible
MSR6 efforts considers BIER-WG to be one of its most important
adjacencies in the IETF and does not want to re-invent anything
unnecessarily that BIER already solves well if applicable to IPv6
only source routing mechanisms.
Likewise, MSR6 efforts would welcome if new technology ideas
originally experimented and used in MSR, but equally applicable to
BIER can also be shared across MSR6 and BIER (such as new
representations of the source-routing information).
Eckert Expires 27 April 2023 [Page 18]
Internet-Draft msr6-problem-statement-eckert October 2022
A.3. MSR6 as experimental (not standards track) work
In the opinion of the author, the ability to experiment with new and
better technology components in MSR6 is easier when an MSR6 WG would
initially be targeted for experimental status - in the same way as
BIER initially was. BIER WG has for some time now been "upgraded" to
standards track work, and also its RFC where upgraded to standard
track, even though this does not show in the documents themselves
(for historical reasons of the RFC publishing process).
With BIER (WG and RFCs) being standards track now, it still can still
adopt experimental work, but when BIER establishes itself as a
deployed solution in the market, it often becomes less easy to
publish new, experimental work with the same name, because this
raises the risk of customers not being able to well distinguish
between more proven, older BIER options and newer, potentially
experimental ones.
Therefore, experimental new technology options may be better first
proven in the context of MSR6 work even when equally applicable to
MSR6 and BIER. In the opinion of the author this specifically
includes the new tree encoding structures proposed by different MSR6
drafts (including the authors [I-D.eckert-msr6-rbs].
The core tenets of MSR6 such as its use of RFC8200 source routing
mechanisms for end-to-end IPv6 multicast could easily be standards
track, but may simply be best targeted as experimental too, solely to
be sure that sufficient practical/operational experience work is
gained - and tracked by appropriate IETF work - before they are
upgraded to standards level.
Author's Address
Toerless Eckert (editor)
Futurewei Technologies USA
2220 Central Expressway
Santa Clara, CA 95050
United States of America
Email: tte@cs.fau.de
Eckert Expires 27 April 2023 [Page 19]
