| Network Working Group | J. Xie |
| Internet-Draft | Huawei Technologies |
| Updates: 8296 (if approved) | L. Geng |
| Intended status: Standards Track | China Mobile |
| Expires: January 14, 2021 | M. McBride |
| Futurewei | |
| R. Asati | |
| Cisco | |
| S. Dhanaraj | |
| Huawei | |
| Y. Zhu | |
| China Telecom | |
| Z. Qin | |
| China Unicom | |
| M. Shin | |
| LG Uplus | |
| G. Mishra | |
| Verizon Inc. | |
| X. Geng | |
| Huawei | |
| July 13, 2020 |
Encapsulation for BIER in Non-MPLS IPv6 Networks
draft-xie-bier-ipv6-encapsulation-08
This document proposes a BIER IPv6 (BIERv6) encapsulation for Non-MPLS IPv6 Networks using the IPv6 Destination Option extension header. This document updates RFC 8296.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] and [RFC8174].
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 14, 2021.
Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Bit Index Explicit Replication (BIER) [RFC8279] is an architecture that provides optimal multicast forwarding without requiring intermediate routers to maintain any per-flow state by using a multicast-specific BIER header.
[RFC8296] defines a common BIER Header format for MPLS and Non-MPLS networks. It has defined two types of encapsulation methods using the common BIER Header, (1) BIER encapsulation in MPLS networks, here-in after referred as MPLS BIER Header in this document and (2) BIER encapsulation in Non-MPLS networks, here-in after referred as Non-MPLS BIER Header in this document. [RFC8296] also assigned Ethertype=0xAB37 for Non-MPLS BIER Header packets to be directly carried over the Ethernet links.
This document proposes a BIER IPv6 encapsulation for Non-MPLS IPv6 Networks, defining a method to carry the standard Non-MPLS BIER header (as defined in [RFC8296]) in the native IPv6 header. A new IPv6 Option type - BIER Option is defined to encode the standard Non-MPLS BIER header and this newly defined BIER Option is carried under the Destination Options header of the native IPv6 Header [RFC8200].
The relationship of this document to BIER core standards is listed in Appendix A.
The relevant extensions to BIER Control-plane Standards are listed in Appendix B.
Readers of this document are assumed to be familiar with the terminology and concepts of the documents listed as Normative References.
The following new terms are used throughout this document:
Destination Options Header and the Options that can be carried under this extension header is defined in [RFC8200]. This document defines a new Option type - BIER Option, to encode the Non-MPLS BIER header. As specified in Section 4.2 [RFC8200], the BIER Option follows type-length-value (TLV) encoding format and the standard Non-MPLS BIER header [RFC8296] is encoded in the value portion of the BIER Option TLV.
This BIER Option MUST be carried only inside the IPv6 Destination Options header and MUST NOT be carried under the Hop-by-Hop Options header.
The BIER Option is encoded in type-length-value (TLV) format as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Next Header | Hdr Ext Len | Option Type | Option Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ BIERv6 Option Data (Non-MPLS BIER Header defined in RFC8296) ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
When a BIERv6 packet is replicated to a next hop BFR, an unicast address of the next hop BFR is used as the destination address of the BIERv6 packet. Considerations of using unicast (or multicast) address is listed in Appendix C.
The unicast address used in BIERv6 packet targeting a BFR SHOULD be advertised as part of the BIER IPv6 Encapsulation. When a BFR advertises the BIER information with BIERv6 encapsulation capability, an IPv6 unicast address of this BFR MUST be selected specifically for BIERv6 packet forwarding. Locally this "BIER Specific" IPv6 address is initialized in FIB with a flag of "BIER specific handling", represented as End.BIER function.
If a BFR belongs to more than one sub-domain, it may (though it need not) have a different End.BIER in each sub-domain. If different End.BIER is used for each sub-domain, implementation SHOULD support verifying the DA of a BIERv6 packet is the End.BIER address bound by the sub-domain of the packet.
For security deployment of BIERv6, the End.BIER address(es) is required to be allocated from an IPv6 address block, and the IPv6 address block is used for domain boundary security policy. See section 5.1 of this document for such security policy. Such kind of security policy using IPv6 address block follows the paradigm settled by the [RFC8754] section 5.
Deployment of BIERv6 in SRv6 network is allowed. In this case, the BIERv6 domain is the same as SRv6 domain, and the End.BIER address is allocated from the locator of SRv6.
To better understand the configuration mode of End.BIER address in BIERv6, [I-D.geng-bier-bierv6-yang] could be referenced.
For the convenience of such co-existence of BIERv6 and SRv6, the indication of End.BIER or "BIER specific handling" in FIB shares the same space as SRv6 Endpoints Behaviors defined in [I-D.ietf-spring-srv6-network-programming].
The following is an example pseudo-code of the End.BIER function:
1. IF NH = 60 and HopLimit > 0 ;;Ref1
2. IF (OptType1 = BIER) and (OptLength1 = HdrExtLen*8 + 4) ;;Ref2
3. Lookup the BIER Header inside the BIER option TLV.
4. Forward via the matched entry.
5. ELSE ;;Ref3
6. Drop the packet and end the process.
7. ELSE IF NH=ICMPv6 or (NH=60 and Dest_NH=ICMPv6) ;;Ref4
8. Send to CPU.
9. ELSE ;;Ref5
10. Drop the packet.
Ref2: The first TLV is BIER type and is the only TLV present in Destination options header.
Ref3/Ref5: Undesired packet is droped because the destination address is the BIER specific IPv6 address (End.BIER function).
Ref4: An ICMPv6 packet using End.BIER as destination address.
As a multicast packet enters the BIER domain in a Non-MPLS IPv6 network, the multicast packet will be encapsulated with BIERv6 Header by the Ingress BFR (BFIR).
Typically a BIERv6 header would contain the Destination Options Header as the only Extensions Header besides IPv6 Header, as depicted in the below figure.
+---------------+------------------+----------------------+
| IPv6 header | IPv6 DO Header | X type of |
| | with BIER Option | C-multicast packet |
| | | |
| Next Hdr = 60 | Nxt Hdr = X | (IPv4/IPv6/Ethernet) |
+---------------+------------------+----------------------+
| | |
|<----------BIERv6 header--------->|<---BIERv6 payload--->|
Format of the multicast packet with BIERv6 encapsulation carrying other extension headers along with Destination Options extension header is required to follow general recommendations of [RFC8200] and examples in other RFCs. [RFC6275] introduces how the order should be when other extension headers carries along with Home address option in a destination options header. Similar to this example, this document requires the Destination Options Header carrying the BIER option MUST be placed as follows:
Source Address field in the IPv6 header MUST be a routable IPv6 unicast address of the BFIR in any case.
BFIR encodes the BIERv6 header in the above mentioned encapsulation format and forwards the BIERv6 packet to the nexthop BFR following the local BIFT table.
BFRs in the IPv6 network, processes and replicates the packets towards the BFERs using the local BIFT table. The BitString field in the BIERv6 Option Data may be changed by the BFRs as they replicate the packet. BFRs MUST follow the procedures defined in section 3.1 as they modify the other fields in the BIERv6 Option Data. The source address in the IPv6 header MUST NOT be modified by the BFRs.
When a multicast packet enters the BIER domain, the Ingress BFR (BFIR) encapsulates the multicast packet with a BIERv6 Header, transforming it to a BIERv6 packet. The BIERv6 header includes an IPv6 header and a BIERv6 Option in IPv6 Destination Options Header. Source Address field in the IPv6 header MUST be set to a routable IPv6 unicast address of the BFIR. Destination Address field in the IPv6 header is set to the End.BIER address of the next-hop BFR the BIERv6 packet replicating to, no matter next-hop BFR is directly connected (one-hop) or not directly connected (multi-hop).
Upon receiving an BIERv6 packet, the BFR processes the IPv6 header first. This is the general procedure of IPv6.
If the IPv6 Destination address is an End.BIER IPv6 unicast address of this BFR, a 'BIER Specific Handling' indication will be obtained by the preceding Unicast DA lookup (FIB lookup). The BIER option, if exists, will be checked to decide which neighbor(s) to replicate the BIERv6 packet to.
It is a local behavior to handle the combination of extension headers, options and the BIER option(s) in destination options header when a 'BIER Specific Handling' indication is got by the preceding FIB lookup. Early deployment of BIERv6 may require there is only one BIER option TLV in the destination options header followed the IPv6 header. How other extension headers or more BIER option TLVs in a BIERv6 packet is handled is outside the scope of this document.
A packet having a 'BIER Specific Handling' indication but not having a BIER option is supposed to be a wrong packet or an ICMPv6 packet, and the process can be referred to the example in section 3.2.
A packet not having a 'BIER Specific Handling' indication but having a BIER option SHOULD be processed normally as unicast forwarding procedures, which may be a behavior of drop, or send to CPU, or other behaviors in existing implementations.
The Destination Address field in the IPv6 Header MUST change to the nexthop BFR's End.BIER Unicast address in BIERv6.
The Hop Limit field of IPv6 header MUST decrease by 1 when sending packets to a BFR neighbor, while the TTL in the BIER header MUST be unchanged on a Non-BIER router, or decrease by 1 on a BFR.
The BitString in the BIER header in the Destination Options Header may change when sending packets to a neighbor. Such change of BitString MUST be aligned with the procedure defined in RFC8279. Because of the requirement to change the content of the option when forwarding BIERv6 packet, the BIER option type should have chg flag 1 per section 4.2 of RFC8200.
The procedures applies normally if a bit corresponding to the self bfr-id is set in the BitString field of the BIERv6 Option Data of the BIERv6 packet. The node is considered to be an Egress BFR (BFER) in this case. The BFER removes the BIERv6 header, including the IPv6 header and the Destination Options header, and copies the packet to the multicast flow overlay. The egress VRF of a packet may be determined by a further lookup on the IPv6 source address instead of the upstream-assigned MPLS Label as described in [RFC8556].
The Fragment Header, AH Header or ESP Header, if exists after the BIER options header, can be processed on BFER only as part of the multicast flow overlay process.
The following diagram shows the whole progression of the multicast packet as it enters the BIERv6 domain on PE1, and leaves the BIERv6 domain on PE2 and PE3.
+-------------+ +-------------+
|{S=PE1,D=P2} | |{S=PE1,D=PE2}|
+-------------+ +-------------+
|[BitStr=0110]| |[BitStr=0010]|
+==========+ +=============+ +=============+ +==========+
|(C-MC Pkt)| >> | (C-MC Pkt) | >> | (C-MC Pkt) | >> |(C-MC Pkt)|
+==========+ +=============+ +=============+ +==========+
CE1-----------PE1------[P1]------P2----------------PE2------------CE2
(BFIR) /(BFR) (BFER, BFR-id=2)
/
/ +-------------+
| |{S=PE1,D=PE3}|
| +-------------+
| |[BitStr=0100]|
\ +=============+ +==========+
\ >> | (C-MC Pkt) | >> |(C-MC Pkt)|
\ +=============+ +==========+
+------[P3]-------PE3------------CE3
(BFER, BFR-id=3)
{S=PE1,D=PE2}: Source address and Destination address in IPv6 header.
[BitStr=0110]: BitString value in IPv6 DO Header.
(C-MC Pkt): Customer MultiCast packet.
BIER IPv6 encapsulation provides a new encapsulation based on IPv6 and BIER to transport multicast data packet in a BIER domain. The BIER domain can be a single IGP area, an anonymous system (AS) with multiple IGP areas, or multiple anonymous systems (ASes) operated by a network operator. A single BIER Sub-domain may be deployed through the whole BIER Domain, as illustrated in [I-D.geng-bier-ipv6-inter-domain].
This section reviews security considerations related to the BIER IPv6 encapsulation, based on security considerations of [RFC8279], [RFC8296], and other documents related to IPv6 extension.
It is expected that all nodes in a BIER IPv6 domain are managed by the same administrative entity. BIER-encapsulated packets should generally not be accepted from untrusted interfaces or tunnels. For example, an operator may wish to have a policy of accepting BIER-encapsulated packets only from interfaces to trusted routers, and not from customer-facing interfaces. See section 5.1 for normal Intra domain deployment.
For applications that require a BFR to accept a BIER-encapsulated packet from an interface to a system that is not controlled by the network operator, the security considerations of [RFC8296] apply.
BIER IPv6 encapsulation may cause ICMP packet sent to BFIR and cause security problems. See section 5.2 for ICMP related problems.
This document introduces a new option used in IPv6 Destination Options Header, together with the special-use IPv6 address called End.BIER in IPv6 destination address for BIER IPv6 forwarding. However the option newly introduced may be wrongly used with normal IPv6 destination address. See section 5.3 for problems introduced by the new IPv6 option with normal IPv6 destination address.
If the multicast data packet of a BIERv6 packet is altered by an intermediate router, contents of the multicast data packet will be damaged. BIER IPv6 encapsulation provides the ability of IPsec to ensure the confidentiality or integrity for multicast data packet. See section 5.4 for this security problem.
If the BIERv6 encapsulation of a particular packet specifies a BitString (together with SI) other than the one intended by the BFIR, the packet is likely to be misdelivered. Some modifications of the BIER encapsulation, e.g., setting every bit in the BitString, may result in denial-of-service (DoS) attacks. This kind of DoS attack is a challenge not only in BIERv6 but also in BIER as specified in [RFC8279] and [RFC8296], as the BitString is required to change on BFR per the BIER forwarding procedures. This document does not provide new mechanisms to improve this kind of weakness.
A BIER router accepts and uses the End.BIER IPv6 address to construct BIFT only when the IPv6 address is configured explicitly, or received from a router via control-plane protocols. The received information is validated using existing authentication and security mechanisms of the control-plane protocols. BIER IPv6 encapsulation does not define any additional security mechanism in existing control-plane protocols, and it inherits any security considerations that apply to the control-plane protocols.
Generally nodes outside the BIER Domain are not trusted: they cannot directly use the End.BIER of the domain. This is enforced by two levels of access control lists:
1. Any packet entering the BIER Domain and destined to an End.BIER IPv6 Address within the BIER Domain is dropped. This may be realized with the following logic. Other methods with equivalent outcome are considered compliant:
* allocate all the End.BIER IPv6 Address from a block S/s
* configure each external interface of each edge node of the domain with an inbound infrastructure access list (IACL) which drops any incoming packet with a destination address in S/s
* Failure to implement this method of ingress filtering exposes the BIER Domain to BIER attacks as described and referenced in [RFC8296].
2. The distributed protection in #1 is complemented with per node protection, dropping packets to End.BIER IPv6 Address from source addresses outside the BIER Domain. This may be realized with the following logic. Other methods with equivalent outcome are considered compliant:
* assign all interface addresses from prefix A/a
* assign all the IPv6 addresses used as source address of BIER IPv6 packets from a block B/b
* at node k, all End.BIER IPv6 addresses local to k are assigned from prefix Sk/sk
* configure each internal interface of each BIER node k in the BIER Domain with an inbound IACL which drops any incoming packet with a destination address in Sk/sk if the source address is not in A/a or B/b.
For simplicity of deployment, a configuration of IACL effective for all interfaces can be provided by a router. Such IACL can be referred to as global IACL(GIACL) .Each BIER node k then simply configs a GIACL which drops any incoming packet with a destination address in Sk/sk if the source address is not in A/a or B/b for the intra-domain deployment mode.
The BIERv6 BFR does not send ICMP error messages to the source address of a BIERv6 packet, there is still chance that Non-BFR routers send ICMP error messages to source nodes within the BIER Domain.
A large number of ICMP may be elicited and sent to a BFIR router, in case when a BIERv6 packet is filled with wrong Hop Limit, either error or malfeasance. A rate-limiting of ICMP packet should be implemented on each BFR.
The ingress node can take note of the fact that it is getting, in response to BIER IPv6 packet, one or more ICMP error packets. By default, the reception of such a packets MUST be countered and logged. However, it is possible for such log entries to be "false positives" that generate a lot of "noise" in the log; therefore, implementations SHOULD have a knob to disable this logging.
This document introduces a new option used in IPv6 Destination Options Header. An IPv6 packet with a normal IPv6 address of a router (e.g. loopback IPv6 address of the router) as destination address will possibly carry a BIER option.
For a router incapable of BIERv6, such BIERv6 packet will not be processed by the procedure described in this document, but be processed as normal IPv6 packet with unknown option, and the existing security considerations for handling IPv6 options apply. Possible way of handling IPv6 packets with BIER option may be send to CPU for slow path processing, with rate-limiting, or be discarded according to the local policy.
For a router capable of BIERv6, such BIERv6 packet MUST NOT be forwarded, but should be processed as a normal IPv6 packet with unknown option, or additionally and optionally be countered and logged if the router is capable of doing so.
IPsec [RFC4301] uses two protocols to provide traffic security services -- Authentication Header (AH) [RFC4302] and Encapsulating Security Payload (ESP) [RFC4303]. Each protocol supports two modes of use: transport mode and tunnel mode. IPsec support both unicast and multicast. IPsec implementations MUST support ESP and MAY support AH.
This document assume IPsec working in tunnel mode with inner IPv4 or IPv6 multicast packet encapsulated in outer BIERv6 header and IPsec header(s).
IPsec used with BIER IPv6 encapsulation to ensure that a BIER payload is not altered while in transit between BFIR and BFERs. If a BFR in between BFIR and BFERs is compromised, there is no way to prevent the compromised BFR from making illegitimate modifications to the BIER payload or to prevent it from misforwarding or misdelivering the BIER-encapsulated packet, but the BFERs will detect the illegitimate modifications to the BIER Payload (or the inner multicast data packet). This could provide cryptographic integrity protection for multicast data transport. This capability of IPsec comes from the design that, the destination options header carrying the BIER header is located before the AH or ESP and the BFR routers in between BFIR and BFERs can process the BIER header without aware of AH or ESP.
For ESP, the Integrity Check Value (ICV) is computed over the ESP header, Payload, and ESP trailer fields. It doesn't require the IP or extension header for ICV calculating, and thus the change of DA and BIER option data does not affect the function of ESP.
For AH, the Integrity Check Value (ICV) is computed over the IP or extension header fields before the AH header, the AH header, and the Payload. The IPv6 DA is immutable for unicast traffic in AH, and the change of DA in BIER IPv6 forwarding for multicast traffic is incompatible to this rule. How AH is extended to support multicast traffic transporting through BIER IPv6 encapsulation is outside the scope of this document.
The detailed control-plane for BIER IPv6 encapsulation IPsec function is outside the scope of the document. Internet Key Exchange Protocol Version 2 (IKEv2) [RFC7296] and Group Security Association (GSA) [RFC5374] can be referred to for further studying.
Allocation is expected from IANA for a BIER Option Type codepoint from the "Destination Options and Hop-by-Hop Options" sub-registry of the "Internet Protocol Version 6 (IPv6) Parameters" registry. The value 0x70 is suggested.
+-----------+-----+-----+-------+-------------+------------+
| Hex Value | act | chg | rest | Description | Reference |
+-----------+-----+-----+-------+-------------+------------+
| 0x70 | 01 | 1 | 10000 | BIER Option | This draft |
+-----------+-----+-----+-------+-------------+------------+
Allocation is expected from IANA for an End.BIER function codepoint from the "SRv6 Endpoint Behaviors" sub-registry. The value 60 is suggested.
+-------+--------+--------------------------+------------+
| Value | Hex | Endpoint function | Reference |
+-------+--------+--------------------------+------------+
| TBD | TBD | End.BIER | This draft |
+-------+--------+--------------------------+------------+
The authors would like to thank Stig Venaas for his valuable comments. Thanks IJsbrand Wijnands, Greg Shepherd, Tony Przygienda, Toerless Eckert, Jeffrey Zhang, Pascal Thubert for the helpful comments to improve this document.
Thanks Aijun Wang for comments about BIER OAM function in BIER IPv6 encapsulation.
Thanks Mach Chen for review and suggestions about BIER-PING function in BIER IPv6 encapsulation.
Gang Yan
Huawei Technologies
China
Email: yangang@huawei.com
Yang(Yolanda) Xia
Huawei Technologies
China
Email: yolanda.xia@huawei.com
| [I-D.geng-bier-bierv6-yang] | Geng, X., Qin, Z. and F. Zheng, "YANG Data Model for Bierv6", Internet-Draft draft-geng-bier-bierv6-yang-00, June 2020. |
| [I-D.geng-bier-ipv6-inter-domain] | Geng, L., Xie, J., McBride, M. and G. Yan, "Inter-Domain Multicast Deployment using BIERv6", Internet-Draft draft-geng-bier-ipv6-inter-domain-01, January 2020. |
| [I-D.ietf-bier-ipv6-requirements] | McBride, M., Xie, J., Dhanaraj, S., Asati, R., Zhu, Y. and G. Mishra, "BIER IPv6 Requirements", Internet-Draft draft-ietf-bier-ipv6-requirements-05, July 2020. |
| [I-D.ietf-bier-ping] | Nainar, N., Pignataro, C., Akiya, N., Zheng, L., Chen, M. and G. Mirsky, "BIER Ping and Trace", Internet-Draft draft-ietf-bier-ping-07, May 2020. |
| [I-D.ietf-spring-srv6-network-programming] | Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., Matsushima, S. and Z. Li, "SRv6 Network Programming", Internet-Draft draft-ietf-spring-srv6-network-programming-16, June 2020. |
| [I-D.xie-bier-ipv6-isis-extension] | Xie, J., Wang, A., Yan, G. and S. Dhanaraj, "BIER IPv6 Encapsulation (BIERv6) Support via IS-IS", Internet-Draft draft-xie-bier-ipv6-isis-extension-01, January 2020. |
| [I-D.xie-bier-ipv6-mvpn] | Xie, J., McBride, M., Dhanaraj, S. and L. Geng, "Use of BIER IPv6 Encapsulation (BIERv6) for Multicast VPN in IPv6 networks", Internet-Draft draft-xie-bier-ipv6-mvpn-02, January 2020. |
The BIER architecture [RFC8279] is inherited in this BIERv6 proposal, and the layering mode of BIER architecture is fully supported with some necessary extension to the data plane as well as the control plane standards.
The focus of this document is BIERv6 data plane, including the BIERv6 encapsulation and packet forwarding procedures. The common BIER header encoding [RFC8296] is maximum reused in this BIERv6 proposal.
To better understand the overall BIER IPv6 problem space and requirements, refer to [I-D.ietf-bier-ipv6-requirements].
The relevant control-plane documents that have done or still to be done are listed below.
BIER is generally a hop-by-hop and one-to-many architecture, and thus the IPv6 Destination Address (DA) being a Multicast Address is a way one may think of as an approach for both the two paradigms in BIERv6 encapsulation.
However using a unicast address has the following benefits:
Some of the above scenarios are assumed part of BIER architecture as described in [RFC8279], and some of them are the scalability aspects for inter-AS stateless multicast this document intends to support. This document intends to fulfil all these requirements (categorized as multi-hop replication), and proposes to use unicast address for both one-hop replication and multi-hop replication.