Internet DRAFT - draft-morishita-dnsop-anycast-node-requirements
draft-morishita-dnsop-anycast-node-requirements
IETF DNSOP Working Group Y. Morishita
Internet-Draft S. Sato
Expires: September 28, 2006 T. Matsuura
JPRS
March 27, 2006
BGP Anycast Node Requirements for Authoritative Name Servers
draft-morishita-dnsop-anycast-node-requirements-03.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 28, 2006.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
IP anycast [1] is a technology to share one IP address for Internet
services with multiple server nodes. It is now being deployed for
improving service reliability, scalability, and stability.
Especially, "Global-Scope" IP anycast is now being deployed for
authoritative name servers, typically for root servers.
Morishita, et al. Expires September 28, 2006 [Page 1]
Internet-Draft Anycast Node Requirements March 2006
RFC 3258 [2] describes a set of practices to apply IP anycast
technology for authoritative name servers. And "Operation of Anycast
Services" Internet-Draft [3] (hereafter, called "Anycast BCP")
describes a series of recommendations and considerations for
distribution of services using IP anycast.
On the other hand, operators of authoritative name servers can also
refer to RFC 2182 [4] and 2870 [5] for general guidances on
appropriate practices for authoritative name servers.
This memo describes the details of requirements and preconditions for
making "Global-Scope" IP anycast nodes for authoritative name
servers, with the conformance of the practices in RFC 2182, 2870,
3258 and Anycast BCP.
And this memo also describes our findings and experiences for making
IP anycast nodes for us. Authors hope that it is useful for DNS
operators that they will walk on the same way in the future,
especially for TLD operators.
1. Introduction
By applying the IP anycast technology to DNS, name server operators
can increase the number of authoritative name server nodes, and
distribute them in topologically and geographically diverse
locations, without violating the DNS protocol limitations [6] [7].
If IP anycast is appropreately operated for DNS server nodes, it
improves the robustness against Denial-of-Service Attack and
Distributed Denial-of-Service Attack and the reliability of DNS
service. And it improves the DNS total response by decreasing RTT
for authoritative name server, and distributes authoritative name
servers' load, too.
However, IP anycast needs more careful operation for achieving the
original goals and improvements. In the IP anycast technology, IP
address does not specify the individual (real) end-point for the
Internet communications any longer. Which means that the real
communication peer can not be specified by the destination IP address
only. It means some new pitfalls and risks the DNS service, for
example, monitoring the availability of the service becomes more
difficult, and data of all DNS server nodes need syncing.
For achieving the original goals of IP anycast, improving the
robustness and the reliability of service. So by introduction of IP
anycast, The reliability of the entire system must not decrease.
Especially, DNS is one of an important infrastructures of the
Morishita, et al. Expires September 28, 2006 [Page 2]
Internet-Draft Anycast Node Requirements March 2006
Internet, so, introducing "Global-Scope" IP anycast in critical DNS
authoritative servers should be carefully.
This memo describes the details of requirements and preconditions for
making "Global-Scope" IP anycast nodes for authoritative name
servers, with the conformance of the practices in RFC 2182, 2870,
3258 and Anycast BCP.
And this memo also describes our findings and experiences for making
IP anycast nodes for us. Authors hope that it is useful for DNS
operators that they will walk on the same way in the future,
especially for TLD operators.
In this memo, authors focus on "BGP anycast" for making "Global-
scope" IP anycast nodes. It is the currently most popular technology
for making IP anycast nodes and it is already used for making some
root and TLD DNS server nodes. Authors think a part of the basic
point of view can be applied to "Local-scope" IP anycast, too.
2. BGP Anycast and DNS Service
BGP anycast is a part of IP anycasting technology. It uses a shared
IP address block and a shared AS number for BGP anycast nodes, and
their nodes are placed in the Internet. Reachability of each nodes
are served by BGP routing protocol [8].
Each anycast nodes propagate the routing information of the shared IP
address block and AS number by BGP. Each BGP routers in the Internet
choose 'nearest' node by BGP's best route selection algorithm. That
is, the accesses to the shared IP address block will be distributed
to the each anycast nodes, depending on the clients' locations and
network topologies.
BGP anycast can control each anycast nodes by configuring as 'local
node' or 'global node' using BGP's routing framework. Concretely,
using the 'NO_EXPORT' [9] or 'NOPEER' [11] community, the 'local
node' operators can limit distributing the routing information of
anycast node only for the directly peering sites. Therefore, the
'local node' can localize the access to anycast node from directly
peering sites. On the other hand, the 'global node' operators apply
the normal BGP anycast for its node.
In this memo, authors focus on 'global node' as main target, but
authors believe it can be applied as 'local node' too.
When one of BGP anycast nodes goes down, routing informations will be
automatically recalculated. The datagrams to the anycast node are
Morishita, et al. Expires September 28, 2006 [Page 3]
Internet-Draft Anycast Node Requirements March 2006
automatically rerouted to other anycast nodes. Thus, BGP anycast can
provide redundancy for the Internet services.
Current BGP anycast is hard to apply for longer-lived transaction
service, because the instability of the dynamic routing protocol is
harmful for them. But most of the DNS queries and answers are based
on a single UDP packet, so, DNS is considered to be one of typical
service in which BGP anycast can be applied.
In this memo, authors focus the critical DNS infrastructure,
especially root and TLD DNS servers. So, authors only focus a case
of "a single IP address for DNS service covered by a single exported
route". It means advertisement and withdrawal of a single covering
prefix can be tightly coupled to the availability of the single
associated service, as described "4.8.1. Multiple Covering Prefixes"
of Anycast BCP.
In this situation, DNS service providers need an exclusive IP address
block which is a provider independent CIDR block and exclusive an AS
number for making each anycast nodes.
3. Requirements and Preconditions for Critical DNS Server Nodes
As described before, IP anycast is one of the effective ways for
improving the robustness and the reliability of service. And in
recent critical authoritative name servers, especially for large
TLDs, ability to handle more data than before, more frequent data
updating, and higher reliability are required.
So, when BGP anycast technology is applied to their critical servers,
carefulness must be necessary for their operators. Authors expect
that the requirements and preconditions which described by this memo
would be useful.
In this section, this memo describes requirements and preconditions
for making BGP anycast nodes for authoritave name servers in the
following two points of view, the Internet access service, and data
center.
3.1. Choosing the Internet Access Service
For making BGP anycast nodes distributed in the wide area, it is
important to make network environment with geographical and network
topological diversity.
In case of making such network environment, each anycast nodes should
have Internet connectivity from different Internet access service
Morishita, et al. Expires September 28, 2006 [Page 4]
Internet-Draft Anycast Node Requirements March 2006
provider (hereafter, called ISP) for ensuring network diversity.
And in case of ensuring the BGP connectivity, the owner of the
authoritative name servers must consider the following preconditions
and requirements to choose the Internet access services.
3.1.1. Reliability of the Backbone Network
When making a critical authoritative name server, higher reliability
for ISP's network itself is needed. For implementing this, it is
desirable for ISP itself to have owned and managed its backbone
network.
In general an ISP, which owns and manages the backbone network
itself, is expected to have stronger responsibility for network
stability. Then it is expectable that the stability of a network is
higher. Of course, it is not absolute requirement, but it will
surely be one of the important elements.
3.1.2. Connectivity of Outside Area
In case of critical authoritative name servers, such as served for
root and/or TLD zones, there are many accesses from not only its
country and its local area but also outside area. Thus, connectivity
for them must be needed. In the same reason of Section 3.1.1, it is
desirable that an ISP which owns and manages the stable connectivity
of all areas.
3.1.3. Peering
When ensuring highly reliable Internet connectivity, it is an
important element for ensuring the diversity of Internet routes
including multiple alternative paths. Moreover, providing DNS
service to multiple ISP networks efficiently, it is desirable for the
ISP to have many BGP peers with other ISPs, and they are much stable.
3.1.4. Connectivity for Provider Independent CIDR Block and AS Number
When making a set of BGP anycast node for critical authritative name
servers, a provider independent CIDR block and an AS number must be
prepare in advance, and they must be used for DNS service at each
anycast nodes. It is also needed for making the multihomed
connectivity.
In this case, the ISP must support propagating CIDR block and AS
number for anycasting service to the Internet widely, and the ISP
must provide connectivity for them from the Internet. Concretely,
the ISP must provide "transit" service.
Morishita, et al. Expires September 28, 2006 [Page 5]
Internet-Draft Anycast Node Requirements March 2006
3.1.5. Connectivity for Administration and Data Synchronization
As in RFC 3258, an Internet connectivity which is different for IP
anycasting must be needed for anycast node administration. And as in
Anycast BCP, the synchronisation of data between anycast nodes will
involve transactions between non-anycast addresses.
3.1.6. Connectivity for IPv6
RFC 3513 [10] prohibited host-based anycast in IPv6. But new version
of RFC 4291 [12] removes this limitation and it RFC 3513 is generally
obsoleted.
So, now there are no protocol limitations for using IP anycasting for
IPv6 network by using the same technology as IPv4. But authors think
that more experiences are needed for deployment of IPv6 anycasting.
That is, the anycast node owner should ensure IPv6 connectivity.
3.2. Choosing the Location
For choosing the BGP anycast node locations for critical
authoritative name servers, RFC 2182 and 2870 can be refered for
useful guidance on appropriate practices for them. By referencing
them, when choosing the location for BGP anycast node for critical
authoritative name servers, the owner of them must consider the
following preconditions and requirements.
3.2.1. Providing Higher Security Level
To realize higher defense performance to physical destruction and/or
the intrusion from the outside, the location must provide higher
security level.
3.2.2. Providing Higher Redundancy of Electrical Power Supply
DNS service requires high continuity and stability, the location must
provide higher redundancy of electrical power supply and urgent power
supply equipment for emergency.
3.2.3. Providing Higher Tolerance Against Disasters
For the same reason of Section 3.2.2, the location must provide
higher tolerance against disasters, for example fire, earthquake and
other disasters.
Morishita, et al. Expires September 28, 2006 [Page 6]
Internet-Draft Anycast Node Requirements March 2006
3.2.4. Providing the Diversity of Locations
For ensuring tolerance and redundancy, the diversity of locations is
needed. Concretely, even if a fatal disaster occurred at one
location, the continuity of critical DNS service must be ensured.
4. Cost and Operational Issue
In the technical point of view, BGP anycast nodes can be made in
numbers of locations. But it is not realistic to prepare them more
than necessity. In general, to satisfy the preconditions and
requirements which is previously described, BGP anycast node needs
high cost, including financial, human and operational resources.
In the current condition, this cost is mandatory for making BGP
anycast node. Especially, to guarantee the quality of service, for
example SLA (Service Level Agreement), needs higher cost than normal
Internet connectivity. This is one of big burden for operating BGP
anycast node. The authors believe that this is one of in the future
issue for deploying IP anycasting.
Furthermore, for administrating remote anycast nodes smoothly, many
human recources are needed, including local and remote technical
staffs and operators. When making BGP anycast node for critical DNS
service, the owner of authoritative name servers must consider about
this issue.
5. Measurement Issue
To evaluate the practical effect of IP anycast, for example, to
verify whether the selections of the IP anycast nodes are appropriate
or not, objective measurement is very important. When making BGP
anycast mesh in the wide area, the measurement must also be carried
out in the wide area.
In such case, there is an effective guideline defined by ICANN,
called "CNNP test" [13]. This guideline is useful for making
critical DNS server nodes.
But the cost of the measurement is very high, and it is hard to make
and maintain for many measurement points/probes.
The continuity is one of an important points for measurement. And
operators should verify that the continuity of DNS service is ensured
by measurement. RIPE NCC's DNSMON service [14] is one of typical
notable project.
Morishita, et al. Expires September 28, 2006 [Page 7]
Internet-Draft Anycast Node Requirements March 2006
6. Our Findings through Making Oversea Anycast nodes
In this section, this memo describes our findings for making oversea
IP anycast nodes for our DNS server. At this point, these server
nodes are still for reserach and development, but when they were
made, we did some useful experiences and got some useful findings.
Authors hope that it is useful for DNS operators that they will walk
on the same way in the future, especially for TLD operators.
6.1. Running Cost is More Dominant
Running cost is more dominant than initial cost. Human resources and
traveling expense for troubleshooting and trouble recovery.
Especially for oversea site, sometimes higher the wall of language
exists. For instance, a simply replacement sometimes may reduce the
total cost from fixing by using remote hands.
6.2. Difference of Custom Practices
Difference of custom of each country is sometimes imporatant issue
for practical server node operation. Some data center requires
damage insurance contract. But it is hard to have contract with
foreign customer. As a result, we had to contact and negotiate a lot
of insurance companies.
6.3. Others to Remind
During our operations, we encountered some unexpected trouble. In
this section, this memo describes the term of them.
6.3.1. Thermal Overheat
Thermal overheat is occurred due to cabinet placement and
ventilation. So, we had to order rack rotation work and additional
cabinet fans.
6.3.2. Broken Hardware Replacement
Due to our contract of agent, broken hardware replacement must be
"parts-based", not a whole of server hardware. So we had to need
additional work for fixing server at oversea place. We cannot order
the remote hands because the work is so complex and difficult.
7. Other Considerations and Issues
In this section, this memo describes other related considerations and
issues for making BGP anycast nodes for critical authoritative name
Morishita, et al. Expires September 28, 2006 [Page 8]
Internet-Draft Anycast Node Requirements March 2006
servers. Authors will describe requirements and preconditions for
these issues, but not yet issued.
o Selection of Node Locations
o Selection of Server Hardware
o Selection of Server Software
o Selection of Remote Maintenance Tools
o Effective Remote Maintenance
o Effective Measurement
8. Security Considerations
IP anycast mitigates Denial-of-Service attack effect and constrains
Distributed Denial-of-Service attack in their local network. It is
one of most important goals of IP anycast.
To keeping higher security level of each DNS server nodes is one of
most important points of managing critical authrotative name servers.
Of couese, it is the same as non-anycasted DNS service, but in IP
anycasting environment, all of IP anycast nodes have the same IP
address for authoritative DNS service, so, it means it is much
important than single server because all of nodes should be applied
the same higher security policy.
In IP anycasting environment, number of nodes increases compared with
non-anycasting environment. It means the place where the safety of
data must be guaranteed also increases. And practical secure data
synchronization method between nodes must be required, typically data
encryption.
9. IANA Considerations
This document requests no action from IANA.
10. Acknowledgements
Paul Vixie and Bill Manning reviewed a previous version of this
document. Joe Abley reviewed a previous version of this document and
provided detailed and useful comments.
Yoshiki Ishida, Tomoya Yoshida, George Michaelson and Peter Koch
provided some useful comments and suggestions.
The authors acknowledge many helpful suggestions from the members of
JPRS Research and Development Department and System administration
Morishita, et al. Expires September 28, 2006 [Page 9]
Internet-Draft Anycast Node Requirements March 2006
Department.
This memo is included in the results of the research activities
funded by National Institute of Information and Communications
Technology (NICT).
11. References
[1] Partridge, C., Mendez, T., and W. Milliken, "Host Anycasting
Service", RFC 1546, November 1993.
[2] Hardie, T., "Distributing Authoritative Name Servers via Shared
Unicast Addresses", RFC 3258, April 2002.
[3] Abley, J. and K. Lindqvist, "Operation of Anycast Services",
draft-ietf-grow-anycast-03.txt (work in progress),
January 2006.
[4] Elz, R., Bradner, S., and M. Patton, "Selection and Operation
of Secondary DNS Servers", RFC 2182, July 1997.
[5] Bush, R., Karrenberg, D., Kosters, M., and R. Plzak, "Root Name
Server Operational Requirements", RFC 2870, June 2000.
[6] Mockapetris, P., "DOMAIN NAMES - CONCEPTS AND FACILITIES",
RFC 1034, November 1987.
[7] Mockapetris, P., "DOMAIN NAMES - IMPLEMENTATION AND
SPECIFICATION", RFC 1035, November 1987.
[8] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)",
RFC 1771, March 1995.
[9] Chen, E. and J. Stewart, "A Framework for Inter-Domain Route
Aggregation", RFC 2519, February 1999.
[10] Hinden, R. and S. Deering, "Internet Protocol Version 6 (IPv6)
Addressing Architecture", RFC 3513, April 2003.
[11] Huston, G., "NOPEER Community for Border Gateway Protocol (BGP)
Route Scope Control", RFC 3765, April 2004.
[12] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, February 2006.
[13] "ICANN Unsponsored TLD Agreement: Appendix D (.info)",
May 2001.
Morishita, et al. Expires September 28, 2006 [Page 10]
Internet-Draft Anycast Node Requirements March 2006
[14] "RIPE-NCC/DNS Server Monitoring", <http://dnsmon.ripe.net/>.
Morishita, et al. Expires September 28, 2006 [Page 11]
Internet-Draft Anycast Node Requirements March 2006
Authors' Addresses
Yasuhiro Orange Morishita
Research and Development Department, Japan Registry Services Co.,Ltd.
Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo 101-0065
Japan
Email: yasuhiro@jprs.co.jp
Shinta Sato
System Administration Department, Japan Registry Services Co.,Ltd.
Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo 101-0065
Japan
Email: shinta@jprs.co.jp
Takayasu Matsuura
System Administration Department, Japan Registry Services Co.,Ltd.
Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo 101-0065
Japan
Email: matuura@jprs.co.jp
Morishita, et al. Expires September 28, 2006 [Page 12]
Internet-Draft Anycast Node Requirements March 2006
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2006). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Morishita, et al. Expires September 28, 2006 [Page 13]
