Internet DRAFT - draft-li-arch-sat

draft-li-arch-sat







Network Working Group                                              T. Li
Internet-Draft                                          Juniper Networks
Intended status: Informational                          21 February 2024
Expires: 24 August 2024


             A Routing Architecture for Satellite Networks
                          draft-li-arch-sat-06

Abstract

   Satellite networks present some interesting challenges for packet
   networking.  The entire topology is continually in motion, with links
   that are far less reliable than what is common in terrestrial
   networks.  Some changes to link connectivity can be anticipated due
   to orbital mechanics.

   This document proposes a scalable routing architecture for satellite
   networks based on existing routing protocols and mechanisms, enhanced
   with scheduled link connectivity change information.  This document
   proposes no protocol changes.

   This document presents the author's view and is neither the product
   of the IETF nor a consensus view of the community.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 24 August 2024.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.





Li                       Expires 24 August 2024                 [Page 1]

Internet-Draft           Routing for Satellites            February 2024


   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Related Work  . . . . . . . . . . . . . . . . . . . . . .   3
     1.2.  Terms and Abbreviations . . . . . . . . . . . . . . . . .   3
   2.  Overview  . . . . . . . . . . . . . . . . . . . . . . . . . .   5
     2.1.  Topological Considerations  . . . . . . . . . . . . . . .   5
     2.2.  Link Changes  . . . . . . . . . . . . . . . . . . . . . .   6
     2.3.  Scalability . . . . . . . . . . . . . . . . . . . . . . .   6
     2.4.  Assumptions . . . . . . . . . . . . . . . . . . . . . . .   7
       2.4.1.  Traffic Patterns  . . . . . . . . . . . . . . . . . .   7
       2.4.2.  User Station Contraints . . . . . . . . . . . . . . .   8
       2.4.3.  Stochastic Connectivity . . . . . . . . . . . . . . .   8
     2.5.  Problem Statement . . . . . . . . . . . . . . . . . . . .   9
   3.  Forwarding Plane  . . . . . . . . . . . . . . . . . . . . . .   9
   4.  IGP Suitability and Scalability . . . . . . . . . . . . . . .  10
   5.  Stripes and Areas . . . . . . . . . . . . . . . . . . . . . .  11
   6.  Traffic Forwarding and Traffic Engineering  . . . . . . . . .  12
   7.  Scheduling  . . . . . . . . . . . . . . . . . . . . . . . . .  14
   8.  Future Work . . . . . . . . . . . . . . . . . . . . . . . . .  16
   9.  Deployment Considerations . . . . . . . . . . . . . . . . . .  16
   10. Security Considerations . . . . . . . . . . . . . . . . . . .  16
   11. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  16
   12. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  17
   13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  17
     13.1.  Normative References . . . . . . . . . . . . . . . . . .  17
     13.2.  Informative References . . . . . . . . . . . . . . . . .  18
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  20

1.  Introduction

   Satellite networks present some interesting challenges for packet
   networking.  The entire topology is continually in motion, with links
   that are far less reliable than what is common in terrestrial
   networks.  Some changes to link connectivity can be anticipated due
   to orbital mechanics.






Li                       Expires 24 August 2024                 [Page 2]

Internet-Draft           Routing for Satellites            February 2024


   This document proposes a scalable routing architecture for satellite
   networks based on existing routing protocols and mechanisms, enhanced
   with scheduled link connectivity change information.  This document
   proposes no protocol changes.

   Large-scale satellite networks are being deployed, presenting an
   unforeseen application for conventional routing protocols.  The high
   rate of intentional topological change coupled with the extreme scale
   are unprecedented in terrestrial networking.  Links between
   satellites can utilize free-space optics technology that allows
   liberal connectivity, but there are limitations due to the range of
   the links and conjunction with the sun, resulting in links that are
   far less reliable than network designers are used to.  In addition,
   links can change their endpoints dynamically, resulting in structural
   changes to the topology.

   This document proposes one approach to provide a routing architecture
   for such networks utilizing current routing technology and providing
   a solution for the scalability of the network while incorporating the
   rapid rate of topological change.

   This document presents the author's view and is neither the product
   of the IETF nor a consensus view of the community.

1.1.  Related Work

   A survey of related work can be found in [Westphal].  Link state
   routing for satellite networks has been considered in [Cao] and
   [Zhang].

1.2.  Terms and Abbreviations

   *  Constellation: A set of satellites.

   *  Downlink: The half of a ground link leading from a satellite to an
      Earth station.

   *  Gateway: An Earth station that participates as part of the network
      and acts as the interconnect between satellite constellations and
      the planetary network.  Gateways have a much higher bandwidth than
      user stations, have ample computing capabilities, and perform
      traffic engineering duties, subsuming the functionality of a
      network controller or Path Computation Element (PCE).  [RFC4655]
      Multiple gateways are assumed to exist, with each serving a
      portion of the network.






Li                       Expires 24 August 2024                 [Page 3]

Internet-Draft           Routing for Satellites            February 2024


   *  GEO: Geostationary Earth Orbit.  A satellite in GEO has an orbit
      that is synchronized to planetary rotation, so it effectively sits
      over one spot on the planet.

   *  Ground link: A link between a satellite and an Earth station.

   *  Earth station: A node in the network that is on or close to the
      surface of the planet and has a link to a satellite.  This
      includes ships, aircraft, and other vehicles below LEO.  [ITU]

   *  IGP: Interior Gateway Protocol.  A routing protocol that is used
      within a single administrative domain.  Note that 'gateway' in
      this context is semantically equivalent to 'router' and has no
      relationship to the 'gateway' used in the rest of this document.

   *  IS-IS: Intermediate System to Intermediate System routing
      protocol.  An IGP that is commonly used by service providers.
      [ISO10589] [RFC1195]

   *  ISL: Inter-satellite link.  Frequently implemented with free-space
      optics that allow signaling using photons without any intervening
      medium.  [Bell]

   *  L1: IS-IS Level 1

   *  L1L2: IS-IS Level 1 and Level 2

   *  L2: IS-IS Level 2

   *  LEO: Low Earth Orbit.  A satellite in LEO has an altitude of
      2,000km or less.

   *  Local gateway: Each user station is associated with a single
      gateway in its region.

   *  LSP: IS-IS Link State Protocol Data Unit.  An LSP is a set of
      packets that describe a node's connectivity to other nodes.

   *  MEO: Medium Earth Orbit.  A satellite in MEO is between LEO and
      GEO orbits and has an altitude between 2,000km and 35,786km.

   *  SID: Segment Identifier [RFC8402]

   *  Stripe: A set of satellites in a few adjacent orbits.  These form
      an IS-IS L1 area.

   *  SR: Segment Routing [RFC8402]




Li                       Expires 24 August 2024                 [Page 4]

Internet-Draft           Routing for Satellites            February 2024


   *  Uplink: The half of a link leading from an Earth station to a
      satellite.

   *  User station: An Earth station interconnected with a small end-
      user network.

2.  Overview

2.1.  Topological Considerations

   Satellites travel in specific orbits around their parent planet.
   Some of them have their orbital periods synchronized to the rotation
   of the planet, so they are effectively stationary over a single
   point.  Other satellites have orbits that cause them to travel across
   regions of the planet gradually or quite rapidly.  Respectively,
   these are typically known as Geostationary Earth Orbits (GEO), Medium
   Earth Orbit (MEO), or Low Earth Orbit (LEO), depending on altitude.
   This discussion is not Earth-specific; as we get to other planets, we
   can test this approach's generality.

   Satellites may have data interconnections with one another through
   Inter-Satellite Links (ISLs).  Due to differences in orbits, ISLs may
   be connected temporarily, with periods of potential connectivity
   computed through orbital mechanics.  Multiple satellites may be in
   the same orbit but separated in space, with a roughly constant
   separation.  Satellites in the same orbit may have ISLs that have a
   higher duty cycle than ISLs between different orbits but are still
   not guaranteed to always be connected.

     User           +-----------------+    Local
     Stations   --- |   Satellites    |--- Gateway --- Internet
                    +-----------------+

                   Figure 1: Overall network architecture

   Earth stations can communicate with one or more satellites that are
   in their region.  User stations are Earth stations that have a
   limited capacity and communicate with only a single satellite at a
   time.  Other Earth stations that may have richer connectivity and
   higher bandwidth are commonly called gateways and provide
   connectivity between the satellite network and conventional wired
   networks.  Gateways serve user stations that are in their geographic
   proximity and are replicated across the globe as necessary to provide
   coverage and meet service density goals.  User stations are
   associated with a single local gateway.  Traffic from one Earth
   station to another may need to traverse a path across multiple
   satellites via ISLs.




Li                       Expires 24 August 2024                 [Page 5]

Internet-Draft           Routing for Satellites            February 2024


2.2.  Link Changes

   Like conventional network links, ISLs and ground links can fail at
   any time.  However, unlike conventional links, there are predictable
   times when ISLs and ground links can potentially connect and
   disconnect.  These predictions can be computed and cataloged in a
   schedule that can be distributed to relevant network elements.
   Predictions of a link connecting are not a guarantee: a link may not
   connect for a variety of reasons.  Predictions of a link
   disconnecting due to orbital mechanics are effectively guaranteed, as
   the underlying physics is extremely unlikely to improve unexpectedly.

2.3.  Scalability

   Some proposed satellite networks are fairly large, with tens of
   thousands of proposed satellites.  [CNN] A key concern is the ability
   to reach this scale and larger, as useful networks tend to grow.

   As we know, the key to scalability is the ability to create
   hierarchical abstractions, so a key question of any routing
   architecture will be about the abstractions that can be created to
   contain topological information.

   Normal routing protocols are architected to operate with a static but
   somewhat unreliable topology.  Satellite networks lack the static
   organization of terrestrial networks, so normal architectural
   practices for scalability may not apply and alternative approaches
   may need consideration.

   In a traditional deployment of a link-state routing protocol, current
   implementations can be deployed with a single area that spans a few
   thousand routers.  A single area would also provide no isolation for
   topological changes, causing every link change to be propagated
   throughout the entire network.  This would be insufficient for the
   needs of large satellite networks.

   Multiple areas or multiple instances of an IGP can be used to improve
   scalability, but there are limitations to traditional approaches.

   The IETF currently actively supports two link-state Interior Gateway
   Protocols (IGPs): OSPF [RFC2328][RFC5340] and IS-IS.

   OSPF requires that the network operate around a backbone area, with
   subsidiary areas hanging off of the backbone.  While this works well
   for traditional terrestrial networks, this does not seem appropriate
   for satellite networks, where there is no centralized portion of the
   topology.




Li                       Expires 24 August 2024                 [Page 6]

Internet-Draft           Routing for Satellites            February 2024


   IS-IS has a different hierarchical structure, where Level 1 (L1)
   areas are connected sets of nodes, and then Level 2 (L2) is a
   connected subset of the topology that intersects all of the L1 areas.
   Individual nodes can be L1, L2, or both (L1L2).  Traditional IS-IS
   designs require that any node or link that is to be used as transit
   between L2 areas must appear as part of the L2 topology.  In a
   satellite network, any satellite could end up being used for L2
   transit, and so every satellite and link would be part of L2,
   negating any scalability benefits from IS-IS's hierarchical
   structure.

   We elaborate on IS-IS-specific considerations in Section 4.

2.4.  Assumptions

   In this section, we discuss some of the assumptions that are the
   basis for this architectural proposal.

   The data payload is IP packets.

   Satellites are active participants in the control and data plane for
   the network, participating in protocols, and forwarding packets.

   There may be a terrestrial network behind each gateway that may
   interconnect to the broader Internet.  The architecture of the
   terrestrial network is assumed to be a typical IS-IS and BGP
   [RFC4271] deployment and is not discussed further.

   The satellite network interconnects user stations and gateways.
   Interconnection between the satellite network and the satellite
   networks of other network operators is outside of the scope of this
   document.

2.4.1.  Traffic Patterns

   We assume that the primary use of the satellite network is to provide
   access from a wide range of geographic locations.  We assume that
   providing high-bandwidth bulk transit between peer networks is not a
   goal.  It has been noted that satellite networks can provide lower
   latencies than terrestrial fiber networks [Handley].  This proposal
   does not preclude such applications but also does not articulate the
   mechanisms necessary for user stations to perform the necessary
   traffic engineering computations.  Low-latency, multicast, and
   anycast applications are not discussed further.

   As with most access networks, we assume that there will be
   bidirectional traffic between the user station and the gateway, but
   that the bulk of the traffic will be from the gateway to the user



Li                       Expires 24 August 2024                 [Page 7]

Internet-Draft           Routing for Satellites            February 2024


   station.  We expect that the uplink from the gateway to the satellite
   network to be the bandwidth bottleneck, and that gateways will need
   to be replicated to scale the uplink bandwidth, as the satellite
   capacity reachable from a gateway will be limited.

   We assume that it is not essential to provide optimal routing for
   traffic from user station to user station.  If this traffic is sent
   first to a gateway and then back into the satellite network, this
   might be acceptable to some operators as long as the traffic volume
   remains very low.  This type of routing is not discussed further.

   We assume that traffic for a user station should enter the satellite
   network through a gateway that is in some close geographic proximity
   to the user station.  This is to reduce the number of ISLs used by
   the path to the user station.  Similarly, we assume that user station
   traffic should exit the satellite network through the gateway that is
   in the closest geographic proximity to the user station.
   Jurisdictional requirements for landing traffic in certain regions
   may alter these assumptions, but such situations are outside of the
   scope of this document.

   This architecture does not preclude gateway-to-gateway traffic across
   the satellite constellations, but it does not seek to optimize it.

2.4.2.  User Station Contraints

   The user station is an entity whose operation is conceptually shared
   between the satellite constellation operator and the operator of the
   cluster of end stations it serves.  For example, the user station is
   trusted to attach MPLS label stacks to end-user packets.  It gets the
   information to do so from some combination of its direct satellite
   and its local gateway, via protocols outside the scope of this
   document.  Equally, it bootstraps communication via an exchange with
   the current local satellite so it can find and communicate with its
   local gateway, again with the details of how that is done being
   outside the scope of this document.

   User stations that can concurrently access multiple satellites are
   not precluded by this proposal, but are not discussed in detail.

2.4.3.  Stochastic Connectivity

   We assume that links in general will be available when scheduled.  As
   with any network, there will be failures, and the schedule is not a
   guarantee, but we also expect that the schedule is mostly accurate.
   We assume that at any given instant, there are enough working links
   and aggregate bandwidth to run the network and support the traffic
   demand.  If this assumption does not hold, then no routing



Li                       Expires 24 August 2024                 [Page 8]

Internet-Draft           Routing for Satellites            February 2024


   architecture can magically make the network more capable.

   Satellites that are in the same orbit may be connected by ISLs.
   These are called intra-orbit ISLs.  Satellites that are in different
   orbits may also be connected by ISLs.  These are called inter-orbit
   ISLs.  We assume that, in general, intra-orbit ISLs have higher
   reliability and persistence than inter-orbit ISLs.

   We assume that the satellite network is connected (in the graph
   theory sense) almost all the time, even if some links are down.  This
   implies that there is almost always some path to the destination.  In
   the extreme case where there is no such path, we assume that it is
   acceptable to drop the payload packets.  We do not require buffering
   of traffic when a link is down.  Instead, traffic should be rerouted.

2.5.  Problem Statement

   The goal of the routing architecture is to provide an organizational
   structure to protocols running on the satellite network such that
   topology information is conveyed through relevant portions of the
   network, that paths are computed across the network, and that data
   can be delivered along those paths, and the structure can scale to a
   very large network without any changes to the organizational
   structure.

3.  Forwarding Plane

   The end goal of a network is to deliver traffic.  In a satellite
   network where the topology is in a continual state of flux and the
   user stations are frequently changing their association with the
   satellites, having a highly flexible and adaptive forwarding plane is
   essential.  Toward this end, we propose to use MPLS as the
   fundamental forwarding plane architecture [RFC3031].  Specifically,
   we propose to use a Segment Routing (SR) [RFC8402] based approach
   with an MPLS data plane [RFC8660], where each satellite is assigned a
   node Segment Identifier (SID).  This allows the architecture to
   support both IPv4 and IPv6 concurrently.  A path through the network
   can be then expressed as a label stack of node SIDs.  IP forwarding
   is not used within the internals of the satellite network, although
   each satellite may be assigned an IP address for management purposes.
   Existing techniques may be used to limit the size of the SR label
   stack so that it only contains the significant waypoints along the
   path.[Giorgetti] This implies that the label stack operates as a form
   of loose-source routing through the network.  If there is an
   unexpected topology change in the network, then the IGP will compute
   a new path to the next waypoint, allowing packet delivery despite ISL
   failures.  While the IGP is converging, there may be micro-loops in
   the topology.  These can be avoided through the use of TI-LFA



Li                       Expires 24 August 2024                 [Page 9]

Internet-Draft           Routing for Satellites            February 2024


   alternate paths [I-D.ietf-rtgwg-segment-routing-ti-lfa], or traffic
   will loop until it is discarded based on its TTL.

   We assume that there is a link-layer mechanism for a user station to
   associate with a satellite.  User stations will have an IP address
   that is assigned from a prefix managed by its local gateway.  The
   mechanisms for this assignment and its communication to the end
   station are not discussed herein but might be similar to DHCP
   [RFC2131].  User station IP addresses change infrequently and do not
   reflect their association with their first-hop satellite.  Gateways
   and their supporting terrestrial networks advertise prefixes covering
   all of its local user stations into the global Internet.

   User stations may be assigned a node SID, in which case MPLS
   forwarding can be used for all hops to the user station.
   Alternatively, if the user station does not have a node SID, then the
   last hop from the satellite to the end station can be performed based
   on the destination IP address of the packet.  This does not require a
   full longest-prefix-match lookup as the IP address is merely a unique
   identifier at this point.

   Similarly, gateways may be assigned a node SID.  A possible
   optimization is that a single SID value be assigned as a global
   constant to always direct traffic to the topologically closest
   gateway.  If traffic engineering is required for traffic that is
   flowing to a gateway, a specific path may be encoded in a label stack
   that is attached to the packet by the user station or by the first-
   hop satellite.

   Gateways can also perform traffic engineering by using different
   paths and label stacks for different traffic flows.  Routing a single
   traffic flow across multiple paths has proven to cause performance
   issues with transport protocols, so that approach is not recommended.
   Traffic engineering is discussed further in Section 6.

4.  IGP Suitability and Scalability

   As discussed in Section 2.3, IS-IS is architecturally the best fit
   for satellite networks, but does require some novel approaches to
   achieve the scalability goals for a satellite network.  In
   particular, we propose that all nodes in the network be L1L2 so that
   local routing is done based on L1 information and then global routing
   is done based on L2 information.

   IS-IS has the interesting property that it does not require interface
   addresses.  This feature is commonly known as 'unnumbered
   interfaces'.  This is particularly helpful in satellite topologies
   because it implies that ISLs may be used flexibly.  Sometimes an



Li                       Expires 24 August 2024                [Page 10]

Internet-Draft           Routing for Satellites            February 2024


   interface might be used as an L1 link to another satellite and a few
   orbits later it might be used as an L1L2 link to a completely
   different satellite without any reconfiguration or renumbering.

   Scalability for IS-IS can be achieved through the use of a proposal
   known as Area Proxy [I-D.ietf-lsr-isis-area-proxy].  With this
   proposal, all of the nodes in an L1 area combine their information
   into a single L2 Link State Protocol Data Unit (LSP).  This implies
   that the size of the L1 Link State Database (LSDB) scales as the
   number of nodes in the L1 area and the size of the L2 LSDB scales
   with the number of L1 areas.

   With Area Proxy, topological changes within an L1 area will not be
   visible to other areas, so the overhead of link state changes will be
   greatly reduced.

   The Area Proxy proposal also includes the concept of an Area SID.
   This is useful because it allows traffic engineering to construct a
   path that traverses areas with a minimal number of label stack
   entries.

   Suppose, for example, that a network has 1,000 L1 areas, each with
   1,000 satellites.  This would then mean that the network supports
   1,000,000 satellites, but only requires 1,000 entries in its L1 LSDB
   and 1,000 entries in its L2 LSDB; numbers that are easily achievable
   today.  The resulting MPLS label table would contain 1,000 node SIDs
   from the L1 (and L2) LSDB and 1,000 area SIDs from the L2 LSDB.  If
   each satellite advertises an IP address for management purposes, then
   the IP routing table would have 1,000 entries for the L1 management
   addresses and 1,000 area proxy addresses from L2.

   In this proposal, IS-IS does not carry any IP routes other than those
   that are in the satellite topology.  In particular, there are no IP
   routes for user stations or the remainder of the Internet.

5.  Stripes and Areas

   A significant problem with any link state routing protocol is that of
   area partition.  While there have been many proposals for automatic
   partition repair, none has seen significant production deployment.
   It seems best to simply avoid this issue altogether and ensure that
   areas have an extremely low probability of partitioning.

   As discussed above, intra-orbit ISLs are assumed to have higher
   reliability and persistence than inter-orbit ISLs.  However, even
   intra-orbit ISLs are not sufficiently reliable to avoid partition
   issues.  Therefore, we propose to group a small number of adjacent
   orbits as an IS-IS L1 area, called a stripe.  We assume that for any



Li                       Expires 24 August 2024                [Page 11]

Internet-Draft           Routing for Satellites            February 2024


   given reliability requirement, there is a small number of orbits that
   can be used to form a stripe that satisfies the reliability
   requirement.

   Stripes are connected to other adjacent stripes using the same ISL
   mechanism, forming the L2 topology of the network.  Each stripe
   should have multiple L2 connections and should never become
   partitioned from the remainder of the network.

   By using a stripe as an L1 area, in conjunction with Area Proxy, the
   overhead of the architecture is greatly reduced.  Each stripe
   contributes a single LSP to the L2 LSDB, completely hiding all of the
   details about the satellites within the stripe.  The resulting
   architecture scales proportionately to the number of stripes
   required, not the number of satellites.

   Groups of MEO and GEO satellites with interconnecting ISLs can also
   form an IS-IS L1L2 area.  Satellites that lack intra-constellation
   ISLs are better as independent L2 nodes.

6.  Traffic Forwarding and Traffic Engineering

   Forwarding in this architecture is straightforward.  A path from a
   gateway to a user station on the same stripe only requires a single
   node SID for the satellite that provides the downlink to the user
   station.

                \
    Gateway -->  X
                  \
                   \
                    X
                     \
                      \
                       X ---> x  User Station
                        \

                            Figure 2: On-stripe forwarding

   Similarly, a user station returning a packet to a gateway need only
   provide a gateway node SID.

   For off-stripe forwarding, the situation is a bit more complex.  A
   gateway would need to provide the area SID of the final stripe on the
   path plus the node SID of the downlink satellite.  For return
   traffic, user stations or first-hop satellites would want to provide
   the area SID of the stripe that contains the satellite that provides
   access to the gateway as well as the gateway SID.



Li                       Expires 24 August 2024                [Page 12]

Internet-Draft           Routing for Satellites            February 2024


    Source S
       |
       |
       V
    Internet
       |
       |
       V          \
    Gateway L -->  X
                    \
                     \     \
                      X --- X
                       \     \
                              \     \    Area A
                               X --- X
                                \     \
                                       \
                                        U ---> D   User Station
                                         \

                            Figure 3: Off-stripe forwarding

   As an example, consider a packet from an Internet source S to a user
   station D.  A local gateway L has injected a prefix covering D into
   BGP and advertised it globally.  The packet is forwarded to L using
   IP forwarding.  When L receives the packet, it performs a lookup in a
   pre-computed forwarding table.  This contains a SID list for the user
   station that has already been converted into a label stack.  Suppose
   that the user station is currently associated with a different stripe
   so that the label stack will contain an area label A and a label U
   for the satellite associated with the user station, resulting in a
   label stack (A, U).

   The local gateway forwards this into the satellite network.  The
   first-hop satellite now forwards based on the area label A at the top
   of the stack.  All area labels are propagated as part of the L2
   topology.  This forwarding continues until the packet reaches a
   satellite that is adjacent to the destination area.  That satellite
   pops label A, removing that label and forwarding the packet into the
   destination area.

   The packet is now forwarded based on the remaining label U, which was
   propagated as part of the L1 topology.  The last satellite forwards
   the packet based on the destination address D and forwards the packet
   to the user station.






Li                       Expires 24 August 2024                [Page 13]

Internet-Draft           Routing for Satellites            February 2024


   The return case is similar.  The label stack, in this case, consists
   of a label for the local gateway's stripe/area, A', and the label for
   the local gateway, L, resulting in the stack (A', L).  The forwarding
   mechanisms are similar to the previous case.

   Very frequently, access networks congest due to oversubscription and
   the economics of access.  Network operators can use traffic
   engineering to ensure that they are getting higher efficiency out of
   their networks by utilizing all available paths and capacity near any
   congestion points.  In this particular case, the gateway will have
   information about all of the traffic that it is generating and can
   use all of the possible paths through the network in its topological
   neighborhood.  Since we're already using SR, this is easily done just
   by adding more explicit SIDs to the label stack.  These can be
   additional area SIDs, node SIDs, or adjacency SIDs.  Path computation
   can be performed by Path Computation Elements (PCE).  [RFC4655]

   Each gateway or its PCE will need topological information from all of
   the areas that it will route through.  It can do this by being a
   participant in the IGP either directly, via a tunnel, or another
   delivery mechanism such as BGP-LS [RFC9552].  User stations do not
   participate in the IGP.

   Traffic engineering for traffic into a gateway can also be provided
   by an explicit SR path for the traffic.  This can help ensure that
   ISLs near the gateway do not congest with traffic for the gateway.
   These paths can be computed by the gateway or PCE and then
   distributed to the first-hop satellite or user station, which would
   then apply them to traffic.  The delivery mechanism is outside of the
   scope of this document.

7.  Scheduling

   The most significant difference between terrestrial and satellite
   networks from a routing perspective is that some of the topological
   changes that will happen to the network can be anticipated and
   computed.  Both link and node changes will affect the topology and
   the network should react smoothly and predictably.













Li                       Expires 24 August 2024                [Page 14]

Internet-Draft           Routing for Satellites            February 2024


   The management plane is responsible for providing information about
   scheduled topological changes.  The exact details of how the
   information is disseminated are outside of the scope of this document
   but could be done through a YANG model
   [I-D.united-tvr-schedule-yang].  Scheduling information needs to be
   accessible to all of the nodes that will make routing decisions based
   on the topological changes in the schedule, so information about an
   L1 topological change will need to be circulated to all nodes in the
   L1 area and information about L2 changes will need to propagate to
   all L2 nodes, plus the gateways and PCEs that carry the related
   topological information.

   There is very little reaction that the network should do in response
   to a topological addition.  A link coming up or a node joining the
   topology should not have any functional change until the change is
   proven to be fully operational based on the usual liveness mechanisms
   found within IS-IS.  Nodes may pre-compute their routing table
   changes but should not install them before all of the relevant
   adjacencies are received.  The benefits of this pre-computation
   appear to be very small.  Gateways and PCEs may also choose to pre-
   compute paths based on these changes, but should be careful to not
   install paths using the new parts of the topology until they are
   confirmed to be operational.  If some path pre-installation is
   performed, gateways and PCEs must be prepared for the situation where
   the topology does not become operational and may need to take
   alternate steps instead, such as reverting any related pre-installed
   paths.

   The network may choose to not do any pre-installation or pre-
   computation in reaction to topological additions, at a small cost of
   some operational efficiency.

   Topological deletions are an entirely different matter.  If a link or
   node is to be removed from the topology, then the network should act
   before the anticipated change to route traffic around the expected
   topological loss.  Specifically, at some point before the topology
   change, the affected links should be set to a high metric to direct
   traffic to alternate paths.  This is a common operational procedure
   in existing networks when links are taken out of service, such as
   when proactive maintenance needs to be performed.  This type of
   change does require some time to propagate through the network, so
   the metric change should be initiated far enough in advance that the
   network converges before the actual topological change.  Gateways and
   PCEs should also update paths around the topology change and install
   these changes before the topology change takes place.  The time
   necessary for both IGP and path changes will vary depending on the
   exact network and configuration.




Li                       Expires 24 August 2024                [Page 15]

Internet-Draft           Routing for Satellites            February 2024


   Strictly speaking, changing to a high metric should not be necessary.
   It should be possible for each router to simply exclude the link and
   recompute paths.  However, it seems safer to also change the metric
   and use the IGP methods for indicating a topology change, as this can
   help avoid issues with incomplete information dissemination and
   synchronization.

8.  Future Work

   This architecture needs to be validated by satellite operators, both
   via simulation and operational deployment.  Meaningful simulation
   hinges on the exact statistics of ISL connectivity, and that
   information is not publicly available at present.

9.  Deployment Considerations

   The network behind a gateway is expected to be a normal terrestrial
   network.  Conventional routing architectural principles apply.  An
   obvious approach would be to extend IS-IS to the terrestrial network,
   applying L1 areas as necessary for scalability.

   The terrestrial network may have one or more BGP connections to the
   broader Internet.  Prefixes for user stations should be advertised
   into the Internet near the associated gateway.  If gateways are not
   interconnected by the terrestrial network, then it may be advisable
   to use one autonomous system per gateway as it might simplify the
   external perception of the network and subsequent policy
   considerations.  Otherwise, one autonomous system may be used for the
   entire terrestrial network.

10.  Security Considerations

   This document discusses one possible routing architecture for
   satellite networks.  It proposes no new protocols or mechanisms and
   thus has no new security impact.  Security for IS-IS is provided by
   [RFC5304] and [RFC5310].

   User stations will interact directly with satellites, potentially
   using proprietary mechanisms, and under the control of the satellite
   operator who is responsible for the security of the user station.

11.  Acknowledgements

   The author would like to thank Dino Farinacci, Olivier De jonckere,
   Eliot Lear, Adrian Farrel, Acee Lindem, Erik Kline, Sue Hares, Joel
   Halpern, and Dean Bogdanovic for their comments.





Li                       Expires 24 August 2024                [Page 16]

Internet-Draft           Routing for Satellites            February 2024


12.  IANA Considerations

   This document makes no requests for IANA.

13.  References

13.1.  Normative References

   [I-D.ietf-lsr-isis-area-proxy]
              Li, T., Chen, S., Ilangovan, V., and G. S. Mishra, "Area
              Proxy for IS-IS", Work in Progress, Internet-Draft, draft-
              ietf-lsr-isis-area-proxy-12, 18 January 2024,
              <https://datatracker.ietf.org/doc/html/draft-ietf-lsr-
              isis-area-proxy-12>.

   [ISO10589] International Organization for Standardization,
              "Intermediate System to Intermediate System Intra-Domain
              Routing Exchange Protocol for use in Conjunction with the
              Protocol for Providing the Connectionless-mode Network
              Service (ISO 8473)", ISO/IEC 10589:2002 , November 2002,
              <https://standards.iso.org/ittf/
              PubliclyAvailableStandards/
              c030932_ISO_IEC_10589_2002(E).zip>.

   [RFC3031]  Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
              Label Switching Architecture", RFC 3031,
              DOI 10.17487/RFC3031, January 2001,
              <https://www.rfc-editor.org/info/rfc3031>.

   [RFC5304]  Li, T. and R. Atkinson, "IS-IS Cryptographic
              Authentication", RFC 5304, DOI 10.17487/RFC5304, October
              2008, <https://www.rfc-editor.org/info/rfc5304>.

   [RFC5310]  Bhatia, M., Manral, V., Li, T., Atkinson, R., White, R.,
              and M. Fanto, "IS-IS Generic Cryptographic
              Authentication", RFC 5310, DOI 10.17487/RFC5310, February
              2009, <https://www.rfc-editor.org/info/rfc5310>.

   [RFC8402]  Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
              Decraene, B., Litkowski, S., and R. Shakir, "Segment
              Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
              July 2018, <https://www.rfc-editor.org/info/rfc8402>.

   [RFC8660]  Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S.,
              Decraene, B., Litkowski, S., and R. Shakir, "Segment
              Routing with the MPLS Data Plane", RFC 8660,
              DOI 10.17487/RFC8660, December 2019,
              <https://www.rfc-editor.org/info/rfc8660>.



Li                       Expires 24 August 2024                [Page 17]

Internet-Draft           Routing for Satellites            February 2024


13.2.  Informative References

   [Bell]     Bell, A. G., "On the Production and Reproduction of Sound
              by Light", American Journal of Science Third Series. XX
              (118): 305-324., DOI 10.2475/ajs.s3-20.118.305, October
              1880, <https://zenodo.org/records/1450056>.

   [Cao]      Cao, X., Li, Y., Xiong, X., and J. Wang, "Dynamic Routings
              in Satellite Networks: An Overview", Sensors (Basel,
              Switzerland), 22(12), DOI 10.3390/s22124552, 2022,
              <https://www.mdpi.com/1424-8220/22/12/4552/
              pdf?version=1655449925>.

   [CNN]      Wattles, J., "Elon Musk's SpaceX now owns about a third of
              all active satellites in the sky", February 2021,
              <https://www.cnn.com/2021/02/11/tech/spacex-starlink-
              satellites-1000-scn/index.html>.

   [Giorgetti]
              Giorgetti, A., Castoldi, P., Cugini, F., Nijhof, J.,
              Lazzeri, F., and G. Bruno, "Path Encoding in Segment
              Routing", IEEE 2015 IEEE Global Communications Conference
              (GLOBECOM), DOI 10.1109/GLOCOM.2015.7417097, December
              2015, <https://doi.org/10.1109/GLOCOM.2015.7417097>.

   [Handley]  Handley, M., "Delay is Not an Option: Low Latency Routing
              in Space", ACM Proceedings of the 17th ACM Workshop on Hot
              Topics in Networks, DOI 10.1145/3286062.3286075, November
              2018, <https://doi.org/10.1145/3286062.3286075>.

   [I-D.ietf-rtgwg-segment-routing-ti-lfa]
              Bashandy, A., Litkowski, S., Filsfils, C., Francois, P.,
              Decraene, B., and D. Voyer, "Topology Independent Fast
              Reroute using Segment Routing", Work in Progress,
              Internet-Draft, draft-ietf-rtgwg-segment-routing-ti-lfa-
              13, 16 January 2024,
              <https://datatracker.ietf.org/doc/html/draft-ietf-rtgwg-
              segment-routing-ti-lfa-13>.

   [I-D.united-tvr-schedule-yang]
              Qu, Y., Lindem, A., Kinzie, E., Fedyk, D., and M.
              Blanchet, "YANG Data Model for Scheduled Attributes", Work
              in Progress, Internet-Draft, draft-united-tvr-schedule-
              yang-00, 11 October 2023,
              <https://datatracker.ietf.org/doc/html/draft-united-tvr-
              schedule-yang-00>.





Li                       Expires 24 August 2024                [Page 18]

Internet-Draft           Routing for Satellites            February 2024


   [ITU]      "ITU Radio Regulations, Article 1", 2016,
              <https://search.itu.int/history/
              HistoryDigitalCollectionDocLibrary/1.43.48.en.101.pdf>.

   [RFC1195]  Callon, R., "Use of OSI IS-IS for routing in TCP/IP and
              dual environments", RFC 1195, DOI 10.17487/RFC1195,
              December 1990, <https://www.rfc-editor.org/info/rfc1195>.

   [RFC2131]  Droms, R., "Dynamic Host Configuration Protocol",
              RFC 2131, DOI 10.17487/RFC2131, March 1997,
              <https://www.rfc-editor.org/info/rfc2131>.

   [RFC2328]  Moy, J., "OSPF Version 2", STD 54, RFC 2328,
              DOI 10.17487/RFC2328, April 1998,
              <https://www.rfc-editor.org/info/rfc2328>.

   [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
              Border Gateway Protocol 4 (BGP-4)", RFC 4271,
              DOI 10.17487/RFC4271, January 2006,
              <https://www.rfc-editor.org/info/rfc4271>.

   [RFC4655]  Farrel, A., Vasseur, J.-P., and J. Ash, "A Path
              Computation Element (PCE)-Based Architecture", RFC 4655,
              DOI 10.17487/RFC4655, August 2006,
              <https://www.rfc-editor.org/info/rfc4655>.

   [RFC5340]  Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF
              for IPv6", RFC 5340, DOI 10.17487/RFC5340, July 2008,
              <https://www.rfc-editor.org/info/rfc5340>.

   [RFC9552]  Talaulikar, K., Ed., "Distribution of Link-State and
              Traffic Engineering Information Using BGP", RFC 9552,
              DOI 10.17487/RFC9552, December 2023,
              <https://www.rfc-editor.org/info/rfc9552>.

   [Westphal] Westphal, C., Han, L., and R. Li, "LEO Satellite
              Networking Relaunched: Survey and Current Research
              Challenges", October 2023,
              <https://arxiv.org/pdf/2310.07646.pdf>.

   [Zhang]    Zhang, X., Yang, Y., Xu, M., and J. Luo, "ASER: Scalable
              Distributed Routing Protocol for LEO Satellite Networks",
              IEEE 46th Conference on Local Computer Networks (LCN),
              Edmonton, AB, Canada, 2021,,
              DOI 10.1109/LCN52139.2021.9524989, 2021,
              <https://doi.org/10.1109/LCN52139.2021.9524989>.





Li                       Expires 24 August 2024                [Page 19]

Internet-Draft           Routing for Satellites            February 2024


Author's Address

   Tony Li
   Juniper Networks
   Email: tony.li@tony.li














































Li                       Expires 24 August 2024                [Page 20]