Source Based Time Variant Routing
Tianpeng Yu
yutianpengietf@163.com
10 July 2024
TVR Workgroup
This document describes a source-based time variant routing methodology that can be used in a network with predicted variations. By using source-based time variant routing, the source node uses stable information (e.g. orbit of satellite) to calculate the forwarding path between source and destination nodes. The intermediate node does not need to maintain routing information, such that in case of neighbor variations (restoration, activation, change of quality, or loss), there are no routing information changes. The routing decision is done by the source node. This makes building a highly scalable network with predicted variations possible (e.g. tens of thousands of satellites).

Introduction
[draft-ietf-tvr-use-cases] describes the use cases of time variant routing. This document describes a source-based time variant routing methodology that can be used in a network with predicted variations. By using source-based time variant routing, the source node uses stable information (e.g. orbit of satellite) to calculate the forwarding path between source and destination nodes. The intermediate node does not need to maintain routing information, such that in case of neighbor variations (restoration, activation, change of quality, or loss), there are no routing information changes. The routing decision is done by the source node. This makes building a highly scalable network with predicted variations possible (e.g. tens of thousands of satellites).

Specification of Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

Terminology
LEO: Low Earth Orbit with the altitude from 180 km to 2000 km.
VLEO: Very Low Earth Orbit with the altitude below 450 km.
GEO: Geosynchronous orbit with the altitude 35786 km.
ISL: Inter Satellite Link.
TN: Terrestrial Network; refers to networks providing connectivity through communication lines that travel on, near, and/or below ground.
UT: User Terminal. Usually known as a dish. Using cellphone as a user terminal will be described in a separate document.
UTID: A permanent ID uniquely identifies a UT. UTID is globally unique over the world.
UTTID: User Terminal Temporary ID.
GS: Ground Station, a device on the ground connecting the satellite and also connecting to TN.
GSID: A permanent ID uniquely identifies a GS. GSID is globally unique over the world.
SL: Satellite Link. Link between UT and satellite or link between GS and satellite.
FP: Forwarding path built via source based routing between UT, satellites and GS. A FP is calculated by UT via source-based routing algorithm. GS maintains a reversed FP used for reversed traffic from GS to UT.
SGDB: Satellite and GS database. Each satellite has a unique identifier (ID). SGDB maintains a database containing information of each satellite, including but not limited:
- Inclination (i)
- Longitude of the ascending node (Omega)
- Eccentricity (e)
- Semimajor axis (a)
- Argument of periapsis (omega)
- True anomaly (nu)
- Number and direction of the lasers (for ISL communication)
SGDB also maintains the information of each GS:
- The position (longitude, latitude) of ground station (the site connecting to PoP providing internet access).
SGDB does not maintain the position of UT(s).
To be clear, these information looks a lot, but these information is quite stable, and does not change due to the movement of the satellites.
For a given time T, the position (longitude, latitude, height) of any satellite is known based on the SGDB. Also the number of ISLs of a satellite is known, the adjacency(adjacencies) of a satellite can be determined and the bandwidth of the ISL(s) can be calculated.

Basic Logic of Source Based Time Variant Routing
One important character of source based time variant routing described in this document is that the routing decision is determined by the source node on the ground (user terminal or ground station), instead of on the satellite. The SGDB does not maintain any MAC and routing tables from users. The decision is made based on SGDB which is a stable database.
There are pre-conditions below to allow source based time variant routing algorithm work:
- It is required that UT and GS have SGDB. The method of synchronizing SGDB is described in section TBD.
- It is required that UT and GS are time synchronized. Detailed considerations on time synchronization is described in section TBD.
- It is required that UT, GS know their location(longitude, latitude).

                T=T0

                    +-+-+-+     +-+-+-+   +-+-+-+
                    |  C  |~~~~~|  D  |~~~|  F  |
                    +-+-+-+     +-+-+-+   +-+-+-+
                      ~     ~
                     ~      ~
+-+-+-+       +-+-+-+      +-+-+-+
|  E  |~~~~~~~|  A  |~~~~~~|  B  |
+-+-+-+       +-+-+-+      +-+-+-+
               *            *
               *             *
               *              *
            +-+-+-+          +-+-+-+   +-+-+-+
            | UT  |          | GS  |===| PoP |
            +-+-+-+          +-+-+-+   +-+-+-+

*** Satellite link
~~~ ISL (Inter Satellite Link)
=== Link on the ground (e.g. fiber)

Figure 1: Topology at time T0

              T=T0+T'

            +-+-+-+   +-+-+-+     +-+-+-+
            |  C  |~~~|  D  |~~~~~|  F  |
            +-+-+-+   +-+-+-+     +-+-+-+
                      ~     ~
                     ~      ~
+-+-+-+       +-+-+-+      +-+-+-+     +-+-+-+
|  G  |~~~~~~~|  E  |      |  A  |~~~~~|  B  |
+-+-+-+       +-+-+-+      +-+-+-+     +-+-+-+
                 *            *
                 *             *
                 *              *
             +-+-+-+          +-+-+-+   +-+-+-+
             | UT  |          | GS  |===| PoP |
             +-+-+-+          +-+-+-+   +-+-+-+

*** Satellite link
~~~ ISL (Inter Satellite Link)
=== Link on the ground (e.g. fiber)

Figure 2: Topology at time T=T0+T'

Figure 1 shows a basic topology diagram of satellite constellations used to explain the terminology of source-based time variant routing.
Let's assume that UT knows its location (Xu, Yu) and the location of the nearest GS (Xg, Yg). UT and GS also know the orbit of the satellites from SGDB. The detailed method of getting this information is described in section 4.
With this information, given a time T=T0, it is not difficult for UT to calculate the shortest path towards GS using the Dijkstra algorithm (or similar ones). Considering a GS can have many UT connected, to minimize the load of GS, GS will maintain a reversed And vice versa from GS to UT, But this is not necessary considering this will bring a significant load on the GS, as a GS can have many UT connected. This is described below.
For the topology in Figure 1, at time T0, the best path to the nearest GS is UT->A->B->GS.
The source node UT encapsulates the entire path containing the source, destination, and a list of IDs of the satellites and the nearest GS, with a pointer toward the ID of the satellite which should process the packet (in the left diagram of Figure 1, UT encapsulates path list {[UTID,(A, B), GSID], current=A}, and points towards A), in front of the original packet.
When A receives the packet, it first checks if the satellite ID (the pointer points to) is equal to the ID itself. If yes, A checks if the next ID is in the adjacency table. If yes, move the pointer to the next ID and forward the packet to the corresponding ISL. So A forwards the packet towards B and moves the pointer pointing to B, the packet becomes {[UTID,(A, B), GSID], current=B}.
After B receives the packet, if the validations pass (same as the process of A described above), it forwards the packet towards GS via SL (air interface) and moves the pointer to the next, without removing the header. So the packet becomes {[UTID,(A,B),GSID], current=GSID}.
After GS receives the packet, it values GSID equal to the ID of itself. If yes, it records a reversed path, which sends traffic back to UT. The reversed path from GS back to UT is {[GSID,(B, A), UTID], current=GSID}. GS can either reply to UT using this path for protocol packets (e.g. UT authentication, IP allocation, etc.) or send the data packet to PoP. For the data packet back from PoP, GS can use the binding relationship between the IP address of the UT allocated by GS and FP, and send traffic to UT based on the destination IP of the packet. GS located the IP address to UT and built a binding relationship between the IP and the FP.
After the process above, the UT and GS build a bi-directional path over the SL and ISL. The path is stateless, which means the satellites do not need to maintain the status of the path.
As the satellites keep moving, after time T', the satellites covering the UT and GS change to E, D, and A as shown in the right diagram in Figure 1. The UT recalculates the path towards the closest GS, based on the SGDB and time T=T0+T', and the new path is UT->E->D->A->GS. The UT sends a path with a notification message to GS with the new path information, GS updates the binding relation between the IP of UT and FP, sends an acknowledge message back to UT, and switches the FP towards UT to the new path. After UT receives the acknowledge message, it switches the FP to the new path.

Maintaining and Synchronizing SGDB

SGDB Synchronization Between Satellites
A control channel(or protocol) between satellites is used to synchronize SGDB. The synchronizing mechanism SHOULD be incremental. Each time an ISL established between satellites, the two satellites compare if the SGDB each other are the same (a checksum based mechanism MAY be used). If SGDBs are not the same, then incremental synchronization(s) are initialized. Also when a new SGDB item is added into SGDB, such information MUST also be flooded to all other satellite neighbors via ISL(s). Instead of synchronizing routing prefixes via IGP(OSPF/IS-IS), SGDB synchronizes orbit of the satellite and location of the ground station, which does not change due to ISLs reestablishments. The detailed mechanism will be described in a separate document.
To be clear that SGDB is a stable database. SGDB does not change due to movements of satellites and up/down of ISLs.

Ground Station Registration
SGDB also needs to maintain the location(longitude, latitude) information of all ground stations. When a new GS is built, and linked with one of the satellite (S), it reports {GSID, (longitude, latitude)} to satellite S. Satellite S add such info into its SGDB and spread it via the mechanism described above.
The organizer information of a GS MAY also be reported. This allows a specific organization (e.g. ISP) to use satellite constellation together with its TN resources to provide specific services.

User Terminal Registration

User Terminal Registration Towards Satellite
When a new UT is connected to one of the satellite, it downloads the SGDB from the satellite. With SGDB, the UT is able to use source-based time variant routing algorithm to calculate the FP (forwarding path) towards the closest GS.
UT MAY also report specific organization code, thus getting the SGDB belonging to specific organization only.
Be aware that the UT information is NOT synchronized in the SGDB.

User Terminal Registration Towards Ground Station
To connect a UT to the internet, it is required that the UT gets the IP address from the GS. GS acts as a gateway and allocate IPv6 address to UT.
GS SHOULD maintain a permanent mapping relationship between UTID and the IPv6 address of UT. This consideration is because a UT MAY be movable (e.g. Plane, Ship), a UT MAY switch to another GS after a period of time. The detailed considerations on roaming UT is described in section TBD.
After UT downloads the SGDB from the connected satellite, the UT itself is able to calculate a FP towards the GS. The UT initiates an IPv6 address allocation request using the FP (DHCPv6 or similar methods MAY be used). GS validates the UTID and allocates the IPv6 address to UT with a reply packet with the reversed FP.

Network Topology Changes Considerations

Overall Considerations on Network Topology Changes
As the satellites keep moving, SLs and ISLs goes up and down (but predictable). The source node (UT) needs to recalculate a new FP before any of the SLs or ISLs becomes unavailable.
The scenarios requiring the source node to recalculate the FP include:
UT/GS moves from coverage of one satellite to another one (or signal strength changes).
GS moves from coverage of one satellite to another one (or signal strength changes).
ISLs goes up/down due to movements of satellites.
The process (3-way handshake) is as below:
T0: UT initiates a path change request using the old forwarding path, with the new path info included in the request towards GS.
T1: GS receives the path change request and records the new path and start to accept packets with the new FP from UT, sends acknowledge packet using the old FP.
T2: UT sends acknowledge packet 2 using the new FP, and start sending traffic with the new FP.
T3: GS receives acknowledge packet 2 from UT. GS stop accepting packets from UT with old FP. GS starts to send traffic towards UT with the new FP.

UT Switching Between Satellites

                #             ^
                   #       ^
                     #  ^
      +-+-+-+  +-+-+^  #           +-+-+-+
      |  A  |**| UT|     #         |  B  |
      +-+-+-+  +-+-+^   #          +-+-+-+
                      #
                     # ^
                   #     ^
                #           ^

   #### coverage of satellite A
   ^^^^ coverage of satellite B
   **** satellite link between satellite and UT

Figure 3: UT Considerations on Satellite Coverage

During time T0 and T3, it is required that UT maintains links towards both of the satellites. UT needs to accept packets from GS with both old FP and new FP.

GS Switching Between Satellites
GS is desired to have multiple antennas towards satellites where possible. This allows GS to have higher bandwidth with satellite link(s).
When GS moves from coverage of one satellite to another one (or signal strength changes), GS notify all connected UT using the SL being switched to initiate a FP switching process. Then UT starts switching process based on the process described in Section 7.1 of this document.
During time T0 and T3, it is required that GS maintains links towards both of the satellites. GS needs to accept packets from GS with both old FP and new FP.

ISL Status Changing
UT needs to initiate a path before any of the ISL being used as current FP. Due to the up/down and quality of ISLs are predicable by UT, it is the responsibility of UT to initiate a FP change request before an ISL becomes unusable. The procedure is same with the process described in Section 7.1. UT and GS do not need to maintain extra SLs during the switch of ISLs.

Load Balancing Considerations
Due to ISLs are established in nearly vacuo space, the bandwidth of ISLs can reach much much higher bandwidth than SLs. So in most of the cases, ISL won't get full.
But still in some cases, one of the ISL MAY be used together by couple of other satellites as an intermediate node. Let's assume A,B,C and D have ISLs based in Figure 5. Traffic from UT connecting to A can have two paths towards the GS connecting to D, which are A-C-D and A-B-D. But UT connecting to B MAY also uses ISL B-D. This may lead to ISL B-D becomes overloaded.
This can be addressed in several ways:
When a source UT have multiple ECMP towards (the number of paths=M) a destination GS, it sorts the path in ascending order based on satellite ID of each hop, then generates a random number N with a seed (the seed CAN use the permanent UTID). Then UT choose the Ith path (I=N%M).
Quality of service mechanism SHOULD be introduced to ensure critical control packets are not lost and important service is not impacted.
When an ISL utilization reaches a threshold, a satellite MAY randomly pick part of the best effort packets and generate FP switch notifications towards UT. UT recalculates the path without the corresponding link. The FP switch process is the same with the process described in section 7.1.

            +-+-+      +-+-+
            | A |~~~~~~| B |
            +-+-+      +-+-+
              ~          ~
              ~          ~
              ~          ~
            +-+-+      +-+-+
            | C |~~~~~~| D |
            +-+-+      +-+-+

      ~~~ ISL (Inter Satellite Link)

      Figure 5: Load Balancing between ISLs

Service Scenarios

L3 services
Source-based time variant solution described in this document can be used to provide L3 services to end users. L3 services can be internet service or L3VPN service. L3VPN service can be achieved via NVO3

L2 services
Source-based time variant solution described in this document does not directly provide L2 service directly. But L2 services (e.g. ELine, ELAN) can be achieved via NVO3.

L1 service
Source-based time variant solution described in this document can be used to provide L1 service (e.g. pseudo wire). Theoretically it is possible to allow two UTs communicate each other via source-based routing, as long as they know the location each other. But from security perspective, the UTs SHOULD be authenticated before being able to

Multicast service
To be studied.

Comparison with Existing Researches and Solutions

Security Considerations
TBD

IANA Considerations
This document does not make any requests from IANA.

Normative References
[draft-ietf-tvr-use-cases]

TVR (Time-Variant Routing) Use Cases
Edward J. Birrane
JHU/APL
Nicolas Kuhn
Thales Alenia Space
Yingzhen Qu
Futurewei Technologies
Rick Taylor
Ori Industries
Li Zhang
Huawei
29 February 2024
Internet-Draft draft-ietf-tvr-use-cases-09

Informative References
[RFC2119]