Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Fri, 26 Mar 2021 20:58:43 +0000

Hi Linda, John,

   When a UE (User Equipment) initiates application packets using the
   destination address from a DNS reply or from its own cache, the
   packets from the UE are carried in a PDU session through 5G Core
   [5GC] to the 5G UPF-PSA (User Plan Function - PDU Session Anchor).
   The UPF-PSA decapsulate the 5G GTP outer header and forwards the
   packets from the UEs to the Ingress router of the Edge Computing (EC)
   Local Data Network (LDN). The LDN for 5G EC, which is the IP Networks
   from 5GC perspective, is responsible for forwarding the packets to
   the intended destinations.

A nit comment about "5G Core" above. When I first started learning 4G/5G It took me a while to realize the 3GPP "core network" concept in vastly different from what IETF people are used to. It's not about topology and now the "core network" functions are being more and more distributed into edges. Therefore, in this context it may be better to simply strike the "through 5G Core [5GC]" wording to reduce the confusion to some readers.

  1.3. Problem #1: ANYCAST in 5G EC Environment

   Increasingly, ANYCAST is used extensively by various application
   providers and CDNs because it is possible to dynamically load balance
   across multiple locations of the same address based on network
   conditions. BGP is an integral part in the way IP anycast usually
   functions. Within BGP routing there are multiple routes for the same
   IP address which are pointing to different locations.

Not only BGP - but all IP routing protocols should work well with anycast. My understanding is that BGP being integral part here is really that the data network here is likely realized by VPNs over the same transport network. Is that a correct understanding?
Of course, BGP does have flexibility in providing better/more control of route selection than IGP does in the context of the companion draft-dunbar-idr-5g-edge-compute-app-meta-data.

   But, having multiple locations for the same ANYCAST address in 5G
   Edge Computing environment can be problematic because all those edge
   computing Data Centers can be close in proximity.  There might not be
   any difference in the routing cost to reach the Application Servers
   in different Edge DCs.   Same routing cost to multiple ANYCAST
   locations can cause packets from one flow to be forwarded to
   different locations, which can cause service glitches.

As pointed out later in this same document, modern routers support "Flow Affinity" and should not cause packets of a flow on a specific router to be forwarded to different locations. The real problem is when a UE moves to a different location, the new router at that location may send it to a different egress router. However, that is the "sticky service" problem described in 1.4.

From draft-dunbar-idr-5g-edge-compute-app-meta-data, I understand that on a specific router it needs to choose a location that can best serve an application based on some non-routing factors. If 1.3 is really for that purpose, it should be reworded accordingly. As I mentioned in an earlier email, the two documents should better align on the problem descriptions.

   Here is the overview of the End-Node based Sticky Service solution:
     - Each ANYCAST Edge Computing server either learns or is informed
        of the unicast Sticky Egress address (Section 3). The goal of
        the network is to deliver packets belonging to one flow to the
        same Sticky Egress address for the ANYCAST address. Section 4.1
        describes how Edge Computing Servers discover their
        corresponding Sticky Egress unicast addresses.
     - When an Edge Computing server sends data packets to a UE (or
        client), it inserts the Sticky-Dst-SubTLV (described in Section
        4.2) into the packets' Destination Option Header.
     - UE (or client) needs to copy the Destination Option Header from
        the received packet to the next packet's Destination Header if
        the next packet belongs to the same flow as the previous packet.

I was really confused by "next packet". I finally realized you may be referring to response packets from the UE to the server, and the "same flow" should be "same service". Better wording is needed here.

     - If the following conditions are true, the ingress router
        encapsulates the packet from the UE in a tunnel whose outer
        destination address is set to the Sticky Egress Address
        extracted from the packet's Sticky-Dst-SubTLV:
          o The destination of the packet from the UE side matches
             with one of the Sticky Service ACLs configured on the
             ingress router of the LDN,
          o the packet header has the Destination Option present with
             Sticky-Dst-SubTLV.

Wouldn't it be better for the UE to put in an SRH with one SID for the server address and set the DA to be the egress router address? That way you don't need the ACL or the DOH (the Sticky-Dst-SubTLV  information in the DOH is not for consumption by the server anyway), and you don't even need tunneling or BGP (unless VPN is used - but that's orthogonal to this). Existing SRv6 function takes care of it.

Also, the Sticky-Dst-SubTLV in DOH of the server->UE traffic would be better renamed as "return waypoint" for more generic purpose.

4.1. Sticky Egress Address Discovery

   To an App server with ANYCAST address, the Sticky Egress address is
   same as its default Gateway address.

   To prevent malicious UEs (or clients) sending DDOS attacks to routers
   within 5G EC LDN, e.g. the Sticky Egress address that is encoded in
   the Destination option header in the packets sent back to the UEs (or
   clients), a proxy Sticky Egress address can be encoded in the
   Destination option header. The proxy Sticky Egress address is only
   recognizable by the 5G EC LDN ingress nodes, i.e. the Ra and Rb in
   the Figure 1, but not routable in other networks. The LDN ingress
   routers can translate the proxy Sticky Egress to a routable address
   for the Sticky Egress node after the source addresses of the packets
   are authenticated.

Why is the 4.1 title called "... discovery"? Does not seem to be about "discovery".

 4.3. Expected behavior at the UE
   ...
   Section 4 describes the network layer processing if UEs do not
   perform the steps described here.

Should be "Section 5".

5. Tunnel based Sticky Service Solutions
5.1. Ingress and Egress Routers Processing Behavior

   The solution assumes that both ingress routers and egress routers
   support at least one type of tunnel and are configured with ACLs to
   filter out packets whose destination or source addresses match with
   the Sticky Service Identifier. The solution also assumes there are
   only limited number of Sticky Services to be supported.
   An ingress router needs to build a Sticky-Service-Table, with the
   minimum following attributes. The Sticky-Service-Table is initialized
   to be empty.
     - Sticky Service ID
     - Flow Label
     - Sticky Egress address
     - Timer

   Editor's Note:
     When a UE moves from one 5G Site to another, the same UE will have
     a new IP address. "Flow Label + Sticky Service ID" stays the same
     when a UE is anchored to a new PSA. Therefore, this solution use
     "Flow Label + Sticky Service ID" to identify a sticky flow. Since
     the chance of different UEs sending packets to the same ANYCAST
     address using the same Flow Label is very low, it is with high
     probability that "Flow Label + Sticky Service ID" can uniquely
     identify a flow. When multiple UEs using the same Flow Label
     sending packets to the same ANYCAST address, the solution described
     in this section will stick the flows to the same ANYCAST server
     attached to the Sticky Egress router. This behavior doesn't cause
     any harm.

It seems that the same flow label is used for traffic of the same service in both directions. So who will assign the flow label?

If two UEs of the same service happen to use the same flow label, then sticky service is not guaranteed. For example, initially they're anchored at different UPFs, and UE1 traffic is sent to egress router 1 while UE2 traffic is sent to egress router 2. When UE 1 relocates to the same UPF as UE 2's, its traffic will be sent to egress node 2 because the same flow label is used.

Therefore, there should be a central controller to assign flow labels based on UE id, and the UE id is not based on IP address (since it could change).

   Note: since there are only small number of Sticky services, the
   Sticky-Service-Table is not very large.

With the above understanding, the table could get large?

   When an ingress router receives a packet from a UE matching with one
   of the Sticky Service ACLs and there is no entry in the Sticky-
   Service-Table matching the Flow Label and the Sticky Service ID, the
   ingress router considers the packet to be the first packet of the
   flow. There is no need to sticking the packet to any location. The
   ingress router uses its own algorithm to select the optimal egress
   node as the Sticky Egress address for the ANYCAST address,
   encapsulates the packet with a tunnel that is supported by the egress
   node. The tunnel's destination address is set to the egress node
   address.

If a UE was using egress router 1 and it relocates to a new UPF, the new ingress router will likely have no corresponding entry for it? What if the new ingress router pick egress router 2?
It seems that the ingress routers need to pre-exchange entries in the table?
I see it's discussed later that the routers do exchange the information. It should be mentioned up front when the table is introduced.

   When an ingress router receives a packet in a tunnel from any egress
   router and the packet's source address matches with a Sticky Service
   ID, the egress router address is set as the Sticky Egress address for
   the Sticky Service ID. The ingress router adds the entry of "Sticky-
   Service-ID + Flow Label + the associated Sticky Egress address +
   Timer" to the Sticky-Service-Table if the entry doesn't exist yet in
   the table. If the entry exists, the ingress router refreshes the
   Timer of the entry in the table.

   When the ingress router receives the subsequent packets of a flow
   from the 5G side matching with an Sticky Service ID and the Sticky-
   Service ID exists in the Sticky-Service-Table, the ingress router
   uses the Sticky Egress address found in the Sticky-Service-Table to
   encapsulate the packet and refresh the Timer of the entry. If the
   Sticky-Service ID doesn't exist in the table, the ingress router
   considers the packet as the first packet of a flow.

The above is what leads me to believe that the flow label is the same in both directions.

 5.3. Scenario 2: With communication with 5G system
   ...
   The ingress and egress router processing are the same as described in
   Section 5.1 except a flow is now uniquely identified by the "Sticky
   Service ID" + "UE address" instead of "Sticky Service ID" + "Flow
   Label".

This confirms my earlier understanding for scenario 1 that "there should be a central controller to assign flow labels based on UE id, and the UE id is not based on IP address (since it could change)" and that the table could get large.

Of course now for scenario 2, you're not using the flow label any more. While the table only contains entries that this ingress router actually need, the following are still true:
- The table could still get large (if the number of attached UEs for the sticky services is large)
- On demand fetching of the table entry may not be fast enough

Additionally, instead of "scenario", "option" or "solution" would be a better wording.

More importantly, this stateful flow steering based on the additional table is just too heavy and complicated. Why not simply have the UEs support SRH so that traffic will be routed via the desired egress router using standard SRv6 mechanism?

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

I have the following questions.

The two related drafts listed the following three problems respectively:

      1.3. Problem#1: ANYCAST in 5G EC Environment.............. 6
      1.4. Problem #2: Unbalanced Anycast Distribution due to UE Mobility.................................................. 7
      1.5. Problem 3: Application Server Relocation............. 7

      1.2. Problem #1: ANYCAST in 5G EC Environment.............. 4
      1.3. Problem #2: sticking to original App Server........... 5
      1.4. Problem #3: Application Server Relocation............. 5

Why is problem #2 different in the two drafts? Is it true that none of the two drafts address problem #3?
The idr draft talk about "soft anchoring" problem and solution - how is that different from the "sticky service"?

Thanks.
Jeffrey

Juniper Business Use Only

---

From: Linda Dunbar <[email-masked]>
Date: Sun, 28 Mar 2021 02:44:50 +0000

Jeffrey, 

Thank you very much for the constructive comments. 
Replies are inserted below:

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]> 
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

[Linda] That is very true. Removed the term per your suggestion. 5G Core refers to all the functions from Radio to UPF.  

[Linda] ANYCAST has traditionally been used for servers or loader balancers that are placed in geographically diverse locations, so that BGP alone is enough for the traffic in one region to be forwarded to one server.  But for the 5G Edge Computing where multiple Servers/load Balancers with the same ANYCAST addresses are placed close proximity, IGP is needed. 

[Linda] Correct. 

[Linda] Correct. 

[Linda] 3GPP has rejected using SRH in the 5G Core. We can think about using them in the N6 interface. 

Also, the Sticky-Dst-SubTLV in DOH of the server->UE traffic would be better renamed as "return waypoint" for more generic purpose.
[Linda]  that is interesting suggestion. 

Why is the 4.1 title called "... discovery"? Does not seem to be about "discovery".
[Linda] it is about remembering which Egress router was used for the flow. Should it be "Sticky Egress Memory"? 

Should be "Section 5".

[Linda] Thank you. 

5. Tunnel based Sticky Service Solutions 5.1. Ingress and Egress Routers Processing Behavior

   Editor's Note:

It seems that the same flow label is used for traffic of the same service in both directions. So who will assign the flow label?
[Linda] The "flow label" from the IPv6 header should be managed by the hosts & servers. 

Therefore, there should be a central controller to assign flow labels based on UE id, and the UE id is not based on IP address (since it could change).
[Linda] Since the "Flow Label" is randomly generated (by Host OS), the chance of two UEs reaching the same service having the same Flow Label is very small.  We can explore the option of getting the Control Plane involved. 

[Linda]? 

If a UE was using egress router 1 and it relocates to a new UPF, the new ingress router will likely have no corresponding entry for it? What if the new ingress router pick egress router 2?
It seems that the ingress routers need to pre-exchange entries in the table?
I see it's discussed later that the routers do exchange the information. It should be mentioned up front when the table is introduced.
[Linda] Would Adding a reference be enough? 

The above is what leads me to believe that the flow label is the same in both directions.
[Linda] they don't have to be the same, do they? 

[Linda] Good suggestion! 

More importantly, this stateful flow steering based on the additional table is just too heavy and complicated. Why not simply have the UEs support SRH so that traffic will be routed via the desired egress router using standard SRv6 mechanism?
[Linda] It is not realistic for UEs (your smart phone) to support SRH. 

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Mon, 29 Mar 2021 14:17:43 +0000

Hi Linda,

You proposed two ways of providing "sticky services" - when a UE moves to a new location, the ingress router at that new location will still route the packets of the same flow to the previous egress router. That flow cannot be identified by the destination address alone, since it is an anycast address that are shared by servers behind different egress routers.

Essentially, you're trying to turn the ingress router into a load balancer, especially with your option #2 (section 5, "tunnel based" solution). I don't think we want the routers to do that - while routers can make use of 5-tuple for ECMP hashing, we don't want to make routers more complicated and do forwarding based on a sticky-service-table with (Sticky Service ID, Flow Label. Sticky Egress address, Timer) entries. It's not only complicated but also does not scale (we can discuss the scaling aspect wrt the flow labels separately).

The variation of option #1 that I suggested would be better, if the following were true:

1. The UE can insert an SRH
2. The ingress router can trust the SRH from the UEs

In that case, it would be better for the UE to learn the egress router via 5G/MEC control plane, instead of relying on the egress router to put that into the DOH of every server->UE packets for sticky services and for the UE to retrieve that information from each incoming sticky service packets. One thing I learned is that the entire 5G system is very much heavy with control/management plane and I would think it is a much better option to provide that information to the UEs.

On the other hand, once you go that way, the control plane can simply provide the regular, non-anycast addresses of the servers instead of the egress router address. Then, all the problems disappear and corresponding proposals are no longer needed, including the ones in draft-dunbar-idr-5g-edge-compute-app-meta-data, and we only need existing simple routing functions.

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

[External Email. Be cautious of content]

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 29 Mar 2021 16:09:09 +0000

Jeffrey, 

We can definitely add the option of UE inserting SRH. I am just not sure how many UEs or end devices will do those actions. If very few UEs can do this action, the solution itself is not useful. However, it doesn't hurt for IETF to specify such a solution so that future IoT or 5G devices can have a reference to do the actions. 

Another point, the number of Sticky Service is not large. The Ingress routers are configured with the policies ( ACLs) to filter those flows. 

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]> 
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

---

From: Vasilenko Eduard <[email-masked]>
Date: Mon, 29 Mar 2021 16:40:46 +0000

It could be the problem.
Because all SR RFCs and drafts clearly say: only inside the domain.
Else could be a huge security risk. UE could not be trusted.
Cross-domain security is the principal question that should be discussed in SPRING first.
Current SR architecture does not try to resolve it yet.

Segment Routing in general and SRv6 in particular are claimed to be designed for Trusted environments only:
- Segment routing architecture (RFC 8402) section 8
- SRH - Segment Routing Header (RFC 8754) section 5
- SRv6 Network Programming (draft-ietf-spring-srv6-network-programming-25) section 9
SRH RFC is especially verbal how to filter-out any SR-related information on the border of "SR domain".
Ed/
-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, March 29, 2021 7:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey, 

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]> 
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

Juniper Business Use Only
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 29 Mar 2021 16:50:24 +0000

Ed, 

Yes, they are in one domain. Here is one example:

5G Connected devices, such as drones for fighting fires or natural disasters or robots in Industry 4.0  environments,  need ultra-low latency  responses from their analytic servers hosted in the Edge data centers. To reach ultra-low latency, there are multiple servers hosting the analytic functions in the Edge DCs.
All the functions (including networking and analytics) and devices are administrated by one operator.  Those functions might be provided by different vendors, therefore needing interoperable solutions. 

Linda 

-----Original Message-----
From: Vasilenko Eduard <[email-masked]> 
Sent: Monday, March 29, 2021 11:41 AM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

It could be the problem.

Segment Routing in general and SRv6 in particular are claimed to be designed for Trusted environments only:
- Segment routing architecture (RFC 8402) section 8
- SRH - Segment Routing Header (RFC 8754) section 5
- SRv6 Network Programming (draft-ietf-spring-srv6-network-programming-25) section 9 SRH RFC is especially verbal how to filter-out any SR-related information on the border of "SR domain".
Ed/
-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, March 29, 2021 7:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey, 

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

Juniper Business Use Only
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4209e8a9acae47b96d1808d8f2d16b8d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637526328578769822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=A39pqHVUBFsssO3DLSqrTUtPpcXAr%2F8pi%2Bmw%2BtIJNME%3D&amp;reserved=0
--------------------------------------------------------------------

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Mon, 29 Mar 2021 17:00:21 +0000

Hi Linda,

> Another point, the number of Sticky Service is not large. The Ingress routers are configured with the policies ( ACLs) to filter those flows.

I assume the context for the above is option #2 - forwarding based on a sticky-service-table with (Sticky Service ID, Flow Label. Sticky Egress address, Timer) entries.

As I mentioned, It's not only complicated (you're turning a router into a load balancer) but also does not scale.

For the scaling aspect, the number of sticky services may not be large, but the number of UEs needing those services could be large. As you say, each UE will randomly generate a flow label for it, and two UEs are likely generating different flow labels. That means the number of entries will be proportional to the number of UEs needing the services.

Also, about why I think the server and the UE need to use the same flow label that is allocated by a controller:

The first paragraph above is about creating an entry based on packets from the server. Let's say flow label is independently/randomly generated by a server connected to egress router 1, and the value is 1000. Another server off a different egress router would also generate flow label 1000. Now when the ingress router gets those packets, the entries will constantly be updated - using egress router 1 for one moment and egress router 2 for another moment (or, the existing entry with egress router 1 will be used for servers off the egress router 2, which is not good either).

The second paragraph is about using an existing entry with a matching flow label for packets from a UE. Similarly, when two UEs happen to pick the same flow label, they will follow the same entry and be sent to the same server (against the original intention), and more importantly, even for the same source UE, at moment 1 the packets will be sent to server1 and at moment 2 the packets will be sent to server 2 as explained above.

Therefore, a controller must be used to generate a consistent flow label; and once you do that, you might as well tell a UE which unicast address (for a server or for a real load balancer) to use.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 12:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Mon, 29 Mar 2021 17:03:55 +0000

Even if you could get over the security/trust hurdle, using a controller to let the UEs know which unicast non-anycast address to use is a much simpler/better solution.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 12:50 PM
To: Vasilenko Eduard <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Ed,

Linda

-----Original Message-----
From: Vasilenko Eduard <[email-masked]>
Sent: Monday, March 29, 2021 11:41 AM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

It could be the problem.

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, March 29, 2021 7:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

Juniper Business Use Only
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6&amp;data=04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&amp;sdata=A39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D&amp;reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z$
--------------------------------------------------------------------

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 29 Mar 2021 17:14:39 +0000

Jeffrey, 

The Devices are moving consistently, it is not reasonable to require them to consistently query DNS for the "correct" non-ANYcast address . 

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]> 
Sent: Monday, March 29, 2021 12:04 PM
To: Linda Dunbar <[email-masked]>; Vasilenko Eduard <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 12:50 PM
To: Vasilenko Eduard <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Ed,

Linda

-----Original Message-----
From: Vasilenko Eduard <[email-masked]>
Sent: Monday, March 29, 2021 11:41 AM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

It could be the problem.

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, March 29, 2021 7:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

Juniper Business Use Only
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6%26amp%3Bdata%3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000%26amp%3Bsdata%3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D%26amp%3Breserved%3D0__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C2b126cc5be8541bd43cc08d8f2d4a7ab%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637526342463012256%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=aOw1DkDcQu0%2FmMi6RfWlUyRcLB2jRcsbBAhcpoaX5yE%3D&amp;reserved=0
--------------------------------------------------------------------

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Mon, 29 Mar 2021 17:20:21 +0000

Since it is for "sticky service", you don't want to get a new server address every time you move - unless the previous one is no longer appropriate. That means it is best for a controller to determine which one to use both initially and later when situation changes (when a UE relocates or server load situation changes), and that does not necessarily mean it is always through DNS.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 1:15 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Vasilenko Eduard <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 12:04 PM
To: Linda Dunbar <[email-masked]>; Vasilenko Eduard <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 12:50 PM
To: Vasilenko Eduard <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Ed,

Linda

-----Original Message-----
From: Vasilenko Eduard <[email-masked]>
Sent: Monday, March 29, 2021 11:41 AM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

It could be the problem.

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, March 29, 2021 7:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

Juniper Business Use Only
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24&amp;data=04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&amp;sdata=aOw1DkDcQu0*2FmMi6RfWlUyRcLB2jRcsbBAhcpoaX5yE*3D&amp;reserved=0__;JSUlJSUlJSUlJSUqKioqKiolJSUqKioqKioqKioqKiolJSUqKioqJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!Q-hLtDzPuot4CQsvyUhfrEcNgHIIBEdRDT4RgyHgVCE1f5Vt6DlvzYC-o7693kZ1$
--------------------------------------------------------------------

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 29 Mar 2021 17:30:13 +0000

Jeffrey, 

The draft is to leverage the network control plane to determine which server is the most appropriate one for the device. 

You can definitely use the Controller to choose the optimal server. To simplify the IP address allocation, the draft suggest using the Egress router addresses as the proxy for reaching the desired server. We can add a section to describe this scenario

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]> 
Sent: Monday, March 29, 2021 12:20 PM
To: Linda Dunbar <[email-masked]>; Vasilenko Eduard <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 1:15 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Vasilenko Eduard <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 12:04 PM
To: Linda Dunbar <[email-masked]>; Vasilenko Eduard <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 12:50 PM
To: Vasilenko Eduard <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Ed,

Linda

-----Original Message-----
From: Vasilenko Eduard <[email-masked]>
Sent: Monday, March 29, 2021 11:41 AM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

It could be the problem.

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, March 29, 2021 7:09 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

Linda

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Monday, March 29, 2021 9:18 AM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda,

Thanks.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Saturday, March 27, 2021 10:45 PM
To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Jeffrey,

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]>
Sent: Friday, March 26, 2021 3:59 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Should be "Section 5".

[Linda] Thank you.

   Editor's Note:

Jeffrey

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang
Sent: Thursday, March 25, 2021 3:46 PM
To: Linda Dunbar <[email-masked]>; Kaippallimalil John <[email-masked]>; IPv6 List <[email-masked]>; idr@ietf. org <[email-masked]>
Subject: questions about draft-dunbar-idr-5g-edge-compute-app-meta-data and draft-dunbar-6man-5g-edge-compute-sticky-service

Hi Linda, John,

Thanks.
Jeffrey

Juniper Business Use Only
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24%26amp%3Bdata%3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000%26amp%3Bsdata%3DaOw1DkDcQu0*2FmMi6RfWlUyRcLB2jRcsbBAhcpoaX5yE*3D%26amp%3Breserved%3D0__%3BJSUlJSUlJSUlJSUqKioqKiolJSUqKioqKioqKioqKiolJSUqKioqJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!Q-hLtDzPuot4CQsvyUhfrEcNgHIIBEdRDT4RgyHgVCE1f5Vt6DlvzYC-o7693kZ1%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C67c907b1d8e643cc5a5108d8f2d6f104%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637526352290503133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Be%2FU8MhJT%2BA5b83e8ZLFCeamlBcSoit3SJ6Xk3X9%2Bz8%3D&amp;reserved=0
--------------------------------------------------------------------

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 29 Mar 2021 14:55:30 -0400

Linda, you are assserting that the answer is "to leverage the network 
control plane".
As Jeffrey has pointed out, given the problem you have described, it is 
not at all clear that is the best answer, or even a good one.

Yours,
Joel

On 3/29/2021 1:30 PM, Linda Dunbar wrote:
> Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24%26amp%3Bdata%3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
>   V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000%26amp%3Bsdata%3DaOw1DkDcQu0*2FmMi6RfWlUyRcLB2jRcsbBAhcpoaX5yE*3D%26amp%3Breserved%3D0__%3BJSUlJSUlJSUlJSUqKioqKiolJSUqKioqKioqKioqKiolJSUqKioqJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!Q-hLtDzPuot4CQsvyUhfrEcNgHIIBEdRDT4RgyHgVCE1f5Vt6DlvzYC-o7693kZ1%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C67c907b1d8e643cc5a5108d8f2d6f104%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637526352290503133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Be%2FU8MhJT%2BA5b83e8ZLFCeamlBcSoit3SJ6Xk3X9%2Bz8%3D&amp;reserved=0

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 29 Mar 2021 21:16:17 +0000

Joel, 

I am confused of your statement. 

The problem I described is:
    An end node (e.g. a drone) moves constantly. There are multiple servers hosted in Edge DCs close to Cell towers to process requests from the end node to achieve ultra-low latency.  It is not realistic for the end node to send DNS query every time it anchors to a new Cell Tower. 

The suggested solution is to leverage network control plane to find the optimal server to serve the end node. 

You stated that "it is not at all clear that is the best answer or even a good one".  Why? 

Linda 

-----Original Message-----
From: Joel M. Halpern <[email-masked]> 
Sent: Monday, March 29, 2021 1:56 PM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Linda, you are assserting that the answer is "to leverage the network control plane".
As Jeffrey has pointed out, given the problem you have described, it is not at all clear that is the best answer, or even a good one.

Yours,
Joel

On 3/29/2021 1:30 PM, Linda Dunbar wrote:
> You can definitely use the Controller to choose the optimal server. To 
> simplify the IP address allocation, the draft suggest using the Egress 
> router addresses as the proxy for reaching the desired server. We can 
> add a section to describe this scenario
> 
> Linda
> 
> To: Linda Dunbar <[email-masked]>; Vasilenko Eduard 
> <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Vasilenko Eduard 
> <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> To: Linda Dunbar <[email-masked]>; Vasilenko Eduard 
> <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> To: Vasilenko Eduard <[email-masked]>; Jeffrey (Zhaohui) 
> Zhang <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang 
> <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> It could be the problem.
> To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> Jeffrey,
> 
> To: Linda Dunbar <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> Hi Linda,
> 
> To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: RE: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> To: Linda Dunbar <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: Questions/comments about 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> Hi Linda, John,
> 
> To: Linda Dunbar <[email-masked]>; Kaippallimalil John 
> <[email-masked]>; IPv6 List <[email-masked]>; 
> idr@ietf. org <[email-masked]>
> Subject: questions about 
> draft-dunbar-idr-5g-edge-compute-app-meta-data and 
> draft-dunbar-6man-5g-edge-compute-sticky-service
> 
> Hi Linda, John,
> 
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000%26amp%3Bsdata%3DaOw1DkDc
> Qu0*2FmMi6RfWlUyRcLB2jRcsbBAhcpoaX5yE*3D%26amp%3Breserved%3D0__%3BJSUl
> JSUlJSUlJSUqKioqKiolJSUqKioqKioqKioqKiolJSUqKioqJSUlJSUlJSUlJSUlJSUlJS
> UlJQ!!NEt6yMaO-gk!Q-hLtDzPuot4CQsvyUhfrEcNgHIIBEdRDT4RgyHgVCE1f5Vt6Dlv
> zYC-o7693kZ1%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C67c9
> 07b1d8e643cc5a5108d8f2d6f104%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C
> 0%7C637526352290503133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJ
> QIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Be%2FU8Mh
> JT%2BA5b83e8ZLFCeamlBcSoit3SJ6Xk3X9%2Bz8%3D&amp;reserved=0
> Administrative Requests: 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%4
> 0futurewei.com%7Ca73842e8d0e7473e591d08d8f2e43b9f%7C0fee8ff2a3b240189c
> 753a1d5591fedc%7C1%7C0%7C637526409383434704%7CUnknown%7CTWFpbGZsb3d8ey
> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C100
> 0&amp;sdata=qkqfI%2B3ZLo76yZFXDIxUxjfwyhA5MNJIqUwzUyTGXqc%3D&amp;reser
> ved=0

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 29 Mar 2021 18:04:21 -0400

Earlier in the problem statement you described the need for flow 
stickiness.  Meaning, as I understand it, that the UE needs to keep 
using the same instance of the server as long as feasible.
The corollary of that seems to be that if the UE is moving frequently 
and not too far, it SHOULDN'T change servers.

Rather, it should keep using the same server (even though it may be 
going through a different router).  Which is quite tricky to pull off in 
routing.  But easy if the UE uses the correct unicast address.

if the UE wants to recheck, it can issue a new query.  And then decide, 
if the answer changes, whether to do any of
1) stick with what it was using;
2) Switch immediately;
3) perform a gradual switch by interacting with the application servers 
in a sensible fashion.
Which of those makes sense depends upon the application and upon related 
properties that only the application knows.

Personally, I would not use some other server infrastructure than DNS 
for the query.  You can if you want.  Or maybe something like ALTO.  Or 
maybe something like LISP.  Or maybe something new.  Figure out what the 
actual constraints are, and then figure out what the answer looks like.

On the other hand, if you do not need flow stickiness, use anycast. 
Just don't put any application information into routing as it won't 
help.  And don't expect perfection.  (For the URLLC case without 
stickiness, one could even use flex-algo to have addresses that are 
routed by lowest cumulative link delay.)  Again, it depends upon what 
problem you really want to solve.

Yours,
Joel

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 29 Mar 2021 22:19:40 +0000

Joel, 

Thank you very much for the elaboration. 

The UEs (or the mobile devices, like Drones) don't know when they are anchored to a new cell tower. Therefore, it is not reasonable to require them to check DNS when they are re-anchored to a new cell tower. 

What we proposed is only on managing delivering packets based on  their destination addresses, nothing else. 
This NANOG talk  from an operator's perspective is more clear:  https://www.youtube.com/watch?v=kpx8u8bCIzs

Linda

-----Original Message-----
From: Joel M. Halpern <[email-masked]> 
Sent: Monday, March 29, 2021 5:04 PM
To: Linda Dunbar <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Earlier in the problem statement you described the need for flow stickiness.  Meaning, as I understand it, that the UE needs to keep using the same instance of the server as long as feasible.
The corollary of that seems to be that if the UE is moving frequently and not too far, it SHOULDN'T change servers.

Rather, it should keep using the same server (even though it may be going through a different router).  Which is quite tricky to pull off in routing.  But easy if the UE uses the correct unicast address.

if the UE wants to recheck, it can issue a new query.  And then decide, if the answer changes, whether to do any of
1) stick with what it was using;
2) Switch immediately;
3) perform a gradual switch by interacting with the application servers in a sensible fashion.
Which of those makes sense depends upon the application and upon related properties that only the application knows.

Personally, I would not use some other server infrastructure than DNS for the query.  You can if you want.  Or maybe something like ALTO.  Or maybe something like LISP.  Or maybe something new.  Figure out what the actual constraints are, and then figure out what the answer looks like.

On the other hand, if you do not need flow stickiness, use anycast. 
Just don't put any application information into routing as it won't help.  And don't expect perfection.  (For the URLLC case without stickiness, one could even use flex-algo to have addresses that are routed by lowest cumulative link delay.)  Again, it depends upon what problem you really want to solve.

Yours,
Joel

On 3/29/2021 5:16 PM, Linda Dunbar wrote:
> <[email-masked]>; 'IPv6 List' <[email-masked]>
> Subject: Re: Questions/comments about 
>> You can definitely use the Controller to choose the optimal server. 
>> To simplify the IP address allocation, the draft suggest using the 
>> Egress router addresses as the proxy for reaching the desired server. 
>> We can add a section to describe this scenario
>>
>> Linda
>>
>> To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) 
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furl
>> defense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook
>> .com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fn
>> am11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ie
>> tf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dun
>> bar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b2
>> 40189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZ
>> sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*
>> 3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2B
>> tIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NE
>> t6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nb
>> jgi-z*24%26amp%3Bdata%3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b12
>> 6cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7
>> C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
>> CJQIjoi
>>

---

From: Joel Halpern Direct <[email-masked]>
Date: Mon, 29 Mar 2021 18:34:44 -0400

I looked again at your draft.

First, one key assumption you make is very dangerous.  You assume that 
there are only a few sticky services.  Then you talk about Drones or 
IOT.  Those will use a multiplicity of different services.    More 
generally,as Mike O'Dell said year ago, scaling is the only hard 
problem.  Anything can be made to work if you pretend it does not need 
to scale.  The experience with our technology shows us that anything 
that is actually useful ends up needing to scale.

With that out of the way, your draft discusses a UE-oriented solution. 
But it assumes that the UE needs to get information at the IP layer.  By 
assumption, in your model, the UE got the "anycast" address from DNS. 
How about, instead of asking DNS in teh first place, it asked something 
else.  And then the application behavior fprovides whatever feedback is 
needed to know when to ask again (e.g. the way the streaming protocols 
do for rate adjustment).

And if you want to keep the UE out of it, don't use data plane sniffing 
to try to teach the edges about tunnels.  It produces a LOT of messes. 
Instead, if you really insist it be at the edge router, use something 
like LISP that can choose what granulartiy to ask for information, and 
has dealt with telling when the answers change, and all the other 
complexities you are sweeping under the covers.

I also note that having these debates in two places is veyr confusing. 
Your ask is indistinguishable from the ask of your colleagues on the 
dyncast list.

Yours,
Joel

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Tue, 30 Mar 2021 13:05:11 +0000

Hi Linda,

You're not "leverage network control plane" - you're adding complications to the network forwarding plane (and the control plane because you need to coordinate among the ingress routers) with that sticky-service-table with (Sticky Service ID, Flow Label. Sticky Egress address, Timer) entries. The 5G/MEC control plane solution I mentioned does not need a UE to query for a new address when its location changes.

The variation of that sticky-service-table based on (Sticky Service ID, UE address, Sticky Egress address, Timer) is slightly better, because it could be generalized to forwarding based on (source, destination), which is not different from IP multicast (except that there is no replication). While that reduces the complication in the forwarding plane, it still has the scaling issue - you will still have a large table and you will still have to coordinate among different ingress routers.

Granted, with the trendy AI, the coordination can be more precisely - the 5G system can predict where a UE is moving to and push the corresponding entry only to the new ingress router and ahead of time. If the UE address does not change after its anchoring UPF changes, the coordination can be simpler (otherwise the new ingress router needs to be able to associate the old and new addresses).

Retaining the same UE address after relocation can be done, and it is documented in " Solution #26: Persistent address allocation for mobile UEs that need MEC access" of 3GPP SA2 TR 23.748. I did not get to defend it during the final stage of study but I still think it is a viable solution (and preferred for application servers that do not deal well with client address changes) - would love to discuss that further with John on 3GPP SA2 mailing list.

Having said all the above, I still believe it's the easiest and most scalable solution if the UEs are told by 5G/MEC control plane what true unicast address of the server or load balancer to use.

Jeffrey

-----Original Message-----
From: Linda Dunbar <[email-masked]>
Sent: Monday, March 29, 2021 5:16 PM
To: Joel M. Halpern <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Joel,

Linda

-----Original Message-----
From: Joel M. Halpern <[email-masked]>
Sent: Monday, March 29, 2021 1:56 PM
To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]>; 'IPv6 List' <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Yours,
Joel

On 3/29/2021 1:30 PM, Linda Dunbar wrote:
> Administrative Requests: https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi__;JSUlJSUlJSUlJSUqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiUlJSoqKioqKioqKio!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8nYVAqHW$
>
> https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Fwww__;JSUl!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8ujmRF-e$ .

---

From: Mark Smith <[email-masked]>
Date: Wed, 31 Mar 2021 00:45:50 +1100

On Wed, 31 Mar 2021, 00:05 Jeffrey (Zhaohui) Zhang, <zzhang=
[email-masked]> wrote:

> Hi Linda,
>
> You're not "leverage network control plane" - you're adding complications
> to the network forwarding plane (and the control plane because you need to
> coordinate among the ingress routers) with that sticky-service-table with
> (Sticky Service ID, Flow Label. Sticky Egress address, Timer) entries. The
> 5G/MEC control plane solution I mentioned does not need a UE to query for a
> new address when its location changes.
>
> The variation of that sticky-service-table based on (Sticky Service ID, UE
> address, Sticky Egress address, Timer) is slightly better, because it could
> be generalized to forwarding based on (source, destination), which is not
> different from IP multicast (except that there is no replication). While
> that reduces the complication in the forwarding plane, it still has the
> scaling issue - you will still have a large table and you will still have
> to coordinate among different ingress routers.
>
> Granted, with the trendy AI, the coordination can be more precisely - the
> 5G system can predict where a UE is moving to and push the corresponding
> entry only to the new ingress router and ahead of time. If the UE address
> does not change after its anchoring UPF changes, the coordination can be
> simpler (otherwise the new ingress router needs to be able to associate the
> old and new addresses).
>
> Retaining the same UE address after relocation can be done, and it is
> documented in " Solution #26: Persistent address allocation for mobile UEs
> that need MEC access" of 3GPP SA2 TR 23.748. I did not get to defend it
> during the final stage of study but I still think it is a viable solution
> (and preferred for application servers that do not deal well with client
> address changes) - would love to discuss that further with John on 3GPP SA2
> mailing list.
>
> Having said all the above, I still believe it's the easiest and most
> scalable solution if the UEs are told by 5G/MEC control plane what true
> unicast address of the server or load balancer to use.
>

Anycast to initially set things up, switch to unicast once that is done so
there is no ongoing "anycast stickiness" required, and therefore no
corresponding ongoing "anycast stickiness" state required in the network
forwarding plane.

I think the existing and coming multipath transport layer protocols can
facilitate that. Initial connection setup via anycast, once that is
established, then establish 1 or more unicast sub-connections (subflows in
MPTCP) for ongoing communication, and abandon the anycast connection once
the first unicast sub-connection is established.

Anycast is used for setup, but not for ongoing communication.

See section 5.7.7 of this ID.

https://tools.ietf.org/id/draft-smith-6man-form-func-anycast-addresses-01.html#rfc.section.5.7.7

Regards,
Mark.

> To: Joel M. Halpern <[email-masked]>; Jeffrey (Zhaohui) Zhang <
> [email-masked]>; 'IPv6 List' <[email-masked]>
>     An end node (e.g. a drone) moves constantly. There are multiple
> servers hosted in Edge DCs close to Cell towers to process requests from
> the end node to achieve ultra-low latency.  It is not realistic for the end
> node to send DNS query every time it anchors to a new Cell Tower.
>
> To: Linda Dunbar <[email-masked]>; Jeffrey (Zhaohui) Zhang <
> [email-masked]>; 'IPv6 List' <[email-masked]>
> > The draft is to leverage the network control plane to determine which
> server is the most appropriate one for the device.
> >
> > Since it is for "sticky service", you don't want to get a new server
> address every time you move - unless the previous one is no longer
> appropriate. That means it is best for a controller to determine which one
> to use both initially and later when situation changes (when a UE relocates
> or server load situation changes), and that does not necessarily mean it is
> always through DNS.
> >
> > Jeffrey
> >
> > The Devices are moving consistently, it is not reasonable to require
> them to consistently query DNS for the "correct" non-ANYcast address .
> >
> > Linda
> >
> > Even if you could get over the security/trust hurdle, using a controller
> to let the UEs know which unicast non-anycast address to use is a much
> simpler/better solution.
> >
> > Jeffrey
> >
> > 5G Connected devices, such as drones for fighting fires or natural
> disasters or robots in Industry 4.0  environments,  need ultra-low latency
> responses from their analytic servers hosted in the Edge data centers. To
> reach ultra-low latency, there are multiple servers hosting the analytic
> functions in the Edge DCs.
> > All the functions (including networking and analytics) and devices are
> administrated by one operator.  Those functions might be provided by
> different vendors, therefore needing interoperable solutions.
> >
> > Linda
> >
> > - SRv6 Network Programming
> (draft-ietf-spring-srv6-network-programming-25) section 9 SRH RFC is
> especially verbal how to filter-out any SR-related information on the
> border of "SR domain".
> > Ed/
> > We can definitely add the option of UE inserting SRH. I am just not sure
> how many UEs or end devices will do those actions. If very few UEs can do
> this action, the solution itself is not useful. However, it doesn't hurt
> for IETF to specify such a solution so that future IoT or 5G devices can
> have a reference to do the actions.
> >
> > Another point, the number of Sticky Service is not large. The Ingress
> routers are configured with the policies ( ACLs) to filter those flows.
> >
> > Linda
> >
> > You proposed two ways of providing "sticky services" - when a UE moves
> to a new location, the ingress router at that new location will still route
> the packets of the same flow to the previous egress router. That flow
> cannot be identified by the destination address alone, since it is an
> anycast address that are shared by servers behind different egress routers.
> >
> > Essentially, you're trying to turn the ingress router into a load
> balancer, especially with your option #2 (section 5, "tunnel based"
> solution). I don't think we want the routers to do that - while routers can
> make use of 5-tuple for ECMP hashing, we don't want to make routers more
> complicated and do forwarding based on a sticky-service-table with (Sticky
> Service ID, Flow Label. Sticky Egress address, Timer) entries. It's not
> only complicated but also does not scale (we can discuss the scaling aspect
> wrt the flow labels separately).
> >
> > In that case, it would be better for the UE to learn the egress router
> via 5G/MEC control plane, instead of relying on the egress router to put
> that into the DOH of every server->UE packets for sticky services and for
> the UE to retrieve that information from each incoming sticky service
> packets. One thing I learned is that the entire 5G system is very much
> heavy with control/management plane and I would think it is a much better
> option to provide that information to the UEs.
> >
> > On the other hand, once you go that way, the control plane can simply
> provide the regular, non-anycast addresses of the servers instead of the
> egress router address. Then, all the problems disappear and corresponding
> proposals are no longer needed, including the ones in
> draft-dunbar-idr-5g-edge-compute-app-meta-data, and we only need existing
> simple routing functions.
> >
> > Thanks.
> >
> > Jeffrey
> >
> > A nit comment about "5G Core" above. When I first started learning 4G/5G
> It took me a while to realize the 3GPP "core network" concept in vastly
> different from what IETF people are used to. It's not about topology and
> now the "core network" functions are being more and more distributed into
> edges. Therefore, in this context it may be better to simply strike the
> "through 5G Core [5GC]" wording to reduce the confusion to some readers.
> >
> > [Linda] That is very true. Removed the term per your suggestion. 5G Core
> refers to all the functions from Radio to UPF.
> >
> > Not only BGP - but all IP routing protocols should work well with
> anycast. My understanding is that BGP being integral part here is really
> that the data network here is likely realized by VPNs over the same
> transport network. Is that a correct understanding?
> >
> > [Linda] ANYCAST has traditionally been used for servers or loader
> balancers that are placed in geographically diverse locations, so that BGP
> alone is enough for the traffic in one region to be forwarded to one
> server.  But for the 5G Edge Computing where multiple Servers/load
> Balancers with the same ANYCAST addresses are placed close proximity, IGP
> is needed.
> >
> > Of course, BGP does have flexibility in providing better/more control of
> route selection than IGP does in the context of the companion
> draft-dunbar-idr-5g-edge-compute-app-meta-data.
> > [Linda] Correct.
> >
> > As pointed out later in this same document, modern routers support "Flow
> Affinity" and should not cause packets of a flow on a specific router to be
> forwarded to different locations. The real problem is when a UE moves to a
> different location, the new router at that location may send it to a
> different egress router. However, that is the "sticky service" problem
> described in 1.4.
> > [Linda] Correct.
> >
> >>From draft-dunbar-idr-5g-edge-compute-app-meta-data, I understand that
> on a specific router it needs to choose a location that can best serve an
> application based on some non-routing factors. If 1.3 is really for that
> purpose, it should be reworded accordingly. As I mentioned in an earlier
> email, the two documents should better align on the problem descriptions.
> >
> > I was really confused by "next packet". I finally realized you may be
> referring to response packets from the UE to the server, and the "same
> flow" should be "same service". Better wording is needed here.
> >
> > Wouldn't it be better for the UE to put in an SRH with one SID for the
> server address and set the DA to be the egress router address? That way you
> don't need the ACL or the DOH (the Sticky-Dst-SubTLV  information in the
> DOH is not for consumption by the server anyway), and you don't even need
> tunneling or BGP (unless VPN is used - but that's orthogonal to this).
> Existing SRv6 function takes care of it.
> >
> > Also, the Sticky-Dst-SubTLV in DOH of the server->UE traffic would be
> better renamed as "return waypoint" for more generic purpose.
> > [Linda] it is about remembering which Egress router was used for the
> flow. Should it be "Sticky Egress Memory"?
> >
> > It seems that the same flow label is used for traffic of the same
> service in both directions. So who will assign the flow label?
> > If two UEs of the same service happen to use the same flow label, then
> sticky service is not guaranteed. For example, initially they're anchored
> at different UPFs, and UE1 traffic is sent to egress router 1 while UE2
> traffic is sent to egress router 2. When UE 1 relocates to the same UPF as
> UE 2's, its traffic will be sent to egress node 2 because the same flow
> label is used.
> >
> > Therefore, there should be a central controller to assign flow labels
> based on UE id, and the UE id is not based on IP address (since it could
> change).
> > [Linda] Since the "Flow Label" is randomly generated (by Host OS), the
> chance of two UEs reaching the same service having the same Flow Label is
> very small.  We can explore the option of getting the Control Plane
> involved.
> >
> > If a UE was using egress router 1 and it relocates to a new UPF, the new
> ingress router will likely have no corresponding entry for it? What if the
> new ingress router pick egress router 2?
> > I see it's discussed later that the routers do exchange the information.
> It should be mentioned up front when the table is introduced.
> > This confirms my earlier understanding for scenario 1 that "there should
> be a central controller to assign flow labels based on UE id, and the UE id
> is not based on IP address (since it could change)" and that the table
> could get large.
> >
> > Of course now for scenario 2, you're not using the flow label any more.
> While the table only contains entries that this ingress router actually
> need, the following are still true:
> > More importantly, this stateful flow steering based on the additional
> table is just too heavy and complicated. Why not simply have the UEs
> support SRH so that traffic will be routed via the desired egress router
> using standard SRv6 mechanism?
> >        1.4. Problem #2: Unbalanced Anycast Distribution due to UE
> Mobility.................................................. 7
> > The idr draft talk about "soft anchoring" problem and solution - how is
> that different from the "sticky service"?
> >
> > Thanks.
> > Jeffrey
> >
> https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3
>
>  d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi__;JSUlJSUlJSUlJSUqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiUlJSoqKioqKioqKio!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8nYVAqHW$
> >

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Tue, 30 Mar 2021 14:03:07 +0000

I missed this thread on the list so I was replying to an earlier email from Linda.

Now I want to re-emphasize that when I say 5G/MEC control plane can tell the UE what address to use, it does NOT mean a UE will query whenever it is anchored to a new UPF. The initial query when it needs a service will return the unicast address of a server based on all kinds of information/constraints, including the server load information talked about in draft-dunbar-idr-5g-edge-compute-app-meta-data. After that, the server won't change until new server is indeed needed (this is what "sticky service" is about, right). When a server change is indeed needed, the 5G/MEC control plane will tell the UE the new unicast address to use, and tell the servers to move the corresponding application data when necessary.

I've learned that many problems are best solved at application level instead of in the network.

Jeffrey

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Joel Halpern Direct
Sent: Monday, March 29, 2021 6:35 PM
To: Linda Dunbar <[email-masked]>; 'IPv6 List' <[email-masked]>; Luigi Iannone <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Yours,
Joel

On 3/29/2021 6:19 PM, Linda Dunbar wrote:
> This NANOG talk  from an operator's perspective is more clear:  https://urldefense.com/v3/__https://www.youtube.com/watch?v=kpx8u8bCIzs__;!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgDJ2j7Q3$
>
>
> Linda
>
>>> https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furl__;JSUl!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgCfRkXA4$
>>> https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Fwww__;JSUl!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgPGX_NBr$ .

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ipv6__;!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgIzVCuMJ$
--------------------------------------------------------------------

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 30 Mar 2021 15:38:16 +0000

Joel, 

Not every service on IoT devices or Drones needs to be sticky. For example, large amount of non-critical sensor data can be piped to databases, via tools like AWS Kenesis, Google Cloud Pub/Sub, or proprietary logging tools. 
Only a small number of ultra-low latency services from those mobile devices need servers placed close to the radio towers, like processing abnormal image captures that requires adjusting the amount of chemical to spray.  And only a subset of those ultra-low latency services need to be sticky. 

We have updated the draft to address comments from the meeting and the mailing list: 
https://datatracker.ietf.org/doc/draft-dunbar-6man-5g-edge-compute-sticky-service/

Replies to other comments are inserted below:

-----Original Message-----
From: Joel Halpern Direct <[email-masked]> 
Sent: Monday, March 29, 2021 5:35 PM
To: Linda Dunbar <[email-masked]>; 'IPv6 List' <[email-masked]>; Luigi Iannone <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

I looked again at your draft.
[Linda] Thank you. Please review the 03 version just uploaded. 

First, one key assumption you make is very dangerous.  You assume that there are only a few sticky services.  Then you talk about Drones or 
IOT.  Those will use a multiplicity of different services.    More 
generally,as Mike O'Dell said year ago, scaling is the only hard problem.  Anything can be made to work if you pretend it does not need to scale.  The experience with our technology shows us that anything that is actually useful ends up needing to scale.
[Linda] Agree with what you said. But can't figure out what exactly the scaling issues raised by the draft. 

With that out of the way, your draft discusses a UE-oriented solution. 
But it assumes that the UE needs to get information at the IP layer.  By assumption, in your model, the UE got the "anycast" address from DNS. 
How about, instead of asking DNS in teh first place, it asked something else.  And then the application behavior fprovides whatever feedback is needed to know when to ask again (e.g. the way the streaming protocols do for rate adjustment).
[Linda]Query DNS is the current practice for virtually all applications. What is the "Something Else" you are referring to? 
Maybe asking DNS to reply the IPv6  Optional Header together with the IPv6 address?  

And if you want to keep the UE out of it, don't use data plane sniffing to try to teach the edges about tunnels.  It produces a LOT of messes. 
Instead, if you really insist it be at the edge router, use something like LISP that can choose what granulartiy to ask for information, and has dealt with telling when the answers change, and all the other complexities you are sweeping under the covers.
[Linda] Yes, we can consider LISP. IP-in-IP with edge nodes being the LDN Ingress routers and Egress routers? How is LISP different from other tunnels?  

I also note that having these debates in two places is veyr confusing. 
[Linda] Is it better to lump all the discussion to the Dyncast list?  We can do. 
Your ask is indistinguishable from the ask of your colleagues on the dyncast list.

Yours,
Joel

On 3/29/2021 6:19 PM, Linda Dunbar wrote:
> This NANOG talk  from an operator's perspective is more clear:  
> youtube.com%2Fwatch%3Fv%3Dkpx8u8bCIzs&amp;data=04%7C01%7Clinda.dunbar%
> 40futurewei.com%7C77620acae4b7451906db08d8f302dc96%7C0fee8ff2a3b240189
> c753a1d5591fedc%7C1%7C0%7C637526540924217102%7CUnknown%7CTWFpbGZsb3d8e
> yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C10
> 00&amp;sdata=5CbWcBskMIyzDdwH94PWTFKf%2BTG7HJFZSATIpWywc0c%3D&amp;rese
> rved=0
> 
> 
> Linda
> 
> if the UE wants to recheck, it can issue a new query.  And then 
> decide, if the answer changes, whether to do any of
>> Zhang <[email-masked]>; 'IPv6 List' <[email-masked]>
>>> To: Vasilenko Eduard <[email-masked]>; Jeffrey 
>>> (Zhaohui) Zhang <[email-masked]>; Kaippallimalil John 
>>> To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil 
>>> John <[email-masked]>; 'IPv6 List' 
>>> <[email-masked]>
>>> To: Jeffrey (Zhaohui) Zhang <[email-masked]>; Kaippallimalil 
>>> John <[email-masked]>; 'IPv6 List' 
>>> <[email-masked]>
>>> IETF IPv6 working group mailing list [email-masked] Administrative 
>>> Requests:
>>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000%26amp%3Bsdata%3DaOw1Dk
>>> D
>>> c
>>> Qu0*2FmMi6RfWlUyRcLB2jRcsbBAhcpoaX5yE*3D%26amp%3Breserved%3D0__%3BJS
>>> U
>>> l
>>> JSUlJSUlJSUqKioqKiolJSUqKioqKioqKioqKiolJSUqKioqJSUlJSUlJSUlJSUlJSUl
>>> J
>>> S
>>> UlJQ!!NEt6yMaO-gk!Q-hLtDzPuot4CQsvyUhfrEcNgHIIBEdRDT4RgyHgVCE1f5Vt6D
>>> l
>>> v
>>> zYC-o7693kZ1%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C67
>>> c
>>> 9
>>> 07b1d8e643cc5a5108d8f2d6f104%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%
>>> 7
>>> C
>>> 0%7C637526352290503133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
>>> C
>>> J
>>> QIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Be%2FU8
>>> M
>>> h
>>> IETF IPv6 working group mailing list [email-masked] Administrative 
>>> Requests:
>>> ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar
>>> %
>>> 4
>>> 0futurewei.com%7Ca73842e8d0e7473e591d08d8f2e43b9f%7C0fee8ff2a3b24018
>>> 9
>>> c
>>> 753a1d5591fedc%7C1%7C0%7C637526409383434704%7CUnknown%7CTWFpbGZsb3d8
>>> e
>>> y
>>> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1
>>> 0
>>> 0
>>> 0&amp;sdata=qkqfI%2B3ZLo76yZFXDIxUxjfwyhA5MNJIqUwzUyTGXqc%3D&amp;res
>>> e
>>> r
>>> ved=0

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 30 Mar 2021 16:26:24 +0000

Jeffrey, 

Questions to your suggestion are inserted below:

-----Original Message-----
From: Jeffrey (Zhaohui) Zhang <[email-masked]> 
Sent: Tuesday, March 30, 2021 9:03 AM
To: Joel Halpern Direct <[email-masked]>; Linda Dunbar <[email-masked]>; 'IPv6 List' <[email-masked]>; Luigi Iannone <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Now I want to re-emphasize that when I say 5G/MEC control plane can tell the UE what address to use, it does NOT mean a UE will query whenever it is anchored to a new UPF. The initial query when it needs a service will return the unicast address of a server based on all kinds of information/constraints, including the server load information talked about in draft-dunbar-idr-5g-edge-compute-app-meta-data. 
[Linda] Are you suggesting having 5G/MEC as the proxy DNS? 5G/MEC is informed of the capacity information of multiple servers? 

After that, the server won't change until new server is indeed needed (this is what "sticky service" is about, right). When a server change is indeed needed, the 5G/MEC control plane will tell the UE the new unicast address to use, and tell the servers to move the corresponding application data when necessary.
[Linda] Does it mean a new protocol has to be created for the 5G/MEC to send a new IP address for the service? 

I've learned that many problems are best solved at application level instead of in the network.
[Linda] 100% agree with. Network is only responsible for forwarding the packets based on packets' destination address. If the Application wants to have multiple servers with the same address to simplify the address management, network is capable to forwarding them. 
This NANOG talk  from an operator's perspective is more clear:  https://www.youtube.com/watch?v=kpx8u8bCIzs
Linda

Jeffrey

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Joel Halpern Direct
Sent: Monday, March 29, 2021 6:35 PM
To: Linda Dunbar <[email-masked]>; 'IPv6 List' <[email-masked]>; Luigi Iannone <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Yours,
Joel

On 3/29/2021 6:19 PM, Linda Dunbar wrote:
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furld
> efense.com%2Fv3%2F__https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dkpx8u8
> bCIzs__%3B!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc4
> 7YKlit19tYgDJ2j7Q3%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%
> 7C43c1aaaec63746e6db3c08d8f3849141%7C0fee8ff2a3b240189c753a1d5591fedc%
> 7C1%7C0%7C637527098001516504%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cf4
> H97UBHp6abX2scn08a%2FJ33b7OPzAHRKjqBoYyUic%3D&amp;reserved=0
>
>
> Linda
>
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fur
>>> ldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlo
>>> ok.com%2F%3Furl%3Dhttps*3A*2F*2Furl__%3BJSUl!!NEt6yMaO-gk!XDfApEJMg4
>>> nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgCfRkXA4%24&amp;data=0
>>> 4%7C01%7Clinda.dunbar%40futurewei.com%7C43c1aaaec63746e6db3c08d8f384
>>> 9141%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527098001516504
>>> %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
>>> 6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=oo2uYLAyN%2FV%2BjxAnRKj1F6r
>>> zzEgLfVjGyhEUJ5Fb2Q0%3D&amp;reserved=0
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fwww__%3BJSUl!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgPGX_NBr%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C43c1aaaec63746e6db3c08d8f3849141%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527098001516504%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=8mVrzjHC1Lt7yEJNaXuoo6rNFGPLtNDN3JaevRCWzEE%3D&amp;reserved=0 .

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6__%3B!!NEt6yMaO-gk!XDfApEJMg4nih6Ax7AEqq3RKhWt2T6FpqVyw1lEj_ykCc47YKlit19tYgIzVCuMJ%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C43c1aaaec63746e6db3c08d8f3849141%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527098001516504%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0s8oELwd%2BmD%2FtGLKWUiSpK%2BYn8ADTl6ZgjkT3iF7p8w%3D&amp;reserved=0
--------------------------------------------------------------------

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 31 Mar 2021 14:10:43 -0400

There seems to be a leap in the description of flow handling in routers.

As far as I know, most routers go to some trouble to make sure that 
packets of a flow follow the same path.
They do NOT, to the best of my knowledge, do this by establishing flow 
state (soft or otherwise).  They do so by hashing fields in the packet 
when selecting among equal cost paths.
Also, they make no effort to provide anything related to flow stability 
when routing changes.

Yours,
Joel

PS: Trying to make anycast do the right thing when servers fail depends 
very heavily on what you define as "right".  If 'whatever routing does" 
works for you, then sure, ti does the "right" thing.  If you want the 
flows to go to the instance which has been sharing operational state 
with the failed instance, then no, anycast will not do what you want.

On 3/30/2021 3:24 PM, Kaippallimalil John wrote:
> Hi Mark, Jeffrey,
> 
> I see the point that Jeffrey raises regarding scaling as the sticky 
> service can result in not just having flow state in routers on path, but 
> that it is also flooded in the network.
> 
> We’ll think of other solutions that do not require flow state to be flooded.
> 
> Re: Mark’s comments and considering just unipath flows for now.
> 
> The assumption we had is that routers on path keep flow soft state so 
> that all packets of a flow follow the same path.
> 
> My understanding is that routers do this already (Jeffrey mentioned this 
> also in the earlier response too)
> 
> Anycast looked attractive because it could provide fault tolerance 
> against server failure, not only at session setup,  but also for the 
> duration of the session.
> 
> For edge computing scenarios with small remote sites/few servers, 
> anycast could offer automatic server reselection on site failure.
> 
> 0-RTT connection setup with QUIC makes this more appealing too (and TLS 
> state, data replicated across other local servers/sites as needed by the 
> application)
> 
> Regards,
> 
> John
> 
> *From:* Mark Smith <[email-masked]>
> *Sent:* Tuesday, March 30, 2021 8:46 AM
> *To:* Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
> *Cc:* Linda Dunbar <[email-masked]>; Joel M. Halpern 
> <[email-masked]>; IPv6 List <[email-masked]>; Kaippallimalil John 
> <[email-masked]>
> *Subject:* Re: Questions/comments about 
> On Wed, 31 Mar 2021, 00:05 Jeffrey (Zhaohui) Zhang, 
> <zzhang=[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     Hi Linda,
> 
>     You're not "leverage network control plane" - you're adding
>     complications to the network forwarding plane (and the control plane
>     because you need to coordinate among the ingress routers) with that
>     sticky-service-table with (Sticky Service ID, Flow Label. Sticky
>     Egress address, Timer) entries. The 5G/MEC control plane solution I
>     mentioned does not need a UE to query for a new address when its
>     location changes.
> 
>     The variation of that sticky-service-table based on (Sticky Service
>     ID, UE address, Sticky Egress address, Timer) is slightly better,
>     because it could be generalized to forwarding based on (source,
>     destination), which is not different from IP multicast (except that
>     there is no replication). While that reduces the complication in the
>     forwarding plane, it still has the scaling issue - you will still
>     have a large table and you will still have to coordinate among
>     different ingress routers.
> 
>     Granted, with the trendy AI, the coordination can be more precisely
>     - the 5G system can predict where a UE is moving to and push the
>     corresponding entry only to the new ingress router and ahead of
>     time. If the UE address does not change after its anchoring UPF
>     changes, the coordination can be simpler (otherwise the new ingress
>     router needs to be able to associate the old and new addresses).
> 
>     Retaining the same UE address after relocation can be done, and it
>     is documented in " Solution #26: Persistent address allocation for
>     mobile UEs that need MEC access" of 3GPP SA2 TR 23.748. I did not
>     get to defend it during the final stage of study but I still think
>     it is a viable solution (and preferred for application servers that
>     do not deal well with client address changes) - would love to
>     discuss that further with John on 3GPP SA2 mailing list.
> 
>     scalable solution if the UEs are told by 5G/MEC control plane what
>     true unicast address of the server or load balancer to use.
> 
> Anycast to initially set things up, switch to unicast once that is done 
> so there is no ongoing "anycast stickiness" required, and therefore no 
> established, then establish 1 or more unicast sub-connections (subflows 
> in MPTCP) for ongoing communication, and abandon the anycast connection 
> once the first unicast sub-connection is established.
> 
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fid%2Fdraft-smith-6man-form-func-anycast-addresses-01.html%23rfc.section.5.7.7&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828345939%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eqBsEMhYzM8WWrL83LjCfbzn8CKCrhBKMUrt%2Fc9br2M%3D&reserved=0>
> 
> Regards,
> 
> Mark.
> 
> 
>     Jeffrey
> 
>     From: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>
>     To: Joel M. Halpern <[email-masked]
>     <mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang
>     <[email-masked] <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>     servers hosted in Edge DCs close to Cell towers to process requests
>     from the end node to achieve ultra-low latency.  It is not realistic
>     for the end node to send DNS query every time it anchors to a new
>     Cell Tower.
> 
>     The suggested solution is to leverage network control plane to find
>     the optimal server to serve the end node.
> 
>     From: Joel M. Halpern <[email-masked] <mailto:[email-masked]>>
>     To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang
>     <[email-masked] <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>     As Jeffrey has pointed out, given the problem you have described, it
>     is not at all clear that is the best answer, or even a good one.
> 
>     Yours,
>     Joel
> 
>      > The draft is to leverage the network control plane to determine
>     which server is the most appropriate one for the device.
>      >
>      > From: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Vasilenko Eduard
>      > <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > Since it is for "sticky service", you don't want to get a new
>     server address every time you move - unless the previous one is no
>     longer appropriate. That means it is best for a controller to
>     determine which one to use both initially and later when situation
>     changes (when a UE relocates or server load situation changes), and
>     that does not necessarily mean it is always through DNS.
>      >
>      > Jeffrey
>      >
>      > From: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>; Vasilenko Eduard
>      > <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > The Devices are moving consistently, it is not reasonable to
>     require them to consistently query DNS for the "correct" non-ANYcast
>     address .
>      >
>      > Linda
>      >
>      > From: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Vasilenko Eduard
>      > <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > Even if you could get over the security/trust hurdle, using a
>     controller to let the UEs know which unicast non-anycast address to
>     use is a much simpler/better solution.
>      >
>      > Jeffrey
>      >
>      > From: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Vasilenko Eduard <[email-masked]
>     <mailto:[email-masked]>>; Jeffrey (Zhaohui)
>      > Zhang <[email-masked] <mailto:[email-masked]>>;
>     Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > 5G Connected devices, such as drones for fighting fires or
>     natural disasters or robots in Industry 4.0  environments,  need
>     ultra-low latency  responses from their analytic servers hosted in
>     the Edge data centers. To reach ultra-low latency, there are
>     multiple servers hosting the analytic functions in the Edge DCs.
>      > All the functions (including networking and analytics) and
>     devices are administrated by one operator.  Those functions might be
>     provided by different vendors, therefore needing interoperable
>     solutions.
>      >
>      > Linda
>      >
>      > From: Vasilenko Eduard <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Jeffrey (Zhaohui) Zhang
>      > <[email-masked] <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > Segment Routing in general and SRv6 in particular are claimed to
>     be designed for Trusted environments only:
>     especially verbal how to filter-out any SR-related information on
>     the border of "SR domain".
>      > Ed/
>      > From: ipv6 [mailto:[email-masked]
>     <mailto:[email-masked]>] On Behalf Of Linda Dunbar
>      > To: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > We can definitely add the option of UE inserting SRH. I am just
>     not sure how many UEs or end devices will do those actions. If very
>     few UEs can do this action, the solution itself is not useful.
>     However, it doesn't hurt for IETF to specify such a solution so that
>     future IoT or 5G devices can have a reference to do the actions.
>      >
>      > Another point, the number of Sticky Service is not large. The
>     Ingress routers are configured with the policies ( ACLs) to filter
>     those flows.
>      >
>      > Linda
>      >
>      > From: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > You proposed two ways of providing "sticky services" - when a UE
>     moves to a new location, the ingress router at that new location
>     will still route the packets of the same flow to the previous egress
>     router. That flow cannot be identified by the destination address
>     alone, since it is an anycast address that are shared by servers
>     behind different egress routers.
>      >
>     solution). I don't think we want the routers to do that - while
>     routers can make use of 5-tuple for ECMP hashing, we don't want to
>     make routers more complicated and do forwarding based on a
>     sticky-service-table with (Sticky Service ID, Flow Label. Sticky
>     Egress address, Timer) entries. It's not only complicated but also
>     does not scale (we can discuss the scaling aspect wrt the flow
>     labels separately).
>      >
>      > In that case, it would be better for the UE to learn the egress
>     router via 5G/MEC control plane, instead of relying on the egress
>     router to put that into the DOH of every server->UE packets for
>     sticky services and for the UE to retrieve that information from
>     each incoming sticky service packets. One thing I learned is that
>     the entire 5G system is very much heavy with control/management
>     plane and I would think it is a much better option to provide that
>     information to the UEs.
>      >
>      > On the other hand, once you go that way, the control plane can
>     simply provide the regular, non-anycast addresses of the servers
>     instead of the egress router address. Then, all the problems
>     disappear and corresponding proposals are no longer needed,
>     including the ones in
>     draft-dunbar-idr-5g-edge-compute-app-meta-data, and we only need
>     existing simple routing functions.
>      >
>      > Thanks.
>      >
>      > Jeffrey
>      >
>      > From: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > From: Jeffrey (Zhaohui) Zhang <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; 'IPv6 List'
>     <[email-masked] <mailto:[email-masked]>>
>      > A nit comment about "5G Core" above. When I first started
>     learning 4G/5G It took me a while to realize the 3GPP "core network"
>     concept in vastly different from what IETF people are used to. It's
>     not about topology and now the "core network" functions are being
>     more and more distributed into edges. Therefore, in this context it
>     may be better to simply strike the "through 5G Core [5GC]" wording
>     to reduce the confusion to some readers.
>      >
>      > [Linda] That is very true. Removed the term per your suggestion.
>     5G Core refers to all the functions from Radio to UPF.
>      >
>     anycast. My understanding is that BGP being integral part here is
>     really that the data network here is likely realized by VPNs over
>     the same transport network. Is that a correct understanding?
>      >
>     balancers that are placed in geographically diverse locations, so
>     that BGP alone is enough for the traffic in one region to be
>     forwarded to one server.  But for the 5G Edge Computing where
>     multiple Servers/load Balancers with the same ANYCAST addresses are
>     placed close proximity, IGP is needed.
>      >
>      > Of course, BGP does have flexibility in providing better/more
>     control of route selection than IGP does in the context of the
>     companion draft-dunbar-idr-5g-edge-compute-app-meta-data.
>      > [Linda] Correct.
>      >
>      > As pointed out later in this same document, modern routers
>     support "Flow Affinity" and should not cause packets of a flow on a
>     specific router to be forwarded to different locations. The real
>     problem is when a UE moves to a different location, the new router
>     at that location may send it to a different egress router. However,
>     that is the "sticky service" problem described in 1.4.
>      > [Linda] Correct.
>      >
>      >>From draft-dunbar-idr-5g-edge-compute-app-meta-data, I understand
>     that on a specific router it needs to choose a location that can
>     best serve an application based on some non-routing factors. If 1.3
>     is really for that purpose, it should be reworded accordingly. As I
>     mentioned in an earlier email, the two documents should better align
>     on the problem descriptions.
>      >
>      > I was really confused by "next packet". I finally realized you
>     may be referring to response packets from the UE to the server, and
>     the "same flow" should be "same service". Better wording is needed here.
>      >
>      > Wouldn't it be better for the UE to put in an SRH with one SID
>     for the server address and set the DA to be the egress router
>     address? That way you don't need the ACL or the DOH (the
>     Sticky-Dst-SubTLV  information in the DOH is not for consumption by
>     the server anyway), and you don't even need tunneling or BGP (unless
>     VPN is used - but that's orthogonal to this). Existing SRv6 function
>     takes care of it.
>      >
>      > Also, the Sticky-Dst-SubTLV in DOH of the server->UE traffic
>     would be better renamed as "return waypoint" for more generic purpose.
>      > [Linda] it is about remembering which Egress router was used for
>     the flow. Should it be "Sticky Egress Memory"?
>      >
>      > If two UEs of the same service happen to use the same flow label,
>     then sticky service is not guaranteed. For example, initially
>     they're anchored at different UPFs, and UE1 traffic is sent to
>     egress router 1 while UE2 traffic is sent to egress router 2. When
>     UE 1 relocates to the same UPF as UE 2's, its traffic will be sent
>     to egress node 2 because the same flow label is used.
>      >
>      > Therefore, there should be a central controller to assign flow
>     labels based on UE id, and the UE id is not based on IP address
>     (since it could change).
>      > [Linda] Since the "Flow Label" is randomly generated (by Host
>     OS), the chance of two UEs reaching the same service having the same
>     Flow Label is very small.  We can explore the option of getting the
>     Control Plane involved.
>      >
>      > If a UE was using egress router 1 and it relocates to a new UPF,
>     the new ingress router will likely have no corresponding entry for
>     it? What if the new ingress router pick egress router 2?
>      > I see it's discussed later that the routers do exchange the
>     information. It should be mentioned up front when the table is
>     introduced.
>      > This confirms my earlier understanding for scenario 1 that "there
>     should be a central controller to assign flow labels based on UE id,
>     and the UE id is not based on IP address (since it could change)"
>     and that the table could get large.
>      >
>      > Of course now for scenario 2, you're not using the flow label any
>     more. While the table only contains entries that this ingress router
>     actually need, the following are still true:
>      > More importantly, this stateful flow steering based on the
>     additional table is just too heavy and complicated. Why not simply
>     have the UEs support SRH so that traffic will be routed via the
>     desired egress router using standard SRv6 mechanism?
>      > To: Linda Dunbar <[email-masked]
>     <mailto:[email-masked]>>; Kaippallimalil John
>      > <[email-masked]
>     <mailto:[email-masked]>>; IPv6 List
>     <[email-masked] <mailto:[email-masked]>>;
>      > idr@ietf. org <[email-masked] <mailto:[email-masked]>>
>      > Subject: questions about
>      > Why is problem #2 different in the two drafts? Is it true that
>     none of the two drafts address problem #3?
>      > The idr draft talk about "soft anchoring" problem and solution -
>     how is that different from the "sticky service"?
>      >
>      > Thanks.
>      > Jeffrey
>      >
>      > [email-masked] <mailto:[email-masked]>
>      > Administrative Requests:
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828355894%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ure%2FcliK7wEXTkuLU9iefC2s5cRqWa%2FwE6eIF%2BHdHfA%3D&reserved=0>
>      > zYC-o7693kZ1%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com
>     <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40futurewei.com%2F&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828355894%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1Uxr81kMC6Cjobd1bqaqiUD13fgwne6OU1q06zG7pu4%3D&reserved=0>%7C67c9
>      >
>      > [email-masked] <mailto:[email-masked]>
>      > Administrative Requests:
>      >
>     https://urldefense.com/v3/__https://nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Fwww__;JSUl!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8ujmRF-e$
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fwww__%3BJSUl!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8ujmRF-e%24&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828365851%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XV7meGtr1JyhhcB2sQnxs6CpK8D7dklhnbyptZuFsMw%3D&reserved=0>
>     .
>      > ietf.org
>     <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fietf.org%2F&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828365851%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vLVhL9VnBIt2q3L1ORqyuEVlgNpI4HyYeqxPUWMYQ7A%3D&reserved=0>%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%4
>      > 0futurewei.com
>     <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F0futurewei.com%2F&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828604803%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xkXn8k5O54INeb7ZjNHWfjdUfROEbZpOlNVU26ubx3c%3D&reserved=0>%7Ca73842e8d0e7473e591d08d8f2e43b9f%7C0fee8ff2a3b240189c
>      >
>     [email-masked] <mailto:[email-masked]>
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&data=04%7C01%7Cjohn.kaippallimalil%40futurewei.com%7C024d9854420c4d3bf75108d8f3823196%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637527087828614761%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MbXMAKV27TYIDRerkpnD7qU%2BBShP%2Bp%2FMkmYTF2Zo%2FQE%3D&reserved=0>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 31 Mar 2021 18:49:49 +0000

Joel, 

To routers with multiple paths to one Address, they can use the cost propagated by the IGP/BGP messages. 
Most commercial routers support Flow Affinity. 

Linda

-----Original Message-----
From: Joel M. Halpern <[email-masked]> 
Sent: Wednesday, March 31, 2021 1:11 PM
To: Kaippallimalil John <[email-masked]>; Mark Smith <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Cc: Linda Dunbar <[email-masked]>; IPv6 List <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

As far as I know, most routers go to some trouble to make sure that packets of a flow follow the same path.
They do NOT, to the best of my knowledge, do this by establishing flow state (soft or otherwise).  They do so by hashing fields in the packet when selecting among equal cost paths.
Also, they make no effort to provide anything related to flow stability when routing changes.

Yours,
Joel

PS: Trying to make anycast do the right thing when servers fail depends very heavily on what you define as "right".  If 'whatever routing does" 
works for you, then sure, ti does the "right" thing.  If you want the flows to go to the instance which has been sharing operational state with the failed instance, then no, anycast will not do what you want.

On 3/30/2021 3:24 PM, Kaippallimalil John wrote:
> service can result in not just having flow state in routers on path, 
> but that it is also flooded in the network.
> 
> We'll think of other solutions that do not require flow state to be flooded.
> 
> Re: Mark's comments and considering just unipath flows for now.
> 
> My understanding is that routers do this already (Jeffrey mentioned 
> this also in the earlier response too)
> 
> 0-RTT connection setup with QUIC makes this more appealing too (and 
> TLS state, data replicated across other local servers/sites as needed 
> by the
> application)
> 
> Regards,
> 
> John
> 
> Anycast to initially set things up, switch to unicast once that is 
> done so there is no ongoing "anycast stickiness" required, and 
> therefore no corresponding ongoing "anycast stickiness" state required 
> in the network forwarding plane.
> 
> I think the existing and coming multipath transport layer protocols 
> can facilitate that. Initial connection setup via anycast, once that 
> is established, then establish 1 or more unicast sub-connections 
> (subflows in MPTCP) for ongoing communication, and abandon the anycast 
> connection once the first unicast sub-connection is established.
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftool
> s.ietf.org%2Fid%2Fdraft-smith-6man-form-func-anycast-addresses-01.html
> %23rfc.section.5.7.7&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7
> C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7
> C1%7C0%7C637528110502834818%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WnFI
> MnsjGb%2B8ycljBXslxndaC5ZI0pRnipaXbv7CokM%3D&amp;reserved=0
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftoo
> ls.ietf.org%2Fid%2Fdraft-smith-6man-form-func-anycast-addresses-01.htm
> l%23rfc.section.5.7.7&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%
> 7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%
> 7C1%7C0%7C637528110502834818%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WnF
> IMnsjGb%2B8ycljBXslxndaC5ZI0pRnipaXbv7CokM%3D&amp;reserved=0>
> 
> Regards,
> 
> Mark.
> 
> 
>     Jeffrey
> 
>     https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502834818%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=TNG%2BMNWM9kwvdNCBeVXZtkNVDihZIuBfRRdwsaDkWXM%3D&amp;reserved=0
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fnam11.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fipv6*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C4209e8a9acae47b96d1808d8f2d16b8d*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526328578769822*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26amp*3Bsdata*3DA39pqHVUBFsssO3DLSqrTUtPpcXAr*2F8pi*2Bmw*2BtIJNME*3D*26amp*3Breserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!NEt6yMaO-gk!QWI34EOzIdgzRLkkdD3rdv_fn4CLHXnnMvDpOOeQB4ELlElbfawu6WXv0nbjgi-z*24*26amp*3Bdata*3D04*7C01*7Clinda.dunbar*40futurewei.com*7C2b126cc5be8541bd43cc08d8f2d4a7ab*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637526342463012256*7CUnknown*7CTWFpbGZsb3&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502834818%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=TNG%2BMNWM9kwvdNCBeVXZtkNVDihZIuBfRRdwsaDkWXM%3D&amp;reserved=0>
>     <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40futurewei.com%2F&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=2xaPm869MrUiDL9%2ByoTEsqBktpf6ID8b9Mp8rxGPs7E%3D&amp;reserved=0>%7C67c9
>      >
>     https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fwww__%3BJSUl!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8ujmRF-e%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=XEOWX%2Fl8GESNNafvu%2BtekCqSSlgs%2F2l8HLGXyw8U%2Fn0%3D&amp;reserved=0
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fwww__%3BJSUl!!NEt6yMaO-gk!VemUe-2PHPVnuVFBu0NeOKMAx3tulW_X0zDToehm7avqQhBqbE3FDykd8ujmRF-e%24&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=XEOWX%2Fl8GESNNafvu%2BtekCqSSlgs%2F2l8HLGXyw8U%2Fn0%3D&amp;reserved=0>
>     .
>      > ietf.org
>     <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fietf.org%2F&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=n3zK2zENL16JEkq4iwW19VBUnSB3ii78O%2FIpsROC7hA%3D&amp;reserved=0>%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%4
>      > 0futurewei.com
>     <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F0futurewei.com%2F&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=XAXtrpwH%2FQ4ij8PcDUP6VViJLdjA1vOpoNifkmdyoQo%3D&amp;reserved=0>%7Ca73842e8d0e7473e591d08d8f2e43b9f%7C0fee8ff2a3b240189c
>      >
>     Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=klLXVxQwPzN2pbIBqpBGuOmFnNbQlk468c60TOiErXY%3D&amp;reserved=0
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=klLXVxQwPzN2pbIBqpBGuOmFnNbQlk468c60TOiErXY%3D&amp;reserved=0>
>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 31 Mar 2021 18:58:02 +0000

Joel, 

When multiple servers have the same IP address (i.e. the ANYCAST), the cost propagated by IGP/BGP  to a failed server will be huge, triggering  routers to select a path to another server with the same IP address. 
That is how ANYCAST can eliminate the bottleneck to one specific server. 

Linda

-----Original Message-----
From: Linda Dunbar 
Sent: Wednesday, March 31, 2021 1:50 PM
To: Joel M. Halpern <[email-masked]>; Kaippallimalil John <[email-masked]>; Mark Smith <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Cc: IPv6 List <[email-masked]>
Subject: RE: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Joel, 

Linda

-----Original Message-----
From: Joel M. Halpern <[email-masked]>
Sent: Wednesday, March 31, 2021 1:11 PM
To: Kaippallimalil John <[email-masked]>; Mark Smith <[email-masked]>; Jeffrey (Zhaohui) Zhang <zzhang=[email-masked]>
Cc: Linda Dunbar <[email-masked]>; IPv6 List <[email-masked]>
Subject: Re: Questions/comments about draft-dunbar-6man-5g-edge-compute-sticky-service

Yours,
Joel

On 3/30/2021 3:24 PM, Kaippallimalil John wrote:
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
> .ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Clinda.dunbar%
> 40futurewei.com%7C58c60724b1fe434f313108d8f4704e73%7C0fee8ff2a3b240189
> c753a1d5591fedc%7C1%7C0%7C637528110502844816%7CUnknown%7CTWFpbGZsb3d8e
> 00&amp;sdata=klLXVxQwPzN2pbIBqpBGuOmFnNbQlk468c60TOiErXY%3D&amp;reserv
> ed=0>
>