Subject: [Dyncast] Feedback on CAN BoF

From: Jens Finkhaeuser <[email-masked]>
Date: Wed, 23 Mar 2022 09:48:00 +0000

Hi,

I was invited yesterday to attend the BoF, and listening to the talks, I have some general feedback - not specifically to any presenter, but to the problem statement as such.

Back in 2006, I worked at Joost, where my team and I brought peer-to-peer video streaming to the Internet, first in an on-demand, and in 2008 in a live fashion. The topics of the talks I saw is very strongly reminiscent of the engineering work we performed back then, which unfortunately is not my intellectual property.

There are, however, a few experiences I can relate that might be of interest to this group. It should go without saying that the purpose of peer-to-peer streaming was very much the same as faced in the edge computing scenarios presented, namely to offload data centers and core internet infrastructure, and utilize such resources at or near the edge as could be found.

As video streaming data apparently constitutes the vast majority of internet traffic, it is maybe somewhat counter-intuitive that throughput matters less for end-user experience than latency. There have been studies (which I do not have at my fingertips, but I can dig up of there is interest) that a skipped video frame or three is barely noticeable, but stutter immediately and severely degrades the viewing experience, which is one of the reasons why players pre-buffer video content.

Without going into too much detail here, relaying live video streams poses several interesting challenges which are most easily solved by assigning different roles to different nodes contributing to a stream. In terms more relevant to this group, even though conceptually "streaming a video" is a single large request, it can be best fulfilled by making many small requests to different nodes providing different services.

Which raises a few points for consideration that are somewhat intended as feedback to comments made during the session:

1.  Session stickiness is not necessarily a desirable feature, or at least requires a precise definition what constitutes a session. While it is clear that each individual small request should be fulfilled by the node to which the request was made, the overall "streaming session" may best be highly distributed. That alone probably provides for some interesting discussion.
2.  As latency matters, so does the latency for resolving the optimal node for each mini request. Whether due to peer-to-peer nodes simply disappearing or an edge node receiving a sudden influx of requests, good and efficient node selection is a challenge.
3.  The metrics question that was raised in various forms is valid, but obscures the real issue: which metrics apply for node selection is highly dependent on the role of the service providing node. The only way this can be effectively generalized is by making routers unaware of the semantics of the metrics, and leave this in the application space. That is, there is only one metric for routers to consider: a sliding scale of "good" to "bad" values for a given application's specific service type. That also makes the selection algorithms comparatively simple. From this follow quite a few implications for how service providing nodes might contribute to this decision making process by providing the respective metric about themselves.
4.  Given such a simple service-defined metric, it is often possible for the client machine to make the optimal routing decision - this is especially the case if it is multi-homed.

I hope this provides some useful feedback. If you have questions or comments, by all means let me know!

I am very happy to see that this group is taking the very specific issues we solved back in the day and tackle the more generalized problem!

Jens

---

From: Dirk Trossen <[email-masked]>
Date: Wed, 23 Mar 2022 11:50:57 +0000

Hi Jens,

Thanks for the feedback and comments below and great that you made the way to Vienna yesterday!

More inline.

Best,

Dirk

From: Dyncast [mailto:[email-masked]] On Behalf Of Jens Finkhaeuser
Sent: 23 March 2022 10:48
To: [email-masked]
Subject: [Dyncast] Feedback on CAN BoF

Hi,

As video streaming data apparently constitutes the vast majority of internet traffic, it is maybe somewhat counter-intuitive that throughput matters less for end-user experience than latency. There have been studies (which I do not have at my fingertips, but I can dig up of there is interest) that a skipped video frame or three is barely noticeable, but stutter immediately and severely degrades the viewing experience, which is one of the reasons why players pre-buffer video content.
[DOT] Agree on the observation and would similarly need to dig out the references but it would be great if we had them.

  1.  Session stickiness is not necessarily a desirable feature, or at least requires a precise definition what constitutes a session. While it is clear that each individual small request should be fulfilled by the node to which the request was made, the overall "streaming session" may best be highly distributed. That alone probably provides for some interesting discussion.
[DOT] Indeed, if we look at a streaming ‘session’ as that of watching a video, an individual chunk retrieval can be the smallest form of a session given its stateless nature (assuming that the name of the chunk suitably points to the relevant chunk, which is the case). So one can easily pull a video from N replicas by sending individual chunk request to any of the N nodes – although the ‘any’ here may be constrained by ‘best’, ‘lowest latency’, ‘least loaded’ or alike.

  1.  As latency matters, so does the latency for resolving the optimal node for each mini request. Whether due to peer-to-peer nodes simply disappearing or an edge node receiving a sudden influx of requests, good and efficient node selection is a challenge.
[DOT] It is this ‘endpoint locator selection’ problem that is key to CAN. Both, an LB or a d-router (using the terminology of the dyncast arch draft) merely select the endpoint, not the path (the path is being defined by the routing policy to each of the possible endpoint locations). Key point though is that the proposed in-path selection is more efficient than using indirection points, which is what Peng pointed out yesterday in his presentation when comparing to, e.g., GSLB.

  1.  The metrics question that was raised in various forms is valid, but obscures the real issue: which metrics apply for node selection is highly dependent on the role of the service providing node. The only way this can be effectively generalized is by making routers unaware of the semantics of the metrics, and leave this in the application space. That is, there is only one metric for routers to consider: a sliding scale of "good" to "bad" values for a given application's specific service type. That also makes the selection algorithms comparatively simple. From this follow quite a few implications for how service providing nodes might contribute to this decision making process by providing the respective metric about themselves.
[DOT] So do you foresee simple numerals being used rather than rich semantic descriptors? I would also argue that apart from the metric itself (semantic-rich or not), the use of the metric matters. This was also discussed in the chat (as Lars pointed out in the LB discussion) since distributing LBs now to every ingress point is all good and nice but if I want consistent behavior for a user moving across those ingress points, the used information and the usage of that information will matter a great deal.

  1.  Given such a simple service-defined metric, it is often possible for the client machine to make the optimal routing decision - this is especially the case if it is multi-homed.

I hope this provides some useful feedback. If you have questions or comments, by all means let me know!
[DOT] Useful, indeed, at least for me – thanks!

I am very happy to see that this group is taking the very specific issues we solved back in the day and tackle the more generalized problem!
[DOT] Thanks, I read this as support for this discussion and work to continue.

Jens

---

From: Jens Finkhaeuser <[email-masked]>
Date: Wed, 23 Mar 2022 12:49:54 +0000

Hi,

simple reply in-line.

------- Original Message -------

> [DOT] It is this ‘endpoint locator selection’ problem that is key to CAN. Both, an LB or a d-router (using the terminology of the dyncast arch draft) merely select the endpoint, not the path (the p…

Indeed!

One of the contributions I made to the Joost stack was to prioritize nearest neighbours based on AS affinity. It's one thing to perform measurements and collect metrics; in some sense this is more of an indicator of past than future suitability (though stable metrics can give an indication of that).

We considered a shortest-path metric, but given the address space, that is expensive in space and time. Instead, I opted for a metric on how many autonomous systems one has to traverse (taking an average of those high level paths). Adding that metric proved to be a good enough estimate of latency in ca. 2007-08 or so that it significantly improved node selection without any prior measurements.

Often it's about "good enough".

> 2.  The metrics question that was raised in various forms is valid, but obscures the real issue: which metrics apply for node selection is highly dependent on the role of the service providing node. The only way this can be effectively generalized is by making routers unaware of the semantics of the metrics, and leave this in the application space. That is, there is only one metric for routers to consider: a sliding scale of "good" to "bad" values for a given application's specific service type. That also makes the selection algorithms comparatively simple. From this follow quite a few implications for how service providing nodes might contribute to this decision making process by providing the respective metric about themselves.
> 

> [DOT] So do you foresee simple numerals being used rather than rich semantic descriptors? I would also argue that apart from the metric itself (semantic-rich or not), the use of the metric matters.…

It's less something I foresee than something I would do, or am doing now. As you rightly point out, distributing LBs is one thing, getting consistent behaviour out of them another.

I suppose it's a space/time tradeoff of sorts. If the metrics are richer than a simple numeric value, then the algorithm processing them needs to be updated, and we run into the consistency behaviour above. If they're rich enough to have semantic meaning, the algorithm can be generalized, but at both a time cost (processing on the router) and a space cost (bandwidth for transmission and memory consumption).

We're exploring very similar concepts in the drone communications world with AnyWi (there are differences, but those are maybe too long for here). We've taken to a construction where packet (request) scheduling decisions are made on a single numeric priority value, while the calculation of said priority is taken outside of the loop, and based on relatively slowly updating richer metrics. Here as well, because our problem domain is relatively specific and includes requirements on low size, weight and power consumption (SWaP), we opted for non-semantic metrics and a less generic algorithm.

But also semantic metrics risk that new semantics are introduced with the algorithm requiring an update.

It's from these points of views that I would consider ideally leaving only the single-metric scheduling decision in the routing component, and moving the node evaluation to the application layer where it can be modified much more easily. Specifically, each service providing node can calculate and advertise an evaluation of its application-specific suitability.

Of course with the AS-aware selection I mentioned above, one has to consider the path, and that decision cannot be made at the node providing the service alone. What the application can do, however, is indicate how much path awareness should weigh in comparison to its own metric.

Maybe that kind of approach yields the best in trade-offs? There is some interesting R&D here.

Jens

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 23 Mar 2022 20:54:41 +0800

Hi Jens,

Thank you for the feedback and comments. There was full discussion in the BoF,  we are summing up it and formulating the promotion plan for the next step. 

Please find the reply inline.

Regards,
Peng

[email-masked]
 
From: Jens Finkhaeuser
Date: 2022-03-23 17:48
To: [email-masked]

I was invited yesterday to attend the BoF, and listening to the talks, I have some general feedback - not specifically to any presenter, but to the problem statement as such.
 [PL] Thanks for joinning and give the feedback.

As video streaming data apparently constitutes the vast majority of internet traffic, it is maybe somewhat counter-intuitive that throughput matters less for end-user experience than latency. There have been studies (which I do not have at my fingertips, but I can dig up of there is interest) that a skipped video frame or three is barely noticeable, but stutter immediately and severely degrades the viewing experience, which is one of the reasons why players pre-buffer video content.
[PL] Yes, as some comments in the BoF, there might be more  use case besides shown in the slides. We have been also working on this, and happy to find the video based one. Moreover, 'Computing' will bring more characteristics, interests and challenges than getting the video itself.

Without going into too much detail here, relaying live video streams poses several interesting challenges which are most easily solved by assigning different roles to different nodes contributing to a stream. In terms more relevant to this group, even though conceptually "streaming a video" is a single large request, it can be best fulfilled by making many small requests to different nodes providing different services.
[PL] Yes! Small request to different nodes is the CAN's aiming use case considering at least 'load status' for computing domain.

Session stickiness is not necessarily a desirable feature, or at least requires a precise definition what constitutes a session. While it is clear that each individual small request should be fulfilled by the node to which the request was made, the overall "streaming session" may best be highly distributed. That alone probably provides for some interesting discussion.
           [PL] Depend on the use case I think. For video, there might not be 'transaction' characteristic to require the session stickiness, while for some 'computing task' there is, and that depends on the what kind of use case               we consider. We may find a suitable way to define this req.
As latency matters, so does the latency for resolving the optimal node for each mini request. Whether due to peer-to-peer nodes simply disappearing or an edge node receiving a sudden influx of requests, good and efficient node selection is a challenge.
         [PL] For those critical latency use case, an in path resolving solution is expected. Of course agree that it is a challenge.
The metrics question that was raised in various forms is valid, but obscures the real issue: which metrics apply for node selection is highly dependent on the role of the service providing node. The only way this can be effectively generalized is by making routers unaware of the semantics of the metrics, and leave this in the application space. That is, there is only one metric for routers to consider: a sliding scale of "good" to "bad" values for a given application's specific service type. That also makes the selection algorithms comparatively simple. From this follow quite a few implications for how service providing nodes might contribute to this decision making process by providing the respective metric about themselves.
          [PL] This part is a interesting issuse that open to discuss,  I reprensented in the BoF there might be two ways, one is single index and another is multi-information since the computing resoure is heterogeneous. 
Given such a simple service-defined metric, it is often possible for the client machine to make the optimal routing decision - this is especially the case if it is multi-homed.

I hope this provides some useful feedback. If you have questions or comments, by all means let me know!
 [PL] Useful feedback and comments!

 [PL] Thanks again~

Jens

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 23 Mar 2022 22:58:34 +0800

Hi, Jens

    I want to explain more about the latency. Hope I have no misunderstanding here.

    In this Dyncast scenario, the Ingress Node can collect the path latency and the status of the service from each potential service node and its corresponding Egress Node.

    BTW, the client of course can make a decision which service node to connect.

    However, if the client has no information about the services or paths, the Ingress Node of the network can do it instead because it can collect the information about the path latency and the status of the service.

Best Regards
Zongpeng Du 

[email-masked] & [email-masked]
 
From: Jens Finkhaeuser
Date: 2022-03-23 20:49
To: Dirk Trossen
CC: [email-masked]
Subject: Re: [Dyncast] Feedback on CAN BoF
Hi,

simple reply in-line.

------- Original Message -------

One of the contributions I made to the Joost stack was to prioritize nearest neighbours based on AS affinity. It's one thing to perform measurements and collect metrics; in some sense this is more of an indicator of past than future suitability (though stable​ metrics can give an indication of that).

Jens