Subject: Re: [core] FW: New Version Notification for draft-fz-core-coap-pm-00.txt

From: Christian Amsüss <[email-masked]>
Date: Sun, 7 Nov 2021 17:46:21 +0100

Hello Giuseppe,

On Fri, Oct 22, 2021 at 04:17:02PM +0000, Giuseppe Fioccola wrote:
> We have just uploaded a new draft on CoAP Performance Measurement
> Option, which we hope to discuss at IETF 112.  This document aims to
> specify a method for the Performance Measurement of CoAP.

skimming through your slides and the document I'm left wondering what
the obtained details would be used for. Estimates for RTT and losses are
produced, for example, also in the FASOR draft[1] -- but these get
applied to retransmission parameters, which are per-hop (ie. up to the
next proxy), where good estimates are direly needed.

What would the derived values be used for?

What are the intermediaries this is intended to work with? CoAP proxies
hide the identity of the client, and moreover often apply caching (which
will at any rate need a bit of consideration if this is to be applied
through proxies). Thus, on the server side the data in the bits would
likely appear completely mixed in presence of more than one client, and
clients would receive mixed signals in presence of cache entries.

So, for both aspects, it'd be helpful if you could sketch out the
envisioned use cases.

Best regards
Christian

[1]: https://datatracker.ietf.org/doc/html/draft-ietf-core-fasor-01

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Giuseppe Fioccola <[email-masked]>
Date: Mon, 8 Nov 2021 08:50:49 +0000

Hi Christian,
Thank you for your comments.
Please find my answers inline as [GF].

Regards,

Giuseppe

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Sunday, November 7, 2021 5:46 PM
To: Giuseppe Fioccola <[email-masked]>
Cc: [email-masked]; Mauro Cociglio <[email-masked]>; Tianran Zhou <[email-masked]>; Massimo Nilo <[email-masked]>; Fabio Bulgarella <[email-masked]>
Subject: Re: [core] FW: New Version Notification for draft-fz-core-coap-pm-00.txt

Hello Giuseppe,

> specify a method for the Performance Measurement of CoAP.

skimming through your slides and the document I'm left wondering what the obtained details would be used for. Estimates for RTT and losses are produced, for example, also in the FASOR draft[1] -- but these get applied to retransmission parameters, which are per-hop (ie. up to the next proxy), where good estimates are direly needed.

[GF]: The derived values (RTT, losses) can be useful for an operator or an enterprise that is managing a constrained, low-power and lossy network. As IoT and M2M keep growing, a mechanism to measure the performance can become useful to meet the operational requirements. Also, it should be a simple mechanism to be developed on constrained nodes. 

What are the intermediaries this is intended to work with? CoAP proxies hide the identity of the client, and moreover often apply caching (which will at any rate need a bit of consideration if this is to be applied through proxies). Thus, on the server side the data in the bits would likely appear completely mixed in presence of more than one client, and clients would receive mixed signals in presence of cache entries.

[GF]: The intermediaries or on-path observers could also be network gateway or probes, that can see deep into application and read the measurements. This information can be used to monitor the network in order to check the operational performance and to employ further network optimization. The measurements can be end-to-end between Client and Server or can be split. If CoAP Proxies are used, the measurements can be done between the Proxies or between the Proxy and the Client. The Server can distinguish the source client, for example by using additional flow information such as the IP addresses. It could also possible to bundle different clients if they are mixed. I think we have to clarify the different scenarios in the next version of the draft.

So, for both aspects, it'd be helpful if you could sketch out the envisioned use cases.

Best regards
Christian

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 9 Nov 2021 18:11:33 +0100

Hello Giuseppe,

On Mon, Nov 08, 2021 at 08:50:49AM +0000, Giuseppe Fioccola wrote:
> [GF]: The derived values (RTT, losses) can be useful for an operator
> or an enterprise that is managing a constrained, low-power and lossy
> network. As IoT and M2M keep growing, a mechanism to measure the
> performance can become useful to meet the operational requirements.
> Also, it should be a simple mechanism to be developed on constrained
> nodes. 

OK, so this sounds more like a "characterization" (to demonstrate
meeting some requirements?) than "feeding these values into the protocol
again", is that correct? Or is concrete action to be taken based on
these? (If so, I'd imagine it would need to be more fine-grained than
across all involved proxies).

> [GF]: The intermediaries or on-path observers could also be network
> gateway or probes, that can see deep into application and read the
> measurements. This information can be used to monitor the network in
> order to check the operational performance and to employ further
> network optimization. The measurements can be end-to-end between
> Client and Server or can be split. If CoAP Proxies are used, the
> measurements can be done between the Proxies or between the Proxy and
> the Client. The Server can distinguish the source client, for example
> by using additional flow information such as the IP addresses. It
> could also possible to bundle different clients if they are mixed. I
> think we have to clarify the different scenarios in the next version
> of the draft.

Let me illustrate what I mean here by describing a use case like those
I'd expect to see it around this document (not necessarily in a document
update -- OK if that helps, but for initial steps this thread may
suffice):

1 -- 2 -- 3 -- 4 -(Internet)- 5

1. Devices: Sensors with 6LoWPAN connectivity, operated by A.
2. Gateway: Proxy (eg. running on a cellular-to-6lowpan router) shipped
   with the sensors and operated by A.
3. Uplink: provided by B.
4. Probe: passive monitoring device operated by B.
5. Backend: Data center operated by A.

If this protocol is run between 1 and 5 (across intermediaries), the
values read at 4 would be total round-trip times over the 6LoPAN, the
cellular network and the remaining Internet. I wouldn't know what to do
with those values; given that 2 is acting as a proxy, they are not the
relevant parameters to tune their CoAP stack, and wouldn't tell anyone
where to add capacity.

(Moreover, data at 4 would look like garbage -- different clients at 1
would send uncoordinated PM values, and at 4 the clients are not
distinguishable any more by design).

If, to set up a similar but different scenario, 4 was also a CoAP proxy,
the PM option were hop-by-hop, and the protocol would be performed
between 2 and 4, then this would result in a characterization of the
cellular uplink, and (in my vastly oversimplifying imagination) might be
used to place additional cell towers there.

The operator situation here does raise an additional concern: If 2 is
operated by A and 4 by B, wouldn't this be prone to A forcing B to take
action (for example) by maliciously flipping the square bit every 62nd
message, giving B the impression that there is message loss? I'm not
asking for security considerations here at this point (I would later,
and privacy on top of that), more about the assumptions underlying this
work.

BR
Christian

---

From: Thomas Fossati <[email-masked]>
Date: Tue, 9 Nov 2021 17:56:10 +0000

Hi Christian, Giuseppe, Mauro, Massimo, Fabio,

> and privacy on top of that), more about the assumptions underlying this

Yep, it looks like the ideal scenario is one where the new CoAP
options are clear-text and integrity protected end-to-end by OSCORE.
Then A and/or B can put their measurement boxes (or functions) in one
or more places to break down the different RTT contributions where it
makes sense (e.g., at the ingress/egress of their respective network
segments).  If we can't assume integrity protection we'd need to take
cheating inventives, trust relationships and all those things into
consideration, which tend to make the analysis much more complicated.

cheers!

> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

-- 
Thomas

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 9 Nov 2021 19:20:27 +0100

> consideration, which tend to make the analysis much more complicated.

Just because it's integrity protected E2E doesn't mean that the ends are
not lying to the observer. The integrity protection would only tell the
ends if someone tampered, and if they are the ones interested in it,
they could encrypt it right away (if it helps *them* assess the link
properties, of which I'm not sure).

How would any of this tell the observer about the contribution of
different hops to the total loss?

BR
c

---

From: Thomas Fossati <[email-masked]>
Date: Tue, 9 Nov 2021 19:17:27 +0000

On Tue, Nov 9, 2021 at 6:20 PM Christian Amsüss <[email-masked]> wrote:
> not lying to the observer.

I am much more worried by the signal being tampered with by a random
on-path attacker rather then the endpoints trolling the observer.
What is their incentive to lie?  If they don't want to (or cannot)
participate they just do nothing.  It's zero cost for them being
removed from the measured set.

> The integrity protection would only tell the

I see the proposed mechanism -- correct me if I'm wrong -- as a way
for network diagnostic tools to do their large-scale measurements
requiring (just a minimal amount of) collaboration from the endpoints.
The information they extract from the options' signals is not
something the endpoints don't already know (possibly with higher
quality).

cheers,

-- 
Thomas

---

From: Giuseppe Fioccola <[email-masked]>
Date: Wed, 10 Nov 2021 17:04:24 +0000

Hi Christian

Regards,

Giuseppe

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Tuesday, November 9, 2021 6:12 PM
To: Giuseppe Fioccola <[email-masked]>
Cc: Mauro Cociglio <[email-masked]>; Massimo Nilo <[email-masked]>; [email-masked]; Fabio Bulgarella <[email-masked]>
Subject: Re: [core] FW: New Version Notification for draft-fz-core-coap-pm-00.txt

Hello Giuseppe,

> Also, it should be a simple mechanism to be developed on constrained 

OK, so this sounds more like a "characterization" (to demonstrate meeting some requirements?) than "feeding these values into the protocol again", is that correct? Or is concrete action to be taken based on these? (If so, I'd imagine it would need to be more fine-grained than across all involved proxies).

[GF]: Yes it can be both. It can surely be used for a characterization but it can also imply concrete actions to be taken based on these measurements.

> think we have to clarify the different scenarios in the next version 

Let me illustrate what I mean here by describing a use case like those I'd expect to see it around this document (not necessarily in a document update -- OK if that helps, but for initial steps this thread may
suffice):

If this protocol is run between 1 and 5 (across intermediaries), the values read at 4 would be total round-trip times over the 6LoPAN, the cellular network and the remaining Internet. I wouldn't know what to do with those values; given that 2 is acting as a proxy, they are not the relevant parameters to tune their CoAP stack, and wouldn't tell anyone where to add capacity.

[GF]: It is correct. Spin bit allows RTT measurement for all the intermediate points but with sQuare Bit and by applying the methodologies in RFC8321, it is also possible to do hop-by-hop measurements for loss and delay. I can include these considerations in the draft. 

(Moreover, data at 4 would look like garbage -- different clients at 1 would send uncoordinated PM values, and at 4 the clients are not distinguishable any more by design).

If, to set up a similar but different scenario, 4 was also a CoAP proxy, the PM option were hop-by-hop, and the protocol would be performed between 2 and 4, then this would result in a characterization of the cellular uplink, and (in my vastly oversimplifying imagination) might be used to place additional cell towers there.

[GF]: This could be possible by applying RFC8321 on the sQuare Bit signal.

The operator situation here does raise an additional concern: If 2 is operated by A and 4 by B, wouldn't this be prone to A forcing B to take action (for example) by maliciously flipping the square bit every 62nd message, giving B the impression that there is message loss? I'm not asking for security considerations here at this point (I would later, and privacy on top of that), more about the assumptions underlying this work.

[GF]: Agree, security considerations need to be taken into account. For similar mechanisms we are considering the precondition of the controlled domain application, but security mechanisms, e.g. OSCORE can also be used.

BR
Christian

---

From: Giuseppe Fioccola <[email-masked]>
Date: Wed, 10 Nov 2021 17:23:55 +0000

Hi Thomas,
Thank you for the inputs,
Please find my replies inline as [GF].

Regards,

Giuseppe

-----Original Message-----
From: Thomas Fossati <[email-masked]> 
Sent: Tuesday, November 9, 2021 8:17 PM
To: Christian Amsüss <[email-masked]>
Cc: Giuseppe Fioccola <[email-masked]>; Mauro Cociglio <[email-masked]>; Massimo Nilo <[email-masked]>; [email-masked]; Fabio Bulgarella <[email-masked]>
Subject: Re: [core] FW: New Version Notification for draft-fz-core-coap-pm-00.txt

On Tue, Nov 9, 2021 at 6:20 PM Christian Amsüss <[email-masked]> wrote:
> > Then A and/or B can put their measurement boxes (or functions) in 
> > one or more places to break down the different RTT contributions 
> > where it makes sense (e.g., at the ingress/egress of their 
> > respective network segments).  If we can't assume integrity 
> > protection we'd need to take cheating inventives, trust 
> > relationships and all those things into consideration, which tend to make the analysis much more complicated.
>
> Just because it's integrity protected E2E doesn't mean that the ends 
> are not lying to the observer.

I am much more worried by the signal being tampered with by a random on-path attacker rather then the endpoints trolling the observer.
What is their incentive to lie?  If they don't want to (or cannot) participate they just do nothing.  It's zero cost for them being removed from the measured set.

[GF]: I agree with you, we can assume that for the typical COAP applications it is less likely that the endpoints are attackers.

> The integrity protection would only tell the ends if someone tampered, 
> and if they are the ones interested in it, they could encrypt it right 
> away (if it helps *them* assess the link properties, of which I'm not 
> sure).

I see the proposed mechanism -- correct me if I'm wrong -- as a way for network diagnostic tools to do their large-scale measurements requiring (just a minimal amount of) collaboration from the endpoints.
The information they extract from the options' signals is not something the endpoints don't already know (possibly with higher quality).

[GF]: Yes it is right.

cheers,

--
Thomas

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 10 Nov 2021 18:29:05 +0100

Hello Giuseppe,

still sounds all very vague to me, but for that I'd guess I'll have to
wait for an update with concrete usage examples.

As proxy traversal has not falled out in the course of this discussion,
let me come back to the question I had hoped wouldn't even need to be
considered (if this would have all become hop-by-hop already):

On the link from a proxy to servers, how would any of that information
be useful? On that link, traffic from different clients would be mixed,
so (eg. in the case of the square bit) there isn't a square wave left,
but just square bits sampling into whichever client that request happens
to come from.

Conversely, on the link from the client to the proxy, communication may
happen with different servers, so just looking at the bit pattern
gernerated by a pair of addresses (client and proxy) would be
insufficient just as well. (Unlike on the other link, here there *is*
the information available, unless the client has an OSCORE context with
the server, in which case it would likely not expose the PM options as
they'd leak whether one or different servers are being used).

So where are probes supposed to find useful information? On the
proxy-server link I wouldn't understand how, and on the client-proxy
link they'd have to parse the remainder of the message to understand
which server this is aimed for. Is that the intention?

BR
c

---

From: Giuseppe Fioccola <[email-masked]>
Date: Wed, 10 Nov 2021 18:01:29 +0000

Hi Christian,
My answers inline as [GF],

Regards,

Giuseppe

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Wednesday, November 10, 2021 6:29 PM
To: Giuseppe Fioccola <[email-masked]>
Cc: Mauro Cociglio <[email-masked]>; Massimo Nilo <[email-masked]>; [email-masked]; Fabio Bulgarella <[email-masked]>
Subject: Re: [core] FW: New Version Notification for draft-fz-core-coap-pm-00.txt

Hello Giuseppe,

still sounds all very vague to me, but for that I'd guess I'll have to wait for an update with concrete usage examples.

As proxy traversal has not falled out in the course of this discussion, let me come back to the question I had hoped wouldn't even need to be considered (if this would have all become hop-by-hop already):

On the link from a proxy to servers, how would any of that information be useful? On that link, traffic from different clients would be mixed, so (eg. in the case of the square bit) there isn't a square wave left, but just square bits sampling into whichever client that request happens to come from.

[GF]: In this case, the proxy can still use the Option to set Spin Bit and sQuare Bit for the bundle of clients. The measurement can be done but it is an information related to a bundle of clients. An alternative can be to use the Option only for a single client at once in order to avoid to do a grouped measurement.

Conversely, on the link from the client to the proxy, communication may happen with different servers, so just looking at the bit pattern gernerated by a pair of addresses (client and proxy) would be insufficient just as well. (Unlike on the other link, here there *is* the information available, unless the client has an OSCORE context with the server, in which case it would likely not expose the PM options as they'd leak whether one or different servers are being used).

[GF]: It is necessary to check the other fields to understand the server.

So where are probes supposed to find useful information? On the proxy-server link I wouldn't understand how, and on the client-proxy link they'd have to parse the remainder of the message to understand which server this is aimed for. Is that the intention?

[GF]: As said, it is possible to manage this use case. It is also worth emphasizing that the first usage of this Option is to do end-to-end measurement between the client and the server. The on-path measurement is the additional feature. But, based on our discussion, it needs to be further explained in the draft in order to consider the different e2e scenarios, the type of proxy implemented and so on.

BR
c