Subject: [Dots] draft-ietf-dots-telemetry: Large responses

From: "Jon Shallow" <[email-masked]>
Date: Mon, 6 Apr 2020 10:35:23 +0100

Hi All,

With support for telemetry status updates, there is a good chance that a
status response will not fit into a single IP packet.

Do we need a new CoAP Option or response code?

These status responses are sent back using CoAP NON-confirmable packets - in
other words the DOTS client does not have to acknowledge receipt of the
packet which works in an environment where there is heavy packet loss in the
same direction as the status packet - the DOTS server is not waiting on
acknowledgement of the previous status packet before sending out the next
one (which too may get dropped).

For the status data that is larger than an IP packet, this can currently be
handled in 2 ways that I can think of
1- Let IP fragmentation be used by sending large data that is fragmented
across several IP packets
2- Make use of the CoAP BLOCK2 option (RFC9759) which is already referred to
in the signal draft to create CoAP fragmentation.

The disadvantage of (1) is that the receiving CoAP library has to have a
large enough receive data buffer size to handle both the encrypted and then
plaintext data.  The libcoap library I use has the default receive buffer
sized for that of an IP packet that is not fragmented.  This buffer can be
made bigger, but how big?
What if the DOTS client is a constrained device with RAM etc. limitations?
There is also a challenge of managing missing IP fragmented packets which is
primarily done by the network stack.

The disadvantage of (2) is that BLOCK2 is a synchronous transmission where
the DOTS client receives a block of data and then has to request the next
block of data.  With a lossy network, if the client does not see a block of
data then this status data transmission will stall and not recover leaving
the DOTS server with outstanding data that cannot be transmitted and hence
the need for garbage collection of failed transmissions.
Furthermore with (2), RFC9759 recommends use of CONfirmable responses to
handle potential packet loss - which does not work with a flooded pipe DDoS
situation.

I agree that if all can fit into one packet then this is not an issue, but
enforcing that is difficult unless there are data reduction policies in
place (such as using Queries or the remainder of the data is not sent
(parameter/values deleted) and a response (may need new code) that says
status is too large to fit into a packet or has been truncated).

Another possibility is to create a new CoAP option that is equivalent to
BLOCK2 in format but does not need to work symmetrically.  Here, the DOTS
server sequentially sends all of the CoAP fragmented data without requiring
the DOTS client to request the next block (just as fragmented IP packets
would be sent).  It would then be up to the DOTS client to do the CoAP plain
text block fragmentation re-assembly (again a limitation requiring more RAM
on the DOTS client and garbage collection if there are missing packets).
Using this new option reduces the load on the DOTS server who could be
serving many DOTS clients.

Thoughts?

Regards

Jon

---

From: <[email-masked]>
Date: Mon, 6 Apr 2020 12:56:37 +0000

Jon,

If we had https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options-08#section-5.7, this would solve the issue. Nevertheless, this is not implemented/deployed. 
Your proposed option mimics the UDP FRAG behavior. 

Because re-assembly would fail if a fragment is missing, notification fragments with the new options should be sent, e.g., three times. 

Ideally, we need a solution that allows to make use of partial responses even if other fragments are lost. 

Before considering a new option, let's see if we can solve this at the DOTS layer. For example, we do have the following to avoid fragmentation: 

   If the total request size exceeds
   the path MTU then the DOTS client MUST split the DOTS signal into
   separate messages; for example, the list of addresses in the 'target-
   prefix' parameter could be split into multiple lists and each list
   conveyed in a new PUT request. 

Does your implementation allow to follow a similar approach when sending notifications? For example, a first notification can include attack-traffic metrics and others include attack-details? 

Because all these notifications are bound to the same tmid, the DOTS client will update its local records with the data received in multiple notifications. 

Cheers,
Med

> -----Message d'origine-----
> De : Dots [mailto:[email-masked]] De la part de Jon Shallow
> Envoyé : lundi 6 avril 2020 11:35
> À : [email-masked]
> Objet : [Dots] draft-ietf-dots-telemetry: Large responses
> 
> Hi All,
> 
> _______________________________________________
> Dots mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dots

---

From: "Jon Shallow" <[email-masked]>
Date: Mon, 6 Apr 2020 15:39:45 +0100

Hi Med,

See inline Jon>

Regards

Jon

> -----Original Message-----
> From: Dots [mailto: [email-masked]] On Behalf Of
[email-masked]
> Sent: 06 April 2020 13:57
> To: Jon Shallow; [email-masked]
> Subject: Re: [Dots] draft-ietf-dots-telemetry: Large responses
> 
> Jon,
> 
> If we had
https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options-08#section-
> 5.7, this would solve the issue. Nevertheless, this is not
implemented/deployed.
> Because re-assembly would fail if a fragment is missing, notification
fragments
> with the new options should be sent, e.g., three times.

Jon> This brings to mind the TCP SACK RFC2018 model which could be
implemented at the DOTS layer, CBOR or CoAP layer (with new CoAP option).
See below.
> 
> Ideally, we need a solution that allows to make use of partial responses
even if
> other fragments are lost.

Jon> This would be nice.
> 
> Before considering a new option, let's see if we can solve this at the
DOTS layer.
> For example, we do have the following to avoid fragmentation:
> 
> Does your implementation allow to follow a similar approach when sending
> notifications? For example, a first notification can include
attack-traffic metrics
> and others include attack-details?

Jon> Hmm, a GET only expects one (potentially re-assembled) response and
then unsolicited observe responses with the same Token if observing.
Additional responses could be sent as "Pseudo Observe" responses with the
same Token, but that assumes that the client is expecting Observe responses
- I think that would work in libcoap and the DOTS layer then taught what to
do.

Jon> There still is a challenge of deciding what to send in each individual
pseudo response such that it is not larger than a packet (i.e. how best to
break down the status tree into component parts that are of the correct size
with CBOR varying length parameter values).  There needs to be some
knowledge of the DTLS + cipher overheads (and padding) to determine what can
fit in the IP MTU.  With DTLS, we cannot use DTLS fragmentation as that is
for Handshake records only.

Jon> I have only got so far as logging responses in the client, but doing
nothing with the data, so this is early days for me.
> 
> Because all these notifications are bound to the same tmid, the DOTS
client will
> update its local records with the data received in multiple notifications.

Jon> I think that this may work, but still am worried about getting data
subsets into a packet and erring on the smaller side giving rise to more
traffic (IP+UDP+DTLS overheads) than necessary which could compound the
lossy issue.

Jon> We also could handle the fragmentation at the CBOR level - a new
parameter of "block" with block-no at the start of a new chunk record, along
with a "max-block" parameter (or perhaps "block" block-no is made up of
maxblock and current block)  and have a GET Query of block=X-Y,Z (missing
range and/or individual blocks, comma separated) to get the missing ones.
~Jon