Subject: Re: [OPSAWG] Call for Agenda Items for Buenos Aires

From: Joe Clarke <[email-masked]>
Date: Wed, 6 Apr 2016 14:17:02 -0400

On 3/18/16 09:41, Petr Lapukhov wrote:
> Folks,
>
> If possible I would like to request a slot on the agenda to present
>
> https://datatracker.ietf.org/doc/draft-lapukhov-dataplane-probe/
>
> This to start work on in-band telemetry format standardization. The
> technology is still at early stages, and I believe it would be helpful
> to agree on common formats to be used across multiple silicon vendors.
>
> Looking for feedback and finding a best place to continue this work at.

Petr, I was going to ask these in the meeting, but given the number of 
questions, I thought it better to ask on the list.

* How do you propose the populated data to be handled normally?  Will 
the probe destination (i.e., the dest address of the UDP packet) be 
expected to understand these, or will loopback mode have to be used? 
The reason I ask is that I'm not sure what happens if all nodes in the 
network do not support this probe.  Could this give more network insight 
to the destination, and thus be a security weakness?

* Related to the question above what would happen if a set a hop count 
of four, but the fourth device (before my destination) has not awareness 
of this telemetry system?

* Why not use ifIndex values for port IDs?  It seems to me that having a 
more standard and known cross-vendor way to identify ports is useful.

* You do not list IANA considerations, but it seems to me that your 
probe types would be well-suited for a registry that perhaps could live 
with IANA?  It would also be useful to have some reserved space for 
private extensions where possible.

* I think you mentioned it at the mic, but the draft doesn't really 
address what happens if a device supports this telemetry system, but 
does not support a specific probe type?  For example, what if I define 
type 50 down the road, but not all of my devices/hardware yet support 
this?  What happens then?

Joe

---

From: Petr Lapukhov <[email-masked]>
Date: Wed, 6 Apr 2016 19:08:12 +0000

Thank for the questions! 

> to the destination, and thus be a security weakness?

One of the main reasons to use regular data packet and not any special OAM format or IPv6 extension header is to let the packet pass through non-compliant devices transparently. The device that understand the request will fill in their data, but others will remain invisible (unless hop-by-hop variant is requests using router-alert option - but I doubt this would be viable given the state of hardware in the DC at the moment).

For security concerns: this is a rather big topic in context of data-plane probe, and I haven't completed the section yet. Easy way out is saying "we control the network in DC and we are aware of security consequences of disclosing the device state". The header marker might help here - a random key value would need to be guessed by possible attacker. In addition, only select applications could be allowed to send the probes to the "special" port. 

> of this telemetry system?

The responsibility for setting the right hop limit is on the sender. It needs to be aware there are devices that do not understand the form of signaling, and adjust hop-limit respectively. All non-compliant hops will be transparent. Worst case, if sender was clueless, the packet will just fall through to the other side just like a regular active probe.

> more standard and known cross-vendor way to identify ports is useful.

This is a good idea: I heard this a few times already and I will change the spec accordingly :) It is hard to number the physical ser/des (e.g. BRCM has a way to do this in SDK, but others would likely be different). The tradeoff is that the ifIndex will need to be stored in some state table in hardware.

> private extensions where possible.

I wasn't sure what kind of code-point could be requested. I think this is a very good point for "Type" registration, to ensure better interop.

> type 50 down the road, but not all of my devices/hardware yet support

Right now the idea is that device will simply set the bit signaling that it does not understand the requested type, and leave the value in TLV empty (zero). The length is known to the device even if it does not understand the type.

---

From: Joe Clarke <[email-masked]>
Date: Sun, 10 Apr 2016 11:18:19 -0400

> For security concerns: this is a rather big topic in context of data-plane probe, and I haven't completed the section yet. Easy way out is saying "we control the network in DC and we are aware of s…

Agreed this will need some fleshing out.  Coming from a technical 
support background, I see the value in obtaining this information in all 
networks, even those without a single source of control.

A thought on this might be that there is a flag that signals local 
storage on the device only.  This way the data is recorded, but not in 
the packet itself.  Instead, local buffers hold this where it can be 
queried later.

> The responsibility for setting the right hop limit is on the sender. It needs to be aware there are devices that do not understand the form of signaling, and adjust hop-limit respectively. All non-…

Understood.  But how does the sender know/discover this?  The network 
can change dynamically, and sending the packet without a good 
understanding of the intermediate nodes' support of the probe could 
result in the destination receiving this telemetry-rich packet when they 
shouldn't be.

> This is a good idea: I heard this a few times already and I will change the spec accordingly :) It is hard to number the physical ser/des (e.g. BRCM has a way to do this in SDK, but others would li…

Whatever value is chosen, it needs to be well-understood and easily 
queryable by the administrator.  Normative language around ifIndex would 
be very clear, but I agree this may not be trivial to obtain in the 
ASIC.  What I want to avoid is having to know vendor-by-vendor, 
hop-by-hop what this value represents and how to obtain a mapping to 
physcal/virtual port.

> I wasn't sure what kind of code-point could be requested. I think this is a very good point for "Type" registration, to ensure better interop.

Maybe the chairs or others in the WG can suggest something.  I would 
personally like a registry with space for reserved probe IDs exactly for 
interoperability and extension.

> Right now the idea is that device will simply set the bit signaling that it does not understand the requested type, and leave the value in TLV empty (zero). The length is known to the device even i…

Ah, right.  The length is known because the sender has filled in all the 
possible fields.

Joe