Subject: Re: [Dots] FW: I-D Action: draft-teague-dots-protocol-00.txt

From: Gilbert Clark <[email-masked]>
Date: Tue, 15 Nov 2016 20:51:04 -0500

Just now getting around to a quick read through this.  A few thoughts:

* I found the reference to protobufs in the document to be rather strange.
I'm not sold on the idea of a tight coupling between a document and a
specific serialization library, no matter how convenient / awesome the
library might be.  I guess protobuf3 might reference a specific
implementation of protobuf, so it would *kind of* count as a protocol
assuming a specific definition existed someplace which could be referenced,
but still ... why use that instead of something more common that might be
easier for others to read and understand offhand?

* On a related note: I'll point out that I really do miss the wire
definition format used in older specifications such as IP, TCP, UDP, and
friends ...

* The port is 4646, which is "(the hex value for the ASCII character "."
twice)"  Why is this relevant?

* 4.3.3 - "The DOTS client may send repeated requests until it receives a
suitable response from the DOTS server by which it may interpret successful
receipt."  Rate control is needed here.

* I'd argue for an explanation of how often the client needs to perform
specific actions, but also how frequently the server should expect to
receive those notifications from the client.  Since the environment can't
be assumed to be terribly stable, I believe that it may be wise to assume
that links are unstable and therefore explicitly enumerate how the most
common failure modes should be addressed.

* I know the signaling protocol is intended to be relatively low overhead,
but an explicit reference to congestion control in the general case might
be useful.

* It's unusual to me to see endpoints in a protocol specification.
Endpoints seem more of an application-level thing than a protocol-level
thing.  I mean, (I think that) I get that endpoints are tightly coupled
with specific operations (and therefore possibly with different formats),
but ... the specific operations here don't really change the *protocol* or
the *encoding*, do they?  Yes, the operations should be enumerated and
defined, but I just don't know that I buy that this document is the right
place to do it.

* Really, the data channel has such an unbelievably tight binding to HTTP
(or REST in general) that I think this document could be a lot shorter if
the data section was simply a note that "The DOTS data channel uses HTTP
(or a derivative thereof)", then add a reference to an external document
with a supported list of endpoints that are listed as MUST, SHOULD, MAY,
etc ...

Cheers,
Gilbert Clark

On Mon, Oct 31, 2016 at 5:19 PM, Teague, Nik <[email-masked]> wrote:

> Hi,
>
> Please see the below referenced protocol draft.
>
> We welcome feedback as always and hope that this draft may lead to some
> stimulating discussion both on-list and in Seoul.
>
> Thanks,
>
> -Nik
>
> On 31/10/2016, 21:15, "I-D-Announce on behalf of [email-masked]"
> <[email-masked] on behalf of [email-masked]>
> wrote:
>
>
>     A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>
>
>             Title           : DDoS Open Threat Signaling Protocol
>             Authors         : Nik Teague
>                               Andrew Mortensen
>         Filename        : draft-teague-dots-protocol-00.txt
>         Pages           : 39
>         Date            : 2016-10-31
>
>     Abstract:
>        This document describes Distributed-Denial-of-Service (DDoS) Open
>        Threat Signaling (DOTS), a signaling protocol for requesting and
>        managing mitigation of DDoS attacks.
>
>        DOTS mitigation requests over the signal channel permit domains to
>        signal the need for help fending off DDoS attacks, setting the scope
>        and duration of the requested mitigation.  Elements called DOTS
>        servers field the signals for help, and enable defensive
>        countermeasures to defend against the attack reported by the
> clients,
>        reporting the status of the delegated defense to the requesting
>        clients.  DOTS clients additionally may use the data channel to
>        manage filters and black- and white-lists to restrict or allow
>        traffic to the clients' domains arbitrarily.
>
>        The DOTS signal channel may operate over UDP [RFC0768] and if
>        necessary TCP [RFC0793].  This revision discusses a transport-
>        agnostic approach to this channel, focusing on the message exchanges
>        and delegating transport specifics to other documents.  The DOTS
> data
>        channel operates over HTTPS or a transport with similar reliability,
>        interaction and security characteristics.
>
>
>     The IETF datatracker status page for this draft is:
>     https://datatracker.ietf.org/doc/draft-teague-dots-protocol/
>
>     There's also a htmlized version available at:
>     https://tools.ietf.org/html/draft-teague-dots-protocol-00
>
>
>     Please note that it may take a couple of minutes from the time of
> submission
>     until the htmlized version and diff are available at tools.ietf.org.
>
>     Internet-Drafts are also available by anonymous FTP at:
>     ftp://ftp.ietf.org/internet-drafts/
>
>     _______________________________________________
>     I-D-Announce mailing list
>     [email-masked]
>     https://www.ietf.org/mailman/listinfo/i-d-announce
>     Internet-Draft directories: http://www.ietf.org/shadow.html
>     or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
>
> _______________________________________________
> Dots mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dots
>

---

From: "Mortensen, Andrew" <[email-masked]>
Date: Wed, 16 Nov 2016 05:54:31 +0000

Thanks, Gilbert. I appreciate the comments. See inline below.

On Nov 15, 2016, at 8:51 PM, Gilbert Clark <[email-masked]<mailto:[email-masked]>> wrote:

* I found the reference to protobufs in the document to be rather strange.  I'm not sold on the idea of a tight coupling between a document and a specific serialization library, no matter how convenient / awesome the library might be.  I guess protobuf3 might reference a specific implementation of protobuf, so it would *kind of* count as a protocol assuming a specific definition existed someplace which could be referenced, but still ... why use that instead of something more common that might be easier for others to read and understand offhand?

I think it’s entirely reasonable to be uneasy about tying a standard to a specific library. The good news is that protobufs isn’t a library, it’s an IDL. We felt protobufs makes a good choice as the IDL for DOTS due its compact wire representation, its support for backward-compatible schema versioning, its language neutrality, and its extensibility. For what it’s worth, Google’s recent effort to standardize gRPC within the IETF mentions protobuf as the IDL, though so far it hasn’t been made a requirement (see <https://tools.ietf.org/html/draft-talwar-rtgwg-grpc-use-cases-00#section-2.2.5>).

Can you describe what you’re thinking of as “more common” and “easier to read” here? We specifically avoided JSON and the like because of the wasted bytes in the serialization.

* The port is 4646, which is "(the hex value for the ASCII character "." twice)"  Why is this relevant?

It’s a weak joke (made weaker by the fact that its the *decimal* and not hex value).

Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import struct
>>> struct.pack("BB", 46, 46)
'..’  <---- DOTS

/rimshot

That does remind me the draft needs an IANA Considerations section for port registration.

* 4.3.3 - "The DOTS client may send repeated requests until it receives a suitable response from the DOTS server by which it may interpret successful receipt."  Rate control is needed here.

Yes, this needs clarification. We’ll update it for the -01 draft.

* I'd argue for an explanation of how often the client needs to perform specific actions, but also how frequently the server should expect to receive those notifications from the client.  Since the environment can't be assumed to be terribly stable, I believe that it may be wise to assume that links are unstable and therefore explicitly enumerate how the most common failure modes should be addressed.

* I know the signaling protocol is intended to be relatively low overhead, but an explicit reference to congestion control in the general case might be useful.

Yes, will clarify.

* It's unusual to me to see endpoints in a protocol specification…. snip ...

* Really, the data channel has such an unbelievably tight binding to HTTP (or REST in general) that I think this document could be a lot shorter if the data section was simply a note that "The DOTS data channel uses HTTP (or a derivative thereof)", then add a reference to an external document with a supported list of endpoints that are listed as MUST, SHOULD, MAY, etc …

This seems like a very reasonable suggestion to me, thanks. I’m hoping further discussion about the use of RESTCONF with respect to DOTS will help set the direction this goes.

andrew

On Mon, Oct 31, 2016 at 5:19 PM, Teague, Nik <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

We welcome feedback as always and hope that this draft may lead to some stimulating discussion both on-list and in Seoul.

Thanks,

-Nik

On 31/10/2016, 21:15, "I-D-Announce on behalf of [email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

    A New Internet-Draft is available from the on-line Internet-Drafts directories.

    Abstract:

       countermeasures to defend against the attack reported by the clients,

       and delegating transport specifics to other documents.  The DOTS data

    Please note that it may take a couple of minutes from the time of submission
    until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org/>.

    [email-masked]<mailto:[email-masked]>

_______________________________________________
Dots mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dots

_______________________________________________
Dots mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dots

---

From: Gilbert Clark <[email-masked]>
Date: Wed, 16 Nov 2016 11:18:15 -0500

Inline:

On Wed, Nov 16, 2016 at 12:54 AM, Mortensen, Andrew <[email-masked]>
wrote:

> On Nov 15, 2016, at 8:51 PM, Gilbert Clark <[email-masked]> wrote:
>
> * I found the reference to protobufs in the document to be rather
> strange.  I'm not sold on the idea of a tight coupling between a document
> and a specific serialization library, no matter how convenient / awesome
> the library might be.  I guess protobuf3 might reference a specific
> I think it’s entirely reasonable to be uneasy about tying a standard to a
> specific library. The good news is that protobufs isn’t a library, it’s an
> IDL. We felt protobufs makes a good choice as the IDL for DOTS due its
> compact wire representation, its support for backward-compatible schema
> versioning, its language neutrality, and its extensibility. For what it’s
> worth, Google’s recent effort to standardize gRPC within the IETF mentions
> protobuf as the IDL, though so far it hasn’t been made a requirement (see <
> https://tools.ietf.org/html/draft-talwar-rtgwg-grpc-use-cas
> es-00#section-2.2.5>).
>
> Can you describe what you’re thinking of as “more common” and “easier to
> read” here? We specifically avoided JSON and the like because of the wasted
> bytes in the serialization.
>
>
My specific concerns with regard to protobuf (in this particular instance)
would be:

* The signaling protocol should be pretty simple - how much more complex
would it be to describe a custom encoding on the wire without the use of
any kind of general-purpose abstraction (read: CBOR / protobuf / JSON /
what have you)?  If that's too hard, I think an important question here
would be "why is this too hard"?

* On a related note, protobuf invites complexity by offering a convenient
abstraction to shield one from having to think about the way the bytes are
actually laid out on the wire.  This can be a win from a rapid development
perspective, but can be a loss in that it introduces a temptation to add
more stuff to the protocol just because it's so easy to do it ...

* The use of protobuf requires an implementer to learn protobuf well enough
that they're able to use it.  Yes, protobuf is an IDL, but one would need
to learn an implementation of protobuf well enough to be able to use it in
the code, and that adds some degree of cognitive burden to the
implementation effort *if one does not already know protobuf*

* The use of protobuf either limits one to platforms for which a functional
implementation of protobuf exists,  OR ...

* The use of protobuf adds implementation overhead in that one must first
implement the subset of protobuf used by DOTS before one can add DOTS
support to that platform.

* To my knowledge, protobuf is not often used in RFCs, so those with
experience reading / implementing protocols defined in other RFCs may have
to spend more time trying to study and understand this document than they
would otherwise.

> >>> struct.pack("BB", 46, 46)

Right.  For what it's worth, I did actually laugh at that once it was
explained.

In any event, I do regret my lack of humour in this instance, and hope that
such will not be interpreted as a poor reflection upon my status as an IETF
member or as a human being.

Cheers,
Gilbert

---

From: "Mortensen, Andrew" <[email-masked]>
Date: Wed, 16 Nov 2016 17:30:47 +0000

Thanks for taking the time to spell out your concerns. I’ve clearly failed to highlight one of the advantages of protobufs—code generation from the schema—and I hope the inline answers help clear things up.

On Nov 16, 2016, at 11:18 AM, Gilbert Clark <[email-masked]<mailto:[email-masked]>> wrote:

Inline:

On Wed, Nov 16, 2016 at 12:54 AM, Mortensen, Andrew <[email-masked]<mailto:[email-masked]>> wrote:
… snip ...
Can you describe what you’re thinking of as “more common” and “easier to read” here? We specifically avoided JSON and the like because of the wasted bytes in the serialization.

My specific concerns with regard to protobuf (in this particular instance) would be:

* The signaling protocol should be pretty simple - how much more complex would it be to describe a custom encoding on the wire without the use of any kind of general-purpose abstraction (read: CBOR / protobuf / JSON / what have you)?  If that's too hard, I think an important question here would be "why is this too hard”?

Are you arguing for TLVs? We could certainly use them, but (assuming protobufs is the alternative) one of the tradeoffs is a significant loss of on-the-wire encoding efficiency. Protobufs uses VarInt encodings for integer values, so e.g. a 64-bit integer field with a value of 0 takes one byte on the wire instead of 8. (See <https://tools.ietf.org/html/draft-rfernando-protocol-buffers-00#section-3.1>.) That has significant advantages

Maybe I should step back from trying to point out the individual advantages to protobufs (as I see them) and ask what it is you think makes more sense. Your references to the 32-bit-wide diagrams from the IP, TCP and UDP specs have me thinking TLVs, but maybe you’ve got something else in mind.

* On a related note, protobuf invites complexity by offering a convenient abstraction to shield one from having to think about the way the bytes are actually laid out on the wire.  This can be a win from a rapid development perspective, but can be a loss in that it introduces a temptation to add more stuff to the protocol just because it's so easy to do it …

I strongly disagree with this. I don’t see how this is different from JSON or any other object serialization format.

* The use of protobuf requires an implementer to learn protobuf well enough that they're able to use it.  Yes, protobuf is an IDL, but one would need to learn an implementation of protobuf well enough to be able to use it in the code, and that adds some degree of cognitive burden to the implementation effort *if one does not already know protobuf*

I don’t follow the argument here. All serialization formats have a cognitive burden when learning the particular language implementation. If one does not already know CBOR, one has to learn CBOR, and use available libraries to manipulate it.

* The use of protobuf either limits one to platforms for which a functional implementation of protobuf exists,  OR …

I’ve obviously not made it clear enough how one goes from protobufs to working code. Let me try to rectify that.

You start by defining a schema, which means assigning data types to numbered fields. You then compile the schema to generate code defining the message classes or structures, and the calls for serialization/deserialization. Google provides protoc, which can generate code for C++, Java, Go, Python, and many others. See <https://developers.google.com/protocol-buffers/docs/reference/overview> and <https://github.com/google/protobuf/blob/master/docs/third_party.md> for a full list of available implementations.

* The use of protobuf adds implementation overhead in that one must first implement the subset of protobuf used by DOTS before one can add DOTS support to that platform.

In protobufs the schema defines the implementation. With the code generated from the schema, there is almost no manual implementation required to serialize and deserialize the messages.

* To my knowledge, protobuf is not often used in RFCs, so those with experience reading / implementing protocols defined in other RFCs may have to spend more time trying to study and understand this document than they would otherwise.

Well, maybe. I’m not sure why this is necessarily an obstacle. The Open vSwitch database management protocol (OVSDB) is RFC 7047. It uses JSON-RPC, which hasn’t been blessed by any standards body.

For what it’s worth, there was an effort to standardize protobufs within the IETF a few years ago, but it stalled:

<https://tools.ietf.org/html/draft-rfernando-protocol-buffers-00#section-3.1>

I still haven’t gotten a satisfactory answer as to why it stalled, though.

andrew

---

From: Gilbert Clark <[email-masked]>
Date: Wed, 16 Nov 2016 13:57:39 -0500

Let's compress this conversation.  Might come back to revisit a few other
points later, but for now ...

There are *always* code generation targets that are unsupported.  Protobuf
will present a barrier to implementation on those unsupported targets, but
at the same time, it will speed development and deployment on targets that
do support it.

My goal with this conversation is to make sure that this has been
considered and is a conscious design decision, 'cause trying to implement
this protocol on e.g. a 33 MHz MIPS without a complete implementation of
libc might kind of suck the way things are written at the moment.

If we don't care, then that's cool ... just want to be sure it's a
conscious thing.

-Gilbert

---

From: "Mortensen, Andrew" <[email-masked]>
Date: Wed, 16 Nov 2016 19:03:59 +0000

> On Nov 16, 2016, at 1:57 PM, Gilbert Clark <[email-masked]> wrote:
> 
> Let's compress this conversation.  Might come back to revisit a few other points later, but for now ...
> 
> There are *always* code generation targets that are unsupported.  Protobuf will present a barrier to implementation on those unsupported targets, but  at the same time, it will speed development and deployment on targets that do support it.  
> 
> My goal with this conversation is to make sure that this has been considered and is a conscious design decision, 'cause trying to implement this protocol on e.g. a 33 MHz MIPS without a complete implementation of libc might kind of suck the way things are written at the moment.

OK, that gives me the missing context, thanks. There are a couple protobuf projects out there targeting embedded/low-resource systems, including nanopb (https://github.com/nanopb/nanopb). I don’t expect that to mean all your concerns should be allayed, but I hope it’s representative of the broad support for and use of protobufs.

andrew