Subject: [Spud] updated draft PLUS charter, rev. 1 June

From: Brian Trammell <[email-masked]>
Date: Wed, 1 Jun 2016 12:44:47 +0200

Greetings, all,

We've taken another editing pass to tighten the charter; results inline below at on GitHub (https://github.com/ietf-plus/charter). We think this is getting close to something it would be useful to discuss and wordsmith at a BoF.

Further comments welcome.

Thanks, cheers,

Brian and Mirja

Path Layer UDP Substrate (PLUS)
===============================

The PLUS working group's goal is to define a common shim layer atop the User
Datagram Protocol (UDP) to provide a transport-independent method to signal
flow semantics under transport and application control, necessary to enable
the deployment of new, encrypted transport protocols within the existing
Internet. UDP provides compatibility with currently deployed middleboxes as
well as ubiquitous support in endpoints, and supports userspace implementation
of new transport protocols. The working group will not specify any new
transport protocols.

The current Internet protocol stack does not provide explicit, in-band,
transport-independent signaling to on-path network devices. This has led to
the deployment of devices which perform implicit discovery of transport
semantics and traffic characteristics via inspection of protocol headers and
payload, a practice made possible when these are sent in the clear.

In order to support more ubiquitous deployment of encryption, and the
encryption of transport headers to allow deployment of new transport
protocols, explicit in-band signaling must be added to the stack. This
signaling must be transport protocol independent, and the types of information
signaled must be based on characteristics that can be independently verified
by devices on path, or that can be usefully applied without requiring a trust
relationship between endpoints and the path. Further, a feedback channel that
provides information from on-path devices back to endpoints and applications,
e.g. for error handling, is essential for the deployment and success of an
explicit cooperation approach.

While IP would seem to be the natural home for this facility, both IPv4 and
IPv6 options and extensions have deployment problems on their own, which makes
it hard to include any additional information in these protocols.

The PLUS working group will specify a new protocol as a Path Layer
UDP Substrate (PLUS), to support experimental deployment of
explicit cooperation between endpoints and devices on path, with the following goals:

- enable ubiquitous deployment of encrypted higher layer protocols
  by providing exposure of basic TCP-like semantics (e.g. SYN, FIN,
  RST flags) to devices on path (e.g. NATs and firewalls).

- allow applications and transport protocols to explicitly provide
  limited information with integrity protection to devices on path

- allow devices on path to provide unencrypted feedback and information
  about the path directly to sending endpoints, under sending endpoint
  control

- allow devices on path to provide unencrypted information about the
  path to receiving endpoints, with encrypted feedback to the
  sending endpoint, under sending endpoint control

This approach explicitly gives the control of information exposure back the
application and/or transport layer protocol on the end host. It is the goal of
PLUS to minimize the information exposed, to make information exposure
transparent, and to limit the level of detail to that useful for network
treatment, while encrypting everything else. Endpoint verification of
signaling integrity, careful design of minimal data structures, and
restrictive policies for registration of signals can help to meet this goal.
This is important to avoid future implicit treatment and resulting
ossification, as well as to minimize the privacy risks presented by explicit
cooperation.

Given that the primary goal of PLUS is to enable the deployment of transport
protocols with encrypted headers, we assume that the higher-layer protocol can
provide an encryption context that can be used by PLUS to provide
authentication, integrity, and encryption where needed. The primary threat
model to defend against will be modification or deletion of exposed
information by middleboxes and other devices on path, by allowing a remote
endpoint to detect modifications.

The working group will start with an initial set of use cases (see draft-
kuehlewind-spud-use-cases) and requirements (see draft-trammell-spud-req),
taken from experience with the Substrate Protocol for User Datagrams (SPUD)
prototype. The working group's main output will be an experimental protocol
specification, together with an initial registry of types of information that
can be exposed using PLUS, clearly aligned to the use cases determined by the
working group. The working group will close if it is not able to come to
consensus on a protocol design to meet these requirements.

The working group will additionally aim to identify and work with other
working groups that could address parts of these requirements within existing
protocols, e.g. by specifying new protocol extensions, or as input for on-
going standardization work. It will aim to work with working groups defining
encryption protocols (e.g. DTLS) which could be used for encryption of
transport protocols running over PLUS.

---

From: Aaron Falk <[email-masked]>
Date: Tue, 7 Jun 2016 13:40:22 -0400

Apologies for being behind on the list and maybe repeating points that have
been made.  A few comments based on my review of the BoF proposal & charter:

   - 1st para of the charter says “The working group will not specify any
   new transport protocols.” and the 5th para says “The PLUS working group
   will specify a new protocol”.  I think the latter statement is correct.
   - I’d like to see a comment on PLUS re-enabling ICMP functionality
   - There’s the obvious & unaddressed question about the relation to QUIC
   - The PLUS Bof description should start something like “This BoF is to
   discuss the proposed PLUS working group.  The wg’s goal is to…”
   - No discussion of wg anti-goals.  I had heard some interest in ruling
   middlebox to middlebox signaling as out of scope.
   - (To ADs:) Would like to see the QUIC and PLUS BoFs scheduled
   back-to-back.  While I think the topics are mostly separable, if the
   discussion gets out of sync it will be a headache.

HTH,

--aaron

On Wed, Jun 1, 2016 at 6:45 AM Brian Trammell <[email-masked]> wrote:

> Greetings, all,
>
> We've taken another editing pass to tighten the charter; results inline
> below at on GitHub (https://github.com/ietf-plus/charter). We think this
> is getting close to something it would be useful to discuss and wordsmith
> at a BoF.
>
> _______________________________________________
> Spud mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spud
>

---

From: Aaron Falk <[email-masked]>
Date: Tue, 07 Jun 2016 17:44:19 +0000

Oh, and there should be some draft deliverables so we can have a fight
about the realism of the associated dates.

On Tue, Jun 7, 2016 at 1:40 PM Aaron Falk <[email-masked]> wrote:

> Apologies for being behind on the list and maybe repeating points that
> have been made.  A few comments based on my review of the BoF proposal &
> charter:
>
>
> --
--aaron

=====Short message from my phone=====

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 7 Jun 2016 21:32:28 +0200

Hi Aaron,

a few comments/answers mostly as proponent (if not indicated differently) in line. 

> Am 07.06.2016 um 19:44 schrieb Aaron Falk <[email-masked]>:
> 
> Oh, and there should be some draft deliverables so we can have a fight about the realism of the associated dates.

As AD, I find this less important then the rest of the text. As proponent, I think the milestones/deliverables are more important than the dates... And there are actually a few open questions here, that I would like to discuss in BoF. But I guess we can start this discussion on the list before the meeting.

> Apologies for being behind on the list and maybe repeating points that have been made.  A few comments based on my review of the BoF proposal & charter:
> 
> 	• 1st para of the charter says “The working group will not specify any new transport protocols.” and the 5th para says “The PLUS working group will specify a new protocol”.  I think the latter statement is correct.

No both is correct. Yes, we want to specify a new protocol, but I wouldn’t call it a transport protocol, because it’s ‚only‘ a signal shim layer, which does not have any of the other typical transport feature because there always has to be a ‚real‘ transport protocol on top. With this first sentences we wanted to make that clear; maybe it’s still not clear enough.

> 	• I’d like to see a comment on PLUS re-enabling ICMP functionality

Yes, that’s a very good use case and clearly in scope. However, there are other good use cases as well, see the use case draft, and we decided to not explicitly outline one (of a subset of them) in the charter.

> 	• There’s the obvious & unaddressed question about the relation to QUIC

I also think that this does not really belong in the charter. It’s for sure important to make this clear in the BoF and that why we have the agenda item on 'Relationship to other work in the IETF‘.

> 	• The PLUS Bof description should start something like “This BoF is to discuss the proposed PLUS working group.  The wg’s goal is to…”

Don’t know; isn’t that implicit for a working group forming BoF…?

> 	• No discussion of wg anti-goals.  I had heard some interest in ruling middlebox to middlebox signaling as out of scope.

That’s a good point. I believe that’s out of scope and would be willing to add that.

> 	• (To ADs:) Would like to see the QUIC and PLUS BoFs scheduled back-to-back.  While I think the topics are mostly separable, if the discussion gets out of sync it will be a headache.

As AD, we already discussed that we would like to see the QUIC BoF first because at least I believe otherwise there is a high risk to have the QUIC BoF in the SPUD BoF. However, I don’t think back to back is a good idea (to have some cooling down phase in the mean time). And given that both BoFs look for a 2.5h slots, they probably will be in two morning slots on different days.

Mirja

---

From: Aaron Falk <[email-masked]>
Date: Tue, 7 Jun 2016 15:53:50 -0400

Hi Mirja!

> On Jun 7, 2016, at 3:32 PM, Mirja Kühlewind <[email-masked]> wrote:
> 
> Hi Aaron,
> 
>> Am 07.06.2016 um 19:44 schrieb Aaron Falk <[email-masked] <mailto:[email-masked]>>:
>> 

I was joking about the argument re: dates.  But I think we are in agreement that there should be discussion about what the deliverables should be.  This would be helped with a strawman in advance.

> On Tue, Jun 7, 2016 at 1:40 PM Aaron Falk <[email-masked] <mailto:[email-masked]>> wrote:

I am unclear.  Maybe others are not.  Are you inventing a new ’thing’ by not calling it a "shim that is not a transport protocol"?  Is there some reason to do so?  

> Yes, that’s a very good use case and clearly in scope. However, there are other good use cases as well, see the use case draft, and we decided to not explicitly outline one (of a subset of them) in…

Fair point but I think the charter would benefit from at least a brief listing of the benefits that come from light weight, in-band signaling.  Besides their important role as scoping documents for working groups, charters (IMO) should help someone figure out what problems you are solving.  This can of course be done at length in a use-case doc but I believe providing a clue for the curious outsider as to whether the work is relevant.  YMMV.

> I also think that this does not really belong in the charter. It’s for sure important to make this clear in the BoF and that why we have the agenda item on 'Relationship to other work in the IETF’.

OK.  I probably should have figured that out by who was presenting.

> Don’t know; isn’t that implicit for a working group forming BoF…?

Meh, I disagree.  Clearly a nit.

> That’s a good point. I believe that’s out of scope and would be willing to add that.

Are there other anti-goals?

> As AD, we already discussed that we would like to see the QUIC BoF first because at least I believe otherwise there is a high risk to have the QUIC BoF in the SPUD BoF. However, I don’t think back…

Fair enough.  My feelings about back-to-back are not strong and I am glad you’ve given it thought.

—aaron

---

From: Ted Hardie <[email-masked]>
Date: Tue, 7 Jun 2016 13:05:12 -0700

Howdy,

No both is correct. Yes, we want to specify a new protocol, but I wouldn’t
call it a transport protocol, because it’s ‚only‘ a signal shim layer,
which does not have any of the other typical transport feature because
there always has to be a ‚real‘ transport protocol on top. With this first
sentences we wanted to make that clear; maybe it’s still not clear enough.

> I am unclear.  Maybe others are not.  Are you inventing a new ’thing’ by
> not calling it a "shim that is not a transport protocol"?  Is there some
> reason to do so?
>
>
Well, I don't want to speak for Mirja, but from my perspective, you get a
different thing if you want to recreate a re-usable shim/toolkit than what
you get from creating a transport.

Imagine for a moment that we are talking about transporting RTP/UDP, and we
would like signalling from the path to give us icmp-like information.  We
could do that by recreating RTP with a  signalling piece built in to a
protocol replacing UDP; we could do that by updating UDP itself; we could
describe how to encapsulate the RTP/UDP in something that carried
signalling but was otherwise relying on the encapsulated protocol for
everything.  The advantage of doing the last is that you don't have to
recreate each transport when you want to use it with signalling.  You can
also use the standard transports without this, if your deployment doesn't
need it.

YMMV, obviously.

Ted

> Yes, that’s a very good use case and clearly in scope. However, there are
> other good use cases as well, see the use case draft, and we decided to not
> explicitly outline one (of a subset of them) in the charter.
>
>
> Fair point but I think the charter would benefit from at least a brief
> listing of the benefits that come from light weight, in-band signaling.
> Besides their important role as scoping documents for working groups,
> charters (IMO) should help someone figure out what problems you are
> solving.  This can of course be done at length in a use-case doc but I
> believe providing a clue for the curious outsider as to whether the work is
> relevant.  YMMV.
>
>
> I also think that this does not really belong in the charter. It’s for
> sure important to make this clear in the BoF and that why we have the
> agenda item on 'Relationship to other work in the IETF’.
>
>
> • The PLUS Bof description should start something like “This BoF is to
> • No discussion of wg anti-goals.  I had heard some interest in ruling
> • (To ADs:) Would like to see the QUIC and PLUS BoFs scheduled
> As AD, we already discussed that we would like to see the QUIC BoF first
> because at least I believe otherwise there is a high risk to have the QUIC
> BoF in the SPUD BoF. However, I don’t think back to back is a good idea (to
> have some cooling down phase in the mean time). And given that both BoFs
> look for a 2.5h slots, they probably will be in two morning slots on
> different days.
>
>

---

From: Aaron Falk <[email-masked]>
Date: Tue, 7 Jun 2016 16:07:30 -0400

> On Jun 7, 2016, at 4:05 PM, Ted Hardie <[email-masked]> wrote:
> 
> Well, I don't want to speak for Mirja, but from my perspective, you get a different thing if you want to recreate a re-usable shim/toolkit than what you get from creating a transport.  
> 
> Imagine for a moment that we are talking about transporting RTP/UDP, and we would like signalling from the path to give us icmp-like information.  We could do that by recreating RTP with a  signalling piece built in to a protocol replacing UDP; we could do that by updating UDP itself; we could describe how to encapsulate the RTP/UDP in something that carried signalling but was otherwise relying on the encapsulated protocol for everything.  The advantage of doing the last is that you don't have to recreate each transport when you want to use it with signalling.  You can also use the standard transports without this, if your deployment doesn't need it.

Hi Ted!

So, we are creating an extension to UDP then, yes?  Why not call it that?

—aaron

---

From: Ted Hardie <[email-masked]>
Date: Tue, 7 Jun 2016 13:25:03 -0700

> So, we are creating an extension to UDP then, yes?  Why not call it that?
Well, because "extension" doesn't sound like you're creating an
encapsulating substrate?

I don't really want to paint the bikeshed too much over this, but if you
think "substrate" is hurting people's understanding of what this effort is
trying to do, we can certainly consider other terms.  Almost all of the
obvious ones are heavily overloaded, at least as I number them off on my
fingers and toes, though, so we may be stuck with phrases ("path-layer UDP
substrate") or invented metasyntactic variables ("fleen").

Ted

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Jun 2016 13:37:14 -0700

On Tue, Jun 7, 2016 at 10:40 AM, Aaron Falk <[email-masked]> wrote:
> 1st para of the charter says “The working group will not specify any new
> transport protocols.” and the 5th para says “The PLUS working group will
> specify a new protocol”.  I think the latter statement is correct.
> I’d like to see a comment on PLUS re-enabling ICMP functionality

What does "re-enabling ICMP functionality" mean?

Thanks,
Tom

> There’s the obvious & unaddressed question about the relation to QUIC
> The PLUS Bof description should start something like “This BoF is to discuss
> the proposed PLUS working group.  The wg’s goal is to…”
> No discussion of wg anti-goals.  I had heard some interest in ruling
> (To ADs:) Would like to see the QUIC and PLUS BoFs scheduled back-to-back.
> While I think the topics are mostly separable, if the discussion gets out of
> sync it will be a headache.
>
> HTH,
>
> --aaron
>
>
>> below at on GitHub (https://github.com/ietf-plus/charter). We think this is
>> getting close to something it would be useful to discuss and wordsmith at a
>> BoF.
>>

---

From: Aaron Falk <[email-masked]>
Date: Tue, 7 Jun 2016 16:41:21 -0400

> On Jun 7, 2016, at 4:37 PM, Tom Herbert <[email-masked]> wrote:
> 

The usefulness of many of the the ICMP signals has been reduced since you have no indication of who is sending them.  With PLUS you can at least have some confidence the sender is on the path.

—aaron

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Jun 2016 13:47:28 -0700

On Tue, Jun 7, 2016 at 1:41 PM, Aaron Falk <[email-masked]> wrote:
> The usefulness of many of the the ICMP signals has been reduced since you
> have no indication of who is sending them.  With PLUS you can at least have
> some confidence the sender is on the path.
>
I'm still missing it. ICMP packets have a source address which of the
router or host that is sending the packet.

Tom

---

From: Aaron Falk <[email-masked]>
Date: Tue, 7 Jun 2016 16:57:47 -0400

> On Jun 7, 2016, at 4:47 PM, Tom Herbert <[email-masked]> wrote:
> 
> On Tue, Jun 7, 2016 at 1:41 PM, Aaron Falk <[email-masked] <mailto:[email-masked]>> wrote:
>> 

You can build equivalent functionality with PLUS.  Would be a different protocol but more trustworthy.

—aaron

---

From: Ted Hardie <[email-masked]>
Date: Tue, 7 Jun 2016 14:17:17 -0700

> > What does "re-enabling ICMP functionality" mean?

ICMP often isn't available to the application; by putting similar
information in in-band signalling, it would reach the application.  For the
RTP application I noted above, for example, that might result in a codec
shift or change in FEC tuning.

regards,

Ted

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 7 Jun 2016 23:25:25 +0200

Hi Aaron,

on this point:

> Am 07.06.2016 um 21:53 schrieb Aaron Falk <[email-masked]>:
> 
>>> 

Actually it’s in the charter (hidden):

"Further, a feedback channel that provides information from on-path devices back to endpoints and applications, e.g. for error handling, is essential for the deployment and success of an explicit cooperation approach.“

Mirja

---

From: Joe Touch <[email-masked]>
Date: Tue, 7 Jun 2016 14:34:07 -0700

On 6/7/2016 1:05 PM, Ted Hardie wrote:
> Imagine for a moment that we are talking about transporting RTP/UDP,
> and we would like signalling from the path to give us icmp-like
> information.  We could do that by recreating RTP with a  signalling
> piece built in to a protocol replacing UDP; we could do that by
> updating UDP itself;

FWIW, this might be a very interesting use of draft-touch-tsvwg-udp-options

(we are developing an implementation this summer after successful
NAT-traversal and legacy endsystem tests last year)

Joe

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 7 Jun 2016 23:57:54 +0200

Hi Joe,

> Am 07.06.2016 um 23:34 schrieb Joe Touch <[email-masked]>:
> 
> 
> 

Yes, I still have this in mind. However, in the mean time I’d prefer a proper architectural solution (and not just a hack). :-)

> NAT-traversal and legacy endsystem tests last year)

Cool. Good to know!

Mirja

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Jun 2016 15:05:34 -0700

> Yes, I still have this in mind. However, in the mean time I’d prefer a proper architectural solution (and not just a hack). :-)
The proper architectural solution is to place network layer
information like what is being proposed in the network layer :-)
Trying to make a network layer out of UDP fails on several accounts
(fragments don't contain UDP headers, ports do not have global
meaning, etc.). IPv6 extension headers are designed for carrying
information like this, I really hope there is more investigation as to
whether they are workable for signaling.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 7 Jun 2016 15:07:56 -0700

> Yes, I still have this in mind. However, in the mean time I’d prefer a proper architectural solution (and not just a hack). :-)

The UDP option field is a vehicle in which an architecture could send
info back in-band without affecting the UDP stream.

It's not the only solution and won't work for other transports, but if
you want ICMP-like feedback in-band for UDP, it will work and is as
architecturally clean as any other new/shim layer would be.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 7 Jun 2016 15:09:47 -0700

> whether they are workable for signaling.

It depends on what layer is actually doing the signaling. If it's
on-path, I agree - that's network layer and should be somewhere besides
the transport header. If'it's on-path but not from the endpoint it also
begs the question of whether IP packet headers should be extended
in-transit (I made the case long ago that this was valid for IPv6 but I
think someone is working to close that loophole or already has).

Joe

---

From: Ted Hardie <[email-masked]>
Date: Tue, 7 Jun 2016 15:18:43 -0700

> architecturally clean as any other new/shim layer would be.
So the problem with talking about any single requirement is that it looks a
good bit easier to match the capabilities required  than it turns out to
be.  If you go back to Section 5 of  draft-trammell-spud-req-04, you'll see
why the tube-id is useful (and necessary to avoid packet injection
attacks).  Unfortunately, the space available in this option field isn't
really sufficient for a reasonable tube-id plus the ICMP (or other
signalling) desired.

Just my view of the requirements, of course; I certainly see that folks
have different views of this.

regards,

Ted

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Jun 2016 15:19:15 -0700

> think someone is working to close that loophole or already has).
There has been a long discussion in 6man for 2460bis as to whether
IPv6 extension headers can be added by middelboxes. The consensus
seems to be that is it not allowed primarily because such actions
change the size of the packet and hence mess up PMTU discovery. I
would think this rationale must apply if devices are allowed to change
UDP payload in flight. Devices should never increase the size of a
packet, if they wish to add information to a packet for transit across
one network then encapsulation is a good alternative.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 7 Jun 2016 15:35:36 -0700

> one network then encapsulation is a good alternative.

Encapsulation doesn't magically solve the PTMU problem.

The only solution is to reserve space - e.g., for the IPv6 header to
have included "NOP" or other variants of reserved space that can be
overwritten.

UDP payloads should NEVER be modified in flight. Only the end system or
its designee should ever do so. I sincerely hope we end up assuming DTLS
or some such to ensure that this is the case.

Joe

---

From: Brian Trammell <[email-masked]>
Date: Wed, 8 Jun 2016 06:27:16 +0200

Hi Joe, all,

Inline...

> On 08 Jun 2016, at 00:35, Joe Touch <[email-masked]> wrote:
> 

I know that Jen Linkova has done some measurement work on this with respect to on-path treatment; I forget the conclusions. Of course that leaves the question of userspace implementation open.

> have included "NOP" or other variants of reserved space that can be

In the earlier thread on how to enforce endpoint control over what middleboxes can say and when, this is the solution we foresaw: reserved space in the PLUS header whose type and length are MAC'd, but the content not (probably implemented as a MAC over a fixed-length array of zeroes), this treatment being a property of the type.

> its designee should ever do so.

In this pattern, the end system explicitly designates the set of PLUS-speaking devices on path to modify that value, in accordance with the type's definition.

> I sincerely hope we end up assuming DTLS

That's my assumption.

I've been thinking of this in terms of a MAC using a key negotiated by the superstrate's crypto layer, which I assume is DTLS. But if we limit superstrates to those over DTLS, we can just use DTLS directly for this.

Cheers, 

Brian

---

From: Brian Trammell <[email-masked]>
Date: Wed, 8 Jun 2016 12:38:22 +0200

> On 08 Jun 2016, at 06:27, Brian Trammell <[email-masked]> wrote:
> 
> Hi Joe, all,
> 
> Inline...
> 

On the question of path transparency to DO extension headers in V6, see this talk from IEPG at IETF91:
http://iepg.org/2014-11-09-ietf91/iepg-ietf91-ipv6-ehs-in-the-real-world-v3.0.pdf (thanks Jen Linkova and Fernando Gont), the takeaway from which is that the minimum observed drop rate of small (8-byte) DO EH on UDP packets is about 6%.

There was also Geoff Huston's IPv6 WG presentation at RIPE a couple of weeks ago: https://ripe72.ripe.net/presentations/67-2016-05-23-bigipv6.pdf: in addition to a nice summary of why ICMP-as-is is broken, he found an upper bound of 16% on in-network fragmentation EH drops.

So the situation with IPv6 EH seems to be slightly worse than the situation with UDP, probably harder to diagnose at runtime, and requires kernel support. So while I agree this is architecturally cleaner, I'm skeptical that it's as deployable as a UDP encapsulation-based approach.

Cheers,

Brian

---

From: Joe Touch <[email-masked]>
Date: Wed, 8 Jun 2016 15:17:11 -0700

> So the situation with IPv6 EH seems to be slightly worse than the situation with UDP, probably harder to diagnose at runtime, and requires kernel support. So while I agree this is architecturally c…
It'd useful to consider the impact of current non-compliant systems on
new approaches, but not to be severely limited by them.

Nothing is universally deployable except HTTP in TCP, and we really
don't need to be reinventing the Internet inside HTTP.

Joe

---

From: Brian Trammell <[email-masked]>
Date: Thu, 9 Jun 2016 10:46:05 +0200

> On 09 Jun 2016, at 00:17, Joe Touch <[email-masked]> wrote:
> 
> 
> 

AFAIC, the marginal difference in brokenness between EH-based and UDP encapsulation-based approaches is way less important than lack of universal userspace access to extension headers. (I'm also not convinced that the types of path signaling envisioned in the use case draft are properly "network layer" -- since they involve the explicit participation of devices on path neither as packet forwarding devices at layer 3 nor as proper endpoints -- but that is a literally academic argument.)

It would be useful if whatever comes out of an eventual PLUS working group could be implemented either over UDP or IPv6 EH, since the lack of access to EH problem is simply a matter of changing APIs. But I'm not convinced that's a hard requirement.

> don't need to be reinventing the Internet inside HTTP.

We also don't have to, since it's already been done.

Cheers,

Brian

---

From: Tom Herbert <[email-masked]>
Date: Thu, 9 Jun 2016 07:55:55 -0700

> So the situation with IPv6 EH seems to be slightly worse than the situation with UDP, probably harder to diagnose at runtime, and requires kernel support. So while I agree this is architecturally c…
Brian,

It's not just a question of being "architecturally cleaner" it's a
question of correctness. I do not believe the neither the standard nor
the architecture of the Internet allows for middleboxes to parse
transport layer payloads (UDP is a transport layer protocol). This
obviously was happening for a long time with devices that were parsing
into TCP payloads to inspect HTTP headers and content, but that was
never done under a standard and we closing that loophole by using TLS.

A major problem is that in order for middleboxes to be able to parse
something they have to positively identify what they are parsing. Port
numbers are not sufficient for this since they only have end to end
meaning (RFC7605). For instance, port 80 does not always have to be
HTTP and conversely not all HTTP traffic has to use port 80. Magic
numbers are the suggested recourse, but regardless of how many bits
are used for the number that technique is still probabilistic to be
correct (btw at Internet scale 32 bits is way to small, I'm not even
sure 64 bits is okay).

This also potentially creates new attack vectors. Consider that an
attacker sends a well formed PLUS packet to a server running a UDP
Echo Protocol (RFC862). The server will happily reflect the data in a
UDP packet so that devices in the network now thinks it sees a PLUS
packets being sent from the server-- this is not correct.

Note that EH does not have these issues since IP protocol numbers have
global and unambiguous meaning. If an IPv6 packet contains 60 in a
next header field, then that next header may always be parsed to be
destination options.

Tom

---

From: Brian Trammell <[email-masked]>
Date: Thu, 9 Jun 2016 18:04:44 +0200

hi Tom,

> question of correctness.

I don't see these questions as distinct.

> I do not believe the neither the standard nor
> transport layer payloads (UDP is a transport layer protocol).

I disagree with the assertion that UDP is actually a transport layer protocol. UDP is a facility to allow processes on multitasking operating systems to safely send as-raw-as-possible datagrams to each other in an environment where one still expects link layer bit errors; i.e. the Internet of 1993.

Basically all of the functions of a real transport layer protocol must be implemented atop UDP-based protocols, and most UDP-based application protocols that actually work in the Internet either really only send one packet at a time or behave like transport-layer protocols.

> This

TLS does nothing to keep the middleboxes from mucking about in the TCP header, which is why we're having this whole discussion in the first place.

> HTTP and conversely not all HTTP traffic has to use port 80.

Too few bits in port numbers as well, yes.

> Magic
> correct (btw at Internet scale 32 bits is way to small,

Agreed.

> I'm not even sure 64 bits is okay

The observation that 64 bit strings at given positions in packets do not appear with uniform probability makes this a bit more okay, but yeah, I'd like to see data too.

> packets being sent from the server-- this is not correct.

I'll point out that the operational error here is connecting a server running Echo to the Internet; it might be that there's lots of echo running out there that people don't care about because its amplification factor is a bit crap, though. And this attack doesn't work against a PLUS that meets requirement 6.6 in the spud-req draft.

I don't see how any of this is germane to the PLUS-in-UDP versus PLUS-in-DO question though.

> next header field, then that next header may always be parsed to be

Using DO to communicate with middleboxes is also not really all that proper -- "destination" is IMO always distinct from "path" -- but since I'm basically saying we need to add a new layer to the stack on top of UDP headers, I probably shouldn't be talking about "proper". :)

So, this is all well and good; a little digging in ip6(7) and sendmsg(2) seems to indicate that I can at least get and set these with the IPV6_DSTOPT socket option, at least on Linux, but the types involved are still unclear from the documentation. I still think getting this right in a cross-platform way tends to indicate an over-UDP implementation. But that's an engineering question.

Cheers,

Brian

---

From: Joe Touch <[email-masked]>
Date: Thu, 9 Jun 2016 10:23:00 -0700

On 6/9/2016 9:04 AM, Brian Trammell wrote:
...
> I disagree with the assertion that UDP is actually a transport layer
> protocol. UDP is a facility to allow processes on multitasking
> operating systems to safely send as-raw-as-possible datagrams to each
> other in an environment where one still expects link layer bit errors;
> i.e. the Internet of 1993. Basically all of the functions of a real
> transport layer protocol must be implemented atop UDP-based protocols,
> and most UDP-based application protocols that actually work in the
> Internet either really only send one packet at a time or behave like
> transport-layer protocols.

By your argument, there are no transport protocols.

UDP *is* a transport by providing all-or-nothing messaging, demuxing to
an upper layer protocol/application, and preserving message boundaries.
Granted that's not everything some apps want, but neither is TCP.

Consider that TCP provides reliable, ordered, bytestream service even
though many apps really want message boundaries. Or concurrent
interleaved messages. Or message chunking-and-muxing to avoid HOL
blocking of large messages. I.e., that's how we ended up with a lot of
what's in HTTP - to adjust for things TCP didn't provide or to undo some
things it did provide.

Your argument that apps often want services existing transports don't
exactly provide is undeniable, but that doesn't mean these aren't
transport protocols.

You've only pointed out the known flaw in the OSI model - that there's
no such thing as a fixed number of protocol layers or any one layer with
a single defining property.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 10 Jun 2016 07:57:45 -0700

> I'll point out that the operational error here is connecting a server running Echo to the Internet; it might be that there's lots of echo running out there that people don't care about because its…
Regardless of what we think of  Echo Server, it is a standard and it
is legitimate to deploy it on the Internet (DOS concerns can be
mitigated by rate limiting). Anti address spoofing does not help since
nothing is being spoofed, all address are properly routable, without
any sort of authentication between senders and network devices I don't
that there is anything we can put into in the UDP payload to resolve
this issue.

> I don't see how any of this is germane to the PLUS-in-UDP versus PLUS-in-DO question though.

Robustness, correctness, and security are germane to any discussion of
a proposed protocol.
> Using DO to communicate with middleboxes is also not really all that proper -- "destination" is IMO always distinct from "path" -- but since I'm basically saying we need to add a new layer to the s…
I was not suggesting DO that was only an example of the invariant
property that IP protocol numbers have and UDP port numbers don't.
Hop-by-hop options are probably more appropriate for signaling to/from
network.

> So, this is all well and good; a little digging in ip6(7) and sendmsg(2) seems to indicate that I can at least get and set these with the IPV6_DSTOPT socket option, at least on Linux, but the types…
AFIACT all the major OSes (Linux, WIndows, IOS, BSDs) implement the
APIs in RFC3542. The caveat is that sending of particular options by
an unprivileged user can be restricted. As options are defined and
shown to be safe to use by unprivileged user they can be whitelisted
and allowed.

Tom

---

From: Brian Trammell <[email-masked]>
Date: Fri, 10 Jun 2016 17:25:44 +0200

> On 10 Jun 2016, at 16:57, Tom Herbert <[email-masked]> wrote:
> 

Reflecting off an echo server doesn't give you return routability.

> Robustness, correctness, and security are germane to any discussion of

You're raising an issue with any protocol that runs over UDP in an Internet with echo servers in it, and my understanding is the arrangement you're proposing here is:

ip
 (EH for path exposure)
udp
  dtls
     (new protocols)

whereas the one I think works is:

ip
udp
  plus for path exposure
    dtls
      (new protocols)

I don't see how these are any different with respect to how they have to handle the echo server attack. What am I missing here?

> Hop-by-hop options are probably more appropriate for signaling to/from

My impression is that HBH options were being more or less deprecated, because they're largely unimplementable in hardware, due to unpredictable header chain lengths and header chain walking time. Am I incorrect here?

(FWIW, sticking this in UDP makes it possible to provide constant offsets when no extension headers are in use and when the superstrate uses the path MTU correctly.)

> APIs in RFC3542.

Good to know; this makes this idea more deployable. :)

Regards,

Brian

> The caveat is that sending of particular options by

---

From: Tom Herbert <[email-masked]>
Date: Fri, 10 Jun 2016 09:03:30 -0700

> You're raising an issue with any protocol that runs over UDP in an Internet with echo servers in it, and my understanding is the arrangement you're proposing here is:

Plus introduces new issues. All prior uses of UDP on the Internet have
been end to end communications, application to application. PLUS is
introducing the notion that UDP is used for application to network and
network to application communications also. For end to end
communications we can apply strong security (e.g. DTLS) so that
spoofed or reflected UDP packets are not accepted.

The issues are even worse if intermediate devices are allowed to
modify payloads. Consider that we can not retroactively reserve a
stream of bits at the beginning of every UDP payload to indicate PLUS.
Regardless of how much data we collect on a set of safe bits to use,
eventually some user will happen send those same exact bits in their
application protocol. If such packets are modified by PLUS we now have
introduced a silent data corruption. A protocol that allows silent
data corruptions cannot be considered correct.

> I don't see how these are any different with respect to how they have to handle the echo server attack. What am I missing here?
The difference is that in case one the EH headers in a packet sent an
echo server and not automatically reflected in the reply packet. In
case #2 the whole UDP payload is reflected so the faux PLUS header is
reflected and devices on the return path will process that information
as signals to the network being sent from the Echo Server. This allows
an attacker to affect the affect the devices in the reverse path.

> My impression is that HBH options were being more or less deprecated, because they're largely unimplementable in hardware, due to unpredictable header chain lengths and header chain walking time. A…
If nodes can't process HBH headers they can be ignored. This needs to
be clarified in 2460bis.

Tom

---

From: Christian Huitema <[email-masked]>
Date: Fri, 10 Jun 2016 20:12:41 +0000

On Friday, June 10, 2016 9:04 AM, Tom Herbert wrote:
> ...
> Plus introduces new issues. All prior uses of UDP on the Internet have been end
> to end communications, application to application. PLUS is introducing the
> notion that UDP is used for application to network and network to application
> communications also. For end to end communications we can apply strong
> security (e.g. DTLS) so that spoofed or reflected UDP packets are not accepted.

I think Tom has a good point here. PLUS does introduce new communication patterns, passing information to intermediate routers and expecting routers to act on the information. These communication patterns can very well introduce new attack vectors. We actually discussed a few of those on the list some time back. For example, an attacker could inject a packet that mimics the closure of a flow, and cause intermediate firewalls to close the holes open for that flow.

I suggest that we recognize the link between new patterns and new attacks in the charter, and have an explicit goal to investigate these attacks and their mitigations. 

-- Christian Huitema

---

From: Joe Touch <[email-masked]>
Date: Fri, 10 Jun 2016 17:14:14 -0700

> whereas the one I think works is:

The former will work with IPsec and is consistent with the Internet
architecture.

The latter is not. A distributed protocol that both relies on IP and is
not consistent with IPsec is, IMO, a non-starter.

Joe

---

From: Brian Trammell <[email-masked]>
Date: Sat, 11 Jun 2016 13:49:12 +0200

> On 10 Jun 2016, at 22:12, Christian Huitema <[email-masked]> wrote:
> 

Except this isn't really a new attack vector; there's no real difference between this and a FIN/RST injection in TCP, except we get a chance to make the space the attacker has to successfully guess in larger.

> I suggest that we recognize the link between new patterns and new attacks in the charter, and have an explicit goal to investigate these attacks and their mitigations.

Absolutely; added an issue to the draft charter, will propose text next week

Cheers,

Brian

---

From: Tom Herbert <[email-masked]>
Date: Sat, 11 Jun 2016 09:23:23 -0700

> Except this isn't really a new attack vector; there's no real difference between this and a FIN/RST injection in TCP, except we get a chance to make the space the attacker has to successfully guess…
By doing TCP over DTLS over UDP we eliminate the possibility of
injection attacks against TCP like this. That is a major reason why we
want to encrypt transport headers.

---

From: Szilveszter Nadas <[email-masked]>
Date: Mon, 13 Jun 2016 15:21:27 +0000

Hi,

> It would be useful if whatever comes out of an eventual PLUS working group
> could be implemented either over UDP or IPv6 EH, since the lack of access to
> EH problem is simply a matter of changing APIs. But I'm not convinced that's a
> hard requirement.

Is a separate protocol number for SPUD also an option? Then the happy eyeball mechanism used could also consider that.

Cheers,
Sz.

---

From: Szilveszter Nadas <[email-masked]>
Date: Mon, 13 Jun 2016 15:38:50 +0000

Hi,

On the topics of what is included and what not. The PLUS protocol kind of reminds me of this paper "Breaking Up the Transport Logjam" http://bford.info/pub/net/logjam.pdf . Though the "Flow Regulation Layer" in the paper can be used for much more than PLUS. Does PLUS intends to be extensible to implement/aid things like this in the far future or do things like this belong to different part of the stack?

Cheers,
Szilveszter

---

From: Joe Touch <[email-masked]>
Date: Mon, 13 Jun 2016 08:46:50 -0700

On 6/11/2016 4:49 AM, Brian Trammell wrote:
> Except this isn't really a new attack vector; 

Not a new type, but it would be a new instance.

> there's no real difference between this and a FIN/RST injection in TCP, except we get a chance to make the space the attacker has to successfully guess in larger.
It would be very useful to be clear about your threat model before
assuming a given solution.

E.g., no increase in number space will prevent middlebox FIN/RST
manipulation of endpoint associations ("connections").

Joe

---

From: Ca By <[email-masked]>
Date: Tue, 14 Jun 2016 05:45:34 -0700

On Monday, June 13, 2016, Szilveszter Nadas <[email-masked]>
wrote:

> Is a separate protocol number for SPUD also an option? Then the happy
> eyeball mechanism used could also consider that.
>
> Cheers,
> Sz.
>
>
All my concerns with spud and quic would be solved by using a new protocol
number since it would allow for the network to, at scale, identify good
spud/quic traffic from the known bad  udp ddos mess

This network operator concern has been repeatedly ignored since it
is easier to access udp in userspace.

This conflict will not end well if not addressed

> Spud mailing list
> [email-masked] <javascript:;>

---

From: Szilveszter Nadas <[email-masked]>
Date: Tue, 14 Jun 2016 14:50:33 +0000

Hi,

I was probably mis-using the term happy eyeballs. I meant that whatever mechanism selecting how SPUD is used may also consider and try the separate protocol number option before falling back to e.g. UDP.

I agree that separate protocol number is not deployable in itself. Whether it is at all practical now and in the foreseeable future, I am not sure, it might be worth a consideration. Some data on this (similarly that for UDP and IPv6 extension header) might help. 

Summarizing, I see it as something to consider during the design, and definitely not the mainstream solution in the short run. 

Cheers,
Sz.

> -----Original Message-----
> From: Brian Trammell [mailto:[email-masked]]
> Sent: Tuesday, June 14, 2016 16:44
> To: Szilveszter Nadas
> Cc: Joe Touch; spud
> Subject: Re: [Spud] updated draft PLUS charter, rev. 1 June
> 
> 
> > On 13 Jun 2016, at 17:21, Szilveszter Nadas
> <[email-masked]> wrote:
> >
> > Hi,
> >
> >
> >> It would be useful if whatever comes out of an eventual PLUS working
> >> group could be implemented either over UDP or IPv6 EH, since the lack
> >> of access to EH problem is simply a matter of changing APIs. But I'm
> >> not convinced that's a hard requirement.
> >
> > Is a separate protocol number for SPUD also an option?
> 
> 
> It doesn't seem to me that using an alternate protocol number for PLUS (or
> QUIC) actually meets the requirements of either effort. One is the ability to run
> on unmodified kernels, which is a somewhat bigger deal for QUIC than for
> PLUS. Another is middlebox transparency, where experience with SCTP has not
> been great. True, we don't have a lot of data here; on our list of path
> transparency measurements to run at ETH is "UDP/TCP with mangled protocol
> number"; I suspect that NAPT breaks in the overwhelming majority of cases
> here.	
> 
> In a world in which every kernel is relatively quickly updatable, NAPT is
> irrelevant, and other middlebox brokenness negligent, alternate protocol
> numbers are an option. We are somewhat closer to that world, I think, than
> we were a decade ago. But I don't think we live in it yet.
> 
> > Then the happy eyeball mechanism used could also consider that.
> 
> I'm not sure how the protocol number is relevant to happy eyeballs... can you
> expand on this a bit?
> 
> Cheers,
> 
> Brian
> 
> > Cheers,
> > Sz.

---

From: Brian Trammell <[email-masked]>
Date: Tue, 14 Jun 2016 16:43:54 +0200

> On 13 Jun 2016, at 17:21, Szilveszter Nadas <[email-masked]> wrote:
> 
> Hi,
> 
> 

It doesn't seem to me that using an alternate protocol number for PLUS (or QUIC) actually meets the requirements of either effort. One is the ability to run on unmodified kernels, which is a somewhat bigger deal for QUIC than for PLUS. Another is middlebox transparency, where experience with SCTP has not been great. True, we don't have a lot of data here; on our list of path transparency measurements to run at ETH is "UDP/TCP with mangled protocol number"; I suspect that NAPT breaks in the overwhelming majority of cases here.

In a world in which every kernel is relatively quickly updatable, NAPT is irrelevant, and other middlebox brokenness negligent, alternate protocol numbers are an option. We are somewhat closer to that world, I think, than we were a decade ago. But I don't think we live in it yet.

> Then the happy eyeball mechanism used could also consider that.

I'm not sure how the protocol number is relevant to happy eyeballs... can you expand on this a bit?

Cheers,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Tue, 14 Jun 2016 16:57:56 +0200

> On 13 Jun 2016, at 17:46, Joe Touch <[email-masked]> wrote:
> 
> 
> 

True, but defending against endpoint state manipulation by a box on path, in the present deployed Internet in which at least endpoint-to-endpoint flows are always identified by a five-tuple, wherein said box can always simply drop packets identified by said five-tuple, is not very useful. The best you can do is detect it happened. At least an RST-injecting on-path device is nice enough to tell you it's killing your flow.

In the case of a PLUS-encapsulated, encrypted transport, the flow state bits would presumably be integrity-protected against change, using a MAC that could only be generated by the sender, so a receiving endpoint could certainly detect that the flow state bits were manipulated, or that the packet was forged. But just as today, it couldn't detect that the middlebox decided simply to start dropping all the packets (or rather, differentiate that state from some other link failure).

You're correct that the number-space defense is primarily useful against off-path attacks, which are more useful to defend against than on-path ones; indeed, if [1] is to be believed (which I tend to think it is), this is still very much a problem in TCP as deployed.

Cheers,

Brian

[1] Luckie et al "Resilience of Deployed TCP to Blind Attacks" http://conferences.sigcomm.org/imc/2015/papers/p13.pdf

---

From: Brian Trammell <[email-masked]>
Date: Tue, 14 Jun 2016 16:59:21 +0200

hi Szilveszter,

> On 14 Jun 2016, at 16:50, Szilveszter Nadas <[email-masked]> wrote:
> 
> Hi,
> 

I'd tend to say that designing PLUS such that it can never run over a protocol number than 17 would be a mistake, yes. But in the short term it's the only thing I see that has a chance to deploy.

Cheers,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Tue, 14 Jun 2016 17:06:01 +0200

> On 13 Jun 2016, at 17:38, Szilveszter Nadas <[email-masked]> wrote:
> 
> Hi,
> 
> On the topics of what is included and what not. The PLUS protocol kind of reminds me of this paper "Breaking Up the Transport Logjam" http://bford.info/pub/net/logjam.pdf .

I'm a big fan of that paper, too. :)

> Though the "Flow Regulation Layer" in the paper can be used for much more than PLUS. Does PLUS intends to be extensible to implement/aid things like this in the far future or do things like this belong to different part of the stack?

In Logjam terminology, PLUS uses UDP as an endpoint layer, and adds additional path communication to that layer (or rather to a Path Layer over that layer). Think of it as "Transport Independent Middlebox Traversal"  (section 2.2.2) greatly expanded.

Some of the functions of Flow Regulation might be implementable with middlebox cooperation over PLUS, but the logjam paper really assumes a bit more understanding and implementation of transport semantics within flow middleboxes than PLUS to date has assumed. Definitely a detailed discussion to have within the BoF or any future WG.

Cheers,

Brian

---

From: Christian Huitema <[email-masked]>
Date: Tue, 14 Jun 2016 19:47:04 +0000

On Tuesday, June 14, 2016 5:46 AM, Ca By wrote:

>
> All my concerns with spud and quic would be solved by using a 
> new protocol number since it would allow for the network to, 
> at scale, identify good spud/quic traffic from the known bad  udp ddos mess

I think we need to dig a bit further there. Botnets could issue spud/quic traffic just fine. Granted, it will take them some time to adapt, but history shows that they will adapt. So, I don't see how painting a fixed set of bits in the packets would help in the long run. Botnets engaged in DOS will just set the same bits, be it a protocol number or a header field.

There may be something specific to do about differentiation from UDP reflection attacks. I am aware that a lot of these are DNS packets. There may be some other UDP services as well. But the general characteristic is that such attacks reuse a deployed infrastructure to their benefits, and thus have specific markers. The infrastructure is not going to change, so the markers will stay. The new protocols should certainly try to differentiate from these existing attacks. 

> This network operator concern has been repeatedly ignored since
> it is easier to access udp in userspace. 

As many said, user space is just one of the problems. The other is the installed base of NATs that won't pass anything else than TCP and UDP. Do we have studies indicating whether that changes with IPv6?

-- Christian Huitema

---

From: 🔓Dan Wing <[email-masked]>
Date: Tue, 14 Jun 2016 13:58:46 -0600

On 14-Jun-2016 06:45 am, Ca By <[email-masked]> wrote: 
> 
> 
> On Monday, June 13, 2016, Szilveszter Nadas <[email-masked] <mailto:[email-masked]>> wrote:
> Hi,
> 
> 
> All my concerns with spud and quic would be solved by using a new protocol number since it would allow for the network to, at scale, identify good spud/quic traffic from the known bad  udp ddos mess
> 
> This network operator concern has been repeatedly ignored since it is easier to access udp in userspace. 
> 

See the just-published https://tools.ietf.org/html/draft-wing-quic-network-req-00#section-2 <https://tools.ietf.org/html/draft-wing-quic-network-req-00#section-2>.

-d

---

From: Brian Trammell <[email-masked]>
Date: Wed, 15 Jun 2016 08:15:14 +0200

hi Christian,

> On 14 Jun 2016, at 21:47, Christian Huitema <[email-masked]> wrote:
> 

Agreed.

> There may be something specific to do about differentiation from UDP reflection attacks. I am aware that a lot of these are DNS packets. There may be some other UDP services as well.

Reflection attacks tend to follow the amplification factor: more amplification, bigger win. NTP (due, I think, to a widely deployed configuration that was amplification-friendly) was the champion here; DNS with DNSSEC looms as the next one (see the work of Roland van Rijswijk-Deij, Anna Sperotto, and Aiko Pras here: [1, 2]).

> But the general characteristic is that such attacks reuse a deployed infrastructure to their benefits, and thus have specific markers. The infrastructure is not going to change, so the markers will stay. The new protocols should certainly try to differentiate from these existing attacks.

This is the rationale behind the selection of the magic number in the SPUD prototype, and the presence of a magic number in the requirements derived therefrom.

I am also intrigued by Tom's earlier suggestion that using IPv6 DO headers might have reflection-reducing properties, given the installed base. We should look into that more deeply.

None of this does *anything* against direct botnet attacks -- there are maybe things you could do in a world with ubiquitous PLUS-speaking middleboxes that distinguished a PLUS magic number flood from a bot from real PLUS-encapsulated traffic. But relying on both ubiquitous deployment (a decade out in the wildly optimistic case) and the kindness of strangers to actually deploy egress filtering (which has worked oh so well in the past) isn't what I'd call a security plan.

> As many said, user space is just one of the problems. The other is the installed base of NATs that won't pass anything else than TCP and UDP. Do we have studies indicating whether that changes with…

No direct numbers, in part because the measurement infrastructures available to us give us much smaller population sizes for v6 measurement. Anecdotally, taken from analysis of the data in [3, 4, 5], I can say that the magnitude of brokenness on v6 in general is lower, but the kinds of brokenness you run into is weirder. Slide 9 in [5] is v4 only because we still haven't been able to explain anomalies in the MTU curve for UDP on IPv6; in both [3] and the detailed analysis linked from [4] we see ECN negotiation anomalies five times more often for v6 than v4.

If I had to make up an explanation for that, I'd say that v6 traffic does indeed tend to pass through fewer middleboxes, but that the v6 functionality in those middleboxes is far less well tested. Happy eyeballs works so well because it quite effectively hides a multitude of sins.

Cheers,

Brian

[1] DNSSEC and its potential for DDoS attacks at IMC '14 https://wwwhome.ewi.utwente.nl/~rijswijkrm/pub/imc101-vanrijswijk.pdf

[2] Making the case for elliptic curves in DNSSEC in CCR, Oct '15 https://wwwhome.ewi.utwente.nl/~rijswijkrm/pub/ccr-ecdsa-2015.pdf

[3] Enabling Internet-wide deployment of ECN, PAM '15
http://ecn.ethz.ch/ecn-pam15.pdf

[4] 70% of popular Web sites support ECM, blog post
https://mami-project.eu/index.php/2016/06/13/70-of-popular-web-sites-support-ecn/

[5] Can we run the Internet over UDP, MAPRG presentation
https://www.ietf.org/proceedings/95/slides/slides-95-maprg-3.pdf

---

From: Ca By <[email-masked]>
Date: Wed, 15 Jun 2016 05:17:31 -0700

On Tuesday, June 14, 2016, Brian Trammell <[email-masked]> wrote:

> hi Christian,
>
> > On 14 Jun 2016, at 21:47, Christian Huitema <[email-masked]
> <javascript:;>> wrote:
> >
> > I think we need to dig a bit further there. Botnets could issue
> spud/quic traffic just fine. Granted, it will take them some time to adapt,
> but history shows that they will adapt. So, I don't see how painting a
> fixed set of bits in the packets would help in the long run. Botnets
> engaged in DOS will just set the same bits, be it a protocol number or a
> header field.
>
> Agreed.
>
> > There may be something specific to do about differentiation from UDP
> reflection attacks. I am aware that a lot of these are DNS packets. There
> may be some other UDP services as well.
>
> Reflection attacks tend to follow the amplification factor: more
> amplification, bigger win. NTP (due, I think, to a widely deployed
> configuration that was amplification-friendly) was the champion here; DNS
> with DNSSEC looms as the next one (see the work of Roland van
> Rijswijk-Deij, Anna Sperotto, and Aiko Pras here: [1, 2]).
>
> > But the general characteristic is that such attacks reuse a deployed
> infrastructure to their benefits, and thus have specific markers. The
> infrastructure is not going to change, so the markers will stay. The new
> protocols should certainly try to differentiate from these existing attacks.
>
> This is the rationale behind the selection of the magic number in the SPUD
> prototype, and the presence of a magic number in the requirements derived
> therefrom.
>
>
Magic numbers do no work on existing router. Just like the constraint you
have that we must work with the intall base so we must use udp. The
solution space is "no change"

> I am also intrigued by Tom's earlier suggestion that using IPv6 DO headers
> might have reflection-reducing properties, given the installed base. We
> should look into that more deeply.
>
> None of this does *anything* against direct botnet attacks -- there are
> maybe things you could do in a world with ubiquitous PLUS-speaking
> middleboxes that distinguished a PLUS magic number flood from a bot from
> real PLUS-encapsulated traffic.

Is there anyone from a middle box vendor here? Do you want to hear what
PLUS is telling you? Or will middleboxes keep doing what they want?

I am concerned plus is saying things nobody is listening to.

>
> But relying on both ubiquitous deployment (a decade out in the wildly
> optimistic case) and the kindness of strangers to actually deploy egress
> filtering (which has worked oh so well in the past) isn't what I'd call a
> security plan.
>

> > As many said, user space is just one of the problems. The other is the
> installed base of NATs that won't pass anything else than TCP and UDP. Do
> we have studies indicating whether that changes with IPv6?
>
> No direct numbers, in part because the measurement infrastructures
> available to us give us much smaller population sizes for v6 measurement.
> Anecdotally, taken from analysis of the data in [3, 4, 5], I can say that
> the magnitude of brokenness on v6 in general is lower, but the kinds of
> brokenness you run into is weirder. Slide 9 in [5] is v4 only because we
> still haven't been able to explain anomalies in the MTU curve for UDP on
> IPv6; in both [3] and the detailed analysis linked from [4] we see ECN
> negotiation anomalies five times more often for v6 than v4.
>
> If I had to make up an explanation for that, I'd say that v6 traffic does
> indeed tend to pass through fewer middleboxes, but that the v6
> functionality in those middleboxes is far less well tested. Happy eyeballs
> works so well because it quite effectively hides a multitude of sins.
>
> Cheers,
>
> Brian
>
>
> [1] DNSSEC and its potential for DDoS attacks at IMC '14
> https://wwwhome.ewi.utwente.nl/~rijswijkrm/pub/imc101-vanrijswijk.pdf
>
> [2] Making the case for elliptic curves in DNSSEC in CCR, Oct '15
> https://wwwhome.ewi.utwente.nl/~rijswijkrm/pub/ccr-ecdsa-2015.pdf
>

---

From: Tom Herbert <[email-masked]>
Date: Mon, 20 Jun 2016 10:10:00 -0700

> I am also intrigued by Tom's earlier suggestion that using IPv6 DO headers might have reflection-reducing properties, given the installed base. We should look into that more deeply.
The solution would be to use hop-by-hop options. These would have a
lot of advantages:

- HBH options work with any transport or IP protocol (TCP, SCTP, UDP,
IPsec, GRE, etc.) not just new UDP based transport protocols. This
means adding host to network signaling could be done as an incremental
change to the existing mostly TCP Internet (I think this is a major
benefit).
- For a fragmented packet each fragment can contain hop-by-hop options
but not each one contains transport headers.
- Hop-by-hop options must be first extension header in the packet so a
network device only needs to parse the IPv6 header and one type of EH.
If a middlebox has to access the transport layer header and payload it
would need to be able to parse over other types of extension headers
(routing, DO, etc.)
- HBH can be replicated as desired in the outer headers if packets go
through an IP tunnel.
- IP protocol numbers are unambiguous. If IPv6 next-hdr value is 0 the
next header in a packet contains an extension header as a global
invariant. No magic numbers needed.
- Extension headers are not automatically reflected by any service.
- It is within the protocol to modify the contents of HBH options (as
specified in the definition of an option). There is also no checksum
that needs to be updated unlike the case of UDP payload being
modified.
- The rules for processing HBH options are being relaxed in 2460bis.
Previously, all nodes in a path were required to inspect them. The new
text allows nodes in the path to ignore them and forward packets as
without processing or modifying HBH options.
- RFC3542 defines a sockets API that seems to be supported by the
major OSes. Applications probably are permitted to set arbitrary
options in packets that may affect the network, however allowing
application specific options that are deemed safe can done by
whitelist.
- HBH options obviously only works with IPv6 and is not available in
IPv4. I actually think this is a good thing since supporting
forwarding looking functionality only in IPv6 would promote continued
transition to IPv6 in the Internet and is in alignment with the early
efforts for sunsetting IPv4.

 Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 22 Jun 2016 12:24:31 -0700

See below.

> change to the existing mostly TCP Internet (I think this is a major

Except that they also increase the chance your packets are just dropped
at routers, or that they will take the slow-path.

> would need to be able to parse over other types of extension headers

Except that they tend to want to jump over only a small number of such
options; increasing that number increases the chance the packet is dropped.

> - HBH can be replicated as desired in the outer headers if packets go
HBH options almost never make sense to copy. They're useful in the
context of the original header, not the new one.

You'd have to define the desire to copy your header if that's the
behavior you want, and you'd need a way to know when that didn't happen.

However, this is a really nonsensical idea - the whole point of a tunnel
is to present a virtual LINK - not an IP subpath. Nothing that happens
on a link needs to be visible to IP unless it's already visible at the
link endpoints -- which is where the tunneled packet's HBH options are
already processed.

> - Extension headers are not automatically reflected by any service.
I'll bet money that they are.

> without processing or modifying HBH options.
That also means that legacy devices might not behave as you expect.
> application specific options that are deemed safe can done by

FWIW, ick. RFCs shouldn't be man pages. IPv6 (more specifically,
RFC2460-bis) should include an abstract API that vendors can support,
exactly as RFC793 did.
> efforts for sunsetting IPv4.
+1 on that point.

---

From: Tom Herbert <[email-masked]>
Date: Wed, 22 Jun 2016 13:45:25 -0700

> options; increasing that number increases the chance the packet is dropped.
Per RFC2460 those devices should not be jumping over any options at
all, this is non standard behavior. This is why we need Happy Eyeballs
in order to work around non-compliant devices. We need workarounds for
IPv6 anyway, and we would need them when trying to replace TCP with
UDP based transport protocol or something else. IMHO I don't believe
that the fact that some number of devices don't comply with the
standard justifies creating a whole new standard to accomplish the
same effect while potentially opening the door for new ways to be
noncompliant.

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 22 Jun 2016 14:05:23 -0700

On 6/22/2016 1:45 PM, Tom Herbert wrote:
> all, this is non standard behavior. 

The two high-order bits of all options determine whether they can be
ignored - at which point, they can be "jumped over". E.g., 00 allows
"skip and continue processing".

You haven't mentioned whether you want that behavior, but if you don't
then the option will die at the first non-supporting router.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 22 Jun 2016 14:18:55 -0700

> then the option will die at the first non-supporting router.
Right, I would say that HBH options like this should be 00. Note
though that the wording in RFC2460bis now allows nodes in the path to
ignore HBH. That means that only nodes that are interested in what is
in the options, like devices that would support the proposed
signaling, need to inspect them.

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 22 Jun 2016 14:22:10 -0700

On 6/22/2016 2:18 PM, Tom Herbert wrote:
> Right, I would say that HBH options like this should be 00. 

If that's the case, you can't know when a router processes a datagram
but ignores this option.

(note - TTL/hopcount tricks won't work - you can't know the difference
between a single router that decrements the TTL/hop by 2 and a second
router that ignores this option)

> Note

Ick. That behavior was already supported by the choice of the option
designer. Taking away the potential to create "must handle" options is a
bad thing.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sat, 25 Jun 2016 14:32:54 -0700

On Wed, Jun 15, 2016 at 5:17 AM, Ca By <[email-masked]> wrote:
>> > but history shows that they will adapt. So, I don't see how painting a fixed
>> > set of bits in the packets would help in the long run. Botnets engaged in
>> > DOS will just set the same bits, be it a protocol number or a header field.
>>
>> Agreed.
>>
>> with DNSSEC looms as the next one (see the work of Roland van Rijswijk-Deij,
>> Anna Sperotto, and Aiko Pras here: [1, 2]).
>>
> have that we must work with the intall base so we must use udp. The solution
> space is "no change"
>
>>
> Is there anyone from a middle box vendor here? Do you want to hear what PLUS
> is telling you? Or will middleboxes keep doing what they want?
>
+1, these are very important questions to be answered for PLUS effort.

> I am concerned plus is saying things nobody is listening to.
Unless somebody's listening, there's no reason to believe applications
are going to bother to say anything.

Tom

>> > installed base of NATs that won't pass anything else than TCP and UDP. Do we
>> > have studies indicating whether that changes with IPv6?
>>
>> indeed tend to pass through fewer middleboxes, but that the v6 functionality
>> in those middleboxes is far less well tested. Happy eyeballs works so well
>> because it quite effectively hides a multitude of sins.
>>
>> Cheers,
>>
>> Brian
>>
>>