Subject: [Spud] SPUD's open/close are unconvincing

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 08 Apr 2015 14:39:16 -0400

Hi folks--

Thanks to those who organized the SPUD BoF at IETF 92.  Despite having
some serious big-picture concerns about SPUD overall, I sympathize with
the general goal of facilitating encrypting more traffic to protect the
confidentiality, integrity and anonymity of Internet communications, so
i appreciate the effort.

I'll write up the big-picture concerns separately, but right now i plan
to focus on some of the concrete details that i find unconvincing about
the idea and the associated proposals.  I'll try to keep each detail
relatively succinct.

This message is about the need for open/close in SPUD, which in the
presentations i saw was frequently the first example brought out to
explain what advantages SPUD can offer to existing on-path equipment.

These signals don't seem to provide any useful advantages to me: Open is
superfluous, and Close is unreliable.

Open is superfluous
-------------------

Given the existing 5-tuple, on-path equipment trivially knows when a
flow starts already: it is a 5-tuple that they haven't seen before.  If
you want to maintain a distinction between a one-side-opened flow and a
replied flow, this is also traceable by looking at the 5-tuple.  It's
not clear that the "open" signal gives on-path equipment any additional
useful information that it didn't have already from the 5-tuple.

Close is unreliable
-------------------

One of the reasons on-path equipment would like a "close" signal is so
that they know when to tear down associations that are no longer needed.
This would reduce memory consumption and make a device more resilient to
DoS attacks that exhaust its connection table.  Without "close", the
on-path equipment has to rely on timers to decide when to tear down the
connection.

Unfortunately, we can't rely on endpoints to send Close.  Legitimate
endpoints crash, run out of power, or have their network connectivity
cut.  So on-path equipment needs to maintain timers anyway if they're
tracking flow state instead of just passing IP traffic statelessly.  And
in a DoS scenario, it's trival for an actively malicious end-point to
send lots of "open" messages without ever sending a "close", so it
doesn't protect against DoS either.

Why should SPUD bother with explicitly signalling open and close?

    --dkg

---

From: Christian Huitema <[email-masked]>
Date: Wed, 8 Apr 2015 19:28:36 +0000

> superfluous, and Close is unreliable.

Did you check the "reset middlebox attack" thread? It was threading much of the same ground. The general notion is that passing session state information in a clear text header allows for a set of denial of service attacks "through the middleboxes." The simplest attack is a fake "stop" causing middleboxes to drop their state and thus terminate the session, even if end-to-end authentication protects the end points against that attack.

> Open is superfluous
> -------------------
> 
> Given the existing 5-tuple, on-path equipment trivially knows when a flow starts
> already: it is a 5-tuple that they haven't seen before.  If you want to maintain a
> distinction between a one-side-opened flow and a replied flow, this is also
> traceable by looking at the 5-tuple.  It's not clear that the "open" signal gives on-
> path equipment any additional useful information that it didn't have already
> from the 5-tuple.

Yes. If it did, we would have to be concerned with route changes, which result in the same five-tuple being routed through a new set of middleboxes. The route change would have to be signaled to the end point, so they repeat the additional "open" information for the benefit of new middleboxes. That looks like a big can of worms.

> Close is unreliable
> -------------------
> 
> One of the reasons on-path equipment would like a "close" signal is so that they
> know when to tear down associations that are no longer needed.
> DoS attacks that exhaust its connection table.  Without "close", the on-path
> equipment has to rely on timers to decide when to tear down the connection.
> 
> Unfortunately, we can't rely on endpoints to send Close.  Legitimate endpoints
> crash, run out of power, or have their network connectivity cut.  So on-path
> equipment needs to maintain timers anyway if they're tracking flow state
> instead of just passing IP traffic statelessly.  And in a DoS scenario, it's trival for
> an actively malicious end-point to send lots of "open" messages without ever
> sending a "close", so it doesn't protect against DoS either.

That, and the injection of fake stop packets by whoever can listen to the path and inject from the side.

> Why should SPUD bother with explicitly signalling open and close?

Agree with you, it would be better if it just did not include these indications.

The initial argument for open/close bit relates to "opening ports in the enterprise firewall." That's probably better handled by an explicit signaling protocol like the Port Control Protocol. In the previous thread, Tiru Reddy pointed to the work in progress to add authentication to PCP, and to the use of PCP to reduce the required frequency of keep alives. That seems more robust than unauthenticated bits in a clear text header.

-- Christian Huitema

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 8 Apr 2015 12:39:21 -0700

Daniel,

a) I think one hope of SPUDs design is to make UDP look enough like
   TCP to persuade FWs (and similar middleboxes) to police / permit it
   as well as TCP flows are policed/permitted.

b) If you don't like how it works, let me generalize the question,
   would like to hear your (and others) answers:

   What is the best possible design we can do to make UDP flows
   be equal or better permissible across WELL BEHAVED FW and similar
   middleboxes ? Please define "WELL BEHAVED"as part of your answer.

c) If i read it correctly, SPUD can 

   a) set up potentially multiple independent pipes across the same 5-tuple.
   b) A single pipe can go across different 5-tuples (multipath/mobility).
   
   These two features should explain why tracking 5-tuple as you
   suggest alone would not be sufficient.

   Btw: I am a fan of b), i don't think a) should be encouraged given
   experience with RTCweb. In that sense, the total number of
   pipes would even be equal or smaller than the number of 5 tuples.

Cheers
    Toerless

P.S.: Cool domain name. Is that indicative of the role you're planning
to take in the discussion ? ;-))) (sorry, i can never resists these questions).

On Wed, Apr 08, 2015 at 02:39:16PM -0400, Daniel Kahn Gillmor wrote:
> _______________________________________________
> Spud mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spud

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 8 Apr 2015 20:53:24 +0000

>   as well as TCP flows are policed/permitted.

Yes.  In talking with enterprise firewall admins, they're comfortable with these properties of TCP:

- they're sure which interface the intent to start an association came in on
- they're pretty sure that subsequent packets match that initial intent
- they're pretty sure that if a box goes down comes back up, or a new box takes the old one's address listening on the same port, that the host will be able to reject traffic left over from an old association
- they know when to clean up state
- they don't feel the need to set as aggressive timeouts as they currently do for UDP, which would cause apps to send lots of extra keep-alives

From a certain perspective, it doesn't necessarily matter if we completely agree with them that today's implicit heuristics are not good enough.  We have evidence that there are enough corporate firewall admins have similar feelings, particularly in places where folks that want to deploy standards-based WebRTC solutions (e.g.), that I believe we need to take their perceptions into account.  The approach that SPUD currently takes is to smell enough like TCP from a policy perspective that we can get corporate firewall admins to allow the traffic by policy.

In other words: these are potentially deployment requirements, not technical requirements.

-- 
Joe Hildebrand

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 08 Apr 2015 14:31:27 -0700

> In other words: these are potentially deployment requirements, not technical requirements.

Joe's analysis highlights an important point - there is no need for SPUD 
to be better than TCP.
If TCP is vulnerable to off-path reset attacks, then it would be ok for 
SPUD to be vulnerable.
We just have to avoid making SPUD *more* vulnerable.

---

From: Brian Trammell <[email-masked]>
Date: Thu, 9 Apr 2015 00:03:40 +0200

hi Daniel, all,

Thanks for your comments; commentcomments per tradition inline...

> On 08 Apr 2015, at 20:39, Daniel Kahn Gillmor <[email-masked]> wrote:
> 
> Hi folks--
> 

I'll defer to Joe's answer on what makes firewall guys happy. :)

I think part of the problem is we're trying to draw a boundary we don't quite understand between the "over" transport (the thing inside SPUD) and the "minimal common" transport that SPUD provides. Which is why we're putting effort into a prototype, on which see my next message (hopefully somewhen tomorrow...)

The explicit signaling of OPEN to devices on path is an accident of the design of TCP. There are devices out there that treat SYNs specially, and the point of signaling OPEN is, in the first order, to allow those devices to do those things they do on SYN. On the other hand, reacting to SYN specially is one of the mechanisms by which these devices make assumptions that can break connectivity, so maybe this is one of those requirements that isn't really a requirement, but rather something common to the over-transport.

Another way to look at OPEN is that it is the inverse of signaling the running state. As a middlebox, seeing packets on a tube for which you didn't see an OPEN or an ACK means:

(1) you rebooted or got a new address
(2) routing is different and you're seeing the middle of a flow
(3) someone is messing with you

It also means that you're missing any ADEC information associated with the tube, and cannot tell whether the tube is valid. It's also not 100% clear you can do anything useful with these.

SPUD's ACK (roughly TCP SYN/ACK) is more interesting. A SYN/ACK in proper response to a SYN means that someone on the other side of the firewall decided a connection could proceed, or more mundanely means there is now actually state on the endpoint so any state the network needs should be there too. For bidirectional transports (i.e., for every transport one should be running over SPUD, since congestion control requires feedback, as does proof of reverse reachability to reduce spoofing) it might be that ACK is sufficient to get us what we need. (In reviewing that paragraph, we probably also need to name it something other than ACK.)

> cut.

Another reason: transports running over SPUD might not even have a signal that maps to CLOSE.

> So on-path equipment needs to maintain timers anyway if they're
> tracking flow state instead of just passing IP traffic statelessly.

Yep. Nobody's saying you can chuck the timers with CLOSE. You can set faster timers once you've seen one, and use state space only for the exceptions. Right now, just looking at a bare UDP datagram, you can't.

It is important to note that if CLOSE is done right, and the active tube ID space remains sufficiently sparse, then only devices on the path or entities cooperating with them -- i.e., those that are in position trivially block or inject traffic -- can fake a CLOSE: you have to be able to fake a valid tube ID.

> And

I don't think we can have devices on path that keep any sort of state without defenses against state exhaustion, and I don't know that it's useful to design those into a signaling encapsulation. You still have to have reverse-reachability, cookies can help, and you probably need some sort of heuristic for rate-limiting. Indeed, these all speak toward making ACK the important OPEN.

Cheers,

Brian

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 08 Apr 2015 18:11:45 -0400

> - they're sure which interface the intent to start an association came in on

This is true of UDP today.

> - they're pretty sure that subsequent packets match that initial intent

This is also true of UDP today; responses to the same 5-tuple (with
source and dest swapped) provide a "pretty sure" match.

> - they're pretty sure that if a box goes down comes back up, or a new
>   box takes the old one's address listening on the same port, that the
>   host will be able to reject traffic left over from an old association

Surely this is a matter for endpoints and not for middleboxes, and
dependent on the protocol implemented by the listener?  For stateless
listeners on an endpoint, they shouldn't mind seeing traffic from an old
"association" because no state is associated with it.  For stateful
listeners, they can simply discard traffic that doesn't align with their
internal state.

> - they know when to clean up state

This is untrue of either TCP or UDP today in an absolute sense.  In both
cases, they must each rely on timeouts because either peer can fall
off-net for many different reasons that are beyond anyone's control.

> - they don't feel the need to set as aggressive timeouts as they
> currently do for UDP, which would cause apps to send lots of extra
> keep-alives

So they're willing to permit longer timeouts for raw TCP SYNs than they
do for UDP?  That sounds like a trivial state exhaustion attack waiting
to happen.  If they treat an unacknowledged TCP SYN with an aggressive
timeout, but then lengthen the timeout after seeing a SYN/ACK, then
surely they can just treat an unreplied UDP 5-tuple with the aggressive
timeout, and lengthen it when they see a response?  This doesn't require
any on-wire format change.

> From a certain perspective, it doesn't necessarily matter if we
> completely agree with them that today's implicit heuristics are not
> good enough.  We have evidence that there are enough corporate
> firewall admins have similar feelings, particularly in places where
> folks that want to deploy standards-based WebRTC solutions (e.g.),
> that I believe we need to take their perceptions into account.  The
> approach that SPUD currently takes is to smell enough like TCP from a
> policy perspective that we can get corporate firewall admins to allow
> the traffic by policy.
>

This argument is also unconvincing to me.  It sounds like you're saying
that we cannot rely on this group of administrators to make reasonable
decisions about how to operate their equipment on the basis of the data
that they currently have, so we're going to do some unnecessary things
to the bits on the wire that will convince them to make reasonable
decisions on the basis of some other unnecessary data.

Why should i believe that this will be the eventual outcome?  A group of
administrators has already provided evidence that they're unwilling to
consider reasonable approaches with data format A; why would equivalent
data format B suggest they would do anything different?

      --dkg

---

From: Christian Huitema <[email-masked]>
Date: Wed, 8 Apr 2015 22:18:34 +0000

> SPUD's ACK (roughly TCP SYN/ACK) is more interesting. A SYN/ACK in proper
> response to a SYN means that someone on the other side of the firewall decided
> a connection could proceed, or more mundanely means there is now actually
> state on the endpoint so any state the network needs should be there too. For
> bidirectional transports (i.e., for every transport one should be running over
> SPUD, since congestion control requires feedback, as does proof of reverse
> reachability to reduce spoofing) it might be that ACK is sufficient to get us what
> we need. (In reviewing that paragraph, we probably also need to name it
> something other than ACK.)

The mapping to actual transport semantics is not entirely clear. Many transport protocols implement some kind of protection against SYN Flooding. One popular such protection is a cookie challenge. So we get a connect/SYN request, a challenge response, a confirmation of the challenge, and only then the actual accept by the server. Which of the server's message map to ACK? The challenge, or the actual confirmation?

> ...
> Yep. Nobody's saying you can chuck the timers with CLOSE. You can set faster
> timers once you've seen one, and use state space only for the exceptions. Right
> now, just looking at a bare UDP datagram, you can't.

Even that is not obvious. What kind of value do you give to the "fast timer?" What if the mobile client is asleep when the fake CLOSE arrives, and cannot invalidate it quickly by sending more traffic? 

> It is important to note that if CLOSE is done right, and the active tube ID space
> remains sufficiently sparse, then only devices on the path or entities cooperating
> with them -- i.e., those that are in position trivially block or inject traffic -- can
> fake a CLOSE: you have to be able to fake a valid tube ID.

Please take a look at the Quantum attacks, which combine passive listening on the path with injection from the side. There is a difference between "block traffic" and "inject traffic." Only the forwarding agents can effectively block traffic. Passive listeners can inject traffic without cooperating with the forwarding agents.

-- Christian Huitema

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 08 Apr 2015 18:21:21 -0400

Hi Toerless--

>    middleboxes ? Please define "WELL BEHAVED"as part of your answer.

As i wrote to Joe, i'm not convinced that this is an answerable question
considering that no one has provided a technical argument yet for why
the enterprise firewall operators can't already do what we're talking
about here.

>    pipes would even be equal or smaller than the number of 5 tuples.

This is an interesting point, thanks.  I'd be curious to hear more
details about the RTCweb experience that have convinced you that (a)
should not be encouraged.

The trouble with (b), of course, is that in the multipath or mobility
case, the tube will continue its flow over different network segments,
which violates the goals people have described for open and close.

As i move to a new path, i'm deliberately not sending a Close message
(because i want to keep the tube open, right?) -- so what do the
middleboxes do?

And after i've moved to a new network and want to continue the same
tube, surely i won't indicate that the flow is opening (it's already
open).  As a result, the equipment on the new path won't see the Open
message -- should they discard it now?

These arrangements seem in conflict to me.

Regards,

        --dkg

> to take in the discussion ? ;-))) (sorry, i can never resists these questions).

:)

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 8 Apr 2015 16:46:06 -0700

Hi all,

One higher-order point: This conversation seems to refer to middlebox
admins as outsiders and represents them as an ossified whole. If this new
architecture is to be a end-middle-end architecture, the middle needs to be
represented in a technical debate about the value of new protocol
machinery. I would sincerely love to see that happen. I think there's a
good bit of serious architectural work that can be done here, but this
requires wrangling with hard questions about trust, delegation, and
placement of network functions. I fear that discussions on what exact bits
to expose so that middlebox admins don't turn into fire-breathing monsters
misses the larger architectural questions.

It's not clear to me that we should negotiate with middlebox admins on what
bits they want -- I'd like some degree of engagement on exactly why they
want it. I'm not keen to see the middlebox admin's wishlist of what headers
they'd like to see. I'm substantially more interested in understanding the
network functions they are looking for. Chances are these are implementable
without the information they think they need.  If not, let's make that case
-- one network function at a time -- do a cost-benefit analysis for the
good of the Internet, and decide whether to expose any bits.

dkg has quite nicely said many of the things I've wanted to say below; I'll
re-echo these and I'll add a couple of comments inline.

> This is true of UDP today.

+1.

> source and dest swapped) provide a "pretty sure" match.

+1.

> listeners, they can simply discard traffic that doesn't align with their

+1.

> off-net for many different reasons that are beyond anyone's control

+1.

> any on-wire format change.

+1. I'll add that it wouldn't be surprising for clients to disappear or for
servers to drop TCP connections without sending a FIN or an RST, for a
variety of reasons. (As it turns out, this can be energy-saving for mobile
devices that don't have to wake the radio up only to receive and process a
FIN! See: http://conferences.sigcomm.org/co-next/2013/program/p211.pdf.)

> > good enough.  We have evidence that there are enough corporate

It obviously does matter if we disagree with them. We're considering a
rework of the Internet's architecture -- I don't think this is what you're
saying, but at what point did we decide to take their needs as absolute?
"They" need to make technical arguments about their needs, if more
information is to be exposed for their use.

> > policy perspective that we can get corporate firewall admins to allow

Doesn't it seem silly that we will be engaging with corporate firewall
admins, but only to learn that *they think* our traffic needs to look like
TCP for them to be comfortable with it? Their perceptions of TCP are most
likely ossified, and quite possibly wrong -- what do we do then? (Perhaps
we should run the "Underhanded TCP Contest" -- make crazy stuff happen but
look like TCP on the wire :-)) TCP FastOpen and Minion both challenge prior
notions of what was (incorrectly but commonly) considered TCP behavior. I
would be quite disappointed if we were to enshrine this ossification in a
wg, and for non-TCP protocols too! This wouldn't be evolution, this would
be ossification on steroids.

> data format B suggest they would do anything different?

+1.

- jana

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 8 Apr 2015 18:22:29 -0700

> the enterprise firewall operators can't already do what we're talking

Sorry, i need to give a multi-tiered answer:

A) Lowest layer what i said initiall - mobility, load-splitting,
multiplexing.

B) Next layer: Yes, the FW admin could do for UDP what she does for TCP,
but in todays ecosystem she will not: 

   UDP has
     I) bad karma due to history of bad UDP apps in the past
    II) More filtering on UDP than equivalent TCP traffic because
        there is so much business critical TCP crap
   III) not a lot of business critical apps riding on UDP
    IV) A historic architecture view of "doing reliable transport
        on top of UDP is a bad, undesirable workaround"

   Seeded by I, II-IV form a vicious cycle. 

C) Decades long a ridiculously slow and cumbersome evolution
   of transport layer functionality. We should have done the bloody
   AQM work a decade or more ago.

   Reason IMHO is that transport layer is in the kernel, resulting
   in a totally broken agility model for transport layer
   compared to middleware/app-layer (which created . 

Alltogether this means to me the goal of the SPUD exercise
architecturally is: 

Redefine the Internet architecture to say:

 -> Transport Layer needs to be split up into sub-layers:

    1) application multiplexing layer - UDP, (running in OS) unchanged
    2) common, middlebox friendly connection/packet signaling layer
    3) transport connection QoE  layer
       (reliability (FEC,ARQ), rate-control, measurement, jitter control,...)
       Aka: short term just existing Internet transport protocols
       (RTP, TCP, SCTP,...) but if one would design a new transport
       protocol, one could not redo whats already in 2). 
    
    2) and 3) can run in app/middleware (per app) and/or in kernel.

SPUD is the first attempt at 2).

If we wouldn't do 2), but just 3) over 1), we will make the
chicken & egg problem of 1) even worse, because we're just asking
FWs to now inspect multiple reliable transports inside UDP instead
of (with 2) giving them one common single layer to inspect. And
of course we also do not solve C), because there is just a bunch
of things like mobility, load-splitting, per-packet marking and so
on that we know we cold get from the network to benefit apps, but
without moving it into user-land, it's not going to go anywhere fast.

> should not be encouraged.

Yeah, let me reword, first time around it wasn't precise:

If an RTCweb session requires 10 5-tuple flows, it may take
10 times as long to set it up, and you may end up with 1/10'th maximum
number of parallel sessions due to the number of NAT/FW 5-tuple pinholes
you need to build.  Because we want NAT/FW to be SPUD tube aware,
SPUD SHOULD NOT go out and claim: 

"you can replace 10 5-tuple UDP flows with 1 UDP flow with 10 SPUD tubes,
 and your NAT/FW pinhole problem goes away". 

Aka: An RTCweb app worried about pinhole setup should opt for one
Tube, but that in itself would already give it a lot of other benefits
mobility, load-splitting, per-packet-marking,....

Btw: IMHO performance of NAT/FW pinhole setup is not a limiting factor
if you just buy the right NAT/FW product, but i do by now don't
think anymore that all packets of a transport level session should
have the same QoS.

SPUD can solve this as well with per-packet markings, and
once you have that, the only reason why you would want multiple
tubes for an app-session IMHO is if you explicitly want to give 
FWs the ability to disect your traffic (eg: drop the video portion
but permit audio).

> (because i want to keep the tube open, right?) -- so what do the

I think the first order of business was not to be worse than TCP,
but not yet trying to finalize what the most widely agreeable improvements
beyond that are.

> These arrangements seem in conflict to me.

I can think of ways to improve the signaling. The main
issue is the IETFs history of onpath signaling and the resulting
resistance forces in the IETF against touching that subject.

But consider the opportunity that redunant or cluster of
NAT/FW will share SPUD tunnel-IDs amongst them to support
the mobility case. No new standardized signaling required.

Cheers
    Toerless

---

From: "Toerless Eckert (eckert)" <[email-masked]>
Date: Wed, 8 Apr 2015 20:42:36 -0700

Jana:

> want it.

IMHO, and stealing a sentence from DanW: FW/Middlebox admins
are charged to stop bad bits to come in (virus), stop good bits to go out
(secrets) and otherwise not impact the business. Thats Mission Impossible,
but there are a bunch of aspects where SPUD can help:

- Filtering based on combination of port-numbers + IP addresses +
  roles of communicating parties (initiator, responder).  This assumes
  you trust at least one side (because of IP) to assign port-numbers
  according to well known or otherwise intended use. What protocols like
  SPUD could help here is to substitute the limited space of 16-bit port
  numbers with better application-intent metadata. It could also decouple
  roles from sequencing of packets. Today, coming from TCP, initiator == 
  client, responder == client, initator sends first. Now imagine you would
  want in IoT servers polling clients for example. By explicitly including
  role as a signaling element (client/server) you can do this, and e
  voila: UDP+SPUD becomes more flexible and FW friendly than TCP.

- Filtering of "malformed"/"unknown" protocol packets. 
  Today, for UDP everything is in some higher layer payload. So if we
  move an equivalent level of connection management as TCP has it into
  a standardized over-UDP level protocol, and will give FW operators the
  same degree of protection as when using TCP.

- Filtering based on signalled cryptographic authenticators (eg:
  cert + challenge/reply snooping/filtering). Can't quite remember
  how much of that was proposed in SPUD, but certainly a worthwhile
  area for real security. 

Just some thoughts. There is of course a lot more.

Cheers
    Toerless

> misses the larger architectural questions.

I'm not keen to see the middlebox admin's wishlist of what headers

---

From: Tom Herbert <[email-masked]>
Date: Wed, 8 Apr 2015 20:46:24 -0700

> without moving it into user-land, it's not going to go anywhere fast.
I think the kernel/user-land argument is a red herring. The problem is
that middleboxes routinely participate in transport layer protocols
which was never architected-- transport layer protocols are inherently
end-to-end protocols.  It's relatively easy to change client and
server sides to accommodate new transport functionality, but pretty
much impossible for us to change all the possible middleboxes in a
path in a timely fashion. Just one middlebox in the path that decides
to drop our packet because it doesn't understand our new option, or
doesn't like our new flags can spoil everything-- it is really
difficult to work around interoperabilities like this.  So, yes, the
net effect of this is that we have become very conservative with
transport layer changes, and when we do make changes at the transport
layer we often have to masquerade these in something that is
considered generally palatable to middleboxes. This is why we intend
to run transport protocols over UDP in the first place, and has even
motivated brazen attempts to overload TCP protocol number with other
things (e.g. STT). SPUD is a good opportunity to generalize and
standardize middlebox/transport protocol interactions, but if the its
benefits are dependent on middleboxes being updated that not is going
to go anywhere fast either!

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 8 Apr 2015 21:15:07 -0700

On Wed, Apr 08, 2015 at 08:46:24PM -0700, Tom Herbert wrote:
> I think the kernel/user-land argument is a red herring.

Elaborate please. To me its probably the biggest motivator for 
SPUD. Otherwise we could design all we need into IP and TCP.

> The problem is
> end-to-end protocols.

Thats 1970'th IETF katechism. I would call middleboxes a big contributor
to todays success of Internet technologies.

> It's relatively easy to change client and
> server sides to accommodate new transport functionality,

Which is why we have OS-level TCP functions that lag state of the art
on average by a decade.

> but pretty much impossible for us to change all the possible middleboxes
> in a path in a timely fashion.  Just one middlebox in the path that decides
> difficult to work around interoperabilities like this.

Everybody and their dog concerned about this problem is already
automatically falling back to IP over SSL on port 443. We simply need to
do the best to evolve away from that. Thats why it is important to offer
benefits to those middleboxes.

> So, yes, the

Any firewall can start out processing SPUD just as UDP and once
FW firmware improves it will simply improve from there. Nobody
says firewalls MUST update, we just say that we want to provide
at least the same toolset for relevant inspection firewalls as what TCP gives
them. 

Cheers
    Toerless

---

From: Tom Herbert <[email-masked]>
Date: Wed, 8 Apr 2015 22:20:48 -0700

> to todays success of Internet technologies.
Maybe, but the consequences of diverging from the end-to-end model is
reality we still need to deal with. When we deploy transport protocols
at scale on the Internet we are still bound in a large part by the
least common denominator network devices out there. There are deployed
devices that choke on our attempts to extend transports. We need to be
concerned about these, we simply can't completely break connectivity
for 1% of the world in order to give everyone else a 5% performance
improvement.

> Which is why we have OS-level TCP functions that lag state of the art
I can't comment on other OSes, but I will say that the rate of change
to TCP and transport layer protocols in Linux and high and
significantly outpaces IETF standards (e.g. IW=10, lowering initRTO,
CC changes). These arguments seem to have more to do with the time for
deployment. This is not a problem with any reasonably modern OS, but
people are still concerned with getting changes into legacy
deployments (Windows is often cited). But, if you have specific OS
features that you think are missing I'd be happy discuss that
off-thread.

> benefits to those middleboxes.
This a general problem with modifying transport protocols. For
instance, it's potential problems with middleboxes that may wreak
havoc with attempts to increase the TCP option space (EDO).

> at least the same toolset for relevant inspection firewalls as what TCP gives

It's pretty clear that for the foreseeable future were pretty much
stuck with using either TCP or UDP on the Internet (barring that
something like SCTP would ever get traction). Since we can't do
reasonably stateful firewalls or stateful NAT with UDP that is a
fundamental problem which constrains us to use TCP. There are needs
for new transport protocols, so encapsulating them in UDP seems to be
pragmatic approach, but we really need the ability for middlesboxes to
track state for such things-- this is what I hope SPUD provides.

Tom

---

From: Eliot Lear <[email-masked]>
Date: Thu, 09 Apr 2015 08:22:17 +0200

Daniel,

> Open is superfluous
Open provides an indication that the flow is not a continuation, that
new state should be instantiated, that any old state of the 5-tuple
should be discarded.  Opens may flow in either direction, but may be
authorized for only some uses.
> Close is unreliable

This is by far the exception and not the rule.  Most state is torn down.

> tracking flow state instead of just passing IP traffic statelessly.

This misses the point.  If there is an implicit understanding of the
behavior of the endpoints, firewalls needn't keep state.  It is
sufficient in many cases to know that SYNs should be blocked whereas
SYN|ACK should be passed.  No state retained on the firewall.  There is
no equivalent in UDP without going up the stack.

Moreover, if a firewall is tracking state, it cannot tell if an incoming
packet is a continuation of a <flow/spud/tube> or an initiation if it is
*not* using timers.  This makes both open and close (as well as perhaps
some other gunk) very useful.

> doesn't protect against DoS either.

In fact, this is precisely the nature of a SYN attack.  Firewalls
regularly detect and block them in TCP.  UDP requires much more effort
and understanding of the higher layers.

Eliot

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 9 Apr 2015 09:09:17 -0400

> SPUD. Otherwise we could design all we need into IP and TCP.

TCP should probably not happen in the kernel either. Nor should
printer drivers be in the kernel or anything that does not require the
intermediation of the security monitor.

Looking at the shoot-yourself-in-the-foot opportunities in the IPv6
encoding, I am not exactly anxious to put all those untrusted code
paths in a position where they can root the machine.

One of the main reasons the current generation of O/S are chronically
insecure is that 90% of the stuff that is inside the security
perimeter has no business being there.

At this point TCP is water under the bridge. But that does not mean we
are obliged to remake the mistake.

When TCP was designed, the mantra was 'everything is a stream'. That
was the right abstraction for Telnet and FTP and Mail. It is probably
not the right abstraction for real time web where an unreliable
sequence of chunks seems a better fit.

---

From: Roland Bless <[email-masked]>
Date: Thu, 09 Apr 2015 15:18:39 +0200

Hi Brian,

On 09.04.2015 at 00:03 Brian Trammell wrote:

> SPUD's ACK (roughly TCP SYN/ACK) is more interesting. A SYN/ACK in 
> proper response to a SYN means that someone on the other side of
> the firewall decided a connection could proceed, or more mundanely
> means there is now actually state on the endpoint so any state the
> network needs should be there too. For bidirectional transports
> (i.e., for

Interesting thought, but only for the very simple TCP model the
endpoint already created state and is thus vulnerable to TCP SYN flood
attacks. So a responding endpoint is probably not a sufficient
indication...see also below.

> every transport one should be running over SPUD, since congestion 
> control requires feedback, as does proof of reverse reachability
> to reduce spoofing) it might be that ACK is sufficient to get us
> what we need. (In reviewing that paragraph, we probably also need
> to name it something other than ACK.)

SYN Flooding would also result in endpoints responding, i.e.,
the SYN/ACK is IMHO not a sufficient indication for a proper
conversation/state setup as the third way (ACK) would probably be.
Moreover, purely unidirectional data flows
(e.g., very sparse measurement data) may have no such prior handshake,
or multicast flows may have no ACKs.

> some sort of heuristic for rate-limiting. Indeed, these all speak 
> toward making ACK the important OPEN.

Maybe, but that is still not sufficient IMHO.

Regards,
 Roland

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 9 Apr 2015 06:55:09 -0700

> intermediation of the security monitor.

Right. As an OS person i would love for someone to implement a
"raw transport socket" option for unprivileged processes. Implement
in linux, go to POSIX, spend a decade trying to get it proliferated across
OSs.

Alas, as a network person, i think this would be an exercise in futility
because the only real benefit would be to create connections between
a legacy kernel TCP stack and a new userland TCP stack and those
connections would likely be mostly have little benefits over a simple
old kernel to old kernel TCP stack:

If you want new functionalities, most of the time, both sides need to support
these, the fastest way to get both sides to support them is to both
run them in userland over UDP and once people get their minds around
the fact that this is good and not just a workaround, the interest in
"native TCP" for new improved transport functions should recede.

> sequence of chunks seems a better fit.

What's missing from SCTP ? 

Cheers
    Toerless

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 9 Apr 2015 10:09:32 -0400

> in linux, go to POSIX, spend a decade trying to get it proliferated across

Yep

> old kernel to old kernel TCP stack:

As I said, water under the bridge. We are now constrained by what
legacy implementations provide.

> "native TCP" for new improved transport functions should recede.

The current trend is actually in the opposite direction. On Windows
the HTTP server is an O/S resource and user mode processes bind to
specific URL paths. This allows different Web services to be
implemented in completely separate processes.

> What's missing from SCTP ?

Quite, do we have the opportunity for a quick fix here? If SCTP has
already done all the necessary design work, can we just bolt that on
top of UDP in place of IP and declare a quick victory?

Worth a look to see...

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 9 Apr 2015 07:21:29 -0700

> implemented in completely separate processes.

But sharing the same transport stack ;-)

> top of UDP in place of IP and declare a quick victory?

You should talk to the SCTP folks or read the specs. I am not
on top of the details but i think it does a lot of what you ask.
For me, userland eg: SCTP would be a pre-requisite to make a solution like
SPUD successful, not an argument to not do SPOD. See my mail where i
was trying to explain the three layers of the transport stack
as i would like to see it.

Cheers
    Toerless

---

From: Brian Trammell <[email-masked]>
Date: Thu, 9 Apr 2015 16:27:07 +0200

> On 09 Apr 2015, at 16:09, Phillip Hallam-Baker <[email-masked]> wrote:
> 

RFC 6951, though it uses a single well-known port for all encapsulated SCTP so is less suitable for userland implementations. You can stick DTLS on top of SCTP (RFC 6083) if you're okay exposing the transport headers, or SCTP on top of DTLS as in WebRTC data channels if not.

Cheers,

Brian

---

From: Eliot Lear <[email-masked]>
Date: Thu, 09 Apr 2015 16:29:33 +0200

> RFC 6951, though it uses a single well-known port for all encapsulated SCTP so is less suitable for userland implementations. You can stick DTLS on top of SCTP (RFC 6083) if you're okay exposing th…

Meh.  If inetd could do it for TCP services...

Eliot

---

From: Tom Herbert <[email-masked]>
Date: Thu, 9 Apr 2015 08:42:07 -0700

> in linux, go to POSIX, spend a decade trying to get it proliferated across
Good luck on finding an OS vendor willing to give random applications
unfettered access to the network. Anyway, you can already do this by
just wrapping whatever you want in UDP-- TCP, SCTP, RDMA, whatever.

Tom

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 9 Apr 2015 09:21:32 -0700

> just wrapping whatever you want in UDP-- TCP, SCTP, RDMA, whatever.

I didn't say "raw ip socket", "pcap socket" or the like. "raw transport
socket"to me means something that behaves pretty much like a UDP
socket only that you can select the IP layer protocol from a list
of eg: OS level configured options. That list could include all the
transport protocols that share the common method of 4-byte demultiplexing
header with Sport/Dport. Eg: TCP, SCTP, and a few others. So the kernel
would still do the demultiplexing bas on either (*,Dport) for sockets
in listening state and (Sport,Dport) for sockets in connected state.

But fully agree with your conclusion. This would be design wise nice and
clean, but has no business opportunity to it to make it worth our while.
Just use UDP and declare it to be the mandatory future OS-level demultiplexer
layer for any future transport (+ the discussion that we likely want to
have that SPUD like layer for common flow-setup).

Cheers
    Toerless

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 09 Apr 2015 09:59:29 -0700

> the enterprise firewall operators can't already do what we're talking
With what information? Are you talking about separate switch
vendors independently implementing a SPUD-like solution, or
are you claiming they already have enough information to perform
this with no assist from the endpoints?

I believe that endpoint assist in the form of declarative information
is necessary to enable optimum network response. Standardizing
that information will make it something vendors and endpoint
stack developers will support. Without standardization you have
a chicken and egg problem that will never be resolved.

The key questions are:
* Are the hints sufficient to provide optimizations that more than
    reward the cost of sending the hints?
* Are we avoiding creating some form of DOS-vector, which would
    result in everyone turning off this feature in the field?

I suspect there are some refinements that may be needed to
avoid enhancing DOS attacks. But we need to set the goal
realistically as avoiding *enhanced* attacks, and not rejecting
SPUD for being vulnerable to the same attacks that would have
worked against TCP.

---

From: Tom Herbert <[email-masked]>
Date: Thu, 9 Apr 2015 10:22:59 -0700

> perimeter has no business being there.
The major Internet security problem now is in embedded systems which
are not maintained (which notably includes middleboxes  like home
routers) (https://www.schneier.com/essays/archives/2014/01/the_internet_of_thin.html).
This is not a OS, userspace, or firmware issue, or even protocol a
issue-- but this is an issue with the software deployment model of a
wide array of products. I don't know how SPUD will be able to help
solve this, but it should at least not make things less secure.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 9 Apr 2015 10:23:39 -0700

+1 on everything you said, but (*grin*) i think we should
layer the analysis:

The base layer is analysing how a "session setup/teardown" signaling
on top of UDP with an arbitrary transport protocol encapsulated
inside of it compares to eg: Native TCP wrt. security and
policy for end-to-end operations and FW/middlebox operations.

The only bar to pass here is that it should not be significantly worse
than TCP.

Then the next level is to add the possible enhancements (or
others) factored into the SPUD proposals and see how each of them
fares. Some of them would have to be compared with other existing
approaches. Eg: if the tube ID is an alernative or enhancement to
MPTCP, then one would have to compare with that.

On the base layer, the main downside i can come up with is that the
filtering of invalid protocol packets or sequence of packets is something
that ideally should become more easy, because the FW would only have
to do it once for SPUD instead of every different transport separately,
but alas that is not true because an attacker can still modify the
encapsulated transport proto signaling elements, so a FW that would
like to provide that state-machinery/packet-header filtering for
transport protocols would need to still understand the whole SPUD+Transport
stack. Just off the top of my head, the best way out would be to
move towards UDP+SPUD+dTLS+reliable-transport, because that would protect
that arbitrary transport protocol headers from attacks and would leave
FWs with only SPUD and dTLS to worry about.

Cheers
    Toerless

---

From: Eliot Lear <[email-masked]>
Date: Thu, 09 Apr 2015 19:36:48 +0200

Hi,

> routers) (https://www.schneier.com/essays/archives/2014/01/the_internet_of_thin.html).

While it's true home routers get 0wned, they are actually not the
biggest problem.  The biggest problem will be the orders of magnitude
more boxes that are behind those boxes (for those keeping score, home
routers= O(1), and a house full might very quickly rise to O(100).  Of
those, maybe O(10) will ever see an update, and those will be general
purpose computing devices, and they will only get an update under the
best of circumstances.  This wouldn't normally be germane to SPUD, but
the "I hate middleboxes" mentality only gets you so far.
> solve this, but it should at least not make things less secure.

It is all of the above, but the fact is that it is also a UI issue, a
trust problem, and an economics problem.

Eliot

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 9 Apr 2015 10:46:03 -0700

Glad you didn't ask how SPUD can solve world peace, because i am
still researching that aspect.

There is a lot of work going on wit security models of device deployment,
you may want to look at 6LO, 6TISCH, ACE, DICE, ANIMA and there is a
lot more outside the IETF. I would see that work as prerequisites
to move forward.

IN general, i think there is one good and sane device operations method,
in IoT which is to NOT upgrade the OS after initial device deployment,
but just upgrade/modify applications. And with SPUD, that means you
now can upgrade/fix/improve transport layer.

Just think of the problem of network based self-upgrade of OS SW in a
reliable fashion with limited Flash/NAND storage or the operational
issues in IoT for a PXE/AMT Sw upgrade workflow. Lots of challenges with that.

Just strip down the OS to the bare minimum for operations with a strong
security perimeter, do the rest at the app-level.

Cheers
    Toerless

---

From: Tom Herbert <[email-masked]>
Date: Thu, 9 Apr 2015 11:40:20 -0700

> now can upgrade/fix/improve transport layer.
Well, there's already over a billion smartphones in the world which
are already regularly updating their OS and this hasn't yet led to
mass anarchy. You can certainly make avoiding OS updates and avoiding
OS in general a design point for your products, but this is not a
relevant design point or requirement for a networking protocol. As I
said, it's a red herring wrt SPUD.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 9 Apr 2015 11:51:21 -0700

If all IoT devices could hace a touchscreen and personal operator
bringing them to places for failed upgrade help and 70% of them
would be 48 monthly throwaways that never get upgraded anyhow,
i would certainly want to clone the smartphone operational model
to those IoT devices. 

Just look at the uproar generated recently for security attacks
against BIOSs. You need clear security perimeters, and the
question is why a reliable transport stack for an app would
need to be in a more secure perimeter than the app itself.

-- 
---
Toerless Eckert, [email-masked]

---

From: Yoav Nir <[email-masked]>
Date: Thu, 9 Apr 2015 23:36:24 +0300

> On Apr 9, 2015, at 9:40 PM, Tom Herbert <[email-masked]> wrote:
> 

There are also over a billion smartphones stuck in an old version of the OS. Android 2.6 is very popular as is the latest iOS version your phone supports (lots of iPhone 4 out there, and they’re not getting iOS 8).

And don’t get me started on desktop operating systems. By NetApp’s numbers just under 17% of desktop and laptop systems are running Windows XP. The majority are running the already outdated Windows 7.

And those phones and desktops are things people are always looking at. Lots of people do upgrade the OS on their phones and computers when given a choice. How many people are going to want to upgrade the OS on their refrigerator, car fuel-injection system, air conditioner, or industrial sensor? Those things are deep into “if it works - don’t fix it” territory.

Yoav

---

From: Jana Iyengar <[email-masked]>
Date: Fri, 10 Apr 2015 15:20:46 -0700

Eliot,

A few responses:

Open provides an indication that the flow is not a continuation, that new
> state should be instantiated, that any old state of the 5-tuple should be
> discarded.  Opens may flow in either direction, but may be authorized for
> only some uses.
>

If used like this, it creates another DoS vector where anyone can trivially
kick someone else's connection off.

>  Close is unreliable

AFAIK, TCP connections are commonly torn down with an RST (not a FIN), and
not infrequently with silence. I would love to know what the truth is -- is
there any data that I can look at to prove myself wrong?

> behavior of the endpoints, firewalls needn't keep state.  It is sufficient
> in many cases to know that SYNs should be blocked whereas SYN|ACK should be
> passed.  No state retained on the firewall.  There is no equivalent in UDP
> without going up the stack.
>

Why does it need to tell the difference? I'm not sure I see a use case at
the middlebox for using this information.

> regularly detect and block them in TCP.  UDP requires much more effort and
> understanding of the higher layers.
>

Why would a DoS attacker set the OPEN bit in SPUD if it will thwart their
attack? That would be silly...

---

From: Jana Iyengar <[email-masked]>
Date: Fri, 10 Apr 2015 15:24:10 -0700

This thread seems to gone far afield. To the original question, I still
don't see clear value in SPUD open/close bits, but it is clear to me how
these can both turn into trivial DoS vectors. Specifically, as I see it
even now,
- An OPEN bit carries redundant information.
- An ACK to the OPEN is a thing apps are likely to do, and we can expect
this to be normal behavior. But we don't yet know why a middlebox may need
this information to be explicitly present on the wire, or what it may do
with it.
- A CLOSE bit is unreliable from the middlebox's point of view, since apps
may not send it, and unauthenticated CLOSE bits are DoS vectors.

If these bits are not authenticated, they are all trivially spoof-able, and
they will be exploited for various sorts of DoS attacks.

On Thu, Apr 9, 2015 at 1:36 PM, Yoav Nir <[email-masked]> wrote:

> There are also over a billion smartphones stuck in an old version of the
> OS. Android 2.6 is very popular as is the latest iOS version your phone
> supports (lots of iPhone 4 out there, and they’re not getting iOS 8).
>
> And don’t get me started on desktop operating systems. By NetApp’s numbers
> just under 17% of desktop and laptop systems are running Windows XP. The
> majority are running the already outdated Windows 7.
>
> And those phones and desktops are things people are always looking at.
> Lots of people do upgrade the OS on their phones and computers when given a
> choice. How many people are going to want to upgrade the OS on their
> refrigerator, car fuel-injection system, air conditioner, or industrial
> sensor? Those things are deep into “if it works - don’t fix it” territory.
>
> Yoav

---

From: Eliot Lear <[email-masked]>
Date: Sat, 11 Apr 2015 09:25:15 +0200

On 4/11/15 12:20 AM, Jana Iyengar wrote:
> Eliot,
>
> A few responses:
>
>     Open provides an indication that the flow is not a continuation,
>     that new state should be instantiated, that any old state of the
>     5-tuple should be discarded.  Opens may flow in either direction,
>     but may be authorized for only some uses.
>
>
> If used like this, it creates another DoS vector where anyone can
> trivially kick someone else's connection off.

Huge vast numbers of devices simply look to see if ACK is set, and if
not, reject most connections, except for a few.  The reason SYN is
needed is so that the firewall simply knows that the host will reject
communications that begin with something else.
>  
> AFAIK, TCP connections are commonly torn down with an RST (not a FIN),
> and not infrequently with silence. I would love to know what the truth
> is -- is there any data that I can look at to prove myself wrong?

I don't know.  But whether one uses RST or FIN is immaterial.  The
concept is roughly similar in terms of signaling state, except that RST
allows for quicker removal of state.
> Why would a DoS attacker set the OPEN bit in SPUD if it will thwart
> their attack? That would be silly...
>  

If it knows that the end point will simply discard the packet without
the OPEN having been seen, it will set it.

Eliot