Subject: Re: [Spud] Details about PLUS BoF

From: Brian Trammell <[email-masked]>
Date: Sun, 3 Jul 2016 15:58:54 +0200

> On 29 Jun 2016, at 20:26, Tom Herbert <[email-masked]> wrote:
> 
> On Wed, Jun 29, 2016 at 10:33 AM, Ted Hardie <[email-masked]> wrote:
>> On Mon, Jun 27, 2016 at 2:28 PM, Tom Herbert <[email-masked]> wrote:
>>> 
>>> On Mon, Jun 27, 2016 at 1:12 PM, Ted Hardie <[email-masked]> wrote:
>>> 
>>>> 
>>>> You appear to be saying that the eliminated implicit path layer simply
>>>> should not be replaced in UDP encapsulations unless evidence comes
>>>> through
>>>> to define what information should be exposed.
>>>> 
>>> Right. Consider that we want to deploy encrypted UDP-based transports
>>> in the near term and that no middlebox supports anything resembling
>>> PLUS. In the simplest model, either a network path forwards UDP
>>> packets without impediment or it doesn't. If it doesn't we will
>>> fallback to TCP and probably alert the user of that at some point. If
>>> PLUS does come on-line then it's worth using only if it provides some
>>> tangible benefit to our traffic or it "fixes" whatever networks are
>>> still blocking UDP.
>>> 
>> 
>> So, having the path forward UDP packets without impediment should be a
>> consistent aim.  The idea behind an explicit path layer is to move from
>> "without impediment" to "with optimization".  If an endpoint is is willing
>> to inform the path of session start and stop for a UDP-based flow, for
>> example, it may get the longer dwell times in NAT bindings that are
>> currently available to flows that reveal that in an implicit path layer.
>> That advantage might translate to avoiding heartbeat traffic, which is quite
>> useful.
>> 
> Ted,
> 
> IMO the idea that the network should be tracking session state like
> that is a red herring because it presumes that the "path" of a flow is
> well defined and has some invariant hops. IP is a packet switched
> network architecture, not circuit switched. There has never been
> either a protocol nor architectural requirement that packets of any
> flow always go through the same intermediate device.

Agreed. This is why, e.g., TCP anycast is such a terrible idea, and is completely undeployable.

Someone should probably point this out to the people deploying it: see e.g. [1], [2]. Cicalese et al had a nice measurement study on this in CoNext last year, too [3].

> In fact, I think
> we are going to see more cases of flows taking alternate paths within
> their lifetime.

I hope we do. Restoring the ability to safely treat IP as a packet-switched architecture is one of the goals of this work (see the SPUD use cases draft, section 6 [4]). As of today, since common wisdom holds that TCP performance *sucks* if you reorder packets, quite a lot of effort goes into reducing reordering within a flow, and it turns out an excellent way to do that is to make sure all the packets in a flow hit all the same queues in the absence of rerouting. How can a transport signal that reordering is tolerable today?

> Consider that any smart phone is now typically
> multi-homed to both a mobile network and a WIFI network. It's pretty
> obvious that we'd like to seamlessly transition from using one network
> to the other without breaking established connections.

This world has existed for a couple of decades now: SCTP (where it deploys) and MPTCP do this already by building a session over multiple transport connections.

> In that world
> I'm not even sure what sort of PLUS state signaling would be
> meaningful (e.g. when transitioning do we need to send a PLUS 'FIN'
> into the old network, and a PLUS 'SYN'  to the new one?).

Simple: map the transport layer signaling for the individual transport connections to the state signaling on PLUS.

> In any case, state signaling to the network seems at best to be a
> hint. Any intermediate devices that tracks tries to track stack must
> realize they could be completely wrong and need to take this fact into
> account.

Absolutely. We spend a fair amount of time making this clear in the requirements document (see requirements 5.7-5.9 [5], though it turns out that the mechanism for 5.7 and 5.8 are probably identical).

> And if the network can't provide any assurances to the host
> given the signaled state, for instance a guarantee that NAT bindings
> are maintained for the duration of the connection, then the only
> recourse for the host has is to fallback to assuming network doesn't
> track state (e.g. still needs to send keepalives to maintain NAT).

This is also covered in the documents (see esp. use case 3). The goal is to reduce the aggregate nonproductive traffic in the network once PLUS-aware middleboxes are deployed, not to make guarantees for any particular flow or path.

> As for the NAT binding issue, I believe that IPv6 is supposed to
> obsolete the need for NAT in the first place.

Heh. Someone should point this out to all the people working on the menagerie of ways to make IPv4-only services accessible on IPv6-only access networks.

Cheers,

Brian

[1] http://blog.catchpoint.com/2015/09/24/tcp-over-ip-anycast/
[2] https://www.nanog.org/meetings/nanog37/presentations/matt.levine.pdf
[3] http://conferences.sigcomm.org/co-next/2015/img/papers/conext15-final100.pdf
[4] https://tools.ietf.org/html/draft-kuehlewind-spud-use-cases-01#section-6
[5] https://tools.ietf.org/html/draft-trammell-spud-req-04#section-5.7 et seq.

---

From: Tom Herbert <[email-masked]>
Date: Sun, 10 Jul 2016 16:35:41 -0500

> I hope we do. Restoring the ability to safely treat IP as a packet-switched architecture is one of the goals of this work (see the SPUD use cases draft, section 6 [4]). As of today, since common wi…
Brian,

I believe that TCP performance *sucks* with OOO packet is mostly a
myth at this point. In the Internet, bad TCP performance is dominated
by packet loss and retransmissions (which I guess leads to OOO but
that's a secondary issue then). In the datacenter where loss is low we
do typically prefer in order delivery to keep variance down. But in
order delivery has never be a requirement there either (except when
people have attempt to use protocols that really require in order like
FCoe, RDMA). There are also results that show all packets could
purposely be sent over different paths in a well engineered with good
effect but would generate OOO
(ttps://engineering.purdue.edu/~ychu/publications/infocom13_pktspray.pdf).
Also, as described in that paper, modern TCP stacks are robust with
some amount of reordering.

A transport can signal that reordering is tolerable by changing the
IPv6 FlowID on every packet. This would be one way to accomplish
Random Packet Spraying without needing special support in network
devices as described in the paper above.

> This world has existed for a couple of decades now: SCTP (where it deploys) and MPTCP do this already by building a session over multiple transport connections.
It's not quite the same, these require that the transport layer or
network understand network topology. That breaks down for instance if
devices are behind a NAT which is change source addresses for existing
connections. As I mentioned previously, NAT state eviction causing
connections to be dropped is one thing we want to fix with UDP based
transport protocols (e.g. by disassociated location).

Tom