Subject: Re: [tsvwg] Comments on draft-pauly-tsvwg-tcp-encapsulation

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 5 Oct 2018 14:19:47 +0200

Hi Tom,

see below.

On 25.09.2018 20:10, Tom Herbert wrote:
> On Tue, Sep 25, 2018 at 8:12 AM, Mirja Kühlewind
> <[email-masked]> wrote:
>> Hi Erik, hi Tommy,
>>
>> I beliebe I promised in Montreal to send some more comments on draft-pauly-tsvwg-tcp-encapsulation; at least that’s what my todo list says (now that I reached down to some bottom part of it).
>>
>> Anyway, here it comes, mainly based on what I already said in the meeting:
>>
>> - Section 3: the example format you define is definitely the right way to go and is what is usually used. However, when I was reading encapsulation, I was thinking that you could probably also just put the whole UDP packet including the UDP header into the TCP stream. Is that a thing that we want maybe discuss as well?
> 
> Mirja,
> 
> Would that include the IP header for for each encapsulated packet? If
> the TCP encapsulation endpoints are coincident with the UDP endpoints
> then the IP headers for reconstituting the UDP/IP packet could be
> derived from the outer IP header- that would save quite a few bytes.
> This makes the inner UDP checksum interesting when the packet goes
> through NAT, but that can be resolved by using the NAT Address
> Checksum option as described in draft-ietf-intarea-gue-extensions.

No I think that a different use case; this draft is mostly about 
end-to-end encapsulation to my understanding and tunneling is just 
mentioned because similar approaches could be used if needed. I was just 
thinking if it might be potentially easier to keep the whole UDP header 
instead of replacing it with a length field. However replacing it 
entirely also saves some bytes.

> 
> On the the topic of encapsulation format, I would suggest the authors
> look at GUE as a possible candidate to make TCP encapsulation generic
> and extensible. The generic part would mean that each TCP encapsulated
> packet could have type, extensible allows options like the one
> described above and there might be some other options that are useful
> like CRC and header authentication. Checksum offload for TCP and UDP
> packets encapsulated in TCP might also be possible to pull off with a
> little protocol help. The only thing needed to turn GUE into GTE
> (Generic TCP Encapsulation) would be adding the length field to
> demarcate messages in the stream.

As I said, as this is not about tunneling, GUE is probably an overkill.

> 
>>
>> - One comment I had in the meeting is actually related to this first point. I said that there should be a stronger recommendation to not change the protocol above. However, if you currently implement your own congestion control and the switch to TCP, it might actually be beneficial to disable that congestion control. I guess the point needs more discussion to figure out what we actually want to recommend, e.g. make minimal change and don’t change the header for easier convergence at gateway…?
>>
> One obvious question that will be raised on this draft is why would an
> application bother to encapsulate UDP in TCP as opposed to just using
> TCP directly. I would presume the answer is that there are some
> applications where the client or server side can't be changed and TCP
> encapsulation would be implemented as a transparent layer beneath the
> application. If that's the case, the congestion control of the
> application protocol probably can't be changed too easily.

Usually, these protocol are already designed for UDP because that's the 
right choice of protocol, e.g. low bandwidth IOT applications. However, 
to work over the Internet, often TCP is the only deployable choice.

I guess if that is not clear from the doc, it would make sense to add a 
small applicability section.

Mirja

> 
> Tom
> 
>> - In section 3.1 it could probably make sense briefly mention the possibility of head-of-line blocking when you multiplex.
>>
>> - Another high level comment from the discussion in Montreal was, that it would probably make sense to separate more clearly the case where encapsulation is used end-to-end or between tunnel endpoints. I guess for the second case, there may be some docs in the INT area that could be worth citing, e.g. draft-touch-intarea-tunnels; not sure if there is more.
>>
>> That’s all I have for now! Hope that helps! Thanks for writing this up!
>>
>> Mirja
>>
>>
>>

---

From: Joe Touch <[email-masked]>
Date: Fri, 5 Oct 2018 07:52:49 -0700

Hi, all,

I don’t quite understand what this draft is addressing. It needs to more clearly explain whether it is aimed at UDP or IP transit. That said:

- 2: the motivation for encapsulation starts by mentioning UDP is used for realtime; it would be important to note that this sort of encapsulation undermines that feature.

- 2.1 says that UDP is blocked but HTTP gets through; this discussion needs to address how HTTP works without DNS, which currently runs almost exclusively over UDP

- 2.2 mentions UDP NAT timeouts, but TCP also has both timeouts and NATs that deliberately inject RSTs to cutoff connections; this should be discussed somewhere as well

- overall, this mechanism should explain why the encapsulation happens at the UDP payload layer rather than at the IP layer; i.e., this encapsulation cannot preserve the correlation of IP headers to UDP headers , e.g., if IP options are used (yes, those options could be used for the TCP connection, but they would not match UDP boundaries anymore.

- the encapsulation mechanism refers to datagrams; are these IP or UDP? Either way, headers need to be included to represent options (for IP and now UDP)

- if this represents IP transit, then please consult draft-ietf-intarea-tunnels for advice that may be important here

- sec 4: the mechanism needs to explain whether transport option numbers are not included in-band; if not, it’s possible that the UDP port pair was available for the UDP stream but might not be available for the TCP stream. Similarly, destination port assignments do not always match between TCP and UDP anymore.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 5 Oct 2018 07:54:56 -0700

On Fri, Oct 5, 2018 at 5:19 AM, Mirja Kühlewind
<[email-masked]> wrote:
>>> I beliebe I promised in Montreal to send some more comments on
>>> draft-pauly-tsvwg-tcp-encapsulation; at least that’s what my todo list says
>>> (now that I reached down to some bottom part of it).
>>>
>>> - Section 3: the example format you define is definitely the right way to
>>> go and is what is usually used. However, when I was reading encapsulation, I
>>> was thinking that you could probably also just put the whole UDP packet
>>> including the UDP header into the TCP stream. Is that a thing that we want
>>> maybe discuss as well?
>>
>>
>> Mirja,
>>
> No I think that a different use case; this draft is mostly about end-to-end
> encapsulation to my understanding and tunneling is just mentioned because
> similar approaches could be used if needed. I was just thinking if it might
> be potentially easier to keep the whole UDP header instead of replacing it
> with a length field. However replacing it entirely also saves some bytes.
>
Hi Mirja,

Please take a look at draft-herbert-tsvwg-gte-00. This allows setting
the size of the message length field and compressing out the GUE
header. In the case that everything is UDP (or other transport
protocol since it's generic) the stream could be just the encapsulated
transport layer packets prefixed by two byte length. The length field
could be compressed out if there's not UDP options. Any encasulated
transport checksum can also be kept correct and efficeint to process.
Compressing out the UDP header could also be configured, and in the
case of encpasulated TCP it would be straightforward to apply Van
Jacobson compression if saving overhead is a big top concern.

Tom

>>> - One comment I had in the meeting is actually related to this first
>>> point. I said that there should be a stronger recommendation to not change
>>> the protocol above. However, if you currently implement your own congestion
>>> control and the switch to TCP, it might actually be beneficial to disable
>>> that congestion control. I guess the point needs more discussion to figure
>>> out what we actually want to recommend, e.g. make minimal change and don’t
>>> change the header for easier convergence at gateway…?
>>>
> right choice of protocol, e.g. low bandwidth IOT applications. However, to
> work over the Internet, often TCP is the only deployable choice.
>
>>> - In section 3.1 it could probably make sense briefly mention the
>>> possibility of head-of-line blocking when you multiplex.
>>>
>>> - Another high level comment from the discussion in Montreal was, that it
>>> would probably make sense to separate more clearly the case where
>>> encapsulation is used end-to-end or between tunnel endpoints. I guess for
>>> the second case, there may be some docs in the INT area that could be worth
>>> citing, e.g. draft-touch-intarea-tunnels; not sure if there is more.
>>>

---

From: Tommy Pauly <[email-masked]>
Date: Fri, 05 Oct 2018 08:07:57 -0700

Hi Joe,

Thanks for the comments! Responses inline.

> On Oct 5, 2018, at 7:52 AM, Joe Touch <[email-masked]> wrote:
> 
> Hi, all,
> 
> I don’t quite understand what this draft is addressing. It needs to more clearly explain whether it is aimed at UDP or IP transit.

The main goal for me, having worked on some specific protocols that needed to have a fallback of sending datagram payloads over TCP, is to have a document that can flesh out the tradeoffs and performance concerns to be referenced from other documents that use similar techniques.

> That said:

That was the intention with some of the concerns in Performance Considerations, but we should link directly there and flesh out the commentary about how this interferes with realtime deadlines.
> - 2.1 says that UDP is blocked but HTTP gets through; this discussion needs to address how HTTP works without DNS, which currently runs almost exclusively over UDP

Yes, that's a good point—this should likely say "when non-DNS UDP traffic is blocked". Essentially, the scenario being described is a network in which only "web-like traffic" is allowed. So, DNS + HTTP/HTTPS.
> - 2.2 mentions UDP NAT timeouts, but TCP also has both timeouts and NATs that deliberately inject RSTs to cutoff connections; this should be discussed somewhere as well

While TCP does have timeouts and RSTs from NATs, the key issue with UDP through NATs is how short the timeout is. TCP timeouts are often as much as an hour; while UDP timeouts are often less than two minutes, and sometimes just 30 seconds.

> - if this represents IP transit, then please consult draft-ietf-intarea-tunnels for advice that may be important here

Regarding IP vs UDP datagrams, the use cases we started with are ones in which only the "application protocol" payload on top of UDP is critical, but I agree that that won't always be the case. This document is describing a technique and caveats more than a specific protocol solution, so different use cases are expected to need different things. We can and should add text for these other cases.

> - sec 4: the mechanism needs to explain whether transport option numbers are not included in-band; if not, it’s possible that the UDP port pair was available for the UDP stream but might not be ava…

Indeed, they may not always match. Being able to communicate that in that case should be described in the document.

Thanks,
Tommy

---

From: Joe Touch <[email-masked]>
Date: Fri, 5 Oct 2018 08:25:45 -0700

> On Oct 5, 2018, at 8:07 AM, Tommy Pauly <[email-masked]> wrote:
> 
> Hi Joe,
> 
> The main goal for me, having worked on some specific protocols that needed to have a fallback of sending datagram payloads over TCP,

You haven’t answered my question - are these IP (and payload transport, etc.) or UDP (is header stripped off), or udp payload?

> is to have a document that can flesh out the tradeoffs and performance concerns to be referenced from other documents that use similar techniques.
> 
>> That said:
>> 

You should the also mention possibly needing to set TCP keepalives.

> Regarding IP vs UDP datagrams, the use cases we started with are ones in which only the "application protocol" payload on top of UDP is critical, but I agree that that won't always be the case. Thi…

Any header you strip off loses information and capability. I suspect those would be three different documents.

> Indeed, they may not always match. Being able to communicate that in that case should be described in the document.

FYI, I’m talking about the case where port 123456 is assigned to different services for TCP and UDP, as is now allowed.

Joe

---

From: Tommy Pauly <[email-masked]>
Date: Fri, 05 Oct 2018 08:35:58 -0700

> On Oct 5, 2018, at 8:25 AM, Joe Touch <[email-masked]> wrote:
> 
> 
> 
>> On Oct 5, 2018, at 8:07 AM, Tommy Pauly <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> Hi Joe,
>> 
>>> On Oct 5, 2018, at 7:52 AM, Joe Touch <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>> Hi, all,
>>> 

The specific protocols I have worked on are application protocol payloads that normally are transmitted as UDP payloads; so, no IP or UDP header. However, the performance considerations of transmitting unreliable datagrams over TCP is intended to be generic for cases in which the headers for IP and UDP are transmitted as well.
> You should the also mention possibly needing to set TCP keepalives.

Yes, that's a good idea for the "outer" TCP connection.
> Any header you strip off loses information and capability. I suspect those would be three different documents.

I would argue that every protocol that needs to have a mapping that can send it over TCP deserves its own document. Some may need the information in UDP and IP headers; some may not; and they almost always will involve some caveats specific to that protocol. The goal for this document is to compile the advice that will be useful to apply across specific protocol mappings.

> FYI, I’m talking about the case where port 123456 <tel:123456> is assigned to different services for TCP and UDP, as is now allowed.

Right!

Thanks,
Tommy

---

From: Tom Herbert <[email-masked]>
Date: Fri, 5 Oct 2018 09:04:38 -0700

On Fri, Oct 5, 2018 at 8:35 AM, Tommy Pauly <[email-masked]> wrote:
> I don’t quite understand what this draft is addressing. It needs to more
> clearly explain whether it is aimed at UDP or IP transit.
>
>
> The main goal for me, having worked on some specific protocols that needed
> to have a fallback of sending datagram payloads over TCP,
>
>
> You haven’t answered my question - are these IP (and payload transport,
> etc.) or UDP (is header stripped off), or udp payload?
>
>
> The specific protocols I have worked on are application protocol payloads
> that normally are transmitted as UDP payloads; so, no IP or UDP header.
> However, the performance considerations of transmitting unreliable datagrams
> over TCP is intended to be generic for cases in which the headers for IP and
> UDP are transmitted as well.
>
>
> is to have a document that can flesh out the tradeoffs and performance
> concerns to be referenced from other documents that use similar techniques.
>
> That said:
>
> - 2: the motivation for encapsulation starts by mentioning UDP is used for
> realtime; it would be important to note that this sort of encapsulation
> undermines that feature.
>
>
> That was the intention with some of the concerns in Performance
> Considerations, but we should link directly there and flesh out the
> commentary about how this interferes with realtime deadlines.
>
>
> - 2.1 says that UDP is blocked but HTTP gets through; this discussion needs
> to address how HTTP works without DNS, which currently runs almost
> exclusively over UDP
>
>
> Yes, that's a good point—this should likely say "when non-DNS UDP traffic is
> blocked". Essentially, the scenario being described is a network in which
> only "web-like traffic" is allowed. So, DNS + HTTP/HTTPS.
>
>
> - 2.2 mentions UDP NAT timeouts, but TCP also has both timeouts and NATs
> that deliberately inject RSTs to cutoff connections; this should be
> discussed somewhere as well
>
>
> While TCP does have timeouts and RSTs from NATs, the key issue with UDP
> through NATs is how short the timeout is. TCP timeouts are often as much as
> an hour; while UDP timeouts are often less than two minutes, and sometimes
> just 30 seconds.
>
>
> - overall, this mechanism should explain why the encapsulation happens at
> the UDP payload layer rather than at the IP layer; i.e., this encapsulation
> cannot preserve the correlation of IP headers to UDP headers , e.g., if IP
> options are used (yes, those options could be used for the TCP connection,
> but they would not match UDP boundaries anymore.
>
> - the encapsulation mechanism refers to datagrams; are these IP or UDP?
> Either way, headers need to be included to represent options (for IP and now
> UDP)
>
> - if this represents IP transit, then please consult
> draft-ietf-intarea-tunnels for advice that may be important here
>
>
> Regarding IP vs UDP datagrams, the use cases we started with are ones in
> which only the "application protocol" payload on top of UDP is critical, but
> I agree that that won't always be the case. This document is describing a
> technique and caveats more than a specific protocol solution, so different
> use cases are expected to need different things. We can and should add text
> for these other cases.
>
>
> I would argue that every protocol that needs to have a mapping that can send
> it over TCP deserves its own document. Some may need the information in UDP
> and IP headers; some may not; and they almost always will involve some
> caveats specific to that protocol. The goal for this document is to compile
> the advice that will be useful to apply across specific protocol mappings.
>

Tommy,

I think the document describes intends the facility to be generic, but
it looks like the discussion and some details in the draft seem to be
focused on UDP. IMO, this sort of mechanism must work equally well
with _ALL_ IP protocols. For instance, this would be a great fallback
technique to get SCTP or DCCP though a firewall (there's even less
chance that a firewall or NAT supports those than UDP). If UDP were to
be the only protocol supported for TCP encapsulation, then it's
perpetuating protocol ossification instead of fighting it!

Also, there might be some other benefits of encapsulating in TCP. For
instance, if I were using this I'd almost certainly want to use TLS
also to prevent firewalls from providing their "value add" to TCP
encapsulation.

Tom

>
>
>
> - sec 4: the mechanism needs to explain whether transport option numbers are
> not included in-band; if not, it’s possible that the UDP port pair was
> available for the UDP stream but might not be available for the TCP stream.
> Similarly, destination port assignments do not always match between TCP and
> UDP anymore.
>
>
> Indeed, they may not always match. Being able to communicate that in that
> case should be described in the document.
>
>
> FYI, I’m talking about the case where port 123456 is assigned to different
> services for TCP and UDP, as is now allowed.
>
>
> Right!
>
> Thanks,
> Tommy
>
>
> Joe
>
>
> Thanks,
> Tommy
>
>
> Joe
>
>

---

From: Derek Fawcus <[email-masked]>
Date: Mon, 8 Oct 2018 18:14:26 +0100

On Fri, Oct 05, 2018 at 09:04:38AM -0700, Tom Herbert wrote:
> For instance, this would be a great fallback technique to get SCTP or DCCP though a
> firewall (there's even less  chance that a firewall or NAT supports those than UDP).

I know of one firewall and/or NAT which has simple support for them (and UDP-Lite);
but not doing any of the draft SCTP NAT support stuff.

Plus one other vendor product which can easily support all three in its s/w ACL code.

DF

---

From: Randall Stewart <[email-masked]>
Date: Mon, 8 Oct 2018 10:24:27 -0700

The FreeBSD Nat/firewall has support for most of the SCTP NAT stuff. I
think there was one small shortcut they took.

R

Sent from my iPhone

> firewall (there's even less  chance that a firewall or NAT supports those than UDP).

I know of one firewall and/or NAT which has simple support for them
(and UDP-Lite);
but not doing any of the draft SCTP NAT support stuff.

Plus one other vendor product which can easily support all three in
its s/w ACL code.

DF

---

From: Tom Herbert <[email-masked]>
Date: Mon, 8 Oct 2018 15:24:37 -0700

> Plus one other vendor product which can easily support all three in its s/w ACL code.
Derek,

No doubt there's some implentatios that deal with (some) non-TCP
protocols, but I doubt there is ubiquitous support that would make
them serious cadidates to use on the Internet.

Tom