Subject: Re: [trill] Mirja Kühlewind's Discuss on draft-ietf-trill-over-ip-16: (with DISCUSS and COMMENT)

From: Donald Eastlake <[email-masked]>
Date: Wed, 4 Apr 2018 23:10:20 -0400

Hi Mirja,

On Thu, Mar 29, 2018 at 10:58 AM, Mirja Kühlewind <[email-masked]> wrote:
> Mirja Kühlewind has entered the following ballot position for
> draft-ietf-trill-over-ip-16: Discuss
>
> ...
>
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> Usage of encapsulation schemes
> ----
> The document does not clearly explain and justify why different encapsulations
> are needed. The document should more extensively discuss the different
> properties and trade-offs for each encapsulation and give clear recommendations
> when you to use which encapsulation. E.g. the document does not clearly specify
> when to use IPSec (besides saying „if security in needed“), however, the
> document needs to explain more clearly what is meant by this: given the
> over-the-Internet path is always not trusted, does that mean that IPSec should
> always be used in that scenario? Why is IPSec specified instead of using TLS or
> DTLS with the TCP or UDP encapsulations respectively?

More information can be added about properties of different encapsulations.

In the latest revision (-16), Appendix A was added (page 48) briefly
discussing the IP security decision: (1) Is the information presented
in Appendix A sufficient? (2) Would you like that material moved into
the body of the document.

> Why is both UDP and TCP
> encapsulation needed, given that UDP is the default that is mandatory to
> implement anyway? Why are the short-comings of UDP and when is it
> recommended/beneficial to switch to TCP?

I would say that the primary problem with UDP is fragmentation.

> DSCP
> -----
> 1) Section 4.3 should also talk about decapsulation as DCSP is often
> overwritten on the path and therefore the DCSP of the inner and other IP
> headers can differ on decapsulation. Please see RFC2983 for further guidance.
> You probably should specify to discard the outer DSCP at tunnel egress in your
> use case.

OK.

> 2) Further it is not clear to me if the use of CS7 in appropriate for this use
> case as RFC 4594 says "   o  CS7 marked packets SHOULD NOT be sent across
> peering points.
>       Exchange of control information across peering points SHOULD be
>       done using CS6 DSCP and the Network Control service class."

Although providing generally similar guidance, I note that RFC 8100 is
a more recent Informational RFC on this topic than RFC 4594.

We can add some guidance that CS6 and CS7 should not be used for
traffic being sent through provider facilities. I believe that
essentially all providers that would be adversely affected by such
traffic provide their own DSCP mapping at their permitter. In any
case, a TRILL campus operator might want to map DSCP themselves before
giving traffic to a provider to reduce the probability of provider
mapping.

> 3) Moreover, if my understanding is correct, the high priority classes in TRILL
> are not exclusively reserved for control data. However, CS6 and CS7 is only
> meant for control and rounting traffic. If those classes are used it must be
> ensure that the traffic send with these DSCP is not overloading the network. I
> think further (security) considerations are needed here.

Current TRILL documents recommend use of the highest available
priority for messages affecting the establishment and maintenance of
adjacency and the next highest priority for other important control
messages (see Section 8.2 of RFC 7780). In case where CS6 and CS7 are
not available, this advice would still apply, it is just that a lower
priority might the be highest available.

We can also add something to the Security Considerations.

> Broadcast Link Encapsulation Considerations
> ------
> Not every transport encapsulation can be used for Broadcast/Multicast. TCP
> cannot be used. This is mention later but also be consider in the text in
> section 5.3

OK.

> TCP Encapsulation
> -----
> If my understanding is correct than TRILL does not know that the connect of a
> TRILL data packet is. That means the data could can also contain traffic that
> is running over TCP, right? Encapsulating TCP in TCP should generally avoided
> if possible and need further considerations as loss in the outer control loop
> that is used on the TRILL IP link appears as strongly varying delays to the
> inner control loop and therefore can have very negative effects.

We can add further considerations on this topic.

> TCP Connection Establishment (section 5.6.1)
> -----
> This section seem to assume for all configured or discovered tunnel endpoints
> should  immediately (at node start up time) and permanently open one/multiple
> TCP connections. I'm in general uncertain if this is the right approach.
> However, even if the connection is not closed, it might not be usable after an
> idle time, as middlebox on the path may have removed their state. Therefore, to
> keep a connection permanently open, the endpoint need probably to send
> keep-alives, or alternative a mechanism to detect such a failure (quickly) and
> re-establish the connection such be used. Alternative, a connection could be
> open and closed when needed. In this case also more recommendation is needed
> when to open and close the connection(s).

BFD is the mechanism provided in TRILL [RFC7175] to detect failures
more rapidly than would be detected by IS-IS Hellos. We can add some
recommendations on this issue.

> Fragmentation
> ----
> The fragmentation problem for Internet links is not sufficiently addressed.
> trill requires a minimum size of 1,470 bytes while most Internet paths only
> support an MTU of 1,470 bytes which will automatically lead to fragmentation
> when encapsulation is added. It can not be assumed that all Internet path
> support IPv6, therefore the recommendation "if fragmentation is anticipated
> with the encapsulations specified in this document, the use of IPv6 is
> RECOMMENDED" is often hard to realize. I assume that many trill IS-IS packets
> are smaller than 1,470 bytes thus fragmentation might not occur, however, for
> larger packets I would rather recommend to implement an additional mechanism to
> enable segmentation above UDP (+ PMTU discovery) or the use of TCP. In any case
> more clear recommendation is needed.

We can add further material on MTU and look into a protocol for
fragmentation and re-assembly over a TRILL hop.

> Port assignment
> ----
> Only one port per service can be assigned (see principles in rfc6335 and
> guidelines in rfc7605). Therefore the spec should be changed to the use of one
> ports and describe how different kinds of packets can be distinguished by other
> means (which should be easily possible to my understanding). Further, it is
> expected that the document confirms that any new security added later on will
> also use the port assigned (i.e., future versions will not be eligible for new
> ports e.g. for a “TLS” variant).

We will respond separately to the port request review comments we have
received.

> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> Editorial coments:
> 1) It is really confusing that the whole document is talking about ports as
> tunnel endpoints while it also often talks about transport (TCP/UDP) ports. It
> makes it really hard to read this document. Maybe these things can be better
> distighlished somehow.

I'm sorry but I don't understand what is confusing. Most of the
occurrences of "transport" were added recently. As far as TRILL is
concerned, the (possibly multi-point to multi-point) TRILL over IP is
just a type of link between TRILL switch ports.

> 2) This document should not re-specify the UDP header (5.4). At minimum the
> text should be changed the following way. OLD "Where he UDP Header is as
> follows:" NEW "Where he UDP Header is as follows as specified in [RFC0768] :"

OK.

Thanks,
Donald
===============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Mon, 9 Apr 2018 08:12:16 -0700

Hi Donald,

sorry for my late reply. Please see below.

> Am 04.04.2018 um 20:10 schrieb Donald Eastlake <[email-masked]>:
> 
> Hi Mirja,
> 

Sorry, I actually missed that appendix. So, yes, I think further guidance needs to be provided in the body and not only explaining why IPSec was selected (focusing on the technical point(s) and not so much on the wg process behind it) but more importantly you need to give more detailed guidance when it needs to/should be used.

Also, what you mean by "Other security protocols can be used by agreeing TRILL over IP transport ports“?

> I would say that the primary problem with UDP is fragmentation.

That is a good point but given that UDP is the mandatory, and only mandatory to implement solution anyway, you need to address the fragmentation problem anyway. In this case it is still not very clear when it is beneficial to negotiate TCP instead.

> a more recent Informational RFC on this topic than RFC 4594.

RFC8100 is only on guidance for interconnected networks. So I would say RFC4594 is actually the more general reference. Anyway the recommendation is the same.
> giving traffic to a provider to reduce the probability of provider

Yes, please do.

> We can also add something to the Security Considerations.

Please do.

> We can add further considerations on this topic.

Please do!

> recommendations on this issue.

TCP is a connected protocol, so you might just receive a RST at some point and the connection is gone. Further if you don’t use the TCP connection for a while (often 30 minutes or hours) it might tear down your network state e.g. at a NAT and you will just not be able to send any packets anymore. That’s a different problem then detecting route failures. You need either to send keep-alive (which TCP provides), or only open and close TCP connection when needed, or provide guidance on when and who has to reestablish the connection if it happens to not work anymore for any reason.

> fragmentation and re-assembly over a TRILL hop.

Please do!

> We will respond separately to the port request review comments we have

Ok.

> just a type of link between TRILL switch ports.

Because a "transport port" is the thing in the transport header that indicates the source and destination ports. It’s just a terminology collision that gave me a hard time at the beginning.
>> follows:" NEW "Where he UDP Header is as follows as specified in [RFC0768] :"

Thanks!
Mirja

---

From: "Susan Hares" <[email-masked]>
Date: Tue, 10 Apr 2018 16:24:50 -0400

Donald and Mirja:

It appears this discussion is going well and that you both are progressing.  

On RFC8100, would you both mind if we included it.  It took me a while to remember which RFC it was, and I found the advice insightful. 

Sue Hares

-----Original Message-----
From: Mirja Kuehlewind (IETF) [mailto:[email-masked]] 
Sent: Monday, April 9, 2018 11:12 AM
To: Donald Eastlake
Cc: The IESG; [email-masked]; Susan Hares; trill IETF mailing list
Subject: Re: Mirja Kühlewind's Discuss on draft-ietf-trill-over-ip-16: (with DISCUSS and COMMENT)

Hi Donald,

>> The document does not clearly explain and justify why different 
>> encapsulations are needed. The document should more extensively 
>> discuss the different properties and trade-offs for each 
>> encapsulation and give clear recommendations when you to use which 
>> encapsulation. E.g. the document does not clearly specify when to use 
>> IPSec (besides saying „if security in needed“), however, the document 
>> needs to explain more clearly what is meant by this: given the 
>> over-the-Internet path is always not trusted, does that mean that 
>> IPSec should always be used in that scenario? Why is IPSec specified instead of using TLS or DTLS with the TCP or UDP encapsulations respectively?
> 

> 
>> Why is both UDP and TCP
>> encapsulation needed, given that UDP is the default that is mandatory 
>> to implement anyway? Why are the short-comings of UDP and when is it 

>> overwritten on the path and therefore the DCSP of the inner and other 
>> IP headers can differ on decapsulation. Please see RFC2983 for further guidance.
>> You probably should specify to discard the outer DSCP at tunnel 
>> egress in your use case.
> 
> OK.
> 

Yes, please do.

> 
>> 3) Moreover, if my understanding is correct, the high priority 
>> classes in TRILL are not exclusively reserved for control data. 
>> However, CS6 and CS7 is only meant for control and rounting traffic. 
>> If those classes are used it must be ensure that the traffic send 
>> with these DSCP is not overloading the network. I think further (security) considerations are needed here.
> 

Please do.

>> Not every transport encapsulation can be used for 
>> Broadcast/Multicast. TCP cannot be used. This is mention later but 
>> also be consider in the text in section 5.3
> 
> OK.
> 
>> TCP Encapsulation
>> -----
>> If my understanding is correct than TRILL does not know that the 
>> connect of a TRILL data packet is. That means the data could can also 
>> contain traffic that is running over TCP, right? Encapsulating TCP in 
>> TCP should generally avoided if possible and need further 
>> considerations as loss in the outer control loop that is used on the 
>> TRILL IP link appears as strongly varying delays to the inner control loop and therefore can have very negative effects.
> 

Please do!

>> This section seem to assume for all configured or discovered tunnel 
>> endpoints should  immediately (at node start up time) and permanently 
>> open one/multiple TCP connections. I'm in general uncertain if this is the right approach.
>> However, even if the connection is not closed, it might not be usable 
>> after an idle time, as middlebox on the path may have removed their 
>> state. Therefore, to keep a connection permanently open, the endpoint 
>> need probably to send keep-alives, or alternative a mechanism to 
>> detect such a failure (quickly) and re-establish the connection such 
>> be used. Alternative, a connection could be open and closed when 
>> needed. In this case also more recommendation is needed when to open and close the connection(s).
> 

>> trill requires a minimum size of 1,470 bytes while most Internet 
>> paths only support an MTU of 1,470 bytes which will automatically 
>> lead to fragmentation when encapsulation is added. It can not be 
>> assumed that all Internet path support IPv6, therefore the 
>> recommendation "if fragmentation is anticipated with the 
>> encapsulations specified in this document, the use of IPv6 is 
>> RECOMMENDED" is often hard to realize. I assume that many trill IS-IS 
>> packets are smaller than 1,470 bytes thus fragmentation might not 
>> occur, however, for larger packets I would rather recommend to 
>> implement an additional mechanism to enable segmentation above UDP (+ PMTU discovery) or the use of TCP. In any case more clear recommendation is needed.
> 

Please do!

> 
>> Port assignment
>> ----
>> Only one port per service can be assigned (see principles in rfc6335 
>> and guidelines in rfc7605). Therefore the spec should be changed to 
>> the use of one ports and describe how different kinds of packets can 
>> be distinguished by other means (which should be easily possible to 
>> my understanding). Further, it is expected that the document confirms 
>> that any new security added later on will also use the port assigned 
>> (i.e., future versions will not be eligible for new ports e.g. for a “TLS” variant).
> 

Ok.

>> 1) It is really confusing that the whole document is talking about 
>> ports as tunnel endpoints while it also often talks about transport 
>> (TCP/UDP) ports. It makes it really hard to read this document. Maybe 
>> these things can be better distighlished somehow.
> 

> 
>> 2) This document should not re-specify the UDP header (5.4). At 
>> minimum the text should be changed the following way. OLD "Where he 
>> UDP Header is as follows:" NEW "Where he UDP Header is as follows as specified in [RFC0768] :"
> 
> OK.

Thanks!
Mirja

> 155 Beaver Street, Milford, MA 01757 USA [email-masked]

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 12 Apr 2018 14:08:31 -0400

Hi Mirja,

OK.

> Also, what you mean by "Other security protocols can be used by agreeing TRILL over IP transport ports“?

Perhaps the sentence you ask about was completely superfluous but I
don't understand what question there is on its meaning. If there is
agreement by the end points of the connection and, instead of the
mandatory to implement IPsec they decide to use something else like
DTLS or SSH, they can certainly do that.

> That is a good point but given that UDP is the mandatory, and only mandatory to implement solution anyway, you need to address the fragmentation problem anyway. In this case it is still not very cl…

We'll see what we can add but it seems unreasonable, in the absence of
field experience, to demand that this document should precisely
indicate when which option should be used.

> TCP is a connected protocol, so you might just receive a RST at some point and the connection is gone.

Yes, that is detectable immediately, but it seems to me you want some
delay before trying to re-establish. I was talking about failures
detected by traffic not getting through.

> Further if you don’t use the TCP connection for a while (often 30 minutes or hours) it might tear down your network state e.g. at a NAT and you will just not be able to send any packets anymore. That’s a different problem then detecting route failures. You need either to send keep-alive (which TCP provides), or only open and close TCP connection when needed, or provide guidance on when and who has to reestablish the connection if it happens to not work anymore for any reason.

Without further mechanisms, TRILL over IP will not establish any
adjacencies through any NAT that is changing IP addresses so there
will not be any such TCP connections. However there might be firewalls
or whatever that do not change IP addresses and have this time-out
problem so we should talk about it.

> Because a "transport port" is the thing in the transport header that indicates the source and destination ports. It’s just a terminology collision that gave me a hard time at the beginning.

I can try to reduce or eliminate occurrences of "transport port".

Thanks,
Donald

---

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 19 Apr 2018 15:08:55 +0200

Hi Donald,

please see inline.

On 12.04.2018 20:08, Donald Eastlake wrote:
>>> On Thu, Mar 29, 2018 at 10:58 AM, Mirja Kühlewind <ietf@kuehlewind..net> wrote:
Yes, I just didn't understand the intent of the that sentence.

> indicate when which option should be used.
It's not about precisely indicating when which option should be used but 
a) provide guidance on what is the best option to use under which 
conditions (given the different solutions have different 
characteristics) and b) maybe also provide guidance when not use 
something, e.g. if it is recommend to always use IPSec for over the 
Internet paths it should say so (however, to be honest, I'm not sure how 
well IPSec would work over the Internet and my recommendation would 
probably rather to use (D)TLS, however that's a topic on its own).

> delay before trying to re-establish.
Not sure. If you have an open connection that was idle for a while, I 
guess you should try and reconnection immediately. If you cannot connect 
at all, you have to treat that as an error. If you try to connect and 
immediately get a RST, you might wait for for a while and try again. So 
there are different cases that need to further specified in the doc.

> I was talking about failures

Yes, that's a different thing.
> problem so we should talk about it.

If you connect over an Internet path, you never know if there is a NAT 
or something else.

Yes, please provide further text.
> I can try to reduce or eliminate occurrences of "transport port".
Or maybe make sure it is clearly separated and clarify somewhere that 
there is this terminology collision.

Thanks!
Mirja