Subject: Warren Kumari's No Record on draft-ietf-6man-rfc1981bis-06: (with COMMENT)

From: Warren Kumari <[email-masked]>
Date: Mon, 08 May 2017 13:01:56 -0700

Warren Kumari has entered the following ballot position for
draft-ietf-6man-rfc1981bis-06: No Record

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-6man-rfc1981bis/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

[ Reminder to myself - I'm leaving this as No Record while looking
more... ]

I sent email about this to the authors on Feb 23rd - I seem to still have
have many of the same questions...

Comments:
1: Sec 1: "Path MTU Discovery relies on such messages to determine the
MTU of the path."
 -- it is unclear which "such" refers to. Perhaps s/such/ICMPv6/ (or
PTB).

2: Sec 3: "Upon receipt of such a message, the
   source node reduces its assumed PMTU for the path based on the MTU
of
   the constricting hop as reported in the Packet Too Big message" --
this says that it reduces it *for the path*. But (as somewhat alluded to
later in the draft) the nodes doesn't know what the path *is* -- it can
decrease for the destination, or flow, or even interface, but (unless it
is strict source routing) it doesn't control or really know the path (see
also #4)

3: Sec 4: "The recommended setting for this timer is twice its minimum
value (10 minutes)." - as above. This was from 1996 - were these metrics
discussed at all during the -bis? I suspect that the average flow is much
shorter these days (more web traffic, fatter pipes, etc) and so a flow of
10 minutes seems really long (to me at least). 

4: Sec 5.2: "The packetization layers must be notified about decreases in
the
   PMTU.  Any packetization layer instance (for example, a TCP
   connection) that is actively using the path must be notified if the
   PMTU estimate is decreased.
      Note: even if the Packet Too Big message contains an Original
      Packet Header that refers to a UDP packet, the TCP layer must be
      notified if any of its connections use the given path."
 - this is related to #2 -- I don't know *which* path my packets take -
once I launch them into the void, they may be routed purely based upon
destination IP address, or they may be hashed based upon some set of
header fields to a particular ECMP link or LSP. Once packets hit a load
balancer, it is probably even *likely* that the UDP and TCP packets end
up on different things. So, if I get a PTB from a router somewhere, I can
probably guess that other packets to the same destination address will
also follow that path, but I cannot know that for sure. I'm fine to
decrease MTU towards that destination IP, but is that what this is
suggesting? If so, please say that. If not, please let me know what I
should do. The above is even more tricky / fun when I'm using flow id as
the flow identifier -- if I get a PTB for flow 0x1234, what do I do? 

 5: Sec 5.3: "Once a minute, a timer-driven procedure runs through all
cached PMTU values, and for each PMTU whose timestamp is not "reserved"
and is older than the timeout interval ...". Please consider providing
clarifications here. The wording implies that I should set a timer to
fire on the minute, and trigger the behavior. If all of the (NTP synced!)
machines in my datacenter do this, and all try send bigger packets (on
1/10th of long flows) their first hop router will get many, many
over-sized packets and it will severely rate-limit the PTBs. 

Nits (Some of these are purely academic.)
I understand that you are trying to limit the changes, so feel free to
ignore these:

1: "A node sending packets much smaller than the Path
MTU allows is wasting network resources and probably getting
suboptimal throughput." - the "much" confuses me. If I'm using anything
less than the MTU I'm wasting network resources and getting suboptimal
throughput - I might not care, but if (used MTU) < (path MTU) I'm wasting
resources.

2: "Nodes implementing Path MTU Discovery and sending packets larger
than
the IPv6 minimum link MTU are susceptible to problematic connectivity
if ICMPv6 [ICMPv6] messages are blocked or not transmitted." The
"implementing Path MTU Discovery and" seems redundant. ALL nodes sending
packets larger than minimum MTU are "susceptible to problematic
connectivity if ICMPv6 [ICMPv6] messages are blocked or not
transmitted.". I get what you are trying to say, but my OCD tendencies
would not allow me to ignore this... 

3: "In the case of multipath routing (e.g., Equal Cost Multipath Routing,
ECMP),"- this is vague / confusing -- (Equal Cost Multipath Routing,
ECMP) makes it sound like either ECMP is an acronym for Equal Cost
Multipath Routing, or that ECMP is something different to Equal Cost
Multipath Routing.
I'd suggest just dropping the "ECMP" (or, "Equal Cost Multipath (ECMP)
routing", but that seems clumsy)

---

From: Suresh Krishnan <[email-masked]>
Date: Tue, 9 May 2017 00:54:46 -0400

Hi Warren,
  Thanks for your comments. Please see responses inline.

> https://datatracker.ietf.org/doc/draft-ietf-6man-rfc1981bis/

This was new text added in the -bis document. In the context of the
sentence (regarding ICMPv6 filtering leading to black hole connection)
I think ICMPv6 works better than singling out PTB.

> is strict source routing) it doesn't control or really know the path (see

Right. This is explored further in Section 5.2

"  However, in most cases a node will not have enough
   information to completely and accurately identify such a path.
   Rather, a node must associate a PMTU value with some local
   representation of a path.  It is left to the implementation to select
   the local representation of a path."

Would you like some additional text in this section?

> 10 minutes seems really long (to me at least).

AFAIK, there was no discussion regarding this number. I have a hard
time seeing how this number relates to the (shortened) length of the
flow, though.

> the flow identifier -- if I get a PTB for flow 0x1234, what do I do?

Yes. You are right. Any out of band mechanism for probing the MTU will
likely end up having the same problem if the probe packets are treated
differently from the payload packets by the load balancers and other
middleboxes.

> over-sized packets and it will severely rate-limit the PTBs.

I am not sure why the first hop router will send PTBs. At timer
expiry, the PMTU estimate will only be set to the MTU of the first-hop
link and hence the first hop router should be able to handle this, no?

> I understand that you are trying to limit the changes, so feel free to

I will let the editor chime in on these.

Thanks
Suresh

---

From: Warren Kumari <[email-masked]>
Date: Tue, 9 May 2017 05:45:38 -0400

> I think ICMPv6 works better than singling out PTB.

Yup, WFM.

> Would you like some additional text in this section?

I don't really know -- I have a weird feeling that the document is
explicit about what you must do under certain circumstances, but then
handwaves away the fact that you cannot really tell. I think that it
would be helpful to point at Sec 5.2 from here.

> time seeing how this number relates to the (shortened) length of the

It's fine if it wasn't discussed (these are just comments), but my
thinking goes:

In 1996 I beleive that average flows were much longer and so flows
would actually hit the 10 minute timer and attempt to increase the
increase MTU process, but these days, with much shorted flows, you
will hit it much less often.

Also, in 1996 a 45Mbps DS3 / T3 was fast. If I started transferring a
large file and my first few packets went down a pipe with a 1280byte
MTU path, which then changed to a path with 1500bytes, after 10
minutes I would have transferred ~3.3GB, and 2.6million packets., for
a "wastage" of 580MB (2.6m * (1500 - 1280)).
On a 10GE, in the same situation, after 10 minutes I would have
transferred 750GB and 585million packets, for a "wastage" of almost
13GB.
(Note: Back of the envelope calculations, may have missed a zero somewhere)

> differently from the payload packets by the load balancers and other

Yup. This is related to the #2. The definition of "path" is very vague
- if I use flow id as my path representation, how do I know what else
to notify?
I think that this should have some better description of what exactly
I adjust if I get the PTB...

If this were a new protocol, which wasn't already implemented
basically everywhere, I'd say that this needs to be much much clearer
and more specific -- but, the fact that there are already
implementations which a: work, and b: new implementations can look at
means that I'm not too fazed about this...
> link and hence the first hop router should be able to handle this, no?

Nope - example scenario...

┌───────────┐               .───────.             ┌────────────┐
│ Computer  │──MTU:1500───▶( Router  )──MTU:1400──▶ "Internet" │
└───────────┘               `───────'             └────────────┘

The outgoing interface from my router is 1400 bytes. This means that
every connection will get back a PTB when it tries to reach the
internet - I *should* change the default MTU on my machine, but, well,
I'm a user...

In many scenarios the lower MTU will be close to the sending machine -
if you synchronize the "try redo  PMTUd" you run the risk of spikes.
Many machines (like my Apple laptop) are now NTP synced - even if the
MTU bottleneck is far away, when everyone tries to use 1500byte
packets at the same time, whatever device has a smaller MTU will have
a bad day...

W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf