Subject: RE: [NSIS] attempted summary of fragmentation discussion

From: "Hancock, Robert" <[email-masked]>
Date: Fri, 13 Jun 2003 00:08:36 +0100

marcus, all,

i still feel i am missing something in this argument.
apologies if this mail is a bit long.

1. georgios' proposal is to have fragmentation functionality 
in the NTLP anyway, and just the option to not use it. 
(another option would be to have no fragmentation functionality 
in the NTLP at all, just 'packet too big' error reporting - 
but that's not what's been proposed.)
==> therefore it is a pure increase in complexity in the NTLP 
compared to 'the NTLP does fragmentation' option.

2. if you believe fragmentation is a 'rare case', do it as 
part of your exception handling software; the code needed for 
fragmentation is trivial (certainly trivial compared to PMTU 
discovery logic which is needed in the NTLP wherever fragmentation 
is done, as Melinda points out below).
==> we don't care either way in terms of NTLP complexity.

3. my more general concern is that the concept of PMTU discovery 
has a number of non-trivial aspects associated with it - check out 
the plpmtud bof notes from the last IETF (and rfc2923) for details 
of how in the real world it doesn't really work as well as one 
would like. while these problems are fixable (and indeed have to 
be fixed for the NTLP, whichever approach we take) they appear to 
me to be pointers that PMTU in the NSLP will architecturally fragile and something to be avoided 
unless there is a pressing performance reason why it has to be done. in particular,
this approach will add an RTT to signalling setup times while the MTU is 
discovered (unless the initial message in the exchange is small), and messages
will be black-holed unless the error reporting (which depends on backwards routing
per flow at every intermediate node) works perfectly.
==> IMHO not doing fragmentation is a mechanism for storing up
deployment and design problems for the future .

4. so, maybe people are really concerned about performance issues.
here is an example:

                     Signalling Messages

 +-----------+                                    +-----------+
 |NSLP Source| +---------\  +------+ Thin Pipe    | NSLP Sink |
 +-----------+ |Fat Pipe  > |NTLP 2| -----------> +-----------+
 |  NTLP 1   | +---------/  +------+              |  NTLP 3   |
 +-----------+                                    +-----------+

(the pipes are fat and thin in the sense of being able to send
big or small messages). This case is I think the *most* favourable
to the "don't fragment at NTLP" viewpoint, because
*) if the NSLP is present at the intermediate nodes the question
of layering is moot
*) if the pipes are the other way round, fragmentation won't happen
anyway

So, let's consider 4 cases for what happens when a big message
is sent source->sink. The emphasis of the analysis is on the 
processing burden at Node 2.
a) Fragmentation at NTLP only:
Node 1 sends message
Node 2 NTLP executes the following
- receive incoming message into NTLP code
- process incoming message
- determine outgoing interface for route
- construct header for outgoing packets
- while (packet payload is left)
  { send chunk of packet payload in fragment }

b) Fragmentation optionally allowed at NSLP:
Because the NSLP designer and implementor don't believe in making
their own lives harder, the NSLP sends its message with 'fragmentation
allowed' anyway
==> end result is the same as case (a) except with more complex
code because of the optional aspects which aren't being used

c) Fragmentation happens at the NSLP (either because
the designer/implementor are blackmailed into it, or
because it is taken out of the NTLP):
Nodes 1 & 2 go through PMTU discovery procedure
Node 1 NSLP generates a sequence of message fragments
Node 1 NTLP bundles them back together into one or two
messages (depending on how much overhead is added)
Node 2 NTLP executes the following (once or twice)
- receive incoming message into NTLP code
- process incoming message bundle header
- for each message in bundle
  {
   - determine outgoing interface for route
   - construct header for outgoing packet
   - send outgoing packet
  }
==> to me, it isn't clear how this makes life easier for
NTLP at Node 2 (and the additional burden applies to any
other nodes between 1 and 2 as well).

d) So, you don't like bundling after all? Take it out
altogether (but remember, the experience leading to 2961 was
that the step called 'receive' here is actually the dominant
element of the processing load):
Nodes 1 & 2 go through PMTU discovery procedure
Node 1 generates a sequence of message fragments, each sent
individually
Node 2 NTLP executes the following
- for each fragment
  {
   - receive incoming message into NTLP code
   - determine outgoing interface for route
   - construct header for outgoing packet
   - send outgoing packet
  }
==> seems hardest of all (again, the extra load applies
everywhere upstream of node 2 as well).

why is this wrong?

cheers,

robert h.

> -----Original Message-----
> From: Melinda Shore [mailto:[email-masked]]
> Sent: 12 June 2003 17:16
> To: Marcus Brunner
> Cc: Hancock, Robert; 'Georgios Karagiannis'; [email-masked]
> Subject: Re: [NSIS] attempted summary of fragmentation discussion 
> 
> 
> > What I have not hear so far is the cost of having
> > fragmentation built into NTLP. Georgios and myself are
> > concerned that the NTLP gets too fat for high-speed
> > processing, so we try to avoid to design for the rare
> > cases such as fragmentation.
> 
> Obviously we don't want to allow IP to fragment the packets,
> since this will remove our ability to support middlebox
> traversal.  That means that either we don't allow any
> fragmentation or we handle fragmentation ourselves.  If we
> handle fragmentation ourselves it's pretty clear that it
> needs to be handled lower in the stack rather than higher in
> the stack, primarily because we need to know the entire
> packet size in order to make correct fragmentation
> decisions.  "Entire packet size" may include bundling but
> will certainly include authentication-related data, which
> can be quite large.  Additionally there may be other TLVs
> in the header (for example, related to reliable delivery)
> that the upper layers will not know about and therefore not
> be able to factor into a fragmentation decision.
> 
> Adding support for fragmentation will certainly increase the
> code load, which may be an issue in memory-constrained
> devices, but it will not meaningfully lengthen the execution
> path since it's a quick check to see if you need to fragment
> (if packet_length > pmtu) on the sender side and a quick
> check to see if reassembly is required (if (frag_bit_is_set)) 
> on the receiver side.  Path MTU will need to be determined
> regardless of where fragmentation occurs and is a constant
> cost unless there's a decision not to fragment at all, which
> brings with it its own costs (and, for that matter, will
> require PMTU discovery as well).
> 
> Melinda
> 
_______________________________________________
nsis mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nsis

---

From: Melinda Shore <[email-masked]>
Date: Fri, 13 Jun 2003 10:53:06 -0400

Those are good points, but I think what's missing from that
analysis is the reassembly costs, which would tend to
dominate the process (as opposed to fragmentation costs,
which are low).  When talking about reassembly you have to
consider what happens when packets arrive out-of-order, are
dropped, etc., and that means additional buffers
(statefulness), additional timers (more statefulness), and
additional execution logic.

The other problem, I think, is that the problem isn't
bundling, it's message size, and that's almost certainly
going to be an issue whether application messages are
bundled or not.

Melinda

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Tue, 17 Jun 2003 09:49:19 +0200

Hi Robert

> but that's not what's been proposed.)

You are right, this could be also seen as a possible alternative.

Best Regards,
Georgios