Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

From: Rick Taylor <[email-masked]>
Date: Tue, 30 Jan 2018 20:47:55 +0000

Brian, Scott

I'm not insisting on any specific behavior just trying to imagine how an implementer solves the following problem in the receive path code:

if (bytes_received < bundle.length)
{
  // What do I do here?
}

I'll respond in more detail tomorrow when I'm not on my phone.

Cheers
Rick

(Sent from my phone)

----- Reply message -----
From: "Brian Sipos" <[email-masked]>
To: "Rick Taylor" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: [dtn] TCP-CL and XFER_SEGMENT
Date: Tue, Jan 30, 2018 18:38

Rick,

There are now some options in TCPCLv4 compared to v3 because of the header extension items. If it's core enough behavior, we can choose to make some kind of CAN_FRAGMENT flag and eat one of the contact header flag bits. If it's not really a core behavior, there can be an extension item to indicate reactive fragmentation (with a default cannot-fragment logic if the extension type is not present).

I don't have a strong preference; there are 7 extra header bits and 65k extra extension types. The header flags are simpler logic to describe/encode.

________________________________
From: Rick Taylor <[email-masked]>
Sent: Tuesday, January 30, 2018 7:17:28 AM
To: [email-masked]; [email-masked]; Brian Sipos
Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

Hi All,

(Chair Hat Off)

So, the plot thickens... Previously the contact header of TCP-CL had an
indicator that reactive fragmentation was allowed/desired, but it has
been swept away in the uplift.

>From a purist perspective, fragmentation is a bundle process, and
should therefore be a function of the bundle processing agent,
however, bundle processing flags exist in the immutable primary block,
and reactive fragmentation is a hop-by-hop decision and therefore any
reactive fragmentation coordination signalling must be either carried
in a non-primary block, or pushed down to the convergence layer (as it
was before).

I'm starting to wonder whether there is benefit in putting some flag(s)
back into the contact header that defines what both parties at the CL
layer should do with partial bundle delivery. So far I can imagine the
following flag values:

'atomic': Bundle transmission will be resumed from the start on next
contact, no need to cache anything.

'resume': Bundle transmission will resume from the first failed segment
on next contact, so the receiver SHOULD retain the segments it already
has.  (This is a store-and-forward mechanism running beneath BP, is
that desired?)

'fragment': The receiver SHOULD create a new first bundle fragment from
what it has already received (and forward?), and a new second bundle
fragment will be transmitted on the next contact. This value MUST NOT
be used when the bundle control flags prevent fragmentation.

If the error codes of TCP-CL allow an implementation to say
"Unsupported Flag" for anything apart from 'atomic', I think we might
have a way forwards.

Thoughts?

Rick

On Mon, 2018-01-29 at 17:53 +0000, Brian Sipos wrote:
> This missing negotiation is a side-effect of my original
> misunderstanding then. I had removed those other negotiated
> parameters/flags and made those other behaviors mandatory to simply
> the protocol. But this fragmentation one is different because it's
> not really the CL itself doing anything with it, but being passed-up
> to the BP agent for its use. Even RFC 7242 really doesn't say what is
> supposed to be done when reactive fragmentation is negotiated as
> enabled.
>
> A question then for the new CL spec: is the symmetric behavior of
> this negotiated TCPCL session parameter in RFC 7242 desirable? So
> that both nodes must support fragmentation in both directions for it
> to be usable in either direction.
>
> It is a relatively straightforward thing to add a contact header flag
> to Section 4.2 and add a new "Reactive Fragmentation Enabled" session
> parameter to Section 4.3. It's still not specifying exactly what the
> BP agent would do with such information, but at least it closes the
> gap that the information is available (as it was in TCPCLv3).
>
> From: Burleigh, Scott C (312B) <[email-masked]>
> Sent: Monday, January 29, 2018 12:19:15 PM
> To: Rick Taylor; [email-masked]
> Cc: Brian Sipos
> Subject: RE: [dtn] TCP-CL and XFER_SEGMENT
>
> Ah, good question.  In RFC 7242 there was a flag in the contact
> header; agreement was reached during contact header exchange.  I have
> not been tracking this thread closely, so I missed the discussion of
> the contact header revisions.
>
> Scott
>
> -----Original Message-----
> From: Rick Taylor [mailto:[email-masked]]
> Sent: Monday, January 29, 2018 8:59 AM
> To: [email-masked]; Burleigh, Scott C (312B) <[email-masked]
> .gov>
> Cc: [email-masked]
> Subject: Re: [dtn] TCP-CL and XFER_SEGMENT
>
> Hi Scott, All,
>
> Thanks for that explanation, it ties with how I imagine one would
> implement reactive fragmentation.
>
> However, I still have an issue that there is no signalling defined
> anywhere I can find (and I may have overlooked a reference) that
> describes how the sending BPA and next-hop BPA agree that the partial
> delivery of the bundle is going to be resumed using fragments.
>
> Without any defined signalling, I can't see how my reactive
> fragmentation will interoperate with your reactive fragmentation (for
> example).
>
> My questions to the list are therefore:
>
> 1) How do 2 BPAs agree that a partial transmission of a bundle should
> be completed by dividing the bundle into the sent part and the unsent
> part, so they can reassembled downstream?
>
> 2) Should the TCP-CL draft mention any of this?
>
> Sorry, I'm not trying to be difficult, but I believe we are eluding
> to functionality that isn't defined anywhere, and I think we should
> either define it, or not mention it.
>
> Cheers,
>
> Rick
>
> On Sun, 2018-01-28 at 18:44 +0000, Burleigh, Scott C (312B) wrote:
> > Rick, I can describe how ION would do it, but of course that is not
> a
> > general answer; all I would offer is what I think is an existence
> > proof that additional customized protocol is not necessary.
> >
> > Briefly, in ION you proactively fragment a bundle of payload length
> Q
> > - into one bundle carrying the first N bytes of payload (N < Q) and
> a
> > second carrying the last Q - N bytes of payload - by making a copy
> of
> > the bundle (in its internal representation), creating two zero-
> copy
> > objects containing the first N and last Q - N bytes of the
> original
> > payload ZCO, delete the original bundle and its payload ZCO, and
> > inserting both new fragmentary bundles into the shared-memory
> linked
> > list of bundles awaiting disposition by the forwarding daemon.
> This
> > fragmentation may occur any point, and at any number of points, in
> the
> > end-to-end path.
> >
> > If ION did reactive fragmentation, I think the sending node would
> > follow the same procedure except that only the fragment carrying
> the
> > last Q - N bytes would be inserted into the linked list; the other
> > fragment would never be created.  The receiving node would form a
> new
> > ZCO containing the first N bytes of the original payload, would
> create
> > a new bundle whose payload is that ZCO, and would insert that
> bundle
> > into its own forwarding queue.
> >
> > Segmentation tracking in TCPCL would enable this procedure: upon
> > detecting connection failure, both the sending and receiving BPAs
> > would use the reception progress offset reported in the last ACK
> > passed from receiver to sender as the value of N, regardless of
> how
> > many additional bytes might actually have been received since that
> ACK
> > was sent.  Anything that the receiver acquired after the first N
> bytes
> > would be wasted transmission that would have to be re-sent (it
> would
> > be at the front of the sender's Q - N fragment), but the sender
> would
> > not need to retransmit the first N bytes of the payload.  The only
> > protocol that is needed is TCPCL, plus detection of TCP connection
> > failure.
> >
> > I think the bpsec spec now says that security blocks only apply to
> the
> > original bundle, not to any fragments, so checking the signature
> or
> > decrypting the payload would in any case only happen at the final
> > destination node upon reassembly of the original bundle from all
> > fragments.  If you did want to sign and/or encrypt a fragmentary
> > payload you could use bundle-in-bundle encapsulation - the
> fragmentary
> > bundle would become the complete payload of a new original bundle.
> >
> > Scott
> >
> > -----Original Message-----
> > From: Rick Taylor [mailto:[email-masked]]
> > Sent: Sunday, January 28, 2018 3:25 AM
> > To: Burleigh, Scott C (312B) <[email-masked]>; dtn@i
> et
> > f.org
> > Cc: Brian Sipos ([email-masked]) <[email-masked]>
> > Subject: RE: TCP-CL and XFER_SEGMENT
> >
> > Hi Scott, All,
> >
> > (Chair hat off)
> >
> > Ironically, I am a fan of reactive fragmentation, but I don't
> think
> > TCP-CL segmentation actually helps.
> >
> > For example, if the transmitting CL reports back to its
> corresponding
> > BPA that 45% of a bundle has been received by the other end, the
> > assumption is that that BPA could react and do some kind of
> > fragmentation to avoid sending the first 45% again.  My issue is
> how?
> >
> > 1) The receiving TCP-CL only has half of what it was expecting.
> How
> > does it know to tell the receiving BPA to retain this, as
> > fragmentation might be occurring.
> > 2) The sending BPA could just split the original bundle at 45%, pop
> on
> > a new header and send that.  However, the receiver now only has
> the
> > second of two fragments. How does this new bundle correlate to the
> > half-a-bundle it's already received.
> > 3) How does any of this 'divide and resend the missing bits' work
> with
> > BPSec integrity checking?
> > 4) I just can't see how one avoids resending the first 45% again as
> a
> > new bundle without changing BPbis.
> >
> > I think there needs to be some consensus over what 'reactive' or
> > 'proactive' fragmentation actually mean, and to my mind, 'reactive'
> > fragmentation has to mean:  The process whereby a bundle is
> possibly
> > fragmented by one or more intermediate nodes along the path from
> > source to destination.  And 'proactive' fragmentation is the
> process
> > whereby a bundle may be fragmented by the sender, to optimize the
> > transmission of the bundle payload from source to destination.
> What
> > we are discussing with TCP-CL segmentation is 'retrospective'
> > fragmentation, which is the process of fragmenting a bundle after
> only
> > a fraction of it has already been received by the next hop.
> >
> > TCP-CL can provide useful data to the BPA to use in implementing
> > reactive fragmentation, such as the Transfer MRU, and the
> transmission
> > success of bundles can be sampled to provide probabilistic
> information
> > about 'what kind of bundles and when'.  This data can drive the
> > reactive fragmentation decision in the BPA, but the presence of
> > segmentation doesn't help.
> >
> > I understand that people probably have running code that does
> > retrospective fragmentation based on feedback from the TCP layer,
> but
> > I imagine it is using earlier/custom protocols, and I would be
> very
> > interested to hear how they think it will work with BPbis and
> BPSec?
> > And if it does, how does it avoid re-sending the original segments
> > that arrived prior to failure.
> >
> > In summary:  Reactive fragmentation is nice to have, and I like
> the
> > idea, but TCP-CL segmentation doesn't actually help provide it.  I
> > just get the feeling that segmentation in TCP-CL makes the CL start
> to
> > look like another store-and-forward mechanism sitting underneath
> BP,
> > and I think that is a mistake.
> >
> > Cheers,
> >
> > Rick
> >
> > > -----Original Message-----
> > > From: dtn [mailto:[email-masked]] On Behalf Of Burleigh,
> Scott
> > > C
> > > (312B)
> > > Sent: 28 January 2018 00:39
> > > To: Rick Taylor; [email-masked]
> > > Cc: Brian Sipos ([email-masked])
> > > Subject: Re: [dtn] TCP-CL and XFER_SEGMENT
> > >
> > > Hi, Rick.  Doubtful as I am of the utility of reactive
> > > fragmentation, if we are going to retain this apparently popular
> > > feature then I'd say we have to continue tracking segmentation
> in
> > > TCPCL.  The periodic, incremental, progressive segment
> > > acknowledgments returned by the receiving node are the only way
> the
> > > sending node can determine the reactive fragmentation point
> (i.e.,
> > > how many leading bytes of the payload must be assumed to have
> been
> > > used by the receiving node to form the received fragment, such
> that
> > > all remaining bytes must be used by the sending node to form the
> > > fragment that needs to be re-forwarded).  TCPCL shouldn't
> perform
> > > the bundle fragmentation itself, but it seems valid for TCPCL to
> > > tell the BPA when and where to fragment.
> > >
> > > So long as it's the BPA itself that always performs the
> > > fragmentation, I think the security questions are covered by the
> > > bpsec specification.
> > > Certainly you're right that eliminating segmentation tracking in
> > > TCPCL would simplify the CL, but I think it will be argued that
> the
> > > reactive fragmentation feature is valuable enough to justify the
> > > additional complexity and overhead.
> > >
> > > Scott
> > >
> > > -----Original Message-----
> > > From: dtn [mailto:[email-masked]] On Behalf Of Rick Taylor
> > > Sent: Saturday, January 27, 2018 5:29 AM
> > > To: [email-masked]
> > > Cc: Brian Sipos ([email-masked]) <[email-masked]>
> > > Subject: [dtn] TCP-CL and XFER_SEGMENT
> > >
> > > Hi All,
> > >
> > > (Chair Hat Off)
> > >
> > > I'm just having another look at TCP-CL and trying to catch up on
> the
> > > comments, and I have a concern about the acknowledged segments
> > > approach in the protocol.
> > >
> > > Given the Bundle Protocol documents do not (intentionally)
> mention
> > > how to do fragmentation, and I believe that there is rough
> consensus
> > > that bundle fragmentation is really the responsibility of the
> BPA,
> > > not the CL, I wonder whether the segmentation algorithms in TCP-
> CL
> > > need to be there at all?
> > >
> > > In my opinion it would be simpler to consider a bundle transfer
> over
> > > TCP-CL (or any other CL) to be an atomic operation:  a BPA
> passes
> > > the next output- queue bundle to the CL to be sent, and it is
> either
> > > sent in its entirety or fails in its entirety, with only a
> > > processing and transmission delay between start of send and
> result.
> > > I.e. the BPA retains 'ownership' of the bundle until the CL says
> it
> > > has been transferred, rather than the CL taking ownership from
> the
> > > BPA.
> > >
> > > The alternate, hinted at in the draft, is that the BPA passes a
> > > bundle to the TCP-CL, and sometime later, the CL announces only
> some
> > > of the bundle has been successfully transmitted.  How is the BPA
> > > supposed to react to this information?  How does the BPA decide
> to
> > > fragment and retransmit a bundle that has been fragmented at a
> lower
> > > layer (the
> > > CL)
> > > in a manner beyond its control, or is the BPA required to have
> > > specific knowledge of the inner workings of the CL, which will
> be
> > > different for every CL?
> > >
> > > I would suggest that a cleaner/simpler approach would be for the
> CL
> > > to refuse/fail a transmission with a response such as "Bundle
> too
> > > big to send" or "Peer hung up early", and let the BPA decide what
> to
> > > do about fragmentation.  To this end, I would remove
> XFER_SEGMENT
> > > altogether, and a have a single XFER_INIT, XFER_ACK pair to
> bracket
> > > a single atomic bundle transmission.  This would simplify some
> of
> > > the text around transmission ordering and pipelining as well.
> > >
> > > I also have a concern that the combination of BPSec and
> > > fragmentation is tricky to get right - by adding TCP-CL level
> > > segmentation, can I as a malicious attacker keep re-requesting a
> > > certain byte-range of encrypted bundles, sampling for a
> potential
> > > cryptographic dictionary attack, with the BPA (that is managing
> the
> > > cryptography) being unaware?  I.e. are we opening ourselves up
> to
> > > replay attacks?
> > >
> > > In summary, I think the recent valid debate around segment
> transfer
> > > acknowledgement and bundle fragmentation is leading me towards
> > > suggesting a 'Keep It Simple' approach of removing fragmentation
> in
> > > TCP-CL and leaving it to the BPA to handle, when there is a
> clearer
> > > idea about how to do bundle fragmentation.
> > >
> > > Thoughts?
> > >
> > > Rick
> > >
> > > _______________________________________________
> > > dtn mailing list
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/dtn
> > >
> > > _______________________________________________
> > > dtn mailing list
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/dtn
> >
> > _______________________________________________
> > dtn mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/dtn

---

From: Brian Sipos <[email-masked]>
Date: Tue, 30 Jan 2018 22:27:20 +0000

Rick,

One thing about your example, from the TCPCL perspective, is that there is no real concept of a "bundle" as such: each transfer is purely opaque octet string for the TCPCL. So the CL doesn't really know the true "bundle length" only the one-hop "transfer length" and in case the concatenated-XFER_SEGMENT-data length is less than the XFER_INIT prescribed length (as your pseudocode indicates) it really amounts to a failed TCPCL transfer.

Any bookkeeping of reactive fragmentation cannot occur at the CL itself anyway. By the time a bundle has failed partway through transfer, the reason why it would have failed would have also caused the TCPCL session itself to fail and at that point the CL has nothing to do; TCPCL can only transfer on live sessions, not on potential future sessions. It is up to the BP agent to establish a replacement TCPCL session and then do 'something' in that new session. And in fact, a reactively fragmented bundle need not have its subsequent fragments sent over the same CL type...

________________________________
From: Rick Taylor <[email-masked]>
Sent: Tuesday, January 30, 2018 3:47:55 PM
To: Brian Sipos; [email-masked]; Scott C Burleigh, (312B)
Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

Brian, Scott

Cheers
Rick

(Sent from my phone)

----- Reply message -----
From: "Brian Sipos" <[email-masked]>
To: "Rick Taylor" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: [dtn] TCP-CL and XFER_SEGMENT
Date: Tue, Jan 30, 2018 18:38

Rick,

Hi All,

(Chair Hat Off)

Thoughts?

Rick