Subject: [dtn] TCP-CL and XFER_SEGMENT

From: Rick Taylor <[email-masked]>
Date: Sat, 27 Jan 2018 13:28:50 +0000

Hi All,

(Chair Hat Off)

I'm just having another look at TCP-CL and trying to catch up on the comments, and I have a concern about the acknowledged segments approach in the protocol.

Given the Bundle Protocol documents do not (intentionally) mention how to do fragmentation, and I believe that there is rough consensus that bundle fragmentation is really the responsibility of the BPA, not the CL, I wonder whether the segmentation algorithms in TCP-CL need to be there at all?

In my opinion it would be simpler to consider a bundle transfer over TCP-CL (or any other CL) to be an atomic operation:  a BPA passes the next output-queue bundle to the CL to be sent, and it is either sent in its entirety or fails in its entirety, with only a processing and transmission delay between start of send and result.  I.e. the BPA retains 'ownership' of the bundle until the CL says it has been transferred, rather than the CL taking ownership from the BPA.

The alternate, hinted at in the draft, is that the BPA passes a bundle to the TCP-CL, and sometime later, the CL announces only some of the bundle has been successfully transmitted.  How is the BPA supposed to react to this information?  How does the BPA decide to fragment and retransmit a bundle that has been fragmented at a lower layer (the CL) in a manner beyond its control, or is the BPA required to have specific knowledge of the inner workings of the CL, which will be different for every CL?

I would suggest that a cleaner/simpler approach would be for the CL to refuse/fail a transmission with a response such as "Bundle too big to send" or "Peer hung up early", and let the BPA decide what to do about fragmentation.  To this end, I would remove XFER_SEGMENT altogether, and a have a single XFER_INIT, XFER_ACK pair to bracket a single atomic bundle transmission.  This would simplify some of the text around transmission ordering and pipelining as well.

I also have a concern that the combination of BPSec and fragmentation is tricky to get right - by adding TCP-CL level segmentation, can I as a malicious attacker keep re-requesting a certain byte-range of encrypted bundles, sampling for a potential cryptographic dictionary attack, with the BPA (that is managing the cryptography) being unaware?  I.e. are we opening ourselves up to replay attacks?

In summary, I think the recent valid debate around segment transfer acknowledgement and bundle fragmentation is leading me towards suggesting a 'Keep It Simple' approach of removing fragmentation in TCP-CL and leaving it to the BPA to handle, when there is a clearer idea about how to do bundle fragmentation.

Thoughts?

Rick

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Sun, 28 Jan 2018 00:39:13 +0000

Hi, Rick.  Doubtful as I am of the utility of reactive fragmentation, if we are going to retain this apparently popular feature then I'd say we have to continue tracking segmentation in TCPCL.  The periodic, incremental, progressive segment acknowledgments returned by the receiving node are the only way the sending node can determine the reactive fragmentation point (i.e., how many leading bytes of the payload must be assumed to have been used by the receiving node to form the received fragment, such that all remaining bytes must be used by the sending node to form the fragment that needs to be re-forwarded).  TCPCL shouldn't perform the bundle fragmentation itself, but it seems valid for TCPCL to tell the BPA when and where to fragment.

So long as it's the BPA itself that always performs the fragmentation, I think the security questions are covered by the bpsec specification.  Certainly you're right that eliminating segmentation tracking in TCPCL would simplify the CL, but I think it will be argued that the reactive fragmentation feature is valuable enough to justify the additional complexity and overhead.

Scott

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of Rick Taylor
Sent: Saturday, January 27, 2018 5:29 AM
To: [email-masked]
Cc: Brian Sipos ([email-masked]) <[email-masked]>
Subject: [dtn] TCP-CL and XFER_SEGMENT

Hi All,

(Chair Hat Off)

Thoughts?

Rick

_______________________________________________
dtn mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dtn

---

From: Rick Taylor <[email-masked]>
Date: Sun, 28 Jan 2018 11:25:08 +0000

Hi Scott, All,

(Chair hat off) 

Ironically, I am a fan of reactive fragmentation, but I don't think TCP-CL segmentation actually helps.

For example, if the transmitting CL reports back to its corresponding BPA that 45% of a bundle has been received by the other end, the assumption is that that BPA could react and do some kind of fragmentation to avoid sending the first 45% again.  My issue is how?

1) The receiving TCP-CL only has half of what it was expecting.  How does it know to tell the receiving BPA to retain this, as fragmentation might be occurring.
2) The sending BPA could just split the original bundle at 45%, pop on a new header and send that.  However, the receiver now only has the second of two fragments. How does this new bundle correlate to the half-a-bundle it's already received.
3) How does any of this 'divide and resend the missing bits' work with BPSec integrity checking?
4) I just can't see how one avoids resending the first 45% again as a new bundle without changing BPbis.

I think there needs to be some consensus over what 'reactive' or 'proactive' fragmentation actually mean, and to my mind, 'reactive' fragmentation has to mean:  The process whereby a bundle is possibly fragmented by one or more intermediate nodes along the path from source to destination.  And 'proactive' fragmentation is the process whereby a bundle may be fragmented by the sender, to optimize the transmission of the bundle payload from source to destination.  What we are discussing with TCP-CL segmentation is 'retrospective' fragmentation, which is the process of fragmenting a bundle after only a fraction of it has already been received by the next hop.

TCP-CL can provide useful data to the BPA to use in implementing reactive fragmentation, such as the Transfer MRU, and the transmission success of bundles can be sampled to provide probabilistic information about 'what kind of bundles and when'.  This data can drive the reactive fragmentation decision in the BPA, but the presence of segmentation doesn't help.  

I understand that people probably have running code that does retrospective fragmentation based on feedback from the TCP layer, but I imagine it is using earlier/custom protocols, and I would be very interested to hear how they think it will work with BPbis and BPSec?  And if it does, how does it avoid re-sending the original segments that arrived prior to failure.

In summary:  Reactive fragmentation is nice to have, and I like the idea, but TCP-CL segmentation doesn't actually help provide it.  I just get the feeling that segmentation in TCP-CL makes the CL start to look like another store-and-forward mechanism sitting underneath BP, and I think that is a mistake.

Cheers,

Rick

> From: dtn [mailto:[email-masked]] On Behalf Of Burleigh, Scott C
> (312B)
> Sent: 28 January 2018 00:39
> To: Rick Taylor; [email-masked]
> Cc: Brian Sipos ([email-masked])
> Subject: Re: [dtn] TCP-CL and XFER_SEGMENT
> 
> Hi, Rick.  Doubtful as I am of the utility of reactive fragmentation, if we are
> going to retain this apparently popular feature then I'd say we have to
> continue tracking segmentation in TCPCL.  The periodic, incremental,
> progressive segment acknowledgments returned by the receiving node are
> the only way the sending node can determine the reactive fragmentation
> point (i.e., how many leading bytes of the payload must be assumed to have
> been used by the receiving node to form the received fragment, such that all
> remaining bytes must be used by the sending node to form the fragment
> that needs to be re-forwarded).  TCPCL shouldn't perform the bundle
> fragmentation itself, but it seems valid for TCPCL to tell the BPA when and
> where to fragment.
> 
> So long as it's the BPA itself that always performs the fragmentation, I think
> the security questions are covered by the bpsec specification.  Certainly
> you're right that eliminating segmentation tracking in TCPCL would simplify
> the CL, but I think it will be argued that the reactive fragmentation feature is
> valuable enough to justify the additional complexity and overhead.
> 
> Scott
> 
> I'm just having another look at TCP-CL and trying to catch up on the
> comments, and I have a concern about the acknowledged segments
> approach in the protocol.
> 
> Given the Bundle Protocol documents do not (intentionally) mention how to
> do fragmentation, and I believe that there is rough consensus that bundle
> fragmentation is really the responsibility of the BPA, not the CL, I wonder
> whether the segmentation algorithms in TCP-CL need to be there at all?
> 
> In my opinion it would be simpler to consider a bundle transfer over TCP-CL
> (or any other CL) to be an atomic operation:  a BPA passes the next output-
> queue bundle to the CL to be sent, and it is either sent in its entirety or fails
> in its entirety, with only a processing and transmission delay between start of
> send and result.  I.e. the BPA retains 'ownership' of the bundle until the CL
> says it has been transferred, rather than the CL taking ownership from the
> BPA.
> 
> The alternate, hinted at in the draft, is that the BPA passes a bundle to the
> TCP-CL, and sometime later, the CL announces only some of the bundle has
> been successfully transmitted.  How is the BPA supposed to react to this
> information?  How does the BPA decide to fragment and retransmit a bundle
> that has been fragmented at a lower layer (the CL) in a manner beyond its
> control, or is the BPA required to have specific knowledge of the inner
> workings of the CL, which will be different for every CL?
> 
> I would suggest that a cleaner/simpler approach would be for the CL to
> refuse/fail a transmission with a response such as "Bundle too big to send" or
> "Peer hung up early", and let the BPA decide what to do about
> fragmentation.  To this end, I would remove XFER_SEGMENT altogether, and
> a have a single XFER_INIT, XFER_ACK pair to bracket a single atomic bundle
> transmission.  This would simplify some of the text around transmission
> ordering and pipelining as well.
> 
> I also have a concern that the combination of BPSec and fragmentation is
> tricky to get right - by adding TCP-CL level segmentation, can I as a malicious
> attacker keep re-requesting a certain byte-range of encrypted bundles,
> sampling for a potential cryptographic dictionary attack, with the BPA (that is
> managing the cryptography) being unaware?  I.e. are we opening ourselves
> up to replay attacks?
> 
> In summary, I think the recent valid debate around segment transfer
> acknowledgement and bundle fragmentation is leading me towards
> suggesting a 'Keep It Simple' approach of removing fragmentation in TCP-CL
> and leaving it to the BPA to handle, when there is a clearer idea about how to
> do bundle fragmentation.
> 
> Thoughts?
> 
> Rick
>

---

From: Carlo Caini <[email-masked]>
Date: Sun, 28 Jan 2018 18:41:40 +0000

Dear Rick,
     BP proactive and reactive fragmentations are clearly defined in RFC4838. Let me cut and paste here these definitions here:

>From RFC4838
Fragmentation and Reassembly

   DTN fragmentation and reassembly are designed to improve the
   efficiency of bundle transfers by ensuring that contact volumes are
   fully utilized and by avoiding retransmission of partially-forwarded
   bundles.  There are two forms of DTN fragmentation/reassembly:

   Proactive Fragmentation

      A DTN node may divide a block of application data into multiple
      smaller blocks and transmit each such block as an independent
      bundle.  In this case, the *final destination(s)* are responsible
      for extracting the smaller blocks from incoming bundles and
      reassembling them into the original larger bundle and, ultimately,
      ADU.  This approach is called proactive fragmentation because it
      is used primarily when contact volumes are known (or predicted) in
      advance.

   Reactive Fragmentation

      DTN nodes sharing an edge in the DTN graph may fragment a bundle
      cooperatively when a bundle is only partially transferred.  In
      this case, the receiving bundle layer modifies the incoming bundle
      to indicate it is a fragment, and forwards it normally.  The
      previous- hop sender may learn (via convergence-layer protocols,
      see Section 6) that only a portion of the bundle was delivered to
      the next hop, and send the remaining portion(s) when subsequent
      contacts become available (possibly to different next-hops if
      routing changes).  This is called reactive fragmentation because
      the fragmentation process occurs after an attempted transmission
      has taken place.

An now your definition of proactive fragmentation:
"And 'proactive' fragmentation is the process whereby a bundle may be fragmented by the sender, to optimize the transmission of the bundle payload from source to destination."

>From the RFC text it is clear that proactive fragmentation can be carried out by whatever DTN node in the path from source to desination, not necessarily by the source, as your proposed definition seems to suggest. Moreover, reactive fragmentation is not carried out to optimize anythig from spurce to destination, but it is carried out by a node (which can either be the source or an intermediate node) by considering what is going to happen in the hop departing from it. E.g. if the residal contact volume of a given contact departing from the current node (not necessarily the source)  it is not large enough to make it possible the tranmission of the current bundle, this bundle may be fragmented.

Your definition of reactive fragmentation:
"to my mind, 'reactive' fragmentation has to mean:  The process whereby a bundle is possibly fragmented by one or more intermediate nodes along the path from source to destination."

This does not match with RFC4838 definition at all.

In conclusions, I fully agree with you that we need clear and unambigous definitions. This is a must all well written standards should comply with.  Concerning fragmentation, in my opinion the definitions from both reactive and proactive given in RFC4838, which is not gooing to be obsoleted and it is also very well written, should be preserved. In my oipinion they are simply perfect.
In my experience reactive implemented in DTN2 TCPCL does exactly what is intended in RFC4838, and works fine (I do not remeber any bug in 12 years of tests). Moreover, bundles obtained by means of reactive fragmentation between two DTN2 nodes have always been correctly reassembled by either DTN2 or ION detsinations, as required by RFC4838, which let implementation free of implementing or not fragmentation, but makes compulsory bundle reassembling, which seems a very sensible policy.  I have less experience on proactive, but the ION code, where it is implemented, does exactly what is said in RFC4838. Every intermediate node can carry out this. 

Yours,
   Carlo 

________________________________________
Da: dtn [[email-masked]] per conto di Rick Taylor [[email-masked]]
Inviato: domenica 28 gennaio 2018 12:25
A: Burleigh, Scott C (312B); [email-masked]
Cc: Brian Sipos ([email-masked])
Oggetto: Re: [dtn] TCP-CL and XFER_SEGMENT

Hi Scott, All,

(Chair hat off)

Cheers,

Rick

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Sun, 28 Jan 2018 18:44:10 +0000

Rick, I can describe how ION would do it, but of course that is not a general answer; all I would offer is what I think is an existence proof that additional customized protocol is not necessary.

Briefly, in ION you proactively fragment a bundle of payload length Q - into one bundle carrying the first N bytes of payload (N < Q) and a second carrying the last Q - N bytes of payload - by making a copy of the bundle (in its internal representation), creating two zero-copy objects containing the first N and last Q - N bytes of the original payload ZCO, delete the original bundle and its payload ZCO, and inserting both new fragmentary bundles into the shared-memory linked list of bundles awaiting disposition by the forwarding daemon.  This fragmentation may occur any point, and at any number of points, in the end-to-end path.

If ION did reactive fragmentation, I think the sending node would follow the same procedure except that only the fragment carrying the last Q - N bytes would be inserted into the linked list; the other fragment would never be created.  The receiving node would form a new ZCO containing the first N bytes of the original payload, would create a new bundle whose payload is that ZCO, and would insert that bundle into its own forwarding queue.

Segmentation tracking in TCPCL would enable this procedure: upon detecting connection failure, both the sending and receiving BPAs would use the reception progress offset reported in the last ACK passed from receiver to sender as the value of N, regardless of how many additional bytes might actually have been received since that ACK was sent.  Anything that the receiver acquired after the first N bytes would be wasted transmission that would have to be re-sent (it would be at the front of the sender's Q - N fragment), but the sender would not need to retransmit the first N bytes of the payload.  The only protocol that is needed is TCPCL, plus detection of TCP connection failure.

I think the bpsec spec now says that security blocks only apply to the original bundle, not to any fragments, so checking the signature or decrypting the payload would in any case only happen at the final destination node upon reassembly of the original bundle from all fragments.  If you did want to sign and/or encrypt a fragmentary payload you could use bundle-in-bundle encapsulation - the fragmentary bundle would become the complete payload of a new original bundle.

Scott 

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Sunday, January 28, 2018 3:25 AM
To: Burleigh, Scott C (312B) <[email-masked]>; [email-masked]
Cc: Brian Sipos ([email-masked]) <[email-masked]>
Subject: RE: TCP-CL and XFER_SEGMENT

Hi Scott, All,

(Chair hat off) 

Cheers,

Rick

> Hi, Rick.  Doubtful as I am of the utility of reactive fragmentation, 
> if we are going to retain this apparently popular feature then I'd say 
> we have to continue tracking segmentation in TCPCL.  The periodic, 
> incremental, progressive segment acknowledgments returned by the 
> receiving node are the only way the sending node can determine the 
> reactive fragmentation point (i.e., how many leading bytes of the 
> payload must be assumed to have been used by the receiving node to 
> form the received fragment, such that all remaining bytes must be used 
> by the sending node to form the fragment that needs to be 
> re-forwarded).  TCPCL shouldn't perform the bundle fragmentation 
> itself, but it seems valid for TCPCL to tell the BPA when and where to fragment.
> 
> So long as it's the BPA itself that always performs the fragmentation, 
> I think the security questions are covered by the bpsec specification.  
> Certainly you're right that eliminating segmentation tracking in TCPCL 
> would simplify the CL, but I think it will be argued that the reactive 
> fragmentation feature is valuable enough to justify the additional complexity and overhead.
> 
> Scott
> 
> Given the Bundle Protocol documents do not (intentionally) mention how 
> to do fragmentation, and I believe that there is rough consensus that 
> bundle fragmentation is really the responsibility of the BPA, not the 
> CL, I wonder whether the segmentation algorithms in TCP-CL need to be there at all?
> 
> In my opinion it would be simpler to consider a bundle transfer over 
> TCP-CL (or any other CL) to be an atomic operation:  a BPA passes the 
> next output- queue bundle to the CL to be sent, and it is either sent 
> in its entirety or fails in its entirety, with only a processing and 
> transmission delay between start of send and result.  I.e. the BPA 
> retains 'ownership' of the bundle until the CL says it has been 
> transferred, rather than the CL taking ownership from the BPA.
> 
> The alternate, hinted at in the draft, is that the BPA passes a bundle 
> to the TCP-CL, and sometime later, the CL announces only some of the 
> bundle has been successfully transmitted.  How is the BPA supposed to 
> react to this information?  How does the BPA decide to fragment and 
> retransmit a bundle that has been fragmented at a lower layer (the CL) 
> in a manner beyond its control, or is the BPA required to have 
> specific knowledge of the inner workings of the CL, which will be different for every CL?
> 
> refuse/fail a transmission with a response such as "Bundle too big to 
> send" or "Peer hung up early", and let the BPA decide what to do about 
> fragmentation.  To this end, I would remove XFER_SEGMENT altogether, 
> and a have a single XFER_INIT, XFER_ACK pair to bracket a single 
> atomic bundle transmission.  This would simplify some of the text 
> around transmission ordering and pipelining as well.
> 
> I also have a concern that the combination of BPSec and fragmentation 
> is tricky to get right - by adding TCP-CL level segmentation, can I as 
> a malicious attacker keep re-requesting a certain byte-range of 
> encrypted bundles, sampling for a potential cryptographic dictionary 
> attack, with the BPA (that is managing the cryptography) being 
> unaware?  I.e. are we opening ourselves up to replay attacks?
> 
> suggesting a 'Keep It Simple' approach of removing fragmentation in 
> TCP-CL and leaving it to the BPA to handle, when there is a clearer 
> idea about how to do bundle fragmentation.
> 
> Thoughts?
> 
> Rick
>

---

From: Rick Taylor <[email-masked]>
Date: Sun, 28 Jan 2018 18:47:57 +0000

Hi Carlo,

Thanks for the clarification, it's been a while since I last read RFC4838 and I had forgotten that the definitions were there.  I agree with you the text in the RFC is good and clear, so I retract my less good definitions.

However, I still regard the behaviour some people suggest XFER_SEGMENT enables as neither 'proactive' or 'reactive', as the fragmentation is occurring after a bundle has started transmission, and the fragmentation decision is being made by the CL, not the BPA.  This 'retrospective' fragmentation is the cause of my concern.

Cheers,

Rick

> From: Carlo Caini [mailto:[email-masked]]
> Sent: 28 January 2018 18:42
> To: Rick Taylor; Burleigh, Scott C (312B); [email-masked]
>      BP proactive and reactive fragmentations are clearly defined in RFC4838.
> Let me cut and paste here these definitions here:
> 
> From RFC4838
> "And 'proactive' fragmentation is the process whereby a bundle may be
> fragmented by the sender, to optimize the transmission of the bundle
> payload from source to destination."
> 
> From the RFC text it is clear that proactive fragmentation can be carried out
> by whatever DTN node in the path from source to desination, not necessarily
> by the source, as your proposed definition seems to suggest. Moreover,
> reactive fragmentation is not carried out to optimize anythig from spurce to
> destination, but it is carried out by a node (which can either be the source or
> an intermediate node) by considering what is going to happen in the hop
> departing from it. E.g. if the residal contact volume of a given contact
> departing from the current node (not necessarily the source)  it is not large
> enough to make it possible the tranmission of the current bundle, this
> bundle may be fragmented.
> 
> "to my mind, 'reactive' fragmentation has to mean:  The process whereby a
> bundle is possibly fragmented by one or more intermediate nodes along the
> path from source to destination."
> 
> In conclusions, I fully agree with you that we need clear and unambigous
> definitions. This is a must all well written standards should comply with.
> Concerning fragmentation, in my opinion the definitions from both reactive
> and proactive given in RFC4838, which is not gooing to be obsoleted and it is
> also very well written, should be preserved. In my oipinion they are simply
> perfect.
> In my experience reactive implemented in DTN2 TCPCL does exactly what is
> intended in RFC4838, and works fine (I do not remeber any bug in 12 years of
> tests). Moreover, bundles obtained by means of reactive fragmentation
> between two DTN2 nodes have always been correctly reassembled by either
> DTN2 or ION detsinations, as required by RFC4838, which let implementation
> free of implementing or not fragmentation, but makes compulsory bundle
> reassembling, which seems a very sensible policy.  I have less experience on
> proactive, but the ION code, where it is implemented, does exactly what is
> said in RFC4838. Every intermediate node can carry out this.
> 
> Yours,
>    Carlo
> 
> 
> For example, if the transmitting CL reports back to its corresponding BPA that
> 45% of a bundle has been received by the other end, the assumption is that
> that BPA could react and do some kind of fragmentation to avoid sending the
> first 45% again.  My issue is how?
> 
> 1) The receiving TCP-CL only has half of what it was expecting.  How does it
> know to tell the receiving BPA to retain this, as fragmentation might be
> occurring.
> 2) The sending BPA could just split the original bundle at 45%, pop on a new
> header and send that.  However, the receiver now only has the second of
> two fragments. How does this new bundle correlate to the half-a-bundle it's
> already received.
> I think there needs to be some consensus over what 'reactive' or 'proactive'
> fragmentation actually mean, and to my mind, 'reactive' fragmentation has to
> mean:  The process whereby a bundle is possibly fragmented by one or more
> intermediate nodes along the path from source to destination.  And
> 'proactive' fragmentation is the process whereby a bundle may be
> fragmented by the sender, to optimize the transmission of the bundle
> payload from source to destination.  What we are discussing with TCP-CL
> segmentation is 'retrospective' fragmentation, which is the process of
> fragmenting a bundle after only a fraction of it has already been received by
> the next hop.
> 
> TCP-CL can provide useful data to the BPA to use in implementing reactive
> fragmentation, such as the Transfer MRU, and the transmission success of
> bundles can be sampled to provide probabilistic information about 'what kind
> of bundles and when'.  This data can drive the reactive fragmentation
> decision in the BPA, but the presence of segmentation doesn't help.
> 
> I understand that people probably have running code that does
> retrospective fragmentation based on feedback from the TCP layer, but I
> imagine it is using earlier/custom protocols, and I would be very interested to
> hear how they think it will work with BPbis and BPSec?  And if it does, how
> does it avoid re-sending the original segments that arrived prior to failure.
> 
> In summary:  Reactive fragmentation is nice to have, and I like the idea, but
> TCP-CL segmentation doesn't actually help provide it.  I just get the feeling
> that segmentation in TCP-CL makes the CL start to look like another store-
> and-forward mechanism sitting underneath BP, and I think that is a mistake.
> 
> Cheers,
> 
> Rick
>

---

From: Rick Taylor <[email-masked]>
Date: Mon, 29 Jan 2018 16:58:35 +0000

Hi Scott, All,

Thanks for that explanation, it ties with how I imagine one would
implement reactive fragmentation.

However, I still have an issue that there is no signalling defined
anywhere I can find (and I may have overlooked a reference) that
describes how the sending BPA and next-hop BPA agree that the partial
delivery of the bundle is going to be resumed using fragments.

Without any defined signalling, I can't see how my reactive
fragmentation will interoperate with your reactive fragmentation (for
example).

My questions to the list are therefore:

1) How do 2 BPAs agree that a partial transmission of a bundle should
be completed by dividing the bundle into the sent part and the unsent
part, so they can reassembled downstream?

2) Should the TCP-CL draft mention any of this?

Sorry, I'm not trying to be difficult, but I believe we are eluding to
functionality that isn't defined anywhere, and I think we should either
define it, or not mention it.

Cheers,

Rick

On Sun, 2018-01-28 at 18:44 +0000, Burleigh, Scott C (312B) wrote:
> Rick, I can describe how ION would do it, but of course that is not a
> general answer; all I would offer is what I think is an existence
> proof that additional customized protocol is not necessary.
> 
> Briefly, in ION you proactively fragment a bundle of payload length Q
> - into one bundle carrying the first N bytes of payload (N < Q) and a
> second carrying the last Q - N bytes of payload - by making a copy of
> the bundle (in its internal representation), creating two zero-copy
> objects containing the first N and last Q - N bytes of the original
> payload ZCO, delete the original bundle and its payload ZCO, and
> inserting both new fragmentary bundles into the shared-memory linked
> list of bundles awaiting disposition by the forwarding daemon.  This
> fragmentation may occur any point, and at any number of points, in
> the end-to-end path.
> 
> If ION did reactive fragmentation, I think the sending node would
> follow the same procedure except that only the fragment carrying the
> last Q - N bytes would be inserted into the linked list; the other
> fragment would never be created.  The receiving node would form a new
> ZCO containing the first N bytes of the original payload, would
> create a new bundle whose payload is that ZCO, and would insert that
> bundle into its own forwarding queue.
> 
> Segmentation tracking in TCPCL would enable this procedure: upon
> detecting connection failure, both the sending and receiving BPAs
> would use the reception progress offset reported in the last ACK
> passed from receiver to sender as the value of N, regardless of how
> many additional bytes might actually have been received since that
> ACK was sent.  Anything that the receiver acquired after the first N
> bytes would be wasted transmission that would have to be re-sent (it
> would be at the front of the sender's Q - N fragment), but the sender
> would not need to retransmit the first N bytes of the payload.  The
> only protocol that is needed is TCPCL, plus detection of TCP
> connection failure.
> 
> I think the bpsec spec now says that security blocks only apply to
> the original bundle, not to any fragments, so checking the signature
> or decrypting the payload would in any case only happen at the final
> destination node upon reassembly of the original bundle from all
> fragments.  If you did want to sign and/or encrypt a fragmentary
> payload you could use bundle-in-bundle encapsulation - the
> fragmentary bundle would become the complete payload of a new
> original bundle.
> 
> Scott
> 
> For example, if the transmitting CL reports back to its corresponding
> BPA that 45% of a bundle has been received by the other end, the
> assumption is that that BPA could react and do some kind of
> fragmentation to avoid sending the first 45% again.  My issue is how?
> 
> 1) The receiving TCP-CL only has half of what it was expecting.  How
> does it know to tell the receiving BPA to retain this, as
> fragmentation might be occurring.
> 2) The sending BPA could just split the original bundle at 45%, pop
> on a new header and send that.  However, the receiver now only has
> the second of two fragments. How does this new bundle correlate to
> the half-a-bundle it's already received.
> I think there needs to be some consensus over what 'reactive' or
> 'proactive' fragmentation actually mean, and to my mind, 'reactive'
> fragmentation has to mean:  The process whereby a bundle is possibly
> fragmented by one or more intermediate nodes along the path from
> source to destination.  And 'proactive' fragmentation is the process
> whereby a bundle may be fragmented by the sender, to optimize the
> transmission of the bundle payload from source to destination.  What
> we are discussing with TCP-CL segmentation is 'retrospective'
> fragmentation, which is the process of fragmenting a bundle after
> only a fraction of it has already been received by the next hop.
> 
> TCP-CL can provide useful data to the BPA to use in implementing
> reactive fragmentation, such as the Transfer MRU, and the
> transmission success of bundles can be sampled to provide
> probabilistic information about 'what kind of bundles and
> when'.  This data can drive the reactive fragmentation decision in
> the BPA, but the presence of segmentation doesn't help.
> 
> retrospective fragmentation based on feedback from the TCP layer, but
> I imagine it is using earlier/custom protocols, and I would be very
> interested to hear how they think it will work with BPbis and
> BPSec?  And if it does, how does it avoid re-sending the original
> segments that arrived prior to failure.
> 
> In summary:  Reactive fragmentation is nice to have, and I like the
> idea, but TCP-CL segmentation doesn't actually help provide it.  I
> just get the feeling that segmentation in TCP-CL makes the CL start
> to look like another store-and-forward mechanism sitting underneath
> BP, and I think that is a mistake.
> 
> Cheers,
> 
> Rick
>

---

From: Rick Taylor <[email-masked]>
Date: Mon, 29 Jan 2018 17:03:30 +0000

A quick P.S.,

I'm half waiting for Vint Cerf to point out that TCP already provides
sufficient ACKs for both parties to detect how many bytes of a stream
have been successfully received, and XFER_SEGMENT/XFER_ACK isn't
actually needed.  I believe he reads this list from time to time.

Cheers,

Rick

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Mon, 29 Jan 2018 17:19:15 +0000

Ah, good question.  In RFC 7242 there was a flag in the contact header; agreement was reached during contact header exchange.  I have not been tracking this thread closely, so I missed the discussion of the contact header revisions.

Scott

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Monday, January 29, 2018 8:59 AM
To: [email-masked]; Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

Hi Scott, All,

Thanks for that explanation, it ties with how I imagine one would implement reactive fragmentation.

However, I still have an issue that there is no signalling defined anywhere I can find (and I may have overlooked a reference) that describes how the sending BPA and next-hop BPA agree that the partial delivery of the bundle is going to be resumed using fragments.

Without any defined signalling, I can't see how my reactive fragmentation will interoperate with your reactive fragmentation (for example).

1) How do 2 BPAs agree that a partial transmission of a bundle should be completed by dividing the bundle into the sent part and the unsent part, so they can reassembled downstream?

Sorry, I'm not trying to be difficult, but I believe we are eluding to functionality that isn't defined anywhere, and I think we should either define it, or not mention it.

Cheers,

Rick

On Sun, 2018-01-28 at 18:44 +0000, Burleigh, Scott C (312B) wrote:
> fragmentation may occur any point, and at any number of points, in the 
> end-to-end path.
> 
> ZCO containing the first N bytes of the original payload, would create 
> a new bundle whose payload is that ZCO, and would insert that bundle 
> into its own forwarding queue.
> 
> many additional bytes might actually have been received since that ACK 
> was sent.  Anything that the receiver acquired after the first N bytes 
> would be wasted transmission that would have to be re-sent (it would 
> be at the front of the sender's Q - N fragment), but the sender would 
> not need to retransmit the first N bytes of the payload.  The only 
> protocol that is needed is TCPCL, plus detection of TCP connection 
> failure.
> 
> I think the bpsec spec now says that security blocks only apply to the 
> original bundle, not to any fragments, so checking the signature or 
> decrypting the payload would in any case only happen at the final 
> payload you could use bundle-in-bundle encapsulation - the fragmentary 
> bundle would become the complete payload of a new original bundle.
> 
> Scott
> 
> 2) The sending BPA could just split the original bundle at 45%, pop on 
> a new header and send that.  However, the receiver now only has the 
> second of two fragments. How does this new bundle correlate to the 
> half-a-bundle it's already received.
> fragmentation, which is the process of fragmenting a bundle after only 
> a fraction of it has already been received by the next hop.
> 
> reactive fragmentation, such as the Transfer MRU, and the transmission 
> success of bundles can be sampled to provide probabilistic information 
> about 'what kind of bundles and when'.  This data can drive the 
> reactive fragmentation decision in the BPA, but the presence of 
> segmentation doesn't help.
> 
> interested to hear how they think it will work with BPbis and BPSec?  
> And if it does, how does it avoid re-sending the original segments 
> that arrived prior to failure.
> 
> just get the feeling that segmentation in TCP-CL makes the CL start to 
> look like another store-and-forward mechanism sitting underneath BP, 
> and I think that is a mistake.
> 
> Cheers,
> 
> Rick
> 
> > Hi, Rick.  Doubtful as I am of the utility of reactive 
> > fragmentation, if we are going to retain this apparently popular 
> > feature then I'd say we have to continue tracking segmentation in 
> > TCPCL.  The periodic, incremental, progressive segment 
> > acknowledgments returned by the receiving node are the only way the 
> > sending node can determine the reactive fragmentation point (i.e., 
> > how many leading bytes of the payload must be assumed to have been 
> > used by the receiving node to form the received fragment, such that 
> > all remaining bytes must be used by the sending node to form the 
> > fragment that needs to be re-forwarded).  TCPCL shouldn't perform 
> > the bundle fragmentation itself, but it seems valid for TCPCL to 
> > tell the BPA when and where to fragment.
> > 
> > So long as it's the BPA itself that always performs the 
> > fragmentation, I think the security questions are covered by the 
> > bpsec specification.
> > Certainly you're right that eliminating segmentation tracking in 
> > TCPCL would simplify the CL, but I think it will be argued that the 
> > reactive fragmentation feature is valuable enough to justify the 
> > additional complexity and overhead.
> > 
> > Scott
> > 
> > Given the Bundle Protocol documents do not (intentionally) mention 
> > how to do fragmentation, and I believe that there is rough consensus 
> > that bundle fragmentation is really the responsibility of the BPA, 
> > not the CL, I wonder whether the segmentation algorithms in TCP-CL 
> > need to be there at all?
> > 
> > TCP-CL (or any other CL) to be an atomic operation:  a BPA passes 
> > the next output- queue bundle to the CL to be sent, and it is either 
> > sent in its entirety or fails in its entirety, with only a 
> > processing and transmission delay between start of send and result.  
> > I.e. the BPA retains 'ownership' of the bundle until the CL says it 
> > has been transferred, rather than the CL taking ownership from the 
> > BPA.
> > 
> > The alternate, hinted at in the draft, is that the BPA passes a 
> > bundle to the TCP-CL, and sometime later, the CL announces only some 
> > of the bundle has been successfully transmitted.  How is the BPA 
> > supposed to react to this information?  How does the BPA decide to 
> > fragment and retransmit a bundle that has been fragmented at a lower 
> > layer (the
> > CL)
> > I would suggest that a cleaner/simpler approach would be for the CL 
> > to refuse/fail a transmission with a response such as "Bundle too 
> > big to send" or "Peer hung up early", and let the BPA decide what to 
> > do about fragmentation.  To this end, I would remove XFER_SEGMENT 
> > altogether, and a have a single XFER_INIT, XFER_ACK pair to bracket 
> > a single atomic bundle transmission.  This would simplify some of 
> > the text around transmission ordering and pipelining as well.
> > 
> > I also have a concern that the combination of BPSec and 
> > fragmentation is tricky to get right - by adding TCP-CL level 
> > segmentation, can I as a malicious attacker keep re-requesting a 
> > certain byte-range of encrypted bundles, sampling for a potential 
> > cryptographic dictionary attack, with the BPA (that is managing the 
> > cryptography) being unaware?  I.e. are we opening ourselves up to 
> > replay attacks?
> >

---

From: Brian Sipos <[email-masked]>
Date: Mon, 29 Jan 2018 17:53:44 +0000

This missing negotiation is a side-effect of my original misunderstanding then. I had removed those other negotiated parameters/flags and made those other behaviors mandatory to simply the protocol. But this fragmentation one is different because it's not really the CL itself doing anything with it, but being passed-up to the BP agent for its use. Even RFC 7242 really doesn't say what is supposed to be done when reactive fragmentation is negotiated as enabled.

A question then for the new CL spec: is the symmetric behavior of this negotiated TCPCL session parameter in RFC 7242 desirable? So that both nodes must support fragmentation in both directions for it to be usable in either direction.

It is a relatively straightforward thing to add a contact header flag to Section 4.2 and add a new "Reactive Fragmentation Enabled" session parameter to Section 4.3. It's still not specifying exactly what the BP agent would do with such information, but at least it closes the gap that the information is available (as it was in TCPCLv3).

________________________________
From: Burleigh, Scott C (312B) <[email-masked]>
Sent: Monday, January 29, 2018 12:19:15 PM
To: Rick Taylor; [email-masked]
Cc: Brian Sipos
Subject: RE: [dtn] TCP-CL and XFER_SEGMENT

Scott

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]]
Sent: Monday, January 29, 2018 8:59 AM
To: [email-masked]; Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

Hi Scott, All,

Cheers,

Rick

---

From: Carlo Caini <[email-masked]>
Date: Mon, 29 Jan 2018 19:11:39 +0100

Dear Rick,
      I had the same thought as you in the past, but I was actually 
wrong. In fact, due to layering, TCP ACKs should not be available to 
upper layers. This is why I think TCPCL on the sender segment a 
bundle into multiple segments, and why these segments are acked by 
the TCPCL on the receiver side. The second fragment, after an 
interruption, starts from the latest segments acked by the TCPCL on 
the Rx side, not from the lasted byte acked by TCP; as said this 
information should not be provided because it is against the layering 
principle. Data sent by TCPCL must be acked by TCPCL, i.e. its peer. 
This is at least my understanding of DTN2 TCPCL as a user. I have 
never inspected the code however.
Yours,
    Carlo

At 18:03 29/01/2018, Rick Taylor wrote:
> > > fragmentation,
> > > fragmentation,
> > > TCPCL
> > > how
> > > the
> > > bundle
> > > to
> > > fragmentation

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Mon, 29 Jan 2018 19:38:39 +0000

Layering should never be strictly enforced for its own sake: therein lies one of the many possible paths to madness.  If I have 128K of RAM, and I have a choice between A) saying something is impossible or B) implementing some cleverly optimized convergence layer that breaks abstraction to share state between my CL and the TCP stack … then guess what I’m going to do?  That doesn’t mean that it’s necessarily a good design decision, it just means that I *can* do it without anyone else ever being the wiser.

I do think that signaling between the agent and the convergence layer is a challenge.  I think that standardizing such an interface only matters if there’s an intent for a CL built by one person to be used with a BP handler built by another.  I don’t think we’re there at the moment, so I personally don’t think that this is a necessary step at this point … but it may be something to revisit eventually.

Instead, I am of the opinion that the immediate key here is picking the right things to focus on so that everyone’s implementation can play nicely together.  Making sure CLs can talk to one another seems like one piece of this, and making sure bundle formats are compatible with one another seems like a second step.  Ensuring deterministic behavior by both CLs and bundle handlers is a third step, so that one side can predict what the other side will do given conditions X, Y, and Z.  As long as we have those things with a sufficiently complete set of conditions, I think it should be good enough.

On a related note, anything that cannot be externally observed and / or measured should be provided in the document on an informational basis *at best*, I think.

-Gilbert

The views expressed in this mail reflect the opinions of the author.  They are, therefore, not intended to reflect official positions of NASA or the U.S. Government.

On 1/29/18, 1:12 PM, "dtn on behalf of Carlo Caini" <[email-masked] on behalf of [email-masked]> wrote:

    Dear Rick,

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Mon, 29 Jan 2018 20:14:12 +0000

That seems to me as though it would be entirely at the whim of the receiving BP agent.  Essentially, I personally don’t see it as an agreement so much as an effort to reduce the end-to-end latency of delivery on a best-effort basis: the other side got a bunch of pieces that you sent, and they told you that they received ‘em … but that data is informational in nature.  It does not necessarily prescribe a specific action unless the sending agent would like to do something with the data it got back from the other side … and the specific action it prescribes is at the discretion of the sending agent.

Which brings me to a different comment: I do think the better question here is “should convergence layers have a mechanism to report partial delivery of bundles?”, which is a fundamentally different question than “how should / does reactive fragmentation work in BP?”  Enabling partial delivery notification in a CL offers the ability to support any kind of implementation of reactive fragmentation in the future … but the mechanics of how that reactive fragmentation might work seem to be beyond the scope of the CL.

-Gilbert

On 1/29/18, 11:59 AM, "dtn on behalf of Rick Taylor" <[email-masked] on behalf of [email-masked]> wrote:

    Hi Scott, All,

    

    

    Cheers,
    
    Rick

---

From: Rick Taylor <[email-masked]>
Date: Tue, 30 Jan 2018 11:44:50 +0000

Hi Carlo,

Thinking about it more, you are right. For 'proper' layering, the TCP-
layer ACKs should trigger CL-layer ACKs, which can then trigger
something else above, for example reactive fragmentation.

Thanks for setting me straight.

Cheers,

Rick

On Mon, 2018-01-29 at 19:11 +0100, Carlo Caini wrote:
> > > To: Burleigh, Scott C (312B) <[email-masked]>; dtn
> > > @iet
> > > f.org
> > > Ironically, I am a fan of reactive fragmentation, but I don't
> > > think
> > > TCP-CL segmentation actually helps.
> > >
> > > 4) I just can't see how one avoids resending the first 45% again
> > > as a
> > > new bundle without changing BPbis.
> > >
> > > > if we are going to retain this apparently popular feature then
> > > > I'd
> > > > say
> > > > form the received fragment, such that all remaining bytes must
> > > > be
> > > > used
> > > > itself, but it seems valid for TCPCL to tell the BPA when and
> > > > where
> > > > to fragment.
> > > >
> > > > fragmentation feature is valuable enough to justify the
> > > > additional
> > > > complexity and overhead.
> > > >
> > > > Scott
> > > >
> > > > I'm just having another look at TCP-CL and trying to catch up
> > > > on
> > > > the
> > > > to do fragmentation, and I believe that there is rough
> > > > consensus
> > > > that
> > > > bundle fragmentation is really the responsibility of the BPA,
> > > > not
> > > > the
> > > > CL, I wonder whether the segmentation algorithms in TCP-CL need
> > > > to
> > > > be there at all?
> > > >
> > > > next output- queue bundle to the CL to be sent, and it is
> > > > either
> > > > sent
> > > > in its entirety or fails in its entirety, with only a
> > > > processing
> > > > and
> > > > to the TCP-CL, and sometime later, the CL announces only some
> > > > of
> > > > the
> > > > bundle has been successfully transmitted.  How is the BPA
> > > > supposed
> > > > to
> > > > retransmit a bundle that has been fragmented at a lower layer
> > > > (the
> > > > CL)
> > > > specific knowledge of the inner workings of the CL, which will
> > > > be
> > > > different for every CL?
> > > >
> > > > refuse/fail a transmission with a response such as "Bundle too
> > > > big
> > > > to
> > > > send" or "Peer hung up early", and let the BPA decide what to
> > > > do
> > > > about
> > > > is tricky to get right - by adding TCP-CL level segmentation,
> > > > can I
> > > > as

---

From: Rick Taylor <[email-masked]>
Date: Tue, 30 Jan 2018 11:53:08 +0000

Hi Gilbert, All,

(Chair Hat Off)

Sorry, I'm being much slower than you (and others). Yes, I agree, the
segmentation and notification of partial delivery is a mechanism that
can be used by the BPA to do 'something', and therefore does have
merit, particularly as Brian has updated the TCP-CL draft to list the
signalling provided by the CL to the BPA.  So I'll stop publicly
chewing over whether XFER_SEGMENTs are needed.  The consensus I'm
seeing is that they have value, even if we are not quite sure how they
can be used.

Cheers,

Rick

On Mon, 2018-01-29 at 20:14 +0000, Clark, Gilbert J. (GRC-LCA0) wrote:
> That seems to me as though it would be entirely at the whim of the
> receiving BP agent.  Essentially, I personally don’t see it as an
> agreement so much as an effort to reduce the end-to-end latency of
> delivery on a best-effort basis: the other side got a bunch of pieces
> that you sent, and they told you that they received ‘em … but that
> data is informational in nature.  It does not necessarily prescribe a
> specific action unless the sending agent would like to do something
> with the data it got back from the other side … and the specific
> action it prescribes is at the discretion of the sending agent.
>
> Which brings me to a different comment: I do think the better
> question here is “should convergence layers have a mechanism to
> report partial delivery of bundles?”, which is a fundamentally
> different question than “how should / does reactive fragmentation
> work in BP?”  Enabling partial delivery notification in a CL offers
> the ability to support any kind of implementation of reactive
> fragmentation in the future … but the mechanics of how that reactive
> fragmentation might work seem to be beyond the scope of the CL.
>
> -Gilbert
>
> The views expressed in this mail reflect the opinions of the
> author.  They are, therefore, not intended to reflect official
> positions of NASA or the U.S. Government.
>
>
> On 1/29/18, 11:59 AM, "dtn on behalf of Rick Taylor" <dtn-bounces@iet
> f.org on behalf of [email-masked]> wrote:
>
>     Hi Scott, All,
>
>     > To: Burleigh, Scott C (312B) <[email-masked]>; d
> tn@iet
>     > f.org
>     > 3) How does any of this 'divide and resend the missing bits'
> work
>     > with BPSec integrity checking?
>     > 4) I just can't see how one avoids resending the first 45%
> again as a
>     > > From: dtn [mailto:[email-masked]] On Behalf Of
> Burleigh,
>     > > Scott C
>     > > (312B)
>     > > if we are going to retain this apparently popular feature
> then I'd
>     > > say
>     > > form the received fragment, such that all remaining bytes
> must be
>     > > used
>     > > CL, I wonder whether the segmentation algorithms in TCP-CL
> need to
>     > > be there at all?
>     > >
>     > > In my opinion it would be simpler to consider a bundle
> transfer
>     > > over
>     > > specific knowledge of the inner workings of the CL, which
> will be
>     > > different for every CL?
>     > >
>     > > refuse/fail a transmission with a response such as "Bundle
> too big
>     > > to

---

From: Rick Taylor <[email-masked]>
Date: Tue, 30 Jan 2018 12:17:28 +0000

Hi All,

(Chair Hat Off)

So, the plot thickens... Previously the contact header of TCP-CL had an
indicator that reactive fragmentation was allowed/desired, but it has
been swept away in the uplift.

From a purist perspective, fragmentation is a bundle process, and
should therefore be a function of the bundle processing agent,
however, bundle processing flags exist in the immutable primary block,
and reactive fragmentation is a hop-by-hop decision and therefore any
reactive fragmentation coordination signalling must be either carried
in a non-primary block, or pushed down to the convergence layer (as it
was before).

I'm starting to wonder whether there is benefit in putting some flag(s)
back into the contact header that defines what both parties at the CL
layer should do with partial bundle delivery. So far I can imagine the
following flag values:

'atomic': Bundle transmission will be resumed from the start on next
contact, no need to cache anything.

'resume': Bundle transmission will resume from the first failed segment
on next contact, so the receiver SHOULD retain the segments it already
has.  (This is a store-and-forward mechanism running beneath BP, is
that desired?)

'fragment': The receiver SHOULD create a new first bundle fragment from
what it has already received (and forward?), and a new second bundle
fragment will be transmitted on the next contact. This value MUST NOT
be used when the bundle control flags prevent fragmentation.

If the error codes of TCP-CL allow an implementation to say
"Unsupported Flag" for anything apart from 'atomic', I think we might
have a way forwards.

Thoughts?

Rick

On Mon, 2018-01-29 at 17:53 +0000, Brian Sipos wrote:
> This missing negotiation is a side-effect of my original
> misunderstanding then. I had removed those other negotiated
> parameters/flags and made those other behaviors mandatory to simply
> the protocol. But this fragmentation one is different because it's
> not really the CL itself doing anything with it, but being passed-up
> to the BP agent for its use. Even RFC 7242 really doesn't say what is
> supposed to be done when reactive fragmentation is negotiated as
> enabled.
>
> A question then for the new CL spec: is the symmetric behavior of
> this negotiated TCPCL session parameter in RFC 7242 desirable? So
> that both nodes must support fragmentation in both directions for it
> to be usable in either direction.
>
> It is a relatively straightforward thing to add a contact header flag
> to Section 4.2 and add a new "Reactive Fragmentation Enabled" session
> parameter to Section 4.3. It's still not specifying exactly what the
> BP agent would do with such information, but at least it closes the
> gap that the information is available (as it was in TCPCLv3).
>
> Ah, good question.  In RFC 7242 there was a flag in the contact
> header; agreement was reached during contact header exchange.  I have
> not been tracking this thread closely, so I missed the discussion of
> the contact header revisions.
>
> Scott
>
> Sorry, I'm not trying to be difficult, but I believe we are eluding
> to functionality that isn't defined anywhere, and I think we should
> either define it, or not mention it.
>
> Cheers,
>
> Rick
>
> create
> ACK
> fragmentary
> > To: Burleigh, Scott C (312B) <[email-masked]>; dtn@i
> et
> > f.org
> > 3) How does any of this 'divide and resend the missing bits' work
> with
> > 4) I just can't see how one avoids resending the first 45% again as
> a
> only
> transmission
> BPSec?
> > > From: dtn [mailto:[email-masked]] On Behalf Of Burleigh,
> Scott
> > > C
> > > (312B)

---

From: "R. Atkinson" <[email-masked]>
Date: Tue, 30 Jan 2018 08:24:04 -0500

> On Jan 29, 2018, at 14:38, Clark, Gilbert J. (GRC-LCA0) <[email-masked]> wrote:
> 

I agree that specifications ought not gratuitously limit an implementation.

That noted, it is quite reasonable to specify standards in a layered manner.

Indeed, layered protocol specification is the just about the only widely demonstrated way
that works well in a world of heterogeneous implementations.

> On a related note, anything that cannot be externally observed and / or measured should be provided in the document on an informational basis *at best*, I think.

The IETF historically has had at least two different philosophical approaches to specifications.
I will use routing protocols to illustrate both the differences in specification approach and also 
the differences in the results.

The OSPF specifications contain a minimal set of “MUST” items and rather a lot of explanatory (i.e., non-normative) text about concepts and advice to implementations.  OSPF interoperability is very very widespread and never has been an operational issue.  OSPF implementation quality varies widely, with some implementations using less memory and more efficient computational approaches to yield demonstrably better performance along various dimensions (speed, scalability, etc.)

The BGP specifications follow the approach above where the specification seeks to limit itself to items that can be “externally observed and/or measured”.  BGP interoperability has been problematic for years.  Indeed the 2 leading router vendors have been very successful in using the paucity of BGP documentation (and the corresponding near total absence of implementation advice) to create a very effective barrier to entry in the backbone/core router equipment market.  While this is _slightly_ better today than 20 years ago, it is still the case that those same 2 equipment vendors still dominate the backbone/core router market in deployed commercial IP networks.  

(A third vendor is vigorously trying to enter that backbone router market and gain non-trivial market share, but published reports have suggested that the 3rd vendor was sued in US Federal Courts for theft of IP routing source code from one of the leading two vendors, which if true illustrates how challenging it is to implement full-feature set BGP from the published specifications.)

The WG could choose to go either way with this.  

Speaking as someone who has implemented multiple widely-deployed protocols over the past 30 years, including writing IPv6 code that apparently is still used today in Cisco IOS, I believe that having well documented implementation advice and explanatory text would increase the quality of typical implementations.  

One approach to this would be to have a standards-track specification for the protocol and then a separate Informational status document that contains information about Concept of Operations, Implementation Advice, and so forth.  The standards-track specification would have an Informational Reference to the separate Informational status document, so that a well-intentioned implementer could easily discover that parallel document.

Yours,

Ran

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Tue, 30 Jan 2018 14:03:03 +0000

Hi,

Sorry for the top post – the mail client I’m using makes it very difficult to do inline.  That’s a problem I’ll eventually need to fix …

With regard to “layered protocol specification is the just about the only widely demonstrated way that works well in a world of heterogeneous implementations”: I absolutely agree that layered protocol specification is good.  Instead, my intended point here was that any abstraction (logical network layering included) should never be completely opaque.  In other words, there needs to be a way to break any abstraction when warranted … given that both the situations and the methods through which specific abstractions may be broken are carefully considered.

With regard to “Speaking as someone who has implemented multiple widely-deployed protocols over the past 30 years, including writing IPv6 code that apparently is still used today in Cisco IOS, I believe that having well documented implementation advice and explanatory text would increase the quality of typical implementations”: Sure, context and recommendations are usually good reading and can be rather helpful (read: the security considerations section in recent protocol specifications) … until it starts to slip into prescribing a specific API and / or implementation as part of the protocol specification.  The original 5050 always struck me as coming perilously close to the latter, which (in hindsight) is probably part of what prompted the comment in this case. 

-Gilbert

On 1/30/18, 8:26 AM, "dtn on behalf of R. Atkinson" <[email-masked] on behalf of [email-masked]> wrote:

    

    

    

    

    Yours,
    
    Ran

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Tue, 30 Jan 2018 18:01:18 +0000

Speaking as an implementer of TCPCL, I strongly prefer that we NOT contemplate a "resume" option; TCPCL data accounting is already complicated enough.  The "fragment" option is okay with me, so long as it is never mandatory; it won't be implemented in ION.

As to error codes, I think what we did before was simply set each implementation's preferred value of each flag in the contact headers that were exchanged, and then - in the event of a divergence - both implementations would simply agree on whichever value was least demanding; no need for additional error signaling.

Scott

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Tuesday, January 30, 2018 4:17 AM
To: [email-masked]; Burleigh, Scott C (312B) <[email-masked]>; [email-masked]
Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

Hi All,

(Chair Hat Off)

So, the plot thickens... Previously the contact header of TCP-CL had an indicator that reactive fragmentation was allowed/desired, but it has been swept away in the uplift.

From a purist perspective, fragmentation is a bundle process, and should therefore be a function of the bundle processing agent, however, bundle processing flags exist in the immutable primary block, and reactive fragmentation is a hop-by-hop decision and therefore any reactive fragmentation coordination signalling must be either carried in a non-primary block, or pushed down to the convergence layer (as it was before).

I'm starting to wonder whether there is benefit in putting some flag(s) back into the contact header that defines what both parties at the CL layer should do with partial bundle delivery. So far I can imagine the following flag values:

'atomic': Bundle transmission will be resumed from the start on next contact, no need to cache anything.

'resume': Bundle transmission will resume from the first failed segment on next contact, so the receiver SHOULD retain the segments it already has.  (This is a store-and-forward mechanism running beneath BP, is that desired?)

'fragment': The receiver SHOULD create a new first bundle fragment from what it has already received (and forward?), and a new second bundle fragment will be transmitted on the next contact. This value MUST NOT be used when the bundle control flags prevent fragmentation.

If the error codes of TCP-CL allow an implementation to say "Unsupported Flag" for anything apart from 'atomic', I think we might have a way forwards.

Thoughts?

Rick

On Mon, 2018-01-29 at 17:53 +0000, Brian Sipos wrote:
> the protocol. But this fragmentation one is different because it's not 
> really the CL itself doing anything with it, but being passed-up to 
> the BP agent for its use. Even RFC 7242 really doesn't say what is 
> A question then for the new CL spec: is the symmetric behavior of this 
> negotiated TCPCL session parameter in RFC 7242 desirable? So that both 
> nodes must support fragmentation in both directions for it to be 
> usable in either direction.
>
> functionality that isn't defined anywhere, and I think we should

---

From: Brian Sipos <[email-masked]>
Date: Tue, 30 Jan 2018 18:38:27 +0000

Rick,

There are now some options in TCPCLv4 compared to v3 because of the header extension items. If it's core enough behavior, we can choose to make some kind of CAN_FRAGMENT flag and eat one of the contact header flag bits. If it's not really a core behavior, there can be an extension item to indicate reactive fragmentation (with a default cannot-fragment logic if the extension type is not present).

I don't have a strong preference; there are 7 extra header bits and 65k extra extension types. The header flags are simpler logic to describe/encode.

________________________________
From: Rick Taylor <[email-masked]>
Sent: Tuesday, January 30, 2018 7:17:28 AM
To: [email-masked]; [email-masked]; Brian Sipos
Subject: Re: [dtn] TCP-CL and XFER_SEGMENT

Hi All,

(Chair Hat Off)

Thoughts?

Rick