Subject: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

From: Brian Sipos <[email-masked]>
Date: Thu, 25 Jan 2018 14:57:52 +0000

All,

One of the comments about BP and its agent-CL interface that has been raised a few times, most recently in Victoria Pritchard's review of TCPCLv4, is the need to understand what is required behavior of any CL, what is expected/optional behavior of a CL, and which would make everything else superfluous to the BP agent.

There is currently in TCPCLv4 draft a behavior for intermediate acknowledgment of segments of a bundle up to, and including, the total size of the bundle. As Victoria has pointed out, there is really no driving need for non-full-bundle acknowledgment as far as the current BPv7 requirements are concerned. There is no notion in BPv7 of the possibility of a partial bundle delivery along its path, and so there is no need for a BP agent to ever be informed by the CL of any kind of intermediate delivery progress. The BP agent really only cares about definitely delivered and definitely failed delivery, as far as I can tell. The unfortunate thing right now is that there is not a hard list of what kinds of requests a CL shall/should accept and what notifications it shall/should provide to a BP agent.

Are there any WG objections to removing intermediate (per-segment) XFER_ACK messages and simplifying the XFER_ACK to simply being sent out once per transfer after the last segment has been received?

The underlying TCP takes care of any sequencing and ARQ issues on the socket, so TCPCL is guaranteed that if the receiver sees the end segment it's already seen all preceding segments.

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 25 Jan 2018 15:28:54 +0000

> Are there any WG objections to removing intermediate (per-segment) XFER_ACK messages

Isn't per-segment ACK needed to support reactive fragmentation?

Fred

From: dtn [mailto:[email-masked]] On Behalf Of Brian Sipos
Sent: Thursday, January 25, 2018 6:58 AM
To: [email-masked]
Subject: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

All,

---

From: Brian Sipos <[email-masked]>
Date: Thu, 25 Jan 2018 15:45:22 +0000

Fred,

Yes that is absolutely correct, but reactive fragmentation is nowhere defined for BPv7.

That is the real core of this issue, that there is no well defined scope of what a CL shall/should do to interact with a BP agent. If reactive fragmentation is truly a thing that a CL may support, then we should leave-in per-segment ACK; otherwise the ACK is unnecessary (and wasteful to an extent).

________________________________
From: Templin, Fred L <[email-masked]>
Sent: Thursday, January 25, 2018 10:28:54 AM
To: Brian Sipos; [email-masked]
Subject: RE: BP Convergence Layer required/optional behavior (and partial-bundle delivery)

> Are there any WG objections to removing intermediate (per-segment) XFER_ACK messages

Isn’t per-segment ACK needed to support reactive fragmentation?

Fred

All,

---

From: Wesley Eddy <[email-masked]>
Date: Thu, 25 Jan 2018 10:54:01 -0500

On 1/25/2018 10:45 AM, Brian Sipos wrote:
> That is the real core of this issue, that there is no well defined 
> scope of what a CL shall/should do to interact with a BP agent. If 
> reactive fragmentation is truly a thing that a CL may support, then we 
> should leave-in per-segment ACK; otherwise the ACK is unnecessary 
> (and wasteful to an extent).
>
>

Without the ACK, decent reactive fragmentation is precluded on TCPCL (or 
at least more difficult to construct a mechanism for), which would be 
really bad.  Our group is using this capability in BPv6, and if v7 can't 
support it that would be tragic.

I don't think you should take it out just for janitorial reasons.

---

From: William Ivancic <[email-masked]>
Date: Thu, 25 Jan 2018 11:25:27 -0500

It appears BP7 allows fragmentation so one would hope the CL would support that.  Reactive fragmentation was proven to be quite useful way back in 2009:

https://www.ietf.org/mail-archive/web/dtn-interest/current/msg00804.html

http://syzygyengineering.com/2010/GlobeCom1569303096.pdf

Will 

From: dtn <[email-masked]> on behalf of Wesley Eddy <[email-masked]>
Date: Thursday, January 25, 2018 at 10:54 AM
To: <[email-masked]>
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Fred,

Without the ACK, decent reactive fragmentation is precluded on TCPCL (or at least more difficult to construct a mechanism for), which would be really bad.  Our group is using this capability in BPv6, and if v7 can't support it that would be tragic.

_______________________________________________ dtn mailing list [email-masked] https://www.ietf.org/mailman/listinfo/dtn

---

From: Brian Sipos <[email-masked]>
Date: Thu, 25 Jan 2018 17:36:53 +0000

Just to be clear. I don't at all disagree with the utility of the behavior.

My concern is that a CL writer/reviewer would have to wade through possibly years of mailing list discussion to be able to piece together all of the things that a CL could possibly be expected to provide. As this topic brings up, it's not necessarily part of the core BP spec that this behavior may reside.

To the WG: Does it make sense to have some kind of after-the-fact BCP (or something) about BPv7 CL-agent interface capabilities?

This would avoid holding things up near-term but would provide an actual document to place these kinds of common shall/should behaviors imposed on a CL.

________________________________
From: dtn <[email-masked]> on behalf of William Ivancic <[email-masked]>
Sent: Thursday, January 25, 2018 11:25:27 AM
To: Wesley Eddy; [email-masked]
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Will

Fred,

---

From: "Marc Blanchet" <[email-masked]>
Date: Thu, 25 Jan 2018 12:49:19 -0500

On 25 Jan 2018, at 12:36, Brian Sipos wrote:

> My concern is that a CL writer/reviewer would have to wade through 
> possibly years of mailing list discussion to be able to piece together 
> all of the things that a CL could possibly be expected to provide. As 
> this topic brings up, it's not necessarily part of the core BP spec 
> that this behavior may reside.
>
>
> To the WG: Does it make sense to have some kind of after-the-fact BCP 
> (or something) about BPv7 CL-agent interface capabilities?

If it is mandatory to implement, then it must be in a Proposed Standard 
RFC. Moreover, if this is going to be required by all CL, then I guess 
it has to go into the BP spec.

Marc.

>
> This would avoid holding things up near-term but would provide an 
> actual document to place these kinds of common shall/should behaviors 
> imposed on a CL.
>
> It appears BP7 allows fragmentation so one would hope the CL would 
> support that.  Reactive fragmentation was proven to be quite useful 
> way back in 2009:
>
>
>
> should leave-in per-segment ACK; otherwise the ACK is unnecessary (and 
> wasteful to an extent).
>
>
>
> Without the ACK, decent reactive fragmentation is precluded on TCPCL 
> (or at least more difficult to construct a mechanism for), which would 
> be really bad.  Our group is using this capability in BPv6, and if v7 
> can't support it that would be tragic.
>
> _______________________________________________ dtn mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/dtn

> _______________________________________________
> dtn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dtn

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 25 Jan 2018 17:49:18 +0000

Brian, I'm doubtful that we need any more specific definition of the scope of convergence-layer adapter behavior.  Section 7 of the bpbis draft covers this briefly, and I don't see the need to be more exhaustive.  What kinds of problems do you think we are opening ourselves up to in the absence of such a BCP (or whatever)?  Isn't it sufficient for a CL specification to conform to BP section 7 and otherwise merely define its own (possibly idiosyncratic) behavior in detail?

Scott

From: dtn [mailto:[email-masked]] On Behalf Of Brian Sipos
Sent: Thursday, January 25, 2018 9:37 AM
To: William Ivancic <[email-masked]>; Wesley Eddy <[email-masked]>; [email-masked]
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

________________________________
From: dtn <[email-masked]<mailto:[email-masked]>> on behalf of William Ivancic <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, January 25, 2018 11:25:27 AM
To: Wesley Eddy; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Will

From: dtn <[email-masked]<mailto:[email-masked]>> on behalf of Wesley Eddy <[email-masked]<mailto:[email-masked]>>
Date: Thursday, January 25, 2018 at 10:54 AM
To: <[email-masked]<mailto:[email-masked]>>
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Fred,

_______________________________________________ dtn mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/dtn

---

From: William Ivancic <[email-masked]>
Date: Thu, 25 Jan 2018 14:28:40 -0500

When this working group started, I believe the consensus was to NOT develop a concept of operations or a requirements document.  So... history is scattered.   Regardless, we probably should put a small requirements section in each convergence layer indicated what functions are expected of it.  IMHO, TCPCL should be able to handle reactive fragmentation (i.e.  reactive fragmentation is a requirement).   LTP, on the other hand, might not need to handle reactive fragmentation.  IMHO, LTP is more for long propagation delays or scheduled outages.  The long propagation delays probably make reactive fragmentation unreasonable.  However, if LTP were to be widely used over short propagation delays, that may not be the case.

Will

On 1/25/18, 12:49 PM, "dtn on behalf of Marc Blanchet" <[email-masked] on behalf of [email-masked]> wrote:

    

    Marc.

---

From: Brian Sipos <[email-masked]>
Date: Thu, 25 Jan 2018 19:38:27 +0000

Scott,

I agree that a CL can do whatever else that BP agent does not directly care about, and it should have no ill effects on the necessary BP processing.

What I was interested in doing was to simplify TCPCLv4 by removing some "unnecessary" behavior (based on WG comment, and because why do a thing if it's not driven by external need) while some consider this behavior fully necessary for their operations, but there's nothing to point to (in the sense of other I-Ds or RFCs) to justify the behavior. The design is being pulled in opposing directions and there is no obvious right way to go, except to leave the ACK behavior as-is because that's what TCPCLv3 did (optionally).

I am fine with leaving in the intermediate ACK behavior along with some rationale for why this may be necessary ("TCPCLv3 did it", some TBD future fragmentation behavior).

There are just no obvious breadcrumbs to leave behind for the next CL author(s), to discover the same undocumented-but-maybe-necessary-for-some-uses types of CL behavior.

________________________________
From: Burleigh, Scott C (312B) <[email-masked]>
Sent: Thursday, January 25, 2018 12:49:18 PM
To: Brian Sipos; William Ivancic; Wesley Eddy; [email-masked]
Subject: RE: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Brian, I’m doubtful that we need any more specific definition of the scope of convergence-layer adapter behavior.  Section 7 of the bpbis draft covers this briefly, and I don’t see the need to be more exhaustive.  What kinds of problems do you think we are opening ourselves up to in the absence of such a BCP (or whatever)?  Isn’t it sufficient for a CL specification to conform to BP section 7 and otherwise merely define its own (possibly idiosyncratic) behavior in detail?

Scott

Will

Fred,

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Thu, 25 Jan 2018 19:44:30 +0000

My interpretation is that 5.8 of the current bpbis draft suggests that reactive fragmentation MAY be supported.  I can’t see a reference to a MUST anywhere … but it also wasn’t an exhaustive read of the draft, so apologies if I just missed something.

With regard to “should all convergence layers be able to support all the capabilities that BP includes?”, my personal thought would be no.  That opinion does seem to agree with my interpretation of 7.2 of the revised 5050. 

With that said, I *do* feel like at least one convergence layer (preferably this one, since it’s going up with the revised 5050) should support *all* required and optional capabilities of BP.

And yes, +1 that reactive fragmentation is extremely important.  Not having a convergence layer capable of supporting it would seem to me to be a pretty significant oversight … but others’ mileage may vary.

-Gilbert

The views expressed in this mail reflect the opinions of the author.  They are, therefore, not intended to reflect official positions of NASA or the U.S. Government.

    

    
    Marc.

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 25 Jan 2018 20:37:01 +0000

Sure, I am all for adding some informative text in the TCPCL specification explaining that incremental ACKs are useful for reactive fragmentation.  I agree that this would save future readers some detective work.

Scott

From: Brian Sipos [mailto:[email-masked]]
Sent: Thursday, January 25, 2018 11:38 AM
To: Burleigh, Scott C (312B) <[email-masked]>; William Ivancic <[email-masked]>; Wesley Eddy <[email-masked]>; [email-masked]
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Scott,

________________________________
From: Burleigh, Scott C (312B) <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, January 25, 2018 12:49:18 PM
To: Brian Sipos; William Ivancic; Wesley Eddy; [email-masked]<mailto:[email-masked]>
Subject: RE: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Scott

From: dtn [mailto:[email-masked]] On Behalf Of Brian Sipos
Sent: Thursday, January 25, 2018 9:37 AM
To: William Ivancic <[email-masked]<mailto:[email-masked]>>; Wesley Eddy <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

Will

Fred,

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 25 Jan 2018 21:06:38 +0000

The word "reactive" appears nowhere in the bpbis specification.  If you design a given CL to be able to cause the BP agent to perform what we have termed "reactive fragmentation" in the past, great.  It's certainly not mandatory.

FWIW, I don't agree that reactive fragmentation is an important capability.  Reactive fragmentation removes the necessity of retransmitting N bytes of some bundle's payload (of length Q, Q > N) in the event that connectivity is interrupted in the midst of transmitting that bundle, after N bytes have been transmitted and received; it also enables the receiving BP agent to forward the received N bytes of that bundle immediately instead of waiting for the rest of the payload to arrive.  That can be valuable in the event that Q is large and the transmission data rate is low.  But in that case, wouldn't you want to proactively fragment that bundle anyway, to prevent that large bundle from imposing head-of-line blocking on the CL connection?  If you proactively fragment bundles such that no bundle requires more than 1 second of transmission time, then on link interruption you've wasted at most 1 second of transmission, right?  And all previously received fragments are bundles that are already ready to be forwarded.  If the cost of supporting reactive fragmentation is additional protocol overhead (in both directions) for every bundle, it's not clear to me that the additional complexity is justified.

But that's not relevant to the real question, which is how to write the TCPCL specification.  I agree that it makes sense to add to that specification some language explaining that the purpose of incremental acknowledgment is to support reactive fragmentation.

Scott

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of Clark, Gilbert J. (GRC-LCA0)
Sent: Thursday, January 25, 2018 11:45 AM
To: Marc Blanchet <[email-masked]>; Brian Sipos <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

-Gilbert

    

    
    Marc.

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Thu, 25 Jan 2018 22:03:42 +0000

With regard to reactive fragmentation: I agree that the technical back-and-forth on reactive fragmentation could rapidly turn into a novel and / or infinitely deep rabbit hole … so might be better to take that offline.

And sure, I’m on board with adding necessary language to TCPCL in order to support the rationale for this option. 

I also don’t think I’d be opposed to a companion document that would act as a kind of template for a convergence layer specification.

Alternatively, I would be okay with (my understanding of) Will’s idea of language that maps BP features to individual convergence layers and explains the corresponding intent / context-sensitive interpretation of where they do / don’t make sense.  I’d envision this as appearing something like a checklist to each CL specification that includes text that covers optional features of BP: if the CL supports it, then great!  Specify how and why.  If the CL doesn’t support it, then too bad: explain why it doesn’t.  That would lead to CL specifications with consistent organization … which is something I believe could be beneficial.

Cheers,
Gilbert

On 1/25/18, 4:07 PM, "Burleigh, Scott C (312B)" <[email-masked]> wrote:

    
    
    Scott
    
    -----Original Message-----
    From: dtn [mailto:[email-masked]] On Behalf Of Clark, Gilbert J. (GRC-LCA0)
    Sent: Thursday, January 25, 2018 11:45 AM
    To: Marc Blanchet <[email-masked]>; Brian Sipos <[email-masked]>
    Cc: [email-masked]
    Subject: Re: [dtn] BP Convergence Layer required/optional behavior (and partial-bundle delivery)

    

    -Gilbert

        

        
        Marc.

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 26 Jan 2018 10:14:25 +0000

Hiya,

I agree this is a bit of an aside but...

On 25/01/18 21:06, Burleigh, Scott C (312B) wrote:
> FWIW, I don't agree that reactive fragmentation is an important
> capability.  Reactive fragmentation removes the necessity of
> retransmitting N bytes of some bundle's payload (of length Q, Q > N)
> in the event that connectivity is interrupted in the midst of
> transmitting that bundle, after N bytes have been transmitted and
> received; it also enables the receiving BP agent to forward the
> received N bytes of that bundle immediately instead of waiting for
> the rest of the payload to arrive.  That can be valuable in the event
> that Q is large and the transmission data rate is low.  But in that
> case, wouldn't you want to proactively fragment that bundle anyway,
> to prevent that large bundle from imposing head-of-line blocking on
> the CL connection?  If you proactively fragment bundles such that no
> bundle requires more than 1 second of transmission time, then on link
> interruption you've wasted at most 1 second of transmission, right?
> And all previously received fragments are bundles that are already
> ready to be forwarded.  If the cost of supporting reactive
> fragmentation is additional protocol overhead (in both directions)
> for every bundle, it's not clear to me that the additional complexity
> is justified.

I'm not sure that's fair. IIRC the situations where folks have
seen benefit from reactive fragmentation were ones where the
agent forwarding the N-byte bundle won't have a contact with the
agent who sent the Q-byte bundle again for a long duration. So
the size of Q isn't the only factor. We saw lots of cases of
that happening in the tests we did back around 2010/11. (From
memory, the record was a bundle that was re-constructed and
for which 75 fragments had at some point been created:-)

I do agree that pro-active fragmentation also solves both issues,
with perhaps a little more overhead, but architecturally, we
started with the idea that bundles carry application data units
that are "naturally" sized for the application. Pro-active
fragmentation requires the application developer to assume
things about the CLs and disruptions that'll happen along the
pats that'll be taken, which can be hard to unknowable or may
be something one can calculate, depending on your environment.
(For situations where one can calculate good bundle sizes, it
would be great if someone documented some rules-of-thumb for
what sizes to use.)

But like I said at the start this is an aside, and oughtn't
affect the current documents.

Cheers,
S.

-- 
PGP key change time for me.
New-ID 7B172BEA; old-ID 805F8DA2 expires Jan 24 2018.
NewWithOld sigs in keyservers.
Sorry if that mucks something up;-)

---

From: "R. Atkinson" <[email-masked]>
Date: Fri, 26 Jan 2018 10:44:59 -0500

On January 25th, 2018, Brian Sipos wrote, in part:
% My concern is that a CL writer/reviewer would have to wade through possibly years 
% of mailing list discussion to be able to piece together all of the things that a CL could 
% possibly be expected to provide. 

All,

With apologies if this is preaching to the choir, it is widely understood within the IETF at large 
that implementers CANNOT be expected to look at any mailing list archives (or similar data sources) 
when coding up an implementation of an IETF protocol specification.  We have decades
of experience with this and naive implementers abound (indeed are growing over the decades 
both in absolute terms and as a percentage of protocol).

It is generally accepted by the IETF at large as reasonable to expect an implementer to look at 
publicly available RFCs (or other freely available protocol specifications) that are cited in References 
when coding up an implementation.  This includes reference Informational-status RFCs and even 
non-IETF Individual-Submissions-Track RFCs — when cited as a Reference in an IETF protocol 
specification.

To me the critical question here is not about the behaviour of particular implementations, 
but about the probability of widespread interoperability of varying implementations.

Anything that hasn’t been documented in an RFC (the RFC having whichever status and 
living on whichever RFC publication track) likely will not interoperate reliably in a heterogenous
deployment.  If DTN and BP are successful, heterogeneous implementations likely will be
very commonplace.

So I would encourage the WG to seriously consider following these principles:
--------------------------------------------------------------------------------------------------------

- Folks interested in interoperability/support for particular behaviours ought to ensure that
  those behaviours are clearly documented in an appropriate place, even if that is only as 
  an Informational status RFC on the Individual-Submission track.  Should there be consensus
  that a behaviour is desirable then probably it is a standards-track RFC on the IETF track.

- The DTN standards-track specifications ought to at least reference those other RFCs, even 
  if only to say that those behaviours are optional (or even possibly not-recommended should
  the DTN WG so decide).  

- Please note that if the referenced specification is not a standards-track RFC or other standard
  (e.g., ANSI, IEEE, ISO), then the reference might be an “Informational Reference” rather than 
  a “Normative Reference” in the standards-track protocol specification RFC.

I’m happy with whatever the WG decides on the specific technical issues raised in this thread.

However, I feel quite strongly that any behaviour that affects interoperability ought to be documented 
in a readily accessible form (e.g., RFC) and referenced in the standards-track protocol specification
RFCs.  That way a naive implementer 3 years from now has a decent chance to code an interoperable
implementation.

Cheers,

Ran

---

From: Rick Taylor <[email-masked]>
Date: Sun, 28 Jan 2018 11:40:19 +0000

All,

(Chair Hat On)

Ran is correct. Forcing people to wade through mailing lists to find required behaviour is wrong, and if the content of mailing list discussion results in required behaviour it must be documented in a 'proper' stand-alone document.

I disagree with Gilbert that any extended fragmentation discussion should be taken offline.  Sure, let's not hog the mic at a meeting on this topic, but the mailing list is absolutely the right place to have this discussion.  If people are already having an offline conversation, can I please request you (re)include the mailing list.

Although the topic of fragmentation is little beyond the scope of the current BP work, it seems to be such an important concept that some kind of consensus needs to be found, and that discussion should happen here.  If we need to produce a document detailing any consensus, than I'm sure we can find volunteers to type something up once consensus starts to form.

Cheers,

Rick

> From: dtn [mailto:[email-masked]] On Behalf Of R. Atkinson
> Sent: 26 January 2018 15:45
> To: [email-masked]
> % My concern is that a CL writer/reviewer would have to wade through
> possibly years % of mailing list discussion to be able to piece together all of
> the things that a CL could % possibly be expected to provide.
> 
> All,
> 
> With apologies if this is preaching to the choir, it is widely understood within
> the IETF at large that implementers CANNOT be expected to look at any
> mailing list archives (or similar data sources) when coding up an
> implementation of an IETF protocol specification.  We have decades of
> experience with this and naive implementers abound (indeed are growing
> over the decades both in absolute terms and as a percentage of protocol).
> 
> It is generally accepted by the IETF at large as reasonable to expect an
> implementer to look at publicly available RFCs (or other freely available
> protocol specifications) that are cited in References when coding up an
> implementation.  This includes reference Informational-status RFCs and even
> non-IETF Individual-Submissions-Track RFCs — when cited as a Reference in
> an IETF protocol specification.
> 
> To me the critical question here is not about the behaviour of particular
> implementations, but about the probability of widespread interoperability of
> varying implementations.
> 
> Anything that hasn’t been documented in an RFC (the RFC having whichever
> status and living on whichever RFC publication track) likely will not
> interoperate reliably in a heterogenous deployment.  If DTN and BP are
> successful, heterogeneous implementations likely will be very commonplace.
> 
>