Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

From: <[email-masked]>
Date: Mon, 2 May 2016 00:18:26 +0000 (UTC)

There are a couple of ways of looking at this:

1. TCP is neither disruption- nor delay-tolerant.
Our handy conceptual chart (Fig 1 of our 'Sharing the Dream'

http://sat-net.com/L.Wood/publications/e-dtn-oct-2009-wood-position-paper.pdf )

shows delay vs disruption, and streaming TCP sits firmly in
the internet cloud at lower right. Extending TCP and its timers
to deep space at top right is a fool's errand (though it doesn't
stop some publishing the idea repeatedly, often by ignoring
SYN/SYN-ACK startup delay...), while extending TCP and its two-way
streaming to disruption tolerance at the left has similar problems,
because TCP has fairly high setup costs, assumes stable endpoints,
etc. So, probably also a fool's errand; like deep-space comms, just
not where TCP lives.

2. What a convergence layer actually needs to do has never
been clearly defined. TCP is not the convergence layer; the
convergence layer in this case uses TCP. But, because,
like the Dark Side, TCP is so seductive and easy to go to,
the minimal one-way streaming and pipelining gets implemented
because that's how TCP works, and then the convergence 

layer turns out to be the bare minimum needed to get TCP
working and declare success. Even though, as shown in 1., TCP
and its streaming (leading to pipelining into TCP's buffers,
where you can't really see what's going on) is really not
suitable for delay- or disruption-tolerant networking.

Still, these are merely theoretical objections to use of TCP
in these scenarios.

Good luck with the practical implementation!

Lloyd Wood
http://sat-net.com/L.Wood/dtn

----- Original Message -----
From: "Burleigh, Scott C (312B)" <[email-masked]>
To: Stephen Farrell <[email-masked]>; Eric Travis <[email-masked]>; Marc Blanchet <[email-masked]>
Cc: Gilbert Clark <[email-masked]>; "[email-masked]" <[email-masked]>
Sent: Saturday, 30 April 2016, 7:31
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of Stephen Farrell
Sent: Friday, April 29, 2016 1:40 PM
To: Eric Travis; Marc Blanchet
Cc: Gilbert Clark; [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

>    ...
>    I'd also be interested if Scott could document the issues he found with the TCPCL. If that assertion is the main basis for picking UDP over TCP then it seems fair to ask for that. If OTOH the main argument is that UDP is simpler and sufficient then documenting the problems with the TCPCL isn't needed now (but would be interesting of course.)

Of course, absolutely fair.  The context for this is that we were implementing TCPCL specifically to deal with the case of a TCP connection being spontaneously lost -- machine rebooted, cable was cut, whatever -- without TCP having had the opportunity to signal the failure.  Here's what I found.

First, the intended operation of LENGTH messages depends on the implementation of REFUSE_BUNDLE messages (1st paragraph of 5.5), but in the general case it is impossible to implement the REFUSE_BUNDLE feature correctly at the bundle sender.

The reason bundle refusal cannot in general be implemented correctly at the bundle sender is that bundles over the TCPCL connection may be pipelined (6th paragraph of 3.0, 2nd paragraph of 3.2).  Since bundles can be pipelined, it is possible for all segments of bundles X and X+1, and the first N segments of bundle X+2, to have been transmitted by the time the sending node receives a BUNDLE_REFUSAL message for message X.

Suppose the receiving node sent a BUNDLE_REFUSAL upon reception of the first segment of bundle X.  The sending node can keep a record of transmitted bundles that have not yet been fully acknowledged, so it knows the unacknowledged oldest bundle and can properly apply the refusal to that bundle.  It can respond correctly to this refusal message by reforwarding bundle X or whatever, depending on the refusal message’s reason code.  No problem.

However, bundle X might have been transmitted in, say, 3 segments.  The 6th paragraph of section 5.4 says that the receiving node SHOULD (but is not required to) refuse the other two segments of bundle X – which were placed in the pipeline long ago – in the same way.  Suppose it does so.  Now the sending node receives two additional BUNDLE_REFUSAL messages.  However, there is no bundle identifier in the BUNDLE_REFUSAL message; the second refusal message might be a refusal of the second segment of bundle X, or it might alternatively be a refusal of the first segment of bundle X+1.  Both behaviors are conformant.  The sending node has no way of distinguishing between the two, so it might erroneously determine that bundles X+1 and X+2 were refused.

There’s a problem with acknowledgments too.  Again, pipelining is the issue.  When an acknowledgment of a TCPCL data segment message arrives it notes the total bundle length received so far.  Clearly, if this length is less than the length previously reported – and the previous ACK didn’t acknowledge the tail end of a bundle – there has been a transmission failure and the bundle that was previously being acknowledged did not arrive and must be retransmitted; the new acknowledgment is acknowledging the next bundle for which it has received data.  But how many bundles immediately following that lost bundle also didn’t arrive (and were not acknowledged)?  Which of those subsequently transmitted bundles is now being acknowledged?  We have no way of knowing, because there are no bundle identifiers in acknowledgment messages.

A last point, which isn't such a big deal: the “request enabling of reactive fragmentation” flag doesn’t actually do anything.  Reactive fragmentation isn’t defined anywhere; not in RFC 7242, and not in RFC 5050.  There is no formal definition anywhere of how to do it, there is no procedure anywhere in TCPCL that depends on it, and there is no definition in RFC 7242 of what is intended when the two ends of a TCPCL connection agree to enable it.  So the operation of the flag can't actually be implemented from the specification.

Scott

_______________________________________________
dtn mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/

---

From: Brian Sipos <[email-masked]>
Date: Mon, 2 May 2016 13:11:30 +0000

I have been following this discussion from the sidelines, and would like to offer my time/effort towards a TCPCL spec. I have a specific need for (something like) TCPCL on ground--ground hops and LTPCL for RF-link hops.

For what it's worth I agree that BPbis + CBOR specs should go together, and having *some* CL spec to go along with it will only ease prototyping and interoperability testing but I wouldn't pin BPbis approval on a CL unless it would significantly improve the approval process itself.
 My (biased) preference would be for TCPCL because it is useful for both testing as well as operational use (i.e. no special congestion control). From browsing RFC 7242 and Scott's comments, it appears that a workable TCPCL is not too far off.

________________________________________
From: dtn <[email-masked]> on behalf of Stephen Farrell <[email-masked]>
Sent: Friday, April 29, 2016 17:57
To: Burleigh, Scott C (312B); Eric Travis; Marc Blanchet
Cc: Gilbert Clark; [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Thanks. That's convinced me that that'll need fixing regardless of
whether the TCPCL is the 1st or last CL done.

Cheers,
S

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 2 May 2016 10:04:54 -0500

Just a note from your friendly AD ...

On Fri, Apr 29, 2016 at 5:07 PM, Gilbert Clark <[email-masked]>
wrote:

>
> Doing some initial research on what transport folks might expect is
>> needed for such a good-enough answer for UDP congestion control in a
>> DTN would seem prudent though - that could be really simple or it
>> might be really complex, I'm just not sure.
>>
>
> The relevant RFC of which I'm aware is 5405 - "Unicast UDP Usage
> Guidelines for Application Designers".  Quoting from 3.1.1 of that:
>
> "Applications that perform bulk transmission of data to a peer over UDP,
> i.e., applications that exchange more than a small number of UDP datagrams
> per RTT, SHOULD implement TCP-Friendly Rate Control (TFRC) [RFC5348],
> window-based, TCP-like congestion control, or otherwise ensure that the
> application complies with the congestion control principles."
>
> My thought would be that we use this as a starting point, and establish /
> work through additional requirements as we encounter them (either before or
> during the review process).  Does this seem sane?

https://datatracker.ietf.org/doc/draft-ietf-tsvwg-rfc5405bis/ is out of the
working group and will probably be entering IESG Evaluation this week.

If people are looking at RFC 5405, I'd suggest looking at its replacement
draft before you get too far down the road ...

Spencer

> -Gilbert
> https://www.ietf.org/mailman/listinfo/dtn
>

---

From: Gilbert Clark <[email-masked]>
Date: Mon, 2 May 2016 12:41:27 -0400

Inline:

On 04/29/2016 03:39 PM, Keith Scott wrote:
> Hmmm...  So as far as the CLA question goes, you're saying that in 
> order for the spec to move forward there has to be *A* CLA definition, 
> and any (later) BPbis implementation claiming compliance has to 
> include compliance with *SOME* specified CLA?  Would you accept (for a 
> later BPbis impl.) a CLA specified by the Consultative Committee for 
> space Data Systems (CCSDS)?  I'm thinking in particular a CCSDS 'Blue 
> Book' which has roughly the same longevity / rigor as an RFC.  I'd 
> like for the space community to be able to use 5050bis in conjunction 
> (possibly) with a CLA adapted to a space-specific link layer that 
> might not see the light of day in the IETF.
>
> Seems reasonable to me.
>

 From a pragmatic perspective: as long as it works reasonably well, is 
written down, is understandable, and everyone agrees to do it ... it 
really doesn't matter which standards body it comes from, I think.

 From an IETF process perspective, I do not know exactly what the 
requirements or caveats might be.  Off the top of my head, though: from 
memory, all normative references included within an RFC must be 
published before the RFC which cites the reference may also be 
published.  Thus, I would think that the referenced CCSDS document would 
need to have already been published at the time the RFC were published 
as well.

Could be wrong, though.

> It does mix things a bit (IMHO) to conflate CLA and BP in a single 
> document.  From a quick read I'd concur that at least 7122 seems like 
> it could be directly applied to 5050bis.  I see your point about 7122 
> being experimental from a purist standpoint, but given that 7122 
> exists, and given that there will probably be a desire to do a new / 
> better TCPCL (and probably others), is it worth holding up BP for it?  
> In my opinion 7122 applied to 5050bis would be sufficient.
>

Basing a proposed standard an experimental draft is never a good thing.  
Anything is only ever as strong as its weakest element, and DTN is no 
exception.  I'd suggest omitting a reference to a CLA draft entirely 
before directly adding a normative reference to 7122.

Informative might be another matter, but it still seems a little sketchy 
to me ...

-Gilbert

---

From: "gclark mti-systems.com" <[email-masked]>
Date: Tue, 3 May 2016 09:03:38 -0400 (EDT)

> On April 29, 2016 at 6:27 PM Stephen Farrell <[email-masked]>
> wrote:
> 
> 
> 
> 
> On 29/04/16 23:07, Gilbert Clark wrote:
> > 
> > i.e., applications that exchange more than a small number of UDP
> > datagrams per RTT, SHOULD implement TCP-Friendly Rate Control (TFRC)
> > [RFC5348], window-based, TCP-like congestion control, or otherwise
> > ensure that the application complies with the congestion control
> > principles."
> > 
> > My thought would be that we use this as a starting point, and establish
> > / work through additional requirements as we encounter them (either
> > before or during the review process).  Does this seem sane?
> 
> Yes. Where it might get tricky is if one re-considers the various
> formulae in 5348 in the context of various kinds of DTN. Perhaps
> there're good-enough ways to side-step having to do that, but if
> not, then it could be a bunch of work to know how to tell a sender
> to be nice with BP/UDP when the sender doesn't know what kinds of
> delay and disruption will be experienced by the packets they send
> to the next BPA.
> 
> A (not *the*, but just *a*:-) danger here is that we end up saying
> that it's only for-sure ok to use the UDPCL where TCP would work,
> leading folks to use a TCPCL instead. If the WG want to encourage
> use of UDPCL even in situations where TCP doesn't work well, then
> that may require tackling the above TFRC-in-a-DTN issues.

Well, this is an interesting perspective, and wasn't something I had really
thought about.  I automatically assumed that a standardized convergence layer
would only be operating in an internet-like setting, because that's where I
assumed cross-implementation interaction would be most likely to occur.  It is
true that this is not necessarily the case, however ...

With the above said, perhaps we could argue for two modes of operation in a
UDPCL: one mode of operation that was *required* if relevant traffic were
passing through the internet, and another mode of operation that would be
allowed in the event that traffic were to remain local to a given DTN.  For
internet operation (read: TCP-friendly environments), we explain how the
convergence layer must conform with 5405bis (thanks Spencer :).  We then add a
section that notes these mechanics *may* be omitted in other, more exotic
situations ... but only when a determination is made that such omission will not
be harmful to the network.  

Notably, this would allow for TFRC (or equivalent mechanisms) to be omitted from
platforms which might have the most difficulty supporting the additional
complexity such code might introduce.

I think that the above text might be sufficient to address both the DTN case and
the internet case, without necessitating the design of a new DTN friendly rate
control (dropping the T, since it needs to operate in places that TCP wouldn't
:).  Generally, I would be hesitant to start down this path, since I have a
feeling that the mechanics of any kind of effective DTN-friendly rate control
would likely depend on the environment into which the rate control was being
deployed.

-Gilbert

---

From: "Marc Blanchet" <[email-masked]>
Date: Mon, 16 May 2016 20:27:20 -0400

We need reviewers and reviews. We can’t forward this document to IESG 
without more reviews. If you have read and agreed without any comment, 
please say so.

deadline: may 25th.

Regards, Marc.

On 27 Apr 2016, at 8:26, Brian Haberman wrote:

> All,
>      This note starts a 4-week DTN Working Group last call on 
> advancing:
>
>         Title           : Bundle Protocol
>         Authors         : Scott Burleigh
>                           Kevin Fall
>                           Edward J. Birrane
>         Filename        : draft-ietf-dtn-bpbis-03.txt
>
> as a Proposed Standard. Substantive comments should be sent to the DTN
> mailing list. Editorial comments can be sent to the document authors.
> This last call will end May 25, 2016.
>
> Regards,
> Brian
>

---

From: <[email-masked]>
Date: Tue, 17 May 2016 00:39:34 +0000

This document is not ready to be a proposed standard, and not ready to forward to IESG.

Lloyd Wood
http://sat-net.com/L.Wood/dtn
________________________________
From: dtn <[email-masked]> on behalf of Marc Blanchet <[email-masked]>
Sent: Tuesday, 17 May 2016 10:27:20 AM
To: [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

We need reviewers and reviews. We can't forward this document to IESG without more reviews. If you have read and agreed without any comment, please say so.

deadline: may 25th.

Regards, Marc.

All,
This note starts a 4-week DTN Working Group last call on advancing:

    Title           : Bundle Protocol
    Authors         : Scott Burleigh
                      Kevin Fall
                      Edward J. Birrane

Regards,
Brian

________________________________

dtn mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 18 May 2016 14:09:27 +0000

I will review and send comments by Wed. 25th.

Fred

From: dtn [mailto:[email-masked]] On Behalf Of Marc Blanchet
Sent: Monday, May 16, 2016 5:27 PM
To: [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

We need reviewers and reviews. We can’t forward this document to IESG without more reviews. If you have read and agreed without any comment, please say so.

deadline: may 25th.

Regards, Marc.

All,

    Title           : Bundle Protocol

    Authors         : Scott Burleigh

                      Kevin Fall

                      Edward J. Birrane

Regards,
Brian

dtn mailing list

---

From: Keith Scott <[email-masked]>
Date: Wed, 18 May 2016 10:37:35 -0400

I have reviewed draft-ietf-dtn-bpbis-03 and have the comments below.

==== Comments ====

######

Section 1 (Introduction)
Just below Figure 1 the document states:
"This document describes the format of the protocol data units..."

maybe change this to:

"This document describes the logical (or abstract?) contents of the data
units..."

Rationale: This document no longer describes PDU formats but the logical
contents of the to-be-constructed PDUs.  This is made clear at the
beginning of section 4.

######

Section 3.1 (Definitions)
Minimum transmission group is defined but never used.  I think this may
have been intended to be "Minimum Reception Group" which *is* used later
(but not defined)?

------

In the definition of a Block:
maybe change
"... one of the protocol data structures..."
to
"... one of the logical (or abstract?) protocol data structures..."

######

Section 4.6 (Extension Blocks)
Should we drop the descriptions of the Manifest and Flow Label blocks since
their specifications are (TBD) and there are no references (even I-Ds) for
them?  The Flow label is also mentioned later.

Without any specification, what does mentioning these here and reserving
block IDs for them get us?

######

5.2 (Bundle Transmission) step 1
The document says:
   If, moreover, custody acceptance by the source node is required but the
   conditions under which custody of the bundle may be accepted are not
   satisfied, then the request cannot be honored and all remaining
   steps of this procedure MUST be skipped.

Is the source node required to take custody if custody transfer is
requested?

######

5.4 (Bundle Forwarding) step 2
What are the two 'o's after the first bullet item under 5.4 step 2?

######

5.12 (Custody Transfer Failure)
As specified, custody transfer is entirely implementation-specific, since
the action taken in response to a custody transfer failure is
implementation-specific.  This means that if there is one node in the path
that takes custody and whose implementation-specific response to custody
failure is 'ignore' then the system is not reliable in the face of:
transmission or node errors resulting in bundle loss; or the need to
re-route bundles that have become 'stuck' (using custody failure as the
bandwidth-efficient mechanism for indicating stuck-ness).

######

I agree that the WG should immediately take up / advance the specification
if at least one encoding mechanism (e.g. CBOR) and one CLA, but I don't
think the need for the additional specifications should hold up the current
document.

The encoding of tunneled bundles might benefit from a recommendation on the
encoding mechanism(possibly a SHOULD and not a MUST).  While the
coordination of encodings and convergence layers for 'adjacent' bundle
routers is a 'local' concern, tunneled bundles could end up many hops from
their head-ends and thus coordination (of the encoding) might be more of an
issue.  This could be addressed in the encoding spec, I think.

=== END ===

On Mon, May 16, 2016 at 8:27 PM, Marc Blanchet <[email-masked]>
wrote:

> ------------------------------
>
> dtn mailing list
> [email-masked]

---

From: Carlo Caini <[email-masked]>
Date: Wed, 18 May 2016 20:44:58 +0200

Dear Keith,
       a few comments about the source taking
custody. It seems weird, but it has, or better
could have, a rationale (maybe 3!):
1) BP on the source is obliged to select a
reliable convergence layer if it accepts the
custody, following RFC4838 (less certain in
RFC5050 and still most uncertain in the draft)
2) If you have a source node that is totally
disconnected, you can still send a bundle (and to
send here means to pass it a content to the BP).
If the custody requested is on, the BP can (it is
not obliged, but usually does) save it in
permanent memory instead of just (possibly only)
in RAM. If the node is restarted and the bundle
is in permanent memory everything is OK. This is
my interpretation of RFC4838 ("e.g. by storing
them in permanent storage" see below)
3) Moreover, the custody may implies also
re-transmissions without intervention of the
application on the source. I.e. it allows the
source application a sort of "fire-and-forget" behavior.

The consideration 2 (permanent memory) holds true
even in the possibly more weird case of BP on
destination taking custody. Once the bundle
arrives at destination, it is usually first saved
in permanent memory if the custody is on, as a
safety measure, then passed to the application.
Usually the two actions are concurrent, but not
always. In fact, you can have the intended
application temporarily switched off, and  the
bundle can be kept into custody until the
application goes on (or the lifetime expires). In
the interval, if the node is rebooted, and you
have the bundle in RAM, the bundle is lost, saved otherwise.

This is all for the rationale. Now I have some
comments about custody definitions in RFCs and in the draft.

In the draft the definition of custody is
indirect, via the definition of custodian (see below).
There is no more mention of reliability of
convergence layer (maybe there was not even in
RFC5050), so point 1) would no more apply.
Actually, the excerpt reported below from
RFC5050bis seems to suggest that the use of
reliable convergence layer and custody are
alternative instead of complementary mechanisms,
which would be in contrast both to RFC4838 and
maybe also to everyday preferences and practice of most users.
I suppose that "for an indefinite period of time"
in the draft implies permanent memory, but is not
certain, as it was not in RFC4838.

In my opinion, these 2 points (reliable CLs and
permanent memory are mandatory or not?) should
deserve further clarification in the draft.
My suggestions are that:
1) the use of a reliable CL is preferable whenever available
2) the use of permanent memory is by contrast mandatory
The rationale of 2) is that it makes robust
bundle transmissions to power offs, reboots and
other node failures (except permanent memory corruption of course!).
Moreover, we could state that reliable CLs and
also re-transmissions, although not strictly
mandatory are recommended in conjunction with
custody acceptance. In most cases they would be
beneficial; therefore, I would state this explicitly. With the key words
should, must etc... as defined in RFC2119, IETF
provides us with a great number of degree of freedom (actually 10!).

Your,s
    Carlo

 From RFC5050bis
Custodian - A custodian of a bundle is a node that has determined
that it will retain a copy of that bundle for an indefinite period
of time, forwarding and possibly re-forwarding the bundle as
appropriate, until it detects one of the conditions under which it
may cease being a custodian of that bundle (discussed later).

However, because custody transfer's only remedy for loss of any part
of a bundle is retransmission of the entire bundle (not just the
lost portion), custody transfer is a less efficient automated
retransmission mechanism than the reliable transport protocols that
are typically available at the convergence layer; configuring BPAs
to use reliable convergence-layer protocols between nodes is
generally the best means of ensuring bundle delivery at the
destination node(s). But there are some use cases (typically
involving unidirectional links) in which custody transfer in BP may
be a more cost-effective solution for reliable transmission between
two BP agents than invoking retransmission protocols at the
convergence layer.

 From RFC5050
Custody - To "accept custody" upon forwarding a bundle is to commit
to retaining a copy of the bundle -- possibly re-forwarding the
bundle when necessary -- until custody of that bundle is
"released".

 From RFC4838
3.6.1
Custody Transfer Requested - requests sent bundles be delivered
with enhanced reliability using custody transfer procedures. Sent
bundles will be transmitted by the bundle layer using reliable
transfer protocols (if available),

Source Node Custody Acceptance Required - requires the source DTN
node to provide custody transfer for the sent bundles. If custody
transfer is not available at the source when this delivery option
is requested, the requested transmission fails. This provides a
means for applications to insist that the source DTN node take
custody of the sent bundles (e.g., by storing them in persistent
storage).

At 16:37 18/05/2016, Keith Scott wrote:
>"This document describes the logical (or
>abstract?) contents of the data units..."
>
>Rationale: This document no longer describes PDU
>formats but the logical contents of the
>to-be-constructed PDUs.  This is made clear at the beginning of section 4.
>
>######
>
>Minimum transmission group is defined but never
>used.  I think this may have been intended to be
>"Minimum Reception Group" which *is* used later (but not defined)?
>
>------
>
>Should we drop the descriptions of the Manifest
>and Flow Label blocks since their specifications
>are (TBD) and there are no references (even
>I-Ds) for them?  The Flow label is also mentioned later.
>
>Without any specification, what does mentioning
>these here and reserving block IDs for them get us?
>
>######
>
>Is the source node required to take custody if custody transfer is requested?
>
>######
>
>As specified, custody transfer is entirely
>implementation-specific, since the action taken
>in response to a custody transfer failure is
>implementation-specific.  This means that if
>there is one node in the path that takes custody
>and whose implementation-specific response to
>custody failure is 'ignore' then the system is
>not reliable in the face of: transmission or
>node errors resulting in bundle loss; or the
>need to re-route bundles that have become
>'stuck' (using custody failure as the
>I agree that the WG should immediately take up /
>advance the specification if at least one
>encoding mechanism (e.g. CBOR) and one CLA, but
>I don't think the need for the additional
>specifications should hold up the current document.
>
>The encoding of tunneled bundles might benefit
>from a recommendation on the encoding
>mechanism(possibly a SHOULD and not a
>MUST).  While the coordination of encodings and
>convergence layers for 'adjacent' bundle routers
>is a 'local' concern, tunneled bundles could end
>up many hops from their head-ends and thus
>coordination (of the encoding) might be more of
>an issue.  This could be addressed in the encoding spec, I think.
>
>
>=== END ===
>
>
>On Mon, May 16, 2016 at 8:27 PM, Marc Blanchet
><<mailto:[email-masked]>[email-masked]> wrote:
>
>We need reviewers and reviews. We canâEUR(tm)t
>forward this document to IESG without more
>reviews. If you have read and agreed without any comment, please say so.
>
>deadline: may 25th.
>
>Regards, Marc.
>
>----------
><mailto:[email-masked]>[email-masked]
><mailto:[email-masked]>[email-masked]

5x1000 AI GIOVANI RICERCATORI
DELL'UNIVERSITÀ DI BOLOGNA
Codice Fiscale: 80007010376

---

From: William Immerman <[email-masked]>
Date: Wed, 18 May 2016 22:07:03 -0400

Carlo—

I completely agree that an important and under-appreciated application for BP is providing “fire-and-forget” behavior to the application/user (why should users or applications in disadvantaged environments assume they have to watch a spinning wheel to ensure that their message has been sent?).

I don’t think, though, that making “permanent storage” a mandatory feature rather than an implementation concern is needed or best … I think there are applications where an implementation might want a bundle forwarder to reside in volatile storage and wouldn’t want that to violate required semantics. It might mean that such a node should not accept custody, or other limitations, but I thought that practical implementation of DTN/BP still assumes it’ll be a “best-effort” protocol. Aside from possible utility, I’m not sure we’ll ever be able to guarantee that remote hardware won’t fail permanently or that contacts will be available to all isolated nodes in time to avoid expiration of bundles, so we might as well embrace “best-effort”.

Bill Immerman

> On May 18, 2016, at 2:44 PM, Carlo Caini <[email-masked]> wrote:
> 
> Dear Keith,

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 19 May 2016 15:49:54 +0000

Thanks, Keith.  Some remarks in-line below.

Scott

From: dtn [mailto:[email-masked]] On Behalf Of Keith Scott
Sent: Wednesday, May 18, 2016 7:38 AM
To: Marc Blanchet <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

==== Comments ====

######

Section 1 (Introduction)

maybe change this to:

"This document describes the logical (or abstract?) contents of the data units..."

Rationale: This document no longer describes PDU formats but the logical contents of the to-be-constructed PDUs.  This is made clear at the beginning of section 4.

>>           Good catch.  But I think it’s cleaner just to remove the words “the format of”.  The document does indeed describe the data units, it just doesn’t describe their formats.

######

Section 3.1 (Definitions)
Minimum transmission group is defined but never used.  I think this may have been intended to be "Minimum Reception Group" which *is* used later (but not defined)?

>>           Another good catch.  “Transmission” needs to be changed to “reception”.

------

>>           Here I disagree.  In this context I think “protocol data structures” are innately abstract; in that sense, I would say a block is a protocol data structure just as a bundle is a protocol data structure.  It’s only the digital representation of the data structures that is omitted from the specification.  I think this is not unlike the notion of data structures as defined in many programming languages: the bit-for-bit representation of a structure is often implementation-dependent.

######

Section 4.6 (Extension Blocks)
Should we drop the descriptions of the Manifest and Flow Label blocks since their specifications are (TBD) and there are no references (even I-Ds) for them?  The Flow label is also mentioned later.

Without any specification, what does mentioning these here and reserving block IDs for them get us?

>>           I like the idea of keeping them as reminders of the intent of the Working Group, but I would be fine with removing them if others agree with you.

######

>>           No, the determination of whether or not the source node is required to take custody when custody transfer is required is an implementation matter.  But we should say so explicitly here.

######

>>           I think they’re an artifact of drafting this document in Word and then converting to text.

######

5.12 (Custody Transfer Failure)
As specified, custody transfer is entirely implementation-specific, since the action taken in response to a custody transfer failure is implementation-specific.  This means that if there is one node in the path that takes custody and whose implementation-specific response to custody failure is 'ignore' then the system is not reliable in the face of: transmission or node errors resulting in bundle loss; or the need to re-route bundles that have become 'stuck' (using custody failure as the bandwidth-efficient mechanism for indicating stuck-ness).

>>           This is true; I think it’s the way RFC5050 always was written.  There was strong resistance within DTNRG to making much of the custody transfer behavior mandatory, as it might exceed the available resources on some limited platforms.

######

I agree that the WG should immediately take up / advance the specification if at least one encoding mechanism (e.g. CBOR) and one CLA, but I don't think the need for the additional specifications should hold up the current document.

>>           I agree with you.

The encoding of tunneled bundles might benefit from a recommendation on the encoding mechanism(possibly a SHOULD and not a MUST).  While the coordination of encodings and convergence layers for 'adjacent' bundle routers is a 'local' concern, tunneled bundles could end up many hops from their head-ends and thus coordination (of the encoding) might be more of an issue.  This could be addressed in the encoding spec, I think.

>>           This is an interesting point.  I think I would contend that in the case of tunneled bundles the head-ends ARE topologically adjacent, within the overlay “network” formed by the tunnel.  So the coordination of encoding is an issue but I think it’s the same issue at each layer of BP, and whatever mechanisms we come up with to achieve this coordination ought to work at all layers.

=== END ===

On Mon, May 16, 2016 at 8:27 PM, Marc Blanchet <[email-masked]<mailto:[email-masked]>> wrote:

deadline: may 25th.

Regards, Marc.

All,

    Title           : Bundle Protocol

    Authors         : Scott Burleigh

                      Kevin Fall

                      Edward J. Birrane

Regards,
Brian

dtn mailing list

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 19 May 2016 16:01:04 +0000

I am inclined to agree with Bill on this point.  We're trying to strike a balance between specifying enough mandatory behavior to make the operation of the network efficient and verifiable but not so much as to exclude resource-limited platforms from participating.  I think the way to do that is to minimize required behavior while always making sure that individual nodes can continue effective operation despite functional lapses at other nodes.  The network might never operate at theoretically optimal performance, but it should never fail altogether.

Scott

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of William Immerman
Sent: Wednesday, May 18, 2016 7:07 PM
To: Carlo Caini <[email-masked]>
Cc: Marc Blanchet <[email-masked]>; Keith Scott <[email-masked]>; [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Carlo—

Bill Immerman

>      a few comments about the source taking custody. It seems weird, 
> but it has, or better could have, a rationale (maybe 3!):
> 1) BP on the source is obliged to select a reliable convergence layer 
> if it accepts the custody, following RFC4838 (less certain in
> 2) If you have a source node that is totally disconnected, you can 
> still send a bundle (and to send here means to pass it a content to 
> the BP).
> If the custody requested is on, the BP can (it is not obliged, but 
> usually does) save it in permanent memory instead of just (possibly 
> only) in RAM. If the node is restarted and the bundle is in permanent 
> memory everything is OK. This is my interpretation of RFC4838 ("e.g. 
> by storing them in permanent storage" see below)
> 3) Moreover, the custody may implies also re-transmissions without 
> intervention of the application on the source. I.e. it allows the 
> The consideration 2 (permanent memory) holds true even in the possibly 
> more weird case of BP on destination taking custody. Once the bundle 
> arrives at destination, it is usually first saved in permanent memory 
> if the custody is on, as a safety measure, then passed to the 
> application.
> Usually the two actions are concurrent, but not always. In fact, you 
> can have the intended application temporarily switched off, and  the 
> bundle can be kept into custody until the application goes on (or the 
> lifetime expires). In the interval, if the node is rebooted, and you 
> This is all for the rationale. Now I have some comments about custody 
> definitions in RFCs and in the draft.
> 
> In the draft the definition of custody is indirect, via the definition 
> of custodian (see below).
> There is no more mention of reliability of convergence layer (maybe 
> there was not even in RFC5050), so point 1) would no more apply.
> Actually, the excerpt reported below from RFC5050bis seems to suggest 
> that the use of reliable convergence layer and custody are alternative 
> instead of complementary mechanisms, which would be in contrast both 
> to RFC4838 and maybe also to everyday preferences and practice of most 
> users.
> in the draft implies permanent memory, but is not certain, as it was 
> not in RFC4838.
> 
> In my opinion, these 2 points (reliable CLs and permanent memory are 
> mandatory or not?) should deserve further clarification in the draft.
> My suggestions are that:
> 2) the use of permanent memory is by contrast mandatory The rationale 
> of 2) is that it makes robust bundle transmissions to power offs, 
> reboots and other node failures (except permanent memory corruption of 
> course!).
> Moreover, we could state that reliable CLs and also re-transmissions, 
> although not strictly mandatory are recommended in conjunction with 
> custody acceptance. In most cases they would be beneficial; therefore, 
> I would state this explicitly. With the key words should, must etc... 
> as defined in RFC2119, IETF provides us with a great number of degree 
> of freedom (actually 10!).
> 
> Your,s
>   Carlo
> 
> 
> From RFC5050bis
> Custodian - A custodian of a bundle is a node that has determined that 
> it will retain a copy of that bundle for an indefinite period of time, 
> forwarding and possibly re-forwarding the bundle as appropriate, until 
> it detects one of the conditions under which it may cease being a 
> custodian of that bundle (discussed later).
> 
> of a bundle is retransmission of the entire bundle (not just the lost 
> portion), custody transfer is a less efficient automated 
> are typically available at the convergence layer; configuring BPAs to 
> use reliable convergence-layer protocols between nodes is generally 
> the best means of ensuring bundle delivery at the destination node(s). 
> But there are some use cases (typically involving unidirectional 
> links) in which custody transfer in BP may be a more cost-effective 
> solution for reliable transmission between two BP agents than invoking 
> retransmission protocols at the convergence layer.
> 
> 
> From RFC5050
> Custody - To "accept custody" upon forwarding a bundle is to commit to 
> retaining a copy of the bundle -- possibly re-forwarding the bundle 
> when necessary -- until custody of that bundle is "released".
> 
> From RFC4838
> 3.6.1
> Custody Transfer Requested - requests sent bundles be delivered with 
> enhanced reliability using custody transfer procedures. Sent bundles 
> will be transmitted by the bundle layer using reliable transfer 
> protocols (if available),
> 
> Source Node Custody Acceptance Required - requires the source DTN node 
> to provide custody transfer for the sent bundles. If custody transfer 
> is not available at the source when this delivery option is requested, 
> the requested transmission fails. This provides a means for 
> applications to insist that the source DTN node take custody of the 
> sent bundles (e.g., by storing them in persistent storage).
> 
> 
>> Rationale: This document no longer describes PDU formats but the 
>> logical contents of the to-be-constructed PDUs.  This is made clear 
>> at the beginning of section 4.
>> 
>> ######
>> 
>> Minimum transmission group is defined but never used.  I think this 
>> may have been intended to be "Minimum Reception Group" which *is* 
>> used later (but not defined)?
>> 
>> ------
>> 
>> Should we drop the descriptions of the Manifest and Flow Label blocks 
>> since their specifications are (TBD) and there are no references 
>> (even
>> As specified, custody transfer is entirely implementation-specific, 
>> since the action taken in response to a custody transfer failure is 
>> implementation-specific.  This means that if there is one node in the 
>> path that takes custody and whose implementation-specific response to 
>> custody failure is 'ignore' then the system is not reliable in the 
>> face of: transmission or node errors resulting in bundle loss; or the 
>> need to re-route bundles that have become 'stuck' (using custody 
>> failure as the bandwidth-efficient mechanism for indicating 
>> stuck-ness).
>> 
>> ######
>> 
>> I agree that the WG should immediately take up / advance the 
>> specification if at least one encoding mechanism (e.g. CBOR) and one 
>> CLA, but I don't think the need for the additional specifications 
>> should hold up the current document.
>> 
>> The encoding of tunneled bundles might benefit from a recommendation 
>> on the encoding mechanism(possibly a SHOULD and not a MUST).  While 
>> the coordination of encodings and convergence layers for 'adjacent' 
>> bundle routers is a 'local' concern, tunneled bundles could end up 
>> many hops from their head-ends and thus coordination (of the 
>> encoding) might be more of an issue.  This could be addressed in the 
>> encoding spec, I think.
>> 
>> 
>> === END ===
>> 
>> 
>> We need reviewers and reviews. We canâEUR(tm)t forward this document 
>> to IESG without more reviews. If you have read and agreed without any 
>> comment, please say so.
>> 
>> deadline: may 25th.
>> 
>> Regards, Marc.
>> 
>> as a Proposed Standard. Substantive comments should be sent to the 
>> DTN mailing list. Editorial comments can be sent to the document authors.

---

From: Keith Scott <[email-masked]>
Date: Thu, 19 May 2016 14:12:39 -0400

Bill and Carlo,

I think the real question goes to 'what does custody transfer mean' and
'what do we want it to do'?

As specified in the current draft, an application's request for custodial
services results in a lot of in-network signaling to manage the question of
'who is the current custodian for this bundle' but, at the critical time
when something fails and it's time for the current custodian to do
something more than pass custody to the next, it's up to the implementation
to decide what to do (including nothing).  The only mandatory action will
be (when the bundle expires) the generation of a deletion status report
(section 5.13).  That's a status report, which does not have the custody
requested bit set, so it really might not even reach the deleted bundle's
report-to endpoint ID.

The draft states that custody transfer's main utility is in recovering from
routing errors rather than as a reliability mechanism (section 3.2):

   The custody transfer mechanism in BP is primarily intended as a
   means of recovering from forwarding failures. When a bundle arrives
   at a node from which it must be forwarded, but forwarding is
   impossible, BP must recover from this error. BP can "return" the
   bundle back toward some node for forwarding along some other path in
   the network, or else it can instead send a small "signal" bundle
   back to such a node, in the event that this node has retained a copy
   of the bundle ("taken custody") and is therefore able to re-forward
   the bundle without receiving a copy. Custody transfer sharply
   reduces the network traffic required for recovery from forwarding
   failures, at the cost of increased buffer occupa

Contrast this with the description of Reliability and Custody Transfer in
section 3.10 of RFC4838:

   3.10.  Reliability and Custody Transfer

   The most basic service provided by the bundle layer is
   unacknowledged, prioritized (but not guaranteed) unicast message
   delivery.  It also provides two options for enhancing delivery
   reliability:  end-to-end acknowledgments and custody transfer.
   Applications wishing to implement their own end-to-end message
   reliability mechanisms are free to utilize the acknowledgment.  The
   custody transfer feature of the DTN architecture only specifies a
   coarse-grained retransmission capability, described next.

   ---- [SNIP] ----

   The nodes
   receiving these bundles along the way (and agreeing to accept the
   reliable delivery responsibility) are called "custodians".  The
   movement of a bundle (and its delivery responsibility) from one node
   to another is called a "custody transfer".  It is analogous to a
   database commit transaction [FHM03].  The exact meaning and design of
   custody transfer for multicast and anycast delivery remains to be
   fully explored.

   Custody transfer allows the source to delegate retransmission
   responsibility and recover its retransmission-related resources
   relatively soon after sending a bundle (on the order of the minimum
   round-trip time to the first bundle hop(s)).

   ---- [SNIP] ----

   Custody transfer in DTN [...] also provides a (weak) mechanism
   for enhancing the reliability of message delivery.  Generally
   speaking, custody transfer relies on underlying reliable delivery
   protocols of the networks that it operates over to provide the
   primary means of reliable transfer from one bundle node to the next
   (set).  However, when custody transfer is requested, the bundle layer
   provides an additional coarse-grained timeout and retransmission
   mechanism and an accompanying (bundle-layer) custodian-to-custodian
   acknowledgment signaling mechanism.  When an application does *not*
   request custody transfer, this bundle layer timeout and
   retransmission mechanism is typically not employed, and successful
   bundle layer delivery depends solely on the reliability mechanisms of
   the underlying protocols.

   ---- [SNIP] ----

If you control the entire network, use reliable convergence layers
everywhere, always store bundles for which you take custody in persistent
storage, and manage all your data transfers so that you'll never have to
delete a custodial bundle due to running out of resources, then everything
will work.  But if you're operating in an internetwork of heterogeneous BP
implementations, you can never trust custody transfer to provide
reliability because some router you DON'T control might take custody of
your bundle and the best you can hope for is to get a deletion status
report.

The people I know who are interested in using BP need reliability.  Of
course they can build their DTN networks with implementations they control
that implement timeouts, and respond to custody failures by retransmitting
/ rerouting / trying something to get bundles forward, but they'll never be
able to interconnect those network with an 'open' BP network (security
notwithstanding) without some better promise of reliability from conformant
BP implementations.

I agree that determining the value for a retransmission timer is, in the
most general case, difficult or impossible.  However we could place
stronger requirements on implementations w.r.t. attempting to deliver
bundles for which they are the custodians, e.g.:
    On receipt of a failed custody signal, the node SHOULD attempt to
re-forward the bundle.  [depending on what the node knows about topology,
this forwarding decision could try to use a different path, or just try
again on the same path, or something].
    Bundle routers SHOULD set a retransmission timeout when transmitting
bundles for which the nodes are custodians. [rather than the current
language that simply states that such a timer is a possibility]

--keith

On Wed, May 18, 2016 at 10:07 PM, William Immerman <[email-masked]>
wrote:

> Carlo—
>
> I completely agree that an important and under-appreciated application for
> BP is providing “fire-and-forget” behavior to the application/user (why
> should users or applications in disadvantaged environments assume they have
> to watch a spinning wheel to ensure that their message has been sent?).
>
> I don’t think, though, that making “permanent storage” a mandatory feature
> rather than an implementation concern is needed or best … I think there are
> applications where an implementation might want a bundle forwarder to
> reside in volatile storage and wouldn’t want that to violate required
> semantics. It might mean that such a node should not accept custody, or
> other limitations, but I thought that practical implementation of DTN/BP
> still assumes it’ll be a “best-effort” protocol. Aside from possible
> utility, I’m not sure we’ll ever be able to guarantee that remote hardware
> won’t fail permanently or that contacts will be available to all isolated
> nodes in time to avoid expiration of bundles, so we might as well embrace
> “best-effort”.
>
> Bill Immerman
>
>

---

From: Carlo Caini <[email-masked]>
Date: Thu, 19 May 2016 20:08:35 +0000

Dear William and Scott,
     I agree with you on not specifying mandatory behaviors if strictly not necessary, and also on best effort. However, the main point in my mail was that what is not mandatory should 1) clearly stated, and 2) recommended if useful. Now custody seems so vague that could be interpreted as "doing nothing", as pointed out by Keith. This doing nothing (from entity Y nodes) would be harmful to other nodes (entity X nodes).

>From the RFC5050bis:
 "A custodian of a bundle is a node that has determined that it will retain a copy of that bundle for an indefinite period of time,"

A part from his "willingness", it is unclear what is mandatory, or simply recommended, and what is not. It is just not stated. Keith points out that this definition does not impose absolutely anything to anyone. Not only does it not impose anything, but it does not recommend anything too, and this is my main objection.
I am not happy with this total liberty left to implementations, because it has a significant cost in terms of robustness, which is the reasons why DTN has been designed for. It seems very unbalanced in favor of  resource limited platforms  but this is just  my personal opinion. However, why not at least to suggest that, whenever possible, bundles should be kept in permanent memory by custodians?  We are all familiar with present implementations of BP that do this. But how can an unfamiliar reader of the draft deduce that this "willingness" means that a node should not only avoid to deliberately cancel the bundle, but also try hard (best effort, of course, nothing more) to preserve the bundle even in face of shutdowns and reboots (e,g. as a result of power failures). It would be enough to state that  a custodian should save it in permanent memory and not in RAM whenever possible, just anything more than this. The same for retransmissions.
Note that a network may consists of nodes belonging to different entities, running different implementations. The source has no knowledge of the implementation of the following nodes on the path towards destination. In such a situation it would be often preferable not to pass custody to nodes that cannot offer any real guarantee of anything.

Custody and Reliable CLs
The following excerpt from the draft is in my opinion misleading in the last few rows, as custody is not an alternative of, but a complement to reliable CLs to augment reliability (provided that custodians do something, such as retransmitting bundles).
Custody is a sort of further step which is left to the source, not to network maintainers, or intermediate nodes owners (the source cannot choose CLs on intermediate links).
It is worth noting that in the classical data mule scenario A-B-C, with TCP on both links, only custody on A, or better on B (but in this case ONLY IF in permanent memory) can save from a power failure of the mule when it has the bundle on board.
This is because reliability of CLs does not protect bundles when stored in intermediate nodes.
To have this protection, custody with permanent memory compulsory or, if you prefer,  at least strongly recommended, is in my opinion the best solution. As a final remark, note that custody on B without permanent memory would be the worst case in the example above, as it would result in a definitive loss.
I hope to have better expressed my view this time.

Yours,
   carlo

________________________________________
Da: dtn [[email-masked]] per conto di Burleigh, Scott C (312B) [[email-masked]]
Inviato: giovedì 19 maggio 2016 18:01
A: William Immerman; Carlo Caini
Cc: Marc Blanchet; Keith Scott; [email-masked]
Oggetto: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Scott

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of William Immerman
Sent: Wednesday, May 18, 2016 7:07 PM
To: Carlo Caini <[email-masked]>
Cc: Marc Blanchet <[email-masked]>; Keith Scott <[email-masked]>; [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Carlo—

Bill Immerman

---

From: Bill Immerman <[email-masked]>
Date: Thu, 19 May 2016 17:09:14 -0400

All--

I may have confused things by "bundling" concerns about custody and
persistent storage (although I think there's a relationship), so let me
take one stab at un-bundling:

RE: Persistent storage: I agree with Carlo that the usual and preferred
implementation would provide persistent storage (i.e., storage that would
survive a restart of the hw host and sw, at least). Since I've considered
at least one application that wouldn't provide that, I was hoping that
wouldn't demand a non-compliant implementation that was "sort-of like BP".
My comment about custody was that a configuration employing such a node
might have to satisfy certain requirements, like refusing custody, and that
would be ok ... it wasn't my understanding that all forwarding nodes had to
agree to custody. My instinct is that the suggestion that most
implementations of BP support mapping bundle storage to persistent storage,
and that most deployments consider using that feature, might be better
suited to an architecture document or an accompanying implementation
guidance document than to the BP spec itself, but I have no experience
authoring standards and would defer to those who do in deciding where such
(useful) guidance should be kept and published.

RE: Custody: I agree with Keith that reliability is very important in most
anticipated uses of BP. I'm not sure I agree that custody is more about
reliability than about resource management. I'd expect that many isolated,
highly disadvantaged nodes lack a number of resources: contact
opportunities, contact throughput, power, storage, ... Say that A is such a
node, B is A's only neighbor, and is still intermittently connected but has
better resources than A (e.g., more contact), and C is a well-connected
neighbor of B. For example, think of A as a remote sensor, B as a DTN
gateway, and C as an Internet connected DTN forwarder. If A could pass data
to B and have a custody request to B acknowledged in a single contact, then
couldn't A relinquish storage of the forwarded data before it entered a
long period of no contact? (therefore the storage would be more available
for the fire-and-forget use by the DTN-enabled sensor application at A?).
Also, wouldn't that enable C to acknowledge receipt directly to B as
custodian, and not require B to ever rely on future contacts with A to
close the loop? In some cases we can count on and predict periodic future
contacts, and in some cases I'm not sure we can ever count on A not
becoming completely partitioned from B. Maybe this isn't what custody is
for, but it seems like a useful case for something other than being another
intermediate forwarding node ...

Bill

On Thu, May 19, 2016 at 4:08 PM, Carlo Caini <[email-masked]> wrote:

> Dear William and Scott,
>      I agree with you on not specifying mandatory behaviors if strictly
> not necessary, and also on best effort. However, the main point in my mail
> was that what is not mandatory should 1) clearly stated, and 2) recommended
> if useful. Now custody seems so vague that could be interpreted as "doing
> nothing", as pointed out by Keith. This doing nothing (from entity Y nodes)
> would be harmful to other nodes (entity X nodes).
>
> From the RFC5050bis:
>  "A custodian of a bundle is a node that has determined that it will
> retain a copy of that bundle for an indefinite period of time,"
>
> A part from his "willingness", it is unclear what is mandatory, or simply
> recommended, and what is not. It is just not stated. Keith points out that
> this definition does not impose absolutely anything to anyone. Not only
> does it not impose anything, but it does not recommend anything too, and
> this is my main objection.
> I am not happy with this total liberty left to implementations, because it
> has a significant cost in terms of robustness, which is the reasons why DTN
> has been designed for. It seems very unbalanced in favor of  resource
> limited platforms  but this is just  my personal opinion. However, why not
> at least to suggest that, whenever possible, bundles should be kept in
> permanent memory by custodians?  We are all familiar with present
> implementations of BP that do this. But how can an unfamiliar reader of the
> draft deduce that this "willingness" means that a node should not only
> avoid to deliberately cancel the bundle, but also try hard (best effort, of
> course, nothing more) to preserve the bundle even in face of shutdowns and
> reboots (e,g. as a result of power failures). It would be enough to state
> that  a custodian should save it in permanent memory and not in RAM
> whenever possible, just anything more than this. The same for
> retransmissions.
> Note that a network may consists of nodes belonging to different entities,
> running different implementations. The source has no knowledge of the
> implementation of the following nodes on the path towards destination. In
> such a situation it would be often preferable not to pass custody to nodes
> that cannot offer any real guarantee of anything.
>
> Custody and Reliable CLs
> The following excerpt from the draft is in my opinion misleading in the
> last few rows, as custody is not an alternative of, but a complement to
> reliable CLs to augment reliability (provided that custodians do something,
> such as retransmitting bundles).
> Custody is a sort of further step which is left to the source, not to
> network maintainers, or intermediate nodes owners (the source cannot choose
> CLs on intermediate links).
> It is worth noting that in the classical data mule scenario A-B-C, with
> TCP on both links, only custody on A, or better on B (but in this case ONLY
> IF in permanent memory) can save from a power failure of the mule when it
> has the bundle on board.
> To have this protection, custody with permanent memory compulsory or, if
> you prefer,  at least strongly recommended, is in my opinion the best
> solution. As a final remark, note that custody on B without permanent
> memory would be the worst case in the example above, as it would result in
> a definitive loss.
> I am inclined to agree with Bill on this point.  We're trying to strike a
> balance between specifying enough mandatory behavior to make the operation
> of the network efficient and verifiable but not so much as to exclude
> resource-limited platforms from participating.  I think the way to do that
> is to minimize required behavior while always making sure that individual
> nodes can continue effective operation despite functional lapses at other
> nodes.  The network might never operate at theoretically optimal
> performance, but it should never fail altogether.
>
> Scott
>

---

From: Carlo Caini <[email-masked]>
Date: Fri, 20 May 2016 11:06:49 +0200

Dear Keith,
       your concerns about custody (vanished from
RFC4838 to the draft) are mine. I was just not
able to express them as clearly as you.
If people have time to read, I will add a few
considerations about standards. Otherwise, it is
enough to say that I fully agree with you.

 From Chiariglione (the father of MPEG) thinking:
The aim of standardization is interoperability
among products of different  producers.
"Standardize the minimum", i.e. only what is
necessary for this aim, so that on the one hand
companies can compete and on the other hand
people are not annoyed by compatibility problems.

My corollary:
The scenario you must have in mind when dealing
with standardization must be heterogeneous. In
networking this means nodes belonging to
different entities and running different
implementations. The opposite of a network where
all nodes belong to the same organization and/or
run the same implementation. Although the latter
can be significant for deployment, in this case
standardization is truly unnecessary. Compare
IEEE 802.3 and Appletalk (maybe the latter was
perfect, but just for a network of Apple's nodes only).

Is the definition of Custody necessary to
inter-operability in a heterogeneous network?
At first glance it seems no, but it actually it
is, as you pointed out. Introducing in a standard
a feature (in my opinion essential to DTN),
without a clear definition in the standard, means looking for troubles.
In fact, when a node accept the custody does 2
things: it accepts the responsibility
of  delivery (to take some effort to this end),
and at the same time, it relieves another node of
that. If for the old custodian custody means
business, while for the new one means
"donothingatall, you have a BIG interoperability
problem. In brief, the first node will be
unwilling to speak to lazy nodes, as you says,
and this would be the end of interoperability.
Only private networks, for which standards are
unnecessary, will survive. In tactical networks,
but not only in these, this would be the end of
DTN. What about an airborne networks, consisting
of half nodes of makers A... (complete as you
like) and the other half of makers B... (complete
again as you like); An A plane passing custody to
a B one (or viceversa), where the latter
interprets custody as doing nothing... We will
immediately have two partitioned networks, A and
B networks (with salesmen saying mine is
better... and customers unwilling to buy because
of scarce or at least questionable interoperability).

 From Tannebaum (rewording of RFC1958)

Look for a good design; it need not be perfect. Often, the designers
have a good design but it cannot handle some weird special
case. Rather than messing up the design, the designers should go
with the good design and put the burden of working around it on the
people with the strange requirements.

I would apply it to nodes that want to be part of
a heterogeneos network, want to act as custodians
(i.e. to offer a reliability enhancement service
to other nodes, maybe belonging of different
companies) but do not have permanent storage or
are unwilling to retranmit. If you cannot save
bundles in permanent memory, or you cannot re
transmit, please, on behalf of all other network
nodes, let the bundle go and refuse the custody!
Again, everything is best effort; HDs sometimes
fail; however this usually happens once every ten
years, while reboots and power offs are an
everyday matter. It is not matter of failures
only, indeed. Why DTN mules should be kept on,
while crossing Sahara? Looking for an Internet café?

Yours,
    Carlo

  20:12 19/05/2016, Keith Scott wrote:
>Bill and Carlo,
>
>I think the real question goes to 'what does
>custody transfer mean' and 'what do we want it to do'?
>
>As specified in the current draft, an
>application's request for custodial services
>results in a lot of in-network signaling to
>manage the question of 'who is the current
>custodian for this bundle' but, at the critical
>time when something fails and it's time for the
>current custodian to do something more than pass
>custody to the next, it's up to the
>implementation to decide what to do (including
>nothing).  The only mandatory action will be
>(when the bundle expires) the generation of a
>deletion status report (section 5.13).  That's a
>status report, which does not have the custody
>requested bit set, so it really might not even
>reach the deleted bundle's report-to endpoint ID.
>
>
>The draft states that custody transfer's main
>utility is in recovering from routing errors
>rather than as a reliability mechanism (section 3.2):
>
>Contrast this with the description of
>Reliability and Custody Transfer in section 3.10 of RFC4838:
>
>If you control the entire network, use reliable
>convergence layers everywhere, always store
>bundles for which you take custody in persistent
>storage, and manage all your data transfers so
>that you'll never have to delete a custodial
>bundle due to running out of resources, then
>everything will work.  But if you're operating
>in an internetwork of heterogeneous BP
>implementations, you can never trust custody
>transfer to provide reliability because some
>router you DON'T control might take custody of
>your bundle and the best you can hope for is to get a deletion status report.
>
>The people I know who are interested in using BP
>need reliability.  Of course they can build
>their DTN networks with implementations they
>control that implement timeouts, and respond to
>custody failures by retransmitting / rerouting /
>trying something to get bundles forward, but
>they'll never be able to interconnect those
>network with an 'open' BP network (security
>notwithstanding) without some better promise of
>reliability from conformant BP implementations.
>
>I agree that determining the value for a
>retransmission timer is, in the most general
>case, difficult or impossible.  However we could
>place stronger requirements on implementations
>w.r.t. attempting to deliver bundles for which they are the custodians, e.g.:
>     On receipt of a failed custody signal, the
> node SHOULD attempt to re-forward the
> bundle.  [depending on what the node knows
> about topology, this forwarding decision could
> try to use a different path, or just try again on the same path, or something].
>     Bundle routers SHOULD set a retransmission
> timeout when transmitting bundles for which the
> nodes are custodians. [rather than the current
>On Wed, May 18, 2016 at 10:07 PM, William
>Immerman <<mailto:[email-masked]>[email-masked]> wrote:
>Carlo-
>
>I completely agree that an important and
>under-appreciated application for BP is
>providing âEURoefire-and-forgetâEUR behavior to the
>application/user (why should users or
>applications in disadvantaged environments
>assume they have to watch a spinning wheel to
>ensure that their message has been sent?).
>
>I donâEUR(tm)t think, though, that making
>âEURoepermanent storageâEUR a mandatory feature
>rather than an implementation concern is needed
>or best ... I think there are applications where
>an implementation mightt want a bundle forwarder
>to reside in volatile storage and wouldnâEUR(tm)t
>want that to violate required semantics. It
>might mean that such a node should not accept
>custody, or other limitations, but I thought
>that practical implementation of DTN/BP still
>assumes itâEUR(tm)ll be a âEURoebest-effortâEUR protocol.
>Aside from possible utility, IâEUR(tm)m not sure
>weâEUR(tm)ll ever be able to guarantee that remote
>hardware wonâEUR(tm)t fail permanently or that
>contacts will be available to all isolated nodes
>in time to avoid expiration of bundles, so we
>might as well embrace âEURoebest-effortâEUR.
>
>Bill Immerman
>
>
> > On May 18, 2016, at 2:44 PM, Carlo Caini
> <<mailto:[email-masked]>[email-masked]> wrote:
> >
> > Dear Keith,
> >> We need reviewers and reviews. We canÃ¢EUR(tm)t
> >> <mailto:[email-masked]><mailto:[email-masked]>[email-masked]
> > DELL'UNIVERSITÃEUR DI BOLOGNA

---

From: Carlo Caini <[email-masked]>
Date: Fri, 20 May 2016 11:48:43 +0200

Dear Bill,
       Concerning persistent storage and
retransmission, I have already said, maybe too
much, in my previous e-mail. A custodian offers a
(best effort) service to other nodes, but at the
same time the new custodian cut off the service
offered by the old custodian; consequently,  the
minimum subset of this service must be clearly defined in the spec.

Concerning custody advantages as resource
management helper, I fully agree with you. The
scenario you mentioned is one (of the many) where custody is useful.

Custody is a very powerful concept (in my opinion
a pillar of DTN), with many important and subtle
implications. The paper "Custody Transfer for
Reliable Delivery in Delay Tolerant Networks" by
Fall and alii (almost 200 citations!) describes
them in a magisterial way. I recommend its
reading to all interested in this topic and that
maybe have not read it yet (it was published at DTN's dawn...).
It considers also (not only, of course) the case you mentioned:

"One of the advantages of custody transfer is to
allow an endpoint to free storage resources
comparatively quickly when a custody transfer
acknowledgment arrives. Given that custody
transfers are likely to take place with
comparatively nearby nodes, the round-trip- time
to complete a custody transfer is expected to be
significantly shorter than an end-to-end round
trip. For end nodes responsible for collecting
data (e.g. remotely deployed sensors), the
ability to free buffers quickly can be of great importance."

  Yours,
    Carlo

At 23:09 19/05/2016, Bill Immerman wrote:
>All--
>
>I may have confused things by "bundling"
>concerns about custody and persistent storage
>(although I think there's a relationship), so
>let me take one stab at un-bundling:
>
>RE: Persistent storage: I agree with Carlo that
>the usual and preferred implementation would
>provide persistent storage (i.e., storage that
>would survive a restart of the hw host and sw,
>at least). Since I've considered at least one
>application that wouldn't provide that, I was
>hoping that wouldn't demand a non-compliant
>implementation that was "sort-of like BP". My
>comment about custody was that a configuration
>employing such a node might have to satisfy
>certain requirements, like refusing custody, and
>that would be ok ... it wasn't my understanding
>that all forwarding nodes had to agree to
>custody. My instinct is that the suggestion that
>most implementations of BP support mapping
>bundle storage to persistent storage, and that
>most deployments consider using that feature,
>might be better suited to an architecture
>document or an accompanying implementation
>guidance document than to the BP spec itself,
>but I have no experience authoring standards and
>would defer to those who do in deciding where
>such (useful) guidance should be kept and published.
>
>RE: Custody: I agree with Keith that reliability
>is very important in most anticipated uses of
>BP. I'm not sure I agree that custody is more
>about reliability than about resource
>management. I'd expect that many isolated,
>highly disadvantaged nodes lack a number of
>resources: contact opportunities, contact
>throughput, power, storage, ... Say that A is
>such a node, B is A's only neighbor, and is
>still intermittently connected but has better
>resources than A (e.g., more contact), and C is
>a well-connected neighbor of B. For example,
>think of A as a remote sensor, B as a DTN
>gateway, and C as an Internet connected DTN
>forwarder. If A could pass data to B and have a
>custody request to B acknowledged in a single
>contact, then couldn't A relinquish storage of
>the forwarded data before it entered a long
>period of no contact? (therefore the storage
>would be more available for the fire-and-forget
>use by the DTN-enabled sensor application at
>A?). Also, wouldn't that enable C to acknowledge
>receipt directly to B as custodian, and not
>require B to ever rely on future contacts with A
>to close the loop? In some cases we can count on
>and predict periodic future contacts, and in
>some cases I'm not sure we can ever count on A
>not becoming completely partitioned from B.
>Maybe this isn't what custody is for, but it
>seems like a useful case for something other
>than being another intermediate forwarding node ...
>
>Bill
>
>On Thu, May 19, 2016 at 4:08 PM, Carlo Caini
>      I agree with you on not specifying
> mandatory behaviors if strictly not necessary,
> and also on best effort. However, the main
> point in my mail was that what is not mandatory
> should 1) clearly stated, and 2) recommended if
> useful. Now custody seems so vague that could
> be interpreted as "doing nothing", as pointed
> out by Keith. This doing nothing (from entity Y
> nodes) would be harmful to other nodes (entity X nodes).
>
> From the RFC5050bis:
>  "A custodian of a bundle is a node that has
> determined that it will retain a copy of that
> bundle for an indefinite period of time,"
>
>A part from his "willingness", it is unclear
>what is mandatory, or simply recommended, and
>what is not. It is just not stated. Keith points
>out that this definition does not impose
>absolutely anything to anyone. Not only does it
>not impose anything, but it does not recommend
>anything too, and this is my main objection.
>I am not happy with this total liberty left to
>implementations, because it has a significant
>cost in terms of robustness, which is the
>reasons why DTN has been designed for. It seems
>very unbalanced in favor of  resource limited
>platforms  but this is just  my personal
>opinion. However, why not at least to suggest
>that, whenever possible, bundles should be kept
>in permanent memory by custodians?  We are all
>familiar with present implementations of BP that
>do this. But how can an unfamiliar reader of the
>draft deduce that this "willingness" means that
>a node should not only avoid to deliberately
>cancel the bundle, but also try hard (best
>effort, of course, nothing more) to preserve the
>bundle even in face of shutdowns and reboots
>(e,g. as a result of power failures). It would
>be enough to state that  a custodian should save
>it in permanent memory and not in RAM whenever
>possible, just anything more than this. The same for retransmissions.
>Note that a network may consists of nodes
>belonging to different entities, running
>different implementations. The source has no
>knowledge of the implementation of the following
>nodes on the path towards destination. In such a
>situation it would be often preferable not to
>pass custody to nodes that cannot offer any real guarantee of anything.
>
>Custody and Reliable CLs
>The following excerpt from the draft is in my
>opinion misleading in the last few rows, as
>custody is not an alternative of, but a
>complement to reliable CLs to augment
>reliability (provided that custodians do
>something, such as retransmitting bundles).
>Custody is a sort of further step which is left
>to the source, not to network maintainers, or
>intermediate nodes owners (the source cannot choose CLs on intermediate links).
>It is worth noting that in the classical data
>mule scenario A-B-C, with TCP on both links,
>only custody on A, or better on B (but in this
>case ONLY IF in permanent memory) can save from
>a power failure of the mule when it has the bundle on board.
>This is because reliability of CLs does not
>protect bundles when stored in intermediate nodes.
>To have this protection, custody with permanent
>memory compulsory or, if you prefer,  at least
>strongly recommended, is in my opinion the best
>solution. As a final remark, note that custody
>on B without permanent memory would be the worst
>case in the example above, as it would result in a definitive loss.
>Da: dtn
>[<mailto:[email-masked]>[email-masked]]
>per conto di Burleigh, Scott C (312B)
>[<mailto:[email-masked]>[email-masked]]
>Inviato: giovedÃ¬ 19 maggio 2016 18:01
>Cc: Marc Blanchet; Keith Scott; <mailto:[email-masked]>[email-masked]
>I am inclined to agree with Bill on this
>point.  We're trying to strike a balance between
>specifying enough mandatory behavior to make the
>operation of the network efficient and
>verifiable but not so much as to exclude
>resource-limited platforms from
>participating.  I think the way to do that is to
>minimize required behavior while always making
>sure that individual nodes can continue
>effective operation despite functional lapses at
>other nodes.  The network might never operate at
>theoretically optimal performance, but it should never fail altogether.
>
>Scott
>
>To: Carlo Caini <<mailto:[email-masked]>[email-masked]>
>Cc: Marc Blanchet
><<mailto:[email-masked]>[email-masked]>;
>Keith Scott
><<mailto:[email-masked]>[email-masked]>;
> >> We need reviewers and reviews. We canÃ¢EUR(tm)t forward this document

---

From: Brian Sipos <[email-masked]>
Date: Fri, 20 May 2016 15:43:20 +0000

Since I have not heard any activity on this in a few weeks, I am proposing to define a new version of TCPCL (to obsolete RFC 7242).
This TCPCL update will address the following:
1. Add a bundle transfer ID to disambiguate bundles (presuming SDNV with implementations required to support at least 2^28 [32-bit encoded] IDs), which should address comments from Scott B. on his implementation issues. This transfer ID would be included in all DATA_SEGMENT, ACK_SEGMENT, REFUSE_BUNDLE, and LENGTH messages.

2. Add messages to support STARTTLS behavior (same as RFC 2595) to allow upgrading a TCP socket to use TLS. This would be an optional and endpoint-negotiated feature. An endpoint which does not support TLS would simply reject a request to upgrade.

3. Add additional message to indicate "received message error" from receiver side. A reason code would be either "message unknown", "message not supported" or "message invalid". This will allow recovery from attempts to STARTTLS to endpoints which don't support it, and could also be used to reject invalid LENGTH messages, etc.

4. As a side effect, this would increment the TCPCL protocol version in the contact header.

Scott, do you think that change #1 would be enough to cover the issues you have run into? Once all bundle transfers are identified, then there should be no ambiguity about correlating messages from each endpoint to determine which segments/bundles succeeded and which did not.

________________________________________
From: dtn <[email-masked]> on behalf of Burleigh, Scott C (312B) <[email-masked]>
Sent: Friday, April 29, 2016 5:31:08 PM
To: Stephen Farrell; Eric Travis; Marc Blanchet
Cc: Gilbert Clark; [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of Stephen Farrell
Sent: Friday, April 29, 2016 1:40 PM
To: Eric Travis; Marc Blanchet
Cc: Gilbert Clark; [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Scott

---

From: Eric Travis <[email-masked]>
Date: Fri, 20 May 2016 12:37:09 -0400

Brian,

As an integral part of such an effort, I would like to suggest including an
expansion of the defined CL interface requirements for the draft 5050bis
document.

>From my reading of Scott's issues description, the RFC 5050 and RFC 7242
documents, there is nothing wrong with RFC 7242 as it met the bundle-in,
bundle-out interface requirements specified in RFC-5050 and draft
RFC5050bis.

It seems that the issue was Scott needed signaling beyond the RFC5050
specified simple bundle in/out service interface.

I* *do* believe that the draft 5050bis document is NOT ready to be moved
forward until a clarification of the CL service interface is provided.*

Eric

On Fri, May 20, 2016 at 11:43 AM, Brian Sipos <[email-masked]> wrote:

> Since I have not heard any activity on this in a few weeks, I am proposing
> to define a new version of TCPCL (to obsolete RFC 7242).
> 1. Add a bundle transfer ID to disambiguate bundles (presuming SDNV with
> implementations required to support at least 2^28 [32-bit encoded] IDs),
> which should address comments from Scott B. on his implementation issues.
> This transfer ID would be included in all DATA_SEGMENT, ACK_SEGMENT,
> REFUSE_BUNDLE, and LENGTH messages.
>
> 2. Add messages to support STARTTLS behavior (same as RFC 2595) to allow
> upgrading a TCP socket to use TLS. This would be an optional and
> endpoint-negotiated feature. An endpoint which does not support TLS would
> simply reject a request to upgrade.
>
> 3. Add additional message to indicate "received message error" from
> receiver side. A reason code would be either "message unknown", "message
> not supported" or "message invalid". This will allow recovery from attempts
> to STARTTLS to endpoints which don't support it, and could also be used to
> reject invalid LENGTH messages, etc.
>
> Scott, do you think that change #1 would be enough to cover the issues you
> have run into? Once all bundle transfers are identified, then there should
> be no ambiguity about correlating messages from each endpoint to determine
> which segments/bundles succeeded and which did not.
>
> >       I'd also be interested if Scott could document the issues he found
> with the TCPCL. If that assertion is the main basis for picking UDP over
> TCP then it seems fair to ask for that. If OTOH the main argument is that
> UDP is simpler and sufficient then documenting the problems with the TCPCL
> isn't needed now (but would be interesting of course.)
>
> Of course, absolutely fair.  The context for this is that we were
> implementing TCPCL specifically to deal with the case of a TCP connection
> being spontaneously lost -- machine rebooted, cable was cut, whatever --
> without TCP having had the opportunity to signal the failure.  Here's what
> I found.
>
> First, the intended operation of LENGTH messages depends on the
> implementation of REFUSE_BUNDLE messages (1st paragraph of 5.5), but in the
> general case it is impossible to implement the REFUSE_BUNDLE feature
> correctly at the bundle sender.
>
> The reason bundle refusal cannot in general be implemented correctly at
> the bundle sender is that bundles over the TCPCL connection may be
> pipelined (6th paragraph of 3.0, 2nd paragraph of 3.2).  Since bundles can
> be pipelined, it is possible for all segments of bundles X and X+1, and the
> first N segments of bundle X+2, to have been transmitted by the time the
> sending node receives a BUNDLE_REFUSAL message for message X.
>
> Suppose the receiving node sent a BUNDLE_REFUSAL upon reception of the
> first segment of bundle X.  The sending node can keep a record of
> transmitted bundles that have not yet been fully acknowledged, so it knows
> the unacknowledged oldest bundle and can properly apply the refusal to that
> bundle.  It can respond correctly to this refusal message by reforwarding
> bundle X or whatever, depending on the refusal message’s reason code.  No
> problem.
>
> However, bundle X might have been transmitted in, say, 3 segments.  The
> 6th paragraph of section 5.4 says that the receiving node SHOULD (but is
> not required to) refuse the other two segments of bundle X – which were
> placed in the pipeline long ago – in the same way.  Suppose it does so.
> Now the sending node receives two additional BUNDLE_REFUSAL messages.
> However, there is no bundle identifier in the BUNDLE_REFUSAL message; the
> second refusal message might be a refusal of the second segment of bundle
> X, or it might alternatively be a refusal of the first segment of bundle
> X+1.  Both behaviors are conformant.  The sending node has no way of
> distinguishing between the two, so it might erroneously determine that
> bundles X+1 and X+2 were refused.
>
> There’s a problem with acknowledgments too.  Again, pipelining is the
> issue.  When an acknowledgment of a TCPCL data segment message arrives it
> notes the total bundle length received so far.  Clearly, if this length is
> less than the length previously reported – and the previous ACK didn’t
> acknowledge the tail end of a bundle – there has been a transmission
> failure and the bundle that was previously being acknowledged did not
> arrive and must be retransmitted; the new acknowledgment is acknowledging
> the next bundle for which it has received data.  But how many bundles
> immediately following that lost bundle also didn’t arrive (and were not
> acknowledged)?  Which of those subsequently transmitted bundles is now
> being acknowledged?  We have no way of knowing, because there are no bundle
> identifiers in acknowledgment messages.
>
> A last point, which isn't such a big deal: the “request enabling of
> reactive fragmentation” flag doesn’t actually do anything.  Reactive
> fragmentation isn’t defined anywhere; not in RFC 7242, and not in RFC
> 5050.  There is no formal definition anywhere of how to do it, there is no
> procedure anywhere in TCPCL that depends on it, and there is no definition
> in RFC 7242 of what is intended when the two ends of a TCPCL connection
> agree to enable it.  So the operation of the flag can't actually be
> implemented from the specification.
>
> Scott
>
>
>

---

From: Brian Sipos <[email-masked]>
Date: Fri, 20 May 2016 17:04:41 +0000

Erik,

My impression was that the issues were due to the use of a single TCP connection to transport more than one bundle. If TCPCL was used as one-bundle-one-connection then there would be no need to identify the bundles being transported and no ambiguity about ack/reject, but it would also be incredibly wasteful of resources for anything but huge individual bundles. TCPCL allows pipelining of multiple bundles (send a few, wait for ack/reject on each) so my understanding of the issue is the lack of ability (in RFC 7242) of individually identifying a bundle. To me this sounds like purely a transport (CL) issue.

________________________________
From: Eric Travis <[email-masked]>
Sent: Friday, May 20, 2016 12:37:09 PM
To: Brian Sipos
Cc: Burleigh, Scott C (312B); [email-masked]
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Brian,

As an integral part of such an effort, I would like to suggest including an expansion of the defined CL interface requirements for the draft 5050bis document.

>From my reading of Scott's issues description, the RFC 5050 and RFC 7242 documents, there is nothing wrong with RFC 7242 as it met the bundle-in, bundle-out interface requirements specified in RFC-5050 and draft RFC5050bis.

It seems that the issue was Scott needed signaling beyond the RFC5050 specified simple bundle in/out service interface.

I *do* believe that the draft 5050bis document is NOT ready to be moved forward until a clarification of the CL service interface is provided.

Eric

On Fri, May 20, 2016 at 11:43 AM, Brian Sipos <[email-masked]<mailto:[email-masked]>> wrote:
Since I have not heard any activity on this in a few weeks, I am proposing to define a new version of TCPCL (to obsolete RFC 7242).
This TCPCL update will address the following:
1. Add a bundle transfer ID to disambiguate bundles (presuming SDNV with implementations required to support at least 2^28 [32-bit encoded] IDs), which should address comments from Scott B. on his implementation issues. This transfer ID would be included in all DATA_SEGMENT, ACK_SEGMENT, REFUSE_BUNDLE, and LENGTH messages.

________________________________________
From: dtn <[email-masked]<mailto:[email-masked]>> on behalf of Burleigh, Scott C (312B) <[email-masked]<mailto:[email-masked]>>
Sent: Friday, April 29, 2016 5:31:08 PM
To: Stephen Farrell; Eric Travis; Marc Blanchet
Cc: Gilbert Clark; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

-----Original Message-----
From: dtn [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Stephen Farrell
Sent: Friday, April 29, 2016 1:40 PM
To: Eric Travis; Marc Blanchet
Cc: Gilbert Clark; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] I-D Action: draft-ietf-dtn-bpbis-03.txt

Suppose the receiving node sent a BUNDLE_REFUSAL upon reception of the first segment of bundle X.  The sending node can keep a record of transmitted bundles that have not yet been fully acknowledged, so it knows the unacknowledged oldest bundle and can properly apply the refusal to that bundle.  It can respond correctly to this refusal message by reforwarding bundle X or whatever, depending on the refusal message's reason code.  No problem.

However, bundle X might have been transmitted in, say, 3 segments.  The 6th paragraph of section 5.4 says that the receiving node SHOULD (but is not required to) refuse the other two segments of bundle X - which were placed in the pipeline long ago - in the same way.  Suppose it does so.  Now the sending node receives two additional BUNDLE_REFUSAL messages.  However, there is no bundle identifier in the BUNDLE_REFUSAL message; the second refusal message might be a refusal of the second segment of bundle X, or it might alternatively be a refusal of the first segment of bundle X+1.  Both behaviors are conformant.  The sending node has no way of distinguishing between the two, so it might erroneously determine that bundles X+1 and X+2 were refused.

There's a problem with acknowledgments too.  Again, pipelining is the issue.  When an acknowledgment of a TCPCL data segment message arrives it notes the total bundle length received so far.  Clearly, if this length is less than the length previously reported - and the previous ACK didn't acknowledge the tail end of a bundle - there has been a transmission failure and the bundle that was previously being acknowledged did not arrive and must be retransmitted; the new acknowledgment is acknowledging the next bundle for which it has received data.  But how many bundles immediately following that lost bundle also didn't arrive (and were not acknowledged)?  Which of those subsequently transmitted bundles is now being acknowledged?  We have no way of knowing, because there are no bundle identifiers in acknowledgment messages.

A last point, which isn't such a big deal: the "request enabling of reactive fragmentation" flag doesn't actually do anything.  Reactive fragmentation isn't defined anywhere; not in RFC 7242, and not in RFC 5050.  There is no formal definition anywhere of how to do it, there is no procedure anywhere in TCPCL that depends on it, and there is no definition in RFC 7242 of what is intended when the two ends of a TCPCL connection agree to enable it.  So the operation of the flag can't actually be implemented from the specification.

Scott

---

From: Brian Sipos <[email-masked]>
Date: Mon, 23 May 2016 19:47:43 +0000

My comments on current BPbis draft are concentrated in Section 7 "Services Required of the Convergence Layer". Beyond the current text "send a bundle" and "deliver a bundle" there is a lack of specific requests and indications which are required/expected/possible to and from the convergence layer. May I suggest CL requirements on:
 1. Whether or not a CL has to indicate or negotiate BP encoding format (CBOR or otherwise); otherwise this is an agent mismatch waiting to happen.
 2. Whether the encoding format is per-CL contact or per-bundle.
 3. Whether and how the CL must indicate a send has finished: either success / failure / no-information (in the case of non-reliable CL).
 4. Require that a CL transmitter be able to distinguish finish status per-bundle. This explicitly requires per-bundle bookkeeping for the sender (which should avoid the earlier TCPCL issue of not being able to cancel/reject individual bundles).
 5. Allow the agent to indicate to the  CL receiver whether or not the agent will accept any new bundles.
 6. Allow the agent to indicate to the CL receiver a maximum acceptable bundle size.
 7. Allow a CL receiver be able to accept/reject individual bundles. This explicitly requires per-bundle bookkeeping for the receiver.

Also, in Section 5.10.1 "Custody Acceptance" there appears to be a cross reference to Section 5.6 which should be Step #2 rather than step #1.

________________________________________
From: dtn <[email-masked]> on behalf of Brian Haberman <[email-masked]>
Sent: Wednesday, April 27, 2016 08:26
To: [email-masked]
Subject: [dtn]  I-D Action: draft-ietf-dtn-bpbis-03.txt

All,

        Title           : Bundle Protocol
        Authors         : Scott Burleigh
                          Kevin Fall
                          Edward J. Birrane

Regards,
Brian

---

From: Manikantan Ramadas <[email-masked]>
Date: Tue, 24 May 2016 15:11:57 +0530

Hi all,

Please find my comments after going through the BP IETF draft - attached. I
have them in MAJOR (semantic issues) and MINOR (mostly syntactic or
minor-sematic issues) sections.

As a suggestion, it would be nice to have Sender and Receiver State
Transition Diagrams. It would help in understanding and implementation. I
know the pain in making ASCII text diagrams; perhaps a PDF version of it -
made with any diagram editor - may be provided as a supplement?

Manikantan Ramadas,
ISTRAC/ISRO

On Wed, Apr 27, 2016 at 6:13 PM, Marc Blanchet <[email-masked]>
wrote:

> Hello,
> FYI, this is the IETF wg last call for comments for the new version of the
> Bundle Protocol, primarily edited by Scott Burleigh. If you have any
> comments, please make sure to send them to the IETF mailing list (
> [email-masked]). This community should provide its comments so that the
> standards are useful for all. Compared to RFC5050, which was done under the
> IRTF (Research) and defined as « experimental », the intent of this work is
> to create an IETF Standard, that would enable various industries and
> companies to implement in standard products.
>
> deadline for comments: May 25th 2016.
>
> Marc, co-chair of IETF dtn working group.
>
> Forwarded message:
>
> From: Brian Haberman [email-masked]
> To: [email-masked]
> Date: Wed, 27 Apr 2016 08:26:39 -0400
>
> All,
> Sis-dtn mailing list
> [email-masked]
> http://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn
>
>

-- 
"By being pleasant always and smiling, it takes you nearer to God, nearer
than any prayer." - Sri Ramakrishna Paramahamsa
------------------------------------------------------------------