Subject: [dtn] TCPCLv4 -07 updates and simplification

From: Brian Sipos <[email-masked]>
Date: Mon, 26 Mar 2018 19:36:19 +0000

All,

After looking into simplifying TCPCLv4 as discussed at the IETF, I see that if a transfer is always simplified into a single XFER_INIT message followed by a single "segment" of data this effectively turns the TCPCL session unidirectional by preventing any other XFER_ACK messages from acknowledging transfers in the other direction within the session. One side effect of segmented transfer data is to allow other messages from the transmitting node between segments by avoiding monopolizing the TCP connection with a single huge message.

I feel like segmentation, as an optional behavior as currently specified, gives the network / node management a way to avoid this kind of message queue blocking behavior and allows more deterministic timing of when an ACK can be expected based on path delays.

For a simple-as-possible workflow, an implementation could choose to disallow a Segment MRU any smaller than the Transfer MRU and could choose to always transfer within a single segment which would minimize the message framing overhead. If interoperability was not necessary then it could also refuse any transfer that includes a segment smaller than the transfer size, so that there would be no need for receive segment concatenation.

Any thoughs on keeping segmentation as a negotiated behavior in order to allow for better administrative control over worst-case session timing?

---

From: Victoria Pritchard <[email-masked]>
Date: Tue, 27 Mar 2018 22:44:50 +0100

Hi Brian,

This is a good point. If one side is busy sending a giant segment, and the
other side wants to initiate a transfer, the side that is already busy
sending cannot send a message in response to that init, e.g. a XFER_REFUSE,
or indeed ACKs. However, I still dont see the point in sending ACKs if we
are not supporting reactive fragmentation for now. I can see the value in
sending data in segments though if this allows us to insert other messages
related to an incoming transfer, to send between segments.

Vicky.

On 26 March 2018 at 20:36, Brian Sipos <[email-masked]> wrote:

> All,
>
> After looking into simplifying TCPCLv4 as discussed at the IETF, I see
> that if a transfer is always simplified into a single XFER_INIT message
> followed by a single "segment" of data this effectively turns the TCPCL
> session unidirectional by preventing any other XFER_ACK messages from
> acknowledging transfers in the other direction within the session. One side
> effect of segmented transfer data is to allow other messages from the
> transmitting node between segments by avoiding monopolizing the TCP
> connection with a single huge message.
>
>
> I feel like segmentation, as an optional behavior as currently specified,
> gives the network / node management a way to avoid this kind of message
> queue blocking behavior and allows more deterministic timing of when an ACK
> can be expected based on path delays.
>
>
> For a simple-as-possible workflow, an implementation could choose to
> disallow a Segment MRU any smaller than the Transfer MRU and could choose
> to always transfer within a single segment which would minimize the message
> framing overhead. If interoperability was not necessary then it could also
> refuse any transfer that includes a segment smaller than the transfer size,
> so that there would be no need for receive segment concatenation.
>
>
> Any thoughs on keeping segmentation as a negotiated behavior in order to
> allow for better administrative control over worst-case session timing?
>
> _______________________________________________
> dtn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dtn
>
>

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Wed, 28 Mar 2018 05:56:35 +0000

Brian, is it possible that this is an implementation matter?  I am thinking that the sending node could have one thread continuously transmitting segment data over the socket while another thread running concurrently could be reading XFER_ACK messages (sent by the receiving node) from the same socket.  Am I overlooking some reason this won't work?

Scott

From: dtn <[email-masked]> On Behalf Of Brian Sipos
Sent: Monday, March 26, 2018 12:36 PM
To: [email-masked]
Subject: [dtn] TCPCLv4 -07 updates and simplification

All,

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Wed, 28 Mar 2018 13:24:38 +0000

Since data is being multiplexed in the segment case, I’d imagine that a single ACK would be necessary to indicate that an entire bundle had been transferred (read: all segments related to that bundle had been received), wouldn’t it?

In my head, one has two separate things (in the abstract): an ACK size (the number of bytes that are transmitted before an ACK is provided) and a segment size (or frame size, or … whatever).  If the ACK size is equal to the size of the bundle, the transfer will not support reactive fragmentation.  If it’s less, reactive fragmentation will be supported at increments of whatever the ACK size is set to be.  The segment size, in the case of no fragmentation, would be set to something sane that allowed the link to be multiplexed.  The segment size, in the case of fragmentation, would be set to be much closer to the ACK size …

… but supporting another layer of generalized fragmentation in this fashion also adds complexity, because one has effectively added another version of MTU to bundle transfers on top of the MTU that IP / TCP need to deal with.  In the event that e.g. one is not a multiple of the other, strange things can start to happen …

Something that might be a little simpler would be to negotiate a fixed number of logical channels (read: parallel streams) that could be used, and to support multiplexing channel transfers over a single link without necessarily supporting multiplexing individual bundles, if that makes sense.  That can make reassembly quite a bit easier, because one only ever has one buffer per channel, avoids the link monopolization issue (as a function related to the maximum number of supported channels), and avoids the layered segment size issue because one could inject markers into data streams to indicate that the active stream / channel has changed without necessarily used fixed frame sizes for the transfer … or something along those lines, anyway ☺

As an aside, SCTP might be a relevant reference if this approach seems like it could be an interesting one.

FWIW,
Gilbert

The views expressed in this mail reflect the opinions of the author.  They are, therefore, not intended to reflect official positions of NASA or the U.S. Government.

From: dtn <[email-masked]> on behalf of Victoria Pritchard <[email-masked]>
Date: Tuesday, March 27, 2018 at 5:45 PM
To: Brian Sipos <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [dtn] TCPCLv4 -07 updates and simplification

Hi Brian,

This is a good point. If one side is busy sending a giant segment, and the other side wants to initiate a transfer, the side that is already busy sending cannot send a message in response to that init, e.g. a XFER_REFUSE, or indeed ACKs. However, I still dont see the point in sending ACKs if we are not supporting reactive fragmentation for now. I can see the value in sending data in segments though if this allows us to insert other messages related to an incoming transfer, to send between segments.

Vicky.

On 26 March 2018 at 20:36, Brian Sipos <[email-masked]<mailto:[email-masked]>> wrote:

All,

_______________________________________________
dtn mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn

---

From: Brian Sipos <[email-masked]>
Date: Wed, 28 Mar 2018 14:00:01 +0000

Scott,

The issue is in the case the two nodes are both attempting a transmission (in opposite directions), and it's a matter of bundle data transmission monopolizing the TCP stream which prevents XFER_REFUSE / XFER_ACK messages from being sent.

For example: Imagine node A sending a large 1GiB bundle "A" to node B within a single segment. Once the start of that segment transmission begins it is committed and there are only two possible outcomes: the node continues to send the single large message to completion, or something interrupts the message and the entire TCPCL session is failed. Halfway through that transfer, node B attempts its own transfer of a small 1KiB bundle "B" to node A within a single segment. The transfer of bundle B is quick and completes in a short time, but the ACK for bundle B has to wait until the entirety of bundle A has completed sending, since it was all sent in a single message, TCPCL messages are atomic, and TCP is single-flow in each direction.

This is not a horrible situation, the ACK will arrive eventually but it makes timing of when to be able to expect an ACK less predictable, and there's a liability of several transfer's ACKs to stack up waiting for a large transfer. An implementation work-around in this case is to only use one TCPCL session for unidirectional transfers (which introduces its own complexity at the BP agent of how to distinguish what purpose/direction a session is 'supposed to be' used for).

Victoria,

If the TCPCL is to be a reliable CL, then XFER_ACK is absolutely necessary. It is there to inform the sender that the receiving BP agent itself has fully received a bundle (not just the OS fully receiving all of its associated TCP packets).

To the WG,

My feeling right now is that segmentation, as already defined and as an implementation decision, is a valuable mechanism to be able to tune some performance aspects of the TCPCL. It's also a direct inheritance from TCPCLv3 so makes logical transition straightforward. I don't see segment reassembly as a large implementation issue, because of the in-order guarantee of segments and the mandatory XFER_INIT allows pre-allocating bundle storage. I can add an explanatory sub-section under Section 2 for recommendations on tuning segmentation for different network goals.

________________________________
From: Burleigh, Scott C (312B) <[email-masked]>
Sent: Wednesday, March 28, 2018 1:56:35 AM
To: Brian Sipos; [email-masked]
Subject: RE: TCPCLv4 -07 updates and simplification

Brian, is it possible that this is an implementation matter?  I am thinking that the sending node could have one thread continuously transmitting segment data over the socket while another thread running concurrently could be reading XFER_ACK messages (sent by the receiving node) from the same socket.  Am I overlooking some reason this won’t work?

Scott

All,

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Wed, 28 Mar 2018 17:45:18 +0000

I agree, Brian - the implementation measure I was talking about would enable immediate ACK flow in response to data flow in the opposite direction, but it certainly doesn't solve the problem when there is data flow in both directions.  Segmentation would enable data and ACKs to be interleaved on the connection.

Scott

From: Brian Sipos <[email-masked]>
Sent: Wednesday, March 28, 2018 7:00 AM
To: Burleigh, Scott C (312B) <[email-masked]>; [email-masked]
Subject: Re: TCPCLv4 -07 updates and simplification

Scott,

Victoria,

To the WG,

________________________________
From: Burleigh, Scott C (312B) <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, March 28, 2018 1:56:35 AM
To: Brian Sipos; [email-masked]<mailto:[email-masked]>
Subject: RE: TCPCLv4 -07 updates and simplification

Scott

From: dtn <[email-masked]<mailto:[email-masked]>> On Behalf Of Brian Sipos
Sent: Monday, March 26, 2018 12:36 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [dtn] TCPCLv4 -07 updates and simplification

All,