Subject: RE: TN3270E Byte-Doubling & Flush Functions

From: "Erickson, Rodger" <[email-masked]>
Date: Sat, 15 Nov 1997 10:34:14 -0800

Greetings,

        I wanted to extend my appreciation to Marty for taking the time
to put a draft together.

        I did have a couple of comments about the draft and also about
Derek's response:

1)      In describing both options you state: "Normally, the client will
start with this option enabled".  I believe that in the interest of
backward compatibility it should state instead that: "Normally, the
client will start with this option disabled".  Note that this may just
be a technicality and that doing it your way doesn't have any adverse
consequences.  However, I think that its cleaner to assume it is off
until turned on (rather than assume that it is on until it isn't
actually turned on).
2)      I *really* don't like this business of how this gets handled
when the server doesn't buffer the entire chain.  It really looks messy.
How about the following approach:
-       the length field contains the number of bytes until the next
TN3270E header
-       an EOR bit is actually turned on/off in the header depending on
if this is the last message in the chain (thus there's no more
<IAC><EOR> in the data stream
-       both client and server must make sure that the same sequence
numbers are used to refer to all pieces of the same chain (i.e. don't
just increment the sequence number because it's s new TN3270E command
unless the EOR bit was set on the last command sent).

                This approach has the benefit of making the case cleaner
where the server doesn't buffer the entire chain as well as eliminating
scanning for <IAC><EOR> in all cases.

        Also, I'm a little unclear about the Flush mechanism.  I'm
uncomfortable with how it is defined.  I don't like all this overloading
or all of these special cases.  As a result of this and other issues
that have come up, I think we should look at adding an extra byte (or
stealing one of the lesser used bytes that's already there) and start
defining command/status bits or even adding new TN3270E commands.  This
would be a lot simpler than messing with overloading of existing Telnet
escape sequences.  I appreciate that we're trying to actually "Abort
some Output" here, but that doesn't mean we should bend over backward to
try and find a way to use <IAC><AO>.

        While I appreciate Derek's comments, I think it would open to
large a can of worms to worry about re-specifying the Telnet protocol to
extend this data-length mechanism to all of telnet.  Really, this kind
of thing is only appropriate to block-mode terminals like 3270 and 5250.
Regular old character-at-a-time or line-at-a-time telnet wouldn't really
benefit from such a mechanism.  I believe that this byte-doubling
nonsense that telnet expects should have never carried over into TN3270E
and we shouldn't let it carry over into TN5250E.
An additional problem with a generic mechanism is how and when to
transfer length information.  TN3270E already has a header so it should
go there.  TN5250, TN3270, and Telnet do not, so where should it go?
Another <IAC> type thing to parse?  If we're doing this different ways
for different flavors, then we've lost most of the benefit from making
it a general mechanism.
Unless I can see a more detailed proposal on how this could be made to
work easily in a general Telnet setting, I vote for just making it part
of the TN3270E sub-negotiation.

I'm looking forward to seeing everyone this week in Raliegh!

Rodger D. Erickson
Technologist
Wall Data Inc.
[email-masked]
(314) 721-4640

        -----Original Message-----
        From:   Derek W. Bolton [SMTP:[email-masked]]
        Sent:   Thursday, November 13, 1997 5:14 PM
        To:     [email-masked]
        Cc:     [email-masked]
        Subject:        Re: TN3270E Byte-Doubling & Flush Functions

        Hi Marty,

        I'm not across the political issues the might be involved,
        but I was hoping that byte-non-doubling could be negotiated
        at the telnet layer rather than as a tn3270e suboption.
        That would bring many benefits:

        - It could be adopted by other telnet-based protocols.
          Ideally, it would be transparent to the higher layers.

        - It would avoid mangling the layers.

          At present, the byte double/undouble is logically at a layer
          below the tn3270/tn3270e layer.  Your proposal changes that in
two
          ways.  First, the higher layer can turn the function off.
That's
          not too bad, but the nasty part is that the lower layer then
          has to understand part of the tn3270e header and dynamically
          turn the function back on.

        - By implementing a more generic mechanism - not tied to tn3270e
          headers - a server that forwards partial chains could still
          avoid ff-doubling.

        Here's the sort of thing I'd prefer.  This is off-the-cuff, so
        there are probably some holes in this, but it conveys the
        flavor:

        - Negotiate new telnet option to permit length-and-data
transmissions.

        - New telnet commands switch into and out of length-and-data
mode.
          The switch applies to the datastream being sent in the same
          direction.  It has no effect on the reverse datastream.

        - While in length-and-data mode, the datastream consists of
          logical packets.  Each packet is either:

          - A telnet command (introduced by 0xFF), or
          - A 2-byte length field (max 0xFEFF) followed by that many
bytes
            of uninterpreted data.

          (Actually, could allow up to 0xFF7F for the length since all
          telnet commands are beyond that.)

        What do you think?

        Regards,
          Derek

        > Please find attached draft proposals for eliminating
byte-doubling
        > and adding a "Flush" function for TN3270E.  Your comments,
        > suggestions, etc are greatly appreciated.

        > Thanks,
        > Marty Williams
        > [email-masked]
        > OpenConnect Systems

---

From: "Derek W. Bolton" <[email-masked]>
Date: Mon, 24 Nov 1997 10:04:29 +1100 (EST)

Hi Rodger,

>         I wanted  to extend my appreciation to  Marty for taking the
> time to put a draft together.

>         I did have a couple of comments about the draft and also
> about Derek's response:

> client will start with this option disabled".  Note that this may
> just be a technicality and that doing it your way doesn't have any
> adverse consequences.  However, I think that its cleaner to assume
> it is off until turned on (rather than assume that it is on until it
> isn't actually turned on).

I agree.

> 2) I *really* don't like this business of how this gets handled when
> the server doesn't buffer the entire chain.  It really looks messy.

> - an EOR bit is actually turned on/off in the header depending on if
> this is the last message in the chain (thus there's no more

> numbers are used to refer to all pieces of the same chain
> (i.e. don't just increment the sequence number because it's s new
> TN3270E command unless the EOR bit was set on the last command
> sent).

>                 This approach has the benefit of making the case
> cleaner where the server doesn't buffer the entire chain as well as
> eliminating scanning for <IAC><EOR> in all cases.

The advantage of eliminating IAC EOR may be illusory.  Presumably
the sender could put some other Telnet command there, so the
receiver still has to look for one.

I'd like to avoid the overhead of a TN3270 header per segment,
particularly if the sequence number is to stay the same for the
duration of the chain.  I'd also like to avoid worrying about
FF doubling in the TN3270E header itself.

An amalgam of these proposals is possible:

- Negotiate length+data as TN3270E suboption.

There are two possibilities as to how we define this:

i. If the option is successfully negotiated, each record can be sent
   either FF-doubled or as length+data.  A new bit in the TN3270E
   header is turned on in the latter case.  The header itself is sent
   FF-doubled.  The data are then sent, perhaps as multiple fragments.

ii. If the option is successfully negotiated, a sender can subsequently
   send commands (new Telnet commands, but only having meaning in the
   TN3270 world) to turn on/off length+data mode.  While the mode is
   on, in a given data direction, each record can be sent
   as one or more fragments.  The first fragment includes the TN3270E
   header, with no FF-doubling.

Then for either definition:

- Each fragment is introduced by a 2-byte length of the following
  data (i.e. non-self-inclusive).  The length must be in the range 0000-
  FEFF (or maybe to FF7F).  Telnet commands can be interspersed
  between the fragments.  The final fragment is followed by IAC EOR.

I prefer choice (ii).  It avoids FF doubling in the TN3270E header
itself, and it could be used in non-E if desired.

An issue which Michael Boe has brought up is the data fragmentation
at the receiver.  If the sender chooses to send multiple fragments
then the receiver winds up with the received data in multiple
discontiguous pieces in memory.  Is that an issue?  It seems to me
that there are three ways a client can handle this:

a. Copy data around to make it contiguous.
b. Have a datastream processing algorithm that copes with
   a multipart source.
c. Turn it into a feature by processing each fragment as it
   arrives, thereby reducing the time to present the completed
   screen.

Option c could induce an unnerving start-stop appearance on the
screen, but maybe the client has a way to blank the whole window
until it has finished.

> uncomfortable with how it is defined.  I don't like all this
> overloading or all of these special cases.  As a result of this and
> other issues that have come up, I think we should look at adding an
> extra byte (or stealing one of the lesser used bytes that's already
> there) and start defining command/status bits or even adding new
> TN3270E commands.  This would be a lot simpler than messing with
> overloading of existing Telnet escape sequences.  I appreciate that
> we're trying to actually "Abort some Output" here, but that doesn't
> mean we should bend over backward to try and find a way to use
> <IAC><AO>.

How about a fragment of length 0 to indicate abort?

Regards,
  Derek

>         While I appreciate Derek's comments, I think it would open
> to large a can of worms to worry about re-specifying the Telnet
> protocol to extend this data-length mechanism to all of telnet.
> Really, this kind of thing is only appropriate to block-mode
> terminals like 3270 and 5250.  Regular old character-at-a-time or
> line-at-a-time telnet wouldn't really benefit from such a mechanism.

I agree, but my guess was that there could be future block-mode
users of the Telnet base.  If you're not going to run in block
mode then you wouldn't negotiate away the use of FF-doubling.

> I believe that this byte-doubling nonsense that telnet expects
> should have never carried over into TN3270E and we shouldn't let it
> carry over into TN5250E.  An additional problem with a generic
> mechanism is how and when to transfer length information.  TN3270E
> already has a header so it should go there.  TN5250, TN3270, and
> Telnet do not, so where should it go?  Another <IAC> type thing to
> parse?  If we're doing this different ways for different flavors,
> then we've lost most of the benefit from making it a general
> mechanism.  Unless I can see a more detailed proposal on how this
> could be made to work easily in a general Telnet setting, I vote for
> just making it part of the TN3270E sub-negotiation.