Subject: Re: [AAA-WG]: OctetString type

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 1 Feb 2001 17:00:12 +0100

>>>>> David Frascone writes:

David> I think removing the basic string type was a bad idea.  With
David> octet string, there is no easy way to tell if the data is
David> printable.  It was nice having a string type, since the data
David> could be printed in log messages, debug strings, etc.

David> I think we should still have two types, Octets, and String, and
David> treat them appropriately.

David> Granted, on the transport side, DIAMETER will treat the two
David> equaly, but it makes debugging/logging much nicer to know when
David> you can print out the data involved.

I totally disagree. The protocol should only distinguish a minimal set
of base data types. In addition, you should have a mechanism to define
application specific "types" that have additional semantics
(e.g. restrictions on the character set) in the data model. This
ensures that you do not need to change the protocol whenever a new
string type is needed. For example, we could introduce UTF8 strings in
SNMP without touching the protocol. And we have lots of other special
purpose octet strings, such as DateAndTime.

The logging argument does IMHO not really count. It is straight
forward to write a piece of code which tests whether a given octet
string is printable or not. And due to internationalization
requirements, you will have to check a UTF8 string for unprintable
byte sequences anyway before printing it since you can't rely on
it being all 7 bit ASCII.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

---

From: David Frascone <[email-masked]>
Date: Thu, 1 Feb 2001 11:48:15 -0600

Since the *type* of the data is not transmitted on the wire, I'm not sure
I see your point.

I'm asking for an internal dictionary representation of the difference 
between data or string.

It does not effect the transmitted protocol at all, since data types are
not transmitted on the wire.

So, once again:  I think we need to diferentiate between OctetString and
PrintableString in the *non-transmitted*, *non-protocol-changing*, 
INTERNAL representation of the data :)

-Dave

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 1 Feb 2001 23:11:31 +0100

>>>>> David Frascone writes:

David> Since the *type* of the data is not transmitted on the wire,
David> I'm not sure I see your point.

David> I'm asking for an internal dictionary representation of the
David> difference between data or string.

David> It does not effect the transmitted protocol at all, since data
David> types are not transmitted on the wire.

I thought you were talking about the String and Time types that were
specified in section 2.2.3 in <draft-calhoun-diameter-17.txt>. These
were in my view protocol data types since the definition of these
types specified the encoding of the AVP and it seems desirable to keep
the number of different encodings to a minimum. (And this was a bigger
issue with the Complex type and actually the reason to get rid of it.)

What is needed is indeed a way to define application specific "types"
on these base data types (or encodings if you will). SMIng has such a
data type mechanism and I am sure Erik will invent one for the XML
proposal as well (if it is not already there). :-)

/js

---

From: David Frascone <[email-masked]>
Date: Thu, 1 Feb 2001 17:08:16 -0600

All I was asking for was the distinction of OctetString and a PrintableString
type.

Basically, so I'd know if it was safe to do a sprintf into a log string.  It
wouldn't affect the transport at all, but it *would* add a type to the AVP
definitions.

So, for example, the User-Name AVP would be of type String.

Which would make a username of 0xdeadbabe illegal.

But, it would allow me to do someting like:

	syslog(LOG_INFO, "Authentication successful for %*s", 
		userAvpDataStructure.dataLength);
		userAvpDataStructure.data);

So, it does affect the draft, and it does affect the dictionary, but it
does NOT affect the protocol, unless we worry about transmitting the 
NULL (which is NOT what I'm asking for)

So, can we have a psuedo-vote?  Does anyone NOT want a printable-vs-
unprintable string type for AVPs?

-Dave

---

From: Barney Wolff <[email-masked]>
Date: Thu, 1 Feb 2001 18:00:11 -0500

A cautious programmer will derive little benefit from an explicitly
printable string, because one must verify anyway that the string
is printable and not too long (the latter was less of a problem
in the "good old days" of the 1-byte length field).  But in the
end I think it's a good idea, as an undifferentiated octet
string is hard to process - do you always print it in ascii if
it happens to all be ascii?  What if it was an IP address or a
binary key that just happened to look printable?

In other words, to even attempt to print a value as a printable
string, you have to know that it's supposed to be one.  That
sort of folklore should be in the dictionary, not in the code.
It does not have to be on the wire, though.

Barney Wolff

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 2 Feb 2001 00:10:28 +0100

>>>>> David Frascone writes:

David> All I was asking for was the distinction of OctetString and a
David> PrintableString type.

Which is OK if it is done right. ;-)

David> Basically, so I'd know if it was safe to do a sprintf into a
David> log string.  It wouldn't affect the transport at all, but it
David> *would* add a type to the AVP definitions.

David> So, for example, the User-Name AVP would be of type String.

David> Which would make a username of 0xdeadbabe illegal.

David> But, it would allow me to do someting like:

David> 	syslog(LOG_INFO, "Authentication successful for %*s",
David> userAvpDataStructure.dataLength); userAvpDataStructure.data);

David> So, it does affect the draft, and it does affect the
David> dictionary, but it does NOT affect the protocol, unless we
David> worry about transmitting the NULL (which is NOT what I'm asking
David> for)

Once again, there are two potential bugs here. First, the assumption
that printable is indeed printable. I can send you UTF-8 printable
strings that will make your syslog files read look really nice.
Second, there is no reason to assume that the actual data you got is
printable just because some spec says its printable. So you need to
check the octets anyway to be on the safe side.

David> So, can we have a psuedo-vote?  Does anyone NOT want a
David> printable-vs- unprintable string type for AVPs?

Are you American? Still proposing to vote? :-)

/js

---

From: David Frascone <[email-masked]>
Date: Thu, 1 Feb 2001 18:14:54 -0600

I am assuming (perhaps incorrectly?) that the implementers know how to write
code and check for error conditions.

But, if I have no idea whether something is printable, I guess I can simply
check (as someone suggested earlier)

So, I get some random blob of data, and starting doing my version of an 
isprint on the string, and low and behold, it's printable.

So, I output the data:

01-FEB-2001 05:15:23 GMT diameterd Received printable data:  Avp NONCE, data "A8p#I3(*@#m%*j#3^&"

I think having a String type makes a little more sense.  Even if a malicious/
buggy implementation might abuse it.

-Dave

---

From: Erik Guttman <[email-masked]>
Date: Fri, 2 Feb 2001 10:05:14 +0100 (MET)

> code and check for error conditions.

We agree then that any implementation will have to check whether a string
is printable before it actually prints it.

> check (as someone suggested earlier)

Yes you do have an idea:  DIAMETER specs define AVPs so as to include both 
their data type and the correct interpretation of the data.

Every attribute of OCTETSTRING data type will also be described.  
Attributes which are UTF8String will be described as such.  Attributes 
which are digital signature (binary) data will be described that way.  

Secondly, the data dictionary will describe this.  Whether we use
SMIng or XML based dictionaries, this information is available.

The SMIng proposal provides this information this way:

class VendorName {
  attribute Utf8String name {
    description "The name of the vendor.";
  }
};

Utf8String is defined by importing the definition:
module TUBS-SMING-DIAMETER {
  import IRTF-NMRG-SMING (Utf8String);
  ...
};

Utf8String is defined in
http://www.ietf.org/internet-drafts/draft-ietf-sming-modules-00.txt

       typedef Utf8String {
           type        OctetString;
           format      "65535t";      // is there a better way ?
           description
              "A human readable string represented using the ISO/IEC IS
               10646-1 character set, encoded as an octet string using
               the UTF-8 transformation format described in RFC 2279.

               Since additional code points are added by amendments to
               the 10646 standard from time to time, implementations must
               be prepared to encounter any code point from 0x00000000 to
               0x7fffffff.  Byte sequences that do not correspond to the
               valid UTF-8 encoding of a code point or are outside this
               range are prohibited.

               The use of control codes should be avoided. When it is
               necessary to represent a newline, the control code
               sequence CR LF should be used.

               The use of leading or trailing white space should be
               avoided.

               For code points not directly supported by user interface
               hardware or software, an alternative means of entry and
               display, such as hexadecimal, may be provided.

               For information encoded in 7-bit US-ASCII, the UTF-8
               encoding is identical to the US-ASCII encoding.

               UTF-8 may require multiple bytes to represent a single
               character / code point; thus the length of a Utf8String in
               octets may be different from the number of characters
               encoded.  Similarly, size constraints refer to the number
               of encoded octets, not the number of characters
               represented by an encoding.

               Note that the size of an Utf8String is measured in octets,
               not characters.";
       };

The XML proposal provides it this way:

   <diameter>
     <base>
        <avp code="266" may-encrypt="true"
         vendor-flag="disallowed" mandatory-flag="disallowed">
          <name> Vendor-Name </name>
          <type printable="true"> OctetString </type>
        </avp>
        <!- lots more stuff ->
      </base>
    </diameter>

> isprint on the string, and low and behold, it's printable.

You want to print traces for arbitrary, unknown DIAMETER extensions?  
The output would look something like this

  01-FEB-2001 05:15:23 GMT diameterd Received 
    Command-Code: ??? AVP-Code: ??? printable data:  P(#OIU#%IOJVD

Question:  Is this really useful?
If you think so then, question:  Can't you print the data whether it 
is a UTF8 string or not?
 - If the octetstring is a well formed UTF8 string, print it as a string.
 - Otherwise, print it as a hex dump.

In this case you do not know a priori how to interpret the AVP, so this
is the best you can do.  If you did know, you'd already have code for
interpreting the AVP correctly (see above).

Erik

---

From: David Frascone <[email-masked]>
Date: Fri, 2 Feb 2001 11:31:35 -0600

Ahh!  That's exactly what I was looking for.  As long as the attributes are
described, then I'm happy!

Consider this thread closed!

-Dave