Subject: ;binary and userCertificate (Was: Private email ...)

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 20 Feb 2002 09:52:08 -0800

;binary has uses beyond those with certificates.  For
example, to request transfer of a DN in BER form instead
of RFC 2253 form.  IMO, an attribute of syntax DN would
serve as a better example of ;binary's "intended" use
(as described in Section 4.1.5.1 of RFC 2251).

It doesn't make much sense to return values in BER form
when the client has not explicitly requested it.  While
RFC 2251 stated: 
   Clients MUST NOT expect servers to return an attribute
   in binary format if the client requested that attribute
   by name without the binary option. 

This did not prevent, generally, a server from running any
attribute in binary format even though requested without
;binary.  Likely it was assumed to be obvious that a server
shouldn't return CN;binary unless it was asked to.  The
revised specification explicitly states that servers must
not use binary transfer when the attribute was requested
without ;binary.

As all attributes explicitly make use of binary transfer
state that values MUST be requested and transferred with
the ;binary option, the revised specification is self
consistent.

In regards to X.509 schema, one should not fix that which
is not broken.  The document should:
	a) indicate edition of X.500 recommendations used
	   is different from those used in LDAPv3 and diccuss
	   impacts of this difference,
	b) add missing matching rules, and
	c) state that values and their encoding forms
	   be preserved.

I see no reason to define a LDAPv3 native (octet)string encoding
for these syntaxes as ;binary works just fine.  Yes, there
may be broken LDAPv3 implementations, they should be fixed.

The fact that LDAPv2 uses a different mechanism is
inconsequential to this discussion.

---

From: Christopher Oliva <[email-masked]>
Date: Wed, 20 Feb 2002 14:35:48 -0500

I am restricting my comments to the Certificate, Certificate List and
Certificate Pair syntaxes. Other functionality for other syntaxes do not
apply and no changes are needed as far as I am concerned.

Comments in-line below.
 
>    by name without the binary option. 

If the Certificate (etc.) syntax is changed so that the native encoding is
the BER encoding, then the quote from RFC 2251 above does not apply. I
believe the above statement applies to attributes whose native syntax is not
BER encoding. Clients that are not aware of the native encoding will
continue to request certificates with the ";binary" transfer option.

> the ;binary option, the revised specification is self

The proposed change is not to alter the binary transfer syntax for
certificates. Instead the native encoding of certificates will change to be
BER encoding therefore the above statement about the binary transfer
mechanism does not apply since the transfer of the native encoding will be
BER. The fact that the native BER encoding for a certificate and the
encoding used for the ";binary" transfer option are the same is
inconsequential.

> inconsequential to this discussion.

In reality there are serious interoperability and backwards compatibility
problems. Vendors have not been able to implement the strict interpretations
of ";binary". The proposed solution is clean, simple and preserves
interoperability in all cases.

I don't think we can simply choose to ignore interoperability and backwards
compatibility.

I have still not seen a compelling reason to enforce the strict ";binary"
interpretation. On the other hand, there are many compelling reasons to
adopt the proposed change.

Chris.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 20 Feb 2002 18:35:51 -0800

Let me boil this down...

The specification says certificates are to be requested and
transferred using ;binary.  There are multiple independently
developed interoperable implementations of this specification.

You state that the specification should be changed because
some other implementors have produced implementations which
don't interoperate with the above implementations.

If these other implementors' interpretation of the specification
was reasonable, that is, a result of ambiguity in the
specification, then it could be argued that specification
should be clarified to one of the two (or more) reasonable
interpretations.

If on the other hand, these implementors simply did not
adhere to the specification, they are broken.  They should
be fixed.

It's my belief that this is a case of the latter.

Kurt

---

From: Christopher Oliva <[email-masked]>
Date: Thu, 21 Feb 2002 11:37:12 -0500

It still hasn't been made clear how the proposed change will have a negative
impact. As explained, the change will only increase interoperability.

I'll explain it again just to be clear (in point form):

- new PKIX spec defining X.509 syntaxes will define their native encoding as
BER
- the native encoding would be identical to the ";binary" transfer syntax
encoding (for X.509 syntaxes)
- this means ";binary" will not be mandatory to request these values
- no changes are needed to the LDAPbis specs (other than to remove any
descriptions of X.509 syntaxes)
- clients that use ";binary" (for x.509 syntaxes) can continue to use this
without any impact
- search add or modify involving such syntaxes can use ";binary" or not -
makes no difference

So you see, it's a win-win situation. What is the downside of making this
change ? It will only increase interoperability. Yes, there are examples of
non-ldapv2 problems but I will not get into specifics on this list. I have
in a previous email detailed many issues on this topic. 

Anyway, I don't see how you can simply ignore ldapv2. How can you provide a
migration path from ldapv2 to v3 if you ignore it? There are customers using
ldapv2 today ... are you supposed to simply ignore that ? Many server
products on the market still accept ldapv2 connections, therefore there is
still a need for ldapv2.

Chris.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Thursday, February 21, 2002 10:21 AM
> To: Kurt D. Zeilenga
> Cc: Ramsay Ron; Christopher Oliva; LDAP BIS
> Subject: Re: ;binary and userCertificate (Was: Private email ...)
> 
> 
> Kurt D. Zeilenga wrote:
>  >
> 
> >>The standard no longer applies, so certificates can go 
> their own way?
> >>
> > 
> > Sure, you can throw out the old specification and start over.
> > I don't believe that is warranted in this case.
> 
> 
> I agree with Kurt. But clearly there are at least two points of view:
> 
> (1) Those who believe ;binary was a bad idea with respect to 
> certificate 
> attributes (I'll leave other uses of ;binary out of this discussion). 
> The introduction of ;binary caused more interoperability 
> problems than 
> it solved.
> 
> (2) Those who believe ;binary was a useful idea with respect to 
> certificate attributes. It allowed people to build standards 
> compliant 
> LDAPv3 clients and servers that interoperate in this area.
> 
> Now, I have never heard of any interoperability problems that 
> involve an 
> LDAPv3 client talking to an LDAPv3 server to search for, retrieve, or 
> modify certificate attributes. In all operations, ;binary is 
> used in the 
> AttributeDescription. There is no ambiguity. Everyone is happy.
> 
> Can someone who is subscribes to (1) provide an example of an 
> interoperability problem that does not involve LDAPv2?  If 
> not, then the 
> real question is whether we care about LDAPv2 any longer. I do not.
> 
> -Mark Smith
>   Netscape
>

---

From: Christopher Oliva <[email-masked]>
Date: Thu, 21 Feb 2002 14:08:24 -0500

It sounds like you agree with the whole concept but there are a few details
to work out (like some strategically placed shoulds and mays). Correct me if
I'm wrong.

comments below.

> At 08:37 AM 2002-02-21, Christopher Oliva wrote:
> >It still hasn't been made clear how the proposed change will 
> have a negative impact. 
> 
> If there are two transfer mechanism, one should be a MUST,
> one should be a SHOULD or MAY.
> 
> As one of the two would be not mandatory to implement,
> then implementations SHOULD NOT use the non-mandatory
> mechanism unless they have knowledge the peer supports
> it.  
> 
> Future implementations must be able to interoperate
> with existing conformant implementations.  Existing
> conformant implementations should not be expected
> to be altered.

Yes and there are existing conformant implementations using RFC 2559. But
the new requirements of the revised RFC 2251 will probably result in some
existing implementations to change. Let me explain:

in RFC 2251, the original definition of attribute options said they should
be treated like attribute subtypes. So, if you define a search filter like
"userCertificate=*", by all rights, this should match any entry that
contains a userCertificate value. According to the revised RFC 2251bis
document, this functionality would no longer work.

> 
> So, servers should be restricted to ;binary unless the
> the client explicitly requested the native string
> representation.
> 
> A client would only request the native encoding if it
> knows the server supports it.  A discovery mechanism
> would have to be provided.  Additionally, client

I disagree. Existing client implementations that only use ";binary" would
continue to work. Once a server supports the native encoding, the same
clients would still continue to work. Clients that are only capable of
requesting the native encoding have no choice therefore a discovery
mechanism will not help them. This sounds more like a SHOULD requirement
directed to the clients that states they should use ;binary - but they don't
have to.

The point is: why would a client requesting the ;binary transfer suddenly
try to switch to use the native encoding? Clients that only support the
native encoding can't switch.

> developer's would have to be warned that some
> servers might use ;binary transfer even though
> the native string was requested.  As this is a
> behavior which the existing specification allowed
> (due to an ambiguity in the specification which,
> I hope, will be removed).
> 

See my above comment about ambiguity in the search filter.

> That is, a client MUST be prepared to interoperate
> with implementations which did not support the
> second transfer mechanism.  The second transfer
> option, hence, should be a MAY.
> 
> As a client would be prepared to interoperate with
> implementations which did not support the second
> transfer mechanism and the second transfer mechanism
> adds no functionality not offered by existing transfer
> mechanism, adding a second transfer mechanism is
> pointless.
> 
> Adding pointless features does harm.

Are you suggesting this is pointless ? I disagree.

The ";binary" was never needed to work with certificates and the proof is
that ldapv2 systems were deployed using RFC 2559 (standard) that did not use
";binary". So I ask you, what do you call a feature that is not necessary ?

Chris.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 21 Feb 2002 12:40:47 -0800

At 11:08 AM 2002-02-21, Christopher Oliva wrote:
>Are you suggesting this is pointless ?

Yes.  Because adding a second transfer mechanism for
these syntaxes solves no interoperability problem between
independently developed implementations of the LDAPv3
technical specification.

In regards to the handling of (userCertificate=*)
versus (userCertificate;binary=*) I would have to
agree that the revision I-D needs to be clarified.
I will offer a minor clarification in a separate
post.

Kurt

---

From: "Ramsay, Ron" <[email-masked]>
Date: Fri, 22 Feb 2002 12:11:33 +1100

I don't understand this business about two transfer syntaxes. Certificate is
to be given a native encoding which is, basically, the certificate.

If you request "certificate", you get a binary blob which is the
certificate.

If you request "certificate;binary", you get the same thing!

As regards compatibility, if a client requests "certificate;binary", the
server returns "certificate;binary". If a client requests "certificate" (and
no current conforming client will), the server returns "certificate".

Seems self-correcting to me!

Ron.

-----Original Message-----
From: Kurt D. Zeilenga [mailto:[email-masked]]
Sent: Friday, 22 February 2002 5:20
To: Christopher Oliva
Cc: [email-masked]'; Ramsay, Ron; Christopher Oliva; LDAP BIS
Subject: RE: ;binary and userCertificate (Was: Private email ...)

At 08:37 AM 2002-02-21, Christopher Oliva wrote:
>It still hasn't been made clear how the proposed change will have a
negative impact. 

Kurt

---

From: Christopher Oliva <[email-masked]>
Date: Thu, 21 Feb 2002 21:50:17 -0500

If you yourself have made this error ... it is very possible that some
vendor has made the same error and might explain broken implementations.

I for one think the spec is ambiguous ... let me explain:

RFC 2251, clause 4.1.5 defines attribute descriptions as subtypes of the
attribute without any options. Neither this clause nor clause 4.1.5.1 says
anything special about this not holding for ";binary". The first problem is
that subtypes are not defined in this RFC. As it turns out, the X.501
definition explicitly says that values can be accessed via the "supertype"
as well as the subtype (I'm paraphrasing).

Now, clause 4.1.5.1 says that "clients MUST NOT expect servers to return an
attribute in binary format if the client requested that attribute by name
without the binary option". The implication is that servers may infact,
return the binary encoding even if the binary option was not used by the
client since the clause says nothing about the behaviour of the servers in
this instance.

So who's wrong here? The server for generating the binary encoding? Not
according to the RFC.

This is in my opinion, sufficient proof that there is ambiguity. There is
sufficient grounds to say that servers should be allowed to return binary
encodings if a client makes a request without ";binary".

Now look at RFC 2252. 

Clause 6.5 defining the certificate syntax says this:

  "... values in this syntax MUST only be transferred using
  the binary encoding, by requesting or returning the
  attributes with descriptions 'userCertificate;binary' or
  'caCertificate;binary'."

The main point being stressed by the MUST is the transfer encoding. One
could argue that the MUST only applies to the transfer using the binary
encoding but not necessarily to the following clause that talks about the
attribute descriptions.

Now look at the word "or" between 'requesting' and 'returning'. It is
possible that the word "or" was used instead of the word "and" in order to
account for the fact that clients might request userCertificates without the
;binary option. This directly conflicts with the MUST NOT of 4.1.5.1 from
RFC 2251 above. This same thing could be used to justify a server returning
userCertificate instead of userCertificate;binary.

Chris.

> Sent: Thursday, February 21, 2002 1:50 PM
> To: Christopher Oliva
> Cc: [email-masked]'; Ramsay Ron; Christopher Oliva; LDAP BIS
> At 10:20 AM 2002-02-21, Kurt D. Zeilenga wrote:
> >Additionally, client
> I mispoke.  The ambiguity I refer to above doesn't
> does apply in the certificate case as the existing
> specification mandates use of ;binary for certificate
> syntaxes.  Hence, this warning would not be needed.
> 
> My apologies for any confusion caused by my error.
> 
> Kurt
> 
>

---

From: "Ramsay, Ron" <[email-masked]>
Date: Fri, 22 Feb 2002 14:28:12 +1100

Hi,
 
Regarding 6.5, this is actually a feature of English, not of the spec. One
may request an attribute or return an attribute (though not both at the same
time) and in each case ;binary should (MUST) be used.
 
Regarding the supertype issue, the new version of the specs propose that
;binary is a transfer ooption and is not related to normal attribute
subtyping. The radicals on the list would like ;binary to be dropped as it
appears to be solving a problem that doesn't actually exist.
 
Ron.

-----Original Message-----
From: Christopher Oliva [mailto:[email-masked]]
Sent: Friday, 22 February 2002 13:50
To: 'Kurt D. Zeilenga'; Christopher Oliva
Cc: [email-masked]'; Ramsay, Ron; Christopher Oliva; LDAP BIS
Subject: RE: ;binary and userCertificate (Was: Private email ...)

Now look at RFC 2252. 

Chris. 

> From: Kurt D. Zeilenga [ mailto:[email-masked]
<mailto:[email-masked]> ]

---

From: "Ramsay, Ron" <[email-masked]>
Date: Fri, 22 Feb 2002 16:00:32 +1100

Kurt,

Excuse me, but 1.3.6.1.4.1.1466.115.121.1.27 is not associated with INTEGER
from the X.500 point of view. INTEGER is pretty simple, but if you consider
PresentationAddress, I would say that the SYNTAX doesn't specify it at all.

This should be clarified by relating it more strongly to the X.500
definition (or including the definitions in, say, a syntax document) and
moving into the X.500 world by indicating that all syntaxes have an ASN.1
form and some also have a string form.

Thanks,

Ron.

-----Original Message-----
From: Kurt D. Zeilenga [mailto:[email-masked]]
Sent: Friday, 22 February 2002 15:23
To: Ramsay, Ron
Cc: Phil Griffin; LDAP BIS
Subject: RE: ;binary and userCertificate (Was: Private email ...)

At 04:57 PM 2002-02-21, Ramsay, Ron wrote:
>Thanks, Phil, but I thought Kurt was referring to LDAP. He has been arguing
>that LDAP defines an ASN.1 type for its syntaxes, because he wants ;binary
>to transfer this type, but I have failed to find specific references in the
>LDAP specs.

Even if ;binary was stricken from RFC 2251, I would argue
that there is the LDAP syntax OID identifies the
corresponding ASN.1 data type in X.500.  Because without
this relationship, little of LDAP would actually make
sense.

For consider,
   ( 2.5.13.29 NAME 'integerFirstComponentMatch'
     SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 )

There is zero text in LDAP RFCs which say how to implement
this, but yet there are multiple interoperable implementations
of this matching rule.  Why?  Because 2.5.13.29 is associated
with the X.500 matching rule integerFirstComponentMatch
and 1.3.6.1.4.1.1466.115.121.1.27 is associated with
the ASN.1 date type INTEGER.

While the LDAP specification uses poor mechanism to relate
OIDs with elements of X.500 schema, most implementors have
figured out not only that there are relationships, but
agree to which OID refers to what.  If they didn't agree,
things wouldn't interoperate.

In revising the specification, it's our job to clarify the
specification so that others may repeatedly development
which interoperate.  This will require being a lot clearer
about the relationship between LDAP and X.500.

I will take your above comments as indications of an area
where we need to provide clarification.

Kurt

---

From: Christopher Oliva <[email-masked]>
Date: Fri, 22 Feb 2002 10:07:06 -0500

> time) and in each case ;binary should (MUST) be used.

You cannot deny that the particular wording in 6.5 is clumsy and can be
interpreted in different ways. A more clear wording should have been used. 

It is possible, because of unclear english, to interpret this clause to mean
that the MUST (strictly) applies only to the encoding and not the use of
;binary in both the request and response.

That is one possible interpretation. I will suggest another in a following
email.

Chris.