Subject: Graham's comments on feature-sets-at-urls-00

From: Bill Newman <[email-masked]>
Date: Thu, 4 Mar 1999 09:56:43 -0800 (PST)

Graham Klyne wrote

> I've just read through the version of your draft submitted to the list --I
> don't know if there are any changes in the published I-D-- and would like
> to make some observations.  In making these observations, I don't think we
> have any fundamental difference about the goals, or the broad means of
> achieving them.

There shouldn't be any changes between the draft to the list
and the published I-D.

I suspect that my goals are roughly a subset of your goals. Of the
five goals in your draft-ietf-conneg-feature-hash-00.txt, we agree on

+  o  to provide an abbreviated form for referencing an arbitrary
+     feature set expression.
+
+  o  the meaning of (i.e. the corresponding feature set expression) a
+     feature set reference should be independent of any particular
+     mechanism that may be used to dereference it.
+
+  o  to be able to verify whether a given feature set expression
+     corresponds to some feature set reference without having to
+     perform an explicit dereferencing operation (i.e. without
+     incurring additional network traffic).

but I don't understand the application of 

+  o  for protocol processors that conform to [1] to be able to
+     sensibly handle a feature set reference without explicit
+     knowledge of its meaning (i.e. the introduction of feature set
+     references should not break existing feature expression
+     processors).

and I don't understand how 

+  o  to allow, but not require, some indication of how to dereference
+     a feature set reference to be included in a feature set
+     expression.

can be the basis of an interoperable system.

> My primary observation is that your draft covers two matters which I would
> prefer to see treated separately:
>   (1) feature set syntax extensions, and
>   (2) profiling of mechanisms for resolving URLs in feature set expressions
> I think that matters of representation and mechanism should be separated so
> that they can be developed independently of each other.

Graham also wrote (I'm quoting slightly out of sequence here) 

> The profiling of resolution mecahisms is a reasonable goal that I have not
> addressed.  This suggests that between us we might aim for two WG submissions:
> 
>    Syntax extensions for abbreviating media feature sets ...
>    Resolving URLs in media feature sets
> 

I agree that they could be developed independently of each other.
However, in the current situation, I think it makes some sense to do
it as a single document. The single document addresses a problem, the
feature-sets-over-thin-pipes scenario that Ted talked about in
Orlando. If we split it into two, neither half by itself address any
problem. For a complicated proposal, I'd be strongly in favor of the
two-way split, or maybe a three-way split (scenarios/requirements
document, extensions document, and resolution mechanism document). But
for this little proposal it seems like overkill -- it's a sufficiently
simple proposal that (modulo my difficulties in expressing myself:-)
it can be easily understood in its entirety.

Any comments from Ted or anyone else on this one?

> I note that the syntax extensions you propose very much overlap the
> proposals in my submission <draft-ietf-conneg-feature-hash-00.txt>.  In the
> scheme of things, the differences I see are quite small so finding
> consensus between these proposals should not be a problem.

I agree. As I understand it, we disagree about the semantics of
whether the association of a URL (or more generally, some mechanism to
recover the definition of a feature set reference) with a feature set
reference should be mandatory or optional. That disagreement limits
what we could do to make the syntax identical, but if I made the
syntax of my proposal as close to yours as possible while respecting
that limit, I'd still be happy with my proposal.

> The remainder of this message contains comments about specific portions of
> your text:
> 
> >Abstract
> >
> >   Other Internet Drafts from the CONNEG working group describe a
> >   syntax[SYNTAX] and vocabulary[FEATURES] for negotiating media feature
> ................................................[---------]
> I'd suggest "representing".  Negotiation, per se, is not covered there.
> 
> >   sets which can be used for transmission of a message.
> [...]

I used this terminology because I'd assumed that reducing two feature
sets to a compatible intersection, using the algorithm described by
the syntax document, was considered "negotiation". However, whether or
not that is considered negotiation, nothing is lost by writing
"representing" instead of "negotiating" here, since this extension is
wholly about the representation.

> >   ... For example, a
> >   feature set may specify that full color output up to 800x600 pixels
> >   is supported, or that output can have up to 300 dots per inch. These
> >   feature sets can be arbitrarily complex, and typical feature set
> .................................................[-----]
> I think it's too soon to say "typical".  Your legitimate meaning is still
> present if this word is deleted.
> >   expressions may be hundreds of bytes in length.
> [...]

OK.

> MD5 is a 128-bit value, which needs 16 bytes or 32 hex digits to represent.

Oops!

> >   syntax document[1]. Thus, it is possible for an implementation to
> >   support both this extension and the extension described in 6.1.3, in
> >   which case a filter will be described by
> >
> >      filter = "(" filtercomp ")" *( ";" parameter )
> >             / "<" absoluteURL ">" *1( ";md5=" md5value )
> ....................................^^^..................^
> RFC2234 allows "[ ... ]" as equivalent, which IMO is easier to read.
> 
> >             / "(" filtercomp *( ";" parameter ) ")"
> >               "where" named-pred-sequence "end"
> 
> Some observations about this syntax:
> 
> (a) Using the notation of RFC2234, this can be described as:
>        filter =/ "<" absoluteURL ">" ...
> i.e. without need to repeat other elements from [SYNTAX].

I wasn't sure what's considered better style, whether detailed
definitions should be repeated or just referenced. Either way is fine
with me, and if there are no other comments I'd be happy change it to
the form that you recommend.

> (b) this formulation syntactically binds "md5" to the URL form, and
> prohibits other parameter values in this position.  Here is another
> possible approach:
>        filter    =/ "<" absoluteURL ">" *( ";" parameter )
>        parameter =/ "md5" "=" 32*HEX
> This would syntactically allow "md5" to be applied to any "filter", and
> allow any other parameter (e.g. "q=") to be applied to the URL form.
> QUESTION:  is this an appropriate approach?

I'm agnostic on this one. It makes the syntax more regular, but that's
not necessarily a good thing when the currently-defined underlying
semantics don't have the same regularity:

                           semantically  allowed      allowed 
                           meaningful?   by your      by draft
                                         new syntax?  syntax?
                           ------------  -----------  --------
    absoluteURL + md5    = yes           yes          yes
    other filter + md5   = no            yes          no
    absoluteURL + q      = yes*          yes          yes
    other filter + q     = yes*          yes          yes
                           ("yes*"="meaningful at top level only")

> >   The absoluteURL field holds a URL where a filter description may be
> >   found. The md5value field, if present, holds a hexidecimal
> >   representation of an MD5 digest[MD5] of the contents of the URL.
> 
> You do not mention whether the md5 is calculated over a normalized form of
> the identified resource.  (I assume that you intend not;  if so, I'd state
> this explicitly so that it is made clear that the resource must be
> transferred *exactly*, with any embedded layout, whitespace, etc.)
> Personally, I favour some simple normalization so that, for example,
> whitespace munging by feature set transports does not affect the validity
> of the result.

I'm opposed to normalization here, because I see the benefits as very
small, and because it adds some complexity to the specification and
implementation. To a large extent, this is probably a matter of taste,
but I'll try to justify it nonetheless.

We've almost reached the new millennium, and it's now routine to set
up a clean communication channel (using some form of encapsulation if
necessary) which doesn't mess with any of the bits in the message,
whitespace or not. I'm not convinced that it's worthwhile to add any
complexity to the standard in order to try to accommodate systems
which won't do this. It adds some cost for the systems which do
communicate cleanly, and it can't adequately address the problem of
systems which don't communicate cleanly, since we can't anticipate all
the ways they might corrupt the message.

Normalizing messages was originally proposed as a way of helping the
same spec to hash to the same value even when the authors introduce
semantically insignificant changes. I dislike that also, for rather
parallel reasons. It adds cost which must be born by everyone, even
those who don't create or use syntactically different, semantically
identical versions of feature sets. And it doesn't adequately address
the problem, since there seems to be consensus that mapping all
identical versions of feature sets onto the same canonical feature set
before hashing would add too much complexity to the spec.

All that said, normalizing whitespace has a rather small cost (mostly
some seldom-used code to handle quoted whitespace, I think), so my
feelings about it are rather mild.

> >Interpretation of URLs
> >
> >   Any implementation of this extension MUST be able to process URLs
> >   with an "http" scheme. Other forms of URL MAY be supported.
> 
> My initial response to this was "why MUST http"?  But, on reflection, I
> accept that it is useful to have some common mechanism defined.  But I can
> imagine cases where "http" is not appropriate to support:  this is one
> reason that I would prefer to see the access mechanisms defined separately,
> so that the syntax can be used without a specific mechanism for
> applications where this is appropriate.

I certainly agree that if we do choose to split this document into
separate syntax and mechanism documents, this requirement belongs in
the mechanism document.

> In summary, I am content with this IFF it is stated separately from feature
> set syntax extensions.

I think I disagree here. This seems to me to be an appropriate
requirement, even if we end up treating syntax extensions and
mechanism as a single monolithic extension to the standard. As I see
it, it's not just useful to have some common mechanism defined, it's
vital. Without specifying support for at least one particular style of
URL, this standard won't provide enough information for independent
implementations to be reliably interoperable.

I can imagine scenarios where it's not appropriate to support "http:"
URLs, but they're obscure scenarios. If the standard isn't applicable
in some obscure scenarios, it's a price that I'm ready to pay for it
being reliably interoperable in the more common case. I'm even more
ready to pay this price because the scenarios I've been able to come
up with tend to involve subnets (not the entire Internet), so that
pure interoperability among strangers is less important. 

That is, 
  (1) I think without a requirement like this, there's no
      interoperability.
  (2) "http:" is the right base choice for the overwhelming majority
      of applications that care about interoperability with 
      systems written by strangers.
  (3) Some other choice may be appropriate for people who don't 
      need interoperability with systems written by strangers, but
      people who don't need interoperability with strangers aren't
      a top-priority design criterion for RFCs.

Do you have some scenario(s) in mind where this requirement would be
more of a problem than I realize?

> >   Any implementation of this extension MUST be able to process data
> >   returned with the MIME type "text". Other types MAY be supported.
> 
> Seems reasonable.
> 
> One thought I had was that a multipart containing human- and machine-
> readable parts might be used (a bit like multipart/report), so that systems
> without content negotiation could offer the user a choice.

I don't understand what you're proposing here well enough to comment on it.

> >Security Considerations
> 
> I thought this section had some good material, and may be an important
> aspect of this memo.

Thank you. But from rereading this section in light of your comments
below, I can see that it could use some rewriting with an eye to
clarity!
 
> >   Further, by specifying the MD5 hash, storing a literal copy of the
> >   desired feature set locally, and being able to repeat a failed
> >   request without URL abbreviation, the device requesting content
> >   negotiation can reduce all these new attacks to degradation of
> >   service. When the adversary doesn't interfere, negotiation would
> >   require a single round trip over the primary communications link,
> >   carrying roughly 40 bytes of compressible URL and 16 bytes of
> >   uncompressible MD5 hash. When the adversary does interfere,
> >   negotiation requires the first round trip (which fails), followed by
> >   retransmissing of the entire literal feature set over the primary
> >   link. Thus, the effect of interference is merely to slow the
> >   negotiation, not to change its result.
> 
> I *think* you are saying here that the MD5 can be used as a cache
> validator, and hence optimize feature set transfers or provide a recovery
> mechanism if a URL cannot be resolved (for reasons of system failure or
> active attack).

I was trying say that the MD5 lets you detect any interference over
the channel which was used for the URL fetch of the feature set, so
nothing worse can happen than that you have to try again with the
ordinary, unextended content negotiation.

> >From a security viewpoint, I don't believe this use of md5 alone can defeat
> an active attack.  Any active attack would, I presume, be capable of
> substituting both features AND md5 values.  But in this case, the attacker
> would need to intercept the negotiation exchange AND the URL resolution
> exchange -- increasing the difficulty of such an attack but by no means
> preventing it.  To fully prevent active attacks, I think some form of
> signature is required.

There are two channels involved, channel X from the CONNEG client to
the CONNEG server, and channel Y from the CONNEG server to some HTTP
server holding the feature set. I think of X as a cellular link, and
since the server and the client are each holding one end, either the
server or the client can impose some secure cryptographic protocol on
it. Thus, in some significant fraction of cases X could be very slow
("22 bytes per second" in the nightmare scenario that I've heard the
wireless people use) and arbitrarily secure. Y is a link over the
public Internet, which could be relatively fast, and there's no
interoperable way for either the client by itself or the server by
itself to force it to be secure. If I understand correctly, what
you're saying above is that the MD5 signature doesn't let you deal
with an attack on channel X, which is true. But the point that I'm
trying to make in the document is that the MD5 signature, transmitted
over a secure channel X, lets you deal with any attack on channel Y.
This is important when X can be made secure in interoperable ways and
Y can't be.

> >   It is expected that only a small minority of content negotation
> >   applications will be sufficiently critical that such interference
> >   will be a significant concern. After all, given that you have an
> >   adversary with the capable of tampering with communications between
> >   your servers, you are likely to have a long list of problems, and
> >   interference with content negotiation may be far from the most
> >   serious problem on that list. For this reason, and because the
> >   primary motivation for this extension is to allow content negotiation
> >   to proceed with minimal bandwidth usage on the primary channel, the
> >   MD5 checksum is optional.
> 
> I think you make a fair point and I don't argue your conclusion, but I
> don't fully agree with its justification.  I'm not sure where the
> expectations of criticality of content negotiation come from.
> 
> If content negotiation becomes a generally used feature of a widely used
> messaging system then its operation starts to become critical, I believe.
> Under these circumstances, I suggest that negotiation exchanges should be
> protected by some form of signature.  Ultimately, this is a feature of the
> application that uses content negotiation.

I'm now inclined to drop this paragraph. If an explanation is needed
for why the hash is optional, it would probably better to just saying
that this feature is primarily useful when channel X is more secure
than channel Y.

Thanks for your comments!

  Bill Newman
  [email-masked]

---

From: Graham Klyne <[email-masked]>
Date: Fri, 05 Mar 1999 12:52:37 +0000

>wholly about the representation.

Please see the terminology section of
<draft-ietf-conneg-requirements-01.txt> (and feel free to disagree, of
course).  I see feature matching as just *part* of a negotiation process.

>the form that you recommend.

I guess that RFC2234 is sufficiently new that a consensus on appropriate
style for use of the new elements has yet to evolve.

>                           meaningful?   by your      by draft

This is a nit, and I don't want to go to war over it.  But I can imagine
that there could be circumsances in which "other filter + md5" is
semantically useful, in that it could be used as an integrity check on a
filter, or applied to other (not-yet-defined) forms of filter.

I don't believe "absoluteURL + q" was allowed by your syntax:

      filter = "(" filtercomp ")" *( ";" parameter )
             / "<"
absoluteURL ">" *1( ";md5=" md5value )
             / "(" filtercomp *( ";"
parameter ) ")"
               "where" named-pred-sequence "end"

unless you do something like:

      (& <Absolute-URL>;md5=foo );q=0.6

As a point of principle, I think it's probably appropriate to use formal
syntax notation to define syntactic constraints, and semantic description
to define semantic constraints, so I'd vote for the regular syntax
approach.  But I'm not hung up on this.

>but I'll try to justify it nonetheless.

Fair enough:  I think that should be stated explcitly.  I don't think
justification is needed.

>the ways they might corrupt the message.

In your applications, I think this is true.  Coming from a more e-mail
oriented perspective I'm less sanguine about clean channels.

>feelings about it are rather mild.

I'm certainly not arguing for complex semantic normalization.  My offering
proposes very simple-minded whitespace normalization.

>separate syntax and mechanism documents, this requirement belongs in

That would suit me just fine!

>vital.

I envisage deployments where http is simply not available.

>URLs, but they're obscure scenarios.

Angonamo!  I've not argued for not supporting http: URLs.  I've only argued
for not requiring HTTP resolution in all cases.  As I type this, I do not
have a live Internet connection;  for e-mail based applications, I believe
that mechanisms that do not depend on continuous Internet connectivity are
appropriate.  I do not believe this is an "obscure" scenario.

[...]
>  (1) I think without a requirement like this, there's no

My view is that the resolution mechanism *requirement* should be defined as
part of the application, not as part of the media feature framework.

For example:  the media feature syntax work is referenced by Extended
Internet Fax:  it is interesting to note that all of the normative
references are contained in the master document;  the remaining documents
are cited for their technical detail, not the requirements that they impose.

>more of a problem than I realize?

E-mail, Internet fax.  (And also some non-IETF work that I am involved with.)

>I don't understand what you're proposing here well enough to comment on it.

This is blue-sky stuff, and I'm not proposing it for immediate
consideration.  But if the URL resolves to something like this:

    multipart/media-features
        text/plain  (or   audio/xxx ?)

        (human comprehensible description of features)

        --
        text/plain

        (media feature expression)

This admits the possibility that the URL used could be used in different
ways by 'conneg' aware and other systems to obtain a description of the
media features.

[...]
>server holding the feature set. 

I don't think we are in fundamental disagreement here, but I had to
double-take when I read "CONNEG client" and "CONNEG server".   I think you
are describing a very specific (but legitimate) scenario.  While I think it
is appropriate to meet the requirements of this scenario, we shouldn't
treat it as the only one.  (In your scenario, I would note that it may be
sufficient to secure just one of the channels by other means, so the
overhead of strong security can be focused on some key systems.)

#g

------------
Graham Klyne
([email-masked])

---

From: Graham Klyne <[email-masked]>
Date: Fri, 05 Mar 1999 12:52:33 +0000

[...]
>+     references should not break existing feature expression

A fair question:  I'll try and explain waht I had in mind.

First, I must acknowledge that in the general case, final determination of
a feature handling capabilities is not possible without resolution of the
feature set abbreviation.  In this, I think we agree.

But, I think that in some circumstances some useful progress can be made in
the
absence of full knowledge.  If a feature set abbreviation can be treated
somewhat like an unknown feature tag, then the feature set matching can
produce a result that may not be perfect, but may be better than no result
at all:

Example 1:
  Party A asserts:  (& (type="image/TIFF") (h.foo) )
  Party B asserts:  (& (type="audio/wav") (h.foo) )
Now whatever the meaning of "h.foo", these parties don't have media
features in common.

Example 2:
  Party A asserts:  (& (type="image/TIFF") (dpi=[100,200,400]) (h.foo) )
  Party B asserts:  (& (type="image/TIFF") (dpi=[50,100]) (h.foo) )
In this case, the feature capabilities can be partially resolved to:
  (& (type="image/TIFF") (dpi=100) (h.foo) )
In this circumstance, a sender *may* choose to assume the (h.foo)
constraint is satisfied and send a 100dpi image in any case.

This kind of partial resolution is possible in my proposal because the
basic feature expression syntax is not extended.  This is what I mean by
"sensibly handle a feature set reference without explicit knowledge of its
meaning".

>can be the basis of an interoperable system.

I am trying to separate *meaning* from *mechanism*.

To put this in concrete terms, if a feature set reference takes the form of
a URL, then that URL contains an "indication" of how it may be dereferenced
(e.g. http).  But, the meaning of that URL does not have to depend on the
availability of that dereferencing mechanism;  in an e-mail environment,
the URL might be used as a key into an LDAP or RESCAP lookup mechanism.

Simply because an identifier contains the name "http" should not be taken
to mean that the http protocol is the only means by which it can be resolved.

Does this help?

>it can be easily understood in its entirety.

I think we have different problems in mind (which is good).

I anticipate an e-mail scenario in which some feature expression
*representation* that we agree is used in conjunction with some mechanism
that is not being considered in this group.  The original thrust of this WG
was that media features be protocol independent.  We are now considering
bringing some protocol specific into the frame:  I'm not arguing with that,
but I really would like to keep some clear separation.

The fact that a common representation may be used with several different
mechanisms actually *requires* that there are multiple documents.  Even if
we proceed and put the expression format and http resolution into one
document, other documents will still be required to deal with other
mechanisms.

>Any comments from Ted or anyone else on this one?

Yes please!

I'll leave this here, and pick up any other points separately.

#g

------------
Graham Klyne
([email-masked])