Subject: The robustness principle, as view by user agent implementors (Re: Working Group Last Call: draft-ietf-httpbis-content-disp-02)

From: Adam Barth <[email-masked]>
Date: Sun, 3 Oct 2010 12:14:37 -0700

Thanks for your thoughtful message.  There's a cultural gap here that
sometimes makes it difficult to understand why user agent implementors
have a different perspective from server operators on some issues.  In
this message, I'll try to explain why we have the perspective we do.
Note that I'm not trying to convince you to adopt our perspective, I'm
just hoping this message will lead to more understanding.

For the remainder of this email, I'd like to ask you to drop all value
judgements about what's "good" or "bad", "valid" or "invalid."  These
are social constructs that we layer on top of the underlying bits and
bytes.  I'll try to present the issue in a value-neutral way, and I
hope you read it in that way.

Imagine, for a moment, that there's a certain network message that
servers send to user agents.  For the sake of discusion, let's call it
the Foo header.  Let's assume there are a number of different
semantics implemented by the different user agents.  The set of all
possible Foo messages breaks down as follows:

All messages
    >=
Messages that are meaningful in at least one semantics
    >=
Messages with the same meaning in all semantics

Now, let's assume servers both generate messages and have an intended
semantics in mind.  Of course, we can't observe the servers intended
semantics, but let's imagine each server has some semantics in mind
for its messages.  Furthermore, let's make the simplifying assumption
that the intended semantics for each message is implemented by at
least one user agent (i.e., the server isn't totally insane).

All messages
  >=
Messages generated by at least one server
  >=
Messages generated by at least one server that can be interpreted with
a single semantic theory (this set is not unique)
  >=
Messages generated that have the same meaning in all semantics

For the time being, lets assume that the existing entities (both user
agents and servers) are immutable.  How is a new entrant into this
market incentivized to behave?

A server wishes the message it generates to be interpreted the same
way by all user agents (or as many as feasible).  If the set of
messages with the same meaning in all semantics is large enough, a new
server entrant will be incentivized to generate messages in that
subset.  Conversely, a user agent wishes understand the intended
semantics of the most number of messages.  Therefore, the user agent
is incentivized to process the largest set of messages generated by at
least one server that can be interpreted by a single semantic theory.

Notice that the semantics selected by the new server entrant is a
subset of the semantics selected by the new user agent entrant.
There's no reason to believe these sets will be equal.  To the extent
that standards explain to new entrants how they ought to behave,
servers wish to know about the set of messages with the same meaning
in all semantics, but user agents wish to know about the largest set
of messages generated by at least one server that can be interpreted
with a single semantic theory.

(There's a little more to the story to explain why user agents also
want a complete semantic theory, one that defines the semantics of all
messages, but I can explain that in another message.)

Adam

On Sat, Oct 2, 2010 at 5:59 PM, Bjoern Hoehrmann <[email-masked]> wrote:
> * Adam Barth wrote:
>>Insulting an important constituency is unlikely to generate consensus
>>by win that constituency over to your point of view.
>
> There is a popular german short story by Peter Bichsel, "Ein Tisch ist
> ein Tisch" (a desk is a desk). He tells of an old man who is trapped in
> his daily routine, everything the same all day. One day he takes a walk
> and for no particular reason he realizes he likes many thing about it.
> He thinks that now everything will change, but when he comes home there
> is still the same desk and still the chairs, the same bed, and he hates
> it.
>
> "Always the same desk", he says, "always the same chairs." And he starts
> wondering why a desk is called "desk" and the chairs called "chairs". He
> wonders why the bed isn't called "ball" and is amused by the thought.
> "Now everything is going to change!", and from now on he calls the bed
> "ball". "I am tired and want to go to ball."
>
> Next day he ponders what to call the other things in his room; his desk
> becomes "carpet", his chair "clock", his clock "photo album" and so on.
> So in the morning he lays in his ball and around nine the photo album
> rings. He is greatly amused and continues renaming things and replaces
> the verbs also. He now had a language all for himself.
>
> Eventually he started having trouble translating between his language
> and the language the people around him use, to a point where he became
> afraid to talk to them and started laughing when he heard others talk.
> He stopped talking.
>
> Meaningful communication requires that sender and recipient agree on the
> meaning of what is being exchanged, otherwise they will fail to communi-
> cate. Saying that the specification of a communication protocol is only
> for "servers" is the equivalent of the old man telling you what words he
> is using, but not telling you how you are to understand them. Since the
> old man and you are using the same words, you would learn nothing from
> that. A specification only for servers would at best make any sense at
> all if there already was a higher-level specification that actually de-
> fines the protocol, and the other specification is just a subset. This
> is not the case here. This specification defines "If you send this, you
> can expected it to be understood in this way"; "If you receive this, you
> should understand it in this way." It addresses senders and receivers
> alike. The idea that a communication protocol specification is only for
> "servers" is silly.
>
> What you rather want the specification to say is that there is a con-
> siderable amount of content that does work in major implementation in
> a specific way, but whose meaning is not defined by the specification,
> and that that is a severe problem implementers will be interested in
> addressing and be reasonably able to do so. So far you have provided
> no evidence of that being the case, and my rudimentary tests suggest
> you are mistaken.
>
> If you can come up with something that works in all of IE6, Firefox,
> and an independent third browser of your choosing, and demonstrate this
> issue comes up for let's say 1 in 10,000 downloads with a content-
> disposition header, where falling back to heuristics for the filename
> would have substantial impact on the user experience, then I'd be more
> than happy to talk about that. In my data I see only unquoted spaces
> that do not work across major browsers, and headers that specify only
> the filename (most of which are no problem because the filename they
> specify is redundant with the typical heuristics) that occur at a
> frequency worth mentioning. You are also more than welcome to propose
> alternate metrics alongside your evidence. Vague suggestions that the
> specification is inadequate are not helpful.
> --
> Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
> Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
> 25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/
>

---

From: "Eric J. Bowman" <[email-masked]>
Date: Sun, 3 Oct 2010 13:47:28 -0600

> messages, but I can explain that in another message.)

I already understand your concerns.  But you still haven't answered my
questions about why the concerns of browsers apply to all user agents,
or anyone else implementing HTTP for any other purpose.  I don't see
the relevance, and the rationale for standardizing nonconformant syntax
still eludes me.

HTTP isn't meant as a guideline for building user agents.  Unlike HTML,
the concerns of browser vendors aren't of paramount importance; and in
this case, not even in-scope.  I doubt HTTPbis would get IETF approval
if it were based on dictating how user agents interpret nonconformant
syntax instead of referring to other RFCs where conformant syntax is
defined, justified by "IE and Chrome do it that way."

-Eric

---

From: "Eric J. Bowman" <[email-masked]>
Date: Sun, 3 Oct 2010 14:00:36 -0600

Considering this proposal elsewhere in HTTPbis:

http://lists.w3.org/Archives/Public/ietf-http-wg/2010JulSep/0388.html

By your logic, since user agents (browsers) would need to continue to
support X-Moz and X-Purpose, it would be incorrect to define the
Purpose header and leave it at that, without also providing some
language instructing user agents how to parse X-Moz and X-Purpose, as
they are in fact used in the wild.

I see describing how to parse the obsoleted headers as standardizing
nonconformant (unspecified) syntax.  Just because it was once done this
way, and adopted in more than one browser, isn't relevant to the
definition of a Purpose header.  Quite relevant to browser development,
not relevant to HTTP.

-Eric

---

From: Adam Barth <[email-masked]>
Date: Sun, 3 Oct 2010 13:03:02 -0700

User agents generate those headers.  They don't need instructions for
how to parse them.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 03 Oct 2010 22:18:49 +0200

>this message, I'll try to explain why we have the perspective we do.

You have an idea how to make the specification better. Submit convincing
evidence that the specification will indeed be better and I will most
certainly support your change proposal. So far you have neither made a
reasonable change proposal, nor have you submitted convincing evidence. 

I have explained to you what I would find a reasonable change proposal
that would seem to address your concern, and I have given you reasonably
objective criteria for what I would find good evidence to support such a
change proposal. There is no cultural gap here, everyone requires good
change proposals and good evidence to support them to make changes. So,
do submit evidence. Do not submit evidence and I will go by my own data,
which satify me that your objection is wrong. So, do submit evidence.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Sun, 3 Oct 2010 13:30:07 -0700

> reasonable change proposal, nor have you submitted convincing evidence.

Concretely, my proposal is that the specification should not forbid
user agents from %-decoding the value of the filename parameter.
Julian has agreed that neither Internet Explorer nor Chrome is going
to stop %-decoding the filename parameter anytime soon.  Forbidding
user agent from processing these messages in this way is fiction.

Now, I'm fine with forbidding servers from generating %-encoded
values.  In fact, I believe that would be desirable.  However, just
because we forbid servers from generating the values does not mean
that we must also forbid user agents from consuming them.

> which satify me that your objection is wrong. So, do submit evidence.

The problem is your change proposal focuses on uncovering the set of
messages with the same meaning in all semantics.  While that is of
interest to servers (generators), that's not of particular interest to
user agent implementors, as explained before.  Instead, user agent
implementors are interested in the largest set of messages generated
by at least one server that can be interpreted with a single semantic
theory.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Sun, 03 Oct 2010 22:47:33 +0200

On 03.10.2010 22:30, Adam Barth wrote:
> ...
> ...

Implementing %-unescaping here is a bug.

I agree it's unlikely to disappear from IE and Chrome anytime soon, so a 
warning is probably a good idea.

I do not believe we can do more than that, because

(a) the 4 other UAs tested do the right thing, and

(b) there's no other way to deliberately send filenames containing these 
character sequences.

> that we must also forbid user agents from consuming them.

I'd be more than happy recommending something else, if there was a 
"something else" we can recommend.

> ...
> ...

I'm sure they are interested in that. But I bet they are also interested in

- unambiguous parsing,

- consistency,

- code reuse (between header fields),

- and compatibility with other UAs.

C-D is a mess, we can all agree on this. RFC 5987 and this ID are an 
attempt to make things better. My main goal is to get to a situation 
where servers can emit C-D headers with filenames containing "arbitrary" 
characters from the Unicode character set (minus whitespace, control, 
path delims, whatnot) without having to do UA sniffing.

To get there, we need IE/Chrome/Safari to implement RFC2231/5987, or 
invent something new everybody is willing to accept. So far I haven't 
seen a serious alternative proposal.

Once everybody implements RFC 5987, broken support for "filename" just 
doesn't matter anymore, because you can always can use the new encoding 
("filename*"). But unfortunately we aren't there yet.

Best regards, Julian

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 03 Oct 2010 22:59:03 +0200

>user agent from processing these messages in this way is fiction.

The file name is advisory only. Implementations removing %-encoding may
be unfortunate, just like implementations stripping all but the first
eight [a-z] characters may be unfortunate, but they are not in violation
of the specification, as the file name is advisory only. That there may
be problems using % in file names is already noted in the draft. So, we
can move on with the draft as it is. I am glad we cleared this up.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Sun, 3 Oct 2010 14:03:05 -0700

> Implementing %-unescaping here is a bug.

That depends what you mean by "a bug."  One operative definition of a
bug is something the implementor wants to change.  You've gotten clear
feedback from one implementor that this bug is WONTFIX, which means
it's not a bug at all.

> (b) there's no other way to deliberately send filenames containing these

The problem with that is your forbidding user agents from implementing
a behavior they're refusing to remove.  That means the spec does not
reflect reality.

> "something else" we can recommend.

You don't have to define what user agents do in this case.  That's
fine.  I object to your requiring them to do something they aren't
going to.

> - and compatibility with other UAs.

Yes.  These are all desirable.

> C-D is a mess, we can all agree on this. RFC 5987 and this ID are an attempt
> to make things better. My main goal is to get to a situation where servers
> can emit C-D headers with filenames containing "arbitrary" characters from
> the Unicode character set (minus whitespace, control, path delims, whatnot)
> without having to do UA sniffing.

I fully support that.  I agree that this document is very useful to
server operators who wish to generate these headers.  However, the
document, as written, is over-broad in that it forbids user agents
from consuming some headers.

Please reduce the scope of the document to align it with your goals.

> To get there, we need IE/Chrome/Safari to implement RFC2231/5987, or invent
> something new everybody is willing to accept. So far I haven't seen a
> serious alternative proposal.
>

That might be true for folks generating the Content-Disposition
header, but folks consuming the header will still need to consume the
"broken" filename header for a long time.

Please open your mind to considering the constraints faced by browsers.

Adam

---

From: Adam Barth <[email-masked]>
Date: Sun, 3 Oct 2010 14:04:51 -0700

> can move on with the draft as it is. I am glad we cleared this up.

So, your position is that the document is meaningless then?  That's
silly.  Why bother introducing filename* then if we don't care how
user agents actually interpret the file name?

Adam

---

From: "Eric J. Bowman" <[email-masked]>
Date: Sun, 3 Oct 2010 15:28:45 -0600

Adam Barth wrote:
>
> > The file name is advisory only. Implementations removing %-encoding
> > may be unfortunate, just like implementations stripping all but the
> > first eight [a-z] characters may be unfortunate, but they are not
> > in violation of the specification, as the file name is advisory
> > only. That there may be problems using % in file names is already
> > noted in the draft. So, we can move on with the draft as it is. I
> > am glad we cleared this up.
> 

Well, that's your position.  I'd say that C-D recognizes the reality
that the sender can't definitively specify a filename, because only the
user agent knows the constraints of its underlying filesystem.  All the
sender can do is conform to MIME syntax, and all the user agent needs
to do is understand MIME syntax.  It would be folly to attempt to make
filename anything but advisory.

-Eric

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 04 Oct 2010 00:35:40 +0200

>user agents actually interpret the file name?

This a specification intended for people who wish to deliver better user
experience. User agent developers who are not interested in delivering a
good user experience will not care if they conform to this specification
or not. The file name cannot be anything but advisory due to varying
constraints, such as limitations of file systems and security consider-
ations.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Oct 2010 09:07:39 +0200

> fine.  I object to your requiring them to do something they aren't

You misunderstood.

*If* we tell producers that they can't send literal % escapes without 
the risk of them being misinterpreted, then it would be good if we could 
suggest an alternative.

Right now there is none, because the clients that have the broken 
percent-handling also do not implement RFC2231/5987.

Best regards, Julian