Subject: draft-ietf-6man-uri-zoneid-02.txt

From: Stuart Cheshire <[email-masked]>
Date: Mon, 16 Jul 2012 17:34:47 -0700

I'm glad we're having this discussion about zone identifiers in IPv6  
address literals, but I'm very disturbed by the direction the  
discussion seems to be going.

To put it simply, today we don't have a problem, but publishing this  
document as it currently stands would be creating one.

I'll explain what I mean by that:

Escaping (e.g. "%" <-> "%25") or some other transformation (e.g. "%"  
<-> "-") is needed *only* when required to allow unambiguous parsing:  
when a delimiter character is also allowed as a data character, then  
escaping or some other data transformation is necessary to  
disambiguate data from delimiters.

Using escaping when it's *not* necessary is a bad idea, because  
escaping is not free. Escaping has a cost, in terms of CPU cycles, in  
terms of increased data size, and -- most importantly -- in terms of  
cognitive cost. When data has different forms in different contexts,  
users and developers have a hard time remembering what goes where.  
Attached are a couple of examples from my daily life. What is "Magic% 
20Bell"? What is "Gabbie&#39;s home"? In both cases here, internal  
data escaping has unintentionally leaked through into the user  
interface.

This cognitive cost can also have a security cost, as draft-ietf-6man- 
uri-zoneid-02.txt points out, when two strings that are textually  
different have the same meaning.

Since escaping has all these costs, escaping should only be used  
where it's actually needed.

And here, in IPv6 addresses in URIs, escaping is *not* actually  
needed to allow unambiguous parsing. Since it's not needed, why  
inflict this pain on ourselves?

The argument seems not to be one reasoning from computer science  
first principles (since there is in fact no need for escaping to  
allow unambiguous parsing) but instead one reasoning from what is  
sometimes called "RFC lawyering".

The document draft-ietf-6man-uri-zoneid-02.txt says as much:

    Some versions of some browsers accept the RFC 4007 syntax for scoped
    IPv6 addresses embedded in URIs, i.e., they have been coded to
    interpret the "%" sign according to RFC 4007 instead of RFC 3986.
    Clearly this approach is very convenient for users, although it
    formally breaches the syntax rules of RFC 3986.

The document concedes that using IPv6 addresses with a "%" zone  
identifier in URIs does actually work, and "is very convenient for  
users", and the only reason not to use it is that it "formally  
breaches the syntax rules of RFC 3986". We should fix RFC 3986 to  
match reality, rather than insisting that reality be constrained by a  
minor mistake in RFC 3986.

In fact, careful reading of RFC 3986 makes it far from clear that  
using IPv6 addresses with a "%" zone identifier would in fact breach  
the syntax rules. RFC 3986 states that percent-encoding should be  
done on a component-by-component basis, and for each component of a  
URI, escaping is done *only* as necessary for that specific  
component. See excerpts from RFC 3986 below. Emphasis (in caps) added  
by me:

1. The percent-encoding rules are per component, not global for the  
URI as a whole:

    A percent-encoding mechanism is used to represent a data octet
    IN A COMPONENT when that octet's corresponding character is  
outside the
    allowed set or is being used as a delimiter of, or within, THE  
COMPONENT.

2. Characters should not be percent-encoded where they are  
specifically allowed by the component in question:

    URI producing applications should percent-encode data octets that
    correspond to characters in the reserved set UNLESS THESE  
CHARACTERS ARE
    SPECIFICALLY ALLOWED BY THE URI SCHEME TO REPRESENT DATA IN THAT  
COMPONENT.

IPv6 literals *do* specifically allow percent signs, and so by this  
reading of RFC 3986 percent signs do not need to be (and should not  
be) escaped.

The document draft-ietf-6man-uri-zoneid-02.txt even concedes that  
this is the most user-friendly approach:

    The authors believe it is feasible, and very convenient for  
users, if
    browsers also allow (in addition to the formal URI syntax defined in
    this document) a syntax that will enable cut and paste.  For  
example:

      http://[fe80::a%en1]

    It seems that modern browsers can be adapted to parse this  
because it
    is inside of the "[" "]"'s.  This would permit the output of  
commands
    like ping6 -w ff02::1%en1 to be "cut and pasted" into a browser
    address bar.  Consequently this document recommends that browsers
    support this syntax in addition to the formal URI syntax defined  
above.

If we are advocating that web browsers accept this syntax (and that  
is indeed what I am myself advocating internally at Apple) then what  
benefit do we gain by specifying that one syntax is acceptable in  
some places and a different syntax is acceptable in other places?

Let's just publish a document which states that the currently- 
accepted and widely-used IPv6 "%zone" notation is also fine for use  
in URIs. This document would also in effect by clarifying RFC 3986 to  
state that "%" characters *only* need to be escaped in those URI  
components that actually use "%xx" escaping, and MUST NOT be escaped  
in URI components like IP-literals that are specified to not use "% 
xx" escaping.

That clears up the ambiguity in RFC 3986, and lets people continue  
using the IPv6 "%zone" notation they already understand.

Stuart Cheshire

---

From: Stuart Cheshire <[email-masked]>
Date: Wed, 18 Jul 2012 15:52:13 -0700

On 16 Jul, 2012, at 20:50, Mark Andrews wrote:

> Stuart,
> 	your mail client botched the Content-type line generation.
> You may want to report it.
>
> 	Content-type: image/png; x-unix-mode=0644; name=Whatis&#39;  
> "?.png"=""
> 	Content-transfer-encoding: base64
> 	Content-disposition: inline; filename="What is &#39; ?.png"
>
> Mark

Mark, your tone sounds very confident that you're absolutely certain  
that you know exactly what botched what, and whose fault it is.

I'll reserve judgement until I actually know what happened, but what  
I can tell you is this: Viewing the outgoing TCP packets with  
tcpflow, this is what my mail client sends on-the-wire to the SMTP  
relay:

Content-Transfer-Encoding: base64
Content-Type: image/png;
	x-unix-mode=0644;
	name=What is &#39; ?.png
Content-Disposition: inline;
	filename="What is &#39; ?.png"

By the time you received the email, Mark, it had been rewritten to  
the form you showed. As to what intermediary (or intermediaries)  
contributed to that rewriting, I do not yet know.

It's ironic that this problem occurs in the midst of a discussion of  
the problems of escaping and message framing. The reality seems to be  
that unless we keep things supremely simple, we can't hope to have  
all programmers get it right in all cases. If there's exactly one  
valid form for a string, then maybe we can hope to have that  
implemented properly. When there are different representations of the  
same string in different contexts, the probability of everyone  
getting it right in all contexts pretty much approaches zero.

Slightly off-topic, I'm told that at least some mail clients  
truncated my original email at the line "unintentionally leaked  
through into the user interface."

As composed on my Mac, there was some introductory text, then two  
images, then the bulk of the text, as it appears in the archive:

<http://www.ietf.org/mail-archive/web/ipv6/current/msg16128.html>

Apparently some mail clients turned the second chunk of body text  
into an attachment.

I'm curious as to how widespread this issue is -- I might have to be  
more careful about where I put images in my email messages in the  
future.

Could people send me a quick private email saying what mail client  
they use and whether it:

1. Showed the entire message as I composed it with the two images  
displayed in-line (like the archive).
2. Showed the entire text of the message, but with the two images as  
attachments (Gmail shows it this way).
3. Showed only the first five paragraphs of text, with the two images  
and remaining text as attachments.

I'll summarize results to the list.

Stuart Cheshire

---

From: Mark Andrews <[email-masked]>
Date: Thu, 19 Jul 2012 10:42:52 +1000

In message <[email-masked]>, Stuart Cheshire wr
> 	filename="What is &#39; ?.png"

Which isn't a syntactically valid Content-Type header (RFC 1341).
 
> contributed to that rewriting, I do not yet know.

It may have been re-written but Garbage In - Garbage Out.
 
> getting it right in all contexts pretty much approaches zero.

It was slightly ironic.

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]