Subject: Reviving HTTP Header Linking: Some code and use-cases

From: Harry Halpin <[email-masked]>
Date: Sun, 09 Mar 2008 14:54:06 -0400

Hi, I'm Harry Halpin, and I'm working on GRDDL (Gleaning Resource
Descriptions from Dialects of Language) with the W3C, a rather neat
technology that helps us get uniform metadata from diverse XML, (X)HTML,
microformats, and (hopefully soon) HTML documents using a simple mix of
widely deployed technologies.

I'd just like to send a e-mail to see if we can revive Mark Nottingham's
"HTTP Header Linking Draft", from which we in particular would like to
see the Link and Profile headers revived [1]. What's the current status
of this idea?

We'd like to see this as an IETF-approved part of HTTP. Reviving
Nottingham's draft would solve a big problem for us. In order for
authors of documents on the Web to tell agents that there is a transform
to get data out of their data, they need a link to the transform and the
ability for the agent to tell that link is actually not just an ordinary
link, but the link of a GRDDL transform. This is absolutely needed for
the cases where the document owner may want to authorize to the
transformation, but may not want to add the information to the header of
*every* document in the collection or have access to the document. A
use-case given by Ian Davis of Talis has been written in more detail [2].

We also have running code that implements this feature in the Jena Java
implementation of GRDDL [3]. It's a simple feature for implementers to
implement, and if even more people gave head's up to this feature I
believe there would be many more implementers who would add this
feature. thanks, harry

[1]http://www.mnot.net/drafts/draft-nottingham-http-link-header-00.txt
[2]http://www.w3.org/TR/grddl-scenarios/#header_use_case
[3]http://jena.sourceforge.net/grddl/

-- 
		-harry

Harry Halpin,  University of Edinburgh 
http://www.ibiblio.org/hhalpin 6B522426

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 10 Mar 2008 23:35:53 +1100

Hi Harry,

I've actually just started working on a new revision of the draft in  
the last week; some side discussions at the APPS area arch workshop  
led me to believe that we may be able to make some progress on this.

I'll hopefully have something submitted in the next few days (once  
submissions are open again).

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 10 Mar 2008 23:09:13 +0100

> widely deployed technologies.

Looking forward to the HTML binding :-)

> see the Link and Profile headers revived [1]. What's the current status

We were sort of stuck with the naming of link relations; is this a 
single space for HTML link elements, Atom link elements, and HTTP Link 
headers? If yes, what's the syntax, and where does the registry go?

I think the last discussion is around 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2007OctDec/thread.html#msg46>.

> use-case given by Ian Davis of Talis has been written in more detail [2].

I totally agree that it would be good to make progress, but I'm not sure 
it's required for your work to proceed.

After all, the Link header *is* defined by RFC2068, so if you stick to 
that definition and do stick to a simple link relation name, I wouldn't 
expect any problems.

Q: do you really need "Profile"?

> ...

BR, Julian

---

From: Harry Halpin <[email-masked]>
Date: Mon, 10 Mar 2008 18:52:12 -0400

> Looking forward to the HTML binding :-)
Ah, that hasn't yet been truly decided  yet either. I think the same
people and code that want URIs in HTTP have also sent a few messages to
HTML 5 saying we'd like something like that kept.
> headers? If yes, what's the syntax, and where does the registry go?
I think Profile would answer that, by allowing the Link's to be typed on
the HTTP level as regards "naming link relations."

I tend to think of all the link relations being one single space, with
serializations in HTML/Atom/HTTP Link. I believe Mark's last draft says
re Profile "The Profile field is semantically equivalent to the profile
attribute of the HEAD element in HTML [W3C.REC-html401-19991224].  Note,
however, that its use is not limited to HTML entities" and for Link "
The Link field is semantically equivalent to the <LINK> element in
HTML." Is there any argument why this should not be so?

It would be good to have a centralized registry of well-known Link
types, with URIs to go along. But it's not necessary I think, and
certainly not mandatory. W3.org space could easily be used for this. In
general, the mechanism is totally decentralized.
> <http://lists.w3.org/Archives/Public/ietf-http-wg/2007OctDec/thread.html#msg46>.
The use of "rel" values should be considered, but are orthogonal to this
discussion at this moment, since we need a HTTP method.
> I totally agree that it would be good to make progress, but I'm not
> sure it's required for your work to proceed.
> that definition and do stick to a simple link relation name, I
> wouldn't expect any problems.
I think it's the combo of Link and Profile that would make us truly
happy, and it's what our code uses. Profile is not in that RFC.
> Q: do you really need "Profile"?
Profile is in general needed to type the Links. Otherwise, it might not
be a GRDDL Link, it could be a Link to anything. Does that make sense?

-- 
		-harry

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 Mar 2008 09:02:44 +0100

> the HTTP level as regards "naming link relations."

Are we talking about the link relation name? Would it be problematic to 
just use "transformation"? (assuming there would be a registry)

> HTML." Is there any argument why this should not be so?

No, I think this is fine.

The problem I see is that the definition of the Profile attribute in 
HTML 4 is sort of weak. It allows multiple profiles, but if you do 
multiple profiles (such as for multiple microformats), it can't be used 
for disambiguation (ha! missing prefixes :-).

So when defining new link relations, I think it's wise to do so in a way 
that doesn't rely on profiles.

> general, the mechanism is totally decentralized.

Yes.

> discussion at this moment, since we need a HTTP method.

...header.

> be a GRDDL Link, it could be a Link to anything. Does that make sense?

Maybe I'm missing something, but 
<http://www.w3.org/TR/2007/WD-grddl-20070302/#grddlvocab> seems to 
indicate that there is a single link relation -- "transform". Are there 
other kinds of GRDDL link relations?

And if there are, exactly what problem does "Profile" use for you? Can 
you provide an example?

BR, Julian

PS: I'm really trying to understand whether Profile is going to help here.

---

From: Harry Halpin <[email-masked]>
Date: Tue, 11 Mar 2008 07:09:07 -0400

Julian Reschke wrote:
> Are we talking about the link relation name? Would it be problematic
> to just use "transformation"? (assuming there would be a registry)
Yep, naming link relations via profile. That would work, but we would
prefer to just keep using URIs in Profile and Link. It's more
decentralized and allows an application to "follow-our nose" to find out
more info about GRDDL in general and the particular GRDDL transform in
particular. 

Right now we use "http://www.w3.org/2003/g/data-view" as our profile URI
both in our HTML and in the code that implements the HTTP Header
test-case (which is non-normative due to dependency on this Profile).
Then for our Link we have a link to the URI with the transform (i.e.
usually a XSLT).
> multiple profiles (such as for multiple microformats), it can't be
> used for disambiguation (ha! missing prefixes :-).
>
> So when defining new link relations, I think it's wise to do so in a
> way that doesn't rely on profiles.
That's a very good point. What our code currently does is, if it sees a
single GRDDL Profile header, then it assumes all the Links might be to
GRDDL transforms. Ditto with multiple Profiles. If it does not see a
GRDDL Profile header, we don't follow the Link header URIs.  While it's
not exact disambiguation (which would be better, but I cannot see how
that would be accomplished without demanding a 1:1 relationship between
Profiles and Headers, which would disturb some other use-cases), it
gives one a smaller space of possible URIs to "follow-our-nose", and
does disambiguate if only one Profile URI or one Link URI is given.

In essence, a GRDDL Profile URI is about the best message one can give
that the author intends the page to be GRDDL'ed. A Link with a GRDDL
transform is also by itself a pretty good message, but not great, since
the author may just be using something like microformats but does not
want their page to have a RDF version. In our spec, we call it author
"licensing" the GRDDL transform.
e say it is possible for non-licensed transforms to be followed, but
they may not preserve the meaning that the author of the document
intends and thus are inherently not safe.

[snip]
> indicate that there is a single link relation -- "transform". Are
> there other kinds of GRDDL link relations?
We use a single URI, "http://www.w3.org/2003/g/data-view" as our profile.
> And if there are, exactly what problem does "Profile" use for you? Can
Since Link is already approved in an earlier RFC and could be used for
all sorts of things (i.e. currently there is large discussion of using
it to distinguish between different types of SemWeb resources in
www-tag, etc.), we don't "follow the link" looking for a GRDDL unless it
has the GRDDL Profile URI specified in the HTTP Header.
> PS: I'm really trying to understand whether Profile is going to help
Thanks for your time and patience. I'm just trying to describe what we
currently do. We can change if it necessary, but we thought HTTP folks
would at least want to know. We thought Profile and Link were a good
combo together.

-- 
		-harry

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 Mar 2008 12:22:32 +0100

> intends and thus are inherently not safe.

Hm.

So as far as I understand you have two sorts of GRDDL transforms, a 
"licensed" one, and another one.

Why don't you just define two separate link relations then, avoiding the 
Profile stuff?

> would at least want to know. We thought Profile and Link were a good

Of course this kind of input is extremely useful, in particular given 
the fact that the WHATWG crowds has removed profile (which I think is a 
problem).

So, let's rephrase this: if a link relation could be a URI (or IRI), 
would we need Profile?

BR, Julian

---

From: Harry Halpin <[email-masked]>
Date: Tue, 11 Mar 2008 07:42:02 -0400

Julian Reschke wrote:
> Why don't you just define two separate link relations then, avoiding
> the Profile stuff?
The links are to the GRDDL transform URIs themselves. And a non-licensed
GRDDL transformation is not technically a GRDDL transformation. That's
why we use Profile in our definition.

> the fact that the WHATWG crowds has removed profile (which I think is
> a problem).
>
Explain precisely how this would work. I'm not sure what document I
should be looking at to explain "link relations" to me. If there's
another header that can do it, that might work.

I can see how there might be conflict if HTML drops Profile and HTTP
re-instates it. It would be better for it to be either re-instated
uniformly or not. But I think Profile does have a pretty good URI-based
extensibility case going for it.

-- 
		-harry

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 Mar 2008 12:59:17 +0100

> why we use Profile in our definition.

Understood. But wouldn't two different link relations have the same 
semantics?

> another header that can do it, that might work.

No, I meant the link header...:

Link: <http://example.com/grddl.xslt>;
   rel="http://www.w3.org/2003/g/data-view/transform"

> extensibility case going for it.

Yes, I totally agree that URI based extensibility is good. I'm just not 
sure it's good enough with Profile, given the fact it doesn't solve the 
disambiguation problem.

BR, Julian

---

From: Harry Halpin <[email-masked]>
Date: Tue, 11 Mar 2008 10:05:33 -0400

> Understood. But wouldn't two different link relations have the same
Operational semantics, yes. But one would be authorized by the creator
of the document while the other's wouldn't. I think that's a valid
distinction. One could "link" to page containing a GRDDL transform and
not want it run, because you do not trust it, it is still experimental,
etc.
>   rel="http://www.w3.org/2003/g/data-view/transform"
Yep, that would be another way to do it, but only *if* rel allows URIs.
I thought it did not. Am I wrong?

But if there is, still think there's an argument for profile, see below.

> Yes, I totally agree that URI based extensibility is good. I'm just
> not sure it's good enough with Profile, given the fact it doesn't
> solve the disambiguation problem.
What profile does is provide a meta-data profile for the resource of the
URI itself, not just the links, although it can be used to disambiguate
the links (albeit imperfectly if lists of profiles and links are used).
In this manner, it's still very useful. For example, it's usage in
microformats in HTML allows microformats to have GRDDL transformations
without a centralized repository. The same case can be made for HTTP.
The only alternative is to "stick everything" in a centralized registry.

So, we could imagine a case where someone wants to use a profile URI for
their microformat-enabled page, and we want GRDDL to go to that profile
and get the GRDDL transform. I guess one could try to do this.

Link: <http://example.com/mymicroprofile>;
  rel="http://example.com/itsaprofile"

But it seems that Profile would work too:

Profile: <http://example.com/mymicroprofile>;

Again, I think I've already made a request for Link to be re-instated.
If Link is re-instated, then we'd also like URIs to be allowed in rel
fields. I think there's no argument here.

We'd also like Profile to be kept, but could probably live if an
alternate mechanism was approved, i.e. URIs in rel fields.

However, this would break existing software and so we'd have to change
it. I think the same would go for the microformat community if HTML 5
got rid of profiles.

-- 
		-harry

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 Mar 2008 15:30:54 +0100

> not want it run, because you do not trust it, it is still experimental,

Yes, but you could define the two link relations to have exactly that 
semantics, couldn't you?

> I thought it did not. Am I wrong?

That's up for discussion, I think.

> The only alternative is to "stick everything" in a centralized registry.

Profile does *help* with microformats name clashes, but it doesn't solve 
them; if you have class="title" in two different microformats, the 
profile attribute won't help you disambiguate.

On the other hand, for link relations we *may* have the option to ground 
them in URI space, so the problem simply would go away.

> alternate mechanism was approved, i.e. URIs in rel fields.

OK, thanks for clarifying.

> However, this would break existing software and so we'd have to change

Which?

> it. I think the same would go for the microformat community if HTML 5

It's (imho) just a related problem that doesn't necessarily need the 
same solution.

BR, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 11 Mar 2008 10:50:59 -0700

> I thought it did not. Am I wrong?

It does not currently allow URIs.  The link relationship space is  
supposed
to be small, universal, and standardized (just like HTTP methods).   
That is
why it is better not to allow an infinite number of relation types,  
but instead
define a few types that allows an infinite number of relations to be  
understood.
However, RDF pretty much screwed us all on that one, so the  
reasonable next
step is to allow URIs and have all flat names be relative to the same
link relationship registry as Atom.

Note, however, that the following performs the same job as Profile  
without
new protocol and without excessive bytes on the wire:

   Link: <http://www.w3.org/2003/g/data-view>; rel="profile"
   Link: <http://example.com/mymicroprofile>;  rel="profile"
   Link: <http://example.com/grddl.xslt>; rel="grddl-rdf"

Yes, there is some tiny possibility that two different profiles will
use the same relationship name for entirely different purposes.
I don't care.  If they can't be distinguished by context, then they
shouldn't be used together.  Profiles are a crutch.

BTW, it is my opinion that both Link and PATCH should be restored to
the main HTTP specs.  HTTP is incomplete without them.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Wed, 12 Mar 2008 13:20:42 +0100

Roy T. Fielding wrote:
> ...
> However, RDF pretty much screwed us all on that one, so the reasonable next

+1.

As IANA currently maintains this as "Atom Link Relations", would we need 
to update that registry somehow?

> Note, however, that the following performs the same job as Profile without

I like that idea; however I'm not totally sure that this is within our 
charter...

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 06:03:47 -0700

Roy T. Fielding wrote:
> reasonable next step is to allow URIs and have all flat names 
> be relative to the same link relationship registry as Atom.

HTTP cannot share the same link registry as Atom unless the Atom link
registry is completely redone. The whole registry is specific to Atom or
feed processing.

Furthermore, the Atom mechanism for registration means that any
registered link relation has two names: "xxx" and
"http://www.iana.org/assignments/relation/xxx". This has made processing
links in Atom feeds unnecessarily tedious. It would be better to come up
with a new mechanism that either required everything to be an IRI or at
least didn't allow registered link relations to be used with the IRI
form.

The Atom mechanism does comparisons character-for-character. An IRI and
its URI equivalent do not match. That means that RFC 3987 IRI-URI
conversion cannot be used for the Link header; instead, something like
percent-encoded Unicode would be needed.

The "title" subfield is also problematic. It must be properly
internationalized, including proper support for Ruby annotations and
BIDI text. If that can happen, then I would like to see a "Title:"
header field too, so that I can HEAD a document to get its title. 

- Brian

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 06:10:10 -0700

Harry Halpin wrote:
> We'd like to see this as an IETF-approved part of HTTP. 
> Reviving Nottingham's draft would solve a big problem for us. 
> In order for authors of documents on the Web to tell agents 
> that there is a transform to get data out of their data, they 
> need a link to the transform and the ability for the agent to 
> tell that link is actually not just an ordinary link, but the 
> link of a GRDDL transform. This is absolutely needed for the 
> cases where the document owner may want to authorize to the 
> *every* document in the collection or have access to the 
> document. A use-case given by Ian Davis of Talis has been 
> written in more detail [2].

Why don't you just create a new HTTP header and register it? The effort
to register a new HTTP header would be the same as the effort to
register a new link relation for the "Link:" header. Plus, you wouldn't
have to wait for "Link:" to become standardized.

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 12 Mar 2008 14:35:15 +0100

> registry is completely redone. The whole registry is specific to Atom or

How so?

> links in Atom feeds unnecessarily tedious. It would be better to come up

I wouldn't call it two names, but just two notations. Before comparing, 
resolve all references against 
""http://www.iana.org/assignments/relation/" and everything should be fine.

> least didn't allow registered link relations to be used with the IRI

IRIs are indeed a problem if we want to use them inside an HTTP header.

> percent-encoded Unicode would be needed.

Sounds like a good reason for not allowing link relations that aren't 
URIs (or URI references).

> header field too, so that I can HEAD a document to get its title. 

Is there a problem with the title parameter (of the link header), except 
for the ugly rules for non ISO-Latin1 characters?

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 07:28:58 -0700

Julian Reschke wrote:
> > HTTP cannot share the same link registry as Atom unless the 
> > Atom link registry is completely redone. The whole registry is 
> > specific to Atom or feed processing.
> 
> How so?

Just as an example, look at "edit". It is defined to always link to an
Atom entry. All the link currently-defined relations are defined by Atom
RFCs. 

> > "http://www.iana.org/assignments/relation/xxx". This has made 
> > processing links in Atom feeds unnecessarily tedious. It would be 
> > better to come up
> 
> I wouldn't call it two names, but just two notations. Before 
> comparing, resolve all references against 

You cannot resolve the references using URI resolution.
"../relation/next" is not the same as "next". You have to compare
everything character-for-character. 

> > The Atom mechanism does comparisons character-for-character. An IRI 
> > and its URI equivalent do not match. That means that RFC 
> > 3987 IRI-URI conversion cannot be used for the Link header; instead,

> > something like percent-encoded Unicode would be needed.
> 
> Sounds like a good reason for not allowing link relations 
> that aren't URIs (or URI references).

That is against IETF policy. New standards have to allow the use of IRIs
wherever URIs are allowed. At least, that is what I was told on the Atom
mailing list. While I have read RFC 2277, I'm not an expert on IETF's
internationalization policy. However, I personally believe it is wrong
to create new standards where things may be named in European languages
but not in non-European languages.

> Is there a problem with the title parameter (of the link 
> header), except for the ugly rules for non ISO-Latin1 characters?

It is just like putting localizable text in an XML attribute. See
http://www.w3.org/TR/2008/NOTE-xml-i18n-bp-20080213/. I'm not sure the
title subfield is worth fussing over too much.

I also think that the Link header is only useful for unregistered link
relations. Any time we would register a link relation, couldn't we just
register a HTTP header with the same name?

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 12 Mar 2008 15:43:54 +0100

So making this a non-specific Atom registry probably requires making 
these relations have generic semantics.

> everything character-for-character. 

RFC 4287 says that relative references 
(<http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.2.7.2>) 
must be simple names:

"Note that use of a relative reference other than a simple name is not 
allowed."

It continues specifying how to compare relations:

"If a name is given, implementations MUST consider the link relation 
type equivalent to the same name registered within the IANA Registry of 
Link Relations (Section 7), and thus to the IRI that would be obtained 
by appending the value of the rel attribute to the string 
"http://www.iana.org/assignments/relation/"."

So it seems to me that this *is* URI resolution.

> mailing list. While I have read RFC 2277, I'm not an expert on IETF's

I don't think this is true.

Also, we're not really talking about something new here; what's being 
discussed is restoring what RFC 2068 already said about "Link:".

> but not in non-European languages.

If this would be a new standard, I would agree.

> register a HTTP header with the same name?

Interesting thought :-)

BR, Julian

---

From: Harry Halpin <[email-masked]>
Date: Wed, 12 Mar 2008 10:46:32 -0400

> have to wait for "Link:" to become standardized.
Link+rel with URIs Headers and Link+Profile Headers are a decentralized
solution to this problem (i.e. would not require a new link relation to
be standardized at all, and in the future). We'd rather have one good
decentralized way of using the Link header be part of HTTP than have to
register a new HTTP header for every technology :)

Perhaps that is just my preference for URI-based extensibility and
decentralization. However, I do think it makes more sense long-term.

-- 
		-harry

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 08:06:59 -0700

Harry Halpin wrote:
> Brian Smith wrote:
> > Why don't you just create a new HTTP header and register it?
> > The effort to register a new HTTP header would be the same
> > as the effort to register a new link relation for the
> > "Link:" header. Plus, you wouldn't have to wait for "Link:"
> > to become standardized.

> Link+rel with URIs Headers and Link+Profile Headers are a 
> decentralized solution to this problem (i.e. would not
> require a new link relation to be standardized at all,
> and in the future). We'd rather have one good decentralized
> way of using the Link header be part of HTTP than have to
> Perhaps that is just my preference for URI-based 
> extensibility and decentralization. However, I do think it 
> makes more sense long-term.

URI-based extensibility for HTTP is RFC 2774
(http://www.ietf.org/rfc/rfc2774.txt):

Opt: "http://example.org/foo"; ns=00
00-My-Custom-Link-Header: http://example.org/bar
00-Another-Custom-Link-Header: http://example.org/baz
Opt: "http://example.com"; ns=01
01-My-Custom-Link-Header: http://example.com/something
01-Yet-Another-Header: http://example.com/something

The RFC 2774 mechanism isn't restricted to hyperlinks, either.

Regards,
Brian

---

From: Harry Halpin <[email-masked]>
Date: Wed, 12 Mar 2008 11:18:25 -0400

> The RFC 2774 mechanism isn't restricted to hyperlinks, either.
That's one experimental way to do it, but again - registering custom
link headers in general I think are a *still centralized* solution in
comparison with a single Link header with extensibility built in. I do
not want to register a custom link header every time I want to have an
application use a link header in a way that needs URI extensibility to
determine its type.

If I use "Opt" do I have to register my link header names, or is it a
free-for-all?

-- 
		-harry

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 08:33:08 -0700

Harry Halpin wrote:
> That's one experimental way to do it, but again - registering 
> custom link headers in general I think are a *still 
> centralized* solution in comparison with a single Link header 
> with extensibility built in. I do not want to register a 
> custom link header every time I want to have an application 
> use a link header in a way that needs URI extensibility to 
> determine its type.
> 
> If I use "Opt" do I have to register my link header names, or 
> is it a free-for-all?

The point of the RFC 2774 mechanism is to allow the use of unregistered
headers without worrying about naming conflicts. It works just like XML
namespaces:

"http://example.org/foo"; ns=00 <=> xmlns:x00='http://example.org/foo'

"http://example.com"; ns=01 <=> xmlns:x01='http://example.com'

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 12 Mar 2008 16:43:42 +0100

> "http://example.com"; ns=01 <=> xmlns:x01='http://example.com'

RFC 2774 has several problems, so in the end it didn't get past 
"experimental".

I personally have used RFC2774 extensions in a very controlled 
environment, but in order to use it on the public web (for instance for 
GRDDL), it would need much more support.

BR, Julian

---

From: Harry Halpin <[email-masked]>
Date: Wed, 12 Mar 2008 12:45:13 -0400

> link relationship registry as Atom.
+1

As regards the notion of putting URIs in rel values, are the
intersection of SGML names and URIs is an empty sub-set? It seems SGML
names don't allow ":", so there's probably no overlap...so, is there a
reason *not* to allow URIs in rel?

Or should we invent another parameter, say "ler" that takes URIs and let
rel keep only SGML names?

Another option might be to say that rel always takes URIs, but if just a
sgml-name is given, it's just a short-hand for the "URI of Registred
Links" + "sgml-name"

    Just wondering...

          harry

-- 
		-harry

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 15:32:59 -0700

Julian Reschke wrote:
> Brian Smith wrote:
> > Julian Reschke wrote:
> >>> HTTP cannot share the same link registry as Atom unless the Atom 
> >>> link registry is completely redone. The whole registry is 
> specific 
> > Atom entry. All the link currently-defined relations are defined by 
> > Atom RFCs.
> 
> So making this a non-specific Atom registry probably requires 
> making these relations have generic semantics.

Some (most?) of them already have pretty generic semantics. But, others
don't, and if you try to make them generic, then they will stop working
for AtomPub. AtomPub requires "edit" to link to an Atom Entry, for
example.

> "Note that use of a relative reference other than a simple 
> name is not allowed."

<snip>

> So it seems to me that this *is* URI resolution.

Right, but only after you've verified that the document is conformant to
RFC 4287. If you aren't sure that the link relation is conformant, then
you can't use IRI resolution to verify it. Also, you have to make sure
that whatever library you are using to do the IRI resolution actually
supports IRIs, and not just URIs.

> >>> 3987 IRI-URI conversion cannot be used for the Link 
> >>> header

> >> that aren't URIs (or URI references).

I think it would be better to only allow link relations where URI -> IRI
conversion is a no-op, and then specify that URI -> IRI conversion is
done when serializing the header, and URI -> IRI conversion must be done
to compare link relations.

> > That is against IETF policy. New standards have to allow the use of 
> > IRIs wherever URIs are allowed. At least, that is what I 
> > was told on the Atom mailing list. While I have read RFC 2277,
> > I'm not an expert on IETF's
> 
> Also, we're not really talking about something new here; 
> what's being discussed is restoring what RFC 2068 already 
> said about "Link:".
> 
> > to create new standards where things may be named in European 
> > languages but not in non-European languages.
> 

If the link header is going to be (re-)standardized, it has to be
specified in a new standard. It is out of scope for HTTPbis. And, RFC
2277 says all new IETF standards must support UTF-8, which implies that
all new IETF standards for URI processing must support IRIs.

- Brian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 12 Mar 2008 16:55:00 -0700

> for AtomPub. AtomPub requires "edit" to link to an Atom Entry, for

That doesn't matter.  If an AtomPub application is following
links in Atom content, it is reasonable to believe that such
a requirement will be true.  If an AtomPub application is
following edit links in random content, then it is acting
outside that protocol and will just ignore the "requirement".
The requirement is on AtomPub, not the meaning of rel="edit".

It is best to have one and only one registry, even if a few
relations have entirely different meanings in different contexts.
It is still valuable to know that those contexts exist.

> supports IRIs, and not just URIs.

No, you can do anything you like with the link relation.  There
are absolutely no universal requirements on the Internet -- they
all exist within the context of a given interoperation.  Outside
of that context, data is just data.  E.g., email addresses are
not just used for sending email.

> all new IETF standards for URI processing must support IRIs.

First, HTTP header fields and methods are a flat namespace.
Link and PATCH were defined in the original HTTP specifications
and only removed to to a lack of known implementations.  They
would only be "new" if their definitions were substantially
changed, which I don't see happening.  Removing Link from 2616
was a very bad editorial decision and I don't see how putting
it back in can be seen as anything other than an editorial
correction -- it is not as if we are changing the meaning of
HTTP in the process.

Second, there is no such requirement for IRIs -- URIs are fully
capable of describing all possible IRIs (by definition) and thus
do not prevent i18n.

....Roy

---

From: "Brian Smith" <[email-masked]>
Date: Wed, 12 Mar 2008 19:59:25 -0700

> It is still valuable to know that those contexts exist.

Still, nobody has justified why a registry of link relations is
necessary for the Link header. If you are going to register a link
relation, you are better off just making a new HTTP header that works
like all the other link headers in HTTP (e.g. Location,
Content-Location).

> > Right, but only after you've verified that the document is 
> > conformant to RFC 4287. If you aren't sure that the link
> > relation is conformant, then you can't use IRI resolution
> > to verify it. Also, you have to make sure that whatever
> > library you are using to do the IRI resolution actually
> No, you can do anything you like with the link relation.  
> There are absolutely no universal requirements on the 
> Internet -- they all exist within the context of a given 
> interoperation.  Outside of that context, data is just data.  
> E.g., email addresses are not just used for sending email.

There must be some miscommunication here. My point is that the Atom way
of having two forms of each registered link relation is more complicated
than necessary, offers no real benefits, and should be avoided. The
easiest way to avoid it is to just not have a registry at all. Barring
that, at least make the rules for parsing link relations simpler than
the Atom rules.

> > 2277 says all new IETF standards must support UTF-8, which implies 
> > that all new IETF standards for URI processing must support IRIs.
> 
> Link and PATCH were defined in the original HTTP 
> specifications and only removed to to a lack of known 
> implementations.  They would only be "new" if their 
> definitions were substantially changed, which I don't see 
> happening.  Removing Link from 2616 was a very bad editorial 
> decision and I don't see how putting it back in can be seen 
> as anything other than an editorial correction -- it is not 
> as if we are changing the meaning of HTTP in the process.

Even with that (very liberal) interpretation of the situation, these
features still cannot be added in HTTPbis, because the working group is
supposed to "remove or deprecate those features that are not widely
implemented." Saying LINK, UNLINK, PATCH, etc. are "not widely
implemented" would be putting it lightly. 

It looks like it will take a lot of time just to resolve the core issues
with RFC 2616, such as how content negotiation interacts with
conditional requests, clarifying "idempotency" (or removing references
to the concept if it is as unimportant as you say), making pipelining
usable, and all the other issues that are in the issue tracker. I'm
pretty sure there is no time to resurrect PATCH, LINK, UNLINK, Link:,
and other such functionality while staying on schedule. And, I don't see
how it is possible to get adequate implementation feedback on these (as
yet unimplemented) old features in time to make any necessary
corrections before October.

I am not against those features, I just think they could easily be
specified in separate RFCs so that HTTPbis can remain focused on the
core problems with the functionality that is defined by RFC 2616.

> Second, there is no such requirement for IRIs -- URIs are 
> fully capable of describing all possible IRIs (by definition) 
> and thus do not prevent i18n.

If URIs were adequate for internationalization, then IRIs would never
have been created. Anyway, I pointed out in a previous message how IRI
link relations can be easily supported in the Link header while using
Atom-like character-by-character comparison, by requiring IRI-to-URI
conversion when writing the header field, and requiring URI-to-IRI
conversion before comparing link relations to each other.

- Brian

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 13 Mar 2008 09:53:29 +0100

> That is against IETF policy. New standards have to allow
> the use of IRIs wherever URIs are allowed. At least, that
> is what I was told on the Atom mailing list.

Then this was IMO more than one error in a simple statement.  

New protocols are supposed to support minimally UTF-8 as per
RFC 2277.  That is not the same as "support IRIs", IRIs can
be in any charset, not only UTF-8.  You'd need to transform
IRIs to UTF-8 first if you want to figure out an equivalent
URI, and you'd need to transform an IDN <ihost> in an IRI
to an ordinary STD 66 <host> with A-labels in the URI, but
that is no "IETF policy", it is simply what RFC 3987 says.

HTTP is no "new" protocol, like mail or news:  2821bis and
2822upd and FWIW RFC.usefor-usefor don't "violate" any IETF
policy.  But atom and xmpp were new, a different situation.

HTTP at the moment allows Latin-1, do you really want to
support the proper subset of all IRIs limited to Latin-1,
for the purpose of HTTP Link: header fields ?  

Please note that RFC 3987 is a proposed standard, HTTP is a
draft standard.  You'd get a downref, and forcing servers
and clients worldwide to do the Unicode 3.2 punycode stunt
(until IDNAbis fixes it) is an interoperability nightmare.

When "keeping Latin-1" is a showstopper, then introducing
IRIs in 2616bis would be a clear "1F" scenario.  You need
a new HTTP version number for this, a restart at PS, and a
new WG Charter.

> I personally believe it is wrong to create new standards
> where things may be named in European languages but not
> in non-European languages.

Strong ACK, let's drop the Latin-1 cruft, and limit 2616bis
to US-ASCII and URIs for now.  HTTP/1.2 is free to tackle
UTF-8, and RFC 3987 offers a strict STD 66 URI for any IRI.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Thu, 13 Mar 2008 09:50:07 +0100

> necessary for the Link header. If you are going to register a link

To avoid relation name collisions?

> like all the other link headers in HTTP (e.g. Location,

Besides avoiding having to many HTTP headers?

One reason is that in (X)HTML, you will need a link relation to include 
the information into the document. So it seems to be a better design to 
have the same mechanism both at document and transport level, in 
particular when it already is specified that way in HTML 4.01 and the 
proposed standard of HTTP/1.1 (which is RFC 2068).

> that, at least make the rules for parsing link relations simpler than

I fail to see how this is a problem in practice:

- reject references that are not simple names
- then resolve against the base URI

> implemented" would be putting it lightly. 

Let's focus on "Link" (the header). I do agree that PATCH, LINK and 
UNLINK (the methods) aren't widely implemented.

"Link" is part of HTTP/1.1, just not as part of the latest draft 
standard. It was left out because of missing support, or because people 
weren't aware of it.

As far as I recall, several user agents now do support Link: to specify 
CSS, so this has changed.

> corrections before October.

Again, "Link:" is implemented.

I do agree that we should prioritize the things that are on our issues 
list, and I also agree that discussing other things won't probably with 
the schedule.

As a matter of fact, I'm concerned that we aren't making sufficient 
progress to have any chance to meet our deadlines.

That being said, people discuss those things they are interested in. If 
these things belong to HTTP/1.1, I'd rather have the discussions here 
then somewhere else.

> core problems with the functionality that is defined by RFC 2616.

Yes, but it's not 100% clear (to me) where to draw the line.

> conversion before comparing link relations to each other.

Understood, but not pretty at all :-)

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Thu, 13 Mar 2008 05:57:15 -0700

Julian Reschke wrote:
> Besides avoiding having too many HTTP headers?

That is a bogus argument. As Roy said earlier in the thread, "the link
relationship space is supposed to be small, universal, and standardized
(just like HTTP methods)." Besides, is somebody going to be able to able
to register a link relation with the same name as a registered HTTP
header, or vice versa? It would be nonsense to register a
"Content-Location" link relation. If somebody registers an "next" link
relation, it would be very confusing to then allow someone else to
register an "next" HTTP header with totally different semantics. So,
effectively HTTP link relations and HTTP headers share the same
namespace.

As far as I understand the LINK and UNLINK methods, they exist soley to
edit "Link:" header field values. But, why have a method that can only
edit one header field, when we could have a method that allows for the
editing of any header for a resource? Roy already said that PATCH is
supposed to have that ability. Once you have that method, the need for
LINK and UNLINK goes away, which also reduces the need for the Link
header.

> One reason is that in (X)HTML, you will need a link relation 
> to include the information into the document. So it seems to 
> be a better design to have the same mechanism both at 
> document and transport level, in particular when it already 
> is specified that way in HTML 4.01 and the proposed standard 
> of HTTP/1.1 (which is RFC 2068).

I would agree, except that Roy already said that the semantics for HTTP
links do not have to match the semantics defined by Atom (or HTML, I
presume) for any link relation. And, the syntax has to be different from
one of them, because Atom link relations are case-sensitive and HTML
link relations are case-insensitive. So, we would end up with something
where the syntax is similar-but-different and the semantics are
similar-but-different. So, it is not really "the same mechanism" at all.

> > There must be some miscommunication here. My point is that the Atom 
> > way of having two forms of each registered link relation is more 
> > complicated than necessary, offers no real benefits, and should be 
> > avoided. The easiest way to avoid it is to just not have a 
> > registry at all. Barring that, at least make the rules for parsing
> > link relations simpler than the Atom rules.
> 

Let's say you want to check for an "edit" link. It would be simpler to
only need to either check if that the relation is "edit" or check that
it is "http://www.iana.org/assignments/link-relations/edit", instead of
always having to check for both. Already, some (many? most?) Atom
implementations fail to understand the longer form, so it is already an
interoperability issue for Atom. 

> That being said, people discuss those things they are 
> interested in. If these things belong to HTTP/1.1, I'd rather 
> have the discussions here then somewhere else.
> 
> > I am not against those features, I just think they could
> > easily be specified in separate RFCs so that HTTPbis can
> > remain focused on the core problems with the
> > functionality that is defined by RFC 2616.
> 

Anything that is already in RFC 2616 should be fixed first, before
worrying about stuff that RFC 2616 removed from RFC 2068. 

> > Anyway, I pointed out in a previous message how IRI 
> > link relations can be easily supported in the Link header 
> > while using Atom-like character-by-character comparison,
> > by requiring IRI-to-URI conversion when writing the
> > header field, and requiring URI-to-IRI conversion
> > before comparing link relations to each other.
> 

Nothing regarding URI-IRI conversion is pretty. But, keep in mind that
it only would affect people who use link relations that where
IRI->URI->IRI conversion is not a no-op. Basically, it would not effect
users of European languages at all, but it would provide users of
non-European languages with a facility for using their own language if
they so desire.

Regards,
Brian

---

From: "Brian Smith" <[email-masked]>
Date: Thu, 13 Mar 2008 06:22:28 -0700

Frank Ellermann wrote:
> > was told on the Atom mailing list.

> be in any charset, not only UTF-8.

Right, but I understand it to mean that a new protocol that supports
URIs must support UTF-8 in URIs, and the only (proposed) standard for
UTF-8 in URIs is RFC 3987.

Also, instead of paraphrasing 2277, let's just directly quote the parts
that apply to HTTPbis:

"Lack of an ability to use UTF-8 is a violation of this policy; such a
violation would need a variance procedure ([BCP9] section 9) with clear
and solid justification in the protocol specification document before
being entered into or advanced upon the standards track."

"For existing protocols or protocols that move data from existing
datastores, support of other charsets, or even using a default other
than UTF-8, may be a requirement. This is acceptable, but UTF-8 support
MUST be possible."

"In documents that deal with internationalization issues at all, a
synopsis of the approaches chosen for internationalization SHOULD be
collected into a section called 'Internationalization considerations',
and placed next to the Security Considerations section."

> policy.  But atom and xmpp were new, a different situation.

RFC 2277 applies to any updates to an existing protocol, as far as I can
tell.

No, that is not what I was saying at all.

> draft standard.  You'd get a downref, and forcing servers and 
> clients worldwide to do the Unicode 3.2 punycode stunt (until 
> IDNAbis fixes it) is an interoperability nightmare.

You don't need to do the IDNA transformation for link relations, because
you are not resolving the hostname of the IRI of the link relation. 

> support the proper subset of all IRIs limited to Latin-1, for 
> the purpose of HTTP Link: header fields ?  

> IRIs in 2616bis would be a clear "1F" scenario.  You need a 
> new HTTP version number for this, a restart at PS, and a new 
> WG Charter.

I am not suggesting that HTTP 1.1 should switch from Latin-1 to UTF-8.
But, I do think that HTTPbis should explain how IRIs are to be used
correctly in HTTP (via URI-IRI conversion, IDNA, etc.). And, I think
that HTTPbis should explain how to encode UTF-8 text in newly registered
header fields. The de-facto mechanism for this, used by Atom and WebDAV,
is percent-encoded UTF-8.

> UTF-8, and RFC 3987 offers a strict STD 66 URI for any IRI.

You seem to know a lot more about IETF policy than me, but I don't see
how it is possible to defer the internationalization considerations of
HTTP any further, while keeping HTTP on the standards track.

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 13 Mar 2008 14:24:06 +0100

> effectively HTTP link relations and HTTP headers share the same

Well, you may wish they did, but they don't.

With Atom relations, we already have name collisions with HTTP header 
("Via").

> LINK and UNLINK goes away, which also reduces the need for the Link

I'm not going to defend LINK and UNLINK here; in particular as long as 
we don't get Link: fixed.

> presume) for any link relation. And, the syntax has to be different from

RFC 2068 says:

"The Link field is semantically equivalent to the <LINK> element in 
HTML.[5]" -- <http://tools.ietf.org/html/rfc2068#section-19.6.2>

...so I don't see how we can change that without rewriting history.

> similar-but-different. So, it is not really "the same mechanism" at all.

...which would be an argument not to consider compatibility with Atom.

> interoperability issue for Atom. 

Well, if you do what the spec says, you don't need to check for both. 
Resolve first, then compare.

And yes, it's always a problem when there are two ways to do the same 
thing. We need test cases for things like that.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 13 Mar 2008 14:47:52 +0100

> UTF-8 in URIs is RFC 3987.

Well, no. URIs are all ASCII. There's is no such thing as "Unicode 
support in URIs". These are called "IRIs".

> and placed next to the Security Considerations section."

All nice in theory, but it hasn't been done in RFC2616.

> RFC 2277 applies to any updates to an existing protocol, as far as I can

I don't see how it could apply to that.

> correctly in HTTP (via URI-IRI conversion, IDNA, etc.). And, I think

HTTP uses URIs, not IRIs.

That being said, it may be necessary to state a few things about HTTP 
IRIs, but that would be in a separate document. Remember our charter?

> is percent-encoded UTF-8.

Note: one instance in WebDAV Delta-V, one in AtomPub.

Are you saying httpbis should recommend that for new headers? I'm not 
against it, but it sounds like something for an update to the HTTP 
header registry.

> HTTP any further, while keeping HTTP on the standards track.

I fear it's the other way around. If we want to keep it on the standards 
track, we can't make any incompatible changes.

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Thu, 13 Mar 2008 08:00:36 -0700

Julian Reschke wrote:
> Brian Smith wrote:
> > "Lack of an ability to use UTF-8 is a violation of this 
> > policy; such a violation would need a variance procedure
> > ([BCP9] section 9) with clear and solid justification
> > in the protocol specification document before being
> > entered into or advanced upon the standards track."
> > 
> > "For existing protocols or protocols that move data
> > from existing datastores, support of other charsets,
> > or even using a default other than UTF-8, may be a
> > requirement. This is acceptable, but UTF-8 support
> > MUST be possible."
> 

The purpose of HTTPbis is to fix problems with RFC2616. That is one of
the problems that needs to be fixed.

> >> HTTP is no "new" protocol, like mail or news:  2821bis and 2822upd 
> >> and FWIW RFC.usefor-usefor don't "violate" any IETF 
> > RFC 2277 applies to any updates to an existing protocol, as 
> > far as I can tell.
> 

Please read what I quoted above. HTTP is an existing protocol, so it can
have a default charset other than UTF-8, but "UTF-8 support MUST be
possible." 

> > I am not suggesting that HTTP 1.1 should switch from 
> > Latin-1 to UTF-8. But, I do think that HTTPbis should
> > explain how IRIs are to be used correctly in HTTP
> > (via URI-IRI conversion, IDNA, etc.). And, I think
> 
> That being said, it may be necessary to state a few things 
> about HTTP IRIs, but that would be in a separate document. 
> Remember our charter?

How does RFC 2277 fit into the standardization process. RFC 2277 itself
says "This document is the current policies being applied by the
Internet Engineering Steering Group (IESG) towards the standardization
efforts in the Internet Engineering Task Force (IETF) in order to help
Internet protocols fulfill these requirements." I take that to mean that
the IESG will reject any specification that is not compliant with RFC
2277 as a matter of policy.

> > that HTTPbis should explain how to encode UTF-8 text in newly 
> > registered header fields. The de-facto mechanism for this, used by 
> > Atom and WebDAV, is percent-encoded UTF-8.
> 
> Are you saying httpbis should recommend that for new headers? 
> I'm not against it, but it sounds like something for an 
> update to the HTTP header registry.

HTTPbis should at least standardize a mechanism for new headers to
support Unicode text. Percent-encoded UTF-8 is one possibility. Or--just
thinking off the top of my head--HTTPbis could allow new headers to
encode UTF-8 text directly in quoted-strings, by starting the quoted
string with the BOM (<EF><BB><BF>, which is "ï»¿" in Latin-1). 

But, it is totally unacceptable to add the Link header with a
non-Unicode-capable title subfield, it is unacceptable to specify any
new headers that have any human-oriented text that is not Unicode
enabled, and any existing headers that have human-oriented text should
be revised (in the most backwards-compatible way possible) to support
Unicode text.

> > You seem to know a lot more about IETF policy than me, but 
> > I don't see how it is possible to defer the
> > internationalization considerations of HTTP any further,
> > while keeping HTTP on the standards track.
> 
> I fear it's the other way around. If we want to keep it on 
> the standards track, we can't make any incompatible changes.

I agree. That is why I have not suggested any incompatible changes.

Regards,
Brian

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 13 Mar 2008 16:29:10 +0100

Brian Smith wrote:

> I understand it to mean that a new protocol that supports URIs
> must support UTF-8 in URIs, and the only (proposed) standard
> for UTF-8 in URIs is RFC 3987.

That is not the case, RFC 3986 is a full Internet Standard, and
it doesn't say "URIs must support UTF-8 in URIs".  Likewise new
protocols supporting mail addresses are not forced to support
EAI.  <recycle> RFC 3987 offers a working STD 66 URIs for any
valid IRI in any charset </recycle>  Better don't get me started
about 'legacy enhanced IRIs' <eg>

> just directly quote the parts that apply to HTTPbis:

That is simple, <quote> </quote>.  HTTPbis is no "new" protocol,
it is based on RFC 2068, and 2068 < 2277.  Similar 2821bis is
no "new" protocol, it's based on older memos.  To some degree
you can do "any charset", e.g., using the 8BITMIME extension of
SMTP (IIRC a SHOULD), or using B64/QP CTEs in MIME bodies, or
using UTF-8 (among others) directly in HTTP bodies.

>| In documents that deal with internationalization issues at all,
>| a synopsis of the approaches chosen for internationalization
>| SHOULD be collected into a section called 'Internationalization
>| considerations', and placed next to the Security Considerations
>| section.

Okay, that is something 2616bis SHOULD do.  They clubbed me on
the SMTP list when I proposed a mere informative reference to
EAI in the I18N considerations of the "email-arch" memo.  

> RFC 2277 applies to any updates to an existing protocol, as

It talks about UTF-8 "for all text".  We can ask Harald what
that precisely means, my first guess is "SDU" (body), not the
complete "PDU" (header + body).  And HTTP is even mentioned in
chapter 3.2 of BCP 15.

> You don't need to do the IDNA transformation for link
> relations, because you are not resolving the hostname of
> the IRI of the link relation. 

We don't need any IRI magic at all if we just follow Roy's and
Julian's proposal.  

> I do think that HTTPbis should explain how IRIs are to be
> used correctly in HTTP (via URI-IRI conversion, IDNA, etc.).

We cold say that implementors of RFC 3987 URI-representations
of IRIs have to implement RFC 3987, not some other algorithm
they saw in say the XML spec., but I guess that is obvious.

> I think that HTTPbis should explain how to encode UTF-8
> text in newly registered header fields. The de-facto 
> mechanism for this, used by Atom and WebDAV, is percent-
> encoded UTF-8.

For a draft standard MIME RFC 2047 comes to mind, for a BCP 
one of the two mechanisms recommended in BCP 113 (RFC 5137).
BCP 113 says that using UTF-8 is typically a bad idea when
looking for an ASCII-compatible encoding.  I'm not hot about
what to use in 2616bis, if anything, but if it ends up in a
single case remotely requiring IDNAbis punycode I scream :-)

For EAI they test the UTF-8 waters with *experimental* RFCs,
2616bis will be a draft standard or better if all goes well.

> You seem to know a lot more about IETF policy than me

NAK, but FWIW Harald was the USEFOR WG Chair when what is 
now RFC.usefor-usefor was Last Called and approved.  He did
not insist on allowing UTF-8 in NetNews header fields, and
several years of the USEFOR WG were wasted to pull this 
feature from an earlier set of Usefor drafts, because it
would break the complete installed base.

Of course RFC.usefor-usefor has I18N considerations, and
you can use UTF-8 in NetNews roughly in the same way (MIME)
as in mail, or roughly as in HTTP (NetNews is 8bit clean):

But NOT "as is" in header fields (NetNews uses RFC 2231, a
successor of RFC 2047 also supporting language tags, for
this purpose).  Another problem with percent-encoded UTF-8,
it offers no indication of language tags (BCP 47).

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Thu, 13 Mar 2008 16:28:48 +0100

> the problems that needs to be fixed.

What exactly is the problem that needs to be fixed?

> have a default charset other than UTF-8, but "UTF-8 support MUST be

<http://tools.ietf.org/html/rfc2277#section-2>:

    Internationalization is for humans. This means that protocols are not
    subject to internationalization; text strings are. Where protocol
    elements look like text tokens, such as in many IETF application
    layer protocols, protocols MUST specify which parts are protocol and
    which are text. [WR 2.2.1.1]

    Names are a problem, because people feel strongly about them, many of
    them are mostly for local usage, and all of them tend to leak out of
    the local context at times. RFC 1958 [RFC 1958] recommends US-ASCII
    for all globally visible names.

    This document does not mandate a policy on name internationalization,
    but requires that all protocols describe whether names are
    internationalized or US-ASCII.

    NOTE: In the protocol stack for any given application, there is
    usually one or a few layers that need to address these problems.

    It would, for instance, not be appropriate to define language tags
    for Ethernet frames. But it is the responsibility of the WGs to
    ensure that whenever responsibility for internationalization is left
    to "another layer", those responsible for that layer are in fact
    aware that they HAVE that responsibility.

So HTTP uses US-ASCII for names; that includes method names, header 
names, relation names, whatever. No problem so far.

Where HTTP headers transport I18Nable text, it should be using the TEXT 
BNF rule, which allows transport of any Unicode character, although in a 
really ugly way.

> string with the BOM (<EF><BB><BF>, which is "ï»¿" in Latin-1).

It already allows that through RFC2047-style encoding.

> be revised (in the most backwards-compatible way possible) to support

The title subfield is Unicode-capable. Read the grammar, plus RFC2047.

(No I don't like that encoding either, but that's what we have).

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Thu, 13 Mar 2008 09:53:35 -0700

> > support Unicode text. 

> Read the grammar, plus RFC2047.

Thank you. I read the BNF for TEXT many times but I've always overlooked
the accompanying text. The RFC 2047 mechanism is truly horrible but I
guess it satisfies the requirements.

What is an accurate BNF grammar for TEXT? It is not clear to me how I am
supposed to parse a quoted-string that contains "=?" but which is not a
valid encoded-word.

Thanks again,
Brian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 13 Mar 2008 18:11:10 +0100

> the accompanying text. The RFC 2047 mechanism is truly horrible but I

Isn't it :-)

BTW: "Words of *TEXT MAY contain characters from character sets other 
than ISO-8859-1 [22] only when encoded according to the rules of RFC 
2047 [14]." really is incorrect and should be rephrased.

> guess it satisfies the requirements.

If it works in practice.

> supposed to parse a quoted-string that contains "=?" but which is not a

Good point. Are recipients of TEXT-typed header contents supposed to 
always run the value through an RFC2047 parser?

BR, Julian

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 13 Mar 2008 13:39:13 -0400

>2277 as a matter of policy.

And you would take it wrong. The IESG isn't a machine: they look at 
what makes sense. A backwards-incompatible change, for example, 
doesn't make sense. Further, because 2616 >> 2277, you can tell that 
the IESG thought that HTTP was already conformant enough.

--Paul Hoffman, Director
--VPN Consortium

---

From: "Brian Smith" <[email-masked]>
Date: Thu, 13 Mar 2008 16:08:25 -0700

Julian Reschke wrote:
> Brian Smith wrote:
> > Thank you. I read the BNF for TEXT many times but I've always 
> > overlooked the accompanying text. The RFC 2047 mechanism is truly 
> > horrible but I guess it satisfies the requirements.

> BTW: "Words of *TEXT MAY contain characters from
> character sets other than ISO-8859-1 [22] only when
> encoded according to the rules of RFC 2047 [14]."
> really is incorrect and should be rephrased.

It is not clear whether or not the RFC 2047 mechanism can be used in
quoted-string, because quoted-string is not defined in terms of "*TEXT",
but rather a similar construct. Given all the places that quoted-string
is used, should the RFC 2047 mechanism really be allowed in all these
places?:

	Accept
	Cache-Control
	Content-Encoding
	Content-Type
	ETag
	Expect
	If-Match
	If-None-Match
	If-Range
	Pragma
	TE
	Transfer-Encoding	
	Warning

Also, the Reason-phrase of the status line is defined as:

	*<TEXT, excluding CR, LF>

But, is the RFC 2047 mechanism allowed in the Reason-phrase?

If the RFC 2047 mechanism is not allowed in quoted-string or the
Reason-phrase, then were *is* it allowed?

> > What is an accurate BNF grammar for TEXT? It is not clear 
> > to me how I am supposed to parse a quoted-string that
> > contains "=?" but which is not a valid encoded-word.

> Good point. Are recipients of TEXT-typed header contents 
> supposed to always run the value through an RFC2047 parser?

Are there any headers fields that have *TEXT in their grammar?

If the specification is read strictly, then the RFC 2047 mechanism has
never been allowed everywhere. And, if it is read liberally, then it is
allowed in way too many places. And, if it is allowed anywhere, there
should be some advice as to what encodings should be supported.

- Brian

---

From: Martin Duerst <[email-masked]>
Date: Fri, 14 Mar 2008 13:47:55 +0900

At 08:08 08/03/14, Brian Smith wrote:
>> > horrible

Very well put.

>> > but I guess it satisfies the requirements.
>
>> If it works in practice.

And, if it is actually used in practice. Any pointers to some
actual usage (both iso-8859-1 and RFC 2047) would be appreciated.

>but rather a similar construct.

My reading the last time I read the spec (years back) is that
Warning: is about the only place where it is allowed.

It may be difficult to fix the truely horrible RFC 2047 on top of
iso-8859-1 mess for existing headers. But in order to move in the
right direction, it would be a very good idea to allow newly defined
headers to specify that they just use UTF-8.

Regards,    Martin.

>Given all the places that quoted-string

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Fri, 14 Mar 2008 13:53:11 +0900

At 02:11 08/03/14, Julian Reschke wrote:
>BTW: "Words of *TEXT MAY contain characters from character sets other than ISO-8859-1 [22] only when encoded according to the rules of RFC 2047 [14]." really is incorrect and should be rephrased.

What exactly is the problem? Do you have any new text?

Regards,   Martin.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 10:36:50 +0100

> headers to specify that they just use UTF-8.

That's related to <http://www3.tools.ietf.org/wg/httpbis/trac/ticket/74>.

This really sounds like the simplest way to do it -- if this applies 
only to new headers, is there even a remote chance of breaking something?

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 10:29:47 +0100

> but rather a similar construct. Given all the places that quoted-string

???

<http://greenbytes.de/tech/webdav/rfc2616.html#basic.rules.quoted-string>:

     quoted-string  = ( <"> *(qdtext | quoted-pair ) <"> )
     qdtext         = <any TEXT except <">>

> is used, should the RFC 2047 mechanism really be allowed in all these

I think this is the intent.

> But, is the RFC 2047 mechanism allowed in the Reason-phrase?

I would think so.

> never been allowed everywhere. And, if it is read liberally, then it is

I disagree.

> should be some advice as to what encodings should be supported.

 From the headers above, where do you think it shouldn't be allowed?

I do agree that if we rely on RFC2047, we may also have to spend some 
time improving that document.

BR, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 14 Mar 2008 20:49:17 +1100

Personally, I am *very* -1 on doing this.

Changing the allowable characters in a protocol element is a *big*  
change, and there is not an interoperability gain to doing so.

There is also not a functionality gain; it is possible (if not pretty)  
to serialise other characters into HTTP headers.

Furthermore, HTTP headers for the most part don't carry user-visible  
data, and when they do, it's often an IRI, for which there is a well- 
understood serialisation into header-safe characters.

Before we go too far down this path again, folks should refresh  
themselves with the last round of discussion, starting at:
   http://lists.w3.org/Archives/Public/ietf-http-wg/2007JulSep/0323.html

Cheers,

On 14/03/2008, at 8:36 PM, Julian Reschke wrote:

> That's related to <http://www3.tools.ietf.org/wg/httpbis/trac/ticket/74 
> >.
>

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 10:23:02 +0100

> What exactly is the problem? Do you have any new text?

As far as I understand, words of *TEXT never contain characters from 
character sets other then ISO8859-1, the same way URIs never contain 
non-ASCII characters.

Wouldn't it be more precise to say:

"Characters from character sets other than ISO-8859-1 must be encoded 
according to the rules of RFC 2047 for use inside TEXT constructs."

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 11:26:59 +0100

> to serialise other characters into HTTP headers.

I think the key question here: is that implemented in practice? (In 
particular, which encoding?) If yes, fine (and maybe let's document what 
works). But if not...?

> data, and when they do, it's often an IRI, for which there is a 
> well-understood serialisation into header-safe characters.

None of the usages in RFC2616 is a IRI (by definition :-).

And, as Brian observed, IRI->URI mapping is not sufficient if you need 
to *exactly* reconstruct the original IRI (such as when the IRI is used 
as a name, e.g. an Atom link relation).

BR, Julian

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 14 Mar 2008 11:45:43 +0100

> Personally, I am *very* -1 on doing this.

I agree (not sure if this means +1 or -1 now, german double negatives  
don't do nothing.)

> change, and there is not an interoperability gain to doing so.

I think such a change is outside 1.1.

//Stefan

--
<green/>bytes GmbH, Hafenweg 16, D-48155 Münster, Germany
Amtsgericht Münster: HRB5782

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 14 Mar 2008 21:46:52 +1100

On 14/03/2008, at 9:26 PM, Julian Reschke wrote:

>> There is also not a functionality gain; it is possible (if not  
>> pretty) to serialise other characters into HTTP headers.
>
> particular, which encoding?) If yes, fine (and maybe let's document  
> what works). But if not...?
>
>> Furthermore, HTTP headers for the most part don't carry user- 
>> visible data, and when they do, it's often an IRI, for which there  
>> is a well-understood serialisation into header-safe characters.
>
> And, as Brian observed, IRI->URI mapping is not sufficient if you  
> need to *exactly* reconstruct the original IRI (such as when the IRI  
> is used as a name, e.g. an Atom link relation).

I think Larry was making much the same point at the app area arch ws,  
IIRC.

My gut feeling on issue 74 is that if any encoding (e.g., IRI->URI,  
RFC2047) is usable in a header, it should be specified on a header-by- 
header basis explicitly, not inherited as a general rule like this.  
That allows the flexibility to use the right mechanism for the job,  
and keeps the overhead low.

Are there any other headers (besides URI-based headers) that we need  
to consider, beyond these?
  * WWW-Authenticate, Proxy-Authenticate, Authorization, Proxy- 
Authorization
  * Content-Disposition
  * From
  * Warning
  * PICS-Label

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 14 Mar 2008 21:53:27 +1100

I've submitted a draft-01 (perhaps against my best judgement; this  
discussion is rapidly consuming the mailing list...), which should  
appear shortly. In the meantime, it's available at <http://www.mnot.net/drafts/draft-nottingham-http-link-header-01.txt 

It is in no way a finished product; it's just a straw-man that tries  
to cover the issues, so it's easier to appreciate what they are (even  
if we don't go in this direction).

Cheers,

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 14 Mar 2008 11:55:33 +0100

Am 14.03.2008 um 11:46 schrieb Mark Nottingham:
>
> I think Larry was making much the same point at the app area arch  
> ws, IIRC.
>
> RFC2047) is usable in a header, it should be specified on a header- 
> by-header basis explicitly, not inherited as a general rule like  
> this. That allows the flexibility to use the right mechanism for  
> the job, and keeps the overhead low.
>
> Are there any other headers (besides URI-based headers) that we  
> need to consider, beyond these?

The whole "Auth" header series does not have the problem, I think.  
All header values are ASCII.

Correct me if I'm wrong: the open issue is how to send user names/ 
passwords with non-ascii characters. But that is a matter of the auth  
mechanism to specify, not an issue with HTTP headers. Example: Basic  
puts Base64 values in the header. Those are always ASCII. But Base64  
operates on octets and Basic just fails to specify how a username is  
converted to octets, right?

//Stefan

---

From: Phil Archer <[email-masked]>
Date: Fri, 14 Mar 2008 11:06:43 +0000

Mark Nottingham wrote:
[snip]
> Are there any other headers (besides URI-based headers) that we need to 
> consider, beyond these?
>  * WWW-Authenticate, Proxy-Authenticate, Authorization, Proxy-Authorization
>  * Content-Disposition
>  * From
>  * Warning
>  * PICS-Label
> 

PICS is, of course, still a W3C Rec - however, it is worth noting that, 
unless something goes horribly wrong, it will be withdrawn in favour of 
POWDER within the next 12 months. My organisation (formerly ICRA, née 
RSAC) has been associated with PICS since it began and we operate the 
PICS labelling system found in Internet Explorer (yes, still in IE7) - 
in other words, if anyone has an interest in PICS, FOSI does - and even 
I'd hate to see anyone spend valuable time updating PICS-Label HTTP 
headers now.

Cheers

Phil.

-- 
Phil Archer
Chief Technical Officer,
Family Online Safety Institute
w. http://www.fosi.org/people/philarcher/

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 13:14:08 +0100

Julian Reschke wrote:

> as Brian observed, IRI->URI mapping is not sufficient if you need 
> to *exactly* reconstruct the original IRI (such as when the IRI

I think he also observed that Latin-1 IRIs are not much, it gives
you 96 additional codepoints u+00A0 up to u+00FF to play with in
HTTP header fields.

The wonders of chapter 5 in RFC 3987 are IMO no 2616bis problem´.

 Frank

---

From: "Brian Smith" <[email-masked]>
Date: Fri, 14 Mar 2008 05:31:22 -0700

Julian Reschke wrote:
> > quoted-string, because quoted-string is not defined in terms of 
> > "*TEXT", but rather a similar construct. Given all the places that 
> > quoted-string
> 
> ???
> 

<any TEXT except <">> is not equivalent to *TEXT.

> I think this is the intent.

Then you run into the question "How are media-ranges and media-types
compared? Are they to be decoded into Unicode and then compared?" When
the specification specifies that ETags must match exactly, is the
comparison character-by-character or octet-by-octet?

> > But, is the RFC 2047 mechanism allowed in the Reason-phrase?

Again, the grammar for reason-phrase is not *TEXT, that is why it isn't
clear

> > And, if it is read liberally, then it is
> 
> I disagree.
> 
> > allowed in way too many places. And, if it is allowed 
> > anywhere, there should be some advice as to what
> > encodings should be supported.
> 

Consider:

  Content-Type: text/plain;charset="=?utf-8?q?utf-8?="
  (how do you compare this against 'text/plain;charset="utf-8"'?)

  ETag: "=?utf-8?q?asdf?="
  (how do you compare this against "asdf"?)

  ETag: "=?"
  (Is this a lexical error?)

> I do agree that if we rely on RFC2047, we may also have to 
> spend some time improving that document.

Keep in mind that RFC2047 has a limit of 75 characters per encoded-word.
And, the grammar seems to allow encoded-words to be mixed with unencoded
words. And, Base-64 encoding to be muxed with quotable-printable. And,
multiple encodings (e.g. UTF-8 and UTF-7) to be mixed. All in the same
*TEXT segment. There is definitely a lot to be improved, but each
improvement would be a incompatible change.

- Brian

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 13:38:44 +0100

Stefan Eissing wrote:

> Basic just fails to specify how a username is converted to octets,
> right?

Ditto passwords and Digest.  RFC 2617 inherits the 2616 *TEXT, and
that is to be interpreted as Latin-1 octets when it's not RFC 2047
encoded.  And you don't 2047-encode user names and passwords used
as input for Basic / Digest / ...  My crystal ball says.  

To "fix" this we'd need SASLPREP UTF-8 as in the 2831bis drafts,
or at least UTF-8 as in 2831, that's one or two additional Digest
parameters.  Won't work directly for Basic - unless we jump from
2616 HTTP/1.1 Latin-1 to a 2616ter HTTP/1.2 UTF-8.  That is no
goal for 2616bis in this round.

 Frank

---

From: "Brian Smith" <[email-masked]>
Date: Fri, 14 Mar 2008 05:51:03 -0700

Mark Nottingham wrote:
> Changing the allowable characters in a protocol element is a 
> *big* change, and there is not an interoperability gain to doing so.
> 
> There is also not a functionality gain; it is possible (if 
> not pretty) to serialise other characters into HTTP headers.

Because RFC 2616 doesn't specify the integration of the RFC 2047 into
HTTP, the RFC 2047 is basically unimplementable. At least, you cannot be
sure that any two implementations are implementing it the same way
because there are ambiguities.

If the integration of RFC 2047 into HTTP was clarified, then at least it
would be implementable. But, because current implementations are not
implementing RFC 2047, you cannot use it for any existing
headers/subfields. So, the new-and-improved RFC 2047 integration would
only be useful for newly-created headers/subfields. But, RFC 2047 is so
terrible that nobody wants to use it anyway, so why waste effort trying
to fix the RFC 2047 integration? Just deprecate it.

But, if RFC 2047 integration is deprecated, then a new mechanism should
be created that can handle UTF-8. Forget the argument that says RFC 2277
requires the HTTP WG to do this. Whether it does or not, it makes sense
to define a sensible, general-purpose way to include UTF-8 in new
headers and new header subfields. The specification for it can be
written up independently of the HTTP WG, and the WG can decide whether
to include it into HTTPbis; if not, it can be published as its own RFC.
IMO, if somebody is willing to create a new, usable mechanism before
HTTPbis is completed, it would be stupid not to include it in HTTPbis.

- Brian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 13:57:38 +0100

> <any TEXT except <">> is not equivalent to *TEXT.

I would argue that the intent of that production is clearly to inherit 
the rules for TEXT.

Funny enough, this issue is one of the remaining blockers for the 
conversion to ABNF; we really need to clarify TEXT, and all productions 
based on TEXT.

Yes.

> comparison character-by-character or octet-by-octet?

That's really not relevant as long as the producer of ETags always uses 
the same representation.

But I do agree what we probably need to look at each case where 
quoted-string is used and decide whether it requires I18N or not.

> Again, the grammar for reason-phrase is not *TEXT, that is why it isn't

But this is indeed one of the cases where I18N makes sense.

Any chance that some of the original authors can explain the history here?

>   (how do you compare this against 'text/plain;charset="utf-8"'?)

I would have hoped that RFC2045 answers this, but that doesn't seem to 
include a definition of quoted-string.

>   (Is this a lexical error?)

For ETag, I'd say it's not a problem. If the server producing the ETags 
wants to cause problems, let it do so.

> improvement would be a incompatible change.

It seems the only way to improve RFC-2047 would be by introducing a new 
encoding that is sane. Such as:

"Any octet sequence starting with EF BB BF (the UTF-8 BOM) is to be 
interpreted as Unicode, encoded in UTF-8."

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 14:04:55 +0100

Mark Nottingham wrote:
> appear shortly. In the meantime, it's available at 
> <http://www.mnot.net/drafts/draft-nottingham-http-link-header-01.txt>.
> 
> It is in no way a finished product; it's just a straw-man that tries to 
> cover the issues, so it's easier to appreciate what they are (even if we 
> don't go in this direction).
> ...

Mark,

I think this absolutely heads into the right direction.

A few comments...:

1.  Introduction

    A means of indicating the relationships between documents on the Web
    has been available for some time in HTML, and was considered as a
    HTTP header in [RFC2068], but removed from [RFC2616], due to a lack
    of implementation experience.

JRE: include reference for HTML

        relationship   = URI-Reference |
                       <"> URI-Reference *( SP URI-Reference) <"> )

JRE: we probably should state that relationship names that include a 
semicolon need to be quoted.

    Relationship values are URIs that identify the type of link.  If the
    relationship is a relative URI, its base URI MUST be considered to be
    "http://www.iana.org/assignments/link-relations.html#", and the value
    MUST be present in the link relation registry.

JRE: why a new base URI? What's wrong with 
"http://www.iana.org/assignments/relation/" 
(<http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.2.7.2>)?

6.1.  Normative References

JRE: I think RFC2434 and RFC3864 could be classified as informative. 

    to map the profiled link relations to URIs.  For example, in HTML:

    <html>
      <head profile="http://example.com/profile1/">
        <link rel="foo" href="/foo">
      </head>
      [...]

    could be represented as a header like this;

    Link: </foo>; rel="http://example.com/profile1/foo"

JRE: do we need to talk about profile URIs where concatenation does not 
work well, such as "http://www.w3.org/2003/g/data-view"?

    HTML defines link relation values as case-insensitive, while the Link
    header's syntax does not.  Therefore, it is important to case-
    normalise relation values in HTML before comparing or converting them
    to Link headers.

JRE: impact on registration procedure? Avoid names that only differ in 
case?

    Atom conveys links in the atom:link element.  When serialising an
    atom:link into a Link header, it is necessary to convert any IRIs to
    URIs, since HTTP headers cannot directly contain UTF-8.

JRE: that's a bit misleading. For instance, a IRI that contains 
non-ASCII Latin-1 characters could be put into an HTTP header, we just 
don't want to allow that and chose URIs as format.

    Additionally, since the base URI for link relations in Link headers
    is fixed, extension links (i.e,. those not in the registry) MUST be
    serialised as absolute URIs.

JRE: s/serialisized/represented/?

BR, Julian

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 14 Mar 2008 14:12:48 +0100

> as input for Basic / Digest / ...  My crystal ball says.

Yes and yes.

> [...]Won't work directly for Basic - unless we jump from

My point is: fix Basic and Digest. HTTP/1.1 needs no additional  
charset for its auth headers. Even if someone comes up with a miracle  
to make HTTP/1.1 send utf-8 headers, it would not make authentication  
work for deployed software. Simply because the code en/decoding auth  
header values is most likely totally separate from any generic header  
parsing stuff.

Deprecate use of 2047-encode in 1.1 headers and close the issue, please.

---

From: Martin Duerst <[email-masked]>
Date: Fri, 14 Mar 2008 20:33:33 +0900

>to compare link relations.

I don't understand the three uses of "URI -> IRI conversion". If the
first is a no-op, that essentially means that URI contains no %-escape
sequences that can be interpreted as UTF-8.

And to put an IRI in a header, one would usually use IRI -> URI conversion,
rather than the other way round. But using that conversion would result
in an URI that wouldn't conform to the first condition.

Can you explain?

>> I don't think this is true.

There is no explicit requirement to use IRIs for new IETF protocols.
There are lots of cases where that makes sense, but there may be
others where it doesn't make sense.

>> > However, I personally believe it is wrong 

It's not about European languages. There are some non-European
languages that can be written with ASCII only (Swahili, Hawai'ian,...),
but most European languages require non-ASCII characters to be
written properly.

>all new IETF standards for URI processing must support IRIs.

As Paul has explained, the IETF isn't a machine. It may make sense
in some cases to make a last minute fix before something goes to
Standard, but on the other hand, even if we decide to allow resource
identifiers for link relationships, the need for internationalized
ones is probably pretty low indeed. After all, these link relationships
are intended to be shared across languages and writing conventions,
I assume.

Regards,   Martin.

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 14:25:55 +0100

[...]
> the grammar seems to allow encoded-words to be mixed with unencoded
> words.

Yes.  FWS between encoded words is removed by decoders, that's how
you get around the length limit, just insert FWS between characters
in the encoder.  Not between input octets, you can't "split" UTF-8.

FWS between encoded and unencoded words is for real (= white space),
like FWS beween unencoded words.  (FWS is "folding white space",
the older MIME RFCs still use "linear white space" like RFC 2616).

> multiple encodings (e.g. UTF-8 and UTF-7) to be mixed.

Yep, I like pc-multilingual-850+euro better than UTF-7, it arrives
faster at the critical "75", especially if combined with a *long*
RFC 4646 language tag as specified in RFC 2231.

Back to your original point, you don't 2047-encode quoted-strings,
RFC 2047 says: 

| + An 'encoded-word' MUST NOT appear within a 'quoted-string'.
[...]
| + An 'encoded-word' MUST NOT be used in parameter of a MIME
|   Content-Type or Content-Disposition field, or in any structured
|   field body except within a 'comment' or 'phrase'.

Roughly the idea is that you can 2047-encode unstructured header
fields as you like (example: Subject in mail or news).  For any
structured header field you must not touch its structure, a
comment must be still a comment, a mail address is must not be
touched at all, ditto quoted-string etc. (see above).

Example: This (is) a test

For a structured header field with name "Example" and field body
"This (is) a test", the four words actually in some interesting
charset, you can 2047-encode "This", "is", "a", "test", "a test".

You cannot encode anything with "(is)", it would obscure the
structure, here the comment "(" and ")".  The goal is, that an
MTA or MUA knowing nothing about MIME at all, can simply treat
(=?us-ascii*tlh?Q?is?=) or similar as some weird ASCII-word in
a comment, it doesn't need to know that it's an US-ASCII Klingon
"is", but it needs to know that it's a comment.  

Example: "umlauted gibberish" 

You can't encode the gibberish within the quoted-string.  But
for structured fields quoted-string *always* (please check this)
is used where unquoted words are allowed.  In other words you
cannot do "=?utf-8?Q?umlauted_gibberish?=" within quotes.  But
you can use =?utf-8?Q?umlauted_gibberish?= without quotes as
ordinary word ("ordinary" from the POV of a MIME-agnostic MTA).

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 14:37:24 +0100

Stefan Eissing wrote:

> My point is: fix Basic and Digest.

I'm not sure that it's allowed by the Charter.  I asked about a
2617bis when the Charter was discussed, but nobody really wants
to tackle it.  And the SASL folks even intend to deprecate the
"better" (wrt I18N) Digest-MD5, attempts to fix 2831 turned out
to be a major headache.  

> Deprecate use of 2047-encode in 1.1 headers

Then 2616bis would be limited to Latin-1, or even to ASCII if
it gets rid of Latin-1.  And *that* would violate RFC 2277, as
Brian wrote, only Latin-1 is not good enough.  RFC 2047 + 2231
are a bit odd, but better than nothing, and implementors in the
mail and news world sometimes get it right, HTTP implementors
likely also manage, or don't they ?  

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 14:49:48 +0100

Brian Smith wrote:
 
> RFC 2047 is basically unimplementable.

It is a draft standard, there must be at least two interoperable
implementations.  Come on, it is no rocket science, the Internet 
uses MIME for ages, RFC 1341 was published in 1992.

> At least, you cannot be sure that any two implementations are
> implementing it the same way because there are ambiguities.

There are no ambiguities, there are only a few dark corners.

> RFC 2047 is so terrible that nobody wants to use it anyway

Parts of RFC 2231 are seriously odd, but "nobody wants it" does
not cut it if the alternatives are HTTP/1.2 UTF-8 or a HTTP/1.1
limited to Latin-1.  

If you are sure that 2047 is either unsupported or doesn't work
as specified it could justify to pull it, but then nobody will
forget what you said RFC 2277, and we can save steps by saying
that HTTP/1.1 is hopleless.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Fri, 14 Mar 2008 14:59:33 +0100

> likely also manage, or don't they ?  

I guess that's the key question.

Do we have any evidence this is implemented somewhere?

And, leaving existing headers aside: if we *would* introduce a way for 
new headers to use UTF-8 (similarly to the "TEXT" rule in RFC2616), 
would that break anything?

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 15:14:09 +0100

Julian Reschke wrote:

> if we *would* introduce a way for new headers to use UTF-8
> (similarly to the "TEXT" rule in RFC2616), would that break
> anything?

Using "raw" UTF-8 would be dubious if the other side hates C1
controls.  Or it could break in interesting ways if the other
side likes some C1 controls (NEL, SS2, SS3).  But I guess you
mean something more sophisticated like BCP 113.

 Frank

---

From: "Brian Smith" <[email-masked]>
Date: Fri, 14 Mar 2008 07:22:39 -0700

Frank Ellermann wrote:
> It is a draft standard, there must be at least two 
> interoperable implementations.  Come on, it is no rocket 
> science, the Internet uses MIME for ages, RFC 1341 was 
> published in 1992.

I followed that with "At least, you cannot be sure that any two
implementations are implementing it the same way because there are
ambiguities." Look at the various interpretations of where it is allowed
or should be allowed in this discussion alone. Mail/News applications
might have figured it out, but HTTP applications haven't. 

The only reason we are discussing RFC 2047 right now is because of the
Link header, which uses quoted-string to contain language-sensitive
text. If RFC 2047 cannot be used in quoted-string, then there is no way
to internationalize the Link header as it is defined in RFC 2068 or in
the latest draft that Mark just linked to. A new mechanism would need to
be created to allow internationalization of the Link header. If a new
mechanism is necessary, we might as well standardize it so that it can
be used in other (new) headers with similar requirements.

Also, Atom and HTML 5 (will) allow internationalized link relation
names, and IRIs as link relation names. If HTTP Link relations names
cannot be internationalized, then HTTP Links will only be useable for a
subset of the relations usable in HTML and Atom. That restriction is
unnecessary.

> Parts of RFC 2231 are seriously odd, but "nobody wants it" 
> does not cut it if the alternatives are HTTP/1.2 UTF-8 or a 
> HTTP/1.1 limited to Latin-1.  
> 
> If you are sure that 2047 is either unsupported or doesn't 
> work as specified it could justify to pull it, but then 
> nobody will forget what you said RFC 2277, and we can save 
> steps by saying that HTTP/1.1 is hopleless.

That is why I suggested that a new mechanism for encoding Unicode text
in header subfields should be created, and that the Link header should
use the new mechanism. Or, put another way, whatever mechanism is used
to make the Link header RFC 2277 compliant should be standardized so
that it can be used by other (new) headers.

The easiest way to internationalize the Link header is to drop the title
subfield completely, and specify that link relations are to be compared
to each other in their IRI (segment) form.

- Brian

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 14 Mar 2008 16:21:35 +0100

Brian Smith wrote:

> If RFC 2047 cannot be used in quoted-string, then there is no way
> to internationalize the Link header as it is defined in RFC 2068
> or in the latest draft that Mark just linked to.

As explained you don't need a <quoted-string> as soon as you have
RFC 2047 encoded words.  Just remove the quotes and 2047-encode
what was within to get an "ordinary" unquoted word =?...?.?...?=

Your premise is apparently wrong.   Looking in RFC 2068 19.6.2.4
I find:

| link-extension = token [ "=" ( token | quoted-string ) ]

Four cases for the right hand side (value):  A simple <token>,
or a (Latin-1) <quoted-string>, or a 2047 encoded word in any
charset, or for longer strings the magic specified in RFC 2231.

Not good enough is this detail:  "title" "=" quoted-string
To "fix" it we use  "title" "=" ( token | quoted-string )  in
2616bis, transformed into the corrresponding proper ABNF, and
then it's the same as above:

Use token, quoted-string, encoded word, or a folded value as
explained in RFC 2231 for a longer non-Latin string.

This covers all <quoted-string> cases in RFC 2068 19.6.2.4.
What's missing, is <sgml-name> not more what it used to be ?

> Atom and HTML 5 (will) allow internationalized link relation
> names, and IRIs as link relation names.

RFC 2068 clearly says URI, and all IRIs have equivalent URIs.
If that is not good enough for Atom, then Atom is in need of
a new protocol to transport it, that is not the fault of HTTP.

 Frank

---

From: "Brian Smith" <[email-masked]>
Date: Fri, 14 Mar 2008 09:29:20 -0700

Frank Ellermann wrote:
> Brian Smith wrote:
> As explained you don't need a <quoted-string> as soon as you 
> have RFC 2047 encoded words.  Just remove the quotes and 
> 2047-encode what was within to get an "ordinary" unquoted 
> word =?...?.?...?=
> 
> Four cases for the right hand side (value):  A simple 
> <token>, or a (Latin-1) <quoted-string>, or a 2047 encoded 
> word in any charset, or for longer strings the magic 
> specified in RFC 2231.

You cannot derive "token | quoted-string | encoded-word" from "token |
quoted-string". RFC 2616 doesn't say that encoded-word may be
substituted for token or quoted-string; it only says encoded-word may be
substituted for *TEXT. RFC 2047 only talks about how to use
encoded-string in RFC 2045 messages with RFC 822 headers, not RFC 2616
messages or RFC 2616 headers. Finally, none of them reference RFC 2231,
and neither does Mark's newest Link header draft, so RFC 2231 is
irrelevant.

> > Atom and HTML 5 (will) allow internationalized link relation names, 
> > and IRIs as link relation names.
> 

RFC 2068 uses sgml-name, not URI. Mark added new functionality to
support URI references in his new draft. Atom allows IRIs or IRI
segments, HTML allows anything that doesn't embed spaces. 

- Brian

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 15 Mar 2008 08:55:30 +1100

Thanks for the feedback. Responses inline...

On 15/03/2008, at 12:04 AM, Julian Reschke wrote:

>> It is in no way a finished product; it's just a straw-man that  
>> tries to cover the issues, so it's easier to appreciate what they  
>> are (even if we don't go in this direction).
>> ...
>
> Mark,
>

Ack

> semicolon need to be quoted.

Ack

> JRE: why a new base URI? What's wrong with "http://www.iana.org/assignments/relation/ 
> " (<http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section. 
> 4.2.7.2>)?

Hmm. That's the actual URI of the registry; we should ping IANA on  
this, but yes.

> JRE: I think RFC2434 and RFC3864 could be classified as informative.

Probably.

> JRE: do we need to talk about profile URIs where concatenation does  
> not work well, such as "http://www.w3.org/2003/g/data-view"?

I'm not sure; I thought about it, but would like to see a use case  
where it's important. Not that there isn't one, but my limited  
imagination couldn't come up with one at 10pm last night.

> JRE: impact on registration procedure? Avoid names that only differ  
> in case?

Nice catch.

> JRE: that's a bit misleading. For instance, a IRI that contains non- 
> ASCII Latin-1 characters could be put into an HTTP header, we just  

hmm.

> JRE: s/serialisized/represented/?

Sure.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 15 Mar 2008 10:10:49 +0100

Mark Nottingham wrote:
> Hmm. That's the actual URI of the registry; we should ping IANA on this, 
> but yes.

Atom specifies <http://www.iana.org/assignments/relation/>.

The page at <http://www.iana.org/assignments/link-relations.html> does 
not provide anchors, so appending a fragment identifier currently has no 
effect.

I was going to claim:

"However, appending the relation name to 
<http://www.iana.org/assignments/relation/> indeed gets you to the 
registration, such as with <http://www.iana.org/assignments/relation/edit>."

...but that is unfortunately not true, because IANA uses 
<http://www.iana.org/assignments/link-relations/> as a base URI, which 
of course is *totally* confusing.

This mismatch should be resolved in some way, be it by fixing the IANA 
web site, or by changing the base URI in the specification (which of 
course will result in Brian complaining even more :-).

> I'm not sure; I thought about it, but would like to see a use case where 
> it's important. Not that there isn't one, but my limited imagination 
> couldn't come up with one at 10pm last night.

I'm also not sure whether it's our problem, but it is a question a 
reader of the spec may ask.

For instance, with the profile URI for GRDDL being 
<http://www.w3.org/2003/g/data-view>, would the absolute URI in the link 
header be

   <http://www.w3.org/2003/g/data-viewtransform>

or

   http://www.w3.org/2003/g/data-view/transform>

or something else?

We may not need to specify that, but if we don't, it may be good to 
state that the party defining the link relation should also specify the 
absolute URI.

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Sat, 15 Mar 2008 10:47:47 +0100

Brian Smith wrote:
 
> You cannot derive "token | quoted-string | encoded-word"
> from "token | quoted-string".

There are two RFCs 2047 and 2231 explaining the technical
details.  If my attempt to express the concept with less
words failed please ignore it, stick to RFC 2047 and 2231.

> RFC 2047 only talks about how to use encoded-string in 
> RFC 2045 messages with RFC 822 headers, not RFC 2616
> messages or RFC 2616 headers.

AFAIK an RFC 2616 message is a variant of message/rfc822,
registered as MIME type message/http.  If MIME RFC 2047
etc. is not more good enough for 2616bis this effort is
doomed.  "Find something better than MIME while claiming
that the result is still related to HTTP/1.1" can't fly.

> Finally, none of them reference RFC 2231

Updating and fixing the references, the numerous errata,
and the horrible syntax, is precisely what this WG tries.

> Mark's newest Link header draft

This draft or HTML 5 have nothing to do with the job to
get RFC 2616 into a shape good enough for its DS status.

If referencing RFC 2231 is really unnecessary it's fine,
after all it would be a downref, and RFC 2231 admits to
be "gibbous".

>> RFC 2068 clearly says URI
[...]
> RFC 2068 uses sgml-name, not URI.

| Link = "Link" ":" #("<" URI ">" *( ";" link-param )
..........................^^^

> HTML allows anything that doesn't embed spaces.

| <!ATTLIST LINK
|  %attrs;                     -- %coreattrs, %i18n, %events --
|  charset %Charset; #IMPLIED  -- char encoding of linked resource --
|  href    %URI;     #IMPLIED  -- URI for linked resource --
............^^^

What are you talking about ?  I'm not aware of a DTD
allowing IRI in this place apart from my homebrewn
XHTML 1 i18n <http://purl.net/xyzzy/-xhtml1-i18n.dtd>

 Frank

---

From: "Brian Smith" <[email-masked]>
Date: Sat, 15 Mar 2008 07:19:05 -0700

> > RFC 2047 only talks about how to use encoded-string in RFC 2045 
> > messages with RFC 822 headers, not RFC 2616 messages or RFC 2616 
> > headers.
> 
> registered as MIME type message/http.  If MIME RFC 2047 etc. 
> is not more good enough for 2616bis this effort is doomed.  
> "Find something better than MIME while claiming that the 
> result is still related to HTTP/1.1" can't fly.

Even if RFC 2616 messages were message/rfc822 messages (which they
aren't, RFC 2616 has a whole section explaining that they are similar
but different), look at RFC 2047:

    * An 'encoded-word' may replace a 'text' token 

    * comment = "(" *(ctext / quoted-pair
              / comment / encoded-word) ")"

    * phrase = 1*( encoded-word / word )

    These are the ONLY locations where an 'encoded-word'
    may appear.

<phrase> is not used at all in RFC 2616. <text> is not used, there is
<TEXT>. However, <TEXT> is only used in three places:

    * In <quoted-string>, where the RFC 2047 mechanism
      is not allowed (according to RFC 2047)
    * In <comment>, where it doesn't matter
    * In reason-phrase
    * In field-content, but its BNF is broken and not
      is not useful for actually parsing HTTP header
      fields.

So, if we interpret everything very liberally, then the RFC 2047
mechanism is allowed in the reason-phrase and in comments *only*. 

> Updating and fixing the references, the numerous errata, and 
> the horrible syntax, is precisely what this WG tries.

Adding a new requirement for implementations to support RFC 2231 in any
existing headers is not a compatible change. 

> What are you talking about ?  I'm not aware of a DTD allowing 
> IRI in this place apart from my homebrewn XHTML 1 i18n 

Please re-read what I wrote:

   Atom and HTML 5 (will) allow internationalized link
   relation names, and IRIs as link relation names.

I was talking about the "rel" and "rev" link parameters ("link relation
names"), not the <URI> part:

   relationship   = sgml-name
                  | ( <"> sgml-name *( SP sgml-name) <"> )

Regards,
Brian

---

From: "Frank Ellermann" <[email-masked]>
Date: Sat, 15 Mar 2008 18:08:57 +0100

Brian Smith wrote:
 
> <phrase> is not used at all in RFC 2616.
> <text> is not used, there is <TEXT>.

Yet another reason to fix RFC 2616.

> Adding a new requirement for implementations
> to support RFC 2231 in any existing headers
> is not a compatible change. 

We both missed the RFC 2231 Last Call, it says
"updates: 2045, 2047, ...".  With 2231 < 2616
folks could have noted these problems earlier,
maybe there is no problem.

> Atom and HTML 5 (will) allow internationalized
> link relation names, and IRIs as link relation
> names.

Atom and HTML 5 will find protocols to transport
them, if it's not HTTP/1.1 that's their problem.

A relation is what RFC 2068, 2070, etc. say:

| sgml-name = ALPHA *( ALPHA | DIGIT | "." | "-" )

In an older HTML DTD I found a reference to...
http://tools.ietf.org/html/draft-ietf-html-relrev
...and that draft stated:

| The REL and REV attributes are defined as NAMES
| in the SGML DTD for the HyperText Markup Language
| (HTML). As such, the legal values which may be
| assigned to the REL or REV attribute are zero or
| more name tokens. Name tokens are case-insensitive,
| must begin with an alpha character, may include
| digits (0-9), period or hyphen, and may be 
| separated by spaces.

RFC 2068 got it right.  I can't tell why HTML 4.01
says CDATA instead of NMTOKENS.  It could be a bug
inherited from 3.2, after the HTML i18n (RFC 2070)
bug "NAMES" instead of NAMES (I think, sometimes
DTDs still confuse me).

But it clearly never was an URI, let alone an IRI.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Sun, 16 Mar 2008 20:15:16 +0100

Hi,

our discussion about RFC2047 reminded me of an interop problem that I 
encountered years ago when serving content "for download", using the 
Content-Disposition header in general and the "filename" parameter in 
particular.

The only documented and partly implemented solution for I18N is RFC2231, 
which is supported by FF and Opera (atleast the essential parts). IE 
doesn't support this, and Safari apparently doesn't yet (but has an 
issue open regarding this).

I raised the topic on the HTML WG's mailing list, but so far nobody 
seems to believe it's something they should include (quite surprisingly 
to me...).

As RFC2616 doesn't really include Content-Disposition, nor requires 
RFC2231, it doesn't seem to be in our charter to do something.

Any suggestions about what should be done here; except for waiting that 
the remaining UA vendors just implement this?

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sun, 16 Mar 2008 20:30:54 +0100

...speaking of which.

In Part3, we currently refer to both RFC1806 (the original definition) 
and RFC2183 (which updated RFC1806).

Wouldn't it make sense to drop the references to the historical document?

Also, I can't help noticing that in Section 8.2 
(<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p3-payload-02.html#rfc.section.8.2>) 
we say

"8.2 Content-Disposition Issues

[RFC1806], from which the often implemented Content-Disposition (see 
Appendix B.1) header in HTTP is derived, has a number of very serious 
security considerations. Content-Disposition is not part of the HTTP 
standard, but since it is widely implemented, we are documenting its use 
and risks for implementors. See [RFC2183] (which updates [RFC1806]) for 
details."

...which is really vague. Are there more considerations than those 
mentioned in <http://tools.ietf.org/html/rfc2183#section-5>?

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 00:11:22 +0100

> RFC2231, it doesn't seem to be in our charter to do something.

The MIME registry points to RFC 4229, which points to RFC 2616, and
from RFC 2616 I find RFC 1806 as amended by RFC 2183.  For obscure
reasons RFC 2183 was updated by RFC 2184, and later by RFC 2231.  
RFC 2184 was later obsoleted by RFC 2231.

That's rather confusing, but with 1806 < 2183 < 2184 < 2231 < 2616
I'd say it's a bug in RFC 2616.  FWIW, RFC 2183 and RFC 2231 are PS,
RFC 1806 is or was an experiment.

What is the idea of a Content-Disposition header field in HTTP/1.1 ?
Why should a filename parameter make more sense than the relevant
part of the URL, with its own security considerations in RFC 2616 ?

 Frank

---

From: Martin Duerst <[email-masked]>
Date: Sat, 15 Mar 2008 09:27:18 +0900

(a side issue only)

At 21:57 08/03/14, Julian Reschke wrote:

>It seems the only way to improve RFC-2047 would be by introducing a new encoding that is sane. Such as:
>
>"Any octet sequence starting with EF BB BF (the UTF-8 BOM) is to be interpreted as Unicode, encoded in UTF-8."

If we are speakind about RFC 2047 itself, then indeed no special sentinel
(such as an UTF-8 BOM) would be neeeded. Any byte with the most significant
bit set would be enough. Also, even for HTTP, mixing iso-8859-1 and UTF-8
might be fine in practice, because it's very easy to distinguish them.

But all this would only make the mess bigger. It's much better to sort this
out on a per header base (unless we can confirm that the current cruft isn't
used at all in practice, which would then allow to go for UTF-8 in all
cases where we need something more than US-ASCII).

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sat, 15 Mar 2008 08:26:48 +0900

At 19:26 08/03/14, Julian Reschke wrote:
>> Changing the allowable characters in a protocol element is a *big* change, and there is not an interoperability gain to doing so.
>> There is also not a functionality gain; it is possible (if not pretty) to serialise other characters into HTTP headers.
>
>I think the key question here: is that implemented in practice? (In particular, which encoding?) If yes, fine (and maybe let's document what works). But if not...?

Exactly. I'm still waiting for somebody to point to a server that
actually serves iso-8859-1 data, or even more, RFC 2045-encoded
data.

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sat, 15 Mar 2008 08:40:35 +0900

>complete "PDU" (header + body).

It's all those protocol fields where you need human-readable
text. The Subject: of an email very clearly qualifies as
text, so it's not only body.

>would break the complete installed base.

That may have been the case for USEFOR. Can you show how
allowing e.g. new HTTP header fields to use UTF-8 would
break anything in the installed base?

>as in mail, or roughly as in HTTP (NetNews is 8bit clean):

There is a big difference between MIME (ASCII+RFC2045)
and HTTP (iso-8859-1+RFC2045).

>it offers no indication of language tags (BCP 47).

I have yet to see a case where the absence of language information
in a header is a problem in practice. Do you know of any?

Regards,   Martin.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 17 Mar 2008 15:36:42 +1100

What would it prove if we found no-one? That they aren't useful  
encodings, or that there aren't use cases for non-ASCII characters in  
headers?

Cheers,

---

From: Martin Duerst <[email-masked]>
Date: Mon, 17 Mar 2008 13:42:35 +0900

>>>>> HTTP is no "new" protocol, like mail or news:  2821bis and 2822upd and FWIW RFC.usefor-usefor don't "violate" any IETF policy.  But atom and xmpp were new, a different situation.
>>>> RFC 2277 applies to any updates to an existing protocol, as far as I can tell.
>So HTTP uses US-ASCII for names; that includes method names, header names, relation names, whatever. No problem so far.

Names in RFC 2277 doen't refer to things such as method names, where
there is only a very small selection. In the above quote, these go under
'protocol'.

'names' above refers to things like domain names, email addresses, and
IRIs, where there is a large number of them, and they may be related to
names of people, companies, or organizations. At the time RFC 2277 was
written, it was not at all clear what should happen with these. I guess
some people felt they were going to be needed sooner or later, while
others thought they were unnecessary, impossible, dangerous, or whatever.

Regards,   Martin.

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 06:21:17 +0100

> text, so it's not only body.

Using RFC 2047 for <unstructured> header fields bodies is
straight forward, no conflict with RFC 2277.  Apparently HTTP
has no unstructured fields, so that might be irrelevant.

Brian has already mentioned comments, using RFC 2047 "as is"
(no 2231 parameter-folding) should be no issue for 2616bis
comments.  I've no idea why anybody would want 4646 language
tags in a comment, but that part of RFC 2231 would be also
straight forward.  And if the other side has no clue what an
encoded comment with language tags is, nothing will break -
or rather I'm not aware of scenarios where comments could be
critical.

> Can you show how allowing e.g. new HTTP header fields to
> use UTF-8 would break anything in the installed base?

I'm sure that the mentioned HTTP/1.0 browser didn't support
UTF-8, and I'm sure that I have not yet tested an IE6 plugin
claiming to offer some kind IDN of support.  Hard to prove a
negative, but FF2 didn't like non-UTF-8 <ipath> in your test
suite, <ihost> and UTF-8 worked, from that I *guess* UTF-8 can
work for this browser.  

2616bis is supposed to work with any server and UA since the
times of RFC 2068, that's why I think we can at best get rid
of the "default Latin-1", but not simply replace it by UTF-8.

Your heuristics to distinguish Latin-1 and UTF-8 depends on
finding 0x80..0x9F (=> potential UTF-8 trail byte, likely no
C1), 0xC0..0xC1 (=> can't be UTF-8), or 0xFE..0xFF (ditto),
and further fine tuning opportunities for the STD 63 UTF-8.

But it's not guaranteed to work for short strings in a header
field, and there's no way to put it into old servers and UAs
supporting only Latin-1.  

> There is a big difference between MIME (ASCII+RFC2045) and
> HTTP (iso-8859-1+RFC2045).

That's why I'd prefer to get rid of "default Latin-1", going
from US-ASCII to UTF-8 later (after 2616bis) is hopefully
simpler than from Latin-1 to UTF-8.  Using a BOM for this
magic is dubious, 0xEFBBBF is valid Latin-1.  

> I have yet to see a case where the absence of language 
> information in a header is a problem in practice. Do you
> know of any?

No.  It would need to be something that's displayed, parsed,
voice output, dunno, anything where the language or script
helps.  Excluding eur-EU, mis-EA, zxx-DG, und-IC, etc. <eg>

 Frank

---

From: Martin Duerst <[email-masked]>
Date: Mon, 17 Mar 2008 14:15:58 +0900

>Personally, I am *very* -1 on doing this.

I don't make this proposal because I think UTF-8 should rule
the world, but because over the years, I have received quite
a few requests along the line of "if we define this new HTTP
header, can't we just say it uses UTF-8?".

>change,

First, most of the HTTP implementations treat this stuff as
bytes-in-byte-out, nothing else. Second, the allowable characters
aren't changed, it's just a different way of encoding.

>and there is not an interoperability gain to doing so.

There is an interoperability gain with the rest of the infrastructure
(APIs,...). This is serious. UTF-8 is used widely these days, telling
a developer that something is in UTF-8 means it can get implemented
quickly. That's not at all the case for an ugly and not particularly
well-defined mish-mash of iso-8859-1 and RFC 2047.

>to serialise other characters into HTTP headers.

In the old days, people were somehow used to this character encoding
stuff being complicated. But these days, there are much more straight-
forward solutions, and tolerance and understanding for this kind of
cruft is getting lower and lower. The chances that something gets
implemented if it's just UTF-8, but not if it's some ugly cruft,
are considerable. At the end of the day, that results in a net
functionality gain.

>data,

Yes indeed. For most part, that's the way it should be. But in some
cases, the complications involved may have been an additional reason
for why it's not done. That's not the way it should be, and we can
fix it.

>   http://lists.w3.org/Archives/Public/ietf-http-wg/2007JulSep/0323.html

I have reread that.

I don't think there is anything seriously prohibiting using raw
UTF-8 in new headers (when more than US-ASCII is needed). The
greatest number of implementations will just treat it as byte-in-byte-out.

An occasional rare implementation may actually interpret the data
as iso-8859-1 and convert it to some internal representation, and
do the reverse on the way back out, but that shouldn't really hurt.
(Character encoding converters usually map the C1 area in iso-8859-1
to the C1 area in e.g. Unicode, if they not just assume windows-1252).

In an even rarer case, that data may then actually be displayed,
resulting in some kind of garbage on screen. But for these cases,
we could just claim that we are doing an encoding on top of
iso-8859-1. Eventually, these few implementations will catch up.

For those implementations that really care about the new header,
things will be a lot, lot easier than if we stayed with the
iso-8859-1+RFC 2047 cruft.

Regards,   Martin.

>> That's related to <http://www3.tools.ietf.org/wg/httpbis/trac/ticket/74 >.
>>

---

From: Martin Duerst <[email-masked]>
Date: Mon, 17 Mar 2008 14:19:32 +0900

>|  href    %URI;     #IMPLIED  -- URI for linked resource --

We have had that discussion before, in another location. The
conclusion is that the HTML DTD allows prety much anything
in attributes such as href, including spaces. The fact
that the entity is named "URI" doesn't mean anything for
an SGML validator. That's the same as trying to force
a variable to be of type "int" by naming it "integer".

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Mon, 17 Mar 2008 14:22:37 +0900

>encodings, or that there aren't use cases for non-ASCII characters in  

Good question. Maybe that the use cases for non-ASCII characters
aren't strong enough to justify the uglyness of the encoding?

Regards,    Martin.

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 07:34:09 +0100

Martin Duerst wrote:
 
> We have had that discussion before, in another location.

On the validator list, and a few days ago somebody posted
a similar question about width and height:

<http://thread.gmane.org/gmane.org.w3c.validator/10560>
 
> the HTML DTD allows prety much anything in attributes

That does not mean that only the DTD minus comments defines
what HTML is, or that URIs are more or less "anything goes".

In another article you wrote:

| we could just claim that we are doing an encoding on top
| of iso-8859-1. Eventually, these few implementations will
| catch up.

If you seriously want this you can use "UTF-4", it encodes:
u+0000..u+007F as 0x00..0x7F (= US-ASCII)
u+00A0..u+00FF as 0xA0..0xFF (= visible iso-8859-1)

u+0080..u+009F as 0x829890 .. 0x82999F (encoded C1)
u+0100 ff.     as 0x83919090 ff. (excl. surrogates)

0x8N is the lead byte for N trail bytes 0x90..0x9F, each
enconding one hex. digit of the (UTF-32BE) codepoint, the
first trail byte cannot be 0x90 (overlong encoding).

In theory, because "UTF-4" doesn't use the minimal C1 set
SS2 and SS3 (0x8E and 0x8F), you can even retrofit it in
an ISO 4873 framework (like ISO-8859).

I could post one of these proposals (XHTM1-i18n or UTF-4)
as I-D if it helps - don't laugh, I saw the "UTF-5" I-D ;-)

 Frank
-- 
<URL:http://purl.net/xyzzy/home/test/utf-4.xml>

---

From: Martin Duerst <[email-masked]>
Date: Mon, 17 Mar 2008 15:23:46 +0900

>not cut it if the alternatives are HTTP/1.2 UTF-8 or a HTTP/1.1

Well, the alternative might be HTTP/1.1 with new headers in UTF-8.

One can discuss whether this is strictly according to the rules
for going to Standard, but then, as Paul has said, the IETF isn't
a machine. Last minute fixes before going to Standard have happened.
Even for Standard, or especially for Standard, the future is longer
than the past.

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Mon, 17 Mar 2008 15:19:21 +0900

At 23:14 08/03/14, Frank Ellermann wrote:
>side likes some C1 controls (NEL, SS2, SS3).

If any party participating in a HTTP transaction is getting
something wrong with iso-8859-1, the most likely thing is
that they interpret iso-8859-1 as windows-1252.

>But I guess you

No idea what BCP 113, RFC 4333, "The IETF Administrative Oversight
Committee (IAOC) Member Selection Guidelines and Process" has to
do with UTF-8 or iso-8859-1.

Regards,   Martin.

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 17 Mar 2008 08:47:42 +0100

> aren't strong enough to justify the uglyness of the encoding?

For style sheets attached through HTTP you'd want non-ASCII characters. So  
you can make meaningfull titles for content that can't be expressed with  
just ASCII characters:

   Link <test>;rel=stylesheet;title="foo"
   Link <alternate>;rel=stylesheet;title="bar"

(imagine foo and bar are not ASCII or latin-1 compatible)

-- 
Anne van Kesteren
<http://annevankesteren.nl/>
<http://www.opera.com/>

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 09:32:40 +0100

Anne van Kesteren wrote:

> For style sheets attached through HTTP you'd want non-ASCII
> characters. So you can make meaningfull titles for content
> that can't be expressed with just ASCII characters:

>|  Link <test>;rel=stylesheet;title="foo"

Warning, stupid question:  What is wrong with link *elements*
for this purpose, and do browsers support link titles in the
elements, let alone in Link header fields ?  As far as they
support titles in Link header fields, can they handle MIME,
and if not why not ?

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 17 Mar 2008 09:47:44 +0100

> RFC 2184 was later obsoleted by RFC 2231.

Oh, I missed the 2184 step :-)

Does anybody know why RFC2183 does not obsolete RFC1806?

> RFC 1806 is or was an experiment.

Let's summarize:

- RFC2616 should stop talking about RFC1806, this is just historic 
information
- RFC2616 will have to continue to document Content-Disposition as 
"additional feature"
- RFC2616 includes some vague statements about security risks, but 
doesn't say what they are; should it do so, or are they adequately 
documented in RFC2183? (in which case we it could just point to them)
- RFC2183 already requires support for RFC2184, later obsoleted by 
RFC2231; so recipients implementing Content-Disposition as per RFC2183 
will need to implement RFC2231 as well.

> part of the URL, with its own security considerations in RFC 2616 ?

Some reasons I can think of right away...:

1) Sometimes the last segment of the URL has no relationship with what 
the filename should be.

2) Even if it does, and that segment contains URI-escaped non-ASCII 
characters, there's no standard way to decode that into a character 
sequence (is it Latin-1? UTF-8? Something else?)

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 09:42:37 +0100

Martin Duerst wrote:

>> I guess you mean something more sophisticated like BCP 113.
 
> No idea what BCP 113, RFC 4333, "The IETF Administrative
> Oversight Committee (IAOC) Member Selection Guidelines and
> Process" has to do with UTF-8 or iso-8859-1.

Nor me, lack of caffeine or something.  My next attempt was
RFC 5237, but BCP 37 "IANA Allocation Guidelines for the
Protocol Field" also isn't related to UTF-8 and iso-8859-1.

Eventually I tried RFC 5137 arriving at BCP 137:  "ASCII 
Escaping of Unicode Characters".  That is what I wanted :-)

Using hex. NCRs for Link titles might be a good idea, after
all they are also supposed to work in the link attributes
of link elements.  But I'm worried about picking different
techniques depending on the header field.

 Frank

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 17 Mar 2008 09:41:18 +0100

On Mon, 17 Mar 2008 09:32:40 +0100, Frank Ellermann  
<[email-masked]> wrote:
> for this purpose,

One use case is having a collection of HTML documents which you do not  
want to modify but still want to style.

> and do browsers support link titles in the
> elements, let alone in Link header fields ?

Yes, it's called the title attribute. Haven't tested the feature with Link  
headers yet, but I think it will work.

> As far as they

I don't understand this question.

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 10:23:50 +0100

> 1) Sometimes the last segment of the URL has no relationship with
> what the filename should be.

Hm...  I could also argue that wild and wonderful filename proposals
might be irrelevant for the client depending on the file system.  If
I store a file my browser lets me pick where, and under which name,
and if I feel like it I put it on the 8+3 FAT partition (example).

I have no partition where case is significant (or maybe I do, but I
have no driver to mount the Debian partition from W2K ;-)  For all
folks laughing about NTFS and HPFS oddities I have one number:  *14*

> sequence (is it Latin-1? UTF-8? Something else?)

Perecent-encoded URIs are always UTF-8, unless they can't be UTF-8,
ignoring pure US-ASCII for the moment, US-ASCII is a proper subset.

That's a consequence of RFC 3987, nothing else makes sense, but we
are not supposed to talk about it:  RFC 3987 does NOT update 3986,
except where it does in its own sneaky way (nothing wrong with it,
but some really old ftp-URIs etc. might strongly disagree ;-) 

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 17 Mar 2008 10:35:34 +0100

Anne van Kesteren wrote:

>> What is wrong with link *elements* for this purpose,

> One use case is having a collection of HTML documents
> which you do not want to modify but still want to style.

Okay, if browsers really support this, I'm not aware that
one of the browsers I use(d) does (or did).  But I'm lost
why such stylesheets would need an I18N title.

>> do browsers support link titles in the elements, let
>> alone in Link header fields ?

> Yes, it's called the title attribute. Haven't tested the
> feature with Link headers yet, but I think it will work.

ACK, I know the attribute, my question was about titles in
the header field.

>> As far as they support titles in Link header fields,
>> can they handle MIME, and if not why not ?

> I don't understand this question.

Will Link: ... title*=utf-8''This%20is%20%2A%2A%2Afun%2A
work ?  Adapted from an example in RFC 2231 section 4.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 17 Mar 2008 10:49:35 +0100

> and if I feel like it I put it on the 8+3 FAT partition (example).

Of course it's only a suggestion.

But tell an Asian customer that they have to retype the name whenever 
they do "Save As".

> folks laughing about NTFS and HPFS oddities I have one number:  *14*

NTFS in fact is case-sensitive, but most Windows APIs hide that. (In 
doubt, try Microsoft's Posix stuff).

> but some really old ftp-URIs etc. might strongly disagree ;-) 

That's not relevant here.

Servers still encode non-ASCII characters using other character sets.

For instance (as far as I recall), for Apache on Unix it simply depends 
on what character set is used in the filesystem, which may depend on the 
  locale of the user who created that file (see, for instance, 
<http://greenbytes.de/tech/webdav/draft-reschke-webdav-url-constraints-latest.html#rfc.section.5.1.2>).

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 17 Mar 2008 10:54:26 +0100

> why such stylesheets would need an I18N title.

I think both FF and Opera do.

For CSS, I would expect the title to appear in the CSS style sub menu.

> work ?  Adapted from an example in RFC 2231 section 4.

Good question. Somebody should try :-)

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 17 Mar 2008 14:27:29 +0100

> Good question. Somebody should try :-)

I just did.

I only have two UAs that use the Link header (FF and Opera), and neither 
of them seems to understand any encoding here:

- when the title parameter is encoded RFC2231-style, the Link doesn't 
show up,

- when it is encoded RCF2047-style, it does show up but it isn't getting 
decoded.

BR, Julian

---

From: Martin Duerst <[email-masked]>
Date: Tue, 18 Mar 2008 13:31:20 +0900

At 22:27 08/03/17, Julian Reschke wrote:

>I only have two UAs that use the Link header (FF and Opera), and neither of them seems to understand any encoding here:
>
>- when the title parameter is encoded RFC2231-style, the Link doesn't show up,
>
>- when it is encoded RCF2047-style, it does show up but it isn't getting decoded.
>
>BR, Julian

What about trying other things, such as:
- The system encoding of the system you use (or the encoding
  of your locale,...)
- The encoding of the html page
- UTF-8
(some or all of the above may be identical)

Regards,   Martin.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 18 Mar 2008 12:33:08 +0100

> (some or all of the above may be identical)

I did one additional test: added an UTF-8 encoded link header, and 
served it with text/html, encoded in UTF-8.

Both FF and Opera display the link, but do not decode the UTF-8 sequence 
(so it assumes it's a single-byte encoding).

BR, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 18 Mar 2008 15:28:03 +0100

> RFC2231, it doesn't seem to be in our charter to do something.

Correct. Content-Disposition is not part of HTTP/1.1 even if commonly
implemented.

> the remaining UA vendors just implement this?

I don't see much else that can be done. This header and how to apply
I18N to it is defined in the MIME world. And there the meaning of
Content-Disposition is defined by RFC2183 (header) + RFC2231 (I18N) +
errata.

We could mention 2231 in the informal reference to 2183 however, but
given that it's an informal reference and also official standards track
update addition to 2183 I don't see why this would be needed. But you
know the rules and guidelines on how to reference other RFCs better than
me.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 18 Mar 2008 16:08:07 +0100

> helps.  Excluding eur-EU, mis-EA, zxx-DG, und-IC, etc. <eg>

In HTTP about the only unstructured protocol field really meant to be
displayed is the human intended status code description. Where most uses
the suggested texts found in 2616.

But the authentication schemes (2617) need to tackle encodings one way
or antother. Today HTTP authentication only allows iso-8859-1 in
usernames or passwords.. 

Well, the character set is not at all defined for digest passwords, but
iso-8859-1 for all else, inherited from the definition of TEXT in 2616.

Regards
Henrik

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 18 Mar 2008 16:31:46 +0100

Henrik Nordstrom wrote:
 
> the authentication schemes (2617) need to tackle encodings one way

Yes, that's bad, SASLPREP UTF-8 or better net-utf8 would be nice.

The SASL folks apparently decided to delay their deprecation of
DIGEST-MD5 until they have something better than CRAM-MD5, I'm a
bit lost who wants what why, but there is a strong consensus for
KISS, with various ideas what KISS precisely means... ;-)

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 24 Mar 2008 10:23:45 +0100

Julian Reschke wrote:
 
> tell an Asian customer that they have to retype the name whenever 
> they do "Save As".

Beats me, I guess that these platforms allow ASCII file names among
others.  I18N of file systems is an interesting topic, but for HTTP
I think we can simply say that RFC 2231 is a possible solution, and
finding something better is no job for a HTTP/1.1 2616bis.  

For a similar problem compare EAI vs. 2821bis:  2821bis + 2822upd
stick to US-ASCII, where EAI needs or only wants something better
it has to say so.  The wannabe-MIME update in EAI is controversial.

8bit clean, e.g. NetNews, or Latin-1, e.g. HTTP/1.1, isn't the same
as "assume UTF-8", unfortunately.  For real net-utf8 support we'd
have to make sure that applications catch any attempts to smuggle
in UTF-7, CESU-8, or overlong encodings in historic UTF-8 variants.
We'd also have to talk about NFC for net-utf8, interesting topics,
but I don't see this as a part of the 2616bis job.

> NTFS in fact is case-sensitive, but most Windows APIs hide that.
> (In doubt, try Microsoft's Posix stuff).

Of course I can have a file AAbb, but I can't have aaBB in the same
directory, and if I look for AABB I find an existing AAbb.  AFAIK
that's not the same idea of case-sensitive as for *NIX file systems
- unless I missed something important.  I'm too lazy to reboot and
test it ;-)

Whenever we get down to individual servers like apache, individual
browsers like IE, or here individual file systems, I think we miss
the point:  2616bis has to specify something working for everybody
based on 2616.  And where apache, IE, HTML5, IIS, atom, etc. do
something else it's their problem.  If they all happen to agree on
what they do, we are free to align the theory with common practice.
And I'll scream if it breaks the oldest curl or wget I can find :-)

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Mar 2008 11:51:56 +0100

> Beats me, I guess that these platforms allow ASCII file names among

And do you really believe that this is acceptable for the average customer?

> finding something better is no job for a HTTP/1.1 2616bis.  

I didn't propose to find something better.

RFC2231 (actually, a small profile of it) works just fine; it just seems 
that some UA vendors (MS, Apple) believe they don't need to implement it.

> Of course I can have a file AAbb, but I can't have aaBB in the same

Yes, you can. You just need to use the proper API to access the file system.

> And I'll scream if it breaks the oldest curl or wget I can find :-)

Well, we have an I18N problem, and a solution for it (RFC2231). Some 
vendors implement it (FF, Opera), others don't. The question is: can we 
improve the documentation of Content-Disposition so that it becomes 
clear that RFC2231 support is required? Furthermore, should we simplify 
things by stating precisely what parts of RFC2231 are really needed (the 
line folding is not needed, and support for a single charset (UTF-8) 
would be sufficient)?

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 24 Mar 2008 16:52:47 +0100

Julian Reschke wrote:

> do you really believe that this is acceptable for the
> average customer?

Don't know, I never try to impose a filename of my choice
on other folks in a MIME or HTTP Content-Disposition, I
would consider it on the border of net abuse to try this,
because I have no idea what their file system supports.

> RFC2231 (actually, a small profile of it) works just
> fine

Good, then let's say so.  In the USEFOR WG I tried about
a year to do anything, anything at all, to avoid RFC 2231,
but eventually I had to accept that all my proposals made
no sense for various reasons.

 [NTFS and case sensitive file names] 
> Yes, you can. You just need to use the proper API to

I tested it now with pax.exe, you're right:

| D:\Programme\bin>pax -v -f test
| -rwx---r-x  0 Frank    Kein           30 Mar 24 16:26 AAbb
| -rwx---r-x  0 Frank    Kein           30 Mar 24 16:28 aaBB
| USTAR format archive
|
| D:\Programme\bin>pax -r -f test
|
| D:\Programme\bin>dir aa*
| Datenträger in Laufwerk D: hat keine Bezeichnung.
| Datenträgernummer: 1C81-013D
|
| Verzeichnis von D:\Programme\bin
|
| 2008-03-24  16:26                   30 AAbb
| 2008-03-24  16:28                   30 aaBB
|               2 Datei(en)             60 Bytes

> should we simplify things by stating precisely what 
> parts of RFC2231 are really needed (the line folding
> is not needed, and support for a single charset (UTF-8) 
> would be sufficient)?

Let's take 2231 "as is", implementations not supporting 
odd charsets can ignore the proposed file name, limiting
it to UTF-8 could be a bad idea if it's actually less
(us-ascii, latin-1, etc.) and the target platform can
only do less.  The strict length limits for RFC 2047 are
no issue for RFC 2231 parameters, or are they ?

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Mar 2008 17:19:32 +0100

> because I have no idea what their file system supports.

Well.

In practice, file systems support the national characters people need. I 
suppose that's Unicode everywhere, although other encodings may be used 
in practice (such as ISO-Latin in Europe, or Big-5 etc in Asia; dunno 
for sure).

Specifying the *original* file name of a document has absolutely nothing 
to do; it provides essential additional information along with an HTTP 
resource.

> (us-ascii, latin-1, etc.) and the target platform can

If it's "less", you don't neede RFC2231-encoding anyway.

> no issue for RFC 2231 parameters, or are they ?

No, they aren't. But I've seen people using the complexity with respect 
to line folding as argument not to support RFC2231 at all. Thus it 
probably would be wise to state the profile that is actually needed in 
the context of HTTP.

BR, Julian