Subject: Re: [W3C BPWG] HTTP header fields X-* and normal ones / question

From: <[email-masked]>
Date: Wed, 21 Jan 2009 22:44:57 +0000

required some more context. I hope the following 
additions will make things clear.

> That's even more insane.  If the request is being 
> rewritten then it is not the same request -- the old 
> request fields should not be sent at all.
> The inbound server doesn't care about these values 
> and sure as hell doesn't want to parse them as part 
> of the request. As such, your deployed proxies are
> irrelevant.

Application servers in the mobile Web are highly dependent
on the Accept-* and User-Agent fields to generate content
that matches the capabilities of the terminal issuing the
request. In particular, the User-Agent serves as identifier
to retrieve information about, for instance, screen dimensions,
acceptable page size, browser quirks, from terminal capability
databases.

Content transformation proxies that intend to convert desktop
Web content to mobile Web content alter these capability
fields to appear as a normal desktop browser (e.g. IE, Firefox), 
but send the backup fields (X-Device-*) along for two reasons:

1. It is impossible a priori to determine reliably whether 
the target server returns desktop content or mobile content;
and
2. servers of mobile Web applications require the original
capabilities to adjust the content for the end terminal.

If a mobile Web server receives the modified request without
the backup fields, it will either fail to return a proper
result (because the terminal appears to be a desktop), or
produce a generic desktop-oriented page instead of a much
more adequate mobile-optimized one.

The question whether the transformation proxies should or 
should not modify the HTTP header fields in the first place
is another question; we are considering the scenario where
they do here.

>> 3) Questions to the IETF.
>>
>> 3.a: What is the IETF procedure, policy, standard, or 
>> best practice regarding the promotion of X- header 
>> fields to normal, registered (provisionally or
>> permanently) header fields?
>
> Never use them in deployed software. The X-prefix is 
> for private use in experiments that are NOT EXPOSED TO 
> THE INTERNET.

This just does not correspond to what is happening on 
the Internet.

Several browsers, notably Microsoft IEMobile, OperaMini, 
and Skyfire do send several X- prefixed HTTP fields that 
are not private, are fully exposed, documented, and that 
are supposed to be used by application servers.

As another example, the Open Mobile Alliance has a published
standard, UAProf, which relies on X-* prefixed HTTP header 
fields (X-Wap-Profile-Warning, X-Wap-Profile-Diff, and 
X-Wap-Profile). These HTTP header fields serve to communicate
terminal descriptions between end-user devices and servers.

The UAProf standard is widely deployed and heavily used on the
Internet, and can be found in the following references:
current version: 
OMA User Agent Profile, OMA-UAProf-v2_0-20030520-C, 2003-05-20.
initial version:
WAG UAProf, WAP-248-UAPROF-20011020-a, 2001-10-20.

I do not know why the OMA did not register proper HTTP 
fields in the IANA registry, but this has been the state
of affairs since 2001.

>> 3.c: What is the procedure, policy, standard or best 
>> practice of the IETF regarding the deprecation and 
>> discontinuance of HTTP header fields?
>
> Just stop using them.

But what if they have been registered? Would that not amount
to leaving chaff behind and cluttering the registry with
obsolete definitions?

Regards

Eduardo Casais
areppim AG
Bern, Switzerland

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Thu, 22 Jan 2009 09:51:14 -0700

> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of [email-masked]
> Sent: Wednesday, January 21, 2009 2:45 PM
> To: [email-masked]
> Subject: Re: [W3C BPWG] HTTP header fields X-* and normal ones /
> question
>

This makes no sense. What you describe is:

1. Mobile Client --(request with device specific headers)--> Proxy
2. Proxy --(request with desktop/generic headers)--> Server
3. Server --(response with desktop/generic/?? representation)--> Proxy
4. Proxy --(transformed response into device specific)--> Mobile Client

The proxy should keep track of what the client requests, get whatever content from the server, adjust that content to the capabilities of the device (regardless of what capabilities the server intended to support), and send it back. The proxy's role is to transform the representation to a device-friendly one, so why does it need to preserve the original headers?

> capabilities to adjust the content for the end terminal.

The proxy should keep track of what was requested locally. If the proxy is just a transformation agent, it should ask the server for the most appropriate representation for the device, and if the response isn't ideal, transform it. If the proxy also caches responses for a wide range of devices, it must always ask for the most generic representation and cache both that, and the transformed response per device type.

> is another question; we are considering the scenario where

Why?

EHL

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jan 2009 16:29:19 -0800

> acceptable page size, browser quirks, from terminal capability

Yeah, so don't change that information.

> capabilities to adjust the content for the end terminal.

A content transformation proxy does not want the "server of
mobile web applications" to return content that is adjusted
specifically for that terminal.  If they did, they wouldn't need
to change all of the request fields in the first place to
pretend not to be that terminal.  They would just forward the
original request without change, or adjust the parameters in
the modified request such that device-specific media types
are preferred over standard media types.

A content server DOES NOT want to know about some request
parameters other than what it is already designed to vary
the representation.  What you suggest would require those
servers to also Vary the content by these X-Device header
fields and specifically not comply with the standard fields
for content negotiation.  In short, you are asking that servers
fail to correctly handle the request as specified and make
caching far less effective at the same time, in addition to
doubling the latency on request parsing due to these useless
header fields being sent to servers that have no interest
in reading them.

In short, whatever problem you think you are trying to solve
will not be solved by the suggested solution, so asking further
about how this bad idea might be registered is pointless.

> is another question; we are considering the scenario where

The answer to that consideration is that the transformed
request must be considered on its own, as requested, and
the fact that it originated from a mobile device is irrelevant.
The origin server is only responsible for responding to the
request as it was requested by the proxy.  If the proxy is
making the wrong transformation, then fix the proxy.

> are supposed to be used by application servers.

And they will never be standardized for that reason.  In fact,
I'll happily add code to strip them out of requests before
the application server has a chance to see them.

> terminal descriptions between end-user devices and servers.

The OMA is not a standards organization.  I don't recommend
following any of their specifications.

> fields in the IANA registry, but this has been the state

Again, I will happily block them if that will solve the problem
of it being considered precedence for not reading the relevant
standards regarding registration.

> to leaving chaff behind and cluttering the registry with

They aren't registered yet.  Even if they were, I'd rather have
a crufty bunch of useless fields in the registry than a crufty
bunch of useless fields on the wire on every request.

....Roy

---

From: <[email-masked]>
Date: Thu, 22 Jan 2009 20:36:07 +0000

optimal mode of operation of transformation proxies -- 
this is not the topic of the original request. 

The question is simple:

There is some technology deployed in the Internet that
makes use of X-* header fields. The W3C is considering
formalizing best practices for this technology,
introducing normal (i.e. non X- prefixed) header fields, 
and registering them at IANA. 

However, there are already commercial systems in 
production use that send these X- prefixed header
fields. Hence, we realize there might be a migration
period where both X- prefixed and non-prefixed fields
may coexist. 

Furthermore, RFC3864 specifies a procedure to register
HTTP header fields. It also pre-defines the set of 
statuses that a header field can have ("standard",
"informational", "historic", etc). There is an X-
prefixed field registerd (X-Archived-At) and marked
as "deprecated", and a corresponding non-prefixed
field Archived-At, permanently registered and marked
"standard". 

All this seems to indicate that the IETF has devised
a life-cycle management scheme for header fields. We
would be grateful to know more about it, especially
in the view of handling the coexistence of X- and 
non-X- prefixed fields, and the phasing out of 
deprecated fields.

As for how transformation proxies work, and why these
strange manipulations of HTTP header fields take place,
this would require a detailed exposition of their 
peculiarities, of the characteristics of mobile browsers, 
and of the specific practices of application development 
in the mobile Internet -- and this is not the right
forum for this.

Best regards

Eduardo Casais
areppim AG
Bern, Switzerland