Subject: Questions about draft-abarth-mime-sniff-00

From: Lisa Dusseault <[email-masked]>
Date: Mon, 6 Apr 2009 09:35:01 -0700

After talking with people at IETF SFO about the reality of MIME  
sniffing, I got around to reading this draft and considering how to go  
forward with it.  Of course I have a few questions too.

1.  Doesn't the threat model of mime sniffing need to assume that some  
servers are malicious?  the last paragraph of section 1 implies that  
without server cooperation, which implies trusting the server, the  
sniffing procedure can lead to security problems.

2.  "Extensions must not be used for determining resource types for    
resources fetched over HTTP."  why is this a needed requirement?  And  
should there be an exception for IETF Standards Track extensions that  
offer more information on resource types?

3.  Having a fixed table of types with patterns to look for, doesn't  
fully address why we will always have content sniffing: new content  
types are created that servers don't recognize, but a client could be  
updated to recognize.  Should the table simply be a starter set and  
clients can extend at will?  Or is there reason (and a reasonable  
way)  to create an IANA registry for new patterns/types?

4.  Are there any best-practice guidelines for working with users?   
E.g. allowing a user to choose "text/html" for unmarked content might  
be a security hazard.  We don't want specific user interface  
requirements, but this document seems like a good place to extend  
security considerations to getting input from users, if there are such  
guidelines.

Thanks,
Lisa

--- Scanned by M+ Guardian Messaging Firewall ---
Messaging Architects sponsors The Spamhaus Project.

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 12:57:12 -0700

Hi Lisa,

On Mon, Apr 6, 2009 at 9:35 AM, Lisa Dusseault
<[email-masked]> wrote:
> servers are malicious?  the last paragraph of section 1 implies that without
> server cooperation, which implies trusting the server, the sniffing
> procedure can lead to security problems.

In most cases, when the server is malicious, the server can simply
specify whatever type it likes in the Content-Type header and doesn't
need to leverage the browser's content sniffing algorithm.  That said,
there is a corner case where it's important to think about a malicious
server: when someone (say a proxy or a firewall) is making a decision
based on the media type of HTTP responses.  For example, a firewall
might wish to block all images (for whatever reason) and then might be
sad when an HTTP response is sniffing as an image.

There isn't really a way to tweak the content sniffing algorithm to
mitigate these issues.  Instead, we win here by having every user
agent that performs sniffing use the same algorithm.  That way,
vendors of proxies and firewalls can take sniffing into account (if
they choose) when deciding which responses to block or transform.

> resources fetched over HTTP."  why is this a needed requirement?  And should
> there be an exception for IETF Standards Track extensions that offer more
> information on resource types?

This requirement is not strictly necessary, but using the extension to
sniff is problematic from a security point of view because the
extension can often be chosen by an attacker.  For example, by
default, PHP let's the attacker choose an arbitrary extension.  In
particular, suppose that example.com uses a PHP script to retrieve
user avatars from its database:

http://example.com/avatar.php?UID=3843

By default, this server will also respond with the same content (which
might be controlled by the attacker) when the user's browser requests
this URL:

http://example.com/avatar.php/foo.html?UID=3843

If we take the extension into account, we're more likely to be tricked
into sniffing "incorrectly."

Can you give an example of something you think ought to be an
exception to this requirement?

> 3.  Having a fixed table of types with patterns to look for, doesn't fully
> address why we will always have content sniffing: new content types are
> created that servers don't recognize, but a client could be updated to
> recognize.  Should the table simply be a starter set and clients can extend
> at will?  Or is there reason (and a reasonable way)  to create an IANA
> registry for new patterns/types?

Which types browsers sniff for has been fairly stable for a reasonable
period of time.  I haven't studied historical sniffing algorithms in
detail, but I believe image/png was the most recent type added to
widely used algorithms.  In my view, content sniffing is most useful
for interoperating with existing Web content.  It might be worth
having a registry for new types, but I'd expect we'd only add a new
type on the order of every 5-10 years.

> 4.  Are there any best-practice guidelines for working with users?  E.g.
> allowing a user to choose "text/html" for unmarked content might be a
> security hazard.  We don't want specific user interface requirements, but
> this document seems like a good place to extend security considerations to
> getting input from users, if there are such guidelines.

As far as I know, none of the major implementations of content
sniffing provide user overrides.  This is in contrast to charset
detection, where most major implementations let the user override.  (I
believe this is because charsets are a huge mess in Asia.)  I think it
makes sense to discuss this in the draft.  I'll add it to the next
version.

Adam

---

From: Lisa Dusseault <[email-masked]>
Date: Mon, 6 Apr 2009 13:07:09 -0700

> they choose) when deciding which responses to block or transform.

That makes more sense now.  It might be nice to specifically mention  
that the threat model assumes that the server can lie about Content- 
Type anyway, and in the security considerations warn that a server  
might trick clients into handling one content type as another if the  
client isn't careful.

> exception to this requirement?

I now think we mean something completely different by "extension".  I  
had assumed "protocol extension", i.e. a specification that extends  
HTTP, but now I see you mean "file name extension".

Thanks,
Lisa

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 06 Apr 2009 22:17:55 +0200

> type on the order of every 5-10 years.

I sort of expect <video src> and <audio src> to do sniffing as well  
(ignoring Content-Type altogether most likely). @font-face ignores  
Content-Type as well because nobody had registered media types for fonts.  
This is context-specific sniffing though.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 13:59:10 -0700

On Mon, Apr 6, 2009 at 1:07 PM, Lisa Dusseault
<[email-masked]> wrote:
> That makes more sense now.  It might be nice to specifically mention that
> the threat model assumes that the server can lie about Content-Type anyway,
> and in the security considerations warn that a server might trick clients
> into handling one content type as another if the client isn't careful.

Will do.

> I now think we mean something completely different by "extension".  I had
> assumed "protocol extension", i.e. a specification that extends HTTP, but
> now I see you mean "file name extension".

I'll clarify this.

Adam

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 14:01:34 -0700

> This is context-specific sniffing though.

That's a good point.  I'll look into how the forthcoming
implementations of these features work.  I suspect we'll have to treat
the context-specific cases that ignore the media type altogether
separately anyhow.

Adam

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 06 Apr 2009 23:05:27 +0200

> the context-specific cases that ignore the media type altogether

Agreed.

There's also mixed cases. E.g. for <img>/background-image/etc. I believe  
Opera first does a check if the Content-Type is image/svg+xml and then  
falls back to sniffing. (I haven't tested this extensively, but I believe  
this is what UAs should do for getting SVG support there.)

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 07 Apr 2009 09:45:13 +1200

Is it safe for all browsers to use the same algorithm here?

 From a purely biological view, having no diversity leads to increased risk.

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Ian Hickson <[email-masked]>
Date: Mon, 6 Apr 2009 21:45:58 +0000 (UTC)

On Mon, 6 Apr 2009, Anne van Kesteren wrote:
> falls back to sniffing. (I haven't tested this extensively, but I 
> believe this is what UAs should do for getting SVG support there.)

HTML5 right now for <img> says to use the "image sniffing rules", which 
first check for SVG and then, if the type doesn't match the SVG type, 
sniff the bytestream. If Adam's draft can become a standards-track 
document, then CSS could just defer to the same algorithm.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 14:47:49 -0700

> Is it safe for all browsers to use the same algorithm here?

Yes.

> From a purely biological view, having no diversity leads to increased risk.

Here the situation is reversed.  Diversity leads to increased security
risk because mismatches in sniffing create cracks that attackers can
exploit.

Adam

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 7 Apr 2009 00:03:28 +0200 (CEST)

On Mon, 6 Apr 2009, Adam Barth wrote:

> Here the situation is reversed.  Diversity leads to increased security risk 
> because mismatches in sniffing create cracks that attackers can exploit.

No, that's the exact same situation as in biology. If there's a single master 
race with no quirks, it will conquer them all. But if that master has a flaw, 
everyone gets hit.

Alas, if the one and only method is found to have a flaw at a future date, 
*all* browsers will have that flaw (assuming all would manage to and want to 
adhere to the same spec). Letting everyone do it there own way of course make 
the risk of them all having the exact same flaw less likely.

-- 

  / daniel.haxx.se

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 15:55:48 -0700

On Mon, Apr 6, 2009 at 3:03 PM, Daniel Stenberg <[email-masked]> wrote:
> No, that's the exact same situation as in biology. If there's a single
> master race with no quirks, it will conquer them all. But if that master has
> a flaw, everyone gets hit.
>
> adhere to the same spec). Letting everyone do it there own way of course
> make the risk of them all having the exact same flaw less likely.

I understand your perspective, but in this case the security issues
caused by mismatched sniffing algorithms are much more common.  For
example, a single byte difference in the content sniffing algorithm
between the server and the client can lead to vulnerabilities.  For a
concrete example we found in Wikipedia's content sniffer, see Section
2.5:

http://www.adambarth.com/papers/2009/barth-caballero-song.pdf

Adam

---

From: Michaeljohn Clement <[email-masked]>
Date: Mon, 06 Apr 2009 17:00:49 -0600

Daniel Stenberg wrote:
> master race with no quirks, it will conquer them all. But if that master
> has a flaw, everyone gets hit.

Ah, the dangers of taking an analogy too far...

In biology we usually talk about whether a species survives or not.  
The analogy fails because in browser security, having an exploitable 
hole in one browser is unacceptable.  The goal isn't to throw a range 
of genetic diversity against a potential extinction event and hope that 
a few individuals make it alive out the other side; the goal is to 
provide a secure browsing experience for *all* users.

-Michaeljohn

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 07 Apr 2009 11:41:03 +1200

> provide a secure browsing experience for *all* users.
sure that's the goal.  But what if you get the algorithm wrong?  It's 
still humans designing this right?  If there is an exploit to the 
algorithm, then potentially any browser that uses it is vulnerable.

It's difficult to foresee the future.  It's also difficult to guarantee 
that the algorithm will be bullet-proof forever and withstand any attack.

The potential down-side if all browsers are found to have a 
vulnerability is difficult to estimate.  It could be enormous.

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 17:07:16 -0700

On Mon, Apr 6, 2009 at 4:41 PM, Adrien de Croy <[email-masked]> wrote:
> sure that's the goal.  But what if you get the algorithm wrong?  It's still
> humans designing this right?  If there is an exploit to the algorithm, then
> potentially any browser that uses it is vulnerable.

What if we get the gzip algorithm wrong?  Maybe we should have
implementations use different compression algorithms just in case
there is an "exploit in the algorithm?"

> It's difficult to foresee the future.  It's also difficult to guarantee that
> the algorithm will be bullet-proof forever and withstand any attack.
>
> The potential down-side if all browsers are found to have a vulnerability is
> difficult to estimate.  It could be enormous.

The security cost of having different user agents use different
algorithms is MUCH greater.  Even mismatches of a single byte can lead
to cracks that an attacker can exploit.  I have previously given an
example of just such a vulnerability in a mismatch between one of the
world's most popular servers and one of the worlds most popular
clients.

I feel like I'm just repeating the argument I made previously.  Do you
have anything more specific to contribute to this discussion beyond a
vague analogy to biological systems?

Adam

---

From: Bill McQuillan <[email-masked]>
Date: Mon, 6 Apr 2009 17:11:16 -0700

> vulnerability is difficult to estimate.  It could be enormous.

Not to worry. I am sure that before all browsers have upgraded to the
standard, someone will come up with an "improved" algorithm (or think she
has) and will implement that instead since it is "just an efficiency
improvement" thus starting a fork that will quickly lead to a "new
species".

And she won't be the only one.

-- 
Bill McQuillan <[email-masked]>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 07 Apr 2009 10:23:04 +0900

On 2009/04/07 5:17, Anne van Kesteren wrote:
> Content-Type as well because nobody had registered media types for
> fonts. This is context-specific sniffing though.

Just a very small historic detail in this context: Around 1996/1997,
there was an attempt to register a top level media type of font/,
but that failed. I think that contributed to the fact that font
types didn't get registered.

Regards,    Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 07 Apr 2009 10:26:08 +0900

> they choose) when deciding which responses to block or transform.

For proxies and firewalls, the story may not be as simple.
How will they be able to distinguish between content sent to
browsers (where they can assume that the algorithm in this draft
applies) and other content?

Regards,    Martin.

---

From: Adam Barth <[email-masked]>
Date: Mon, 6 Apr 2009 18:34:54 -0700

> applies) and other content?

Their behavior will depend on their purpose.  Some proxies will wish
to be conservative and err on the side of safety (i.e., filter/block
assuming both sniffing and non-sniffing user agents).  Ultimately,
having two kinds of clients (sniffing and non-sniffing) is an
improvement on having the N types of clients we have today (where each
rolls its own sniffing algorithm).

Another option we could consider is supporting something like IE8's
X-Content-Type-Options: nosniff (whereby proxies and servers could
instruct their clients not to sniff).

Adam

---

From: Blake Frantz <[email-masked]>
Date: Mon, 6 Apr 2009 20:40:42 -0500

I would argue that the algorithm can't be "wrong" provided all content consumers attempting to sniff content type use the same algorithm.

I agree with Adam that, from a security perspective, we need to standardize on how content consumers determine the type of the content. The need for such convergence will continue to grow as web applications continue to take ownership of other's content.  Adam has demonstrated this in http://www.adambarth.com/papers/2009/barth-caballero-song.pdf but the same condition effects many other sites. 

Blake

---

From: Julian Reschke <[email-masked]>
Date: Tue, 07 Apr 2009 09:02:52 +0200

Adam Barth wrote:
> ...
> ...

I think that is incorrect; it even has been discussed over here in this 
very context:

<http://lists.w3.org/Archives/Public/ietf-http-wg/2008JanMar/0173.html>

and

<http://blogs.msdn.com/ie/archive/2005/02/01/364581.aspx#364853>

BR, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 7 Apr 2009 00:23:25 -0700

On Tue, Apr 7, 2009 at 12:02 AM, Julian Reschke <[email-masked]> wrote:
> I think that is incorrect; it even has been discussed over here in this very
> context:
>

Honestly?  Undocumented, numerical registry keys are hardly a user
interface.  By contrast, I can override the charset encoding in Safari
(which is the browser I happen to have in front of me) by selecting
"Text Encoding" from the View menu.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 07 Apr 2009 09:57:58 +0200

> "Text Encoding" from the View menu.

It is badly documented, yes. But it is a user override.

BR, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 7 Apr 2009 01:29:14 -0700

> It is badly documented, yes. But it is a user override.

I disagree, but I'm not sure what the point of this discussion is.  Is
there something specific you're advocating we change in the spec?

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 07 Apr 2009 10:34:49 +0200

> there something specific you're advocating we change in the spec?

No. I was just replying to your statement that UAs do not offer user 
overrides for this.

IE does, although badly documented. It would be nice if other UAs did it 
better, instead of not doing it at all.

BR, Julian