Subject: Privacy and HTTP intermediaries

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 3 May 2011 09:10:09 +0800

The issue of logging HTTP requests has come up in a discussion in another working group.

The goal is not just to prevent someone from learning that a certain person requested a certain resource, but to protect the identity of the resource.  That is, the very existence of the resource is a secret.

I understand that with CONNECT an intermediary only really knows that a particular server has been contacted, but what about unsecured HTTP?  Does the value of the Cache-Control header have any bearing on whether something is logged?

What sort of logging does an HTTP intermediary typically do?

--Martin

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 3 May 2011 11:47:45 +1000

> I understand that with CONNECT an intermediary only really knows that a particular server has been contacted, but what about unsecured HTTP?  Does the value of the Cache-Control header have any bea…

Nope.

I suppose you could read Cache-Control: no-store has having those semantics, but it doesn't in any implementation I'm aware of. Perhaps we need to clarify that.

> What sort of logging does an HTTP intermediary typically do?

The Squid format is fairly common; see:
  http://wiki.squid-cache.org/SquidFaq/SquidLogs#access.log
  http://www.squid-cache.org/Doc/config/logformat/

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 3 May 2011 10:16:15 +0800

On 2011-05-03 at 11:47:45, Mark Nottingham wrote:
> > Does the value of the Cache-Control header have any bearing on whether 
> > something is logged?
> 
> Nope.
> 
> I suppose you could read Cache-Control: no-store has having those 
> semantics, but it doesn't in any implementation I'm aware of. Perhaps 
> we need to clarify that.

With my privacy nut hat on, it would be nice if that could be added.  It's certainly consistent with the definition of no-store.

I'm not expecting the guidance to have any teeth, nor for it to have any impact on implementations, but there's a definite advantage to having text to that effect.

There is the question about non-caching intermediaries that might otherwise perform logging.  They aren't always going to look at Cache-Control unless they need to (for no-transform), so a caveat along the lines of "this is NOT a reliable or sufficient mechanism" might need to be added for this.

That leaves me with (for p6, S3.2.1 & S3.2.2):

  An intermediary that performs logging (whether or not it implements a cache) MUST NOT perform logging for requests or responses with a no-store directive.

--Martin

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 3 May 2011 12:23:16 +1000

>  An intermediary that performs logging (whether or not it implements a cache) MUST NOT perform logging for requests or responses with a no-store directive.

The problem is that a) this will make lots of existing implementations non-conformant, and b) people will start emitting no-store everywhere to control logging, killing caching in the process. 

I'm afraid the horse has already bolted here...

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 3 May 2011 07:18:40 +0200

Hi Martin,

>   An intermediary that performs logging (whether or not it implements a cache) MUST NOT perform logging for requests or responses with a no-store directive.

Many intermediaries will still log regardless of whatever new directive
you add, and there are a lot of places where logging will be mandatory
regardless of the cache-control header (which should control caching and
not logging).

Also, concerning the privacy, I see no reason for not logging something
that is exchanged in clear text. This has always been the case for decades
with the query string in GET requests etc... ; if you want some privacy,
you know you need SSL.

There are some places where logging is performed by capturing all network
traffic, and other places where network traffic is captured in order to
make post-mortem analysis of network incidents. The presence of whatever
header, or the use of a CONNECT will not change anything there, the captures
will still contain all the exchanges. Once again the only solution there is
to encrypt the traffic.

I think it'd be more efficient to remind the reader about that in the spec,
so that implementers leave the choice to their users (accessibility vs
privacy). Right now when I connect to Yahoo mail in clear text from some
customer's, I know I'm taking a risk on my privacy but I have my access.
With WS it should be the same. When you connect to some services in clear
text, you accept a risk.

What we should discourage is a transparent and automatic fallback from
SSL to clear text, because the user might think his privacy is respected
while it's not.

Regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 03 May 2011 07:35:59 +0200

On 03.05.2011 07:18, Willy Tarreau wrote:
> ...
> ...

Logging is one thing, preserving logs is another thing. Reminder: in 
some countries, the IP address is considered relevant for privacy (and I 
agree), thus preserving HTTP logs containing IP information for too long 
is not allowed.

Best regards, Julian

---

From: David Morris <[email-masked]>
Date: Mon, 2 May 2011 22:55:14 -0700 (PDT)

On Tue, 3 May 2011, Julian Reschke wrote:

> Logging is one thing, preserving logs is another thing. Reminder: in some
> countries, the IP address is considered relevant for privacy (and I agree),
> thus preserving HTTP logs containing IP information for too long is not
> allowed.

I think log management policies are out of scope for the HTTP protocol 
specification other than to note that traffic might be logged, requests
might be logged, etc. Logging may be required for system management and
problem resolution, or it might be required to conform to local laws.
Conversely, logging (and/or retention) may be limited by local laws.

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 3 May 2011 14:07:58 +0800

On 2011-05-03 at 15:18:40, Willy Tarreau wrote:
> I think it'd be more efficient to remind the reader about that in the 
> spec, so that implementers leave the choice to their users 
> (accessibility vs privacy). Right now when I connect to Yahoo mail in 
> clear text from some customer's, I know I'm taking a risk on my 
> privacy but I have my access.
> With WS it should be the same. When you connect to some services in 
> clear text, you accept a risk.

This discussion probably needs to include a quote of the relevant, and existing, disclaimer.

      This directive is NOT a reliable or sufficient mechanism for
      ensuring privacy.  In particular, malicious or compromised caches
      might not recognize or obey this directive, and communications
      networks might be vulnerable to eavesdropping.

As this disclaimer says, there's no accounting for those who choose to disrespect your wishes.  But that doesn't mean that you should suffer indignities quietly, or not even bother trying.

I think that a quick skim of http://tools.ietf.org/html/draft-morris-policy-cons might be enlightening for those who aren't aware of some of the more advanced privacy work currently going on.

--Martin

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 3 May 2011 08:16:57 +0200

> As this disclaimer says, there's no accounting for those who choose to disrespect your wishes.  But that doesn't mean that you should suffer indignities quietly, or not even bother trying.

OK but still my point remains that cache-control is irrelevant to logging.
Cache-* is for caches only. Proxies, l7 firewalls, load balancers, WAFs,
compressors, URL filters, anti-virus, etc... all do log and will not inspect
cache-* because they are not caches (and it should remain this way).

Probably that HTTP will need some extensions to handle privacy. For instance,
a header might be used to indicate what part of the URI may be logged (eg:
max length), if the request body may be logged, and the same for response
(eg: redirects). But anyway, most of the time logging is made mandatory on
whatever information is available, for troubleshooting as well as because
of legal obligations, and the same legal obligations might for you to destroy
those logs past a certain delay.

Regards,
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 3 May 2011 14:26:28 +0800

On 2011-05-03 at 16:16:57, Willy Tarreau wrote:
> Cache-* is for caches only. Proxies, l7 firewalls, load balancers, 
> WAFs, compressors, URL filters, anti-virus, etc... all do log and will 
> not inspect

That was my initial thought too.  Until I saw the description of no-transform, which almost all of those examples will have to respect...if they want to remain compliant.

As you say, the alternative is to add a new header (DNT?), but when there is already so much overlap, it seemed cleaner this way.

> [...] legal obligations [...]

That's the golden rule, isn't it?  Policy trumps all.  Irrespective of what an RFC says, someone can enact a policy that overrides the specification.  Add that to the cases covered by the caveat (under compromised, if you like).

--Martin

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 03 May 2011 06:41:04 +0000

In message <[email-masked]>

>That was my initial thought too.  Until I saw the description of no-transform,
>which almost all of those examples will have to respect...if they want 
>to remain compliant.

This is the crux of the matter:  How can you as a privacy-desiring user
know if they want to be compliant ?

Squezing non-effective "please-protect-my-privacy" requests into HTTP
is not going to have any practical effect at all, so we should not do it.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 3 May 2011 15:02:26 +0800

On 2011-05-03 at 16:41:04, Poul-Henning Kamp wrote:
> This is the crux of the matter:  How can you as a privacy-desiring 
> user know if they want to be compliant ?
> 

It's a fun argument.  Much like the one that goes: "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place."

That's not the point.  Privacy sometimes requires something more than a technical solution.  See aforementioned paper.  I know a lot of people are quite negative towards this approach to the problem, but the alternatives (capitulation, TLS) aren't superb either.

And maybe you really do trust your intermediary.  In a lot of cases you have chosen to use it, after all.  Maybe logging for all those other transactions is quite handy.

Or maybe you aren't given any choice in the matter and this gives you a way to salvage the important stuff.  And you don't want to suppress all logging, because that's incentive for the policy makers to ignore this sort of guidance.

--Martin

---

From: Dzonatas Sol <[email-masked]>
Date: Tue, 03 May 2011 14:05:26 -0700

> SSL to clear text, because the user might think his privacy is respected

Within one single layer, that may be obvious as you have stated. Within 
multiple layers, in-between some layers may see clear-text and outside 
those two the same text is appears encrypted. Don't mistake this as 
false sense of security. We could state many use-cases, yet the obvious 
is binary on hardware. Something decodes the image and feeds that to the 
hardware for execution, for example.

The fix for this has been in niche areas, yet it may be there was some 
good reason left open in the public commerce. After decades. we just 
didn't make it obvious even if one thing leads to another. With 
possibilities of quantum computers (on demand), maybe we are not worried 
given current trends of organic development.

-- 
--- https://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering, Virtual Reality, Consultant

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 04 May 2011 10:39:41 +1200

I think the cases where client-controlled suppression of logging will be 
useful are few and far between.

In most cases, the user should not have authority to control logging 
function on an intermediary.

Only if the intermediary explicitly allows such control (and I can't 
imagine any corporate sys admins turning such a feature on) could it be 
allowed by policy.  It's also completely open to abuse (proxy advertises 
ok, it will do it, but then doesn't).

I don't know if I buy the argument "at least we should try".  This will 
just bloat http requests out with yet another bandwidth-wasting header.

On 3/05/2011 6:41 p.m., Poul-Henning Kamp wrote:
> In message<[email-masked]>

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 3 May 2011 23:44:17 +0100

On 3 May 2011, at 03:16, Thomson, Martin wrote:

> With my privacy nut hat on, it would be nice if that could be added.

If what could be added? A clarification that no-store suddenly has an associated semantic that no implementation associates with it currently?

>  It's certainly consistent with the definition of no-store.
> 

If you do not expect "the guidance to have any teeth, nor for it to have any impact on implementations", what exactly are you hoping to achieve?

>  An intermediary that performs logging (whether or not it implements a cache) MUST NOT perform logging for requests or responses with a no-store directive.

Apart from the fact that I'm not aware of an implementation that associates such a semantic with no-store, given the reasons that logging is often enabled (e.g. debugging, reporting, billing, etc.) I simply don't see that it is likely that implementations will adhere to the stated requirement to not log requests/responses containing no-store and therefore having a specification mandate something that few if any implementations will adhere to seems entirely pointless to me. It has nothing to do with inter-operation and at best will give some subset of folks the impression that using no-store will provide some additional level of privacy that in practice will be non-existent.

Ben

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 4 May 2011 09:58:13 +1000

Right. 

I don't see an issue here, unless someone wants to argue that we need to explicitly add logging to this text:

Cheers,