Subject: [Pearg] Fwd: New Version Notification for draft-learmonth-pearg-safe-internet-measurement-02.txt

From: Iain Learmonth <[email-masked]>
Date: Thu, 16 May 2019 11:26:29 +0100

-------- Forwarded Message --------
Subject: New Version Notification for
draft-learmonth-pearg-safe-internet-measurement-02.txt
Date: Thu, 16 May 2019 03:25:03 -0700
From: [email-masked]
To: Iain R. Learmonth <[email-masked]>, Iain Learmonth
<[email-masked]>

A new version of I-D, draft-learmonth-pearg-safe-internet-measurement-02.txt
has been successfully submitted by Iain R. Learmonth and posted to the
IETF repository.

Name:		draft-learmonth-pearg-safe-internet-measurement
Revision:	02
Title:		Guidelines for Performing Safe Measurement on the Internet
Document date:	2019-05-16
Group:		Individual Submission
Pages:		7
URL:
https://www.ietf.org/internet-drafts/draft-learmonth-pearg-safe-internet-measurement-02.txt
Status:
https://datatracker.ietf.org/doc/draft-learmonth-pearg-safe-internet-measurement/
Htmlized:
https://tools.ietf.org/html/draft-learmonth-pearg-safe-internet-measurement-02
Htmlized:
https://datatracker.ietf.org/doc/html/draft-learmonth-pearg-safe-internet-measurement
Diff:
https://www.ietf.org/rfcdiff?url2=draft-learmonth-pearg-safe-internet-measurement-02

Abstract:
   Researchers from industry and academia will often use Internet
   measurements as a part of their work.  While these measurements can
   give insight into the functioning and usage of the Internet, they can
   come at the cost of user privacy.  This document describes guidelines
   for ensuring that such measurements can be carried out safely.

Note

   Comments are solicited and should be addressed to the research
   group's mailing list at [email-masked] and/or the author(s).

   The sources for this draft are at:

   https://github.com/irl/draft-safe-internet-measurement

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Sara Dickinson <[email-masked]>
Date: Tue, 21 May 2019 11:23:09 +0100

Hi All,

This email starts a two week Call for Adoption of draft-learmonth-pearg-safe-internet-measurement

The draft is available at: https://datatracker.ietf.org/doc/draft-learmonth-pearg-safe-internet-measurement/ <https://datatracker.ietf.org/doc/draft-learmonth-pearg-safe-internet-measurement/>

Please review this draft to see if you think it is suitable for adoption by PEARG and send comments to the list, clearly stating your view.

Please also indicate if you are willing to contribute text, review, etc.

This call for adoption ends on 4th April 2019.

Sara.

---

From: Sara Dickinson <[email-masked]>
Date: Tue, 21 May 2019 11:24:22 +0100

> On 21 May 2019, at 11:23, Sara Dickinson <[email-masked]> wrote:
> 
> Hi All,
> 

That should be 4th June 2019 - apologies :-)

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 21 May 2019 11:51:36 +0100

> This email starts a two week Call for Adoption of draft-learmonth-pearg-safe-internet-measurement

I read it before the last IETF meeting and would be
very happy to see it adopted and published.

S.

---

From: Amreesh Phokeer <[email-masked]>
Date: Tue, 21 May 2019 15:59:41 +0400

I believe it's an important piece of document for researchers carrying
Internet-wide measurements. I would like to see it adopted and published.

--
Amreesh

On Tue, May 21, 2019 at 2:51 PM Stephen Farrell <[email-masked]>
wrote:

> > This email starts a two week Call for Adoption of
> draft-learmonth-pearg-safe-internet-measurement
>
> > The draft is available at:
> <
> > Please review this draft to see if you think it is suitable for adoption
> by PEARG and send comments to the list, clearly stating your view.
> >
> --
> Pearg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/pearg
>

-- 
Amreesh Phokeer

---

From: Vittorio Bertola <[email-masked]>
Date: Thu, 23 May 2019 09:41:03 +0200 (CEST)

Il 21 maggio 2019 12:23 Sara Dickinson <[email-masked]> ha scritto:

Hi All,

It's a useful document, I am in favour of adoption.

Separately, I think I have comments on the "consent" section - it's fine to discuss what would be nice to do, but at least in GDPR-land there is hard law on what requires end-user consent and what does not, which of course takes precedence.

Ciao,

--

Vittorio Bertola | Head of Policy & Innovation, Open-Xchange
[email-masked]
Office @ Via Treviso 12, 10144 Torino, Italy

---

From: Iain Learmonth <[email-masked]>
Date: Thu, 23 May 2019 09:10:33 +0100

On 23/05/2019 08:41, Vittorio Bertola wrote:
> Separately, I think I have comments on the "consent" section - it's fine
> to discuss what would be nice to do, but at least in GDPR-land there is
> hard law on what requires end-user consent and what does not, which of
> course takes precedence.

This document is not meant to offer legal advice, but rather set a
baseline for safety that may be further restricted by local laws. I'd
like to just say this once in the scope section, and not have to repeat
it throughout the document.

On the other hand, if GDPR has specific places that would require
consent then I would be happy to cite those as referenced examples to
help readers. I think specific examples definitely have a place in this
document.

Thanks,
Iain.

---

From: Mallory Knodel <[email-masked]>
Date: Thu, 23 May 2019 11:57:07 +0300

> by PEARG and send comments to the list, clearly stating your view.

Very happy to see that this draft presents so clearly so much, already
at this stage. I think that it should be adopted by the RG.

I could help with writing text for a human rights considerations
section. Also happy to review.

-Mallory

-- 
Mallory Knodel
Head of Digital :: article19.org
gpg fingerprint :: E3EB 63E0 65A3 B240 BCD9  B071 0C32 A271 BD3C C780

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 27 May 2019 06:34:47 -0700

I have reviewed this document and while I think some of the advice
here is potentially useful, I don't think the recommendations really
match what's current practice or what's practical. As such, I don't
think it should be adopted without quite a bit more work.

As background, here is a partial list of the kinds of studies that
are commonly done:

- A/B tests of browser behavior where we randomize users into
  a control and treatment group and then measure some parameter
  (e.g., connection success rate) under normal browsing usage.
  This kind of A/B test is standard practice for many browser
  features, at least for Chrome and Firefox. See:
  https://textslashplain.com/2017/10/18/chrome-field-trials/
  https://support.mozilla.org/en-US/kb/shield

- Forced tests where we have the browser perform some action
  (e.g., connect to a server) with various parameters and
  measure differences between the parameter settings.
  This is less common, but we used these during the roll out
  of TLS 1.3 and are currently using these for DoH.
  See:
https://blog.mozilla.org/futurereleases/2018/11/27/next-steps-in-dns-over-https-testing/

- Studies where we use an ad network to initiate some behavior
  in the browser. This is pretty common approach when you don't
  control a browser measurement platform. See, for instance:
  https://labs.apnic.net/?p=341

https://www.usenix.org/conference/usenixsecurity13/technical-sessions/paper/lian

- Wide-scale scans (e.g., https://zmap.io/).

I don't really want to get into a long debate about whether any
particular study type is appropriate. Rather, these are common study
types and so if the advice in this document is to be useful, then it
needs to reasonably match what people do -- or at least have a much
stronger argument that people should change what they do than is
offered here.

S 2. CONSENT
The text in this draft leans pretty heavily on getting consent,
either direct consent (including all users of the shared network)
or "proxy consent".

However, many of these kinds of studies don't really lend themselves
to detailed consent from individual users of the browser -- let alone
to from every user on the network they are on. As a concrete example,
ad-type studies don't generally get any kind of consent at all.  For
instance, here's the experimental setup for APNIC's DNSSEC
measurements: https://labs.apnic.net/?p=341

    The experiment uses an online advertisement campaign to deliver
    the test code to end systems. When the end system is passed an ad
    that is carrying the experiment the system runs embedded Adobe
    Flash code. The code is executed when the ad is passed to the
    user, and does not rely on a user "click" or any other user
    trigger action. The active code interrogates one of two experiment
    controllers by performing a URL fetch. The contents of the fetched
    experiment control URL are a dynamically generated sequence of
    four URLs. These four URLs are the substance of the test setup.

It's worth noting at this point that the Web is a platform for running
remote code, and by browsing you're opting into that, and ad studies
just leverage that behavior.

As another example, Mozilla's Shield Studies system
(https://wiki.mozilla.org/Firefox/Shield/Shield_Studies),is generally
opt-out, with a specific opt-in for when the study collects
more sensitive data:

    Shield Studies are available on all channels. Individual studies
    can be opt-out or opt-in depending on the nature of the study and
    the data being collected. Opt-out Shield Studies can only collect
    Type 1 and 2 data. This is the same type of basic interaction data
    we collect by default as defined in our privacy policy. There may
    be instances where we want to collect data that is not covered by
    our default data collection policies. In those instances you will
    be prompted to opt-in to the study. A complete description of the
    study and any additional data that would be collected will be
    disclosed before enrolling. A great example of this is Pioneer,
    which is actually an opt-in Shield Study currently in the wild.

As above, in the cases where it's opt-in, the primary question is
about the user's private data, and it's when that data is implicated
that we get more consent. There's no practical way to get consent from
other users of the same network. So, the consent section of this draft
doesn't seem to really apply well.

Now, obviously, one could argue that this kind of study should follow
a different set of practices, but I don't think that's really
right. There seem to be two core issues here:

- The effects of various changes on the user or the network they are
  on.

- The data collection inherent in doing the study.

WRT to the first point, as a general matter, modern browsers
auto-update, so the user has generally opted into regularly getting
whatever new code the vendor thinks makes the best browser. We use
studies to determine whether a given change is good but in many cases
the alternative is just to roll the change out to everyone,
effectively doing the study on the whole population. So, either
we can have that effect on everyone or only on a limited subset
(what we do now). We run a lot of experiments and having explicit
consent for each one would make that much harder, resulting in
testing much more on the full population.

Similarly, on the topic of data collection, browsers report back quite
a bit of technical data about their behavior. In both Chrome and
Firefox, this is on by default, though you can turn it off. As
suggested by the quote above on Shield studies, many studies just
gather this kind of data (and often data that the browser would
already report back) and Mozilla, at least, has a pretty
well-developed framework for determining what kind of data requires
what level of consent
(https://wiki.mozilla.org/Firefox/Data_Collection).

S 3.4.

   When deciding on the data to collect, assume that any data collected
   might become public.  There are many ways that this could happen,
   through operation security mistakes or compulsion by a judicial
   system.

This seems like it might be a good practice under some circumstances,
but impractical in many other cases. Moreover, given the enormous
amounts of personal data that users routinely hand to Web sites,
this seems like an unreasonably high standard.

S 3.5.
   For all data collected, consider whether or not it is really needed.

This section (and Section 4) seem really undeveloped. As a comparison
point, consider Mozilla's Data Collection principles
(https://wiki.mozilla.org/Firefox/Data_Collection). I'm not saying
that those principles are perfect, but they're fairly complete and
thought through. if you're going to publishing an RFC with proposed
principles for what data is collected, I would expect it to be
comparably thorough.

---

From: Vittorio Bertola <[email-masked]>
Date: Mon, 27 May 2019 17:23:21 +0200 (CEST)

Il 27 maggio 2019 15:34 Eric Rescorla <[email-masked]> ha scritto:

S 2. CONSENT

or "proxy consent".

measurements:

That's not very accurate, at least under European regulations. It's not "by browsing you're opting into that", it's rather that each website shows you a cookie consent popup, and that's where you provide your consent and opt into receiving various types of cookies, including the ones that make ad-based experiments work. If you reject advertising/customization cookies, of course the ad-based experiment should not work. On top of that, the text asking for consent to these cookies should specify that you will not just be shown customized ads, but that personal information about you will be collected.

By the way, the fact that "these kinds of studies don't really lend themselves

to detailed consent from individual users of the browser" is not a legally valid justification for doing these studies without explicit user consent, at least if they collect personal information and unless they fall into a well specified set of exceptions.

Ciao,

--

---

From: Iain Learmonth <[email-masked]>
Date: Mon, 27 May 2019 16:39:40 +0100

Hi Eric,

> think it should be adopted without quite a bit more work.

In my opinion, a lot of current practice is not safe. This document does
not aim to set out current practice. It aims to raise the bar on user
safety when it comes to performing Internet measurement.

> As background, here is a partial list of the kinds of studies that

Thanks for these examples. I see no reason why each of these could not
be done safely. I have refrained so far from relating to specific
examples to try to keep the guidelines general but an "applied" section
may be useful to readers to better understand how to use the guidelines.

> stronger argument that people should change what they do than is

Not necessarily. If it turns out that upon analysis, a lot of studies
are dangerous for users, this document should not weaken its guidelines
to allow those studies to continue. That would be silly.

You can do more dangerous things as long as either you have the consent
of the users, or you believe that what you are doing would be expected
by the users anyway. Neither of these things removes risk, but ensures
that the user is taking an informed risk.

> either direct consent (including all users of the shared network)

This version contains a first draft of this section. Perhaps the
"implied consent" portion could do with more work.

>     four URLs. These four URLs are the substance of the test setup.

This is great until you run on a user machine in a country which has
some censorship/monitoring infrastructure in place that has
misinterpreted the URL as some proscribed content, landing the user in
trouble.

At the very least you may have generated costs for the user's bandwidth
there, which are uncompensated.

> just leverage that behavior.

Tell that to the 1,543,235 users that have installed NoScript from
Firefox Add-ons. Now you could say that they have opted out and the code
won't be run, but the way you've phrased this makes me think that
actually you just haven't understood the wider range of Internet users
that exist.

> opt-out, with a specific opt-in for when the study collects

Exactly, when a study would collect more data than a user might expect
would be collected, you ask for informed consent. When a study is
looking at non-personal information then it is OK to go with implied
consent.

> There seem to be two core issues here:

This is a good split that I had not considered before.

> whatever new code the vendor thinks makes the best browser.

Say a mobile phone vendor wanted to test out how its camera was doing.
It ships you an auto-update that sends back every photo you've taken to
work out things like light levels, noise, and what might need tuned.
This is for the purpose of improving the camera.

> (https://wiki.mozilla.org/Firefox/Data_Collection).

The question is not about whether or not you can report back data. The
question is about what safeguards you can take to protect your users
from harm. You can collect data and report it back safely. We have
techniques for doing this in ways that respect users. The only reason to
not use them is if you don't actually care about protecting your users.

This draft is still in its early stages and there's a lot to flesh out
still. This feedback is useful and I'll be incorporating some of the
topics, but I feel that you've maybe missed the point of it.

Thanks,
Iain.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 27 May 2019 08:44:05 -0700

On Mon, May 27, 2019 at 8:23 AM Vittorio Bertola <
[email-masked]> wrote:

> That's not very accurate, at least under European regulations. It's not
> "by browsing you're opting into that", it's rather that each website shows
> you a cookie consent popup, and that's where you provide your consent and
> opt into receiving various types of cookies, including the ones that make
> ad-based experiments work. If you reject advertising/customization cookies,
> of course the ad-based experiment should not work.
>
On top of that, the text asking for consent to these cookies should specify
> that you will not just be shown customized ads, but that personal
> information about you will be collected.
>

You may be misunderstanding my point here, as these studies don't really
rely on tracking cookies. If you're browsing the Web with JS on, your
browser will execute whatever code it gets served. That's just the nature
of the Web.The reason you use an ad network isn't primarily to take
advantage of tracking behavior but rather to cheaply get a large sample of
users (note that big sites like Google or Facebook can do this without
taking advantage of an ad network). Now, as it happens, ad networks do
generally use tracking cookies, and it's sometimes useful to use that for
targeting, but in general these studies run fine even in environments where
third party cookies are completely disabled, as in Safari ITP or Firefox
ETP.

By the way, the fact that "these kinds of studies don't really lend
> themselves
> to detailed consent from individual users of the browser" is not a legally
> valid justification for doing these studies without explicit user consent,
> at least if they collect personal information and unless they fall into a
> well specified set of exceptions.
>

This isn't really the appropriate place to discuss the legal parameters of
various methodologies, especially as they vary from jurisdiction to
jurisdiction. I would advise people to consult their own legal/compliance
departments rather than relying on advice they got on the Internet.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 27 May 2019 09:13:55 -0700

> safety when it comes to performing Internet measurement.

Yes, I appreciate that. However, as written I don't think it captures those
boundaries well and so is not very useful.

> to allow those studies to continue. That would be silly.

Hence the text you are directly quoting above.

"or at least have a much stronger argument that people should change what
they do than is offered here.

Neither the draft as written, nor your response, seems to me to offer any
such argument.

> misinterpreted the URL as some proscribed content, landing the user in

Well, this isn't my study, so I'm not going to account for how it's
constructed,
but it seems like you're missing the context here that this is served
as part of an ad, and that the way that ads work *in general* is to load
content from remote servers. It's not clear to me why you think that
the URL that the study checks is somehow more likely to be interpreted
as proscribed than the ad content that would normally be served.

With that said, we also do studies where the browser loads specific URLs
we provide it, and we do try to make them innocuous.

> there, which are uncompensated.

Again, this is served via an ad network, so the user would be loading
some other ad that used up random amounts of bandwidth instead of
the ad that runs the study. This seems like a pretty weak argument.

> actually you just haven't understood the wider range of Internet users

I'm quite aware of NoScript (you do know that I work for Mozilla, right?)
but this is a trivially small fraction of the browsing user base. As a
general
matter, the Web is a platform for executing code sent by the server,
which of course users can disable if they choose.

> This is for the purpose of improving the camera.

I think you've misunderstood my point. I'm not saying that the browser
vendor  -- or a mobile phone vendor -- can just send malicious code to
the user (and for avoidance of doubt I would consider reporting every photo
to be malicious).

Rather, what I'm saying is that browsers already do auto-updates, and so
it's not
useful to have a much stricter set of rules for experiments/measurements
that are delivered to some users than vendors do for code they would deliver
to all users as part of an update. I.e., it's not that useful to focus on
studies
rather than on overall behavior.

> still.

Understood. I don't think at this early stage it's ready for adoption. Once
you've fleshed it out, I'll be happy to take another look.

-Ekr

---

From: Vittorio Bertola <[email-masked]>
Date: Tue, 28 May 2019 10:18:54 +0200 (CEST)

Il 27 maggio 2019 17:44 Eric Rescorla <[email-masked]> ha scritto:

On top of that, the text asking for consent to these cookies should specify that you will not just be shown customized ads, but that personal information about you will be collected.

You may be misunderstanding my point here, as these studies don't really rely on tracking cookies. If you're browsing the Web with JS on, your browser will execute whatever code it gets served. That's just the nature of the Web.

Ok, let's see it from this viewpoint then: the fact that I visit a web page that executes code (client and/or server side) is not a valid authorization for that web page to collect any personal information about me. If it were so, then no consent request for data collection would ever be necessary on the web. Instead, whenever the web page wants to record personal information about me, it needs to ask for explicit and specific consent (again, this is true in the European framework, which is anyway the model for most other privacy regulations).

Privacy protection and consent management works by purpose: if I access a web site with the objective of watching a video, you may imply consent to any data processing activity which is strictly necessary to make me watch the video, but not to other activities such as running your tests (if they gather personal information). That would require another, separate and optional, expression of consent.

All in all, this problem mostly goes away if the data gathered by your tests is anonymized or at least pseudonymized in a way that makes it impossible to associate it back with the actual person (IP address included). This is advice that can be given to testers, and it would be interesting to understand, from your experience, if there is any reason why these tests cannot always be made in anonymized form.

This isn't really the appropriate place to discuss the legal parameters of various methodologies, especially as they vary from jurisdiction to jurisdiction.

True, but if the I*TF wants to provide advice on how to make Internet tests without harming privacy, one would expect it to set the bar at least at the same level of the best privacy protection rules that are currently in force around the world.

Ciao,

--

---

From: Sara Dickinson <[email-masked]>
Date: Fri, 31 May 2019 10:08:22 +0100

> On 27 May 2019, at 17:13, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
> On Mon, May 27, 2019 at 8:39 AM Iain Learmonth <[email-masked] <mailto:[email-masked]>> wrote:
> Hi Eric,
> 
> "or at least have a much stronger argument that people should change what they do than is offered here.
> 
> Neither the draft as written, nor your response, seems to me to offer any such argument.
> 

I’d like to support addressing these concerns in the way that Iain proposed i.e. with a clear separation in the document:

- Retain a ‘Recommendations' section which remains close to a description of desired behaviour for Safe Measurement practices

- Add a ‘Current Practices’ section which can cite studies such as the ones mentioned with the analysis of if and why they fall short of the recommendations and the specific risks to users due to that difference. Where possible suggest modifications to mitigate those risks.

I agree it would be useful for the document to capture acknowledged or existing practical limitations in more detail and these can be reference from the recommendations. However I would argue the primary goal should remain providing strong standalone guidance on best practices for these and future studies. 

Sara.