Subject: Christian's review of draft-mm-wg-effect-encrypt-13

From: Christian Huitema <[email-masked]>
Date: Mon, 4 Dec 2017 08:23:36 -0800

The high level summary is that draft-mm-wg-effect-encrypt version 13 is
significantly improved from previous versions, but that the document
would benefit a lot from additional work. I am not convinced that the
sections on data center and enterprises belong in this specific document
- they seem too high level to bring serious information to readers.
Maybe stage a separate document to survey enterprise issues in some
depth? I also feel that section 7 is way too speculative for a survey
document.

I have been using the "side by side" feature of the IETF web site to
compare the last draft that I reviewed (draft 10) and the current draft
(draft 13). That tools shows the high volume of changes and
reorganization that happens between these two versions. These changes
generally make the document more balanced, and that's a welcomed
development. There are however many places where I find that the
documents lists controversial techniques without sufficient caveats. I
will list some of them in the reminder of this review.

The introduction and the summary have been rewritten to clearly place
the document in perspective. The previous versions could be read as a
plea to stop encrypting traffic so previous management practices could
continue. The new introduction makes clear that the IETF needs to
prevent pervasive monitoring and improve the users' privacy, and that
the goal of the document is to list areas in which current management
practice need to evolve, or areas where substitute practices have to be
found. I also appreciate the note that "some of  (current management
practices) have been considered controversial from a technical or
business perspective or contradictory to previous IETF statements" --
and the implicit implication that the IETF will not attempt to restore
all of these.

2.  Network Service Provider Monitoring

Section 2, "Network Service Provider Monitoring", has been reorganized
to focus on management goals rather than simply provide a list of
existing management tools. The description of the trouble shooting tasks
in section 2.2 is useful. It makes the point that "application server
operators using increased encryption should expect to be called upon
more frequently to assist with debugging and troubleshooting", and that
could lead to some interesting work in the IETF.

There is paragraph at the end of section 2.1.2, Troubleshooting, that
states that "the push for encryption by application providers has been
motivated by the application of the described techniques." I think that
paragraph is misplaced. As far as I can tell, the application providers
are first concerned with "content management" techniques that modify 
the data stream. Any change of content has the potential to generate
bugs that are difficult for the application provider to fix. The second
concern is "ossification", when traffic characterization based on
inferred features of the application traffic leads to adverse
consequences when the application or transport protocols evolve. Neither
of those is directly relevant to the "troubleshooting" task. Maybe move
that paragraph higher in the document, e.g. in the introduction of
section 2? If not that, then maybe move it to section 2.2.2, since one
purpose of application encryption is indeed to defeat differential
treatment in the network.

I find the discussion of load balancers in section 2.2.1 somewhat
confusing. It seems to cover three functions: load balancers in data
centers, load balancers integrated with the network, and a network
management function that tries to maintain proper connectivity to
anycast addresses services in the presence of mobility. It might be
useful to move the discussion of "classic" load balancers to section 3,
and to discuss the problem of anycast continuity in a separate
subsection. The anycast discussion seems to assume that the network
operator alone has to deal with the supposed inadequacies of the
application providers. It seems obvious that this problem would be much
better solved by improved handling of mobility in content distribution
networks, rather than by some complex machinery in the network itself.
This might need to be stated.

Section 2.2.2 on Deep Packet Inspection could state that it is very
often possible to classify traffic based on analysis of the encrypted
data. Audio stream, video streams and web traffic have very different
signatures, even when encrypted. At the same time, it should also note
that many application providers are actively working to defeat the
"unilateral" traffic classification enabled by these techniques,
complementing encryption with various techniques like multiplexing or
padding. We could well observe an arms race between more powerful
network based analysis and smarter application hiding.

The discussion of performance enhancing proxies in section 2.2.3 states
that "This optimization at network edges measurably improves real-time
transmission over long delay Internet paths or networks with large
capacity-variation (such as mobile/cellular networks)." This is not a
consensual statement. Operators do indeed hope that deploying such
proxies will improve performance, but independent measurements have
shown that such proxies often in fact degrade performance. The studies
that show improvement tend to be based on old network technologies, or
on ancient TCP stacks. If the authors want to keep a statement like
that, they should add references to actual measurements. At a minimum,
the text should note that many application providers disagree with the
assessment presented here, and that the development of encrypted
transports such as QUIC is largely motivated by the desire to mitigate
the negative effects of such "performance-dehancing" proxies.

The discussion of caching in section 2.2.5 correctly states the tension
between network usage and application control. It could also state the
inherent privacy risk associated with network based caches: they will
provide a log of which users accessed what cached content. There is a
reference to draft-thomson-http-bc-01, but as far as I know the authors
have abandoned it, in part because they could not solve the related
privacy issues. In any case, that draft expired several month ago, and
the reference is probably not appropriate.

In section 2.3.3, Application Layer Gateways, I was wishing it would say
something about IPv6. But then of course most IPv6 deployments today
involve a form of NAT64...

Section 2.3.4 documents the "HTTP Header Insertion" technique. The
relation between that technique and "Network Service Provider
Monitoring" is unclear -- header insertion is certainly not a network
monitoring tool. It is also a highly controversial tool, as documented
for example in
https://www.theverge.com/2016/3/7/11173010/verizon-supercookie-fine-1-3-million-fcc.
I wonder whether it is appropriate to describe this at all in a document
dedicated to network management, and my simple suggestion would be to
just remove that section altogether. Failing that, the text needs to be
modified to note the controversial nature of the process, and its impact
on privacy. The authors could also note that the function could be
trivially implemented in the client's browsers if it was really needed
and approved by the users. There is no technical need to have anything
like that "in the network".

3.  Encryption in Hosting SP Environments

After examining network monitoring in section 2, the draft continues
with an analysis of "Hosting SP Environments" in section 3, and section
4 describes "Encryption for Enterprises". I assume that the initials SP
stand for "Service Provider" -- spelling it out would not hurt. I really
wonder whether these sections belong in the document at all, rather than
being published in separate documents. "Hosting Service Provider
Environments" appears to be a subset of the general "Data Center"
problem. It is true that some network providers also provide data center
services for their customers, but these network providers represent only
a small fraction of the service hosting industry. Similarly, some
network providers provide services to enterprises, but there is a wide
variety of enterprises. It is hard to believe that the authors of an
individual draft have authority to speak at the same time about network
services, data centers, and enterprises. In my opinion, it would be
simpler to just excise section 3 and 4 from this draft, and use the
content as input for specific drafts describing issues in data centers
and enterprises.

In any case, I am puzzled by the reference to Data Loss Prevention (DLP)
in the introduction of section 3.1. Data exfiltration is indeed a
security issue, but I knew it primarily as an issue in enterprise
networks. It does indeed become an issue in data centers when an
enterprise application is hosted outside the data center, but it is a
bit strange to see the first reference there. I already suggested to
move section 3 and 4 out to a different document. Failing that, I would
suggest reversing the order of section 3 and 4, i.e., discuss enterprise
issues first and data center issues next.

The discussion of Customer Access Monitoring in section 3.1.1 is a bit
strange. Most applications control customer access based on the customer
identity, not based on the IP addresses of the customer -- the whole
point of the "cloud" is that applications can be accessed from anywhere.
Some applications do perform additional checks, mainly as a defense
against stolen credentials, and would attempt to block access if the
network location does not look plausible for this specific user. These
are useful techniques, but the relation with encryption of data is
somewhat thin. It seems to reinforce my point that data center issues
would best be discussed in a separate document.

The reminder of section 3 appears to be a high level tutorial on the
operation of data centers. It is not clear that there is a particular
problem with encryption there. Indeed, I note that a lot of operators of
big data centers, such as for example AWS, Azure or Google, have
voluntarily pushed for increased used of encryption. I don't learn much
by reading these sections, and I question whether they belong in the draft.

4.  Encryption for Enterprises

The discussion on encryption in enterprises would probably benefit from
input by a variety of enterprise network managers. I found the
discussion somewhat hard to read. It seems that the authors want to
tackle three issues: the enterprise as a target for security attacks,
the enterprise as an application provider, and the enterprise as a
network provider. These are discussed in sections 4.1.1, 4.1.2, and
4.1.3.   

The description of attacks in 4.1.1 is somewhat high level. It starts
from the statement that "A significant portion of malware hides its
activity within TLS or other encrypted protocols" to draw a requirement
to monitor encrypted traffic, when in practice there are many other
monitoring points, from endpoint monitoring to data base activity logs
to logs at network authentication servers -- as stated in the last
paragraph of the section.

The monitoring of application performance in enterprises appears
strangely focused on the "IPv6 Destination Option Header (DOH)
implementation of Performance and Diagnostic Metrics (PDM)". I
understand that most big applications solve their monitoring need by
implementing some form of telemetry, which is not affected at all by
encryption, yet I see no mention of this telemetry approach in the
discussion.

I had a hard time reading section 4.1.3, Enterprise Network Diagnostics
and Troubleshooting. It seems to cover a variety of techniques meant to
monitor application services without actually instrumenting the
application, and as such is not very convincing.

The meat of section 4 appears to be in section 4.2, which is covering
the issue of data loss prevention, and generally detection of data
exfiltration. Again, this is an issue that would be worth a specialized
draft.

5.  Security Monitoring for Specific Attack Types

Looks fine, and this review is already very long.

6.  Application-based Flow Information Visible to a Network

Do we need this section at all? It seems that most of the information
could be captured by adding a small subsection to 2.1. Passive Monitoring.

7.  Impact on Mobility Network Optimizations and New Services

This section appears to be a mix of replication of statements already
made in section 2, and some speculation on the effect of transport
header encryption, such as deployed in Web RTC (SCTP over DTLS) or
planned in QUIC. There are active discussions in the QUIC WG to provide
alternative to transport header inspection for RTT monitoring, and
possibly also for packet loss monitoring.

Contrarily to the rest of the document, this section seems speculative
in nature. It discusses the possible effects of transport header
encryption on the possible deployment of new services, which do not
appear to be based on any IETF standard. I think the document would be
stronger if some of the content of section 7 was moved to the
appropriate part of section 2, and if the speculative statements were
published as a separate document.

8.  Response to Increased Encryption and Looking Forward

Looks reasonable.

-- Christian Huitema

---

From: Kathleen Moriarty <[email-masked]>
Date: Fri, 8 Dec 2017 11:25:40 -0500

Thank you, Christain and others for your reviews.  Our running draft
has addressed most comments received by mid-IETF week.  We are working
on the others.  We also received some comments from Kyle Rose and
Brandon Williams and are working to address those as well.

We'll respond on list when we get to this set of comments.

Best regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: james woodyatt <[email-masked]>
Date: Fri, 8 Dec 2017 10:53:24 -0800

> involve a form of NAT64...

That section leaves unmentioned a vast array of open questions related to IPv6-layer security filters and the problem of application layer gateways.

It contains an extended discussion of the ALG functions in one example case-- and not a particularly good one I would say-- the RTSP protocol, which in the presence of an ALG, depends on the parties performing it in cleartext, or it alternatively requires a proxy being a participant in the security protocol. The issues it describes apply to RTSP over both IPv4 and IPv6 in the presence of stateful filters that deny unsolicited inbound flows. Both NAT44 and NAT64 depend on such stateful filters, and "Simple Security in IPv6 Gateway CPE” [RFC 6092] is an example of a place where the issue applies in the absence of any network address translation.

Some history might be worth recalling here, at the risk of yet again being That Guy Who Told You So.

Because I’m a bitter old man, I like to go back to “Local Network Protection for IPv6” [RFC 4864]. Specifically, §4.2 IPv6 and Simple Security. This document, while Informational, is the point in our history where IETF formally published a statement acknowledging that IPv6 would absolutely always have the same problem with needing trustworthy ALGs and proxies in the network that IPv4 does.

The key sentence there is this one:

>> These should provide a default configuration where incoming traffic is limited to return traffic resulting from outgoing packets (sometimes known as reflective session state).

After this document was published, and the howling subsided, some of us thought it would be a good idea to describe the behavioral requirements of this IPv6 "Simple Security" function in the same way that RFC 4787 and RFC 5382 do for UDP and TCP, respectively, in the IPv4/NAT case. Which is why we now have "Simple Security in IPv6 Gateway CPE” [RFC 6092].

When RFC 6092 was in development, an early draft revision had a section about recommendations for various application layer gateways, e.g. RTSP+RTP, L2TP+IKE+IPsec/ESP, FTP, et cetera. The final draft didn’t include any of that, because consensus emerged around the idea that IPv6 simple security functions should not *need* any ALGs, and they should instead include a generic service for applications to use in soliciting inbound flows from unknown remote addresses. This entailed having the usual IETF tussle over competing non-standard solutions, i.e. UPnP-IGD vs. NAT-PMP. It was unresolved at the time RFC 6092 was published. It would still take some time before "Port Control Protocol (PCP)" [RFC 6887] would settle that debate.

Which is why REC-48 is written with such curiously vague text:

>> Internet gateways with IPv6 simple security capabilities SHOULD implement a protocol to permit applications to solicit inbound traffic without advance knowledge of the addresses of exterior nodes with which they expect to communicate.

Were we to revise RFC 6092 today, I would like to see this amended to recommend PCP specifically. If there is an RFC anywhere that specifically asserts that deploying PCP servers in NAT64 and IPv6 firewall devices is Best Current Practice, then I’m unaware of it. It would be nice.

With hindsight, I was right to harbor pessimism about the rosy predictions, favored by some IETF participants, for the future of IPv6 without simple security gateways everywhere, entailing the need for an array of ALGs and proxies in every gateway, with the consequential impediment to application protocol evolution created thereby. It turns out that “Local Network Protection for IPv6” [RFC 4864] really was a BCP in an Informational drag, which is the case I remember making here all those years ago.

With respect to NAT64, it’s worth reviewing “NAT64 Deployment Options and Experience” [RFC 7269], which is Informational, and it discusses both PCP and "An FTP Application Layer Gateway (ALG) for IPv6-to-IPv4 Translation” [RFC 6384], as an example of a case where we recognized that ubiquitous stateful filtering and denial of unsolicited inbound flows would entail the placement of application layer gateways and proxies in the network. Why do IPv6 FTP clients require RFC 6384? Because they don’t have PCP clients in them, and there are no PCP servers anyway.

Now it’s almost 2018, and we are finally discovering that pervasive encryption requires ALGs and proxies to be trusted by all parties in the application protocol. And this is painful. Because new host application protocols aren’t even developed to use PCP. Because PCP servers haven’t been deployed anywhere. Because everything is terrible, and everyone would love to keep it that way. Or, at least, it seems so to me. Because I’m a bitter old man.

I honestly don’t know if there is anything useful to say about ALGs in this document beyond simply acknowledging somewhere that they’re all necessarily broken by pervasive end-to-end encryption and operators who value administrative privacy, user tracking and/or topology hiding should consider the utility of secure application level proxies instead.

--james woodyatt <[email-masked]>