Subject: [Jmap] Unconditional HTTPS requirement and server development debugging

From: Andri Möll <[email-masked]>
Date: Sun, 16 Sep 2018 21:47:36 +0000

Hi,

I raised a similar issue with JSCalendar and I thought I'd seek what 
others think here, too.

This is about unconditionally requiring HTTPS 
<https://tools.ietf.org/html/draft-ietf-jmap-core-08#section-8> in 
Section 8 with no explicit (e.g. a warning like from today's mail apps) 
HTTP fallback. Long story short, suppose I come across a 3rd party JMAP 
client on iOS that fails to work with my JMAP server. How am I expected 
to debug the problem if I can't direct the app to a development server 
on a local machine?

I'll quote myself from before:

> I've spent hours debugging and working around iOS's (and other 
> platforms') CalDAV quirks and managed to so by directing the apps to a 
> development machine. Yet if CalDAV had demanded HTTPS to the server, I 
> would've been hosed. You can't get a legit certificate for a local IP 
> address nor do I have a Mac to install custom root certificates to an 
> iPhone, for example. If that's even a thing anymore. Some client 
> software have their own bundled roots. There it would've impossible to 
> productively debug HTTPS-only services.
>
> This is a weird thing for me to advocate, as I'm usually the first to 
> jam encryption down people's throats, but my latest experience with 
> implementing CalDAV has left me with quite a cynical view. While 
> people _have_ to follow standards, they don't. They make mistakes. And 
> that's where obstacles to debugging annoy me.

I'd like to see some considerations for the implementors of [JMAP] 
servers here, too.

Andri

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 17 Sep 2018 12:17:14 +0200

> How am I expected to debug the problem if I can't direct the 
> app to a development server on a local machine?

You direct the client to a development server for which you have the 
private key, then install that private key in wireshark and tell wireshark 
to decrypt the connection. One of these should help:

https://www.google.com/search?hl=eo&q=wireshark+tls+private+key

Arnt

---

From: Andri Möll <[email-masked]>
Date: Mon, 17 Sep 2018 10:46:16 +0000

Well, unfortunately that's not going to cut it. Debugging 3rd party 
software is more than getting the plain text. It's an iterative process 
of trial and error. Secondly, the biggest problem is getting an accepted 
certificate for development in the first place which I described in my 
original email. The more "correct and secure" the 3rd party app is, the 
less possible it is to test with it _if_ it insist on using HTTPS.

On 9/17/18 10:17 AM, Arnt Gulbrandsen wrote:
>> How am I expected to debug the problem if I can't direct the app to a 
>> development server on a local machine?
>
> private key, then install that private key in wireshark and tell 
> wireshark to decrypt the connection. One of these should help:
>
> _______________________________________________
> Jmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jmap

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 17 Sep 2018 13:00:42 +0200

If you need more than just getting the plaintext, then relaxing the HTTPS 
requirement won't help you either.

Arnt

---

From: Andri Möll <[email-masked]>
Date: Mon, 17 Sep 2018 11:04:26 +0000

I think we're imaging something different. I'm talking about a 
JMAP-compatible client application providing a way to talk to a given 
server over HTTP just like I can configure IMAP or CalDAV compatible 
clients today to talk to a local development server over plain IMAP and 
HTTP respectively.

On 9/17/18 11:00 AM, Arnt Gulbrandsen wrote:
> If you need more than just getting the plaintext, then relaxing the 
> HTTPS requirement won't help you either.
>
> Arnt
>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 17 Sep 2018 13:26:46 +0200

Maybe I misunderstand... what does requiring support for plain-HTTP 
transport achieve compared to decrypting connections?

Arnt

---

From: Aki Tuomi <[email-masked]>
Date: Mon, 17 Sep 2018 14:28:07 +0300

> transport achieve compared to decrypting connections?

Or you implement debug logging in your client / server so that it can be
instructed to store the raw chatter in some log file?

Aki Tuomi

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 17 Sep 2018 12:34:13 +0100

> instructed to store the raw chatter in some log file?

+1.

---

From: Andri Möll <[email-masked]>
Date: Mon, 17 Sep 2018 11:39:37 +0000

> instructed to store the raw chatter in some log file?

Have either of you implemented some server (be it IMAP, DAV+CalDAV etc) 
by spec and then discovered that none of the major client-side 
implementations work as they should and all require hours of trial and 
error with multiple hacks to properly work? Printing to syslog on 
production and deploying every time to test miniscule changes is a 
brain-dead way to develop, not to mention it only works if you have an 
Internet connection. Don't kid yourself that the implementations of 
tomorrow's standards will in any way be better than yesterday's.

> transport achieve compared to decrypting connections?

Arnt, I answered that in my previous email, but if it's not clear, 
imagine, how would you get a certificate for 10.1.2.3 that iOS would 
accept? What if the client app comes with its own bundled CA roots? You 
won't get anything to decrypt in the first place.

---

From: Nazim Can Bedir <[email-masked]>
Date: Mon, 17 Sep 2018 14:47:14 +0300

Hi,

Instead of trying to get a certificate for 10.1.2.3, why don't
    - create (let's say) dev3.uber-jmap-client.acme.com record with 
public IP address first
    - get a certificate for it
    - change DNS record to point 10.1.2.3
    - use dev3.uber-jmap-client.acme.com instead of IP address in the client

Yes, I understand what you are saying. But this kind of change (relaxing 
HTTPS requirement) could open an attack opportunity for bad guys.

Regards,
Nazim Can.

On 17/09/2018 14:39, Andri Möll wrote:
> Have either of you implemented some server (be it IMAP, DAV+CalDAV 
> etc) by spec and then discovered that none of the major client-side 
> accept? What if the client app comes with its own bundled CA roots? 
> You won't get anything to decrypt in the first place.
>
>

---

From: Andri Möll <[email-masked]>
Date: Mon, 17 Sep 2018 12:09:56 +0000

I was waiting for someone to offer that solution. ^_^ While doable, 
quite a pain in the ass. It requires you have access to the router _and_ 
that the router has been "jailbroken" to run a DNS server with which you 
can overwrite public domains. If not, you're again required to have an 
always-on Internet connection to just write some code by yourself. 
Having to solve a self-imposed development-problem is just a facepalm 
IMO. I'd much rather avoid the problem in the first place.

> But this kind of change (relaxing HTTPS requirement) could open an 
> attack opportunity for bad guys.
A decent warned override method or a toggle in the advanced settings of 
a client app is entirely sufficient to permit reasonable development and 
prevent accidental insecure connections. As said, something every 
mail/calendar app today already provides.

On 9/17/18 11:47 AM, Nazim Can Bedir wrote:
> Yes, I understand what you are saying. But this kind of change 
> (relaxing HTTPS requirement) could open an attack opportunity for bad 
> guys.
>
> Regards,
> Nazim Can.
>
>> implementations work as they should and all require hours of trial 
>> and error with multiple hacks to properly work? Printing to syslog on 
>> brain-dead way to develop, not to mention it only works if you have 
>> an Internet connection. Don't kid yourself that the implementations 
>> of tomorrow's standards will in any way be better than yesterday's.
>>

---

From: Sudi <[email-masked]>
Date: Mon, 17 Sep 2018 06:20:09 -0700

This should work? 
https://www.charlesproxy.com/
-Sudi
📲 

> On Sep 17, 2018, at 5:09 AM, Andri Möll <[email-masked]> wrote:
> 
> I was waiting for someone to offer that solution. ^_^ While doable, quite a pain in the ass. It requires you have access to the router _and_ that the router has been "jailbroken" to run a DNS server with which you can overwrite public domains. If not, you're again required to have an always-on Internet connection to just write some code by yourself. Having to solve a self-imposed development-problem is just a facepalm IMO. I'd much rather avoid the problem in the first place.
> 
>> But this kind of change (relaxing HTTPS requirement) could open an attack opportunity for bad guys.
> A decent warned override method or a toggle in the advanced settings of a client app is entirely sufficient to permit reasonable development and prevent accidental insecure connections. As said, something every mail/calendar app today already provides.
>>    - create (let's say) dev3.uber-jmap-client.acme.com record with public IP address first
>> Yes, I understand what you are saying. But this kind of change (relaxing HTTPS requirement) could open an attack opportunity for bad guys.
>> 
>> Regards,
>> Nazim Can.
>> 
>>> Have either of you implemented some server (be it IMAP, DAV+CalDAV etc) by spec and then discovered that none of the major client-side implementations work as they should and all require hours of trial and error with multiple hacks to properly work? Printing to syslog on production and deploying every time to test miniscule changes is a brain-dead way to develop, not to mention it only works if you have an Internet connection. Don't kid yourself that the implementations of tomorrow's standards will in any way be better than yesterday's.
>>> 
>>> Arnt, I answered that in my previous email, but if it's not clear, imagine, how would you get a certificate for 10.1.2.3 that iOS would accept? What if the client app comes with its own bundled CA roots? You won't get anything to decrypt in the first place.
>>> 
>>> 
>>>

---

From: Ted Lemon <[email-masked]>
Date: Mon, 17 Sep 2018 10:09:52 -0400

> A decent warned override method or a toggle in the advanced settings of a client app is entirely sufficient to permit reasonable development and prevent accidental insecure connections. As said, so…

Yes, and these are routinely used to phish or snoop on naive users.   There is no technical reason why you should need to be able to force a downgrade.   It's just the debugging flow you're comfortable with.   That's understandable, but hardly reason enough to make the protocol insecure.   E.g., if you had only HTTPS CalDAV, which I think is what you should only have, then you would indeed have to work harder to debug the problem.   And that would be okay.

We had a very long discussion about this in the TLS WG over the past year, and the conclusion we reached was "no, we're not going to break TLS to make debugging easier."

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 17 Sep 2018 10:35:36 -0400

Hi Andri,

--On September 17, 2018 at 11:39:37 AM +0000 Andri Möll <[email-masked]> 
wrote:

> accept?

On this one issue: you can use Apple's Configurator.app (available on Mac 
app store) to install a trusted root certificate on an iOS device. What I 
typically do is:

1) Use Keychain.app on macOS to create a "Certificate Authority" certificate
2) Export that cert as a PEM file
3) Use Keychain.app to create a "leaf" certificate signed by the CA cert
4) Install the leaf in the server (and include the CA cert in the server's 
"chain" file)
5) Use Configurator.app to create a certificate profile containing the CA 
cert
6) Use Configurator.app to install the profile on an iOS device

Steps 1-5 only need to be done once, then you can install on as many 
devices as you want.

I have used this process for both development and for my personal 
"production" server.

-- 
Cyrus Daboo

---

From: Brandon Long <[email-masked]>
Date: Mon, 17 Sep 2018 22:18:30 -0700

On Mon, Sep 17, 2018 at 7:10 AM Ted Lemon <[email-masked]> wrote:

> A decent warned override method or a toggle in the advanced settings of a
> client app is entirely sufficient to permit reasonable development and
> Yes, and these are routinely used to phish or snoop on naive users.
> There is no technical reason why you should need to be able to force a
> downgrade.   It's just the debugging flow you're comfortable with.   That's
> understandable, but hardly reason enough to make the protocol insecure.
> E.g., if you had only HTTPS CalDAV, which I think is what you should only
> have, then you would indeed have to work harder to debug the problem.   And
> that would be okay.
>
> We had a very long discussion about this in the TLS WG over the past year,
> and the conclusion we reached was "no, we're not going to break TLS to make
> debugging easier."
>

+1.  You're making things less secure for millions of users in order to
make your life a bit easier.

Brandon