Subject: Re: [Acme] case in point of usability

From: Stephen Farrell <[email-masked]>
Date: Wed, 01 Apr 2015 10:15:48 +0100

Hi Carl,

On 01/04/15 06:16, Carl Mehner wrote:
> If we do want to put these type of considerations in the draft,
> maybe the security considerations section is the best place.
> Something along the lines of:
> 
> When preparing to use the new certificate received from a issuance or
> refresh, the client software should check that the OCSP response from
> the certificate authority is valid before enabling the new certificate
> for use in the server system. If the OCSP response is requested too
> early by the server system, a 'revoked' or 'unknown' OCSP response may
> be cached and cause browsers to fail connection attempts.

Maybe. OTOH, my "client s/w" in this case is the openssl CLI
and that's fairly gigantically crap. I did get it to emit an
OCSP request that was sent somewhere but only ever got an
error response. Before I figured that out I found the set of
postings from other folks who'd suffered the same issue so
I stopped playing.

I'd argue that stuff like this ought be catered for by acme,
but we can have that discussion down the road when/if we're
standardising the protocol in a wg.

S.

PS: My Cullen-moment here was also extended because I got bad
warnings from browsers - FF said "bad issuer" and it was only
when I got to my phone and running the qualsys tests that I
saw that it was actually an ocsp issue.

---

From: Rob Stradling <[email-masked]>
Date: Wed, 01 Apr 2015 10:44:13 +0100

<snip>
> be cached and cause browsers to fail connection attempts.

The CA's OCSP infrastructure might consist of many servers that are not 
necessarily perfectly synchronized.  So the ACME client may be able to 
obtain a "good" OCSP response for a recently issued certificate, but 
some other clients may get a different response.

Only the ACME server (the CA) could possibly know for certain that all 
of the servers in its OCSP infrastructure have become aware of the 
recently issued certificate.

Perhaps an ACME client should be able to ask an ACME server "Is your 
OCSP infrastructure fully aware of this cert yet?"

Or perhaps the ACME draft should simply say that TLS servers SHOULD 
enable OCSP Stapling, so that TLS clients are less likely to encounter a 
"requested too early" OCSP response.

-- 
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 01 Apr 2015 13:19:18 +0100

And in a happy ending for this thread, when I whacked in
the new cert today it all worked.

Interestingly, there was a point where it was fine in all
but one browser - that lasted an hour or so before they
were all ok, not quite sure why, but there's clearly more
going on with ocsp caching than I know about;-)

S.

On 31/03/15 16:11, Stephen Farrell wrote:
> 
> So today I was updating a web server cert as I do a few
> times a year. And I have a usability story to tell...
> 
> I got the new cert and installed it in apache without any
> Cullen-like problems:-) That cost me €0.00 in payment and
> about 5-10 minutes. All good so far.
> 
> Chrome was happy, but FF/opera/my phone weren't.
> 
> I then messed about for 30 minutes checking to see if
> a new intermediate cert was needed etc. (i.e. I was back
> to Cullen-mode:-)
> 
> Turns out after a bit of searching, I'd installed the new
> cert too soon, and when I tested it, a "dunno" OCSP
> response was sent before the responder had seen the new
> cert and that OCSP response has now been cached for some
> unknowable (to me) number of hours in who-knows-what
> places. And that caching behaviour has changed since the
> last time I got a cert from the same provider a few months
> ago. So I reverted my apache to the old cert and will
> try install the new cert again tomorrow.
> 
> That's exactly that kind of thing I'd love to see fixed
> with acme and that is not handled by CMP, CMC, PKCS#10,
> EST or SCEP. At least I don't believe there's a standard
> way of getting the right thing to happen with those
> without some proprietary extension/surroundings.
> 
> And one big reason CMP etc don't support that is that we
> didn't have that requirement when we had the big fight
> that lead to CRMF back nearly 20 years ago. (Since OCSP
> didn't exist then and we didn't know how folks would be
> updating web servers, and we're much more intolerant of
> Cullen-like messing about being needed these days, and
> rightly so.)
> 
> I would like acme defined so that when I get the cert
> back all the PKI stuff has happened already and is
> working. I'm sure some other semantics could also work
> out, (e.g. if acme had a "ready-yet?" query I could
> emit after getting the cert), but those are the kind
> of problems we're currently facing that are killers and
> that we can address, now that we know the deployment
> requirements much better than we did in 1996.
> 
> I hope this helps those who are worried that acme is
> only about business models. In my head what acme ought
> be about is getting rid of that 1 hour of silly sysadmin
> time I just spent - the system-automated web server s/w
> update should just have done all of this for me without
> me even having to know a new cert was needed until I
> get the system update email tomorrow.
> 
> Cheers,
> S.
> 
> PS: Apologies, Cullen but it's your own fault:-)
> 
> _______________________________________________
> Acme mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/acme
>

---

From: Ben Laurie <[email-masked]>
Date: Wed, 1 Apr 2015 13:38:29 +0100

> going on with ocsp caching than I know about;-)

Are you sure it wasn't the intermediate problem you were originally
investigating?

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 01 Apr 2015 13:51:04 +0100

> Are you sure it wasn't the intermediate problem you were originally

I am sure in the sense of "I did what the forum post said and
it worked," yes:-) [1]

And while the CA has produced a new chain, the old intermediates
chain works fine and is what's in place now and working with all
browsers. I did check the new chain differs from the old one, but
I didn't bother checking exactly how, other than the dates. So I
guess it really was OCSP.

But from the acme POV, it doesn't really matter what was going on
in the OCSP infrastructure - the point was that the renewal process
didn't take that into account at all, leading to user-visible errors.
Bleh. And something where we can do better with acme now we (well,
some of we), know how OSCP caching happens in CDNs and all that.
(Or, as Rob said, we could go all stapled with acme maybe.)

S.

[1] https://forum.startcom.org/viewtopic.php?f=15&t=2654

---

From: Warren Kumari <[email-masked]>
Date: Wed, 1 Apr 2015 09:00:34 -0400

> Chrome was happy, but FF/opera/my phone weren't.

[ Long tale of woe snipped ]

> get the system update email tomorrow.

Something that is obvious, but probably worth mentioning is that you
actually have /some/ clue about this sort of thing, and were still
foiled.
Most admins have no idea what OSCP, CMP, CMC, PKCS#10, EST, SCEP, CRMF
or even a PKI *are* -- nor should they have to....

> PS: Apologies, Cullen but it's your own fault:-)

I missed Acme, but Cullen's sadness has been mentioned many times now
-- sounds like I missed much entertainment...
W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 1 Apr 2015 09:18:29 -0400

> (Or, as Rob said, we could go all stapled with acme maybe.)

One of the reasons that I do not like unnecessary complication or
variation is that it leads to this sort of situation.

Right now most servers are administered in bespoke fashion and so the
variation in configuration can be quite large. Automating the process
takes the human out of the loop and makes it much easier to get
consistent results.

It is the same problem I have with UNIX admin. Anyone who has ever
tried to administer a machine where the previous sysadmin has spent
their time trying to make Ubuntu look like Solaris and reorganized the
directories accordingly will know what I mean. Every administration
action becomes a hunting expedition. Is the file in the usual place or
was it moved?

Today I am going to have to spend a couple of hours changing a code
generator because someone thought "x5c#256" was a good name for a
structure id. Sure, this is legal JSON but why make unnecessary work?
I now have to implement a hack to serve as a work around. And every
other implementer is also going to have to do a work around. And that
means that all our APIs are going to be a little different when they
should have been the same.

---

From: Nico Williams <[email-masked]>
Date: Wed, 1 Apr 2015 11:47:06 -0500

> try install the new cert again tomorrow.

So the problem is caching of "unknown" status OCSP Responses.

I think this is a separate and separable problem, though one that ACME
should work on anyways.

Nico
--