Subject: [DNSOP] .PR servfails due to wrong key in DLV

From: Paul Wouters <[email-masked]>
Date: Tue, 8 Sep 2009 11:58:35 -0400 (EDT)

On Tue, 8 Sep 2009, Stephane Bortzmeyer wrote:

[added [email-masked] to the reply]

> Subject: [Unbound-users] .PR servfails with Unbound but not with BIND

> % dig SOA pr.

> I get the key through DLV.

It's outdated and wrong and missing the new key.

On Aug 19 2009, pr added this key:

> PR. IN DNSKEY 257 3 5 AwEAAeDPv9lQ7Ej5Ld9Fz/FKLhdOajwtEXsWykj65ugIa4Di1nY6ti9n
dkeR4kp1aSNlvf6N7KsjunfMJj4SccBwcY77DrxmQ+g9nI09ePMZvxF2
U63Lv9BftGaIguYdkYZVSwHd1q7DdXqNkLaD4tZEHiN0h/3wBdTQUPH1
IoskD1vGxiPw2egftk6sVQdvOJWaAgSpmG0eq+/e90WVTNX4/xhA17Pr
dQQJIheZQ3+EsDoil8kyJZC12KoHYpFklx7+aCiR2u8Fumy6ARFR4PP0
n7bnBaKOgMpVzz+KI79a3USDkj9RhNog50iSWgaBM75Xu0IBNEpcCVYZ
YjwDESgiDXc=

And on Sep  4 2009, pr removed this keys:

< PR. IN DNSKEY 257 3 5 AwEAAc6SkFSHw00wJFUWd1Td/efsxhfX+UTrxrzqQXNuZ8Qj2PiP6p/m
BxysJt06XgSCB41CPhkgvgqrtdaJ/hXKG81xNXUcGfqvV9wYMJnN+oBB
/lLaQU/39fWaNc4fBGiRI2dNDVKPry2YX6y04YrEGRM+wf6HWHVdW1Js
xuMuDOSr

> % dig DLV pr.dlv.isc.org.

> ;; ANSWER SECTION:
> pr.dlv.isc.org.		3255	IN	DLV	62704 5 2 57E017A982196D194B3F52CDD39F86A9A33DED75064F285A9242BA7A 448A659C
> pr.dlv.isc.org.		3255	IN	DLV	62704 5 1 AFA72CB11D4C97657D82338AF6D569ED614166EB

These are the old key, and that DLV record should be removed. The new DLV record should be:

pr.dlv.isc.org. IN DLV 6277 5 2 6966580bb25c608540e8224039561c7b2a1488d1f927c5cdbd137f4ef3d31528
pr.dlv.isc.org. IN DLV 6277 5 1 05d02dce8385974d958a5db409f6ff3658293b2

I guess we need a MUCH better communication method between TLD's, iTAR and ISC's DLV. This is bad.

Paul

---

From: Edward Lewis <[email-masked]>
Date: Tue, 8 Sep 2009 12:52:05 -0400

Trimmed the cc to avoid cross-posting.

>and ISC's DLV. This is bad.

What do we need?

(I'm asking in the sense of agreement, but the follow up to this 
needs a broader scope.  This isn't about the symptoms, it's about 
shoring up the operational aspects of DNS such that the security 
doesn't break operations.)

The general problem is the transfer of provisioning information from 
one zone administration to another.  This has been an ignored link, 
treated in EPP by RFC 4310, in itself a decent effort but in the 
larger scheme of things still a drop in the bucket.  (EPP isn't even 
from the DNS operator of the child to the DNS operator of the parent, 
EPP is between other elements in the food chain.)

When an SEP is minted, the public and private components have to be 
provisioned in a number of places.  The public key has to get into 
the zone.  The public and private key files have to be made available 
to the signing application.  And the public key has to be in position 
to become a DS RR and shipped to the parent.  This is true whether 
the child is a registrant in a registrant-registrar-registry model 
and the parent is TLD, or when the child is a TLD and the parent is a 
combination of the root zone and a trust anchor repository.

None of these interfaces have been treated by the standards bodies. 
When designing a TLDs DNSSEC mechanics the interface and protocol for 
publicizing the SEP is still a grey "TBD" box.  That's a risk area.

How do we securely automate a DNS provisioning interface/mechanism? 
It's more than secure dynamic update.

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

As with IPv6, the problem with the deployment of frictionless surfaces is
that they're not getting traction.

---

From: David Conrad <[email-masked]>
Date: Tue, 8 Sep 2009 09:56:58 -0700

Hi,

> It's outdated and wrong and missing the new key.

Out of curiosity (since I'm not on the unbound-users list), why did it  
work with BIND and not Unbound?

> iTAR and ISC's DLV. This is bad.

3 points:

1) Get used to these sorts of failures.  In the universe of TLD  
operators, there are a non-trivial number that have limited technical  
skills.  I can easily imagine folks hiring consultants who come in,  
set up DNSSEC, and then leave.  Time passes, keys expire, servers  
change, etc.

2) ISC redistributes the ITAR without any formal or even informal  
interaction with the ICANN staff that runs the ITAR.  In some  
particularly unpleasant (at least to me) alternative universe, there  
could be a myriad of DLV registries.  How should ICANN interact with  
these DLV registries?

3) Of course, DLV deals with more than TLDs so you can probably take  
point (1) above and multiply the fun by some non-trivial amount.

Regards,
-drc

---

From: Paul Wouters <[email-masked]>
Date: Tue, 8 Sep 2009 13:13:51 -0400 (EDT)

>> and ISC's DLV. This is bad.

- A tool for key owners to detect proper progatation of their new key, so
   they know when it is safe to complete the rollover by removing the old
   trust anchor. This should be part of any DNSSEC Signer solution.

- Better checks in the ISC DLV. It should have spotted this. Perhaps it did,
   but did not know who to contact.

- A tool for resolvers to detect and roll over trust anchors (eg autotrust
   deployed)

> None of these interfaces have been treated by the standards bodies. When 
> designing a TLDs DNSSEC mechanics the interface and protocol for publicizing 
> the SEP is still a grey "TBD" box.  That's a risk area.

At least for TLD's, we expect this to be fixed in December with a signed root.

I am not sure what appliance or software setup '.pr' uses, but it should have
never allowed to finish the key rollover with the bad key in the ISC DLV.

Paul

---

From: Edward Lewis <[email-masked]>
Date: Tue, 8 Sep 2009 13:23:51 -0400

>At least for TLD's, we expect this to be fixed in December with a signed root.

The root may be signed by then.  But I have not seen plans for an 
interface for the TLDs to use to submit key material.

>never allowed to finish the key rollover with the bad key in the ISC DLV.

Avoiding addressing just the symptoms, I will say that the most 
annoying part in documenting SEP supercession is that we have to 
"wave our hands" at that point and say "when we notice that the 
$parent has changed the DS record we proceed" to the next step. 
Besides this hand waving, it's impossible to script this into a test 
plan.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

---

From: David Conrad <[email-masked]>
Date: Tue, 8 Sep 2009 11:23:16 -0700

> never allowed to finish the key rollover with the bad key in the ISC  

Hmm.

So, in order to roll a key, you have to ensure DLV registries have  
replaced the keys, even when the DLV registries obtain the originals  
indirectly?

Seems a bit broken to me.

Regards,
-drc

---

From: Chris Thompson <[email-masked]>
Date: 08 Sep 2009 19:52:13 +0100

>never allowed to finish the key rollover with the bad key in the ISC DLV.

ISC supposedly get their data for TLDs from the IANA ITAR. That's certainly
up to date now at https://itar.iana.org/anchors/anchors.xml but it would be
more than interesting to know how long that has been the case. (As I recall,
PR had to be chivvied into joining the ITAR in the first place.)

Given that the ITAR *is* now up to date, can anyone from ISC comment?
(I had expected to find it already fixed after coming back from the pub,
but no...)

-- 
Chris Thompson
Email: [email-masked]

---

From: Edward Lewis <[email-masked]>
Date: Tue, 8 Sep 2009 15:12:09 -0400

>Seems a bit broken to me.

That's not broken, that's reality.  My guidance is that we 
(operators) have to take reasonable steps to prevent relying parties 
from suffering consequences.  When an emergency supercession is 
needed, a nasty choice may need to be made.

Instead of complaining more about this, we need to figure out what 
why this is a weakness in DNSSEC operations and come up with a 
solution.

It's too late for weeping and wailing...
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

---

From: Paul Wouters <[email-masked]>
Date: Tue, 8 Sep 2009 15:26:51 -0400 (EDT)

On Tue, 8 Sep 2009, Edward Lewis wrote:

>> Hmm.
>> 
>> So, in order to roll a key, you have to ensure DLV registries have replaced 
>> the keys, even when the DLV registries obtain the originals indirectly?
>> 
> That's not broken, that's reality.  My guidance is that we (operators) have 
> to take reasonable steps to prevent relying parties from suffering 
> consequences.  When an emergency supercession is needed, a nasty choice may 
> need to be made.

Indeed. And TLD's should check the iTAR and DLV. Perhaps also leave the old
key a little bit longer then two weeks for people to have a little bit more
time to respond.

Putting TLD's into DLV was not ideal anyway. I understand it was easier on a
lot of people. I've always seen TLD configuration (as long as the root is not
signer) as a seperate issue from putting domains in unsigned parents into the
DLV. Hence the use of dnssec-conf to configure the TLD keys along side the
DLV usage.

Paul

---

From: [email-masked]
Date: Tue, 8 Sep 2009 19:30:12 +0000

a few of us actually did a little work in this area three or four years
ago - did working proof of concepts - and were promptly ignored.
(the claim was - this work was premature)

--bill

On Tue, Sep 08, 2009 at 01:23:51PM -0400, Edward Lewis wrote:
> _______________________________________________
> DNSOP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsop

---

From: David Conrad <[email-masked]>
Date: Tue, 8 Sep 2009 12:35:46 -0700

Ed,

On Sep 8, 2009, at 12:12 PM, Edward Lewis wrote:
>> replaced the keys, even when the DLV registries obtain the  
>> originals indirectly?
> That's not broken, that's reality.

I disagree. Requiring every zone administrator that has signed their  
zone to update folks they may not have the slightest idea about before  
being able to roll a key is fundamentally broken.

I do not know the circumstances that resulted in ISC's DLV getting out  
of sync with the ITAR, however since ISC harvests ITAR data, I believe  
the responsibility is on ISC to keep the data up to date.  In theory,  
we (ICANN) could broadcast changes to re-distributors of the ITAR as  
the occur, but what protocol should we use and what happens if  
somebody misses an update?

> My guidance is that we (operators) have to take reasonable steps to  
> prevent relying parties from suffering consequences.

Lovely idea.  When Bill's Bait and DLV Registry gets the ITAR data  
from a third party and you, as a signed TLD zone operator, have no  
relationship with either the third party or Bill's B&DLVR, how do you  
propose doing this?  Now extend that down into the rest of the tree.   
How are zone administrators going to propagate these changes?

> When an emergency supercession is needed, a nasty choice may need to  
> be made.

Well, yes.  But somewhat irrelevant.

> why this is a weakness in DNSSEC operations and come up with a  

Correct me if I'm wrong, but the architecture of DNSSEC assumed  
(rightly or wrongly) a single hierarchical deployment model.  ISC has  
now thrown a non-hierarchical component to deployment, DLV, into the  
mix. It isn't particularly surprising that this has an impact on  
DNSSEC operations.  Perhaps the solution is to not use DLV?

> It's too late for weeping and wailing...

As current US politics empirically prove, it is NEVER too late for  
weeping and wailing.

Regards,
-drc

---

From: Kim Davies <[email-masked]>
Date: Tue, 8 Sep 2009 12:42:18 -0700

> (I had expected to find it already fixed after coming back from the pub,

I do not intend to speak for anyone at ISC, but I believe they stated on
this list previously they sync with the ITAR data on a weekly basis.

As for when the current .PR key was listed on the interim trust anchor
repository at IANA, 2009-09-01 21:45:06.072 UTC would be the precise time.

Kim

---

From: Edward Lewis <[email-masked]>
Date: Tue, 8 Sep 2009 16:19:30 -0400

At 12:35 -0700 9/8/09, David Conrad wrote:

>Correct me if I'm wrong, but the architecture of DNSSEC assumed (rightly or
>wrongly) a single hierarchical deployment model.

Ok, if I must.  DNSSEC does not assume a single hierarchical 
deployment model.  If it did, there would not be a reason to identify 
the signer in the RRSIG.  (Archeological evidence also includes - 
2065 was DNSSEC I, 2535 was DNSSEC II, but it was not until RFC 3008 
that the recommendation was made to have the authorization to sign 
data follow the tree.)

The short version of why RFC 3008 came about was that we found it too 
hard for our meager research effort to define a policy language (to 
express acceptable signatures for data) and implement it in the BIND 
8 we were hacking to support the dream of a fully general 
authorization model.  Mind you, the dreams then were fueled by 
researcher more in with the security crowd than with the DNS crowd.

(I do remember many strawman models I worked up, one that I 
visualized with a chain of paper clips.  Yes, basic research.)

>ISC has now thrown a non-hierarchical component to deployment, DLV, into
>the mix.

ISC has exercised the principle of "local policy overrides" all.  DLV 
is perfectly within the heart of the soul of DNSSEC.

I should acknowledge that I am not a fan of DLV.  I am a little 
fearful of it.  There are many things about it I don't like.  Still, 
what it is attempting to do is within limits.

>It isn't particularly surprising that this has an impact on DNSSEC
>operations.  Perhaps the solution is to not use DLV?

I'm not one to throw stones, considering I live in a glass house 
(albeit the glass is still covered by plywood), but one "solution" 
regarding DLV by ISC is to continue to stamp out the evil operations 
gremlins.  I don't mean to pick, but ISC's operational expertise is 
mostly in the operation of F root, not in operating a provisioning 
registry.  20-20 hindsight, it's now apparent that in (say) January 
2009, ISC had a lot to learn ahead of it.  (I'm not going to make any 
statement about their operations chops now.)

>As current US politics empirically prove, it is NEVER too late for weeping
>  and wailing.

What would impress me is a proposal to sign the root zone that 
included a real good look at the provisioning interface as well as an 
OT&E.  For instance, promise me a response to a DS change in a matter 
of hours or minutes, not a day.

PS - To date we have more operational experience with DLV than a 
signed root.  More and more I will know what to anticipate when 
rolling DNSSEC out the door, at least as far as DLV is concerned.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 8 Sep 2009 22:32:01 +0200

On Tue, Sep 08, 2009 at 11:23:16AM -0700,
 David Conrad <[email-masked]> wrote 

> Seems a bit broken to me.

You use the plural but there is today only one DLV registry in active
use. Since the root is not signed, a reasonable TLD operator which
signs with DNSSEC has only one point to watch before dismissing his
old key.

If it is too much work, then, indeed, something is broken.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 8 Sep 2009 22:35:49 +0200

On Tue, Sep 08, 2009 at 12:35:46PM -0700,
 David Conrad <[email-masked]> wrote 
 a message of 56 lines which said:

> Perhaps the solution is to not use DLV?

Sure, every DNS resolver sysadmin should follow the various trust
anchors (remember there are not only TLD) and update them when they
change. And you claim it would be operationally easier and safer than
DLV?

Today, very few people check the DNSSEC signatures (otherwise, the
failure would have been detected much earlier). Not using DLV would
make this number go from "a very small number of geeks" to "near
zero".

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 8 Sep 2009 22:38:20 +0200

On Tue, Sep 08, 2009 at 09:56:58AM -0700,
 David Conrad <[email-masked]> wrote 

> work with BIND and not Unbound?

Probably a caching effect and not a real difference between the
resolvers.

My BIND resolver now fails as well.

---

From: Jim Reid <[email-masked]>
Date: Tue, 8 Sep 2009 22:19:29 +0100

On 8 Sep 2009, at 21:19, Edward Lewis wrote:

> included a real good look at the provisioning interface as well as  
> an OT&E.  For instance, promise me a response to a DS change in a  
> matter of hours or minutes, not a day.

Ed, what would impress me is a signed root zone. Delivered some time  
before my yet unborn grandchildren retire. :-)

Yes, the provisioning interface and an OT&E environment would be nice.  
So would tools and processes to get DS changes propagated in the  
timescales you mentioned (or better). However these can come later  
IMO. The first thing is to establish the solid foundation of a signed  
root.

There will be plenty of time after there's a signed root to work on  
these things. Few TLDs are signed yet and few of them are signing  
delegations. There's only a small bunch of people who have switched on  
validation. So the impact of a dud TLD key or failed rollover isn't  
going to be a big deal. [Did the recent problem with .pr cause the sky  
to fall in?] At least not today. And maybe not for another year or two.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 09 Sep 2009 10:24:58 +1000

In message <[email-masked]>, Paul Wout
> /lLaQU/39fWaNc4fBGiRI2dNDVKPry2YX6y04YrEGRM+wf6HWHVdW1Js

	Which is a ridiculously short key rollover period (16 days)
	when the parent is not signed.  Lots of tracking of TLD
	keys is completely manual.
 
> > pr.dlv.isc.org.		3255	IN	DLV	62704 5 2 57E017A982196
> D194B3F52CDD39F86A9A33DED75064F285A9242BA7A 448A659C
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Mark Andrews <[email-masked]>
Date: Wed, 09 Sep 2009 10:57:48 +1000

In message <[email-masked]>, David Conrad
> Seems a bit broken to me.

David, is there a published maximum poll interval for ITAR.  i.e. you
must poll this regularly or you may miss a rollover event.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 09 Sep 2009 11:07:21 +1000

> repository at IANA, 2009-09-01 21:45:06.072 UTC would be the precise time.

So ITAR consumers had 2 days to respond to this key rollover event.

Did PR inform you immediately the DNSKEY was added to the PR zone?

What happened in the 14 days between the DNSKEY being added to the
zone and it appearing in ITAR?

Mark

---

From: Kim Davies <[email-masked]>
Date: Tue, 8 Sep 2009 18:14:03 -0700

> zone and it appearing in ITAR?

The ITAR listing process is essentially automatic, but relies on the TLD
operator actually submitting a request to list via a web form. It is up to
the TLD operator to submit trust anchors to us when they are ready. The only
check we do is we will not list a trust anchor until there is a matching
DNSKEY in their zone.

We have no unique insight into the key management policies of the TLD
operators. We do not monitor TLD zones for DNSKEYs that are not in the ITAR
and give them courtesy notes that they are absent (maybe we should?).

I think the questions on rollover planning are best left for each TLD to
provide, it is not something we have any restrictions on.

kim

---

From: Mark Andrews <[email-masked]>
Date: Wed, 09 Sep 2009 11:26:47 +1000

In message <[email-masked]>, David Conrad
> being able to roll a key is fundamentally broken.

Normal operations with a signed parent doesn't require you to inform
everyone.  "DS TTL + Parent Refresh" after the visible parents have
changed should be enough.  The child's key manager can work this
out from data published in the DNS.  You just don't start the timer
ticking until you see the parent react.
 
The root should be using RFC 5011, which provides timing guidance,
for all clients.  RFC 5011 support will be built into the validators
if it is not already there.

> DNSSEC operations.  Perhaps the solution is to not use DLV?

The ITAR is also not part of the standard proceedures .... 
The root should have been signed years ago ....
There's lots of places to point fingers.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 09 Sep 2009 11:56:09 +1000

In message <[email-masked]>, Kim Davies writes:
> >> repository at IANA, 2009-09-01 21:45:06.072 UTC would be the precise tim> e.
> > 
> the TLD operator to submit trust anchors to us when they are ready. The onl> y

So PR took 14 days to submit to ITAR then pulled the old one 2 days later
leaving a effective rollover period of 2 days for anyone using ITAR.

I've talked to the DLV project manager and PR's DLV record is now updated.

Mark

---

From: Michael Graff <[email-masked]>
Date: Wed, 09 Sep 2009 05:27:08 -0500

> this list previously they sync with the ITAR data on a weekly basis.

I believe it may be time to switch to daily updates.  It's a simple
https pull from a web site for us, there is no reason we could not do it
more frequently.

- --Michael

---

From: Chris Thompson <[email-masked]>
Date: 09 Sep 2009 11:57:46 +0100

>https pull from a web site for us, there is no reason we could not do it

It might be helpful if IANA offered advice on that. To quote from
https://itar.iana.org/instructions/

| You should also ensure you have procedures in place to keep the trust
| anchors up to date as the ITAR will regularly add and remove keys.

but they don't define "regularly". If they had said "you must check
at least once a week", and also made that policy known to their TLD
clients, then we could castigate PR for the removing their old KSK
too soon. As it is, the finger of blame is oscillating wildly.

Ob-mention-of-RFC5011: of course. But incremental policy improvements
would help meanwhile.

-- 
Chris Thompson               University of Cambridge Computing Service,
Email: [email-masked]    New Museums Site, Cambridge CB2 3QH,
Phone: +44 1223 334715       United Kingdom.

---

From: Michael Graff <[email-masked]>
Date: Wed, 09 Sep 2009 06:04:24 -0500

> too soon. As it is, the finger of blame is oscillating wildly.

If no one actually did anything in malice, there is no finger of blame
here IMHO.  Just many fingers of learning.

Even with automated scripts, ITAR importing for most people is a
granular action; it's not done dynamically based upon some published TTL
or expiry.  Perhaps something as simple as "valid until" might be useful
for an ITAR like thing, but only so long as it comes from the submitter
of that key.

- --Michael

---

From: Chris Thompson <[email-masked]>
Date: 09 Sep 2009 13:06:40 +0100

On Sep 9 2009, Michael Graff wrote:

>           Perhaps something as simple as "valid until" might be useful

The https://itar.iana.org/add/ form implies that ITAR subscribers do
provide a "listing period" and an "effectivity period". Maybe these 
could be added to the exported version? After all, one point of using
XML ought to be that it is possible to add new tags transparently!

---

From: Edward Lewis <[email-masked]>
Date: Wed, 9 Sep 2009 10:38:56 -0400

At 22:19 +0100 9/8/09, Jim Reid wrote:

>delegations. There's only a small bunch of people who have switched
>on validation. So the impact of a dud TLD key or failed rollover isn't

As a TLD operator engineer - the lack of a specified interface and 
OT&E is an impact to me now.  For instance, I've already been asked 
"how are we going to make the public key public?"

It isn't a matter of getting a signed root OR having an interface and 
OT&E.  It's a matter of solving what we can solve now.  Not that 
anyone is just lying around, waiting for some politically based 
decree that will approve the signing of the root, but if they are, 
there are things we can work on.

More generally, we still have the issue of getting the SEP from one 
administration to another.  This problem also hits between SLD's (TLD 
registrants) and TLDs.  In the ICANN world - which I know is not 
every case - here's no direct link from DNS operator to the Registry 
and a protocol there would be nice to have.  And that is something we 
can engineer now.

---

From: Edward Lewis <[email-masked]>
Date: Wed, 9 Sep 2009 10:51:01 -0400

>for an ITAR like thing, but only so long as it comes from the submitter

...We take the first step down the path of certificates, X.509, and 
that sort of thing. ;)

We need to be careful.  The jargon used for DNSSEC has included 
phrases like, "a ZSK should expire in 3 months".  DNSKEY RRs do not 
have time in them, RRSIG RRs do - and only in there is absolute time 
ever apparent inside the DNS (system).  If we now attach absolute 
times to the presence of records (in this case the DS RR that is the 
result), we are introducing a new paradigm.

What happens when the parent disposes the DS RR at the end of the 
key's effectivity date?  Will the child know to remove the same 
DNSKEY, or at least mark it as revoked in RFC 5011 speak?  Do we need 
to define a signal mechanism from the parent to child?  These are the 
kinds of things that need to be thought about thoroughly - and why I 
say we need to treat the disease and not the symptoms.

Brainstorming is good and we do need that, but be aware we need to 
arrive at a fully fleshed-out solution.

---

From: Mark Andrews <[email-masked]>
Date: Thu, 10 Sep 2009 09:49:58 +1000

> arrive at a fully fleshed-out solution.

I don't think we need to make ITAR more complicated than it already
is as it is intended to go away once the root is signed.  Adding a
recommendation of daily checks would be a good thing.

It would be useful for IANA to send the TLD's that are using ITAR
something about this so they become aware of the need to factor in
a reasonable rollover period after adding the new key to the ITAR
along with a description of what happened when this didn't happen.

We can debate reasonable but 2-4 weeks seams to be the right ballpark.

Mark

---

From: David Conrad <[email-masked]>
Date: Thu, 10 Sep 2009 12:03:30 -0700

On Sep 8, 2009, at 1:19 PM, Edward Lewis wrote:
>> (rightly or
> deployment model. [...] but it was not until RFC 3008 that the  
> recommendation was made to have the authorization to sign data  
> follow the tree.)

Are you saying that 3008 has been deprecated?  If so, why the big  
stink about signing the root?

> ISC has exercised the principle of "local policy overrides" all.

Yep. If you decide to shoot yourself in the head, that's up to you.   
What I object to is tying the ability of zone managers to roll a key  
to you putting the gun to your temple.

> DLV is perfectly within the heart of the soul of DNSSEC.

Heart and soul?  Seriously?  Then perhaps it shouldn't be  
informational and the implementation should conform to the spec (or  
vice versa).

DLV is a bletcherous hack that should not exist and I publicly  
apologize to the world for mentioning its predecessor to Paul. Mea  
culpa.  Of course, one of the reasons I dropped it was that after  
thinking about it, I realized that it wouldn't scale and had  
stunningly bad operational and deployment characteristics if it ever  
gained any sort of following.

If you want DLV to be a long-standing component of the DNS  
infrastructure, I'd suggest:

- get the RFC out of informational and into standards track
- get somebody to implement the RFC (or, alternatively, document what  
has been implemented)
- figure out some way of propagating changes to potentially numerous  
unrelated DLV registries

> Still, what it is attempting to do is within limits.

And within the limits of local policy, that's fine.  What is simply  
broken is having that local policy have global impact.

> matter of hours or minutes, not a day.

Did you provide these comments to NTIA?  Have you provided comments to  
IANA staff regarding potential improvements to the ITAR?

> rolling DNSSEC out the door, at least as far as DLV is concerned.

There are multiple DNSSEC testbeds.  Are you making use of them?

Regards,
-drc

---

From: David Conrad <[email-masked]>
Date: Thu, 10 Sep 2009 12:17:55 -0700

Stephane,

On Sep 8, 2009, at 1:32 PM, Stephane Bortzmeyer wrote:
> use.

I was told previously that there was more than one.  Perhaps this was  
mistaken.  In any event, I don't believe DLV was designed to be  
monopolistic.

> Since the root is not signed, a reasonable TLD operator which

The issue is that ISC harvests data from the ITAR and they didn't  
synch the change submitted by PR to the ITAR.

Regards,
-drc

---

From: David Conrad <[email-masked]>
Date: Thu, 10 Sep 2009 12:25:31 -0700

> change. And you claim it would be operationally easier and safer than

Not at all.  However, one issue that was raised with DLV back when it  
was just a bad idea was that it would reduce the pressure on parents  
to sign zones.  I find I have greater sympathy with this view now.

Regards,
-drc

---

From: Edward Lewis <[email-masked]>
Date: Thu, 10 Sep 2009 15:36:04 -0400

At 12:03 -0700 9/10/09, David Conrad wrote:
>>  Ok, if I must.  DNSSEC does not assume a single hierarchical deployment
>>model. [...] but it was not until RFC 3008 that the recommendation was made
>>to have the authorization to sign data follow the tree.)
>
>Are you saying that 3008 has been deprecated?  If so, why the big stink about
>signing the root?

How did you draw the conclusion that RFC 3008 was deprecated?  It has 
been obsoleted by DNSSEC III (RFC 4033-4035).  What is said above is 
"the architecture of DNSSEC assumed a single ... model" is a false 
statement.

>What I object to is tying the ability of zone managers to roll a key to
>you putting the gun to your temple.

Again, you are jumping to conclusions.  DLV is well within the 
original concept for DNSSEC.  Whether or not there are operational 
issues with the implementation.

>Heart and soul?  Seriously?  Then perhaps it shouldn't be informational and
>the implementation should conform to the spec (or vice versa).

Yes, seriously.  Don't let paperwork get in front of code, i.e., it 
probably should be better documented, but that's not the bottleneck 
here.

>DLV is a bletcherous hack that should not exist and I publicly apologize to
>the world for mentioning its predecessor to Paul. Mea culpa.  Of course, one
>of the reasons I dropped it was that after thinking about it, I realized
>that it wouldn't scale and had stunningly bad operational and deployment
>characteristics if it ever gained any sort of following.

That may also be true.  In as much as the single hierarchical model 
didn't appear "until" RFC 3008, it did appear for a reason.  The 
reason was that trying to have a general purpose authorization model 
proved to be too onerous for the three of us writing code then and we 
"punted it for future generations."

>If you want DLV to be a long-standing component of the DNS infrastructure,
>I'd suggest:
>
>- get somebody to implement the RFC (or, alternatively, document what has
>   been implemented)

Again jumping to conclusions.  1) DLV fits the original concept.  2) 
The updated concept set that aside.  3) What we have today has 
exhibited operational problems.

At this point you can throw it out or work on it to make it better. 
That's not my call, but, at NANOG in Philly, one operator of a 
recursive server expressed a desire to have someone to rely upon for 
trust anchor management.

>And within the limits of local policy, that's fine.  What is simply broken
>is having that local policy have global impact.

The local policy of "trusting DLV" is not having a global impact, 
just a local impact on the parties relying on the cache with that 
policy.

>>  What would impress me is a proposal to sign the root zone that included a
>>real good look at the provisioning interface as well as an OT&E.  For
>>instance, promise me a response to a DS change in a matter of hours or
>>minutes, not a day.
>

To NTIA, yes, with acknowledgement.  As far as IANA, I've exchanged 
private email with Kim Davies and Richard Lamb about this (May 27 and 
28, 2009).  If you want, I can forward copies of the messages to you 
privately.

>>PS - To date we have more operational experience with DLV than a signed root.
>>More and more I will know what to anticipate when rolling DNSSEC 
>>out the door,
>>at least as far as DLV is concerned.
>

No - none have been appropriate for what we would want to test.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

---

From: David Conrad <[email-masked]>
Date: Thu, 10 Sep 2009 13:37:47 -0700

On Sep 10, 2009, at 12:36 PM, Edward Lewis wrote:
>> broken

Again, I am not objecting to people using DLV. I think it is ucky, but  
that's just me. What I am objecting to is the suggestion made here  
that _before a TLD that has submitted its keys to the ITAR rolls its  
keys, it must notify the (potentially multiple?) folks who run a DLV  
registry, of which the TLD may have no knowledge, who have harvested  
ITAR data and wait_.  That's just crazy talk.

> No - none have been appropriate for what we would want to test.

What do you want to test?

Regards,
-drc

---

From: Paul Wouters <[email-masked]>
Date: Thu, 10 Sep 2009 17:09:21 -0400 (EDT)

On Thu, 10 Sep 2009, David Conrad wrote:

> Again, I am not objecting to people using DLV. I think it is ucky, but that's 
> just me. What I am objecting to is the suggestion made here that _before a 
> TLD that has submitted its keys to the ITAR rolls its keys, it must notify 
> the (potentially multiple?) folks who run a DLV registry, of which the TLD 
> may have no knowledge, who have harvested ITAR data and wait_.  That's just 
> crazy talk.

A TLD should do due diligence. I mean, its their core business. Its the
ONLY thing they should do right. Make sure theit zone file works.

Even when this issue was found, they could have easilly added their old
key to the zone to ensure DLV would work until it got updated.

Arguing there might be 15 unknown DLV's is kind of beside the point.
Reality is, there is only on DLV they need to worry about.

Paul

---

From: Mark Andrews <[email-masked]>
Date: Fri, 11 Sep 2009 10:14:01 +1000

In message <[email-masked]>, David Conrad
> synch the change submitted by PR to the ITAR.

The issue is that ITAR has no policy on often the users of ITAR
should check or what a minimum key roll over period should be after
a update is submitted to ITAR.

Annually is "regularly". Monthly is "regularly". Weekly is "regularly".
Daily is "regularly".  Hourly is "regularly". Every "minute" is
"regularly".  Every second is "regularly".   Which "regularly" did
IANA mean when it said "regularly"?
 
PR created a effective key rollover period of 2 days which was
insanely short when there will be sites that are updating from ITAR
manually.

---

From: Mark Andrews <[email-masked]>
Date: Fri, 11 Sep 2009 10:38:20 +1000

In message <[email-masked]>, David Conrad
> ITAR data and wait_.  That's just crazy talk.

ITAR is supposed to being used by people to configure trust anchors
for TLDs until the root is signed.  There will always be a lag
between when data is entered into ITAR and when it is turned into
trust-anchors.  Whether it is updating named.conf directly or DLV
there will be a lag.

PR waited way too long to add the record to ITAR.  It should have
done it immediately after it published the DNSKEY.  There is not
point in waiting once the DNSKEY is published.  Instead PR waited
2 weeks then added the record to ITAR.  PR then waited 2 day and
hoped that everyone would have seen the ITAR change and updated
their configuration and pulled the record.

Now when ITAR goes away and the root is signed then the timing would
have been reasonable but not perfect as the timing would normally
have allowed the old DS RRset with only the old DNSKEY reference
in it to flush from the system.  To be perfect you need to wait
<parent expire> + <ds ttl> so that stealth parent servers that are
out of contact with their masters have expired the zone and any
records they served have also flushed from caches.

We need the EDNS EXPIRE option I proposed so the these timings can
be made deterministic rather than hope that there isn't a loop in
the axfr transfer graph which keeps a zone alive indefinitely.

Mark

---

From: Jim Reid <[email-masked]>
Date: Fri, 11 Sep 2009 09:42:06 +0100

Apologies for using a meaningful Subject: header....

> ONLY thing they should do right. Make sure theit zone file works.

True. Now please suggest how someone can perform that due diligence in  
circumstances where there are unknown (to them) actors between how  
they sign the zone and how others may or may not verify those  
signatures.

> key to the zone to ensure DLV would work until it got updated.

That's not the issue. [Let's leave aside the bizarre logic of "after  
you've had your heart transplant, keep the old, diseased one handy in  
case you might have to go back to it one day"."] Anyone lodging keys  
with the ITAR has no formal knowledge whatsoever over who is slurping  
data from the ITAR or what they might be doing with that data. Or who  
depends on whatever flavour of the month trust anchor that third party  
may have constructed. AFAICT there are no (service level) agreements  
underpinning these arrangements.

Rather than consider this in the abstract, let's now look at the  
specifics.
Is there an agreement between .pr and IANA about the ITAR? Probably. I  
don't know. Does anyone retrieving data from the ITAR have an  
agreement with IANA covering that? I very much doubt it. Has ISC  
told .pr "Be informed we're slurping your keys from the ITAR and  
feeding them to our DLV engine. This happens once a day/week/whatever.  
Bear this in mind if you ever roll a key"? I doubt it. Is there an  
agreement or written understanding between .pr and ISC on how both  
parties will behave wrt publishing or rotating .pr's keys? Probably  
not. IIUC ISC provides its DLV service under a best-efforts, hold  
harmless basis and there are no agreements between ISC and those using  
its DLV service.

So is it any surprise that Bad Things happen if a zone signer has to  
take into account someone else's processes and procedures that they  
don't know anything about whenever they do a key rollover?

> Reality is, there is only on DLV they need to worry about.

Prove it.

FYI, there have been occasional postings on bind-users about how to  
use and set up DLV. So some people have been playing with it. They  
might even have it working behind a campus/corporate firewall.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 11 Sep 2009 11:48:58 +0200

[Working for a TLD, so I believe I understand the problem of
interacting with the root, with the local Internet community, etc.]

On Fri, Sep 11, 2009 at 09:42:06AM +0100,
 Jim Reid <[email-masked]> wrote 
 a message of 55 lines which said:

> Now please suggest how someone can perform that due diligence in
> circumstances where there are unknown (to them) actors

If dlv.isc.org is an unknown actor to a DNSSEC-signed TLD, they indeed
have a big problem! 

The best proof that there was *no* "due diligence" is that on the very
own Web site of ".pr", they still publish the old key
<http://dnssec.nic.pr/serverconf.php>. Can you blame DLV, ISC or an
unknown actor for that?

I fully agree with Paul Wouters here: the responsability of a TLD
manager is to make sure it works. Not to blindly follow a
procedure. When you deploy a new name server, you test it from many
places, you don't just publish the route with BGP, you test with many
unknown actors that it does work.

> Has ISC told .pr "Be informed we're slurping your keys from the ITAR
> and feeding them to our DLV engine. This happens once a
> day/week/whatever. Bear this in mind if you ever roll a key"? I
> doubt it.

Currently, almost no one (outside of this mailing list) uses
DNSSEC. Yet, you say that there are people in this very small
community that are not aware of dlv.isc.org and not aware that it
mirrors ITAR? Come on.

>> Reality is, there is only on DLV they need to worry about.

There is today only one big DLV registry. If you know others, please
inform the community, it is an important thing to know.

---

From: Jim Reid <[email-masked]>
Date: Fri, 11 Sep 2009 12:20:19 +0100

> If dlv.isc.org is an unknown actor to a DNSSEC-signed TLD, they indeed

Stephane, you are picking nits instead of addressing the points I made.
These focus on the fact that there are informal undocumented  
arrangements around the operation of a DLV registry, how and where  
that registry gets its keying material, how that registry publishes  
the info, what its operating practices and service commitments are,  
how others use that DLV registry for validation, what problem  
reporting and escalation procedures are available, etc, etc. These are  
the unknown actors I was referring to.

It appears there was no procedure for .pr to give ISC a heads-up about  
the key rollover. Or for ISC to nudge .pr about the problem caused by  
the changed key. That worries me. It should worry you too.

At one level this is no big deal in the context of minimal DNSSEC  
deployment and only some fraction of that being dependent on  
dlv.isc.org. However we are talking about something that affects core  
Internet infrastructure, namely a TLD. IMO, more care is needed. If  
the Internet people don't do that, there is a strong possibility their  
toys will one day get taken away because governments will decide they  
have intervene to impose order on something that appears to them to be  
(a) important; (b) unsatisfactorily supervised.

> Can you blame DLV, ISC or an unknown actor for that?

I am not blaming anyone or seeking to apportion blame. I'm simply  
pointing out that it's not wise for the operation of things like DLV  
to be based on unstated assumptions and goodwill best efforts (however  
noble these things are) instead of documented processes/procedures.

> community that are not aware of dlv.isc.org and not aware that it

No I'm not saying that. I am saying that it is not wise to work from  
assumption (correct or otherwise) that dlv.isc.org is the only DLV in  
town.
If someone claims there is no other DLV, then the onus on them is to  
prove that.

---

From: Paul Wouters <[email-masked]>
Date: Fri, 11 Sep 2009 09:35:32 -0400 (EDT)

> Stephane, you are picking nits instead of addressing the points I made.

Uhm no. The above statement is a good summary.

> At one level this is no big deal in the context of minimal DNSSEC deployment 
> and only some fraction of that being dependent on dlv.isc.org.

If you install a nameserver on Fedora 11 or never, it comes with TLD keys preloaded
and DLV enabled. What are you calling "minimal"?

> However we are 
> talking about something that affects core Internet infrastructure, namely a 
> TLD. IMO, more care is needed. If the Internet people don't do that, there is 
> a strong possibility their toys will one day get taken away because 
> governments will decide they have intervene to impose order on something that 
> appears to them to be (a) important; (b) unsatisfactorily supervised.

Are you suggesting that if a TLD makes a big mistake, we the internet people are
to blame for any government consequences?

> I am not blaming anyone or seeking to apportion blame. I'm simply pointing 
> out that it's not wise for the operation of things like DLV to be based on 
> unstated assumptions and goodwill best efforts (however noble these things 
> are) instead of documented processes/procedures.

TLD's should have an internal documentation process on key rollovers. If they
want to share their documents with the community, even better. But the
community is in no position to dictate procedure nor being the party to assign
blame to when things go wrong.

> assumption (correct or otherwise) that dlv.isc.org is the only DLV in town.

If there is another DLV Registry in use, those people should better speak
up in the community to make others aware. It's not our job to hunt down
DLV Registries. 
Since one can only configure 1 DLV in the two popular nameservers
supporting DLV, it seems unlikely another public DLV is deployed. If someone is
deploying a non-public DLV, they beter be aware that they cannot be notified,
and therefor need a good working active key verification procedure for their DLV.

> If someone claims there is no other DLV, then the onus on them is to prove 
> that.

Ever heard about Betrand Russels tea pot around Mars argument?

Paul

---

From: Roy Arends <[email-masked]>
Date: Fri, 11 Sep 2009 16:48:16 +0200

> synch the change submitted by PR to the ITAR.

Indeed.

DLV registries that choose to sync with ITARs should do that  
regularly. Synching once a week is ridiculously low.

Why should the operators of PR care about this DLV? What if some other  
DLV decides to synch with itar once a month? How does PR or some other  
registry with keys in ITAR know which DLV synch's how often with what  
TAR?

The problem is DLV. Not ITAR. Not PR.

Roy

---

From: Roy Arends <[email-masked]>
Date: Fri, 11 Sep 2009 17:07:36 +0200

On Sep 10, 2009, at 11:09 PM, Paul Wouters wrote:

>> Again, I am not objecting to people using DLV. I think it is ucky,  
>> but that's just me. What I am objecting to is the suggestion made  
>> here that _before a TLD that has submitted its keys to the ITAR  
>> rolls its keys, it must notify the (potentially multiple?) folks  
>> who run a DLV registry, of which the TLD may have no knowledge, who  
>> have harvested ITAR data and wait_.  That's just crazy talk.
>

Paul. their zone file works. has worked. DLV was the problem. PR has  
nothing to do with it. Stop blaming PR.

> Reality is, there is only on DLV they need to worry about.

I'd recommend that domain holders who do NOT want their dnskey (or  
hashed derivative) end up in some DLV, copyright their public keys. I  
also recommend that, when submitting TLD DNSKEYS to IANA, IANA allows  
option that the keys will NOT be published in their ITAR and solely be  
distributed via the root zone (in that 6 month period when both exist).

Roy

---

From: joao damas <[email-masked]>
Date: Fri, 11 Sep 2009 17:40:04 +0200

On 11 Sep 2009, at 17:07, Roy Arends wrote:

>>> who run a DLV registry, of which the TLD may have no knowledge,  
>>> who have harvested ITAR data and wait_.  That's just crazy talk.
>>

Not DLV, but ISC's DLV management, to be specific.
As for .pr, they ought to update their keys with the IANA as soon as  
they put them in the zone, perhaps even earlier if the ITAR allows for  
deferred publication (whatever the IANA ends up doing I hope deferred  
publication is in the system requirements, independently of how the  
key is made visible to the world)

> hashed derivative) end up in some DLV, copyright their public keys.  
> I also recommend that, when submitting TLD DNSKEYS to IANA, IANA  
> allows option that the keys will NOT be published in their ITAR and  
> solely be distributed via the root zone (in that 6 month period when  
> both exist).

Well, I hope not. In fact I hope the ITAR never goes away and I have a  
means of cross checking the info the IANA has and publishes directly  
against what ends up in the root zone. I also hope they are the same  
all the time, but it is just nice to be able to check what things look  
like when they enter the pipeline and when they come out.

Joao

---

From: Roy Arends <[email-masked]>
Date: Fri, 11 Sep 2009 17:54:19 +0200

>> solely be distributed via the root zone (in that 6 month period  
>> when both exist).
>
> Well, I hope not. In fact I hope the ITAR never goes away and I have  
> a means of cross checking the info the IANA has and publishes  
> directly against what ends up in the root zone.
> I also hope they are the same all the time, but it is just nice to  
> be able to check what things look like when they enter the pipeline  
> and when they come out.

There can be private channels for debugging, no? Would you want that  
viewable publicly by any DLV cowboys?

Roy

---

From: Jim Reid <[email-masked]>
Date: Fri, 11 Sep 2009 18:20:20 +0100

On 11 Sep 2009, at 14:35, Paul Wouters wrote:

> If you install a nameserver on Fedora 11 or never, it comes with TLD  
> keys preloaded and DLV enabled.

Yawn.

> What are you calling "minimal"?

The dictionary definition will do. Or, failing that, a percentage of  
the people using a specific release/variant of a Linux distro. Which  
is a percentage of the installed base of that distro. Which is a  
percentage of the overall Linux distros. Which is a small-ish  
percentage of the devices connected to the Internet. Seems fairly  
minimal to me. Your mileage may of course vary.

> Are you suggesting that if a TLD makes a big mistake, we the  
> internet people are to blame for any government consequences?

No. I'm saying that if we, the Internet experts, make a half-assed job  
of putting processes and procedures in place around important Internet  
infrastructure, someone one day is going to come along take the toys  
away and provide adult supervision. Making the odd mistake is not the  
problem. We're all human after all. Not having transparent mechanisms  
for resolving those problems or minimising their impact is an issue.

Suppose a government cared about DNSSEC, do you think they'be be happy  
for their ccTLD key(s) to go into a DLV registry without some kind of  
agreement or understanding between the ccTLD, the DLV registry and  
perhaps even the government itself? Or let's re-ask that question in a  
different context. If you were responsible for a TLD would you be  
happy for its keys to end up passing from the ITAR to some DLV  
repository where they could be used for who knows what without you  
having any influence over that? [Like only checking the ITAR for new  
keys once in a blue moon.] Hint: if others used an explicit TA for  
your TLD, DNS resolution/validation for your TLD won't depend on the  
DLV registry or its DNS infrastructure. And if there's a problem  
there's a binary answer: either you screwed up the signing or the  
other guy got the TA wrong. If DLV was involved, the path would be  
longer and murkier.

> But the community is in no position to dictate procedure nor being  
> the party to assign blame to when things go wrong.

You haven't understood what I said. So let me repeat it Paul. I am not  
blaming anyone or looking to apportion blame. I am not trying to  
dictate procedure either: just that there should be procedure(s). I am  
saying that if TLD keys go into the ITAR and from there into some DLV  
registry (or whatever), there need to be agreements documenting how  
this works, the respective roles and responsibilities and how the  
parties to that agreement interact with each other, particularly for  
things like rollovers or emergency problem resolution. The actual  
mechanics of that are not procedures or processes dictated from above:  
they're worked out between the parties involved.

Hey it's not like we're in the 1990's when few TLDs had agreements  
with whoever provided slave service for their zone, is it?

>> If someone claims there is no other DLV, then the onus on them is  
>> to prove that.
>

Of course. But IMO its you who's making an assertion about the  
existence of that teapot (a single DLV registry) orbiting Mars. Let's  
not waste time debating whether there's more than one teapot up there.  
It's a distraction from the real issues.

---

From: joao damas <[email-masked]>
Date: Fri, 11 Sep 2009 19:58:18 +0200

On 11 Sep 2009, at 17:54, Roy Arends wrote:

>>> hashed derivative) end up in some DLV, copyright their public  
>>> keys. I also recommend that, when submitting TLD DNSKEYS to IANA,  
>>> IANA allows option that the keys will NOT be published in their  
>>> ITAR and solely be distributed via the root zone (in that 6 month  
>>> period when both exist).
>>
>> Well, I hope not. In fact I hope the ITAR never goes away and I  
>> have a means of cross checking the info the IANA has and publishes  

why not? I don't really care what people do in their spare time.
Private channels for debugging is a "silly" idea. Private to whom? are  
some of us more equal than others?

Joao

---

From: Roy Arends <[email-masked]>
Date: Fri, 11 Sep 2009 20:27:03 +0200

On Sep 11, 2009, at 7:58 PM, joao damas wrote:

>>>> I'd recommend that domain holders who do NOT want their dnskey  
>>>> (or hashed derivative) end up in some DLV, copyright their public  
>>> be able to check what things look like when they enter the  
>>> pipeline and when they come out.
>>
>> There can be private channels for debugging, no? Would you want  
>> that viewable publicly by any DLV cowboys?
>
> Private channels for debugging is a "silly" idea. Private to whom?
> are some of us more equal than others?

1) for .INFO, you want only Afilias to be able to add or change the DS/ 
DNSKEY in the ITAR. Yes?
2) Subsequently, before it is published in the root, you'd like to see  
if it is correct. Yes?
3) after publication, just use dig (tm) to see what things look like  
when they come out.

At least for Nominet, I want (2) to do cross-checking, be able to  
check what things look like before they enter the pipeline, preferably  
using the same channel as (1). Before I push the 'publish' button, I  
want to check it in private. After I push the publish button, its in  
the root, and world can check it with me, using dig.

If some folks, just as equal as I am, with a lot more spare time, have  
that urge to publish my key, just because they can, but synch only  
once a week with the ITAR, and then point the finger at me for not  
telling them I changed my key 4 days ago, I'd tell them to Just Go Away.

roy

---

From: joao damas <[email-masked]>
Date: Fri, 11 Sep 2009 21:03:01 +0200

On 11 Sep 2009, at 20:27, Roy Arends wrote:
>
> 1) for .INFO, you want only Afilias to be able to add or change the  
> DS/DNSKEY in the ITAR. Yes?

I want afilias, and only afilias, to tell the IANA and IANA to publish  
it, yes.

> 2) Subsequently, before it is published in the root, you'd like to  
> see if it is correct. Yes?

not necessarily before (not me as a user, the publisher might want to  
see it before). As a consumer of the information, at the same is good  
enough.

> 3) after publication, just use dig (tm) to see what things look like  

you mean after the validator has failed and the phone is red hot?

> check what things look like before they enter the pipeline,  
> preferably using the same channel as (1). Before I push the  
> 'publish' button, I want to check it in private. After I push the  
> publish button, its in the root, and world can check it with me,  
> using dig.

I want an out of band channel, just like I can download the root zone  
using ftp today (only that I would prefer to do it from IANA rather  
than the rs.internic.net machine)

>
> If some folks, just as equal as I am, with a lot more spare time,  
> have that urge to publish my key, just because they can, but synch  
> only once a week with the ITAR, and then point the finger at me for  
> not telling them I changed my key 4 days ago, I'd tell them to Just  
> Go Away.
>

and you would be right do so, IMHO. the fact that dlv.isc.org didn't  
pick up the .pr key is an issue that rest entirely with ISC (then  
again, we are all learning operational practices in this DNSSEC  
business at this time)

Joao

---

From: Joe Abley <[email-masked]>
Date: Fri, 11 Sep 2009 18:22:24 -0400

> again, we are all learning operational practices in this DNSSEC  

(to pick a message more or less at random to respond to)

Is there an implication from the presence of this thread on this list  
that:

(a) general, non-wg, operational matters are on-topic for this list, or

(b) this is a support channel for the ITAR, or ISC's DLV, or the .PR  
registry?

I can see that some feedback about novel operational problems relating  
to DNSSEC are useful to raise since they might stimulate a desire to  
work on documents. But the current state of the thread has me  
scratching my head a bit. Perhaps I just need to get a life.

My $0.02 Canadian, on a rainy Friday afternoon.

Joe

---

From: Mark Andrews <[email-masked]>
Date: Sat, 12 Sep 2009 09:01:22 +1000

I see no need for any TLD to check that ISC has picked up records
from the ITAR for the DLV registry.  What I do see is a need to
formalise the timing parameters involved an ensure that all parties
involved are aware of them.  That way if a party is too slow or if
a party is too fast we know who to blame.  At the moment there is
wishy washy language which doesn't help anyone do the right thing

IANA still has not provided timing guidance.

IANA can you please specifiy a maximum polling interval on this
page and inform the TLD's using ITAR of what it is.  A minimum
polling interval would also be useful but is not crucial.

As a reminder, the trust anchor repository is temporary. It will
be decommissioned once the root zone is signed. Any software
configured to use it should be configured to continue to work when
this event happens. You should also ensure you have procedures in
place to keep the trust anchors up to date as the ITAR will regularly
add and remove keys.

---

From: Kim Davies <[email-masked]>
Date: Fri, 11 Sep 2009 16:21:47 -0700

Hi Mark,

> polling interval would also be useful but is not crucial.

I can't provide good guidance, because the ITAR is "live". If we get a valid
add request then it goes in, if we have a valid remove request it comes out.
We could get three different changes in the span of an hour, or we could get
no requests in the span of a whole month. Revisions to the registry are not
batched into scheduled intervals, like, say, the root zone. Maybe they
should be?

Right now, since the initial activity populating the repository, there are
only a couple of ITAR change events per month, but we have no pattern as to
when they can occur.

kim

---

From: Kim Davies <[email-masked]>
Date: Fri, 11 Sep 2009 16:30:23 -0700

> only a couple of ITAR change events per month, but we have no pattern as to

Actually, I can be more specific:

2009-01-20 sign_zones generating new registry view (serial 2 -> 3)
2009-01-22 sign_zones generating new registry view (serial 3 -> 4)
2009-01-29 sign_zones generating new registry view (serial 4 -> 5)
2009-02-01 sign_zones generating new registry view (serial 5 -> 6)
2009-02-12 sign_zones generating new registry view (serial 6 -> 7)
2009-02-17 sign_zones generating new registry view (serial 7 -> 8)
2009-02-21 sign_zones generating new registry view (serial 8 -> 9)
2009-02-23 sign_zones generating new registry view (serial 9 -> 10)
2009-03-12 sign_zones generating new registry view (serial 10 -> 11)
2009-04-01 sign_zones generating new registry view (serial 11 -> 12)
2009-04-06 sign_zones generating new registry view (serial 12 -> 13)
2009-04-06 sign_zones generating new registry view (serial 13 -> 14)
2009-04-14 sign_zones generating new registry view (serial 14 -> 15)
2009-04-17 sign_zones generating new registry view (serial 15 -> 16)
2009-06-24 sign_zones generating new registry view (serial 16 -> 17)
2009-07-15 sign_zones generating new registry view (serial 17 -> 18)
2009-08-22 sign_zones generating new registry view (serial 18 -> 19)
2009-09-01 sign_zones generating new registry view (serial 19 -> 20)
2009-09-02 sign_zones generating new registry view (serial 20 -> 21)
2009-09-03 sign_zones generating new registry view (serial 21 -> 22)

Kim

---

From: Mark Andrews <[email-masked]>
Date: Sat, 12 Sep 2009 09:47:02 +1000

In message <[email-masked]>, Kim Davies writes:
> I can't provide good guidance, because the ITAR is "live". If we get a vali> d
> add request then it goes in, if we have a valid remove request it comes out> .
> We could get three different changes in the span of an hour, or we could ge> t

When they happen is not the issue.  It how often the people
using the ITAR should be checking for changes that is the
issue.

With any key rollover there are timing constraints.  For a ZSK you
need to publish the ZSK at least one DNSKEY TTL before you depend
on it (sign only with it).  You also need to wait for all the
authoritative severs to be serving the zone with that key in it
before you start the timer.

There are similar constrains on how to roll over a KSK.  The old
DS RRset needs to have cleared the caches before you depend on the
new DS RRset.  Similarly the old DNSKEY RRset needs to have cleared
caches before you can depend on the new DNSKEY RRset.

With the data in the DNS it is easy as you can see the TTL on the
records.  You will most probably publish DS's with 1 or 2 day ttls
and everyone knows when the old data should have cleared caches.

Publish new DNSKEY, publish new DS, wait at least the max TTL of
the old DS/DNSSKEY TTLs.  Remove old DS, remove old DNSKEY.

The same thing should be happening with ITAR.  Publish new DNSKEY,
publish new DS, wait the prescribed period for the trust achors to
be updated, remove old DS, remove old DNSKEY.

At the moment no one knows how long to wait as you havn't told
anyone what that period should be. 

Mark

---

From: Mark Andrews <[email-masked]>
Date: Sat, 12 Sep 2009 10:58:39 +1000

In message <[email-masked]>, Mark Andrews writes:
> > >=20
> > >=20
> > I can't provide good guidance, because the ITAR is "live". If we get a vali=
> > d
> > add request then it goes in, if we have a valid remove request it comes out=
> > .
> > We could get three different changes in the span of an hour, or we could ge=
> > t

For another example RFC 5011 is Publish new DNSKEY, wait the
prescribed period (1 month, which really should have been specfied
in days as we now have to track leap years etc.), remove the old
DNSKEY.

---

From: Mark Andrews <[email-masked]>
Date: Sat, 12 Sep 2009 12:17:31 +1000

> The problem is DLV. Not ITAR. Not PR.

Actually there is blame all round.

PR for not performing a key rollover correctly.  They still havn't
updated their own web site <http://dnssec.nic.pr/serverconf.php> so
there will be people who haven't been informed yet.

	trusted-keys {
	"pr." 256 3 5
	"BQEAAAABvS8Q64q8v62DW3y4EtUmsHr0dpU9Mizo63NXFMlEA4UaO88s B5il79Mb
	J0dzmRZ7M+j/E5pVSTTazJsK6LMnncBF3bwMWo4/nVVB0d9E 6CsClsJFU+A0a8kWI
	Z+aXuqUHO7QZ88qG7cwLbTNwHeo1X+ArvXgXmU6 OaemL3v5+eU=";

	"pr." 257 3 5 
	"AwEAAc6SkFSHw00wJFUWd1Td/efsxhfX+UTrxrzqQXNuZ8Qj2PiP6p/m BxysJt06
	XgSCB41CPhkgvgqrtdaJ/hXKG81xNXUcGfqvV9wYMJnN+oBB /lLaQU/39fWaNc4fB
	GiRI2dNDVKPry2YX6y04YrEGRM+wf6HWHVdW1Js xuMuDOSr";
	};

ITAR for not publishing a polling interval.   I would recommend
that it be in line with the TTL that they are going to use on the
DS records.  I would expect this to be 1-2 days.

ISC for having a highish polling interval.  I've recommending that
ISC drops the poll interval down to 1 hour which is inline with the
TTL of the DLV record.

ISC needs to make sure its polling interval of the ITAR + DLV TTL
is less than the maximim ITAR polling interval so that no extra
delay is added.

Mark

---

From: Edward Lewis <[email-masked]>
Date: Fri, 11 Sep 2009 23:45:03 -0400

Ok, I've about had it with the tone of this thread.

>Actually there is blame all round.

I think this is uncalled for and is detrimental to the list.

---

From: Mark Andrews <[email-masked]>
Date: Sat, 12 Sep 2009 14:55:25 +1000

> I think this is uncalled for and is detrimental to the list.

Did all the other users of ITAR successfully follow this key rollover?
Did all the other users of ITAR fail to follow this key rollover?
Would they admit it if they did?

I know I took several days to update my trusted-key clause for PR
in my named.conf's.  They are all updated manually.  If I wasn't
asking PR directly, by doing DNSKEY queries, but instead used ITAR's
collection I would have not followed the key rollover.

PR should have updated ITAR immediately.  They didn't.  PR should
have taken their delay into account before removing the old key.
They didn't.  2 days was not reasonable when every other key rollover
took ~1 month.

ITAR should be providing guidance on how often to poll.  They didn't.
No TAR can be reliable without this guidance because nobody can
know what to expect.

PR rolled keys faster than any other TLD has ever rolled keys in
the past once they went operational (GOV rolled sub 24 while testing).
PR rolled keys much faster than the recommended timings in RFC 5011.
Weekly polling was quite reasonable based on RFC 5011 and historic
TLD key rollover periods.

Mark

---

From: Kim Davies <[email-masked]>
Date: Mon, 14 Sep 2009 09:01:12 -0700

Hi Mark,

> anyone what that period should be.

Are you suggesting ITAR needs to add TTLs, or ITAR should be somehow
technically enforce sufficiently long overlap periods, or should just
provide rules that TLD operators are expected to abide by?

The assumption right now is it is for the TLD operators to act responsibly
and make changes as appropriate. ITAR is just an oblivious republisher of
data that they have submitted, and has subsequently verified is genuine. It
seems to me the problems you describe are ones of encouraging best practice
amongst TLD operators, rather than a specific defect in ITAR.

Kim

---

From: joao damas <[email-masked]>
Date: Mon, 14 Sep 2009 18:53:12 +0200

Kim,
could the ITAR have a serial number that could be checked without  
having to download and parse the whole file, to enable quick checks  
from consumers of the ITAR information that would not overwhelm either  
end of the communication?
Joao
PS: or even, could it be published as a DNS zone of some name, with a  
TLL and an SOA?

---

From: Kim Davies <[email-masked]>
Date: Mon, 14 Sep 2009 10:12:49 -0700

Hi Joao,

> end of the communication?

There is a serial number which can be checked, and I guess we could publish
it independently, however you can use the inherent HTTP mechanism
(If-Modified-Since header) to check for updates and not get the actual file
if it hasn't updated. e.g. The '-N' flag with wget.

> PS: or even, could it be published as a DNS zone of some name, with a

Perhaps, although I am hoping a signed root zone is not too far off...

kim

---

From: David Blacka <[email-masked]>
Date: Mon, 14 Sep 2009 14:45:38 -0400

> provide rules that TLD operators are expected to abide by?

I would say that the latter would be sufficient, but it would be rules  
that the TLD operators *and* ITAR consumers would have to abide by.

I think it works to simply say this:

   * The ITAR should be checked for changes once per 24 hour period.

Then:

   * consumers (e.g., dlv.isc.org, me) will know to check at least  
once per day;
   * TLD operators would know to pre-publish the new DS at least 24  
hours before doing the KSK roll.

As it is, I don't expect any TLD operator to have any idea of how long  
they must pre-publish, nor any consumer to know how often to poll IANA  
for any changes.

> amongst TLD operators, rather than a specific defect in ITAR.

But without any timing advice, it is quite difficult for TLD operators  
to know if they are acting responsibly or not.

--
David Blacka                          <[email-masked]>
Sr. Engineer          VeriSign Platform Product Development

---

From: Thierry Moreau <[email-masked]>
Date: Mon, 14 Sep 2009 15:01:23 -0400

> to know if they are acting responsibly or not.
Is it the role of IANA to provide such advice?

What if a zone manager (e.g. a TLD registry) wishes to use very short 
signature keys with very short rollover periods and finds its parent 
zone manager responsive in terms of DS change turnover times? Maybe I 
missed something and a specifications document somewhere prevents short 
RSA signature keys, I which case I apologize in advance for the noise on 
this mailing list.

Regards,

- Thierry Moreau

---

From: Edward Lewis <[email-masked]>
Date: Mon, 14 Sep 2009 15:07:41 -0400

At 14:45 -0400 9/14/09, David Blacka wrote:

>   * TLD operators would know to pre-publish the new DS at least 24 hours
>     before doing the KSK roll.

Plus an allowance for TTLs.

It's a bit more complicated I think, unless I missed a fine point in 
the message.

To nit pick: "TLD operators would know to pre-publish the new DS" is 
incorrect per se - the DS set for a zone is authoritative at the 
parent.

When I want to change a SEP, first I plan to have it in my DNSKEY set 
for a long enough period to satisfy at least RFC 5011 and the time it 
takes for all caches to time out all DNSKEY set versions prior to the 
new SEP-to-be.

Next I add the new SEP signature to the DNSKEY set and ask the parent 
to change the DS record in my DS RRSet.  Change means take out the 
old and put in the new (per algorithm/etc.).  I don't remove anything 
at this point.

I don't think the parent's TTL considerations matter because I am 
placing two SEP's in to play while waiting.  As soon as I see the new 
DS record appear in the zone (and the old DS gone), I start a timer 
based on my TTL for the DNSKEY set.  Once that timer expires I 
RFC5011-revoke the old SEP (and sign the whole key set with old and 
new SEP key).

According to RFC 5011 rules I then remove the old SEP and any 
signature at the appropriate time.

>As it is, I don't expect any TLD operator to have any idea of how 
>long they must pre-publish, nor any consumer to know how often to 
>poll IANA for any changes.

Polling is suboptimal when an event is anticipated but it is a good 
catch all to avoid missing an event notification.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 15 Sep 2009 11:16:51 +1000

In message <[email-masked]>, Technical Support writes:
> Hi Everyone,
> 
> We're sorry for all the inconvenience this issue has caused. Regarding 
> the rollover procedure, we did follow it by using a double signature in 
> the rollover phase. The new keys are posted at 
> http://dnssec.nic.pr/trust_keys.php instead of 
> "http://dnssec.nic.pr/serverconf.php" as listed in one of the posts. 
> Also, an email was sent to our mailing list. Even though we didn't 
> upload the new keys in a timely manner and the maintenance key rollover 
> script removed the old keys from the zones, our rollover didn't last 2 
> days.

It effectively lasted 2 day for ITAR users.  The new key was added
on the 1st and the old key removed on the 3rd.  The total rollover
was 16 days.

> We are looking forward to avoid such issues in the future. Once 
> again sorry for this misunderstanding and any feedback is welcome.
> 
> 
> Cheers,
> 
> 
> _______________________
> Technical Staff
> .prTLD
> 787.372.3804 - 787.689.5868
> 
> 
> 
> 
> Mark Andrews wrote:

---

From: Mark Andrews <[email-masked]>
Date: Tue, 15 Sep 2009 11:57:54 +1000

> provide rules that TLD operators are expected to abide by?

Provide rules that both the TLD's and consumers are expected to
abide by.  This should prevent disconnects like what happened here
from occuring.

TLD's will need to wait between submitting the new key to IANA and
removing the old key or validation will break.  This is irrespective
of whether IANA is publishing the ITAR or only updating the root
zone.  If you wish to maintain consistancy of operations as you
transition from ITAR to root zone updates only I would recommed you
make this delay match the TTL you expect to use for the DS records
in the root zone then tell the consumers of the ITAR that they need
to ensure that update their trust anchors within this period.

> and make changes as appropriate.

The problem is that no one has defined what is appropriate.  We are
all working in a information vacumn.  The 1 week poll was choosen
assuming TLD operators would follow RFC 5011 timings as they don't
yet have a signed parent zone and would inform IANA in a timely
manner.  Neither of these assumptions turned out to be true.

> ITAR is just an oblivious republisher of

ITAR is missing the information that would be available when the
root is signed which is timing.  ITAR is special as it is in a
position to provide that as IANA is producer of both ITAR and the
root zone.

There are generic issues for the operations of TAR's here which need
to be explored but they are not applicable to ITAR.

Mark

---

From: Mark Andrews <[email-masked]>
Date: Tue, 15 Sep 2009 12:36:18 +1000

> takes for all caches to time out all DNSKEY set versions prior to the 

Iff the zone being managed according to RFC5011.  RFC5011 is not
required for zones that that have a secure parent.

RFC 5011 is a real pain and unless you have a reason to use it I
would not recommend saying you are following RFC 5011.

With a secure parent there are several different strategies that
can be used to roll a DNSKEY cleanly.

Add the new DNSKEY.  Add the new DS to the parent zone.  Wait for
both the old DS and DNSKEY RRsets to expire from caches.  Remove
the old DS and the old DNSKEY.

Add new DNSKEY, wait DNSKEY TTL, replace old DS with new DS, wait
DS ttl, remove old DNSKEY.

Generate DNSKEY, publish new DS, wait DS ttl, replace DNSKEY, wait
DNSKEY ttl, remove old DS.  ** This is not compatible with how ITAR
is defined to work.

Note all of these have a wait "DS ttl" after the new DS is published.
Consumers of ITAR need to update in this window.

> old and put in the new (per algorithm/etc.).  I don't remove anything 

You will break validators which use different caches for DNSKEY and
DS lookups doing this.
 
> RFC5011-revoke the old SEP (and sign the whole key set with old and 

DS ttl always come into play.  You need to assume a validator can
be using different caches to retieve the DS and DNSKEY RRsets.

---

From: Holger Zuleger <[email-masked]>
Date: Tue, 15 Sep 2009 11:38:28 +0200

> DS ttl, remove old DNSKEY.
To nit pick again: wait *old* DS ttl.
The problem here is that you don't know the exact point in time, when
the new DS is published by the parent. The timer starts when *all* the
authoritative servers of the parent zone publish the new DS record.

For sure, you can dig for the new DS, but in the world of anycast name
servers you can't be sure that the DS is distributed to all of them.

-- 
Holger Zuleger / Zur Röderburg 6 / D-35315 Homberg/Ohm-Höingen /
xmpp:[email-masked] / http://www.hznet.de / tel:+49 6633 642022

---

From: Mark Andrews <[email-masked]>
Date: Tue, 15 Sep 2009 21:38:55 +1000

> servers you can't be sure that the DS is distributed to all of them.

If you are going to be that picky then it is old expire + old DS
ttl after the serial has changed to allow all stealth slaves to
have caught up or to have stop serving the zone hoping there isn't
a loop in the zone transfer graph or they are all transfering from
the primary master because no-one wanted the EDNS EXPIRE option
that would have addressed that issue.