Subject: [DNSOP] Updated KSK Sentinel document

From: Warren Kumari <[email-masked]>
Date: Mon, 12 Feb 2018 15:28:50 -0500

<author hat only>

Hi all,

Sorry it has taken so long to get a new version of this document
posted - you deserve better.

Anyway, we've finally posted an updated version -
https://datatracker.ietf.org/doc/draft-ietf-dnsop-kskroll-sentinel/

This version includes a (hopefully easily understood) description of
how this would actually be used, and not just "here's a protocol, k,
thnx, bye!". I've tried to layout what each party does, and how it all
fits together - please let me know if it isn't clear. This section is
towards the top of the document - we will likely make it an Appendix
before publication.

I've also updated it to use the kskroll-sentinel-is-ta-<id> format. It
is easy to change again in the future, but this seemed to be what the
working group liked. I also updated my demo implementation
(http://www.ksk-test.net) to use this naming scheme.

This version also clarifies that the test is "Is the Key ID a DNSSEC
root KSK?" Originally my view was that it should be "Is there *any*
key in the trust store with this keyID?", but after running some
numbers I decided that there is a significant chance of false
positives.

As I mentioned, it took an embarrassingly long time to post the update
- please let us know if we missed your comments.

W
-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Bob Harold <[email-masked]>
Date: Tue, 13 Feb 2018 12:10:56 -0500

> - please let us know if we missed your comments.
Thanks, the explanation helped.  I finally realized that it only works if
all (or most) validating resolvers are updated to support this new feature,
otherwise we have a bunch of responses that are uncertain.

Thinking out loud...
If an entry could be put in the root zone, that is signed only with the new
key, then could users query that and always get a yes/no answer to whether
they will be affected?
I realize it might be difficult politically or process-wise.
Not sure if it could be just a pair of A and AAAA records, or a delegation
to a temporary TLD.
And is it even possible, since it probably needs a new ZSK based on the new
KSK?

-- 
Bob Harold

---

From: Vladimír Čunát <[email-masked]>
Date: Tue, 13 Feb 2018 18:45:35 +0100

On 02/13/2018 06:10 PM, Bob Harold wrote:
> [...] If an entry could be put in the root zone, that is signed only
> with the new key, then could users query that and always get a yes/no
> answer to whether they will be affected? 

I don't think that's possible.  This is about the _single_ root DNSKEY
RRset - switching which key signs the set (tags 19036 and 20326). 
Resolvers will either successfully validate this RRset or not, and
consequently they either can validate all other signatures in the root
zone or they can't trust anything in the whole DNS tree.

--Vladimir

---

From: Joe Abley <[email-masked]>
Date: Tue, 13 Feb 2018 13:18:29 -0500

On 13 Feb 2018, at 12:10, Bob Harold <[email-masked]> wrote:

> Thanks, the explanation helped.  I finally realized that it only works if all (or most) validating resolvers are updated to support this new feature, otherwise we have a bunch of responses that are uncertain.

Based on numbers of queries arriving at ORG servers, the top dozen or so ASes account for over half of all resolver traffic. Most of those ASes (there are a couple of exceptions) originate almost all of this DNS traffic from a small number of source addresses.

So "most", in practice, at least by those metrics, might not be a very large number. There is a very long tail.

Joe

---

From: "Wessels, Duane" <[email-masked]>
Date: Tue, 13 Feb 2018 18:49:34 +0000

> On Feb 13, 2018, at 9:10 AM, Bob Harold <[email-masked]> wrote:
> 
> If an entry could be put in the root zone, that is signed only with the new key, then could users query that and always get a yes/no answer to whether they will be affected?  

This doesn't work because when the new key is published in the zone (and signed by the old key, as it must be), then the new key becomes trusted by the validator.  Thus, there is still a valid chain-of-trust to those records for those with only the old TA.  

DW

---

From: Warren Kumari <[email-masked]>
Date: Tue, 13 Feb 2018 14:02:12 -0500

On Tue, Feb 13, 2018 at 1:49 PM, Wessels, Duane <[email-masked]> wrote:
> This doesn't work because when the new key is published in the zone (and signed by the old key, as it must be),

Yup - this is the critical bit -- a number of us keep going down the
"Oooh! This is easy, we just publish Im-only-signed-with-2222. in the
root, and then people who cannot resolve that know that they don't
have 2222". And then killjoys like yourself point out that DNSSEC
doesn't actually work like that.... :-)

W

>then the new key becomes trusted by the validator.  Thus, there is still a valid chain-of-trust to those records for those with only the old TA.
>
> DW
>
> _______________________________________________
> DNSOP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsop

---

From: 神明達哉 <[email-masked]>
Date: Wed, 14 Feb 2018 14:05:09 -0800

At Mon, 12 Feb 2018 15:28:50 -0500,

> https://datatracker.ietf.org/doc/draft-ietf-dnsop-kskroll-sentinel/

I've read draft-ietf-dnsop-kskroll-sentinel-01 (this is my first
careful read of this draft) and found it generally well-written.

I have some comments on 01.  These are basically editorial.

- General
  I think it's worth pointing out that SERVFAIL can be returned for
  various other reasons and the detection mechanism relying on it
  isn't reliable.  This is probably okay given the purpose of this
  detection, but I think it's better to note it explicitly.

- Section 1.1
   Address Record: Throughout this document we use the term Address
   Record (AR) to mean an A or AAAA record.  [...]

  I actually didn't find this term other than here and Section 9
  (change log).

- Section 1.1 and throughout the draft

   AAAA records and the IPv6 documentation prefix (2001:DB8::/32) as

  I'd suggest using lower-case letters to show IPv6 addresses and
  prefixes, following the recommendation of RFC5952.  It's not a big
  deal but it would be better if we can be more consistent on the
  choice of letter case in RFCs.

- Section 2

   Charlie's resolvers are validating, but they have not been upgraded
   [...]
   resolve it).  He is able to fetch both of the other resources - from
   this he knows (see the logic below) that he is using legacy, non-
   validating resolvers.  [...]

  Do you mean "he is using legacy validating resolvers"?  If it's not
  a typo, the behind logic isn't obvious to me and it would help if
  you explain it in more detail (instead of just referring to
  'below').

- Section 3

   [...] Note that
   the test is "Is there a key with this KeyID in the resolver's current
   trust store for the DNS root", not "Is there any key with this KeyID
   in the trust store", nor "Was a key with this KeyID used to validate
   this query?".

  Unless I miss something, the draft doesn't clarify its use is
  limited to root KSK until the next paragraph of this, so I suspect
  this statement will confuse a fresh reader who reads this doc from
  top to bottom without a particular knowledge of background
  discussion.  I'd suggest noting it somewhere before this part,
  perhaps in the introduction (and maybe also in abstract).

- Section 3

   [...]  Note
   that the <tag-index> is specified in the DNS label using hexadecimal
   notation.

  I suggest clarifying whether '<tag-index>' contains leading 0s
  somewhere in the document (this may not be the best place to do so,
  but this is the first reference to 'tag-index' that made the
  question occur to me).

- Section 3

   If the resolver has not placed a
   Root Zone Key Signing Key with tag index value matching the value
   specified in the query into the local resolver's store of trusted
   keys, then the resolver should return a response indicating that the
   response contains authenticated data according to section 5.8 of
   [RFC6840].

  Should we perhaps define "store of trusted keys" more precisely?
  For example, if a key is in the "AddPend" status (per RFC5011) for
  the resolver, should it be considered in the "store of trusted
  keys"?

- Section 3

   This mechanism is to be applied only by resolvers that are performing
   DNSSEC validation, and applies only to RRset responses to an A or
   AAAA query (Query Type value 1 or 28) where the resolver has
   authenticated the response RRset according to the DNSSEC validation
   process and where the query name contains either of the labels
   described in this section as its left-most label.

  Is the RRset in 'response RRset' intentionally added (or in other
  words can't it just be 'response')?  Perhaps is the intent to
  exclude negative responses?  In any case I think the intent should
  be clearer here.  And if a mere 'response' suffices it's probably
  less confusing to just say so.

  BTW, you might want to say 'a query for an Address Record' or 'an
  Address Record query' instead of 'an A or AAAA query' (I guess
  that's the intent of the introduction of this term.  See also my
  first comment on Section 1.1 above).

- Section 4

   o  Vleg: A DNSSEC-Validating resolver that does not implement this
      mechanism will respond with an A or AAAA RRSET response for
      "kskroll-sentinel-is-ta", an A record response for "kskroll-
      sentinel-not-ta" and SERVFAIL for the invalid name.

  + s/RRSET/RRset/?
  + 'an A record response' should be 'an A or AAAA record response' or
    it's revised using the "Address Record" term.  Same comment
    applies to other part of this section including the table.

--
JINMEI, Tatuya

---

From: "John Dickinson" <[email-masked]>
Date: Fri, 16 Feb 2018 17:35:44 +0000

Hi,

I like what this draft is trying to do.

I am a bit concerned about adding a invalid RR in to a otherwise 
correctly signed zone. It suspect that there may be a variation in how 
validating resolvers treat authoritative servers that appear to have 
sent bogus data. Some might retry, retry other auth servers, stop using 
that server altogether etc etc…

I suggest that the example A/AAAA RRs on page 4 be written fully 
qualified so there can be no doubt that this draft does not imply new 
special names at the root (which is what I first thought).

In the discussion of Charlie’s resolvers I think “from
    this he knows (see the logic below) that he is using legacy, non-
    validating resolvers.”

should have the “non-“ removed.

regards
John

> - please let us know if we missed your comments.

John Dickinson

http://sinodun.com

Sinodun Internet Technologies Ltd.
Magdalen Centre
Oxford Science Park
Robert Robinson Avenue
Oxford OX4 4GA
U.K.

---

From: Geoff Huston <[email-masked]>
Date: Mon, 19 Feb 2018 07:21:37 +1100

Hi John,

thanks for the review of this draft

> On 17 Feb 2018, at 4:35 am, John Dickinson <[email-masked]> wrote:
> 
> Hi,
> 
> I am a bit concerned about adding a invalid RR in to a otherwise correctly signed zone. It suspect that there may be a variation in how validating resolvers treat authoritative servers that appear to have sent bogus data. Some might retry, retry other auth servers, stop using that server altogether etc etc…

I have been doing this for many years in an ad-based measurement campaign. When a validating resolver is incapable of validating a response it sends back a SERVFAIL response code of course. Some years back “incapable of validating a response” implied an exhaustive search through all NS’s of all parent zones to see if any path exists that can validate the RRSIG - these days many resolvers simply accept the first invalid response and pass back SERVFAIL. Obviously the SERVFAIL response will prompt a stub resolver to pass the query tyo any other recursive resolvers that it is configured to query. This is of course the same behaviour as one would expect from a validating recursive resolver that has failed to track a KSK roll. I have not observed any signal that a client resolver accepts a SERVFAIL response from a recursive resolver as anything other than a failure for the query itself. 

> 
> I suggest that the example A/AAAA RRs on page 4 be written fully qualified so there can be no doubt that this draft does not imply new special names at the root (which is what I first thought).

“example.com” appears four times on page 4 - are you suggesting that this be altered to read “example.com.”?

Or are you suggesting that the 5 instances of the label kskroll-sentinel-(is|not)-ta-2222 which refer to a “resoirce” be edited to read "kskroll-sentinel-(is|not)-ta-2222.example.com"?

Or both?

> should have the “non-“ removed.

yes - that’s correct. That was a typo.

---

From: "John Dickinson" <[email-masked]>
Date: Mon, 19 Feb 2018 10:18:48 +0000

On 18 Feb 2018, at 20:21, Geoff Huston wrote:

>> correctly signed zone. It suspect that there may be a variation in 
>> how validating resolvers treat authoritative servers that appear to 
>> have sent bogus data. Some might retry, retry other auth servers, 
>> stop using that server altogether etc etc…
>
> I have been doing this for many years in an ad-based measurement 
> campaign. When a validating resolver is incapable of validating a 
> response it sends back a SERVFAIL response code of course. Some years 
> back “incapable of validating a response” implied an exhaustive 
> search through all NS’s of all parent zones to see if any path 
> exists that can validate the RRSIG - these days many resolvers simply 
> accept the first invalid response and pass back SERVFAIL.

OK great - I just wondered if there was all sorts of strange differences 
between different implementations.

> Obviously the SERVFAIL response will prompt a stub resolver to pass 
> the query tyo any other recursive resolvers that it is configured to 
> query. This is of course the same behaviour as one would expect from a 
> validating recursive resolver that has failed to track a KSK roll. I 
> have not observed any signal that a client resolver accepts a SERVFAIL 
> response from a recursive resolver as anything other than a failure 
> for the query itself.
>
>
>>
> “example.com” appears four times on page 4 - are you suggesting 
> that this be altered to read “example.com.”?
>
> Or are you suggesting that the 5 instances of the label 
> kskroll-sentinel-(is|not)-ta-2222 which refer to a “resoirce” be 
> edited to read "kskroll-sentinel-(is|not)-ta-2222.example.com"?
>
> Or both?

The second one but only for the 3 examples

       invalid IN AAAA 2001:DB8::1

       kskroll-sentinel-is-ta-2222 IN AAAA 2001:DB8::1

       kskroll-sentinel-not-ta-2222 IN AAAA 2001:DB8::1

regards
John

John Dickinson

http://sinodun.com

---

From: Warren Kumari <[email-masked]>
Date: Wed, 21 Feb 2018 08:53:23 -0500

> careful read of this draft) and found it generally well-written.

Thank you!

> I have some comments on 01.  These are basically editorial.

Even more thank you!

>   detection, but I think it's better to note it explicitly.

Good point. I had some trickiness writing this - how is "The
techniques describes in this document rely on (DNSSEC validating)
resolvers responding with SERVFAIL (RCODE 2) to valid answers. Note
that a slew of other issues can also cause SERVFAIL responses, so
false positive or negative results may sometimes occur." ?

>   I actually didn't find this term other than here and Section 9

Ah, thanks, removed.

>   choice of letter case in RFCs.

Good point. Fixed.

>   you explain it in more detail (instead of just referring to

Wow, fixed! Yup, Charlie is using a legacy, *validating* resolver.
Thanks for catching this!.

>   perhaps in the introduction (and maybe also in abstract).

Thanks, done (it was actually mentioned much earlier, but is was easy to miss).

>   but this is the first reference to 'tag-index' that made the

Hmmm... Good point. I personally actually preferred having these as
"decimal" keys.
RFC1034, Sec 5.3: The DS RR Presentation Format sayeth:
   " The Key Tag field MUST be represented as an unsigned decimal
integer.", things like dig +multiline DNSKEY . shows it as decimal,
etc.
My initial demos (www.ksk-test.net) also all used decimal. Petr
recently pointed out to me that I'd messed up, and so I converted my
demo to use hex, as the draft states.
What does the WG prefer? Is the new KSK called "20326" or it is "4f66"?

Note that Knot Resolver 2.1  already implements (thank you!) this, and as hex.
But, yes, either way, we need to note if it is padded.
[TODO(WK): Raise the hex vs Dec issue]

>   the resolver, should it be considered in the "store of trusted

Nope. Well, yes, we need to better define it, but no, a key in AddPend
is not considered.
We explicitly mean "keys which are active and can be used for
validating entries in the root zone."

We have: "In particular, this response mechanism can be used to
determine whether a certain Root Zone KSK is ready to be used as a
trusted key within the context of a key roll by this resolver."
and I added:
"An active key is one which could currently be used for validation (ie
not in AddPend or Revoked state (RFC5011)).

Sounds reasonable?

>   less confusing to just say so.

Good point. I removed responses:
"This mechanism is to be applied only by resolvers that are performing
DNSSEC validation, and applies only to responses to an A or AAAA
query..."
There are some other places where RRSet is still needed, so I left
some of them in.

>   first comment on Section 1.1 above).

Actually, there are small enough number of occurrences that explicitly
listing them (and not creating a new term) seemed better, so (see
ablove) I removed the "Address Record" term and left in the "A or
AAAA" stuff - please let me know if you agree.

>     applies to other part of this section including the table.

Thank you, good catch. I s/RRSET/RRset/ and added "or AAAA".

Thank you very much for the comments and feedback - I've just pushed a
new version to GitHub (
https://github.com/APNIC-Labs/draft-kskroll-sentinel ) and will post a
new draft on a few minutes.

W

---

From: Warren Kumari <[email-masked]>
Date: Wed, 21 Feb 2018 09:02:52 -0500

On Fri, Feb 16, 2018 at 12:35 PM, John Dickinson <[email-masked]> wrote:
> I am a bit concerned about adding a invalid RR in to a otherwise correctly
> signed zone. It suspect that there may be a variation in how validating
> resolvers treat authoritative servers that appear to have sent bogus data.
> Some might retry, retry other auth servers, stop using that server
> altogether etc etc…
>
> I suggest that the example A/AAAA RRs on page 4 be written fully qualified
> so there can be no doubt that this draft does not imply new special names at
> the root (which is what I first thought).

Thank you, that's a good catch -- I've made the names in the
"zonefile" all be fully qualified. The fact that these labels can live
anywhere in the tree was a common souce of confusion, so I should have
caught that earlier.

> should have the “non-“ removed.

Yup, apologies, that was an unfortunate cut-n-paste error. Fixed, and
I just posted a new version incorporating your and Jinmei's comments.

Review and feedback appreciated,
W

---

From: Vladimír Čunát <[email-masked]>
Date: Wed, 21 Feb 2018 17:14:00 +0100

> integer.", things like dig +multiline DNSKEY . shows it as decimal,

Yes, using the (unpadded) presentation format would certainly feel more
natural to me (rfc4034), as that seems the generally preferred format. 
I may have missed some arguments in favor of the current style.

--Vladimir  @  cz.nic

---

From: 神明達哉 <[email-masked]>
Date: Wed, 21 Feb 2018 10:44:51 -0800

At Wed, 21 Feb 2018 08:53:23 -0500,

> false positive or negative results may sometimes occur." ?

Works for me, except that I might avoid 'false positive/negative' as
it's often ambiguous or confusing (exactly what "false positive" means
depends on the context).  I'd personally say something like "..., so
the sentinel processing (Section 4) may sometimes result in incorrect
conclusions".  But that's probably minor in the entire context of this
document, so I'd leave it to you.

> [TODO(WK): Raise the hex vs Dec issue]

Personally I don't have a strong opinion/preference on hex vs decimal
or with or without leading zeros as long as it's clearly specified.
But I see the point of preferring the decimal representation you
showed above.  https://data.iana.org/root-anchors/root-anchors.xml
also uses the decimal representation, so especially if we expect the
usage of this query by a human operator who builds the query name by
hand, it would be more convenient if it's consistent with other common
representations, which is currently decimal.  But again, I don't have
a strong opinion.

> validating entries in the root zone."

To be clear, I didn't think a key in the "AddPend" state is considered
to be in the "store of trusted keys".  I just pointed out that it may
not be super clear.  And so...

> not in AddPend or Revoked state (RFC5011)).

yes it does.

> AAAA" stuff - please let me know if you agree.

Works for me.

--
JINMEI, Tatuya

---

From: Warren Kumari <[email-masked]>
Date: Wed, 21 Feb 2018 14:09:59 -0500

> document, so I'd leave it to you.

Thanks, I changed it (in the editor copy) to ", so the sentinel
processing (Section 4) may sometimes result in incorrect conclusions".
When I initially added the false positive/false negative text I spent
some time trying to figure out if it was a false positive or false
negative... and then just listed both :-)

> representations, which is currently decimal.  But again, I don't have

I'll start a separate thread on this question. I have a feeling that
I'm bikeshedding here, but decimal seems preferable to me.

> not be super clear.  And so...

Yup - you are right, it wasn't super clear (and I'd assumed that you
didn't think that AddPend counted, but it doesn't hurt to clarify in
the doc).
Thanks.

>> not in AddPend or Revoked state (RFC5011)).

Yay!
>> AAAA" stuff - please let me know if you agree.

Excellent!
W

---

From: "Paul Hoffman" <[email-masked]>
Date: Tue, 27 Feb 2018 13:28:02 -0800

On 12 Feb 2018, at 12:28, Warren Kumari wrote:

> I also updated my demo implementation

I would very much like to see draft-ietf-dnsop-kskroll-sentinel move 
forward, but was concerned that the result might be something where an 
end-user might not be able to reliably test the resolver they were using 
for it. Warren's Javascript code from that page uses the idea that an 
image that could not be loaded would have a height of zero:

       if (img_invalid.height === 0) {invalid = false;}
       if (img_is_ta.height === 0) {is_ta = false;}
       if (img_not_ta.height === 0) {not_ta = false;}

That seems to work for some browsers, but I worried that some browsers 
might implement something different for their Javascript. (This is not 
to knock Warren's code: he admitted it was a quick hack.)

After some investigation, I have a different method that should work in 
all browsers that follow the HTML and Javascript standards. It does not 
rely on any non-standard assumptions in the client-side Javascript.

In the <head> of an HTML document, first include something like this:
    <script>var collected_names = [];</script>
... to create a global variable that holds a list. Then include:
    <script 
src="http://kskroll-sentinel-is-ta-4f66.example.com/is-ta.js"></script>
    <script 
src="http://kskroll-sentinel-not-ta-4f66.example.com/not-ta.js"></script>
    <script src="http://invalid.example.com/invalid.js"></script>
The files that are attempted to be retrieved have one line of code, 
different for each file:
    collected_names.push("is_ta");
       or
    collected_names.push("not_ta");
       or
    collected_names.push("invalid");
The result is that the collected_names list now contains an entry for 
each of the domains where the .js file was fetchable. The Javascript (in 
yet another <script> block) can then process that list to determine the 
sentinel status.

I have some very simple sample code that tests this for the Vleg and 
nonV cases, but I'm still waiting for a resolver that runs 
kskroll-sentinel properly to test for all four. (Kudos to Knot Resolver 
for being first to implement, but there is a small bug that they say 
should be fixed soon.)

So, for folks waiting to see if draft-ietf-dnsop-kskroll-sentinel could 
really work in practice, this is a very good sign.

--Paul Hoffman