Subject: [DNSOP] remarks on draft-ietf-dnsop-5966bis-01

From: Andrew Sullivan <[email-masked]>
Date: Tue, 17 Mar 2015 21:38:06 -0400

Dear colleagues,

I have read draft-ietf-dnsop-5966bis-01.  I have some comments.

To begin with, I must say that the draft is in really very good shape
and I think it more or less ready to ship.  Good work.  I support the
document and believe it should be published as an RFC once a couple
small issues are addressed:

In section 5, there's "In essence, TCP SHOULD be considered as valid a
transport as UDP."  I don't think that's a 2119 SHOULD.  It's just an
aspriational statement, really.

In section 6, there's this:

    The server MUST NOT enforce these rules for a particular
   client because it does not know if the client IP address belongs to a
   single client or is, for example, multiple clients behind NAT.

I don't think that MUST NOT is reasonable.  I could accept SHOULD NOT,
I suppse.  This topic looks to me like operational policy, and an
operator could easily decide to enforce such a limit in case (for
instance) that it knows that there are no NATs of the sort in question
in the network in question.  Mostly I think this just wants a big fat
warning that if you _do_ create such limits, NATs may be broken in
surprising ways, which is a good reason not to do that.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Wed, 18 Mar 2015 01:59:56 -0700

i have read draft-ietf-dnsop-5966bis-01, and i have some comments.

tl;dr: this document is nowhere near ready to ship.

> ...
>
>    This document therefore updates the core DNS protocol specifications
>    such that support for TCP is henceforth a REQUIRED part of a full DNS
>    protocol implementation.

this language is too strong, and breaks working configurations. many DNS
servers are completely functional for all intents and purposes even
though an upstream firewall or other middlebox prevents them from
hearing TCP requests.

i suggest some adjective other than REQUIRED here. perhaps STRONGLY
RECOMMENDED.

>
>    There are several advantages and disadvantages to the increased use
>    of TCP as well as implementation details that need to be considered.
>    This document addresses these issues and therefore extends the
>    content of [RFC5966], with additional considerations and lessons
>    learned from new research and implementations
>    [Connection-Oriented-DNS].

i object to this reference. [Connection-Oriented-DNS] is a controversial
paper, which i've reviewed separately, and should not be referenced in
standards work.

>    Whilst this document makes no specific requirements for operators of
>    DNS servers to meet,

if that's true, then you really can't say REQUIRED (see above).

> ...
>
>    However, transport of UDP packets that exceed the size of the path
>    MTU causes IP packet fragmentation, which has been found to be
>    unreliable in some circumstances.

can i ask, as the RFC 2671 author, that you change the word "some" to be
instead the word "many" ?

> 6.  Connection Handling
>
>    One perceived disadvantage to DNS over TCP is the added connection
>    setup latency, generally equal to one RTT.  To amortize connection
>    setup costs, both clients and servers SHOULD support connection reuse
>    by sending multiple queries and responses over a single TCP
>    connection.
>
>    DNS currently has no connection signaling mechanism.  Clients and
>    servers may close a connection at any time.

this is factually wrong. servers are not permitted to close a connection
at any time. (this is what the spec says, and the expectations of any
implementation that faithfully followed the spec must be called
"reasonable").

>   Clients MUST be prepared
>    to retry failed queries on broken connections.

this is factually wrong. a client can give up on a server for at least
the duration of the current transaction if it closes a connection
prematurely.

>    Section 4.2.2 of [RFC1035] says:
>
>       If the server needs to close a dormant connection to reclaim
>       resources, it should wait until the connection has been idle for a
>       period on the order of two minutes.  In particular, the server
>       should allow the SOA and AXFR request sequence (which begins a
>       refresh operation) to be made on a single connection.  Since the
>       server would be unable to answer queries anyway, a unilateral
>       close or reset may be used instead of a graceful close.
>
>    Other more modern protocols (e.g., HTTP/1.1 [RFC7230]) have support
>    for persistent TCP connections and operational experience has shown
>    that long timeouts can easily cause resource exhaustion and poor
>    response under heavy load.  Intentionally opening many connections
>    and leaving them dormant can trivially create a "denial-of-service"
>    attack.

noting: this is why i frequently refer to the RFC 1035 connection
management logic as "unfortunate".

>    It is therefore RECOMMENDED that the default application-level idle
>    period should be of the order of seconds, but no particular value is
>    specified.  In practice, the idle period may vary dynamically, and
>    servers MAY allow dormant connections to remain open for longer
>    periods as resources permit.

noting, as a recommendation, this is harmless to the installed base, as
long as the installed base remembers that a faithful implementation of
RFC 1035 4.3.2 will not be doing this, and avoids perturbing those
servers with long-lived TCP sessions that their connection management
logic is ill-equipped to handle.

>    To mitigate the risk of unintentional server overload, DNS clients
>    MUST take care to minimize the number of concurrent TCP connections
>    made to any individual server.

not just concurrent, but also, idle. no idle connections can be
permitted unless the client knows absolutely that the server is modern
and is following the specification now being reviewed.

> ...
>    For reasons of efficiency, implementations SHOULD wherever possible
>    attempt to coalesce the two byte length field and subsequent DNS
>    payload data into a single packet.

i think you can word this more strongly. as in: "Since TCP responders
are sensitive to timeout conditions, some extant implementations will
abort a TCP session if the first TCP window does not contain both the
two-octet length field and the entire request message described by that
length field. TCP initiators should therefore take care to transmit the
two-octet length field, and the message described by that length field,
in a single TCP window."

>    If a server finds that a client has closed a TCP session, or if the
>    session has been otherwise interrupted, before all pending responses
>    have been sent then the server MUST NOT attempt to send those
>    responses.  Of course the server MAY cache those responses.

this is a no-op, since there would be nobody to send the responses to. i
recommend deleting this paragraph.

> 7.  Query Pipelining
>
>    Due to the use of TCP primarily for zone transfer and truncated
>    responses, no existing RFC discusses the idea of pipelining DNS
>    queries over a TCP connection.
>
>    In order to achieve performance on par with UDP DNS clients SHOULD
>    pipeline their queries.  When a DNS client sends multiple queries to
>    a server, it should not wait for an outstanding reply before sending
>    the next query.  Clients should treat TCP and UDP equivalently when
>    considering the time at which to send a particular query.

add: "TCP initiators should be aware that some responders will answer in
order no matter what, and they should not therefore expect parallelism.
If performance is a consideration, the use of UDP will be a better
guarantor of same, unless the initiator is sure that the responder is
acting according to this amendment to the DNS protocol."

>    DNS servers (especially recursive) SHOULD expect to receive pipelined
>    queries.  The server should process TCP queries in parallel, just as
>    it would for UDP.  The handling of responses to pipelined queries is
>    covered in the following section.
>
>    When pipelining queries over TCP it is very easy to send more DNS
>    queries than there are DNS Message ID's.  Implementations MUST take
>    care to check their list of outstanding DNS Message ID's before
>    sending a new query over an existing TCP connection.  This is
>    especially important if the server could be performing out-of-order
>    processing.

i think you mean "since" rather than "if", as in, "since the server
could be performing out-of-order processing." that is, if an initiator
is sure that the responder is answering in order, as most TCP responders
do today, then the initiator would be free to re-use the same TXID over
and over, as some TCP initiators are known to do today.

>   In addition, when sending multiple queries over TCP it
>    is very easy for a name server to overwhelm its own network
>    interface.  Implementations MUST take care to manage buffer sizes or
>    to throttle writes to the network interface.

i cannot imagine what this could mean, in real terms, to a resolver
implementer, having only UNIX or Windows system calls to work with. can
you provide some examples?

> 8.  Response Reordering
>
>    RFC 1035 is ambiguous on the question of whether TCP responses may be
>    reordered -- the only relevant text is in Section 4.2.1, which
>    relates to UDP:
>
>       Queries or their responses may be reordered by the network, or by
>       processing in name servers, so resolvers should not depend on them
>       being returned in order.
>
>    For the avoidance of future doubt, this requirement is clarified.

you can't do that. let me explain:

>    Authoritative servers and recursive resolvers are RECOMMENDED to
>    support the sending of responses in parallel and/or out-of-order,
>    regardless of the transport protocol in use.

this works as a recommendation for responders, which initiators had
better be prepared to cope with, since the only time they have to cope
with it is if they use pipelining.

>   Stub and recursive
>    resolvers MUST be able to process responses that arrive in a
>    different order to that in which the requests were sent, regardless
>    of the transport protocol in use.

this does not work as a MUST, because existing implementations which
were faithful to RFC 1035 might not be able to do this -- and you know
this or you would not be trying to clarify the point at this late date.
i think that if you want new behaviour you have to negotiate for it, not
merely mandate it, since to mandate it would invalidate previously-valid
implementations and break previously-working configurations.

> ...
> 9.  TCP Fast Open
>
>    This section is non-normative.
>
> ...
>
>    Adding support for this to existing name server implementations is
>    relatively easy, but does require source code modifications.  On the
>    client, the call to connect() is replaced with a TFO aware version of
>    sendmsg() or sendto(). ...

i object to the use of UNIX syscalls being used to describe a protocol,
even non-normatively.

let me close by quoting my final remarks in the review of
[Connection-Oriented-DNS]:

--------

the backdrop of these comments is that the DNS is a large, complex, and mostly-working system. as the stewards of its protocol, we have an obligation to do no avoidable harm. since a TCP/53 listener may have been developed and deployed by someone not within the sound of DNSOP's voice today, but who faithfully followed the unfortunate advice in RFC 1035 regarding TCP connection management, we have to respect their choices. what that really means is we have to avoid saturating their small TCP connection capacity (as managed by their long session timeouts) since that TCP capacity is the only hope of any UDP initiator who hears TC=1 from them. i argue that while any attacker can take down that TCP capacity at will with a trivial one-line perl script, that does not give us the right to take it down willfully, by using persistent TCP by default.

--------

see also:

http://www.circleid.com/posts/20130913_on_the_time_value_of_security_features_in_dns/

http://queue.acm.org/detail.cfm?id=1242499

paul vixie

---

From: Tony Finch <[email-masked]>
Date: Wed, 18 Mar 2015 10:08:23 +0000

Andrew Sullivan <[email-masked]> wrote:
> I don't think that MUST NOT is reasonable.

I think the recommended limits in that paragraph are OK only if viewed
from the perspective of individual client programs. The above quote is
correct in that there's no sensible way for the server to enforce the
limit - never mind NAT, if a user starts up a second browser, will it be
locked out of TCP queries until the first one closes its connection?

I suggest:

   To mitigate the risk of unintentional server overload, DNS clients
   MUST take care to minimize the number of concurrent TCP connections
   made to any individual server. It is RECOMMENDED that for any given
   client - server interaction a client SHOULD limit itself to no more
   than one connection for regular queries, one for zone transfers and
   one for each protocol that is being used on top of TCP, for example,
   if the resolver was using TLS.  Servers MAY impose limits
   on the number of concurrent TCP connections being handled for any
   particular client. These limits SHOULD be much looser than the client
   guidelines above, because it does not know if the client IP
   address belongs to a single client or is, for example, multiple
   resolvers on a single machine, or multiple clients behind NAT.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Faeroes: West or southwest 5 or 6, becoming variable 4, then becoming south or
southeast 5 or 6. Moderate or rough. Occasional rain. Good, becoming moderate
or poor.

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 18 Mar 2015 08:23:48 -0400

> locked out of TCP queries until the first one closes its connection?

I agree with this; it's only the MUST NOT that I think goes too far,
since that's a restriction on what an implementation might offer.  I
like the proposed alternative text (which I elided).

A

-- 
Andrew Sullivan
[email-masked]

---

From: John Kristoff <[email-masked]>
Date: Wed, 18 Mar 2015 07:37:24 -0500

On Wed, 18 Mar 2015 01:59:56 -0700
Paul Vixie <[email-masked]> wrote:

> >    This document therefore updates the core DNS protocol
> > specifications such that support for TCP is henceforth a REQUIRED
> > part of a full DNS protocol implementation.

> this language is too strong, and breaks working configurations.

That is the exact same language used in IETF RFC 5966.  The original
document, along with this draft still says  it "makes no specific
recommendations to operators of DNS servers".  It is written for
"implementors".  This seems to be a common misunderstanding.

John

---

From: Ray Bellis <[email-masked]>
Date: Wed, 18 Mar 2015 13:38:47 +0000

> On 18 Mar 2015, at 12:37, John Kristoff <[email-masked]> wrote:
> 

That's exactly correct, and was required to get 5966 through IESG review.

Ray

---

From: Paul Vixie <[email-masked]>
Date: Wed, 18 Mar 2015 08:49:04 -0700

> That's exactly correct, and was required to get 5966 through IESG review.

for avoidance of doubt, let's try writing what we actually mean, and arguing it to IESG if necessary. the current philosophy of DNSOP is to write documents in the "If you want to do this, here's how to do it interoperably" style, rather than the "If you're going to do this at all, you must do it this way" style. 

-- 
Paul Vixie

---

From: "W.C.A. Wijngaards" <[email-masked]>
Date: Thu, 19 Mar 2015 09:56:56 +0100

Hi Paul,

On 18/03/15 09:59, Paul Vixie wrote:
>> Stub and recursive resolvers MUST be able to process responses
>> that arrive in a different order to that in which the requests
>> were sent, regardless of the transport protocol in use.
> 
> this does not work as a MUST, because existing implementations
> which were faithful to RFC 1035 might not be able to do this -- and
> you know this or you would not be trying to clarify the point at
> this late date. i think that if you want new behaviour you have to
> negotiate for it, not merely mandate it, since to mandate it would
> invalidate previously-valid implementations and break
> previously-working configurations.

+1.  Backwards compatibility means you cannot specify that existing
implementations have to change.  You can specify that 'upgraded'
implementations perform new actions, of course.

Best regards,
   Wouter

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Mar 2015 06:56:45 -0400

> implementations perform new actions, of course.

No RFC can make changes that apply to implementations that do not
conform with that RFC.  This is true by definition.  The draft does
not somehow magically retroactively change the text in RFC 1035.  It
specifies new behaviour, and old implementations that don't conform to
this new text will therefore obviously not exhibit the new behaviour.
What's the problem?

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Thu, 19 Mar 2015 03:58:47 -0700

> Andrew Sullivan <mailto:[email-masked]>
> Thursday, March 19, 2015 3:56 AM
>

the problem is that the definition of DNS doesn't change incompatibly.

if you want new behaviour, negotiate for it.

-- 
Paul Vixie

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Mar 2015 07:05:12 -0400

> if you want new behaviour, negotiate for it.

So you want a way to signal the new behaviour and negotiate that?

It strikes me that the EDNS0 option that Paul Wouters wrote up was
intended to offer such signalling, and people thought it was not
necessary.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Thu, 19 Mar 2015 04:27:48 -0700

> So you want a way to signal the new behaviour and negotiate that?

i don't care whether there's new behaviour. but if new behaviour is
going to be incompatible with existing implementations, then that new
behaviour must be negotiated. that's how we rebuilt this airplane in
flight over and over again from 1986 to the present. it's all "DNS".

> intended to offer such signalling, and people thought it was not

i was not one of those people. 

-- 
Paul Vixie

---

From: Florian Weimer <[email-masked]>
Date: Sat, 21 Mar 2015 15:12:10 +0100

* W. C. A. Wijngaards:

> implementations have to change.

Does it matter if they do not exist or are not considered practically
relevant?

As a counterexample, RFC 6891 requires FORMERR responses without OPT
RRs from implementations which do not support EDNS:

   Responders that choose not to implement the protocol extensions
   defined in this document MUST respond with a return code (RCODE) of
   FORMERR to messages containing an OPT record in the additional
   section and MUST NOT include an OPT record in the response.

---

From: Paul Vixie <[email-masked]>
Date: Sat, 21 Mar 2015 11:08:48 -0700

> Does it matter if they do not exist or are not considered practically

not usually. if there's a standard for it, our burden is to assume that
there are faithful implementations of it. only if there is no way to
negotiate for the new behaviour (as there would be for tcp stay-open, so
this exception does not apply to the draft under consideration) would we
even consider assuming otherwise.

>    section and MUST NOT include an OPT record in the response.

this language was very careful. older implementations cannot "choose" behaviour relative to this standard. so, as an example of what you're trying to convey, this text fails. 

-- 
Paul Vixie

---

From: Florian Weimer <[email-masked]>
Date: Sun, 22 Mar 2015 19:55:43 +0100

* Paul Vixie:

> this language was very careful. older implementations cannot
> "choose" behaviour relative to this standard. so, as an example of
> what you're trying to convey, this text fails.

They could patch the RCODE in the header and reflect the packet.  That
used to be a valid implementation option, and it was sort-of assumed
by the previous EDNS0 version with its extended label types.  It was
the only way to generate a FORMERR response when you could not parse
even the QNAME.  (Or you could say that EDNS0 encouraged matching
responses based on transaction IDs only, but let's not go there.)