Subject: [DNSOP] MIXFR: Smaller IXFR in the DNSSEC case

From: Matthijs Mekking <[email-masked]>
Date: Thu, 15 Jan 2015 11:13:10 +0100

Hi wg,

IXFR with DNSSEC is suddenly not so small anymore. Do you recognize
this? Olafur and I have some ideas on keeping those zone transfers
small. Your feedback is appreciated.

  http://www.ietf.org/internet-drafts/draft-mekking-mixfr-01.txt

Best regards,
  Matthijs

---

From: 神明達哉 <[email-masked]>
Date: Thu, 15 Jan 2015 10:33:54 -0800

At Thu, 15 Jan 2015 11:13:10 +0100,

>   http://www.ietf.org/internet-drafts/draft-mekking-mixfr-01.txt

I see the motivation, and the proposed approach of MIXFR may make
sense.  But, just like for any kind of optimization ideas, I would
wonder whether this could be a premature one.  Do you have any
measurement of the effect of this idea?

On the draft text (also related to this higher level point):

   The goal of this proposal is to allow small changes to be
   communicated over UDP, and remove as much redundant information from
   the zone transfer as possible.

We still need to send new RRSIGs, and since the main concern is the
size of them (whether they are to be removed or added), I guess
sending a non-negligible number of RRSIGs could easily require TCP,
even if we can omit a half of them.  So I'm not sure how often we can
avoid falling back to TCP (M)IXFR thanks to this in practice.  Again,
some actual measurement or at least a quantitative analysis may help.

Regarding Section 5 (IXFR Gone Wild: Even more optimized transfers):
if we go this far, I wonder whether we might just use generic and more
efficient compression and exchange compressed data.

--
JINMEI, Tatuya

---

From: Matthijs Mekking <[email-masked]>
Date: Fri, 16 Jan 2015 09:27:27 +0100

> measurement of the effect of this idea?

Not yet, but it is on my wish list to investigate this more and measure
some use cases. My idea here is to look at TLDs, since those are large
and dynamic zones.

> sending a non-negligible number of RRSIGs could easily require TCP,

We would always have to send new RRSIGs, I can't really see how we could
live without?

> efficient compression and exchange compressed data.

I think reusing the current format of the protocol aids deployability.
If we are indeed going this far and introduce a new transfer format I
indeed believe we could very well end up with a more generic compression
solution.

Thanks,
  Matthijs

---

From: Olafur Gudmundsson <[email-masked]>
Date: Fri, 16 Jan 2015 10:51:17 -0500

> On Jan 15, 2015, at 1:33 PM, 神明達哉 <[email-masked]> wrote:
> 
Jinmei,  
thank you for your good comments. 

> measurement of the effect of this idea?

This is a real good point, I would hope we (or others) have some 
information on that before we standardize, right now this just an idea to discuss. 
The quick back of the envelope calculations say 30-45% for DNSSEC signed zones, that are just
being resigned. The milage on other operations my differ. 

This begs the question what is the best way to measure? 

> some actual measurement or at least a quantitative analysis may help.

not sending the OLD RRsig’s is a big savings on its own, in particular when people
use large RSA keys. Close to 50% when all you are just refreshing a signature on a one or two RRsets. 

> efficient compression and exchange compressed data.
One of the oldest ideas on that was from Andreas Gustafsson was to wrap
XFR transmission inside compressed transmission. 
But the biggest reason not to do compression is that RRSIG rdata is not compressible. 

I have a much more radical zone transfer proposal in the works that is over persistent TCP
connections and that is ripe for secured and compressed transmission. 

    Olafur

---

From: Miek Gieben <[email-masked]>
Date: Fri, 16 Jan 2015 15:59:12 +0000

[ Quoting <[email-masked]> in "Re: [DNSOP] MIXFR: Smaller IXFR in ..." ]
>connections and that is ripe for secured and compressed transmission.

I think it makes sense to talk about *that* instead of optimizing *XFR.

/Miek

--
Miek Gieben

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 16 Jan 2015 08:45:23 -0800

> This begs the question what is the best way to measure? 

Indeed. We can look at zones and measure, but not all zones use IXFR to update. Some zones don't use IXFR, but instead use other protocols to move the DNS RRs as data. One person said their zones use rsync-of-small-files because it is just as fast and yet is much easier to implement and monitor. Whatever measurements are done, they should be done on zones that are known to use IXFR.

> connections and that is ripe for secured and compressed transmission. 

That is not "radical" for zone admins who are already doing it. :-)

--Paul Hoffman

---

From: 神明達哉 <[email-masked]>
Date: Fri, 16 Jan 2015 09:29:54 -0800

At Fri, 16 Jan 2015 09:27:27 +0100,

> > some actual measurement or at least a quantitative analysis may help.

Apparently I was not clear about my point.  Let me rephrase it:
If (part of) the goal is "to allow small changes to be communicated
over UDP", and if the proposed solution is to be a useful tool for
that goal, then we should have the following situation:

  sizeof(MIXFR) <= client's UDP payload size < sizeof(IXFR) (*)
  (where both 'MIXFR' and 'IXFR' contain a non-negligible number of
  RRSIGs, and 'IXFR' would be twice as large as 'MIXFR')

It's certainly true sizeof(MIXFR) is much smaller than sizeof(IXFR),
but since MIXFR itself would still contain some amount of large data
(i.e., new RRSIGs), I guess it may not be uncommon that
sizeof(MIXFR) > client's UDP payload size.
In this case, MIXFR doesn't help in terms of avoiding the fallback to
TCP.

On the other hand, if the size of new RRSIGs is quite small, it would
also be possible that sizeof(IXFR) <= client's UDP payload size.
MIXFR doesn't help in terms of avoiding the fallback to TCP in this
case either.

So my point is that it was not immediately clear how often we see the
situation where the condition (*) holds in practice.  And, hence "also
related to this higher level point" (= I wonder whether there's a
measurement result).

I hope I'm clear enough this time.

--
JINMEI, Tatuya

---

From: 神明達哉 <[email-masked]>
Date: Fri, 16 Jan 2015 09:32:25 -0800

At Fri, 16 Jan 2015 10:51:17 -0500,

> use large RSA keys. Close to 50% when all you are just refreshing a signature on a one or two RRsets.

I know it's a big savings.  My (apparently unclear) point was it was
not clear to me how effective this "savings" is in terms of avoiding
the fallback to TCP.  See my other message I just sent to Matthijs
(and the list).

--
JINMEI, Tatuya

---

From: Paul Vixie <[email-masked]>
Date: Fri, 16 Jan 2015 09:58:32 -0800

> Olafur Gudmundsson <mailto:[email-masked]>
> Friday, January 16, 2015 7:51 AM
> ...

late BIND4 and early BIND8 had something called ZXFR that did this. it
never worked out of the box, but frederico neves in brazil fixed it and
had it running in production for his inter-site synchronization some
time in the mid/late 1990's. it's worth asking him if it was worthwhile
(noting, this was before incompressible DNSSEC signatures were added.)

on the topic of this draft, the current IXFR encoding requires
transmitting the old RRsets, which in the case of DNSSEC are large. the
benefit of this is that it provides extra assurance of synchronization
-- if the receiver does not have the old RRsets then we know that IXFR
can't work, and AXFR is tried.

my proposal at the time IXFR was being worked on back in DNSEXT was to
use the UPDATE encoding, which allows RRset deletion or replacement
without transmitting the old RRset. i still think that's a good plan,
and if...

> connections and that is ripe for secured and compressed transmission. 

...if we were going to do zone synchronization over a persistent TCP
session, then i would recommend once again the UPDATE encoding for the
individual deltas.

-- 
Paul Vixie

---

From: Bob Harold <[email-masked]>
Date: Fri, 16 Jan 2015 17:04:54 -0500

There seem to be a lot of "set CLASS to ANY" in the spec.  But I thought
that a.b.c class IN was totally unrelated to a.b.c class CHAOS, and
deleting or changing one should not affect the other.  Or am I confused?
And everything in the zone must be in the same CLASS, so can we drop the
CLASS completely?

-- 
Bob Harold
hostmaster, UMnet, ITcom
Information and Technology Services (ITS)
[email-masked]
734-647-6524 desk

On Fri, Jan 16, 2015 at 12:58 PM, Paul Vixie <[email-masked]> wrote:

>
>
>   Olafur Gudmundsson <[email-masked]>
> never worked out of the box, but frederico neves in brazil fixed it and had
> it running in production for his inter-site synchronization some time in
> the mid/late 1990's. it's worth asking him if it was worthwhile (noting,
> this was before incompressible DNSSEC signatures were added.)
>
> benefit of this is that it provides extra assurance of synchronization --
> if the receiver does not have the old RRsets then we know that IXFR can't
> work, and AXFR is tried.
>
> my proposal at the time IXFR was being worked on back in DNSEXT was to use
> the UPDATE encoding, which allows RRset deletion or replacement without
> transmitting the old RRset. i still think that's a good plan, and if...
>
> _______________________________________________
> DNSOP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsop
>
>

---

From: Paul Vixie <[email-masked]>
Date: Fri, 16 Jan 2015 14:11:38 -0800

> Tony Finch <mailto:[email-masked]>
> Friday, January 16, 2015 11:19 AM
>
> I discussed that in some detail with Olafur and Matthijs, and section
> 3 of the MIXFR draft is basically adding UPDATE syntax to IXFR. (I
> thought MIXFR should be called UXFR.)

unless you have some reason to dislike code-reuse and library
modularity, i hope you'll make it "exactly like UPDATE" rather than
"basically adding UPDATE".

-- 
Paul Vixie

---

From: Mark Andrews <[email-masked]>
Date: Sat, 17 Jan 2015 12:51:37 +1100

> "basically adding UPDATE".

Sending UPDATE messages instead of IXFR deltas is dangerous.  UPDATE
is tolerent of a lots of things that you really don't want to allow
when doing a incremental update of a zone.

Delete of non-existant data is not a error with UPDATE.  It "cannot
happen" with IXFR so if you see you know you are out of sync.

CNAME related changes may or may not happen depending upon the state
of the zone.

Adds of existing data is not a error with UPDATE.  This is "cannot
happen" with IXFR so if you see you know you are out of sync.

Out of sync does happen.  That code does get exercised.  People
mis-manage master zones.  Database delta -> IXFR has been problematic
in the past.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Fri, 16 Jan 2015 17:58:05 -0800

> Mark Andrews <mailto:[email-masked]>
> Friday, January 16, 2015 5:51 PM
>

yes, it's dangerous, for the reason you've stated.
> mis-manage master zones. Database delta -> IXFR has been problematic

some constraints, like "prereq must always be specified", could be added
for the "UXFR" case.

-- 
Paul Vixie

---

From: Tony Finch <[email-masked]>
Date: Sat, 17 Jan 2015 02:02:22 +0000

The main difference from UPDATE is the "replace" function, which combines delete and add. This is crucial for efficient RRSIG handling.

If you re-sign a subset of the RRsets at an owner, you can't say which RRSIG records to delete (which algorithm and type-covered) without providing the complete RDATA in UPDATE syntax. So UPDATE does not give you an advantage over IXFR in this common case.

MIXFR allows you to provide just the new RRSIGs, and the receiver knows they replace old RRSIGs with matching owner algorithm and type-covered.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at

> On 16 Jan 2015, at 22:11, Paul Vixie <[email-masked]> wrote:

> unless you have some reason to dislike code-reuse and library modularity, i hope you'll make it "exactly like UPDATE" rather than "basically adding UPDATE".

---

From: Matthijs Mekking <[email-masked]>
Date: Mon, 19 Jan 2015 09:50:19 +0100

On 16-01-15 23:04, Bob Harold wrote:
> deleting or changing one should not affect the other.  Or am I

To clarify: A record with its CLASS set to ANY does *not* mean to
delete/change the record in all available classes. Note that an XFR is
encapsulated in SOA records that determine the zone name and class (see
Figure 2). Only changes in the zone matching that name and class will be
made.

> confused?  And everything in the zone must be in the same CLASS, so can
> we drop the CLASS completely?

Because everything in the zone must be in the same CLASS, we could drop
it, bit we cam also reuse the CLASS field to introduce new
functionality. That is what is being done here.

Best regards,
  Matthijs

> [email-masked] <mailto:[email-masked]>
> 734-647-6524 desk
> 
> On Fri, Jan 16, 2015 at 12:58 PM, Paul Vixie <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> 
> 
>     late BIND4 and early BIND8 had something called ZXFR that did this.
>     it never worked out of the box, but frederico neves in brazil fixed
>     it and had it running in production for his inter-site
>     synchronization some time in the mid/late 1990's. it's worth asking
>     him if it was worthwhile (noting, this was before incompressible
>     DNSSEC signatures were added.)
> 
>     transmitting the old RRsets, which in the case of DNSSEC are large.
>     the benefit of this is that it provides extra assurance of
>     synchronization -- if the receiver does not have the old RRsets then
>     we know that IXFR can't work, and AXFR is tried.
> 
>     my proposal at the time IXFR was being worked on back in DNSEXT was
>     to use the UPDATE encoding, which allows RRset deletion or
>     replacement without transmitting the old RRset. i still think that's
>     a good plan, and if...
> 
>     session, then i would recommend once again the UPDATE encoding for
>     the individual deltas.
> 
>     -- 
>     Paul Vixie
> 
>     [email-masked] <mailto:[email-masked]>

---

From: Frederico A C Neves <[email-masked]>
Date: Wed, 21 Jan 2015 20:13:20 -0200

> time in the mid/late 1990's. it's worth asking him if it was worthwhile

This was late 1990's, in a time that we didn't have registry backend
journals, way before bind started providing ixfr-from-differences on
9.3, OSs still struggled with the long fat pipe problem and we happen
to have some secondaries on the 350ms vicinity... it was definitely
worthwhile. But today I guess in the same scenario a vpn based
transport solution would be easier.

> (noting, this was before incompressible DNSSEC signatures were added.)

True only for individual records. I guess the authors are aiming at
small zones and Olafur had this in mind when doing this comment. For a
even a few thousand records signed zone I see quite good compression
only taking in account RRSIGs RDATA.

Taking synchronization efficiency in perspective, reducing the amount
of redundant information, explicitly suppressing it, is the best
approach. MIXFR ideas are worth pursuing.

Fred