Subject: [BEHAVE] Using DNS64 to load balance to NAT64

From: Cameron Byrne <[email-masked]>
Date: Mon, 19 Oct 2009 23:01:19 -0700

Folks,

One of the major problems many operators have is scaling the CGN in a
reliable way. Scaling in the number of session per seconds as well as
raw concurrent sessions continues to be an issue, and will continue to
be an issue.  Additionally, given that state and ALGs are being used,
boxes are known to reboot when protocols act in unexpected ways, so
fault isolation is another desire (array of independent boxes).

Does the scenarios where i have boxes NAT64-A, NAT64-B, and NAT64-C
(all unique state boxes or high-available pairs of active/standby
boxes, with unique prefix64 A, B, and C respectively announced from
each) and  the DNS64 load balances traffic (for example, round-robin
assignments of Prefix64-A, B, and C to resolving clients) destine to
each NAT64-A, B, C systems evenly. This way, the DNS64 drives 1/3 of
the requests to each of the 3 NAT64 systems.  And, it scales linearly
in terms of NAT64 boxes.  This assumes the DNS64 system scales higher
than the NAT64 systems, which i think is a good assumption.

Additionally, the NAT64 and the DNS64 should be able to communicate in
such a way that the NAT64 can signal an overloading situation such
that the DNS64 drops the overloaded NAT64 out of the round-robin
prefix64 synthesis for some period of time.  This could be implemented
as simple SNMP GET sampling of the sessions that NAT64 systems are
currently processing.

Does this make sense to have this level of load-balancing between an
array of independent NAT64 boxes and feedback / back-pressure
communicated to the DNS64 system that is ultimately driving the load
to the NAT64 boxes?

Does this fit within the existing drafts?  Or is this implementation
minutia that does not require being covered here?  At a minimum, i
think it would be nice to have the DNS64 draft examine how the DNS64
system can drive load to multiple unique NAT64 systems.

Thoughts?

Regards,

Cameron Byrne

Cameron

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 20 Oct 2009 14:58:54 +0800

Hi Cameron,

> system can drive load to multiple unique NAT64 systems.

In the following draft
(http://tools.ietf.org/id/draft-xu-behave-nat64-standby-00.txt), some
consideration about the load-balancing together with redundancy is
discussed. Of course, this demo does not discuss the issues with achieving
best balance by adjusting the numbers of hosts adopting different prefix64s
since there are many possible options to achieve this, e.g., by using DNS64,
or using prefix-learning mechanism defined in
[draft-wing-behave-learn-prefix-03].

Xiaohu

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Oct 2009 09:21:45 -0400

> than the NAT64 systems, which i think is a good assumption.

As a general matter, DNS-based load balancing is broken.  If nothing
else, there are way too many examples of applications holding onto an
IP address and continuing to use it, which means that your attempts to
take things in and out of service based on what you think the DNS is
giving out will cause failures.  (It's also almost certainly going to
cause you heartburn with DNSSEC, but I'm just assuming here that
you're planning to give up on DNSSEC in this context.)  Moreover,
there are often interdependencies among different requests such that
the attempts to load balance via DNS breal the interdependencies and
cause otherwise working arrangements to fail.

Based on that previous experience, I suspect that the above plan is
not a good idea.  What I would prefer to see suggested is some sort of
multiple-redundant NAT64 set, with multiple-redundant DNS64.  A client
should not use inconsistent sets of DNS64 servers (on a given
interface), but should stick to one set.  

Nevertheless, I am realistic enough to realise that your proposal is
bound to be implemented by someone, because we've invented the DNS64.
I suspect it will be fragile in exactly the ways DNS load balancing is
today.
 
> as simple SNMP GET sampling of the sessions that NAT64 systems are

That does not seem to be consistent with the current DNS64 draft,
which says that the DNS64 and NAT64 share one common configuration but
are otherwise not coupled.  It sounds like you're proposing to
reinvent the NAT-PT DNS ALG, and if we were going to do that why
wouldn't we just write "RFC 4966 considered harmful" and be done with
it?

> system can drive load to multiple unique NAT64 systems.

I would need quite a bit more argument in favour of this technique in
order to be co-operative about adding that sort of discussion to the
draft.  It is _very hard_ to control what people will do with the data
they get out of the DNS, and even the limited number of assumptions we
have in there will clearly break some installations.  (I have a
gateway device sitting on my desk that would almost certainly
immediately break were it to be put behind a DNS64 service, because
it's a mixed-mode iterator and forwarder.)  I'm not really keen on
inventing new ways to make DNS64 more fragile.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 20 Oct 2009 16:10:14 +0200

On 20 okt 2009, at 08:01, Cameron Byrne wrote:

> prefix64 synthesis for some period of time.

I have one problem here in addition to Andrew's objections:

If make a given host move to a different NAT64, that means that for  
some time, there will be an inconsistency between the external IPv4  
addresses associated with different sessions. This makes protocols  
like STUN fail, and as such, should be avoided.

You really don't want to run so close to your translator capacity that  
systems like this become useful.

---

From: Cameron Byrne <[email-masked]>
Date: Tue, 20 Oct 2009 08:33:12 -0700

On Tue, Oct 20, 2009 at 7:10 AM, Iljitsch van Beijnum
<[email-masked]> wrote:
> If make a given host move to a different NAT64, that means that for some
> time, there will be an inconsistency between the external IPv4 addresses
> associated with different sessions. This makes protocols like STUN fail, and
> as such, should be avoided.
>

Unfortunately,  there is no box that meets my translation capacity
needs today.  I do over 10 million busy hour simultaneous translations
today in the USA, and that number is going to grow and grow (100%+
growth a year is the trend) with more users and more things like AJAX,
always on connections, application NAT keepalives .....  So, the
scaling mechanism as put forth today, is broken in NAT64 / DNS64 since
it is bounded and coupled to pairings of NAT64 and DNS64 boxes.

The NAT44 and NAT64 designs, as is , require a lot of routing magic to
balance the traffic and meet the demand.  Today, i have to have arrays
of boxes that scale linearly as part of the NAT44 topology design.
There is no other way.  I have to use policy based routing on routers
to carve up the NAT44 demand from my customers and assign it to NAT44
boxes.   So even in a highly distributed deployment, I have nearly 20
locations today in the USA where i have arrays of NAT44 capacity,
scaling is not ideal.  It is ugly to look at for a purest... but it
works and scales linearly.

It would be nice if we could bake in an ability for DNS64 / NAT64 to
scale linearly (10x boxes / Prefix64 is practical). Today, coupling
DNS64 capacity with NAT64 capacity won't work for me.  I will again
have to slice and dice capacity with router tricks to balance the load
over an array of capacity.  Not ideal.  We need a better way.

Scaling is a practical and real problem that we must address.  We
cannot turn our noses up at this problem as a minor issue.

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Oct 2009 11:39:07 -0400

> cannot turn our noses up at this problem as a minor issue.

I don't think anyone is turning up noses.  The plain fact is that
using "DNS load balancing" to scale these sorts of things is broken
_today_, and DNS64 can't be any better (indeed, it can make things
worse).  

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Cameron Byrne <[email-masked]>
Date: Tue, 20 Oct 2009 08:41:08 -0700

> [draft-wing-behave-learn-prefix-03].

The mechanics in this draft are very similar to the state of the
practice today in any vendors NAT44 implementation.  Today, most NAT44
products do the things described in this draft.  I am specifically
looking for something that scales linearly leverages the control plane
(DNS64?) to drive the traffic.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 20 Oct 2009 17:49:07 +0200

On 20 okt 2009, at 17:33, Cameron Byrne wrote:

> Today, coupling
> DNS64 capacity with NAT64 capacity won't work for me.

(Did we land on a conclusion on whether it would be ok to put a  
caching DNS server behind a DNS64?)

The requirement is that the DNS64 that a host uses and the NAT64 that  
a host uses agree on the prefix. One very simple way to do this is to  
have one prefix that is shared by a bunch of NAT64s (like a well-known  
prefix). The disadvantage of that approach is that you can't use  
different prefixes to make traffic move to different NAT64s, which is  
probably something you want in a setup like yours.

You can also first move the DNS request to the right (cluster of) DNS64 
(s) by simply giving different hosts different DNS resolver addresses.  
So unlike NAT44, you're pretty free to place the boxes where you want  
rather than be forced to place them where the traffic happens to be.

I would also talk to my vendors and ask them for the ability to send  
different DNS64 replies to different hosts based on their source  
addresses. This limits the number of DNS64 boxes that you need.

> I will again

IPv6, no NAT?  :-)

> cannot turn our noses up at this problem as a minor issue.

We weren't saying that. What I was saying is that dynamically  
balancing load over different NAT64s has problems for NAT traversal  
and what Andrew was saying is that it probably won't work (if I'm not  
mistaken).

---

From: Cameron Byrne <[email-masked]>
Date: Tue, 20 Oct 2009 08:57:35 -0700

> As a general matter, DNS-based load balancing is broken.  If nothing

Can you elaborate on the scale of the brokenness ?  It is my
understanding that DNS based load-balancing is very common on the
Internet today.  From a purest point of view i guess i understand, but
my issue is a practical one and i see DNS based load balancing as a
practical and common tool today, broken or not.

> interface), but should stick to one set.

Via DHCP i could load balance out different DNS64 address which could
help me control the loading, but then my ability to spread load is in
increments of users, not flows ... and the loading and failover (back
pressure sensitivity)  is now dependent on DHCP lease times.  In
reality, this would not be a huge issue and would address the
problem.... But are we just shuffling the ugliness from one area to
another?  I still feel DNS64 is the more germane solution.
> I suspect it will be fragile in exactly the ways DNS load balancing is

I agree.  It will be implemented, better to provide guidance from the
community here than leave it to a product manager.

> wouldn't we just write "RFC 4966 considered harmful" and be done with

No.  Not in the least.  I want the DNS64 to be more like a control
plane that statelessly drives traffic to our fragile and
scale-burdened NAT64 boxes.

> inventing new ways to make DNS64 more fragile.

Agreed, this is will make DNS64 marginally more fragile, but i believe
it makes the DNS64 / NAT64 system as a whole more robust.  I am most
concerned with making the general case work at a high scale.  Anything
that needs to by-pass the NAT64 for purity / complexity  sake has an
incentive to move to  end-to-end IPv6 communication.

> A
>
> --
> Andrew Sullivan
> [email-masked]
> Shinkuro, Inc.
> _______________________________________________
> Behave mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/behave
>

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Oct 2009 12:18:36 -0400

> practical and common tool today, broken or not.

You'll get different answers depending on whom you ask, but the basic
issue is that people who get the data out of the DNS might not do what
you think they will with it.  There are a few basic problems:

    1.  Some clients cache the results of their first A record query
    and then keep using that, sometimes even after the TTL is
    expired.  

    2.  Some clients might re-query after an expired TTL even if
    they're still using a connection (for instance, because they need
    to make a different connection).  If you split them across
    different target servers, then depending on how your application
    works, you have broken state in the system,

    3.  There are unpredictable interactions with intermediate
    caches and new queries, so you can end up with inconsistent data
    in the cache (which results in resolution failures).

You're quite right, however, that we can describe lots of things we
see on the Internet as "DNS based load-balancing", and some result in
more failures than others.  Very often the load balancing stuff works
just fine for stateless protocols like http, and since that's the bulk
of the traffic any trouble it causes is all but invisible.

> another?  I still feel DNS64 is the more germane solution.

It does move ugliness around, but at least then users are presented
with a consistent view, which means that they don't get surprise
transient failures that are almost impossible to debug.  (In DNS load
balancing, since you get different answers from "the same server"
depending on when you ask, you can be all but sure that if you try to
diagnose a problem you won't be able to see it.  It's almost like the
cat box, where your mere act of trying to troubleshoot the problem
causes traffic which alters the problem to be solved.)

> scale-burdened NAT64 boxes.

And that's not the way the draft reads today, in which there is not
supposed to be any shared state between the different services:

   In general, the only shared state between the DNS64 and the IPv6/IPv4
   translator is the Pref64::/n and an optional set of static
   parameters.  The Pref64::/n and the set of static parameters must be
   configured to be the same on both; there is no communication between
   the DNS64 device and IPv6/IPv4 translator functions.

> incentive to move to  end-to-end IPv6 communication.

I have pretty grave doubts that this sort of system is going to work
in a general case at high scale.  But I'm content to be proven wrong.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Oct 2009 12:21:05 -0400

On Tue, Oct 20, 2009 at 05:49:07PM +0200, Iljitsch van Beijnum wrote:
>
> (Did we land on a conclusion on whether it would be ok to put a caching 
> DNS server behind a DNS64?)

Yes.  It will never work.  We have assured that in the latest draft by
saying that the additional section MUST NOT be altered.

> We weren't saying that. What I was saying is that dynamically balancing 
> load over different NAT64s has problems for NAT traversal and what Andrew 
> was saying is that it probably won't work (if I'm not mistaken).

Yes, that's what I'm saying.  At least for some values of "work".  If
you think that "Internet access" means "Web access" (and one of the
largest ISPs in Canada seems to think that, for instance), then maybe
it'll mostly work well enough that users won't notice the breakage.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 21 Oct 2009 09:43:59 +1300

Cameron,

Perhaps if you rephrase your requirement as "load sharing" instead
of "load balancing", you can find a scalable solution. Basically,
I would suggest that if you calculate that the maximum safe
load for a DNS64/NAT64 pair is N hosts, you need to configure
the network in such away that only N hosts will actually use
each pair.

Yes, that means that each DNS64/NAT64 pair becomes a single
point of failure for N hosts. But that's been part of the price
of NAT since NAT was invented. It also means a different
Pref64::/n for each pair, but so what?

I can't see anything but a sea of troubles in dynamic load
balancing and failover, for all the reasons given by others.

   Brian

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Oct 2009 16:49:27 -0400

> Pref64::/n for each pair, but so what?

I like that approach also because it seems not to violate the KISS
principle.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Cameron Byrne <[email-masked]>
Date: Tue, 20 Oct 2009 15:15:09 -0700

> of "load balancing", you can find a scalable solution. Basically,

Sharing or balancing both work for me.  I understand the semantics can
be important, but load sharing works plenty fine for me.

> Pref64::/n for each pair, but so what?

Yeah, using many Pref64 is no big deal.  Also, there is no reason to
have a single point of failure.  I generally pass 2 DNS servers to my
customers today.  I can do the same in the future, but in this case i
am passing them 2 DNS64s what are coupled with 2 NAT64, all
transparent to the user.  Also, most NAT44 deployments in carriers are
1+1 redundant.  Today, my NAT44 deployment is 1+1 active / standby +
load sharing using sourced / policy based routing.  Scaling linearly
is a requirement in my world.
> balancing and failover, for all the reasons given by others.

When I learned about NAT64 a few months back, i thought it was great
that the DNS-ALG / DNS64 was decoupled from the NAT64.  I assumed that
was to facilitate better end to end IPv6 communication and to assist
in the scaling of the NAT64 -- the part with the failure prone ALGs
and limited resources in session setup, packets per second ...   At
that time, I assumed there would be many NAT64 boxes to 1 DNS64 box
deployment and it made sense.  Today, I have many NAT44 boxes and 1
anycast (many servers with the same IP) IPv4 DNS deployment.   I
understand to some degree the groups concerns that it may be ugly, but
I have worked for more than one network operator ... and yes,  things
are sometimes ugly, they are robust and at the same time fragile,
especially when we have to react on the fly and make tactical
decisions about scaling.  But, personally, i think geo-DNS  and DNS
based load sharing are good things in today's internet.  I really
don't think the Internet would be where it is today (the good parts)
without making these trade-offs in favor of scaling.

I think tinkering with DHCP DNS servers may work, but ultimately
vendors that give me, the network operator, more control will win the
RFP.   Added control and "reliability" features frequently are the
headlines for vendor differentiation.  At least, when i look at my
infrastructure today, that is what i see.  Geo-load balanced DNS and
all.

Cameron

---

From: Xu Xiaohu <[email-masked]>
Date: Wed, 21 Oct 2009 10:27:56 +0800

> probably something you want in a setup like yours.

Another disadvantage of using the same prefix on different NAT64 boxes for
load-balancing is that an internal topology change of the IPv6 network will
possibly cause the packets of the same session to traverse different NAT64
boxes. In order to make the session survive from the NAT64 change, the
mapping states have to be synchronized among these NAT64 boxes. 

> addresses. This limits the number of DNS64 boxes that you need.

Besides using different prefix64s for address synthesis according to the
different DNS clients, the DNS64 server could also use different prefix64s
according to different A records in the received DNS replies.

Similarly, if the IPv6 host itself implements the IPv6 address synthesis by
using the prefix64 learnt from DNS, DHCPv6 or RA (see
draft-wing-behave-learn-prefix), the DNS and DHCP server could also provides
different prefix64s according to different clients, and the gateways(default
routers) could announces different prefix64s to different subnets. Of
course, multiple prefix64 can also be offered to the IPv6 hosts and make
them choose which prefix64 to use according to the similar rules as
mentioned above.

Xiaohu

---

From: Xu Xiaohu <[email-masked]>
Date: Wed, 21 Oct 2009 11:19:20 +0800

> products do the things described in this draft.  

Hi all,

Is it necessary to standardize the redundancy and load-balancing mechanisms
of large-scale NAT (also called CGN), including NAT64, NAT44 and even NAT46?

Xiaohu

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 21 Oct 2009 09:26:05 +0200

> than the NAT64 systems, which i think is a good assumption.

this would be one way of doing it with the considerations expressed by 
Andrew.

However, you could also play with the IGP, right?

I mean, suppose you use the same Pref64/n for all the NAT64 boxes.
Suppose now that the IPv6 representation of the IPv4 address X is 
created as Pref64:X:SUFFIX

Now, you could configure the different NAT64 boxes propagating several 
more specific routes of Pref64/n, for instance propagating 
Pref64:abcd::/n+i (for different values of abcd)

The result afaiu would be similar to the one you achieve today when you 
have multiple egress routers.
I understand that ISPs and big sites alreayd do this, so this wouldn't 
change that operational practice. Would that make sense to you?

> as simple SNMP GET sampling of the sessions that NAT64 systems are

you could do that, but i am afraid that you could end up with 
significant stability problems depending on the parameters and the 
response time, the RTT and the lifetime of the flows.
I mean this looks like a lot to routing based on congestion and we 
already had experienced stability issues with that.

I am not saying this cannot be done, i am saying that stability is a 
concern and that defining this behavious is not only defining the snmp 
extensions, but some stability analysis is granted

> system can drive load to multiple unique NAT64 systems.
AFAIU, we already have the request to support multiple prefixes in the 
dns64 draft so i think this would fit in there, so nothing else would be 
needed
for the load sharing using the igp, the main consideration is about the 
address format, which is still under discussion
For the dynamic thing, as i said, i think a stability analysis is needed.

Regards, marcelo

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 21 Oct 2009 09:39:51 +0200

> like STUN fail, and as such, should be avoided.
but you could solve that problem by making the pref64 used by the dns64 
dependent on the source address of the query, right?

I mean, in that way, you could make sure that all the DNS queries coming 
to the dns64 server from the same source IP address get the same Pref64 
bt different sources get different Pref64

This would result in per source host load sharing and woudl address this 
particular issue i think

Regards, marcelo

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 21 Oct 2009 10:23:40 +0200

> but you could solve that problem by making the pref64 used by the  
> dns64 dependent on the source address of the query, right?

Yes. But that still doesn't give you dynamic load balancing, or  
perhaps you only assign "new" sources to different NAT64s dynamically.  
(New = haven't been active for some hours or just (re)connected to the  
network.)

---

From: Xu Xiaohu <[email-masked]>
Date: Wed, 21 Oct 2009 18:04:46 +0800

> -----邮件原件-----
> 发件人: [email-masked] [mailto:[email-masked]] 代表
> marcelo bagnulo braun
> 发送时间: 2009年10月21日 15:40
> 收件人: Iljitsch van Beijnum
> 抄送: Cameron Byrne; [email-masked]
> 主题: Re: [BEHAVE] Using DNS64 to load balance to NAT64
> 

If I understood it correctly, the external address for the internal host
learnt from STUN is used by the external host to initiate communication
session. So would it cause any problem when allocating prefix according to
destinations (i.e. A record contained in DNS reply)?

Xiaohu

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 21 Oct 2009 07:22:26 -0700

> dependent on the source address of the query, right?

This, in part, is exactly what i am looking for.  This level of load
sharing would help me a great deal in delivering a solutions that
scales linearly.  I can keep adding pref64 to the one DNS64 and keep
adding NAT64 boxes as demand arises.

>
> I mean, in that way, you could make sure that all the DNS queries coming to
> the dns64 server from the same source IP address get the same Pref64 bt
> different sources get different Pref64

Excellent.

One other thing.  I am not interested in including IGP in any of this
PREF64.  I do not want temporary topology changes (link down) to swing
around NAT64 traffic when it is not needed.  I want a DNS64 to hand
out a a specific and consistent PREF64 all the time.  Perhaps a
sourced based hash that results in buckets 1, 2, 3 ... and each bucket
is a PREF64-1, -2, 3.

Also, keep in mind, the DNS64 will likely be deployed as an anycast in
many geographic locations ... so it will be subject to IGP changes...
but, i dont want the PREF64 to be dependent on IGP beyond the anycast
DNS64.  For me, PREF64 should be statically tied to a single NAT64
entity (which will be 1+1 boxes).  It's possible i did not understand
what was meant by Marcelo wrt to IGP.

> This would result in per source host load sharing and woudl address this

YES! :)

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 22 Oct 2009 09:06:16 -0700

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Cameron Byrne
> Sent: Wednesday, October 21, 2009 7:22 AM
> To: marcelo bagnulo braun
> Cc: [email-masked]
> Subject: Re: [BEHAVE] Using DNS64 to load balance to NAT64
> 
> entity (which will be 1+1 boxes). 

That is exactly the model I had in mind, as well.

One thing to note is that if NAT state is shared between several
NATs (as with draft-chen-behave-olnat or 
draft-xu-behave-nat-state-sync) it would be possible to allow
packets to egress any of the NATs that have the necessary
state, and thus allow the IGP to attract traffic to a lightly-
loaded NAT.

-d

> It's possible i did not understand

---

From: Cameron Byrne <[email-masked]>
Date: Thu, 22 Oct 2009 10:20:13 -0700

> state, and thus allow the IGP to attract traffic to a lightly-

I would be very hesitant to creep the NAT functions into the IGP or
BGP or any sort of network topology technology.  The WAN guys have
enough trouble today figuring out why flows work the way they do on
the network with IGP / BGP / MPLS / TE / VPN / multi-topology /...
They don't need another variable to look at when they are
troubleshooting shooting basic network topology issues.

This is why i am suggesting that the DNS64 have a hand in either "dumb
round robin" or "intelligent NAT64 system aware" load sharing, since
DNS64 is assigning the PREF64 to the client resolver, it is the driver
directing the load on the network / NAT64.  This is an important
point.  The DNS64 system drives the load to unique PREF64 / NAT64 that
is hosted on a specific location in the network topology.  One of the
main advantages of DNS64/NAT64 is that it is not required to be part
of shortest path routing topology between the user and the content
like NAT-PT was, thus the entire system encourages and benefits from
NAT-free end to end IPv6 communication.  The DNS64 / PREF64 provides a
very elegant detour from the shortest path string of routers and into
a NAT64 service specific / optimized node, only when NAT64 is needed.

In my NAT44 experience, there is  such a thing as a "bad state"....
protocols act in unexpected ways, which triggers unforeseen software
defects and unforeseen feature interaction that can results in
catastrophic system failure.  I have seen this several times from
several NAT / Firewall vendors where there is a 1+ 1 pair with state
sync... some SIP packet comes in the firewall with an awkward mix of
parameters, and both the Active and Standby firewall reboot because
the active wall setup the "bad" session then replicated it to the
standby box ...and now both active and standby hang and / or reboot...
Rebooting being the better scenario than just locking up.  This is the
reality of NAT from all vendors.

That said, i really don't like state sync beyond local 1+1 pairs,
network wide state sync does not fly ... think cascading failures
across the entire network.  Given how fragile these state machines
are, the fault domain needs to be small and simple. I would also not
overestimate the ability to replicate state across many systems in
small region.  Replicating state requires a lot of CPU cycles and is
very time / distance / latency sensitive.  I assume many HTTP sessions
conclude on the initiating NAT system before the session is replicated
and installed on the standby system.

Cameron

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 22 Oct 2009 13:56:07 -0400

> a NAT64 service specific / optimized node, only when NAT64 is needed.

This view is exactly how we got so much sludge from various local DNS
systems leaking out onto the Internet.  For instance, we have a _whole
system_ (the AS112 system) of anycast name servers just to sink the
traffic that used to arrive at the in-addr.arpa zone due to RFC 1918
address space allocations.  The idea that these DNS answers are not
going to leak, are not going to get out onto the wider Internet, and
so on is completely fanciful.  They will, and the only question is how
we can arrange things so that they do the least damage.

Therefore, the trickier the strategy one proposes for DNS64, the more
my alarm bells ring.  I think we want to take DNS64 for what it is --
a cute trick that allows us to make two completely different networks
talk to one another -- and accept that, as all cute tricks, it has a
number of limitations some of which will be, "Performance is rotten."
Anything that smacks of DNS tricks that already break all over the
place in the plain old DNS world (and that will break even more as
DNSSEC gets turned on) sounds to me like the sort of trouble we
shouldn't be borrowing.  I'm aware that people are going to do those
things, but if I have to give them the sort of advice that will really
help them with deployment, my advice is "don't".

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 22 Oct 2009 12:15:54 -0700

> [mailto:[email-masked]] On Behalf Of Andrew Sullivan
> Sent: Thursday, October 22, 2009 10:56 AM
> To: [email-masked]

It's important that, as best we can, the public IPv4 address stay
the same for all translations by a client.  Otherwise 
some applications will break.  FTP certainly breaks.  Other 
IP-based auth/authz systems break, too, and I am told some 
websites use cookies that encode IP addresses.  That's why 
REQ-1 is written in 
http://tools.ietf.org/html/draft-nishitani-cgn-02#section-3

If the DNS64 were to change the Prefix as a NAT64's load changed,
it would cause flows to take a different NAT64.  If that different
NAT64 is not sharing state with other NAT64s, a different public
IPv4 address would be assigned.

So, a safer approach is to load balance when the IPv6 address is
assigned, using DHCPv6 to assign a DNS64 to a host is safer, and 
have that DNS64 always use the same Prefix for its synthesized 
answers.  This is a natural thing to do, anyway -- if you have 
5 NAT64 devices you want 20% of users to go to each one.  So 
20% of your users are pointed to one DNS64, 20% to another 
DNS64, and so on.  Each DNS64 could, actually, be the same 
DNS64 box -- which means for this scheme the  DNS64 should use 
the *destination* address of the arriving DNS query (rather 
than source address) to decide which Prefix to use in the 
synthesized response.  Would that be reasonable to put into
the DNS64 spec?

-d

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 22 Oct 2009 16:10:16 -0400

> synthesized response.  Would that be reasonable to put into

This sounds like a suggestion about a section detailing different
strategies for managing multiple Pref::64/n, in case multiple such
prefixes are assigned.  This exact path was what I was wary of
travelling when the whole multicast problem came up, because it sounds
like we're going to start recommending ways to make this thing quite a
bit more complicated than what I'd hoped.  But if we really want that,
ok, let's get some text and start figuring out how to add it.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 23 Oct 2009 13:01:19 +0200

Dan Wing escribió:
>>> This is why i am suggesting that the DNS64 have a hand in 
>>>       
>> either "dumb
>>     
>>> DNS64 is assigning the PREF64 to the client resolver, it is 
>>>       
>> the driver
>>     
>>> point.  The DNS64 system drives the load to unique PREF64 / 
>>>       
>> NAT64 that
>>     
>>> is hosted on a specific location in the network topology.  
>>>       
>> One of the
>>     
>>> NAT-free end to end IPv6 communication.  The DNS64 / PREF64 
>>>       
>> provides a
>>     
>>> very elegant detour from the shortest path string of 
>>>       
>> routers and into
>>     
>>> a NAT64 service specific / optimized node, only when NAT64 
>>>       
>> is needed.
>>
> synthesized response. 
right, this seems cleaner to me as well

>  Would that be reasonable to put into
> the DNS64 spec?
>
>   
mmm, afaict this doens't change the spec at all.

I think it may make sense to document reccomended deployment practices 
(with cool tricks like this one), but maybe in a separated BCP document?

Regards, marcelo

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 23 Oct 2009 10:15:14 -0400

> (with cool tricks like this one), but maybe in a separated BCP document?

I think this is a really good idea.  Actual operation of a protocol is
quite a bit different from the definition of the protocol.  I think it
would be nice to avoid adding a lot of operational scenarios to the
DNS64 document, partly because I don't believe we completely
understand how it's really going to be used (and usable) in practice,
except that we know some things will break for sure.

Would it be acceptable to push ahead with just the definition of the
service first, and then create a later document describing how
everything might be used?

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 23 Oct 2009 07:45:05 -0700

> From: marcelo bagnulo braun [mailto:[email-masked]] 
> Sent: Friday, October 23, 2009 4:01 AM
> To: Dan Wing
> Cc: 'Andrew Sullivan'; [email-masked]

Agreed.

> I think it may make sense to document reccomended deployment 
> practices (with cool tricks like this one), but maybe in 
> a separated BCP document?

That sounds reasonable.

-d