Subject: [v6ops] comments for draft-ietf-v6ops-nat64-experience [was draft-ietf-v6ops-nat64-experience WGLC]

From: GangChen <[email-masked]>
Date: Tue, 5 Mar 2013 17:01:52 +0800

wg,

There are offline comments from Randy Bush suggest to consolidate
NAT64 statements with the principle of e2e preservation and "smart
edge & stupid core". It was presented at
http://archive.psg.com/120229.apops-v4-life-extension.pdf

Personally, I think it's worth to add some texts of those principle as
a part of NAT64 deployment considerations, since it would beneficial
to advance IPv6 deployment.

I would like to seek wg opinions on this point or futher comments
regarding to http://tools.ietf.org/html/draft-ietf-v6ops-nat64-experience.

Please take the time to review the document in order to make sure the
draft doesn't lose anything at next update.

Best Regards

Gang

---

From: "George, Wes" <[email-masked]>
Date: Wed, 6 Mar 2013 14:29:39 -0500

Apologies for not getting it out before the conclusion of WGLC, but it seems that the authors are still looking for comments before revising it, so I've reviewed this document.

Introduction, third paragraph. I think this sentence is out of place and should be removed:
"There is also the troublesome trend of access
   network providers squatting on IPv4 address space that they do not
   own."
It really doesn't tie into the rest of that paragraph, nor does it relate overmuch to the reasons for deploying NAT64. Carriers that need to support customers that have end devices which still require IPv4 connectivity are going to do so by whatever means they feel appropriate, and NAT64 doesn't solve that problem. NAT64 solves the problem of an SP that has the means to force all (or a large subset of) devices to IPv6-only but still needs a way for those users to reach IPv4-only content and services.

3.2 - this section is currently pretty weak. Are there any references to the different methods of HA in NAT64 that you could provide, either via vendor implementation or in the standards themseles? Is the assertion that most traffic is short-lived and therefore cold standby is ok backed up by any production or lab testing demonstrating the user experience for NAT64? It needs to discuss what assumptions you made about how short-lived is short-lived, how long a cold-standby failover should take before it becomes an unacceptable impact to customers, and whether there are certain types of traffic more sensitive to this problem (TCP vs UDP, streaming vs surfing, etc). It may also need to discuss true cold standby vs warm standby, because in most cases, I think what you're referring to as cold standby is more like warm standby (online at all times, but not actively exchanging all state information). This likely also ties into section 3.4, since the choice here will affect the quality of experience. I would like to see more effort to tie the two ideas together. IOW, if you are expecting that this will be a second-class service that only covers the major traffic (http, etc) then it might be ok for there to be cold standby and session resets. If you're trying to make this a first-class alternative (within the limits of the technology of course), then session resets might not be acceptable.

I would also move 3.5 so that it is immediately after 3.2. These are both talking about similar areas (scale and resiliency) and make more sense when more closely tied together, especially when bolstered with some discussion of how load balancers are used for resiliency and scale for normal use, and how that might be different for NAT64/FE use.

3.3 - there have been a couple of drafts floating around trying to build a NAT/CGN mib, whether generic or specific to NAT64. They haven't really gone anywhere. Might be useful to discuss whether the lack of a mib is making this harder, or if it doesn't matter.

3.6 - I'm not following the explanation of the MTU issues here. It seems to assert that IPv6 can't deal with packets smaller than 1280, which really isn't true, it just requires end host to end host fragmentation. Therefore, IMO part of a NAT64 device's job is going to be to manage the PTB messages and MTU discovery between the protocols and act as the IPv6 destination end host so that it can facilitate any required IPv6 endhost-to-endhost fragmentation if the required MTU is below 1280 - it basically has to detect the outgoing MTU and enforce it back to its end host. It's a lot of state to maintain probably, but doable. This is also a corner case, because IPv4 allows almost all packets (except those with the DF bit set) to be fragmented by middleboxes if their MTU is smaller than the offered packet, making it largely transparent to the end hosts. The IPv6 host can send whatever size packet they want, and the IPv4 side will just fragment as necessary unless the NAT64 box is doing something dumb like setting DF on the outgoing IPv4 packets it is generating.
Is this meant to discuss the situation where a NAT64 box receives IPv4 packets smaller than 1280 and then has to send them to the IPv6 host?

4 - I'm not certain figure 2 is helpful in its current form. I had to look 3 times before I figured out what was even different from figure 1, and the addition of one word doesn't really require a new diagram.
Honestly, this entire section repeats a lot of the content from the previous sections, and I think the draft would be much tighter and readable if you simply integrated the little bit of additional information into the previous sections, either as subsections or just as part of the overall discussion. I don't think it adds much value as a standalone section. Generally, the drafts that you reference here do a better job of discussing the use of a loadbalancer as a method to provide IPv6 to a mostly IPv4-only backend and vice versa.

Regarding Randy's comments on e2e/smart edge/stupid core and the interaction with NAT64, I think they're mostly incompatible. While NAT64 makes running an IPv6-only network more possible while we wait for ubiquitous IPv6 deployment on the content/service side, it's still fundamentally a NAT, a stateful box in the middle that interferes with end to end communications and may require ALGs for full function. It also still requires investment in those boxes in the middle, which means that you're sort of stuck with them until they depreciate, unless you can repurpose them for some other use. The only thing you could do here is make the point that in order to keep the intelligence at the edge, NAT64/DNS64 implementations should be pushed as far down into the network as possible. But even then there are other tradeoffs on placement to consider, such as the scale of an individual NAT64 box or the number of devices that would be required by a more decentralized placement, or network design and topology (as you have alluded to with the mobile examples).

By the ruler that Randy was proposing in his slides, this is less bad because at least it sets up a network to be actually running IPv6 instead of extending IPv4, and only does IPv4 to compensate for those who haven't deployed it yet. But it's still limited to places that are unencumbered by a need to continue supporting legacy IPv4 devices, which limits its applicability in the Access Provider community.

Nits: 3.1 has a number of spelling errors

Wes George

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of GangChen
> Sent: Tuesday, March 05, 2013 4:02 AM
> To: v6ops
> Subject: [v6ops] comments for draft-ietf-v6ops-nat64-experience [was
> draft-ietf-v6ops-nat64-experience WGLC]
>
> wg,
>
> NAT64 statements with the principle of e2e preservation and "smart edge
> & stupid core". It was presented at http://archive.psg.com/120229.apops-
> v4-life-extension.pdf
>
> Personally, I think it's worth to add some texts of those principle as a
> part of NAT64 deployment considerations, since it would beneficial to
> advance IPv6 deployment.
>
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: GangChen <[email-masked]>
Date: Sat, 9 Mar 2013 07:32:37 +0800

Hi Wes,

Many thanks for the review and detailed comments.
My reply is inline.

2013/3/7, George, Wes <[email-masked]>:
> Apologies for not getting it out before the conclusion of WGLC, but it seems
> that the authors are still looking for comments before revising it, so I've
> reviewed this document.
>
> It really doesn't tie into the rest of that paragraph, nor does it relate
> overmuch to the reasons for deploying NAT64. Carriers that need to support
> customers that have end devices which still require IPv4 connectivity are
> going to do so by whatever means they feel appropriate, and NAT64 doesn't
> solve that problem. NAT64 solves the problem of an SP that has the means to
> force all (or a large subset of) devices to IPv6-only but still needs a way
> for those users to reach IPv4-only content and services.

I won't object to remove it. That seems superfluous for a clear
description. PS: The original intent of the sentence is to describe
the incentive of enabling access network with IPv6. Doing NAT64 will
bridge those IPv6 access network with dominated IPv4 services.

> 3.2 - this section is currently pretty weak. Are there any references to the
> different methods of HA in NAT64 that you could provide, either via vendor
> implementation or in the standards themseles?

One early work was documented at
http://tools.ietf.org/html/draft-xu-behave-stateful-nat-standby-06, in
which three modes were described. I guess the draft could revisit
those considerations to consolidate the statement.

> Is the assertion that most
> traffic is short-lived and therefore cold standby is ok backed up by any
> production or lab testing demonstrating the user experience for NAT64? It

There is statistic of service traffic in a mobile network could be the
evidence. Most traffic (almost 94% of amount of traffic) is accounted
by Http, WAP browsing, which contribute this assertion.

> needs to discuss what assumptions you made about how short-lived is
> short-lived, how long a cold-standby failover should take before it becomes
> an unacceptable impact to customers, and whether there are certain types of
> traffic more sensitive to this problem (TCP vs UDP, streaming vs surfing,

Good suggestion. More concrete data backing up statement would be
added to verify the assumptions at next version.

> etc). It may also need to discuss true cold standby vs warm standby, because
> in most cases, I think what you're referring to as cold standby is more like
> warm standby (online at all times, but not actively exchanging all state
> information). This likely also ties into section 3.4, since the choice here
> will affect the quality of experience. I would like to see more effort to
> tie the two ideas together. IOW, if you are expecting that this will be a
> second-class service that only covers the major traffic (http, etc) then it
> might be ok for there to be cold standby and session resets. If you're
> trying to make this a first-class alternative (within the limits of the
> technology of course), then session resets might not be acceptable.

I guess it's worth to do some tests regarding those ties. Current
draft didn't cover because the lack of evaluation on user experiences
impacts of different modes. Let's try to start with the testing.

> I would also move 3.5 so that it is immediately after 3.2. These are both
> talking about similar areas (scale and resiliency) and make more sense when
> more closely tied together, especially when bolstered with some discussion
> of how load balancers are used for resiliency and scale for normal use, and
> how that might be different for NAT64/FE use.

Will a change at next version.

> 3.3 - there have been a couple of drafts floating around trying to build a
> NAT/CGN mib, whether generic or specific to NAT64. They haven't really gone
> anywhere. Might be useful to discuss whether the lack of a mib is making
> this harder, or if it doesn't matter.

I would prefer to discuss the mib issues at a separated section other
than section 3.3 "Traceability" because that is different topics.
Current NAT64 equipment didn't offer MIB AFAIK.

> 3.6 - I'm not following the explanation of the MTU issues here. It seems to
> assert that IPv6 can't deal with packets smaller than 1280, which really
> isn't true, it just requires end host to end host fragmentation. Therefore,
> IMO part of a NAT64 device's job is going to be to manage the PTB messages
> and MTU discovery between the protocols and act as the IPv6 destination end
> host so that it can facilitate any required IPv6 endhost-to-endhost
> fragmentation if the required MTU is below 1280 - it basically has to detect
> the outgoing MTU and enforce it back to its end host. It's a lot of state to
> maintain probably, but doable. This is also a corner case, because IPv4
> allows almost all packets (except those with the DF bit set) to be
> fragmented by middleboxes if their MTU is smaller than the offered packet,
> making it largely transparent to the end hosts. The IPv6 host can send
> whatever size packet they want, and the IPv4 side will just fragment as
> necessary unless the NAT64 box is doing something dumb like setting DF on
> the outgoing IPv4 packets it is generating.
> Is this meant to discuss the situation where a NAT64 box receives IPv4
> packets smaller than 1280 and then has to send them to the IPv6 host?

The draft discuss the case where a NAT64 receives IPv6 packets with
fragmentation header in which M=0 & offset=0(that is caused if IPv6
nodes receive PTB<1280 via NAT64 box). Those packets likely impact
other fragments already queued with the same set of {IPv6 Source
Address, IPv6 Destination Address, Fragment Identification}. If the
NAT64 box is compliant with RFC5722, there is risk that all the
fragments have to be dropped. The draft recommends doing
I-D.ietf-6man-ipv6-atomic-fragments on NAT64-CGN to exclude the risk.

> 4 - I'm not certain figure 2 is helpful in its current form. I had to look 3
> times before I figured out what was even different from figure 1, and the
> addition of one word doesn't really require a new diagram.
> Honestly, this entire section repeats a lot of the content from the previous
> sections, and I think the draft would be much tighter and readable if you
> simply integrated the little bit of additional information into the previous
> sections, either as subsections or just as part of the overall discussion. I
> don't think it adds much value as a standalone section. Generally, the
> drafts that you reference here do a better job of discussing the use of a
> loadbalancer as a method to provide IPv6 to a mostly IPv4-only backend and
> vice versa.

Will try to restruct the contents at next update. Hopefully, it's more
clearer than current form if the draft integrated NAT-FE&CGN in one
section.

> Regarding Randy's comments on e2e/smart edge/stupid core and the interaction
> with NAT64, I think they're mostly incompatible. While NAT64 makes running
> an IPv6-only network more possible while we wait for ubiquitous IPv6
> deployment on the content/service side, it's still fundamentally a NAT, a
> stateful box in the middle that interferes with end to end communications
> and may require ALGs for full function. It also still requires investment in
> those boxes in the middle, which means that you're sort of stuck with them
> until they depreciate, unless you can repurpose them for some other use. The
> only thing you could do here is make the point that in order to keep the
> intelligence at the edge, NAT64/DNS64 implementations should be pushed as
> far down into the network as possible. But even then there are other
> tradeoffs on placement to consider, such as the scale of an individual NAT64
> box or the number of devices that would be required by a more decentralized
> placement, or network design and topology (as you have alluded to with the
> mobile examples).

I have a different understanding because Randy's proposal is likely
applied to stateless NAT64 in the core. Those principles are
incompatible with stateful NAT64 consideration. That reminds me a
point: should the draft cover the case of stateless NAT64?
FWIW, both stateful&stateless NAT64 has customers in real world. They
have different incentives. A tentative showcase is
stateful NAT64-CGN: NAT64 on Core router or in a mobile network(464xlat)
stateful NAT64-FE: an HTTP proxy for IPv4 backend in DC
stateless NAT64-CGN: MAP/4rd/A+P kinds of solutions
stateless NAT64-FE: I-D.anderson-siit-dc

Is it of value to discuss both in the I-D.ietf-v6ops-nat64-experience?

> By the ruler that Randy was proposing in his slides, this is less bad
> because at least it sets up a network to be actually running IPv6 instead of
> extending IPv4, and only does IPv4 to compensate for those who haven't
> deployed it yet. But it's still limited to places that are unencumbered by a
> need to continue supporting legacy IPv4 devices, which limits its
> applicability in the Access Provider community.
>

Will fix at next update

Best Regards

Gang

> This E-mail and any of its attachments may contain Time Warner Cable
> proprietary information, which is privileged, confidential, or subject to
> copyright belonging to Time Warner Cable. This E-mail is intended solely for
> the use of the individual or entity to which it is addressed. If you are not
> the intended recipient of this E-mail, you are hereby notified that any
> dissemination, distribution, copying, or action taken in relation to the
> contents of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify the
> sender immediately and permanently delete the original and any copy of this
> E-mail and any printout.
>

---

From: Philip Matthews <[email-masked]>
Date: Mon, 11 Mar 2013 17:34:32 -0400

I agree with Wes's comments (below), so I won't repeat them.  I will simply add a few additional comments of my own:

3.1  This section makes the comment
            "It is advantageous from the vantage-point of troubleshooting and traffic engineering to carry the IPv6 traffic natively for as long as possible within an access network and translate only near the edge."
The question of where to place the NAT64  (near the subscriber, near peering routers, somewhere in the middle) is an interesting one.  I would like to see a lot more discussion on this topic. The document makes some brief comments on why placing it near peering routers might be useful, but I am sure more can be said. I can also see cons for placing it there: as one example, this would make geo-location based on IPv4 addresses rather inaccurate. Furthermore, if one has multiple NAT64 boxes located near different peering routers, then might mean some flows went through one NAT64 box, while others went though a different NAT64 box -- not a good situation.   

3.3  I don't understand what is meant by "online" vs. "offline" in this section.   It would be good to properly define these, or give a reference to a document that defines them.
Furthermore, the bullet points in this section seem somewhat random   --- if there is some cohesive picture which the authors are trying to present, then I don't see it.

- Philip

On 2013-03-06, at 14:29 , George, Wes wrote:

> 3.2 - this section is currently pretty weak. Are there any references to the different methods of HA in NAT64 that you could provide, either via vendor implementation or in the standards themseles? Is the assertion that most traffic is short-lived and therefore cold standby is ok backed up by any production or lab testing demonstrating the user experience for NAT64? It needs to discuss what assumptions you made about how short-lived is short-lived, how long a cold-standby failover should take before it becomes an unacceptable impact to customers, and whether there are certain types of traffic more sensitive to this problem (TCP vs UDP, streaming vs surfing, etc). It may also need to discuss true cold standby vs warm standby, because in most cases, I think what you're referring to as cold standby is more like warm standby (online at all times, but not actively exchanging all state information). This likely also ties into section 3.4, since the choice here will affect the quali
> ty of experience. I would like to see more effort to tie the two ideas together. IOW, if you are expecting that this will be a second-class service that only covers the major traffic (http, etc) then it might be ok for there to be cold standby and session resets. If you're trying to make this a first-class alternative (within the limits of the technology of course), then session resets might not be acceptable.
> 
> 3.6 - I'm not following the explanation of the MTU issues here. It seems to assert that IPv6 can't deal with packets smaller than 1280, which really isn't true, it just requires end host to end host fragmentation. Therefore, IMO part of a NAT64 device's job is going to be to manage the PTB messages and MTU discovery between the protocols and act as the IPv6 destination end host so that it can facilitate any required IPv6 endhost-to-endhost fragmentation if the required MTU is below 1280 - it basically has to detect the outgoing MTU and enforce it back to its end host. It's a lot of state to maintain probably, but doable. This is also a corner case, because IPv4 allows almost all packets (except those with the DF bit set) to be fragmented by middleboxes if their MTU is smaller than the offered packet, making it largely transparent to the end hosts. The IPv6 host can send whatever size packet they want, and the IPv4 side will just fragment as necessary unless the NAT64 box is d
> oing something dumb like setting DF on the outgoing IPv4 packets it is generating.
> Regarding Randy's comments on e2e/smart edge/stupid core and the interaction with NAT64, I think they're mostly incompatible. While NAT64 makes running an IPv6-only network more possible while we wait for ubiquitous IPv6 deployment on the content/service side, it's still fundamentally a NAT, a stateful box in the middle that interferes with end to end communications and may require ALGs for full function. It also still requires investment in those boxes in the middle, which means that you're sort of stuck with them until they depreciate, unless you can repurpose them for some other use. The only thing you could do here is make the point that in order to keep the intelligence at the edge, NAT64/DNS64 implementations should be pushed as far down into the network as possible. But even then there are other tradeoffs on placement to consider, such as the scale of an individual NAT64 box or the number of devices that would be required by a more decentralized placement, or network d
> esign and topology (as you have alluded to with the mobile examples).
>

---

From: GangChen <[email-masked]>
Date: Thu, 14 Mar 2013 07:13:17 +0800

Hi Philip,

2013/3/12, Philip Matthews <[email-masked]>:
>             "It is advantageous from the vantage-point of troubleshooting
> and traffic engineering to carry the IPv6 traffic natively for as long as
> possible within an access network and translate only near the edge."
> The question of where to place the NAT64  (near the subscriber, near peering
> routers, somewhere in the middle) is an interesting one.  I would like to
> see a lot more discussion on this topic. The document makes some brief
> comments on why placing it near peering routers might be useful, but I am
> sure more can be said. I can also see cons for placing it there: as one
> example, this would make geo-location based on IPv4 addresses rather
> inaccurate.

Will add more discussions on the point. Regarding the geo-location, I
guess IPv6 address is more preferable than converted IPv4 address.
IPv4 will loss geographical semantic in any case if NAT is on the
path.

> Furthermore, if one has multiple NAT64 boxes located near
> different peering routers, then might mean some flows went through one NAT64
> box, while others went though a different NAT64 box -- not a good situation.

Not sure if your concern is on asymmetric routing? Otherwise, those
you described seem a usual practice of load balance.

>
> 3.3  I don't understand what is meant by "online" vs. "offline" in this
> section.   It would be good to properly define these, or give a reference to
> a document that defines them.
> Furthermore, the bullet points in this section seem somewhat random   --- if
> there is some cohesive picture which the authors are trying to present, then
> I don't see it.

"online" vs. "offline" is differentiated by various applications
needs. We will fix that by adding proper definitions and logic in next
version.

Best Regards

Gang