Subject: [v6ops] draft-ietf-v6ops-464xlat-optimization-02 general concerns

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Fri, 17 Jul 2020 11:40:12 +0200

Hi Erik,

I think it will be good to respond in different threads to simplify the discussion and try to find appropriate text for each of the issues that you mention.

This one is on your general concerns, below in-line.

If any of the general concerns is further discussed in the rest of your email, I will try to tackle it also in the follow-up emails, so we can concentrate in every possible issue in the specific email to avoid multiple parallel threads on the same issue.

Tks a lot!

El 15/7/20 7:54, "v6ops en nombre de Erik Nygren" <[email-masked] en nombre de [email-masked]> escribió:

On Thu, Jul 9, 2020 at 11:58 PM Jen Linkova <[email-masked]> wrote:

OK, I"ve read the document and before I finish the write up I'd like
to discuss a number of comments I have.
My apologies for not raising them during the WGLC - I was not paying
attention, got busy with work...

Prodded by Jen's response here I also re-read draft-ietf-v6ops-464xlat-optimization-02.
(My apologies also for not raising these during WGLC -- similar excuses to Jen...  ;-)

While this continues to close corner-cases, I'm still concerned
that there is significant possibility for breakage in a way that
will result in content that is now dual-stacked getting switched
to be IPv4-only.

It would be good to add some examples as to why
some of the invalidation logic needs to actually work
(and the risk of corner cases where it doesn't).
To that end, it may make sense to also have some more
use of normative rfc2119 language for clearly spelling out behaviors
where there are sharp edges.

[Jordi] Often when an informational document goes to the IESG they complain about the use of normative language. One open question is, if we advance RFC6877 to standards then, we could consider that this optimization should be also in the standards track. Anyway, at the time being I will keep it as informational, but I will add normative language to make everything very clear and more examples as needed in every section.

Some example cases of breakage or significant risk of breakage:
* A site is IPv4-only (breaks for or denies all IPv6 users) but shares an A record with a dual-stack site.  If this isn't detected properly, it will break users who get sent over IPv6 to the IPv4-only site.

[Jordi] My goal has been to ensure that the optimization doesn’t break any case, and especially when the site only offers A records. Unless I’m missing something, I think this is perfectly resolved in 5.2.2, because the EAMT entry will be only created if the site is only IPv4-only.

* some CDNs use TLS SNI for demuxing IPv4 traffic but the IPv6 address for demuxing IPv6 traffic (both to determine which cert to serve).  This means that the IPv4:IPv6 associations are one to many and using the wrong one will result in cert errors.

[Jordi] This was discussed in a previous version (I recall it was after the presentation in Prague) and resolved by incorporating the FQDN in the EAMT entry and a section 5.2.4, which forces the forwarding path via a stateful NAT44.

* Some CDNs may use cluster A for IPv4 and cluster B for IPv6 for one customer/tenant, but then cluster A for IPv4 and cluster C for a different customer/tenant.  An stale or invalid association will result in going to the wrong cluster which may not be able to serve the request properly.

[Jordi] Same as the previous one. Because the EAMT entry is created by every CE, the association between a given IPv4 and IPv6 FQDN/addresses is done the same way as if the same CE is actually dual-stack instead of IPv6-only. If I got it correctly, in a CDN, any change in terms of what cluster is serving what, depends on DNS TTLs and maybe routing, right? If that’s the case, routing is transparent because the entries are created in each CE, and to the CDN they appear as a dual-stack eyeball. Similarly, we rely in the CDN provided TTLs.

** if the user changes DNSs, unexpected consequences may happen, but there is no difference vs the case when there is no optimization. Any change in the DNSs done by users, may break thinks (for example, CDN contents are server from another cluster). Nevertheless, usually users change DNS in end-hosts (Windows, Linux, MacOS, etc.), it is not so common to change those in the CE. Is even less common to change them in the smartTV or STB, because in many cases they don’t allow it. This is described in 5.2.10. If needed I can improve that section.

While the recent revisions help some, there are still a bunch of cases
where things could still break.  I fear that when they do break it's
not something that content providers or CDNs or their customers will
debug and the result will be for content to get switched back
to IPv4-only (which is not what we want happening, but will
appear to "fix" issues here).

[Jordi] Fully agree here, we need to make sure that we don’t break anything.

I think there may also be some significant security issues not called out
that are independent of the DNS spoofing issue.

[Jordi] I will move this to independent threads.

Another area not discussed is clients caching DNS lookups well past
their TTL.  This is fairly common in some clients (eg, some Java versions)
which could result in entries getting expired from the EAMT but then used
with a new association that should be invalid but wasn't detected as such
as the first client is caching well past the expiry..

[Jordi] I’m confused here.

You mean a client ignoring DNS TTLs ? This could actually be a problem without the optimization. If the CDN is changing the IPv4 or IPv6 addresses, but the client is still using them, the content will be unreachable or something different served, right? So, the optimization is not the problem here it is a broken client already …

Note that if the EAMT expired and the client tries to use it again, the EAMT entry will be re-created with the new (correct) information from the CDN DNSs, so we may be potentially solving the broken client problem. If the EAMT, for whatever reason is not recreated, the optimization will not be used, so the client will be still doing the “correct” thing according to its expected behavior (non-optimized NAT46+NAT64) – whether it is a broken behavior or not.

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.

---

From: Philip Homburg <[email-masked]>
Date: Fri, 17 Jul 2020 12:14:02 +0200

>    * Some CDNs may use cluster A for IPv4 and cluster B for IPv6
>    for one customer/tenant, but then cluster A for IPv4 and cluster
>    C for a different customer/tenant.  An stale or invalid association
>    will result in going to the wrong cluster which may not be able
>    to serve the request properly.
> 
>    [Jordi] Same as the previous one. Because the EAMT entry is
>    created by every CE, the association between a given IPv4 and
>    IPv6 FQDN/addresses is done the same way as if the same CE is
>    actually dual-stack instead of IPv6-only. If I got it correctly,
>    in a CDN, any change in terms of what cluster is serving what,
>    depends on DNS TTLs and maybe routing, right? If that's the
>    case, routing is transparent because the entries are created in
>    each CE, and to the CDN they appear as a dual-stack eyeball.
>    Similarly, we rely in the CDN provided TTLs.

I don't understand this.

Suppose an IPv4-only client looks up a.example.com and b.example.com (in
parallel). Both return 192.0.2.1

However, the IPv6 address for a.example.com is 2001:DB8::1 and the one for
b.example.com is 2001:DB8::2

Now the client connects twice to 192.0.2.1 to set up two tls connections, one for
a.example.com and one for b.example.com.

How does the CPE know that it needs to connect one tcp connection to 
2001:DB8::1 and the other to 2001:DB8::2?

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Fri, 17 Jul 2020 12:20:13 +0200

Hi Philip,

Because the EAMT contains the FQDN so to avoid creating a trouble, in this case the EAMT entry is set to invalid, and not used. This is described in 5.2.6 with a similar example.

    I don't understand this.

---

From: Philip Homburg <[email-masked]>
Date: Fri, 17 Jul 2020 12:37:51 +0200

In your letter dated Fri, 17 Jul 2020 12:20:13 +0200 you wrote:
>Because the EAMT contains the FQDN so to avoid creating a trouble, in this case 
>the EAMT entry is set to invalid, and not used. This is described in 5.2.6 with 
>a similar example.

Can you explain what happens if both A queries and the first AAAA query is
served from a local resolver's cache and then the second AAAA query is delayed?

I guess Section 5.2.2 can be made explicit that the CPE must not return the
answer to an A query until an answer to the AAAA query has been received.

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Fri, 17 Jul 2020 13:03:03 +0200

[Jordi] If the 2nd AAAA query is delayed past the "Resolution Delay", the EAMT entry will not be created (this time, it may happen later), so the optimization will not be performed for this IPv4 destination. See section 5.2.3.

[Jordi] Good point. May be:

"The CE DNS proxy MUST NOT return the answer to an A query until either an answer to the AAAA query has been received or the Resolution Delay has passed"

If you think this resolves it, I may need to move the text relevant to "what is the resolution delay" to 5.2.2, to make it clearer.

This may be in the actual 5.2.2 or a new section after that to be more explicit, something like "CE DNS proxy responses"

---

From: Philip Homburg <[email-masked]>
Date: Fri, 17 Jul 2020 17:05:08 +0200

>"The CE DNS proxy MUST NOT return the answer to an A query until either an answe
>r to the AAAA query has been received or the Resolution Delay has passed"

Yes, I think this is important. Otherwise small differences in DNS timing
may cause hard to diagnose bugs.

Another suggestion is that if the AAAA query either gets a timeout or SERVFAIL
then any existing EAMT entry for the corresponding IPv4 address should be
set to invalid.

Then there is DNSSEC. For example, the AAAA lookup MUST/SHOULD/MAY be
verified using DNSSEC local validation. (Or, the AD bit of the reply could
be used instead). A reply that fails DNSSEC validation MUST/SHOULD/MAY not
result in the creation of an EAMT entry (and MUST/SHOULD/MAY result in
marking an exiting entry invalid)

One question that doesn't seem answer, what if the upstream resolver is
doing DNS64. In that case an AAAA query would get an answer for an IPv4-only
service. Maybe it doesn't matter, because the mapping will be the same.
Otherwise setting the CD flag in the query may help to get only 'real' 
AAAA records.

---

From: Erik Nygren <[email-masked]>
Date: Fri, 17 Jul 2020 11:57:32 -0400

At the high-level, not causing problems relies heavily on assumptions about
client behavior,
both for STBs/SmartTVs as well as for other devices in the home.  If the
EAMT entries
also contained the internal client IPv4 address then it might make it so
that
at least devices couldn't interfere with each other, but absent that there
is also
plenty of possibility for wacky interactions.  The more we look for
corner-cases
that will cause production problems, the more I suspect we'll find.
On some of the assumptions:

* We know clients cache DNS lookups for some amount of time past their TTL.
   (ie, a client will do a "gethostbyname" and then just keep using the
result
   for multiple connections).  CDNs have to mostly deal with this (eg, wait
a few
  hours after removing an IP from being handed out before assuming the rate
  of new connections drops to an acceptable level).  I suspect some of the
STBs/SmartTVs
  may be particularly bad about this (eg, within a viewing session).
  This generally works fine today and no one notices.  But if entries drop
  out of the EAMT cache after 20 seconds but clients keep using them
  for 15 minutes, then it is likely another lookup for another hostname
  (perhaps from another device) would have created another EAMT entry
  that would have marked the first invalid but doesn't see it due to the
early expiry.
  Keeping EAMT entries in a "stale" state for some amount of time following
TTL expiry
  (an hour or two?) where they can't be used but can be transitioned to
invalid
  and also kept in invalid for an hour or two would help.  The question is
whether
  this is too much state and if it would help enough.

* The problem with the "user changes DNS" is that it is not just the user.
   There are plenty of devices in the field that do this today, including
   SmartTV/STB platforms but also desktops/laptops.
   In-particular, Chromecast-based devices (a major STB platform)
   use their own resolver of 8.8.8.8 and 8.8.4.4.
   It would not surprise me (but I haven't tested) that Chromecast does
   synchronous A/AAAA lookups in a way that may break the "IPv4-only
detection".
   Apple has also announced that starting this Fall, apps will be able
   specify a DoH server to use (and it's possible this might apply to
AppleTV as well?).
   Chrome and Firefox also now use DoH servers outside of the home
   in a bunch of places.  The whole DoH/ADD situation has gotten very
   messy in the past year.  If one dual-stacked device in the home using DoH
   get requests for IPv4-only sites hijacked and sent to an IPv6 address,
   that's going to be a real problem.    Keying the EAMT entry by client IP
might
   help, but won't help in the iOS/OSX case where apps can specify
   their own DoH server.

More comments inline below on topics not covered above...

On Fri, Jul 17, 2020 at 5:40 AM JORDI PALET MARTINEZ <
[email-masked]> wrote:

> [Jordi] My goal has been to ensure that the optimization doesn’t break any
> case, and especially when the site only offers A records. Unless I’m
> missing something, I think this is perfectly resolved in 5.2.2, because the
> EAMT entry will be only created if the site is only IPv4-only.
>

[Erik]  Ah, I think the invariant here is that DNS64 will always guarantee
that *everything* has a AAAA record
available to the CPE resolver.  If there are ever cases where no AAAA is
returned (eg, where the CLAT would get used)
then problems can show up because "IPv4-only" (no AAAA record) entries
won't be validated against the EAMT table
and won't cause EAMT entries to get created.  It may be that we should be
creating "invalid" EAMT entries
for {IPv4,FQDN} pairs where no AAAA is returned.  Also, we should be
invalidating EAMT entries
when an IPv4 address in an EAMT entry is returned for a different FQDN.

> * some CDNs use TLS SNI for demuxing IPv4 traffic but the IPv6 address for
> demuxing IPv6 traffic (both to determine which cert to serve).  This
> means that the IPv4:IPv6 associations are one to many and using the wrong
> one will result in cert errors.
>
>
>
> [Jordi] This was discussed in a previous version (I recall it was after
> the presentation in Prague) and resolved by incorporating the FQDN in the
> EAMT entry and a section 5.2.4, which forces the forwarding path via a
> stateful NAT44.
>

> * Some CDNs may use cluster A for IPv4 and cluster B for IPv6 for one
> customer/tenant, but then cluster A for IPv4 and cluster C for a different
> customer/tenant.  An stale or invalid association will result in going to
> the wrong cluster which may not be able to serve the request properly.
>
> [Jordi] Same as the previous one. Because the EAMT entry is created by
> every CE, the association between a given IPv4 and IPv6 FQDN/addresses is
> done the same way as if the same CE is actually dual-stack instead of
> IPv6-only. If I got it correctly, in a CDN, any change in terms of what
> cluster is serving what, depends on DNS TTLs and maybe routing, right? If
> that’s the case, routing is transparent because the entries are created in
> each CE, and to the CDN they appear as a dual-stack eyeball. Similarly, we
> rely in the CDN provided TTLs.
>

[Erik]  My question for both of these is if we should provide some context
or examples in the doc
to explain the invariants and things that must be tested for?  While these
conversations happened on-list,
having summaries of the risks (and what is being relied on to mitigate
them) seems like it may be valuable
in an Informational draft.

> You mean a client ignoring DNS TTLs ? This could actually be a problem
> without the optimization. If the CDN is changing the IPv4 or IPv6
> addresses, but the client is still using them, the content will be
> unreachable or something different served, right? So, the optimization is
> not the problem here it is a broken client already …
>
>
>
> Note that if the EAMT expired and the client tries to use it again, the
> EAMT entry will be re-created with the new (correct) information from the
> CDN DNSs, so we may be potentially solving the broken client problem. If
> the EAMT, for whatever reason is not recreated, the optimization will not
> be used, so the client will be still doing the “correct” thing according to
> its expected behavior (non-optimized NAT46+NAT64) – whether it is a broken
> behavior or not.
>

The case is:

1) a.example.com (TTL 20s) is resolved but cached by App A for 30 minutes
2) Five minutes later, b.example.com is resolved (TTL 600s) is resolved an
EAMT entry is created
   (but shares an IPv4 address with a.example.com but a different IPv6
address)
3) Two minutes later, the IPv4 address from a.example.com gets used (past
the DNS TTL)
     and will now use the IPv6 EAMT entry for b.example.com (which breaks).

Does this help clarify?  While you could argue this client is 'broken",
anecdotal experience
is that this is a very common behavior for some client software that
servers already
need to work around by padding operational actions for an hour or two past
the DNS TTL.

     Erik

---

From: Erik Kline <[email-masked]>
Date: Fri, 17 Jul 2020 17:36:33 -0700

> This may be in the actual 5.2.2 or a new section after that to be more explicit, something like "CE DNS proxy responses"

I don't think waiting helps anything.  If there's packet loss and the
2nd AAAA never comes back, the client's TCP SYN to 192.0.2.1 will be
sent to the wrong IPv6 host.

I think this is hopelessly broken.

---

From: Philip Homburg <[email-masked]>
Date: Sat, 18 Jul 2020 12:00:45 +0200

>sent to the wrong IPv6 host.

That can be fixed by deleting the entry upon timeout.

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 19 Jul 2020 12:57:31 +0200

Actually, it is much simpler than that.

Because we are looking for EAMT entries which also contain the FQDN, there will be no match, so no optimization performed if the AAAA response doesn't come timely.

    _______________________________________________
    v6ops mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/v6ops

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 19 Jul 2020 16:18:33 +0200

Hi Philip,

[Jordi] Yes both of those are also very good points. Maybe, we can summarize all those changes and some of my previous responses to Erik and Eduard in the list with the text (tentative) that I'm adding at the end of this email.

[Jordi] This is already resolved by NOT creating an EAMT entry if only a synthetized AAAA is received. Section 5.2.2:
" Note that the NAT46/CLAT MUST already
   know the WKP or NSP being used in that network.  If the response
   contains at least one IPv6 address not using the WKP/NSP, it means
   that the destination is IPv6-enabled (because at least one of the
   IPv6 addresses is not synthesized).  This means that it is possible
   for the NAT46/CLAT, to create an Explicit Address Mapping"

[Jordi] I'm still trying to tidy up all the text according to other inputs and respond to them. Too soon to send a new version, but close ...

Tentative changes:

5.2.2.  Detection of IPv4-only hosts

   The goal is to ensure that only IPv4-only hosts are optimized.
   Towards that, the CE MUST use ARP and ND (and their relevant caches,
   if the information of a hosts has been already learnt) each time a
   new host starts a connection.  If it is not possible to bind the same
   MAC address to both an IPv4 and IPv6 address, then the host is not
   IPv4-only (it may be IPv6-only or dual-stack), and MUST NOT be
   optimized.

   The CE MUST maintain a table of IPv4-only hosts and ensure that if
   any IPv4-only hosts become IPv6-enabled, it is properly handled.

   This mechanism to detect IPv4-only hosts has two drawbacks:

   1.  If a subscriber has intermediate dual-stack routers in between
       the IPv4-only host and the NAT46/CLAT, the IPv4-only hosts will
       be detected as dual-stack, so no optimization will be performed.
       This is not the most common scenario, as typically the devices
       that are more relevant to the optimization (in terms of those
       that generate more IPv4-only traffic) are directly attached to
       the CE, or in bridged interfaces.

   2.  If a host is dual-stack, but has some IPv4-only applications,
       because the host will be detected as dual-stack, none of the
       applications will be optimized.  This is a good trade-off,
       considering the most important traffic to optimize is typically
       coming from real IPv4-only devices such as old SmartTVs/STBs.
       Furthermore, this avoid breaking other mechanisms present only in
       dual-stack hosts, such as Happy Eyeballs [RFC8305] and simplifies
       troubleshooting.

   It needs to be remarked that, if the detection of the IPv4-only host
   is done incorrectly (either not detecting it or by a false
   detection), the goal is that no harm is caused.  In the worst case,
   optimization MUST NOT be performed.

5.2.3.  Detection of IPv6-enabled service

   In the case of an IPv4-only detected host, the DNS proxy/stub
   resolver MUST actually perform an additional AAAA query, unless the
   information is already present in the Additional Section, as per
   Section 3 of [RFC3596].

   Note that the NAT46/CLAT MUST already know the WKP or NSP being used
   in that network.  If the response contains at least one IPv6 address
   not using the WKP/NSP, it means that the destination is IPv6-enabled
   (because at least one of the IPv6 addresses is not synthesized).
   This means that it is possible for the NAT46/CLAT, to create an
   Explicit Address Mapping ([RFC7757]).

5.2.4.  CE DNS proxy responses

   In the case of an IPv4-only detected host, the CE DNS proxy MUST only
   return the answer to an A query once any of the following happens:

   1.  An answer to the AAAA query has been received.

   2.  A SERVFAIL has been received.

   3.  The "Resolution Delay" has passed.

   The Resolution Delay MUST be set to the same value (50 milliseconds)
   as indicated by Happy Eyeballs [RFC8305].

5.2.5.  Creation of EAMT entries

   An EAM Table (EAMT used for short, across the rest of this document)
   MUST be created/maintained automatically by the NAT46/CLAT, which is
   responsible to prioritize any available entries in the EAMT, versus
   the use of any synthetic AAAA.

   The EAMT entry MUST only be created if all the following conditions
   are met:

   1.  The source host is IPv4-only.

   2.  The DNS proxy is ready to return the A answer (according to
       Section 5.2.4).

   3.  There is at least one non-synthesized AAAA response.

   4.  If DNSSEC is available, the response has been locally validated
       or the AD bit has been set by a trusted resolver, as per
       [RFC3655].

   This avoids a slight NAT64 overload and flapping between destination
   addresses (IPv4/IPv6), which may impact some applications, at the
   cost of a small extra delay for the initial communication setup, when
   the EAMT entry doesn't yet exist.

   Each EAMT entry MUST contain, the fields already described in
   [RFC7757] and a few new extensions (as per section 3.1 of [RFC7757]):

   1.  ID: EAMT Entry Index (optional).

   2.  MAC address: Identify the host to which this EAMT entry belongs.

   3.  Destination IPv4 address/prefix: By default, the prefix length is
       32 bits.

   4.  Destination IPv6 address/prefix: By default, the prefix length is
       128 bits.  Only non-synthesized addresses are allowed.

   5.  TTL: It MUST be set to the minimum value from the AAAA/A RR pair.
       In normal conditions the TTL for both A and AAAA records, of a
       given FQDN, should be the same, so this ensures a proper behavior
       if there is any DNS mismatch.

   6.  FQDN: The one that originated the A query for this EAMT entry.
       Required in order to ensure a correct detection of cases such as
       the use of reverse-proxy with a single IPv4 address to multiple
       IPv6 addresses.

   7.  Valid/Stale/Invalid: When set to Stale, means that this EAMT
       entry MUST NOT be used for new connections.  When set to Invalid,
       means that this EAMT entry can be deleted, unless the Auto/Static
       bit is also set.

   8.  Auto/Static: When set to 1, means that this EAMT entry has been
       manually/statically configured, for example by means of an
       explicit configuration (GUI, CLI, provisioning system, etc.).

   Note that allowing destination IPv4 and IPv6 prefixes, together with
   the Valid/Stale/Invalid and Auto/Static flags, allows manual explicit
   optimization and non-optimization configuration for specific parts of
   Internet.

   When a new EAMT entry is first automatically created, it is flagged
   as "Valid" and "Auto".  If a subsequent A query, with a different
   FQDN, results in an IPv4 address that has already an EAMT entry and a
   different IPv6 address, it means that some reverse-proxy or similar
   functionality is being used by the IPv6-enabled service.  In this
   case, the existing EAMT entry will be marked as "Stale".  No new EAMT
   entry is created for that IPv4 address.  Otherwise, the optimization
   will only allow to access the first set of IPv4/IPv6/FQDN, which may
   break the access to other FQDN that share the same IPv4 address and
   different IPv6 addresses.

   In this case the EAMT entry will still become "Invalid" according the
   TTL, which allows to re-enable optimization if a new query for the A
   record has changed the situation.  For example, maybe the reverse-
   proxy has been removed, or there is now only a single device using
   it, so at the time being, the optimization is again possible without
   creating troubles to other hosts.

   An EAMT entry marked as "Stale" or "Invalid", only affects the
   relevant host.  Other hosts have their own EAMT entries or they are
   using the regular NAT46/CLAT+NAT64 path (without the optimization).

5.2.6.  Forwarding path via stateful NAT for existing EAMT entries

   Following this approach, if there is a valid EAMT entry, for a given
   pair of source-MAC-address/IPv4-destination, the IPv6-native path
   pointed by the IPv6 address of that EAMT entry, will take precedence
   versus the NAT64 path, so the traffic will not be forwarded to the
   NAT64.

   However, this is not sufficient to ensure that individual
   applications are able to keep existing connections.  In many cases,
   audio and video streaming may use a single TCP connection lasting
   from minutes to hours.  Instead, the CDN TTLs may be configured in
   the range from 10 to 300 seconds in order to allow new resolutions to
   switch quickly and to handle large recursive resolvers (with hundreds
   of thousands of clients behind them).

   Consequently, the EAMT entries MUST NOT be used directly to establish
   a forwarding path, but instead, MUST be used to create a stateful NAT
   entry for the 4-tuple for the duration of the session/connection.
   This means that to implement the optimization the NAT46 MUST be
   stateful.  Typically stateful NAT46 are implemented by means of a
   stateful NAT44 (which often maybe hardware off-loaded), followed by a
   stateless NAT46.  If the SoC/code is able to do stateless NAT46, this
   still could be used when the optimization is disabled.

5.2.7.  Maintenance of the EAMT entries

   An EATM entry with the Auto/Static bit set, MUST NOT be automatically
   modified.

   An EAMT entry with the Auto/Static bit clear, MUST be set to Stale
   in case of:

   1.  TTL time-out.

   2.  Or the conditions for creation of the EAMT entry (Section 5.2.5)
       aren't longer met.

   Entries in Stale state MUST be set to Invalid once existing
   connections time-out.

   The Invalid EAMT entries MUST be deleted, unless the Auto/Static bit
   is set.  This allows users/operators to set explicit rules for
   diagnostics or resolution of issues in special situations.

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 19 Jul 2020 22:49:16 +0200

Hi Erik,

I think that the changes that I’ve introduced with the Eduard’s idea of doing an alternative detection of the IPv4-only host, together with having the MAC address in the EAMT entry and the “Stale” state, solve most of the issues:
Only IPv4-only hosts are now using the optimization.
We keep the EAMT entries until the connection is gone.
HE is not affected.
IPv4 hosts that use vendor defined DNSs (such as 8.8.8.8), will not take advantage of the optimization, because they don’t reuse existing entries – unless they are manually configured by the user or operator.

We lose applying the optimization to specific applications  (i.e., a dual-stack host running an IPv4 only app), but because the main goal is the “big” traffic generators (SmartTVs/STBs), I think this is just fine.

I’m responding in-line to some points.

El 17/7/20 17:57, "Erik Nygren" <[email-masked]> escribió:

At the high-level, not causing problems relies heavily on assumptions about client behavior,

both for STBs/SmartTVs as well as for other devices in the home.  If the EAMT entries

also contained the internal client IPv4 address then it might make it so that

at least devices couldn't interfere with each other, but absent that there is also

plenty of possibility for wacky interactions.  The more we look for corner-cases

   (ie, a client will do a "gethostbyname" and then just keep using the result

   for multiple connections).  CDNs have to mostly deal with this (eg, wait a few 

  of new connections drops to an acceptable level).  I suspect some of the STBs/SmartTVs

  that would have marked the first invalid but doesn't see it due to the early expiry.

  Keeping EAMT entries in a "stale" state for some amount of time following TTL expiry

  (an hour or two?) where they can't be used but can be transitioned to invalid 

  and also kept in invalid for an hour or two would help.  The question is whether

[Jordi] I’m not sure if instead of defining time is sufficient to wait for the connection to die. If it never dies, it will be there for as much time as the CE needs to clean up entries, because lack of memory. We may need to define this in a much better way. Note however, that I think the number of devices that are IPv4 only and the number of connections they stablish, in a typical subscriber, should not be a big problem. A couple of SmartTVs and STBs, may be some other devices. Are we talking about a few tens of entries?

   synchronous A/AAAA lookups in a way that may break the "IPv4-only detection".

   specify a DoH server to use (and it's possible this might apply to AppleTV as well?).

   that's going to be a real problem.    Keying the EAMT entry by client IP might

   their own DoH server.

[Jordi] We don’t create/reuse EAMT entries for those cases, neither use DNS for the detection. Clearly, we will not optimize those,  but we *don’t need to* because I expect devices being updated to support DoH, etc., are also supporting IPv6, specially Apple case and also Chromecast/Android.

 

On Fri, Jul 17, 2020 at 5:40 AM JORDI PALET MARTINEZ <[email-masked]> wrote:

[Erik]  Ah, I think the invariant here is that DNS64 will always guarantee that *everything* has a AAAA record

available to the CPE resolver.  If there are ever cases where no AAAA is returned (eg, where the CLAT would get used)

then problems can show up because "IPv4-only" (no AAAA record) entries won't be validated against the EAMT table

and won't cause EAMT entries to get created.  It may be that we should be creating "invalid" EAMT entries 

for {IPv4,FQDN} pairs where no AAAA is returned.  Also, we should be invalidating EAMT entries 

[Jordi] That was solved since the first version. We only create EAMT entries if we have an AAAA and is not using the NAT64 prefix (so we make sure isn’t synthesized). I’ve added more text to make it more explicit, and not using. I don’t see if I’m missing something, but I believe there is no way to create an EAMT which doesn’t really have the right set of parameters (FQDN, IPv4, IPv6). We are also already setting to stale EAMT entries if a new FQDN starts to use a different IPv6 with the same IPv4, so will not be longer optimized for new connections, but the existing one is kept.

 

[Erik]  My question for both of these is if we should provide some context or examples in the doc

to explain the invariants and things that must be tested for?  While these conversations happened on-list,

having summaries of the risks (and what is being relied on to mitigate them) seems like it may be valuable

[Jordi] Yes, no problem on that. I’m happy to have more examples. We first need to make sure that the “new direction” is appropriate so not to lose right track.

The case is:

2) Five minutes later, b.example.com is resolved (TTL 600s) is resolved an EAMT entry is created 

   (but shares an IPv4 address with a.example.com but a different IPv6 address)

3) Two minutes later, the IPv4 address from a.example.com gets used (past the DNS TTL) 

Does this help clarify?  While you could argue this client is 'broken", anecdotal experience

is that this is a very common behavior for some client software that servers already

need to work around by padding operational actions for an hour or two past the DNS TTL.

[Jordi] No, this can’t happen. Now even less with the new way (stale status, entries per host, no duplicate entries with the same IPv4). Even if your example is the same host, a.example.com will be optimized, but b and c will not. Further when b happens, a.example.com will become stale (I’m not sure if this is any longer needed … I need to double-think about that).

     Erik

---

From: Erik Kline <[email-masked]>
Date: Sun, 19 Jul 2020 14:29:15 -0700

Does any of the below solve the problem of an IPv4-only host trying to
learn it's external IPv4 address via a STUN service that is
dualstacked?

---

From: Vasilenko Eduard <[email-masked]>
Date: Mon, 20 Jul 2020 08:59:16 +0000

Hi Jordi,
> [Jordi] I'm still trying to tidy up all the text according to other inputs and respond to them. Too soon to send a new version, but close ...

There is a theory that average person has a cache for 7 different things.
Then he/she has to substitute 1 thing by another.
And like in CPU case - productivity would considerably degrade.

I mean: Jin, Eric, Philip have pushed you much over tolerable 7 (or even 10) considerable changes.
Nobody could keep all these advises in mind - re-reading of e-mails is needed, that is not productive.
Draft "-03" is needed to flush human cache. For effectiveness.

No problem to finish on draft "-04" in the future. Do not put too much efforts to finish on "-03" .

>        if there is any DNS mismatch.

[Eduard] This is wrong. If host would not pay attention to DNS TTL and continue to use session (or even create new sessions) - we have no rights to drop his experience.

>    still could be used when the optimization is disabled.

[Eduard] May be I have not read all your e-mails, but do you still have NAPT44?
You are already talking about individual context for every host.

>    Entries in Stale state MUST be set to Invalid once existing
[Eduard] it is not properly decided yet how to understand what is "stale" - see above comments.

Eduard
-----Original Message-----
From: v6ops [mailto:[email-masked]] On Behalf Of Erik Kline
Sent: 20 июля 2020 г. 0:29
To: JORDI PALET MARTINEZ <jordi.palet=[email-masked]>
Cc: v6ops list <[email-masked]>
Subject: Re: [v6ops] draft-ietf-v6ops-464xlat-optimization-02 general concerns

Does any of the below solve the problem of an IPv4-only host trying to learn it's external IPv4 address via a STUN service that is dualstacked?

---

From: Vasilenko Eduard <[email-masked]>
Date: Mon, 20 Jul 2020 09:20:38 +0000

Hi Erik,

What is your concern on STUN?
If somebody would access server by Literal - it would go to PAT - no problem.
RFC 5389 section 9: Hard-coding the domain name of the server into software is NOT RECOMMENDED in case the domain name is lost or needs to change for legal or other reasons.
RFC 5389 section 19 (changes):   o  Added support for IPv6.  Made it clear that an IPv4 client could get a v6 mapped address, and vice versa.
Therefore, If STUN server is available over IPv6 - then it is the owner problem for it to operate properly. Agree?

Eduard

---

From: Philip Homburg <[email-masked]>
Date: Mon, 20 Jul 2020 11:57:10 +0200

> Because we are looking for EAMT entries which also contain the
> FQDN, there will be no match, so no optimization performed if the
> AAAA response doesn't come timely.

I don't understand this.

parallel).

The CPE generates additional AAAA queries for a.example.com and b.example.com.

The A queries for both a.example.com and b.example.com return 192.0.2.1.
The AAAA query for a.example.com returns 2001:DB8::1 and the AAAA query 
for b.example.com results in a timeout.

When the AAAA query for a.example.com returns, the CPE installs an EAMT
entry for a.example.com. When b.example.com gets a timeout, the CPE does
nothing

Later the host connects twice to 192.0.2.1. The CPE sees the EAMT entry
and connects the host twice to 2001:DB8::1

---

From: Vasilenko Eduard <[email-masked]>
Date: Mon, 20 Jul 2020 10:49:37 +0000

I do not see good resolution for this corner case (presented by Philip):
"for the corner case when the owner of Internet resource has partially finished his applications migration to IPv6, may be some applications accessible by IPv4-only, despite sharing same dual-stack server (or servers)"

But I see work-around (more complexity on draft - additional algorithm complexity):
if such situation would happen (destination IPv4 has the collision for the same source IPv4 host, but at the same time second IPv6 is not resolved) then previous EAM entry should be put in "disabled" state, no new sessions for this pair <source IPv4, destination IPv4> should be processes, current flow could continue.
The biggest problem that we should memorize this situation (in "disabled" state) forever, because request to 1st (IPv6 available) resource could happen later - then traffic to 2nd resource (IPv4 only) would be disrupted (I assume it was flowing to PLAT).

Such complicated logic would never come to hardware, but SoC chips have ARM cores for this - should not be a problem.

Of course, it is not good to keep states for the purpose "not to use".
Any better idea?

Eduard
-----Original Message-----
From: v6ops [mailto:[email-masked]] On Behalf Of Philip Homburg
Sent: 20 июля 2020 г. 12:57
To: [email-masked]
Cc: JORDI PALET MARTINEZ <jordi.palet=[email-masked]>
Subject: Re: [v6ops] draft-ietf-v6ops-464xlat-optimization-02 general concerns

> Because we are looking for EAMT entries which also contain the FQDN, 
> there will be no match, so no optimization performed if the AAAA 
> response doesn't come timely.

I don't understand this.

Suppose an IPv4-only client looks up a.example.com and b.example.com (in parallel).

The A queries for both a.example.com and b.example.com return 192.0.2.1.
The AAAA query for a.example.com returns 2001:DB8::1 and the AAAA query for b.example.com results in a timeout.

When the AAAA query for a.example.com returns, the CPE installs an EAMT entry for a.example.com. When b.example.com gets a timeout, the CPE does nothing

Later the host connects twice to 192.0.2.1. The CPE sees the EAMT entry and connects the host twice to 2001:DB8::1