Subject: [v6ops] Fwd: New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt

From: Davey Song <[email-masked]>
Date: Fri, 21 Sep 2018 14:31:50 +0800

Hi folks,

I just submited a new draft intending to provide better connectivity from
network side function . Comments are welcome.

Best regares,
Davey

---------- Forwarded message ---------
From: <[email-masked]>
Date: Fri, 21 Sep 2018 at 14:15
Subject: New Version Notification for
draft-v6ops-xie-network-happyeyeballs-00.txt
To: Chongfeng Xie <[email-masked]>, Linjian Song <
[email-masked]>

A new version of I-D, draft-v6ops-xie-network-happyeyeballs-00.txt
has been successfully submitted by Linjian Song and posted to the
IETF repository.

Name:           draft-v6ops-xie-network-happyeyeballs
Revision:       00
Title:          Network-side Happy Eyeballs based on accurate IPv6
measurement
Document date:  2018-09-20
Group:          Individual Submission
Pages:          9
URL:
https://www.ietf.org/internet-drafts/draft-v6ops-xie-network-happyeyeballs-00.txt
Status:
https://datatracker.ietf.org/doc/draft-v6ops-xie-network-happyeyeballs/
Htmlized:
https://tools.ietf.org/html/draft-v6ops-xie-network-happyeyeballs-00
Htmlized:
https://datatracker.ietf.org/doc/html/draft-v6ops-xie-network-happyeyeballs

Abstract:
   During the period of IPv6 transition, both ISP and ICP (Internet
   Content Provider) care about user's experience in dual-stack network.
   They hesitate to provide IPv6 to their users due to the fear of poor
   IPv6 performance.  Network-based Happy Eyeballs (NHE) is proposed in
   this memo as a approach to provide better connectivity to end users
   by doing accurate measurement and comparison on IPv6/IPv4 performance
   on the network side other than client-side Happy Eyeballs (HE)
   ([RFC8305]).  It works independently with client's adoption of HE.

   REMOVE BEFORE PUBLICATION: The source of the document with test
   script is currently placed at GitHub [NHE-GitHub].  Comments and pull
   request are welcome.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Fri, 21 Sep 2018 08:51:31 +0200

Hi,

I like this document.

In fact, looks to me, somehow, as an alternative approach to what I was trying in:

https://tools.ietf.org/html/draft-palet-v6ops-he-reporting-00

However, I’m missing some way to actually alert the ISP about the failures.

If you think that this makes sense, I’m happy to contribute to that, as I was already working in an update of my document and alternatives to using syslog.

Regards,

Jordi

De: v6ops <[email-masked]> en nombre de Davey Song <[email-masked]>
Fecha: viernes, 21 de septiembre de 2018, 8:32
Para: <[email-masked]>
Asunto: [v6ops] Fwd: New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt

Hi folks,

I just submited a new draft intending to provide better connectivity from network side function . Comments are welcome.

Best regares,

Davey 

---------- Forwarded message ---------
From: <[email-masked]>
Date: Fri, 21 Sep 2018 at 14:15
Subject: New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt
To: Chongfeng Xie <[email-masked]>, Linjian Song <[email-masked]>

Name:           draft-v6ops-xie-network-happyeyeballs
Revision:       00
Title:          Network-side Happy Eyeballs based on accurate IPv6 measurement
Document date:  2018-09-20
Group:          Individual Submission
Pages:          9
URL:            https://www.ietf.org/internet-drafts/draft-v6ops-xie-network-happyeyeballs-00.txt
Status:         https://datatracker.ietf.org/doc/draft-v6ops-xie-network-happyeyeballs/
Htmlized:       https://tools.ietf.org/html/draft-v6ops-xie-network-happyeyeballs-00
Htmlized:       https://datatracker.ietf.org/doc/html/draft-v6ops-xie-network-happyeyeballs

Abstract:

The IETF Secretariat

_______________________________________________ v6ops mailing list [email-masked] https://www.ietf.org/mailman/listinfo/v6ops 

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.consulintel.es
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.

---

From: Davey Song <[email-masked]>
Date: Sat, 22 Sep 2018 15:26:01 +0800

Hi Jordi,

Thanks for your support. I read your draft. I think we target different
problems. NHE can avoid the operational issues but it is not the main
purpose. If you would like to improve NHE to do better performance
measurement or other extension, the authors are open for suggestions.

Btw, it is the vacation of Mid-Autumn day in China. Wish you all Happy
Mid-Autumn day : )

Best regards,
Davey

On Fri, 21 Sep 2018 at 14:51, JORDI PALET MARTINEZ <
[email-masked]> wrote:

> If you think that this makes sense, I’m happy to contribute to that, as I
> was already working in an update of my document and alternatives to using
> syslog.
>
>
> Regards,
>
> Jordi
>
>
>
>
>
>
>
> *De: *v6ops <[email-masked]> en nombre de Davey Song <
> [email-masked]>
> *Fecha: *viernes, 21 de septiembre de 2018, 8:32
> *Para: *<[email-masked]>
> *Asunto: *[v6ops] Fwd: New Version Notification for
> _______________________________________________ v6ops mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/v6ops
>
> This electronic message contains information which may be privileged or
> confidential. The information is intended to be for the exclusive use of
> the individual(s) named above and further non-explicilty authorized
> disclosure, copying, distribution or use of the contents of this
> information, even if partially, including attached files, is strictly
> prohibited and will be considered a criminal offense. If you are not the
> intended recipient be aware that any disclosure, copying, distribution or
> use of the contents of this information, even if partially, including
> attached files, is strictly prohibited, will be considered a criminal
> offense, so you must reply to the original sender to inform about this
> communication and delete it.
>
>

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sat, 22 Sep 2018 10:13:08 +0200

Hi Davey,

My point is that in the document you mention “Network-based Happy Eyeballs (NHE) is proposed in this memo as a approach to provide better connectivity to end users by doing accurate measurement…”.

If those measurements are done, I think the next logical step is also to notify somehow the IPv4-fallbacks to the operator. Don’t you think so?

Regards,

Jordi

De: Davey Song <[email-masked]>
Fecha: sábado, 22 de septiembre de 2018, 9:26
Para: <[email-masked]>
CC: <[email-masked]>
Asunto: Re: [v6ops] Fwd: New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt

Hi Jordi,

Thanks for your support. I read your draft. I think we target different problems. NHE can avoid the operational issues but it is not the main purpose. If you would like to improve NHE to do better performance measurement or other extension, the authors are open for suggestions. 

Btw, it is the vacation of Mid-Autumn day in China. Wish you all Happy Mid-Autumn day : )

Best regards,

Davey

On Fri, 21 Sep 2018 at 14:51, JORDI PALET MARTINEZ <[email-masked]> wrote:

Hi,

I like this document.

Regards,

Jordi

Hi folks,

Best regares,

Davey 

---------- Forwarded message ---------
From: <[email-masked]>
Date: Fri, 21 Sep 2018 at 14:15
Subject: New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt
To: Chongfeng Xie <[email-masked]>, Linjian Song <[email-masked]>

Abstract:

The IETF Secretariat

---

From: Davey Song <[email-masked]>
Date: Sun, 23 Sep 2018 00:22:05 +0800

Sure, helping ISP to record the fallback cases is additional gain from NHE
which adds more motivation for ISP to deploy it. If people like NHE and
encourage us to proceed, then we can discuss it after we move a step
forward.

Right now, people may be still doubt about the performance of NHE itself
(the accuracy and the cost of it ). We need some tests and data to
demonstrate the capacity. People may suspects that NHE may cause more
aggressive IPv4-fallback, so we need to determine a good threshold value
for NHE when comparing IPv6/IPv4 round-trip delay.  We need to know the
ratio of HE-enabled apps in top 1000 apps, and if NHE can help
None-HE-enabled Apps. Now it is unknown yet for me.

A good news recently is Apple release a ios version 12.1 which finally
support IPv6 in 4G LET data plane. And China telecom is planning for a
testing to find answers on above questions. If any ISP operators think NHE
is interesting, we can do some work together. Or if I missed any work
already done, please let me know. Thanks.

Best regards,
Davey

On Sat, 22 Sep 2018 at 16:13, JORDI PALET MARTINEZ <
[email-masked]> wrote:

> Hi Davey,
>
>
>
> My point is that in the document you mention “Network-based Happy Eyeballs
> (NHE) is proposed in this memo as a approach to provide better connectivity
> to end users by doing accurate measurement…”.
>
>
>
> If those measurements are done, I think the next logical step is also to
> notify somehow the IPv4-fallbacks to the operator. Don’t you think so?
>
>
> Regards,
>
> Jordi
>
>
>
>
>
>
>
> *De: *Davey Song <[email-masked]>
> *Fecha: *sábado, 22 de septiembre de 2018, 9:26
> *Para: *<[email-masked]>
> *CC: *<[email-masked]>
> *Asunto: *Re: [v6ops] Fwd: New Version Notification for

---

From: 神明達哉 <[email-masked]>
Date: Mon, 24 Sep 2018 11:14:40 -0700

At Fri, 21 Sep 2018 14:31:50 +0800,

> network side function . Comments are welcome.

Some quick observations:

- I don't see why the intended status is Standards Track.  It seems to
  be a document about an operational practice rather than a new
  protocol feature.

- In general, I wouldn't be excited about placing such complicated
  functionality in the network rather than end hosts.  Sometimes it
  may be justified as a least evil option, but the current description
  of the draft didn't fully convince me

- I suspect the discussion on breaking DNSSEC is way too hand-waving.
  In my general understanding it's generally not accepted at dnsop to
  justify breaking DNSSEC just by saying "it's okay as validation at
  end hosts is not typical".  Especially if it really intends to be
  published as Standards Track I suspect some more detailed discussion
  with a stronger justification will be needed.

--
JINMEI, Tatuya

---

From: Fred Baker <[email-masked]>
Date: Mon, 24 Sep 2018 16:31:00 -0700

> On Sep 24, 2018, at 11:14 AM, 神明達哉 <[email-masked]> wrote:
> 

I think we would have a charter problem as well. V6ops, per charter, can produce BCPs and informational documents.

> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

--------------------------------------------------------------------------------
Victorious warriors win first and then go to war,
Defeated warriors go to war first and then seek to win.
     Sun Tzu

---

From: Davey Song <[email-masked]>
Date: Tue, 25 Sep 2018 10:30:05 +0800

Hi Jinmei,

Thanks for your comments. I once had same questions in my mind when this
draft is conceived. I reply inline.

>   be a document about an operational practice rather than a new

I agree with your observation here if the draft intends to share
information and practice. It now goes with Standards Track because the
authors would like to provide a new HE feature(or a framework) on network
side as a peering funtion of client-side HE. I would like to see more
feadback on this issues.

>   of the draft didn't fully convince me

Before I put down this draft, I talked with some CPs (content/app
providers) and ISPs, they have motivations and requirement on this.  One
example is Tencent, they are planning to deploy a complicated measurement
network to info their Apps (with a SDK ) which address family they should
try first (their apps use dns over http).  I'm told that Tencent think HE
implemented on each client is too complicated (they have comlicated apps)
and resource consuming. ISP's motivation is easy to understood, they always
try to improve their network connenctivity on both IPv4 and IPv6. I have no
more comment on your saying of evil option：） Any technology can be used as
evil purpose I think.

>   with a stronger justification will be needed.

Sorry, I admit current txt on breaking DNSSEC is weak. Now it is put
into Security
Considerations as well as the issues on incoherency. I think one way to
avoid it as suggested in  Security Considerations is to artificially delay
the AAAA answers, other than omitting the AAAA record and its RRSIG

Best regards,
Davey

---

From: Gert Doering <[email-masked]>
Date: Tue, 25 Sep 2018 09:20:15 +0200

Hi,

On Tue, Sep 25, 2018 at 10:30:05AM +0800, Davey Song wrote:
> and resource consuming. 

This hints at a general misunderstanding on how "The Internet" works.

Talking to the local ISP on "what would you recommend, IPv4 or IPv6?"
can give an indication, but if the server you're trying to talk to 
is on the other side of the world, the local ISP's preference for 
IPv4-vs-IPv6 might be the opposite of what actually works better to
reach said server.  There are many networks on the path...

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                      Vorstand: Sebastian v. Bomhard, Michael Emmer
Joseph-Dollinger-Bogen 14        Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                 HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444         USt-IdNr.: DE813185279

---

From: Davey Song <[email-masked]>
Date: Tue, 25 Sep 2018 17:09:13 +0800

To get to know the current status of HE , I did a simple test on my handy
devices visit top 20 website or open existing apps (currently only DNS). I
captured AAAA&A DNS query pairs (intervel less than 10s) and count HE query
pairs whoes interval less than 100ms. There is a prelimiary result:

1) My laptop with web browsers (Chrome, firfox, Sogou, QQ, 360). All
browers support HE in DNS in which  more than 90% AAAA&A query pairs are HE
query pairs.
2) My Andriod phone(V7.1.1).  262 AAAA&A query pairs out of  802 pairs . It
means only 32.6% DNS queries supported by HE. Not 32.6% Apps on my phone,
because some Apps may sent more DNS queries than others. I made a
statisitcs on interval of all AAAA& A pairs (statistics if no HE pairs
included), the average interval is 409ms (534ms), the median delay is
198ms(324ms), and the maximum is 7,135ms. If hostname resolution is not
support HE, the connection attampt will also be delayed accrodingly.
3) My ipad (V11.4.1).  1208 AAAA& A query pairs out of 1216 pairs. It means
more than 99% DNS queries supported by HE. It is true that IOS asynchronous
DNS API. But it is unkonwn on how many apps make connection asynchronously.
It will be investigated later.

In conlusion, HE is not widely supported at least in Android  platform.
Large companies can invest and implement it for their users. But it is not
fair for small Apps providers to endure such competitive disadvantage due
to IPv6 deployment.

Davey

---

From: Davey Song <[email-masked]>
Date: Tue, 25 Sep 2018 17:30:48 +0800

> Talking to far-end server need measurement on end-to-end path.So the
accurracy of end-to-end measurement is the key. There are acturally lots of
practice in the fields of NPM and APM. People can do it well.

Davey

---

From: Ole Troan <[email-masked]>
Date: Tue, 25 Sep 2018 12:25:16 +0200

> Talking to far-end server need measurement on end-to-end path..So the accurracy of end-to-end measurement is the key. There are acturally lots of practice in the fields of NPM and APM. People can do it well. 

But in the general case the network cannot.
Think host multi-homing.

Ole

---

From: Davey Song <[email-masked]>
Date: Wed, 26 Sep 2018 10:49:17 +0800

> But in the general case the network cannot.

Yes or no.

Generally speaking the races of IPv6 and IPv4 connections on both network
and client are going to be suffered by netowrk dynamics, including
Multi-homing,  route flaps, roaming, or other network falilures. Extremely,
a client can get a better IPv6 connection in one second (when IPv6 win the
race), and lose it in next second. In such case, more sophisticated
measurement should be done(on client or network) , for a longer period, on
statistics of RTT and Failure rate, or combinations of them. But in IMHO,
the assumption of HE is relatively stable network for short exchange
connections. The dynamics exits but relatively rare or no notable impact on
HE. So I see no such discussion in RFC8035.

Davey

---

From: George Michaelson <[email-masked]>
Date: Wed, 26 Sep 2018 12:58:30 +1000

I have said before, but don't know if I still adhere to it, but
anyways, here's a question: How *long* do people think a biassing
mechanism like HE is a good idea?

 * is it a good idea *forever*

 * or is it a transition path mechanism which has an end-of-life?

 * how do we know, when its at end-of-life?

I used to love HE. I now have a sense, I'm more neutral. Maybe, we
actually don't want modified, better happy eyeballs, because we want
simpler, more deterministic network stack outcomes with less bias
hooks?

I barely register if I an on v4 any more. I assume I'm on 6 on many
networks. This is as an end-user. I guess if I am really an end user,
this belief I understand TCP and UDP is false, and I should stop
worrying (as an end user)
On Wed, Sep 26, 2018 at 12:49 PM Davey Song <[email-masked]> wrote:
> Generally speaking the races of IPv6 and IPv4 connections on both network and client are going to be suffered by netowrk dynamics, including Multi-homing,  route flaps, roaming, or other network falilures. Extremely, a client can get a better IPv6 connection in one second (when IPv6 win the race), and lose it in next second. In such case, more sophisticated measurement should be done(on client or network) , for a longer period, on statistics of RTT and Failure rate, or combinations of them. But in IMHO, the assumption of HE is relatively stable network for short exchange connections. The dynamics exits but relatively rare or no notable impact on HE. So I see no such discussion in RFC8035.
>
> Davey
> _______________________________________________
> DNSOP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsop

---

From: Owen DeLong <[email-masked]>
Date: Tue, 25 Sep 2018 21:25:16 -0700

It was never a good idea. It was a necessary evil (kind of like NAT in that regard) to expeditiously deal with a somewhat tenacious (at the time) problem which has since been given a significantly better solution, but so long as the workaround appears to be working, people are loathe to put in the effort of implementing the actual solution.

sigh… Human nature.

Owen

> On Sep 25, 2018, at 19:58 , George Michaelson <[email-masked]> wrote:
> 
> v6ops mailing list

---

From: Joe Abley <[email-masked]>
Date: Tue, 25 Sep 2018 22:24:34 -0700

What better idea did you mean?

Being able to select a protocol based on what works best for the
end-user does not seem like a terrible end-state for the end-user,
short- or long-term.

> On Sep 25, 2018, at 21:25, Owen DeLong <[email-masked]> wrote:
>

---

From: George Michaelson <[email-masked]>
Date: Wed, 26 Sep 2018 15:28:24 +1000

I'm not speaking for Owen. I'm speaking for myself. I asked a
question. Is this really a long-term defensible thing to do? Do we
want HE forever?

run a race? thats fine. But, as the thread here notes, the
second-by-second conditions which leads one TCP to return SYN-ACK
before another can be volatile.

run a race, but bias the race towards the one you like? okaaaay.. But
once we're beyond a world where the V6 needs the bias, for anyone
stuck on the vestigial 4-is-better space, this means they incurred
*additional* connection penalty. wheres the control knob?

now we're talking about tuning the bias, weighting the sum, tumbling
the dice. I thought it was a crap shoot anyway...

-G

---

From: Joe Abley <[email-masked]>
Date: Tue, 25 Sep 2018 22:43:40 -0700

Having a predictable bias seems better than an unpredictable one.

> On Sep 25, 2018, at 22:28, George Michaelson <[email-masked]> wrote:
>

---

From: Ole Troan <[email-masked]>
Date: Wed, 26 Sep 2018 08:10:02 +0200

Davey,

If we’re discussing host based versus network based happy eyeballs, would it be naive to think that the network based HE would interfere with the client’s HE?

A router knows very little about end to end properties of a connection. It could of course do those measurements by looking deeply into packets, but it would still be restricted to it’s own topological location. Compare that with the data available to an MP-TCP host stack. 

And note I think HE can’t just be between v4 and v6, but between all the candidate connections between source and destination. 

Cheers,
Ole

---

From: Davey Song <[email-masked]>
Date: Wed, 26 Sep 2018 14:23:30 +0800

Hi George,

Actually the idea of NHE is inspired partially by CDN stuff, which involve
lots of measurments
and route users to visit a best path against network dynamics.  It proves
to be a good practice
for morden Internet. No doubt. I'm wondering CDN is also breaking DNSSEC to
stub-resolver, right?

Davey

On Wed, 26 Sep 2018 at 13:28, George Michaelson <[email-masked]> wrote:

> > > It was never a good idea. It was a necessary evil (kind of like NAT in
> that regard) to expeditiously deal with a somewhat tenacious (at the time)
> problem which has since been given a significantly better solution, but so
> long as the workaround appears to be working, people are loathe to put in
> the effort of implementing the actual solution.
> > >
> > > sigh… Human nature.
> > >
> > > Owen
> > >
> > >
> > >>> Generally speaking the races of IPv6 and IPv4 connections on both
> network and client are going to be suffered by netowrk dynamics, including

---

From: Gert Doering <[email-masked]>
Date: Wed, 26 Sep 2018 08:36:09 +0200

Hi,

> *additional* connection penalty. wheres the control knob?

We need a bias for one or the other protocol.  Otherwise you'll end up
flipflopping between v4 and v6 for subsequent connections if both are
equal (because the networks in beween just run v4 and v6 over the same
links, same devices, same peerings, ...) - and that is a major annoyance
when trying to troubleshoot things.  Like, web server access that 
"sometimes works and on reload brings an error" (because apache ACLs
are not in sync for v4 vs. v6).

In the long run, we want a single protocol, whichever it is.  Dual-stack
is dual-work.

Gert Doering
        -- NetMaster

---

From: Mark Smith <[email-masked]>
Date: Wed, 26 Sep 2018 17:05:27 +1000

On Wed., 26 Sep. 2018, 16:10 Ole Troan, <[email-masked]> wrote:

> Davey,
>
> If we’re discussing host based versus network based happy eyeballs, would
> it be naive to think that the network based HE would interfere with the
> client’s HE?
>
> A router knows very little about end to end properties of a connection. It
> could of course do those measurements by looking deeply into packets, but
> it would still be restricted to it’s own topological location. Compare that
> with the data available to an MP-TCP host stack.
>
> And note I think HE can’t just be between v4 and v6, but between all the
> candidate connections between source and destination.
>

Yes, multipathing (+ encryption) has a lot of impacts on what is possible
to do and commonly done on the network today. A lot of those activities
would have to be shifted to the host.

Start at slide 63 -

http://www.users.on.net/~markachy/The_Rapid_Rise_of_the_MMHH.pdf

I can't seem to find it now, however I think Fred Baker wrote a draft a
number of years ago that observed that multi-addressing on a single
interface is also a form of multi-homing, meaning that it isn't a
requirement for a "multi-homed" host to have multiple interfaces attached
to multiple and different networks. That means most IPv4 hosts, and all
IPv6 hosts, by design, are potential multi-homed hosts.

Regards,
Mark.

---

From: Davey Song <[email-masked]>
Date: Wed, 26 Sep 2018 15:14:42 +0800

> it be naive to think that the network based HE would interfere with the

Currently this draft only considers IPv4/IPv6 racing situation. The general
address
racing is already done for all clients (rfc6724). As mentioned in the
draft, NHE can
work independenly(without interferance) with the adoption of Client-side
HE.
If client's already support HE, it will query A & AAAA at the same time. NHE
can
help reduce the unnecessary traffic emitted by HE client becuase the AAAA
record
will be omitted or delayed if IPv6 connectivity is poorer. I don't see any
interferance now.

> it would still be restricted to it’s own topological location.
>
It depends on how far you do the measurement to simulate the end users.
Some APM (application performance measurement) vendor like New Relic
provides sdk implementing on Apps to provide measurement data for analysis
and deliever a good performance monitoring and traffic engineering.
Some provide probes close to clients and edge.

I think maybe there is a crose-layer issues on this draft which should be
simple
 enough and aim IP layer issues. But right now lots of complected
application
layer complexities (like CDN, APM) are involved.

> Compare that with the data available to an MP-TCP host stack.
>

I think MP-TCP or other transmition protocol are based on stable,
high-performance ip layer.

> candidate connections between source and destination.

HE(RFC8035) only focus on IP connection seletion right now accroding to the
problem statement:

   "Many communication protocols operating over the modern Internet use
   hostnames.  These often resolve to multiple IP addresses, each of
   which may have different performance and connectivity
   characteristics.  Since specific addresses or address families (IPv4
   or IPv6) may be blocked, broken, or sub-optimal on a network, clients
   that attempt multiple connections in parallel have a chance of
   establishing a connection more quickly.  This document specifies
   requirements for algorithms that reduce this user-visible delay and
   provides an example algorithm."

If there are other candidate connections, they also need a better IP
connectiviy as a bais I think.
NHE at least can give a indicaiton of poor connection for certain address
family (or address) .
It is potentially a low level routing founctions other than hight level
traffic scheduling tools.

Davey

---

From: Gert Doering <[email-masked]>
Date: Wed, 26 Sep 2018 09:28:39 +0200

Hi,

On Wed, Sep 26, 2018 at 03:14:42PM +0800, Davey Song wrote:
> NHE can

I'm not sure how often I've heard the well-meaning suggestion "just do
not deliver DNS records of type <x> if <condition y>".

It was a bad idea at all times, and it is still a bad idea.  Never withhold
legitimate records.

[..]
> high-performance ip layer.

Read up again what MP-TCP does :-)

Gert Doering
        -- NetMaster

---

From: "Chongfeng Xie" <[email-masked]>
Date: Wed, 26 Sep 2018 15:56:15 +0800

I'm working in China telecom. End users' experience is the uppermost important for us, either is over IPv4 or IPv6.  People have fears on IPv6 transition because they are not sure of IPv6 performance. Actually we have received user's complains when IPv6 was deployed. IPv6-only is attractive，but it will take long time.  In the early stage of dual-stack deployment, we can not expect the IPv6 has matched performance with IPv4. Many countries and regions still in early develpment shown in APNIC's global IPv6 performance monitoring (stats.labs.apnic.net/v6perf). NHE is a promising solution to enable network operator to provide their clients with better connectivity in dual-stack, which will also be helpful to SP/CPs.  In addition, IPv6 situation in one region will persistent for longer period compared to the network dynamics. It will be easier to handle.
Chongfeng Xie

[email-masked]
 
From: George Michaelson
Date: 2018-09-26 10:58
To: dnsop WG
CC: <[email-masked]>
Subject: Re: [v6ops] [DNSOP] New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt
I have said before, but don't know if I still adhere to it, but
anyways, here's a question: How *long* do people think a biassing
mechanism like HE is a good idea?

---

From: Gert Doering <[email-masked]>
Date: Wed, 26 Sep 2018 10:01:27 +0200

Hi,

On Wed, Sep 26, 2018 at 03:56:15PM +0800, Chongfeng Xie wrote:
> In the early stage of dual-stack deployment, we can not expect the 
> IPv6 has matched performance with IPv4. 

This is true.  But "early stage of dual-stack deployment" was 15 years ago,
so this is not a valid excuse today.

Gert Doering
        -- NetMaster

---

From: Philip Homburg <[email-masked]>
Date: Wed, 26 Sep 2018 11:27:50 +0200

>simpler, more deterministic network stack outcomes with less bias

In my own implementation of HE, I globally (i.e. across all processes) keep
track of the time it takes to establish a TCP connection for individual
addresses, and I compute values for IPv4 and IPv6.

Conceptually I like the idea that if you know (from past measurements) that 
it takes 100ms or less to establish a connection, you don't wait for a
couple of seconds for an attempt to fail. In addition, instead of trying
one address at a time, you can add a new connection attempt when the
previous one is still running.

All of this provides for a better user experience, at the cost of being
less deterministic and masking failures.

Note that in my implementation, I select the best address (giving a 30ms
preference to IPv6) wait for the expected time for the TCP connection
to complete and only if that timer expires, move to the next address.
So there is no race.

---

From: Tony Finch <[email-masked]>
Date: Wed, 26 Sep 2018 11:58:07 +0100

> It was a bad idea at all times, and it is still a bad idea.  Never withhold

I think it's even worse in this case.

The point of happy eyeballs is to work around crappy or broken networks,
and this NHE idea sounds to me like making a huge effort to work around
broken or crappy networks rather than making the effort to fix them. It
seems misdirected to me.

If it's a third party network with crappy v6 connectivity, why not just
drop their v6 prefixes and let normal HE work as it should?

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
defend the right to speak, write, worship, associate, and vote freely

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 26 Sep 2018 13:41:56 +0000

> The point of happy eyeballs is to work around crappy or broken networks,

That's my recollection, too. And I'm really struggling to figure out which are the broken IPv6 networks NHE is trying to work around. The possible networks are: LAN, ISP/access, core/transit, and destination. I'm not aware of issues with core/transit networks. I'm also not aware of major issues with IPv6 broken at the destination (authoritative DNS servers supplying AAAA records, but those IPv6 addresses aren't actually reachable over the Internet or have incredibly slow IPv6 connectivity to the Internet).

My recollection is that the brokenness being worked around was primarily caused by non-functioning IPv6 connections that were being advertised to hosts, were provided AAAA addresses during DNS resolution (over IPv4), but didn't have actual IPv6 connectivity to The Internet. Teredo and 6to4 in CE routers and host OSs were primary causes of the problem, exacerbated by DNS resolvers (sitting on IPv4-only networks) providing AAAA records to DNS queries over IPv4. The destination network was *not* the source of the broken IPv6. The problem was very, very local. HE wasn't in response to minor performance differentials between IPv4 and IPv6 (to pick the one that worked better). It was about not waiting an entire 60 seconds (or more) for a 100% broken advertised IPv6 connection to fail before using IPv4. 

Some questions I have are:
Why would an ISP choose to deploy partial or broken IPv6 + NHE, rather than properly functioning IPv6? I would think we should be recommending deploying properly functioning IPv6 rather than broken IPv6 + NHE. I'm not aware of a compelling reason at this stage of the game for non-IPv6 or partial-IPv6 ISPs to be providing AAAA to DNS queries over IPv4. Not providing AAAA over IPv4 DNS in this case is much easier than the proposed NHE.

As for CDN statements made in this thread: CDNs are in the business of optimizing connections to their content. HE isn't about optimizing connections -- it's about more quickly reacting to the likelihood of outright brokenness. But it sounds like NHE *is* about optimizing (where IPv6 isn't broken, but is just really slow)? I'm a bit confused about this.

I've done various IPv4/IPv6 latency tests from inside my home, over time. 
Back when I had an ISP that was experimenting with IPv6 by having a single 6rd BR in the center of the US and that ISP was having other "we don't know what we're doing" issues, the IPv6 and general IP-connectivity performance was pretty abysmal. Now that I have an ISP (for the past 6 years) that has deployed a rock solid IPv6 solution (and I successfully got all the Teredo, 6to4, and ISATAP disabled on every device in my home), I have no problem. Which suggests IPv6 issues related to poor IPv6 deployments in destination networks isn't an issue for me. If it's an issue elsewhere in the world, please provide statistics -- I haven't seen any statistics presented by proponents of NHE related to the severity of whichever problem they're trying to solve. My experience suggests that ISPs who deploy IPv6 properly have no need for NHE.

It really sounds like the main thing NHE is trying to "solve" is "I'm an ISP who wants to deploy bad, partial, or broken IPv6 and need something that will keep my users from suffering too horribly from my ineptitude." I'm not sure that's a compelling reason for an RFC. I also question whether such inept ISPs would make use of NHE, even if it were available.
Barbara

---

From: Tony Finch <[email-masked]>
Date: Wed, 26 Sep 2018 16:15:52 +0100

STARK, BARBARA H <[email-masked]> wrote:

> Why would an ISP choose to deploy partial or broken IPv6 + NHE, rather
> than properly functioning IPv6?

That was my initial reaction too :-) I think the actual idea is to work
around brokenness on third party networks, e.g. the ISP has working v6,
the web site has working v6, but there's bad connectivity between them.
(It doesn't help that there are some stupid peering wars going on between
tier 1 v6 providers, as mentioned in the v6 section of [1].)

The right fix, I think, is to get better peering or to filter broken
routes. It would seem to me to be better to implement the fix in the
network layer not the DNS layer, since it's a network problem not a DNS
problem.

[1] https://blog.apnic.net/2018/09/21/2018-national-internet-segments-reliability-report/

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
South Utsire, Forties, Cromarty: Westerly 5 to 7, perhaps gale 8 later, but
variable 4 for a time. Moderate or rough, occasionally very rough in South
Utsire. Drizzle, rain later. Moderate or good.

---

From: Tommy Pauly <[email-masked]>
Date: Wed, 26 Sep 2018 08:37:52 -0700

+1 to the point that the proposal for NHE is essentially a mechanism for the ISP and/or content provider to work around a broken deployment that they should be in a position to fix themselves, or between one another as Tony describes.

The point of client-side Happy Eyeballs is to work around broken or suboptimal situations that the client has no control over. Following the analogy, the server or ISP should only employ a parallel mechanism in order to work around some problem it has no ability to fix otherwise—such as hiding IPv6 from clients with somehow broken IPv6 stacks. This isn't what the situation is like, however.

The part that remains interesting is suggestions for networks to collect more reports on the relative health of IPv4 and IPv6 connections, to help fix bugs and broken deployments. Focusing on these improvements rather than filtering DNS answers seems like it would be a more fruitful direction.

Best,
Tommy

> On Sep 26, 2018, at 8:15 AM, Tony Finch <[email-masked]> wrote:
>

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Wed, 26 Sep 2018 12:51:37 -0300

Hi Barbara,

In my experience, there are two ways IPv6 can be broken:
1) ICMPv6 being filtered, so PMTUD doesn't work.
2) IPv6 deployment issues (including having AAAA records with no or bad IPv6 connectivity).

HE only helps for 2).

But 2 can be caused in different points between the source and destination, and in most of the cases is bad transit/peering more often than the LANs or the ISP deploying IPv6 badly in its own network.

What is missing is that, unfortunately, many folks don't do the same level of monitoring on their IPv6 networks than IPv4 ones, or if the destination has AAAA, but they don't have IPv6 enabled, of course, they aren’t monitoring at all a non-existing IPv6 network.

Also, if the problem is caused by 1, is the ISP deploying IPv6, then one that need to get some kind of alert from the customers about failing end-sites, and contact them or their transits ...

At the end they need to be able to cooperate resolving the problem.

There is an example about this. I've detected several years ago that 1&1 deploys IPv6 to their customers, but even if I tried personally contacting them they did nothing. I kept insisting for 3 years, but I already give up.

This means that thousands of web site with IPv6 aren't reachable for many customers. Of course, because in this case is due to 1), you can't reach those web sites neither with IPv6 or IPv4 (unless you manually disable IPv6 in the browser or OS).

Regards,
Jordi

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Wed, 26 Sep 2018 12:54:33 -0300

And this is what I tried to say several times:

We need to take advantage of HE and NHE, or other means, to "report" to the ISPs about brokenness.

Regards,
Jordi

-----Mensaje original-----
De: v6ops <[email-masked]> en nombre de Tommy Pauly <[email-masked]>
Fecha: miércoles, 26 de septiembre de 2018, 12:38
Para: Tony Finch <[email-masked]>, "STARK, BARBARA H" <[email-masked]>, dnsop <[email-masked]>, "[email-masked]" <[email-masked]>, Davey Song <[email-masked]>
Asunto: Re: [v6ops] [DNSOP] New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt

    
    
    Best,
    Tommy

---

From: Paul Vixie <[email-masked]>
Date: Wed, 26 Sep 2018 09:05:44 -0700

JORDI PALET MARTINEZ wrote:
> ...

perhaps we can choose a flag day to turn on a new option in our ipv6 
stacks-- add an ipv6 option to all our syn and syn-ack packets, and/or 
require pmtud for all connections. anyone who filters, won't be on v6.

-- 
P Vixie

---

From: Fred Baker <[email-masked]>
Date: Wed, 26 Sep 2018 10:27:39 -0700

Speaking for myself, I see HE as a mechanism whose usefulness as described (selecting between IPv4 and IPv6 addresses) will wane, but which applied in a different way may have value long term. The latter has to do with access from or to multi-addressed services and selecting the one that seems to work best most of the time - even if all of the addresses are IPv6 addresses (or, in the RFC 6555 timeframe, all of the addresses were IPv4 addresses). As IPv4 winds down, yes, HE as described will become anachronistic - there won't be a choice to make. It will become, as NAT64 will, a service that is more work to remove than it's worth, but eventually isn't implemented in new implementations and nobody notices. But in the more general sense of multihoming, it may have longer term value.

Again speaking for myself, if it's a technology with declining value, it seems to me to not be something to invest in long term. RFC 6555 addressed a very real and measurable problem - host applications could find themselves waiting longer than their users could bear to a connection to complete. RFC 8305 improved on that. Further investment needs, I think, to have the same kind of investigation a monetary investment needs - is there a real need being addressed, and does the investment address it?

I haven't drawn any conclusions, but that gives you an idea of the questions in my head.

> On Sep 25, 2018, at 7:58 PM, George Michaelson <[email-masked]> wrote:
> 

--------------------------------------------------------------------------------
The fact that there is a highway to hell and a stairway to heaven is an interesting comment on projected traffic volume...

---

From: Fred Baker <[email-masked]>
Date: Wed, 26 Sep 2018 17:31:31 -0700

On Sep 26, 2018, at 10:27 AM, Fred Baker <[email-masked]> wrote:
> As IPv4 winds down, yes, HE as described will become anachronistic - there won't be a choice to make. It will become, as NAT64 will, a service that is more work to remove than it's worth, but eventually isn't implemented in new implementations and nobody notices.

This time I'm speaking as a co-chair of the working group. This is in part a question: if folks don't see things the way I do, I, and we, need to understand the various viewpoints. But it's also a statement of what I expect the next few years to look like.

As to the status of IPv6, let me comment on its history. I saw an article the other day whose headline announced that "IPv6 has failed; let's pick something else." RFC 2460 was published in 1998, but it didn't define IPv6 as a usable protocol. It needed IPv6-in-DNS, IPv6-in-DHCP, IPv6-in-your-favorite-routing-protocol, vendor implementation, working vendor implementation (oh, is that important?), random features like uRPF etc, ROAs and IRRs, and a long list of whatever else. ICANN didn't approve a policy for distribution of prefixes to RIRs, and the RIRs to networks, until 2006, per OECD. In 2007, about 350 IPv6 prefixes were allocated by ARIN for testing purposes, and in 2009-2011 there was a "run on the bank" in Asia-Pacific as the result of a JPNIC study indicating that IPv4 depletion was a realistic scenario.

In other words, the development of the technologies implied by IPv6 took almost as long to develop as the technologies required to use IPv4 (and involved a few mis-steps). However, the impetus to use them was much lower - whenever a technology was unavailable to make IPv6 work, "well, you can always use IPv4". IPv6 didn't become "important" until "always" became untrue. That started in 2011, when the IPv4 address space available from the IANA (and eventually the RIRs) was depleted; it is becoming increasingly untrue as the cost of purchasing an IPv4 address spirals upward.

"Pick something else" - I have to wonder what "something else" might be in view. We have a number of research projects on the table, such as ICN/CCN/NBN, and they likely have their uses. But I don't see the architectural or business implications following through; if they were rationally available, they would have started a decade back, I suspect. So I see IPv6 deployment as the only rational option on the table.

What I am increasingly hearing from ISPs and other networks is that it is less expensive to deploy an IPv6-only network with (or without) an IPv4 overlay  than to deploy a dual stack network. There are a few networks that have drawn that conclusion; as more and more do, IPv4 will become less of a requirement.

I'm not sure what to expect next, and maybe people can inform me. One option would be two networks that implement IPv6 natively and IPv4 as an overlay service connect, meaning that the address of a 464XLAT PLAT (or whatever) from one carrier is an address in a different carrier for some part of one of their networks. Or, maybe, a network simply stops providing an IPv4 PLAT or other support for IPv4 as an overlay. Maybe they are running lw4o6 and it becomes necessary to do IPv4 routing inside that overlay, so that they want to charge for it as a service. The obviously implied next question is "when" - I'm guessing early 2020s or something like that.

So, what do I expect the next few years to look like? I see three broad classes of ISPs, IXPs, and networks at the edge: those that see IPv6 as the least expensive endpoint and drive to that end, those that have enough IPv4 for the moment and aren't thinking any further ahead than that, and those that are balanced on a knife's edge in between. We have those now. I expect to see a shifting balance, with different markets adopting different perspectives and changing over time. I see a slow but definitely discernible motion from "IPv4 works" to "https://www.youtube.com/watch?v=_y36fG2Oba0" "I suppose we ought to give it a try". One at a time, I hear operators say "I don't want to buy any more IPv4 addresses". The networks that are fixed in the "IPv4 laggard" camp tend to be networks that aren't faced with IPv4 address purchases and ask for a business justification that doesn't involve a monetary outlay.

Ron, Lee, and I asked for a change to the v6ops charter to discuss IPv6-only networking with that in view. What I want to see in v6ops is more of that - lets move IPv4 to an overlay service, and watch it become less relevant over time.

So I personally expect transition technologies of various kinds - lw4o6, 464xlat, MAP-E, MAP-T, and so on - to remain important, but I expect that they will become the aftermarket more than the market. If cloud services are accessible by IPv6, I expect residential access to become IPv6 service to become larger and IPv4 to become smaller. If you want a picture of that, look at Trinidad and Tobago. As I understand it, there are some services (Google, Facebook, etc) that only offer IPv6 access there, so the "IPv6" picture is a picture of their market dominance. https://stats.labs.apnic.net/ipv6/CC?x=1&s=1&p=1&w=30&c=TT

I expect service providers to do as little with IPv4 as they can get away with, and for IPv4 to increasingly become ragged around the edges. There are still a variety of places where I would get tarred and feathered and sent home wearing a barrel for saying that, but the SNA people were of that opinion too - until APPN made SNA networks irrelevant and people started deploying IP-native applications.
--------------------------------------------------------------------------------
Victorious warriors win first and then go to war,
Defeated warriors go to war first and then seek to win.
     Sun Tzu

---

From: Davey Song <[email-masked]>
Date: Thu, 27 Sep 2018 11:09:39 +0800

Hi Barbara,

Thanks for your comments and sharing. I need to note that NHE is not
competing with
Client-side HE or competing with deployment of fully functioning IPv6. The
background of
NHE is :

1) the dualstack is real. Client is facing the choice ipv4 or ipv6.
2) the poor or unpredictable IPv6 performance is real (
https://stats.labs.apnic.net/v6perf ).
3) Low adoption of Client-side HE is realy.

So the suffering of users is real in dualstack.

What we are trying to do is to make the dualstack less painful for users in
these regions.
We can not say "you are starving or poor becuase you are not rich". We can
not jump
directly to build a fully functioning IPv6 in one day (like roma city). I
feel so sad the region
I live failed to catch up the average level of IPv6 development. But it is
true the IPv6
development situation varies in different places.

Maybe your observation is true that local ISP is the major source of broken
IPv6. However,
though I'm not network operator, I had at least two observations of IPv6
failure caused
by far-end and midst. One case is a real user complaint from China telecom.
It proved
that user experinced long delay when Servfail is received when asking AAAA
for many times.
It is the problem of authoritative of that site.Another case is a real
routring problem of my office
which lasts for two days due to the middle transit (Hurricane Electric) not
acept my
company's prefix. There are other cases may introduced by my co-author who
have
more experince run ipv6 network.

Maybe you (and other guys) missed that the most important argument in this
draft is if HE is
not widely deployed, NHE can enable ISP to deloy it and gain.

> As for CDN statements made in this thread: CDNs are in the business of
> optimizing connections to their content. HE isn't about optimizing
> connections -- it's about more quickly reacting to the likelihood of
> outright brokenness. But it sounds like NHE *is* about optimizing (where
> IPv6 isn't broken, but is just really slow)? I'm a bit confused about this.
>

As to this question,  NHE does the same thing HE did racing the IPv6 and
IPv4 connections to avoid both broken and slow IPv6. But NHE does
optimizing as well in terms of stopping sending additional traffic if IPv6
connection is slow (not broken) , becuase AAAA record will not be recieved
or AAAA record is received too late (one promising direction for next
version of NHE draft).

Davey

---

From: Davey Song <[email-masked]>
Date: Thu, 27 Sep 2018 11:28:45 +0800

Hi Tommy,

Thanks for your suggestions.

On Wed, 26 Sep 2018 at 23:38, Tommy Pauly <[email-masked]> wrote:

> +1 to the point that the proposal for NHE is essentially a mechanism for
> the ISP and/or content provider to work around a broken deployment that
> they should be in a position to fix themselves, or between one another as
> Tony describes.
>

 Fair enough. I think it is worth of doing it.

The part that remains interesting is suggestions for networks to collect
> more reports on the relative health of IPv4 and IPv6 connections, to help
> fix bugs and broken deployments. Focusing on these improvements rather than
> filtering DNS answers seems like it would be a more fruitful direction.
>

>From the feadback of this list, now I'm think of dividing NHE draft into
two pieces: One is network-side IPv6 measurment and failure reporting which
is in aim to provide information to workaround broken deployment in a
position to fix by ISP themselves. ISP can decide by themself to take
advantege of measurement information. It can be standard track or
informational.

Another one is Selectively delaying AAAA record other than filtering DNS
records. It served an experimental/informational case of ISP taking IPv6
mearement infromation as a feed to quickly and temporarily fix the problem
if operator think it is neccesary.

How does this sound ?

Davey

---

From: Simon Hobson <[email-masked]>
Date: Thu, 27 Sep 2018 13:10:24 +0100

Fred Baker <[email-masked]> wrote:

> So, what do I expect the next few years to look like?

I largely agree with your predictions, contrary to the naysayers IPv6 is here to stay and while it's going to take a long time, it's going to become dominant.
As to timescales, I don't think we'll see the end of IPv4 for a long long long time - and having watched the lack of activity from my own ISP (who isn't exactly a dinosaur) I think early 2020s for IPv4 to be relegated to an overlay/tunnelled/whatever protocol for "most" networks is probably a bit optimistic.

And then I think back to last year when I was having a bit of a play with Azure at work, and I don't recall seeing any mention of IPv6 at all with that. Indeed, from what I could see, their model for providing service mobility relies on IPv4 and NAT. I imagine the same is true to some extend with all the large providers working with flexible pools of resources.
That sort of thing is going to be a drag on ditching IPv4.

On the upside, I've seen hosting providers where IPv6 is included and IPv4 is a paid for extra. Or even IPv6 only with NAT64 to reach the IPv4 internet https://ipv6onlyhosting.com/en-us/cms/

I think the key single point that's going to drag things out compared to how IPv4 killed off things like AppleTalk and IPX is that IPv4 offered a compelling advantage (global routing, ubiquitous support once it and "the internet" "took off") - while for many people, IPv6 is just "the same thing with added complexity".

---

From: Gert Doering <[email-masked]>
Date: Thu, 27 Sep 2018 14:13:03 +0200

Hi,

> So the suffering of users is real in dualstack.

This is a misconception.

"A few applications are lazy, so performance for them is poor" is better
describing things.

Gert Doering
        -- NetMaster

---

From: Gert Doering <[email-masked]>
Date: Thu, 27 Sep 2018 14:18:39 +0200

Hi,

On Wed, Sep 26, 2018 at 05:31:31PM -0700, Fred Baker wrote:
> ICANN didn't approve a policy for distribution of prefixes to RIRs, and 
> the RIRs to networks, until 2006, per OECD. 

First /35 handed out by the RIPE NCC was in August 1999.

(I agree with the general statement, but these numbers are just not right)

The formal "/12 to the RIRs" policy took a few years longer, so ICANN was
operating under a provisional "/23 (IIRC) to each RIR" policy at the time,
but nevertheless, these was a regular RIPE policy and the prefixes are
official and valid.

(I'm a bit sensitive on that topic, as a large part of the excuses why
people haven't deployed IPv6 is "because assignment/allocation policies
are not right!!!" - which just wasn't true for the majority of use cases
for well over 10 years now)

Gert Doering
        -- NetMaster

---

From: Fred Baker <[email-masked]>
Date: Fri, 28 Sep 2018 08:49:43 -0700

Thanks for the correction. The OECD document I found the date in was probably referring to the /12 to the RIRs policy.

> On Sep 27, 2018, at 5:18 AM, Gert Doering <[email-masked]> wrote:
> 
> Hi,
>

---

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 28 Sep 2018 19:01:14 +0200

For me it looks quite challenging to develop a meaningful vision in this
space.

On one hand, this Internet witnesses fast digital evolutions, and
disruptions, every other day. On another hand IPv6 overcoming IPv4 seems 
to advance very slowly. On a same timescale Windows leaped from 95 to 
10, while IP still steps from 4 to 6.

In that sense, I think IPv6 and IPv4 side by side as a double stack is
still a viable evolution in the next decades, although not very fancy.
The translation from one to another seems like an artificial path that
keeps reinventing itself but with no clear advantage of one translation
mechanism over another. These IPv4-IPv6 translation mechanisms are like
satisfaction for our needs of getting there faster.

(but I also speculated at a time that Twitter would disappear in its
infancy because finger existed - the reverse happened; I also speculated
that iPhone 4 would do _only_ IPv4 and iPhone 6 _only_ IPv6 - it did not
happen).

Le 27/09/2018 à 02:31, Fred Baker a écrit :
>> As IPv4 winds down, yes, HE as described will become anachronistic
>> - there won't be a choice to make. It will become, as NAT64 will, a
>> service that is more work to remove than it's worth, but eventually
>> isn't implemented in new implementations and nobody notices.
> 
> This time I'm speaking as a co-chair of the working group. This is in
> part a question: if folks don't see things the way I do, I, and we,
> need to understand the various viewpoints. But it's also a statement
> of what I expect the next few years to look like.
> 
> 
> As to the status of IPv6, let me comment on its history. I saw an
> article the other day whose headline announced that "IPv6 has failed;
> let's pick something else." RFC 2460 was published in 1998, but it
> didn't define IPv6 as a usable protocol. It needed IPv6-in-DNS,
> IPv6-in-DHCP, IPv6-in-your-favorite-routing-protocol, vendor
> implementation, working vendor implementation (oh, is that
> important?), random features like uRPF etc, ROAs and IRRs, and a long
> list of whatever else. ICANN didn't approve a policy for distribution
> of prefixes to RIRs, and the RIRs to networks, until 2006, per OECD.
> In 2007, about 350 IPv6 prefixes were allocated by ARIN for testing
> purposes, and in 2009-2011 there was a "run on the bank" in
> Asia-Pacific as the result of a JPNIC study indicating that IPv4
> depletion was a realistic scenario.
> 
> In other words, the development of the technologies implied by IPv6
> took almost as long to develop as the technologies required to use
> IPv4 (and involved a few mis-steps). However, the impetus to use them
> was much lower - whenever a technology was unavailable to make IPv6
> work, "well, you can always use IPv4". IPv6 didn't become "important"
> until "always" became untrue. That started in 2011, when the IPv4
> address space available from the IANA (and eventually the RIRs) was
> depleted; it is becoming increasingly untrue as the cost of
> purchasing an IPv4 address spirals upward.

I would like to react to this.

We should reckon frankly what happened with this depletion.  It has been 
hugely advertised.  It is a few years now.

It depleted.  But it is still possible to get IPv4 as long as the unused 
space gets back.

We can talk about refuelling IPv4.

Maybe we need to wait for several cycles of depletion-refuelling in 
order to sense a stronger need of IPv6.

Alex

> 
> "Pick something else" - I have to wonder what "something else" might
> be in view. We have a number of research projects on the table, such
> as ICN/CCN/NBN, and they likely have their uses. But I don't see the
> architectural or business implications following through; if they
> were rationally available, they would have started a decade back, I
> suspect. So I see IPv6 deployment as the only rational option on the
> table.
> 
> 
> What I am increasingly hearing from ISPs and other networks is that
> it is less expensive to deploy an IPv6-only network with (or without)
> an IPv4 overlay  than to deploy a dual stack network. There are a few
> networks that have drawn that conclusion; as more and more do, IPv4
> will become less of a requirement.
> 
> I'm not sure what to expect next, and maybe people can inform me. One
> option would be two networks that implement IPv6 natively and IPv4 as
> an overlay service connect, meaning that the address of a 464XLAT
> PLAT (or whatever) from one carrier is an address in a different
> carrier for some part of one of their networks. Or, maybe, a network
> simply stops providing an IPv4 PLAT or other support for IPv4 as an
> overlay. Maybe they are running lw4o6 and it becomes necessary to do
> IPv4 routing inside that overlay, so that they want to charge for it
> as a service. The obviously implied next question is "when" - I'm
> guessing early 2020s or something like that.
> 
> 
> So, what do I expect the next few years to look like? I see three
> broad classes of ISPs, IXPs, and networks at the edge: those that see
> IPv6 as the least expensive endpoint and drive to that end, those
> that have enough IPv4 for the moment and aren't thinking any further
> ahead than that, and those that are balanced on a knife's edge in
> between. We have those now. I expect to see a shifting balance, with
> different markets adopting different perspectives and changing over
> time. I see a slow but definitely discernible motion from "IPv4
> works" to "https://www.youtube.com/watch?v=_y36fG2Oba0" "I suppose we
> ought to give it a try". One at a time, I hear operators say "I don't
> want to buy any more IPv4 addresses". The networks that are fixed in
> the "IPv4 laggard" camp tend to be networks that aren't faced with
> IPv4 address purchases and ask for a business justification that
> doesn't involve a monetary outlay.
> 
> Ron, Lee, and I asked for a change to the v6ops charter to discuss
> IPv6-only networking with that in view. What I want to see in v6ops
> is more of that - lets move IPv4 to an overlay service, and watch it
> become less relevant over time.
> 
> So I personally expect transition technologies of various kinds -
> lw4o6, 464xlat, MAP-E, MAP-T, and so on - to remain important, but I
> expect that they will become the aftermarket more than the market. If
> cloud services are accessible by IPv6, I expect residential access to
> become IPv6 service to become larger and IPv4 to become smaller. If
> you want a picture of that, look at Trinidad and Tobago. As I
> understand it, there are some services (Google, Facebook, etc) that
> only offer IPv6 access there, so the "IPv6" picture is a picture of
> their market dominance.
> https://stats.labs.apnic.net/ipv6/CC?x=1&s=1&p=1&w=30&c=TT
> 
> I expect service providers to do as little with IPv4 as they can get
> away with, and for IPv4 to increasingly become ragged around the
> edges. There are still a variety of places where I would get tarred
> and feathered and sent home wearing a barrel for saying that, but the
> SNA people were of that opinion too - until APPN made SNA networks
> irrelevant and people started deploying IP-native applications. 
> Defeated warriors go to war first and then seek to win. Sun Tzu
> 
> 
>

---

From: Lee Howard <[email-masked]>
Date: Fri, 28 Sep 2018 13:47:23 -0400

On 09/28/2018 01:01 PM, Alexandre Petrescu wrote:
> disruptions, every other day. On another hand IPv6 overcoming IPv4 
> seems to advance very slowly. On a same timescale Windows leaped from 
> 95 to 10, while IP still steps from 4 to 6.
Windows 95 was a significant rebuild over Windows 3.11.  Each of the 
next several versions were incremental upgrades, and all were both 
forward and backward compatible.

Meanwhile, lots of work had to go into making IPv6 deployable, and it 
didn't really start until 2011. In that time, Windows has evolved from 
Windows 7 to Windows 10 (skipping 9, so only two desktop upgrades).

> satisfaction for our needs of getting there faster.

How does a network run dual-stack when it's out of IPv4 addresses?
If it buys addresses, why does it run dual-stack?
I guess you consider IPv6-only plus translation to be dual-stack?

Agreed, for now different people are choosing different mechanisms, and 
there isn't yet a clear winner.

> that iPhone 4 would do _only_ IPv4 and iPhone 6 _only_ IPv6 - it did not
I haven't seen finger enabled on a system since 1992.

> We should reckon frankly what happened with this depletion.  It has 
> been hugely advertised.  It is a few years now.
>
> It depleted.  But it is still possible to get IPv4 as long as the 
> unused space gets back.
At about US$20 each.

> order to sense a stronger need of IPv6.

Right now, the cloud and CDN companies are essentially all of the IPv4 
demand. In terms of number of addresses and dollars spent, they are more 
than 90% of the market. The CDNs are mostly okay in IPv6 support, even 
if they ingest over IPv4.
The cloud providers are in a pile of trouble. AWS is way ahead of anyone 
else for IPv6 support, and last I heard they're still buying IPv4 
addresses to make sure they have enough cushion. Azure can do a VM or a 
Load Balancer, but not both, which is more than Google Cloud can do 
(only LB). Oracle will allow you to open a ticket requesting IPv6, but 
unless it's a Service Cloud, they'll close the ticket with no action. 
Alibaba promises an IPv6-only cloud by 2025, but so far has nothing but 
dreams.

Something like seven years ago I predicted 500M IPv4 addresses in the 
market. About half have moved, but the market didn't really get moving 
until about 2015. It's important to note that large net blocks are 
valued by large companies: nobody wants to manage hundreds of /18s. 
Unless the US government/Department of Defense releases several /8s, the 
high end of the market is all but gone.
Yes, there may be thousands of /16s still available, for over a million 
USD each and rising.

2018 has seen an inflection in pricing. But of course, nobody knows what 
happens next. My concern is that instead of "permissionless innovation" 
the IPv4 Internet will require significant capital just to get connected.

We're also on track for 50% of the world to reach Google over IPv6 in 
two years. Several mobile networks are already IPv6-only internally, and 
several large landline ISPs are either there or on their way.

So, what do you advocate we wait and see? How long should we wait, and 
what are we looking for?
More importantly, once we see it, how much time will we have to do 
something about it, and will it be enough time?

Lee

---

From: Fred Baker <[email-masked]>
Date: Fri, 28 Sep 2018 11:26:25 -0700

> still a viable evolution in the next decades, although not very fancy.

As I theorized the other day, I suspect that the IPv4 Internet becomes something strictly at the edge. It might very well be dual stack within the edge network, but it will come to a NAT of some kind before it leaves the premises. The ISPs run IPv6-only, and they serve residential and mobile customers/services using it. CLAT-PLAT translation means that the ISP doesn't have to think about it too much; lw4o6 tunnels become a network of their own that someone has to manage. As much as I dislike LISP, LISP might be a rational tunneling technology; it allows for the dynamic creation of tunnels whenever and wherever needed, which would be useful for enterprise connection to its hosting sites and business partners.

Also, I suspect that said aftermarket business becomes a PITA that people charge to support. Given the choice of a bill for one protocol or a bill for two separate protocols with some sort of PLAT or tunnel endpoint support built in, expect customers to stop asking about a business case and start asking about reducing their costs.

---

From: Ole Troan <[email-masked]>
Date: Fri, 28 Sep 2018 20:47:37 +0200

Fred,

> On 28 Sep 2018, at 20:26, Fred Baker <[email-masked]> wrote:
> 
> it. CLAT-PLAT translation means that the ISP doesn't have to think about it too much; lw4o6 tunnels become a network of their own that someone has to manage.

I don’t understand this statement. 
You seem to imply that there should be less management with 464XLAT aka double translation than tunneling? Why do you believe that?

Ole

---

From: Fred Baker <[email-masked]>
Date: Fri, 28 Sep 2018 12:32:00 -0700

> On Sep 28, 2018, at 11:47 AM, Ole Troan <[email-masked]> wrote:
> 
> Fred,
> 

With translation, there are two translators. With a tunnel, there are two tunnel endpoints. I see the number two in both statements.

In addition, in both cases the connection gets to an IPv4 Internet. However, at some point, that network starts to break down. If, for example, it becomes a set of tunnels leading to each neighboring ISP, then there is routing somewhere in the lw4o6-or-whatever system to select the right neighboring ISP. If it's translation, then that routing can be accomplished using the IPv6 prefix plus some subset of the IPv4 address in question.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 28 Sep 2018 17:30:33 +0000

Hi Davey,
Thanks for the info. That’s enlightening.
I would not support this as either a standard or a best practice. The proposed NHE mechanism appears to be a capability only of interest to ISPs in countries/regions where (because of how they have chosen to set up their IPv6 peering) they are impacted by the actions of specific transit providers (and other ISPs) doing bad IPv6-related things; and hosts aren’t doing HE. I would be fine with something informative along the lines of “if an ISP finds itself in this situation, here is a work-around they might consider doing, as a last resort (after futilely trying to get the underlying causes fixed or trying to ensure multiple transit paths are available)”. Designing a network with a single point of failure is never a good idea. But the preferred approach should be to recommend connecting to multiple IPv6 peers, rather than using IPv4 as the backup to failure of a single IPv6 peer.

The map makes this problem look very regional, national, or even company-specific. I don’t think it’s a good idea to create standards or best practices for work-arounds to regional/national/company-specific problems.

I did see the email from Tony pointing to problems inside the US and Europe. But they don’t seem impactful enough for ISPs in these regions to be wanting to do this sort of work-around.
Barbara

From: Davey Song <[email-masked]>
Sent: Wednesday, September 26, 2018 11:10 PM
To: STARK, BARBARA H <[email-masked]>
Cc: Tony Finch <[email-masked]>; Gert Doering <[email-masked]>; dnsop <[email-masked]>; [email-masked]
Subject: Re: [DNSOP] [v6ops] New Version Notification for draft-v6ops-xie-network-happyeyeballs-00.txt

Hi Barbara,

Thanks for your comments and sharing. I need to note that NHE is not competing with
Client-side HE or competing with deployment of fully functioning IPv6. The background of
NHE is :

1) the dualstack is real. Client is facing the choice ipv4 or ipv6.
2) the poor or unpredictable IPv6 performance is real (https://stats.labs.apnic.net/v6perf<https://urldefense.proofpoint.com/v2/url?u=https-3A__stats.labs.apnic.net_v6perf&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=LoGzhC-8sc8SY8Tq4vrfog&m=UClJsJsCXhi0G8zfGtyd-jc6xlVyeP-pDfD9cuU89OU&s=ciUdcpoxV8G6M_g6RNYgM_3VzGft9mc2TVlGm3EQKy0&e=> ).
3) Low adoption of Client-side HE is realy.

What we are trying to do is to make the dualstack less painful for users in these regions.
We can not say "you are starving or poor becuase you are not rich". We can not jump
directly to build a fully functioning IPv6 in one day (like roma city). I feel so sad the region
I live failed to catch up the average level of IPv6 development. But it is true the IPv6
development situation varies in different places.

Maybe your observation is true that local ISP is the major source of broken IPv6. However,
though I'm not network operator, I had at least two observations of IPv6 failure caused
by far-end and midst. One case is a real user complaint from China telecom. It proved
that user experinced long delay when Servfail is received when asking AAAA for many times.
It is the problem of authoritative of that site.Another case is a real routring problem of my office
which lasts for two days due to the middle transit (Hurricane Electric) not acept my
company's prefix. There are other cases may introduced by my co-author who have
more experince run ipv6 network.

Maybe you (and other guys) missed that the most important argument in this draft is if HE is
not widely deployed, NHE can enable ISP to deloy it and gain.

As to this question,  NHE does the same thing HE did racing the IPv6 and IPv4 connections to avoid both broken and slow IPv6. But NHE does optimizing as well in terms of stopping sending additional traffic if IPv6 connection is slow (not broken) , becuase AAAA record will not be recieved or AAAA record is received too late (one promising direction for next version of NHE draft).

Davey

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 29 Sep 2018 12:07:33 +1200

> How does a network run dual-stack when it's out of IPv4 addresses?

I think it all depends on what you mean by "network". The network I'm
on right now has all the IPv4 addresses it can possibly need, and my laptop
sees a full dual stack service. It doesn't seem to care that its IPv4
address is in 192.168.178.0/24. It also has no idea whether the upstream
link is dual stack or not. (My upstream, above the NAT, is dual stack,
but my laptop doesn't need to know that.)

OK, I'm belabouring the obvious, but I want to cement the notion that
today and for many years to come, most hosts need to see a dual stack
network for everything to more or less work. Nothing new there, I've
been saying so since 1994 in my first RFC. (Well, there's a bit new,
in that 464XLAT is now an alternative to tunnels as a way to simulate
a dual stack if you don't actually run one. My laptop wouldn't know or
care if the CE used 464XLAT.)

> I guess you consider IPv6-only plus translation to be dual-stack?

If it looks that way to the end host, yes. Dual-stack isn't a religion,
it's a coexistence technique. Although people insist on using the
T-word (transition), everything's been about coexistence all along,
and still is.

> there isn't yet a clear winner.

And there doesn't need to be, as far as I can see.
 
> something about it, and will it be enough time?

Obviously I (Brian) don't think we should wait and see. But we have to
clearly separate the separable problems. I won't try and do that right
here, except that it's very clear that moving cloud services to IPv6
is an utterly different problem from moving user LANs and end systems to
dual stack coexistence. I think we (v6ops) do need another round of problem
analysis.

   Brian

---

From: Ole Troan <[email-masked]>
Date: Sat, 29 Sep 2018 08:06:42 +0200

Fred,

> On 28 Sep 2018, at 21:32, Fred Baker <[email-masked]> wrote:
> 
> 
> 

A future where IPv4 routing is injected into IPv6 routing. Didn’t we drop that idea when deprecating auto-tunnels and IPv4 compatible addresses?

It might be one future, but personally I doubt the “network starts to break down” part. 

Cheers 
Ole