Subject: [MMUSIC] ICE candidate address selection update draft

From: Ari Keranen <[email-masked]>
Date: Wed, 01 Aug 2012 16:35:22 -0700

Folks,

Unfortunately there was not enough time to present the updated ICE 
address selection draft during the WG session today. You can still check 
out the slides from here:
http://tools.ietf.org/agenda/84/slides/slides-84-mmusic-7.pdf

And the draft here:
http://tools.ietf.org/html/draft-keranen-mmusic-ice-address-selection-01

The draft seems to be pretty mature, is fixing a known problem, and the 
MMUSIC WG is the obvious place to do the work, so WG adoption would be 
the natural next step.

Give the doc a quick read if you already haven't (it's only couple of 
pages), let us know if there's something to fix, and if you think it's a 
good idea, feel free to +1 for WG adoption.

Cheers,
Ari

---

From: Simon Perreault <[email-masked]>
Date: Wed, 01 Aug 2012 16:54:19 -0700

> good idea, feel free to +1 for WG adoption.

+1 for WG adoption

I just read the draft. It's simple, it makes sense, and it fixes a real 
bug in the ICE spec.

One comment:

>    o  Candidate addresses from Unique Local Addresses (ULAs) MUST NOT be
>       combined with any other candidates except other ULA candidates.

That would fail if you're behind an NPTv6 thingie that maps the ULA to a 
global prefix. So maybe remove that rule.

>    o  Local relayed candidates MUST NOT be combined with remote host
>       candidates from IPv4 private address space [RFC1918] or IPv6 link-
>       local addresses or ULAs.

Same comment here about the "or ULAs" part.

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: Ari Keranen <[email-masked]>
Date: Wed, 01 Aug 2012 19:00:18 -0700

Hi Simon,

Thanks for the review!

> global prefix. So maybe remove that rule.

(Without going into discussion why an NPTv6 thingie is probably a bad 
idea :) running STUN should give one the global address. Then, that 
address should be used instead of the ULA with peer's global addresses.

Cheers,
Ari

---

From: Simon Perreault <[email-masked]>
Date: Wed, 01 Aug 2012 21:58:28 -0700

> address should be used instead of the ULA with peer's global addresses.

Hmmm... Right so STUN will never give a ULA. The only time you will have 
a ULA candidate will be for host addresses. And you still want to try 
those matched with your peer's global addresses.

Simon

---

From: "Pal Martinsen (palmarti)" <[email-masked]>
Date: Thu, 2 Aug 2012 12:33:56 +0000

+1

.-.
Pål-Erik

On Aug 2, 2012, at 1:35 AM, Ari Keranen <[email-masked]> wrote:

> Folks,
> 
> Unfortunately there was not enough time to present the updated ICE address selection draft during the WG session today. You can still check out the slides from here:
> The draft seems to be pretty mature, is fixing a known problem, and the MMUSIC WG is the obvious place to do the work, so WG adoption would be the natural next step.
> 
> Give the doc a quick read if you already haven't (it's only couple of pages), let us know if there's something to fix, and if you think it's a good idea, feel free to +1 for WG adoption.
> 
> 
> Cheers,
> Ari
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Ari Keranen <[email-masked]>
Date: Fri, 03 Aug 2012 11:58:00 -0700

> those matched with your peer's global addresses.

Yes, you should not get ULAs from the STUN server as long as the STUN 
server is properly located (on the Internet, outside of the organization 
perimeter and on the other side of any NATs). And the peer reflexive 
candidate (which would be global address) can be then matched with 
peer's globals.

Cheers,
Ari

---

From: Ari Keranen <[email-masked]>
Date: Fri, 03 Aug 2012 12:01:29 -0700

On 8/3/12 11:58 AM, Ari Keranen wrote:
> And the peer reflexive candidate

...meant to say "server reflexive candidate"

---

From: Simon Perreault <[email-masked]>
Date: Fri, 03 Aug 2012 12:03:40 -0700

> candidate (which would be global address) can be then matched with

But reflexive candidates don't always work. I know I said NPTv6, but we 
shouldn't prevent ICE from working behind evil NAT66. And an ICE client 
may not be able to reach a STUN server so you can't rely on having 
reflexive candidates. So you should still try to match ULAs with globals.

Simon

---

From: Ari Keranen <[email-masked]>
Date: Fri, 03 Aug 2012 12:27:53 -0700

> reflexive candidates. So you should still try to match ULAs with globals.

OK, this is quite a corner case, but I'm afraid you're right. However, 
if you have a global address on the interface (i.e., you're not behind 
an evil NAT66), matching ULAs with globals doesn't make sense. So, I'd 
suggest text along the lines of:

    o  Candidate addresses from Unique Local Addresses (ULAs) SHOULD NOT
       be combined with any other candidates except other ULA candidates.
       However, if an interface does not have any global addresses, the
       ULA SHOULD be used.

(changed MUST to SHOULD and added the second sentence)

Cheers,
Ari

---

From: Jonathan Lennox <[email-masked]>
Date: Sat, 4 Aug 2012 04:09:55 -0400

> (changed MUST to SHOULD and added the second sentence)

Remembering that the two ICE endpoints need to agree on which candidates match, does this mean that I need to know whether my peer's ULA and my peer's global address are on its same interface?

As a more general comment, one of the virtues of ICE is that by matching everything to everything, you're guaranteed to find a route, if one's possible, no matter how stupid or evil your network architects have been.  I don't think we want to break that property for IPv6, except in cases where it's absolutely impossible (e.g., due to host behavior) for a route to work.  If there's any circumstance where a network admin being "helpful" could make a connectivity check on a candidate pair succeed, I think we want to try that pair.  Even if the IETF says that the network MUST NOT be configured so as to make such a route work.

That said, if there were a well-defined algorithm to prioritize sensible checks over absurd ones, that'd be fine.

I'm afraid I don't understand v6 addressing well enough to make concrete text suggestions based on this principle, though.

--
Jonathan Lennox
[email-masked]

---

From: "Pal Martinsen (palmarti)" <[email-masked]>
Date: Mon, 6 Aug 2012 08:02:48 +0000

+1 

I think you make a very good point here. Just make sure we don´t encourage bad/evil network configurations. Those configurations should finish their connectivity checks last.

.-.
Pål-Erik

---

From: Ari Keranen <[email-masked]>
Date: Mon, 06 Aug 2012 17:06:29 +0200

On 8/4/12 10:09 AM, Jonathan Lennox wrote:
>>>> Yes, you should not get ULAs from the STUN server as long as
>>>> the STUN server is properly located (on the Internet, outside
>>>> of the organization perimeter and on the other side of any
>>>> NATs). And the peer reflexive candidate (which would be global
>>>> address) can be then matched with peer's globals.
>>>
>>> But reflexive candidates don't always work. I know I said NPTv6,
>>> but we shouldn't prevent ICE from working behind evil NAT66. And
>>> an ICE client may not be able to reach a STUN server so you can't
>>> rely on having reflexive candidates. So you should still try to
>>> match ULAs with globals.
>>
>> OK, this is quite a corner case, but I'm afraid you're right.
>> However, if you have a global address on the interface (i.e.,
>> you're not behind an evil NAT66), matching ULAs with globals
>> doesn't make sense. So, I'd suggest text along the lines of:
>>
>> o  Candidate addresses from Unique Local Addresses (ULAs) SHOULD
>> NOT be combined with any other candidates except other ULA
>> candidates. However, if an interface does not have any global
>> addresses, the ULA SHOULD be used.
>>
> Remembering that the two ICE endpoints need to agree on which
> candidates match, does this mean that I need to know whether my
> peer's ULA and my peer's global address are on its same interface?
>
> As a more general comment, one of the virtues of ICE is that by
> matching everything to everything, you're guaranteed to find a route,
> if one's possible, no matter how stupid or evil your network
> architects have been.  I don't think we want to break that property
> for IPv6, except in cases where it's absolutely impossible (e.g., due
> to host behavior) for a route to work.  If there's any circumstance
> where a network admin being "helpful" could make a connectivity check
> on a candidate pair succeed, I think we want to try that pair.  Even
> if the IETF says that the network MUST NOT be configured so as to
> make such a route work.

You have a good point. My main concern is that when you have a long list 
of different kind of IPv6 addresses for each interface (global, 
link-local, ULA, some temporary addresses, etc.), you will end up with a 
huge checklist of host-host candidates (which have the highest 
priority). If you have broken v6 connectivity, it's gonna take ages 
before you move to v4 candidates that work.

That being said, perhaps we should keep matching ULAs to globals but 
just give that combination a lower priority than other host-host 
candidate pairs. Link-locals we should still match only to link-locals 
(right?).

> That said, if there were a well-defined algorithm to prioritize
> sensible checks over absurd ones, that'd be fine.

This is the tricky part. Right now the candidates use same priority 
regardless of the other candidate's type. What we'd like to say is "for 
ULA candidate with ULA pair you need priority X and with other types 
priority Y". Then another question is what's the sensible trade-off 
between complexity of prioritization and making these corner cases work..

> I'm afraid I don't understand v6 addressing well enough to make
> concrete text suggestions based on this principle, though.

I can try to craft something for the next revision. All suggestions are 
welcome of course.

Cheers,
Ari

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 6 Aug 2012 22:07:53 -0400

> candidate pairs. Link-locals we should still match only to link-locals 

I understand the problem, and I agree it'll be an issue.

Furthermore, if you need to fall back to v4, it's not enough to just
prioritize all the weird cases below the other host-host pairs -- the v4
address that works will probably be server-reflexive, which you'd prefer.

(If hosts really treat link-local as link-local -- i.e. you'll never have a
router for link-local, and trying to send from link-local to non-link-local
is a black hole or socket error at the sender -- then yes, there's no point
in pairing them with non-link-local addresses.  That's the case I was
thinking of as "impossible")

> between complexity of prioritization and making these corner cases work..

My quick-and-dirty idea would be to take all the candidate pairs that the
current draft lists as "MUST NOT" combine, and put them in a secondary
("stupid") check list which follows all the other checks.  Within the
secondary list, checks are in the standard ICE priority order.

One question would be what the relative priority of weird v6 combinations
vs. v4 relays should be.

-- 
Jonathan Lennox
[email-masked]

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 7 Aug 2012 00:15:04 -0700

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Jonathan Lennox
> Sent: Monday, August 06, 2012 7:08 PM
> To: Ari Keranen
> Cc: [email-masked]
> Subject: Re: [MMUSIC] ICE candidate address selection update draft
> 

Mine would be to take the list of IPv6 and IPv4 addresses and try them
in the order described by ICE (which currently recommends following
the OS's default, which is sometimes hard to get depending on the
OS).  But if the first IPv6 candidate didn't return a connectivity
checks quickly (let's say, 150ms), initiate a connectivity check
on the highest priority IPv4 address next.  In that 150ms, based
on ICE's pacing, many IPv6 addresses will have been tried.  150ms
gives plenty of time for IPv6 to 'win', before using an IPv4
resource that is likely shared with IPv4-only devices.

-d

---

From: Jonathan Lennox <[email-masked]>
Date: Tue, 7 Aug 2012 05:53:08 -0400

On Tuesday, August 7 2012, "Dan Wing" wrote to "'Jonathan Lennox', 'Ari Keranen', [email-masked]" saying:

> the OS's default, which is sometimes hard to get depending on the OS).  
> But if the first IPv6 candidate didn't return a connectivity checks 
> quickly (let's say, 150ms), initiate a connectivity check on the 
> highest priority IPv4 address next.  In that 150ms, based on ICE's 
> pacing, many IPv6 addresses will have been tried.  150ms gives plenty 
> of time for IPv6 to 'win', before using an IPv4 resource that is 
> likely shared with IPv4-only devices.

Can't this result in the endpoints getting out of sync with the order of the checklist?  If one side has started IPv4 while the other hasn't, you won't have the outbound packets coming from one side to create the port bindings.

-- 
Jonathan Lennox
[email-masked]

---

From: "Pal Martinsen (palmarti)" <[email-masked]>
Date: Tue, 7 Aug 2012 18:30:19 +0000

>From a ICE multi platform library developer that is somewhat of a nightmare. 
Guess it can be solved by having a reasonable interface that the application 
that uses the library can fill in the defaults. Having reasonable default values
is important as some platforms might not be able to set the values. 

> Can't this result in the endpoints getting out of sync with the order of the checklist?  If one side has started IPv4 while the other hasn't, you won't have the outbound packets coming from one sid…
It can at least potentially introduce more "kamikaze" packets and delay the result. The STUN transaction have 10 retransmits and would timeout long after the checklist is empty.

To maintain the speed of ICE negotiation and keep call setup times low I think we should consider looking at the pacing of the conn checks. The rationale behind the pacing was not to overwhelm the NATs. Have the world move slightly towards better NATs? Looking from the perspective of an endpoint sending 1080p60 video streams the pacing seem a bit conservative? And how does the different paths (IPv4, IPv6, multi homed) impact the pacing? Could we do more in parallel without overwhelm the NATs?

I am wondering if we could do something smart with prioritising the checklist and triggered checks to make sure IPv6 wins if there is connectivity? I have to little understanding if IPv6 and all the different addresses to clearly formulate the idea at the moment. (Hoping this may trigger something for someone.)

.-.
Pål-Erik

---

From: Tom Taylor <[email-masked]>
Date: Tue, 07 Aug 2012 14:58:56 -0400

People involved in this discussion might be interested in

  http://datatracker.ietf.org/doc/draft-ietf-6man-rfc3484bis/

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 8 Aug 2012 09:05:14 -0700

> From: Pal Martinsen (palmarti) [mailto:[email-masked]]
> Sent: Tuesday, August 07, 2012 11:30 AM
> To: Jonathan Lennox
> Cc: Dan Wing (dwing); Ari Keranen; [email-masked]
> OS).
> > Can't this result in the endpoints getting out of sync with the order
> of the checklist?  If one side has started IPv4 while the other hasn't,
> you won't have the outbound packets coming from one side to create the
> port bindings.
> >
> It can at least potentially introduce more "kamikaze" packets and delay
> the result. The STUN transaction have 10 retransmits and would timeout
> long after the checklist is empty.
> 
> To maintain the speed of ICE negotiation and keep call setup times low
> I think we should consider looking at the pacing of the conn checks.
> The rationale behind the pacing was not to overwhelm the NATs.

Yes.  And also to reduce the concern that ICE connectivity checks 
could flood a link.

> Have the
> world move slightly towards better NATs? Looking from the perspective
> of an endpoint sending 1080p60 video streams the pacing seem a bit
> conservative?

The problem, even at the time, was not pps or bandwidth throughput of
the NATs.  Rather, it was that NATs would not create new mappings
quick enough and drop the STUN Binding Request.  This would be re-
transmitted and make it through the NAT, but that retransmit was 
obviously harmful for call setup times.  As all the connectivity
checks are supposed to be sent from the same source address and
port on the client, a new mapping is only necessary for NATs which
have "address dependent mapping" or "address and port dependent
mapping" behavior.

I just checked, and couldn't find a test of UDP mapping speed
at http://nattest.net.in.tum.de/results.php, 
draft-jennings-behave-test-results, or 
http://eggert.org/papers/2010-imc-hgw-study.pdf.  Perhaps there
are tests available somewhere else.

> And how does the different paths (IPv4, IPv6, multi
> homed) impact the pacing? Could we do more in parallel without
> overwhelm the NATs?

I'm sure we could.

We could also be more aggressive with STUN retransmissions, and
less aggressive at trying new candidates.  That depends on our 
confidence that the candidates are in the proper order, and if
we can move a connection to a new candidate.  RTP can be moved
pretty easily; TCP cannot.  Before ICE completes, we could move
RTP sessions around without draft-wing-mmusic-ice-mobility.

It is probably worth backing up and enumerating the problem(s) we 
have with ICE, and then deciding what we want to solve.

> I am wondering if we could do something smart with prioritising the
> checklist and triggered checks to make sure IPv6 wins if there is
> connectivity? I have to little understanding if IPv6 and all the
> different addresses to clearly formulate the idea at the moment.
> (Hoping this may trigger something for someone.)

ICE was written before there was wide acceptance that the IPv6 path
is sometimes broken.  Tweaking ICE to better accommodate that reality
seems worthwhile.

-d

---

From: "Pal Martinsen (palmarti)" <[email-masked]>
Date: Thu, 9 Aug 2012 17:13:29 +0000

> are tests available somewhere else.
Thanks for clearing that up.

> RTP sessions around without draft-wing-mmusic-ice-mobility.
More agressive STUN retransmits would help speeding up things for RFLX 
candidates.

> have with ICE, and then deciding what we want to solve.
Yes, you are right.

Keeping the number of connectivity checks at a minimum by removing the 
ones that will never work as this documents describes is probably what we
want from this document. 

> is sometimes broken.  Tweaking ICE to better accommodate that reality

Prioritising IPv6 over IPv4 is all about choosing from the valid_list and that is 
currently up to the different implementations to decide. Is it worthwhile having 
a separate document describing that?

.-.
Pål-Erik

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 9 Aug 2012 10:40:33 -0700

> Sent: Thursday, August 09, 2012 10:13 AM
> To: Dan Wing (dwing)
> Cc: Jonathan Lennox; Ari Keranen; <[email-masked]>
> ones that will never work as this documents describes is probably what
> we want from this document.

Absolutely.

Solving additional problems might belong in other documents (e.g.,
faster STUN retransmissions).  

> currently up to the different implementations to decide. Is it
> worthwhile having a separate document describing that?

To my mind, that seems appropriate for this ICE Candidate Address
Selection Update document.  Right now ICE is pretty circumspect in
its IPv6/IPv4 recommendation and says to follow what the OS would
order, if possible, but doesn't provide strong recommendations if
the OS doesn't have an order.

I know there are one or two APIs in Windows that accept a bunch
of IP addresses and will re-order them.  I dunno if a similar API
is available for Android, Linux, or iOS/OS X.  Without an API on
those OSs, the ICE Candidate Address Selection Update is a good
place to make recommendations for those OSs.

-d

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 22 Aug 2012 17:59:05 -0700

> From: Jonathan Lennox [mailto:[email-masked]]
> Sent: Tuesday, August 07, 2012 2:53 AM
> To: Dan Wing; 'Ari Keranen'
> Cc: [email-masked]
> Subject: RE: [MMUSIC] ICE candidate address selection update draft
> 

Don't we already have that problem if both peers are dual-stack, and 
one side has many more IPv6 candidates than the other side?

But, to avoid the problem you describe, seems we should try IPv6
and try IPv4 in a somewhat parallel fashion.  Echos of Happy 
Eyeballs.

-d

---

From: Jonathan Lennox <[email-masked]>
Date: Thu, 23 Aug 2012 09:55:16 -0400

Dan Wing writes:

> Don't we already have that problem if both peers are dual-stack, and one side has many more IPv6 candidates than the other side?

No, because the check list consists of candidate *pairs*, and under the current ICE rules both sides see the same list of pairs, and sort them in the same order.

> and try IPv4 in a somewhat parallel fashion.  Echos of Happy 

ICE connectivity checks are inherently parallel, so some ordering like "likely v6 pairs", "v4 pairs", "unlikely v6 pairs" should accomplish this.

For the full Happy Eyeballs treatment, I suppose we could recommend that an endpoint prioritize its candidates based on which ones worked previously, but that's an optimization and doesn't need to be standardized (so long as these priorities are signaled in the standard manner).

-----Original Message-----
From: Dan Wing [mailto:[email-masked]] 
Sent: Wednesday, August 22, 2012 8:59 PM
To: Jonathan Lennox; 'Ari Keranen'
Cc: [email-masked]
Subject: RE: [MMUSIC] ICE candidate address selection update draft

> > pacing, many IPv6 addresses will have been tried.  150ms gives 
> > plenty of time for IPv6 to 'win', before using an IPv4 resource that 
> > is likely shared with IPv4-only devices.
> 
> of the checklist?  If one side has started IPv4 while the other 
> hasn't, you won't have the outbound packets coming from one side to 
> create the port bindings.

Don't we already have that problem if both peers are dual-stack, and one side has many more IPv6 candidates than the other side?

-d

---

From: "Pal Martinsen (palmarti)" <[email-masked]>
Date: Thu, 23 Aug 2012 14:24:17 +0000

> one side has many more IPv6 candidates than the other side?

But the checklist consist of candidate pairs. So the checklists on remote and local side should be fairly similar in size? 

> and try IPv4 in a somewhat parallel fashion.  Echos of Happy 
That is a nice speed optimisation.

.-.
Pål-Erik