Subject: Re: [v6ops] I-D Action: draft-ietf-v6ops-conditional-ras-01.txt

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Fri, 2 Mar 2018 16:21:32 +0000

Hi,

In Abstract:

"The problem of enterprise multihoming without address
   translation of any form has not been solved yet as it requires both
   the network to select the correct egress ISP based on the packet
   source address and hosts to select the correct source address based
   on the desired egress ISP for that traffic."

Should be:
"The problem of enterprise multihoming without address
   translation of any form has not been solved yet as it requires
   the hosts to select the correct source address based
   on a destination address for that traffic, 
   the hosts to select a first hop router based on the packet
   source address, and the network to select the correct egress ISP based on the packet
   source address"

This introduction would be more suited for Figures 2&3. If not, please clarify the intent of the draft.

3.2.1.  Single Router, Primary/Backup Uplinks

What "if{packet_destination_address is in 2001:db8:1::/48 or 2001:db8:2::/48}"? Will the next hop still be determined by the source address? If yes, is the destination address check needed?

3.2.2.  Two Routers, Primary/Backup Uplinks
Similar as above. 

Why would R1 have a policy that sends a packet to "next-hop is ISP_B_uplink", if it does not talk to ISP_B?

With R1 and R2 policies, H1 would have only one prefix all the time. How will H1 do load balancing or send one application stream over ISP_A and another over ISP_B, if needed? Maybe the first sentence needs to state that only one ISP can be used.

3.2.3.  Single Router, Load Balancing Between Uplinks
If load balancing is used, R1 sends packets to an ISP_x based on the source address. This means the host must know which ISP (source address) to select. How does host know this?

3.2.5.  Topologies with Dedicated Border Routers
"For load-balancing case ...
   prefix 2001:db8:2:1::/64 {
     if ISP_B_uplink_route is present then preferred_lifetime = 0
       else preferred_lifetime = 604800
    }"

Is this just copy and paste from the NON-load-balancing policy? Why would H1 use 2001:db8:2:1::/64 if "ISP_B_uplink_route is NOT present" and send traffic over R2 - R3 - ISP_A with 2001:db8:2:1::x source address?

Page 10:
Typo: "trgger"

Regards,
Dusan Mudric.

> -----Original Message-----
> From: v6ops [mailto:[email-masked]] On Behalf Of Jen Linkova
> Sent: Tuesday, February 27, 2018 3:03 PM
> To: [email-masked]
> Subject: Re: [v6ops] I-D Action: draft-ietf-v6ops-conditional-ras-01.txt
> 
> The draft (draft-ietf-v6ops-conditional-ras)has been updated to
> address comments received in Singapore.
> 
> Any feedback is appreciated, especially if you have any suggestions on
> how to improve the document.
> 
> On Wed, Feb 28, 2018 at 6:57 AM,  <[email-masked]> wrote:
> >
> > A New Internet-Draft is available from the on-line Internet-Drafts directories.
> > This draft is a work item of the IPv6 Operations WG of the IETF.
> >
> >         Title           : Using Conditional Router Advertisements for Enterprise
> Multihoming
> >         Authors         : Jen Linkova
> >                           Massimiliano Stucchi
> >         Filename        : draft-ietf-v6ops-conditional-ras-01.txt
> >         Pages           : 17
> >         Date            : 2018-02-27
> >
> > Abstract:
> >    This document discusses most common scenarios of connecting an
> >    enterprise network to multiple ISPs using an address space assigned
> >    by an ISP.  The problem of enterprise multihoming without address
> >    translation of any form has not been solved yet as it requires both
> >    the network to select the correct egress ISP based on the packet
> >    source address and hosts to select the correct source address based
> >    on the desired egress ISP for that traffic.
> >    [I-D.ietf-rtgwg-enterprise-pa-multihoming] proposes a solution to
> >    this problem by introducing a new routing functionality (Source
> >    Address Dependent Routing) to solve the uplink selection issue and
> >    using Router Advertisements to influence the host source address
> >    selection.  While the above-mentioned document focuses on solving the
> >    general problem and on covering various complex use cases, this
> >    document describes how the solution proposed in
> >    [I-D.ietf-rtgwg-enterprise-pa-multihoming] can be adopted for limited
> >    number of common use cases.  In particular, the focus is on scenarios
> >    where an enterprise network has two Internet uplinks used either in
> >    primary/backup mode or simultaneously and hosts in that network might
> >    not yet properly support multihoming as described in [RFC8028].
> >
> >
> > The IETF datatracker status page for this draft is:
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__datatracker.ietf.org_doc_draft-2Dietf-2Dv6ops-2Dconditional-
> 2Dras_&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhI
> oUWB8YLZU23029sMQGQ2kY&m=Tvwolb891s9E__RcQ05Srqho90KVleyHRFiv
> eiJiW4g&s=4GiE9981xqO7DwH-RffXvRXuxwZGkG_QrwoHaFUVsc4&e=
> >
> > There are also htmlized versions available at:
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_draft-2Dietf-2Dv6ops-2Dconditional-2Dras-
> 2D01&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIo
> UWB8YLZU23029sMQGQ2kY&m=Tvwolb891s9E__RcQ05Srqho90KVleyHRFive
> iJiW4g&s=1q5WZlStkpgafuIZcNyITzCYHMR9sxnxPjqRsanSv0w&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__datatracker.ietf.org_doc_html_draft-2Dietf-2Dv6ops-2Dconditional-
> 2Dras-
> 2D01&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIo
> UWB8YLZU23029sMQGQ2kY&m=Tvwolb891s9E__RcQ05Srqho90KVleyHRFive
> iJiW4g&s=3Yki_IS8Q3O9n3vWhd8JTcchlcUF0_8nCIMTKUFeBc8&e=
> >
> > A diff from the previous version is available at:
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_rfcdiff-3Furl2-3Ddraft-2Dietf-2Dv6ops-2Dconditional-
> 2Dras-
> 2D01&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIo
> UWB8YLZU23029sMQGQ2kY&m=Tvwolb891s9E__RcQ05Srqho90KVleyHRFive
> iJiW4g&s=EzOsCZBnjl_2C2UOT7sBNzyvAVc7XN7ULjrSXwlcL34&e=
> >
> >
> > Please note that it may take a couple of minutes from the time of submission
> > until the htmlized version and diff are available at tools.ietf.org.
> >
> > Internet-Drafts are also available by anonymous FTP at:
> > https://urldefense.proofpoint.com/v2/url?u=ftp-3A__ftp.ietf.org_internet-
> 2Ddrafts_&d=DwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iC
> rhIoUWB8YLZU23029sMQGQ2kY&m=Tvwolb891s9E__RcQ05Srqho90KVleyHR
> FiveiJiW4g&s=k-4jLyp-ruXqFPp_GKnLNTWz_VXishJWQtLDfIixeFU&e=
> >
> > _______________________________________________
> > v6ops mailing list
> > [email-masked]
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_v6ops&d=DwICAg&c=BFpWQw8bsuKpl1
> SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIoUWB8YLZU23029sMQGQ2kY&m=Tvwol
> b891s9E__RcQ05Srqho90KVleyHRFiveiJiW4g&s=jY0VQvQFNS1fCWqzD1ewRO
> p807KXejHGqSOM79nXAWA&e=
> 
> 
> 
> --
> SY, Jen Linkova aka Furry
> 
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_v6ops&d=DwICAg&c=BFpWQw8bsuKpl1
> SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIoUWB8YLZU23029sMQGQ2kY&m=Tvwol
> b891s9E__RcQ05Srqho90KVleyHRFiveiJiW4g&s=jY0VQvQFNS1fCWqzD1ewRO
> p807KXejHGqSOM79nXAWA&e=

---

From: Gert Doering <[email-masked]>
Date: Fri, 2 Mar 2018 19:56:57 +0100

Hi,

>    source address, and the network to select the correct egress ISP based on the packet

Actually the new text is completely missing the point.  Source address
selection *for a destination address* is something already addressed
nicely (RFC6724).

Selecting the right source address *to select the egress ISP to use* is
the part not addressed in host stacks at all yet.

The network part "select egress based on proper source address" is halfway 
solved with the homenet protocol suite (source address dependent babel, 
hncp, etc.) - but that is stuff not usually found on Enterprise routers.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Fri, 2 Mar 2018 22:29:13 +0000

" hosts to select the correct source address based
>    on the desired egress ISP for that traffic."
will not work in the load sharing example. Unlike the router, when the host has two prefixes it does not know which one is 'desired'. Possible option is to assign a prefix preference in RA.

"Actually the new text is completely missing the point. Selecting the right source address *to select the egress ISP to use* is
the part not addressed in host stacks at all yet"
If so, the draft needs to make it clear. The underlined assumption is that the host selected the right source address and the router uses that address to send a packet to a right ISP.

Regards,
Dusan.

> From: Gert Doering [mailto:[email-masked]]
> Sent: Friday, March 02, 2018 1:57 PM
> To: Mudric, Dusan (Dusan)
> Cc: Jen Linkova; [email-masked]

---

From: Jen Linkova <[email-masked]>
Date: Tue, 6 Mar 2018 13:53:52 +1100

Hi Dusan,

First of all, thanks for your comments!

>    source address, and the network to select the correct egress ISP based on the packet

As Gert has pointed out already, your text does not describe the
problem the draft is trying to solve.

1) "the hosts to select the correct source address based on a
destination address for that traffic," - this is default address
selection RFC.
If all information the host has (all information which needs to be
taken into account) is the destination IP and the set of source IPs,
then the RFC6724 describes how it should be done.

2) " the hosts to select a first hop router based on the packet source
address, and the network to select the correct egress ISP based on the
packet source address"

If hosts in the given multihomed network can use any/all uplinks (the
draft refers to it as 'load-balancing  scenario'), then yes, it would
work but only while all uplinks are available. As soon as one IPS
uplink can not be used (because it's failed or because of the policy -
the draft refers to it as 'active/backup uplink scenario') we need a
mechanism to close the feedback loop and signal back to hosts that
source addresses need to be selected based on the ISP uplink
available.

> What "if{packet_destination_address is in 2001:db8:1::/48 or 2001:db8:2::/48}"? Will the next hop still be determined by the source address? If yes, is the destination address check needed?

I see. I think I need to change 'next-hop' to 'default'. Obviously
intra-site routing should not be affected, this example policy is for
determining the ISP uplink only. All destinations are sent there, it's
about choosing between the default route pointing to the ISP_A and
default route pointing to the ISP_B.

I'll update the text to make it more clear.

> Why would R1 have a policy that sends a packet to "next-hop is ISP_B_uplink", if it does not talk to ISP_B?

Because R1 still may receive packets from hosts with source addresses
belonging to ISP_B address space.

> With R1 and R2 policies, H1 would have only one prefix all the time. How will H1 do load balancing or send one application stream over ISP_A and another over ISP_B, if needed? Maybe the first sente…

It's explained in
https://tools.ietf.org/html/draft-ietf-v6ops-conditional-ras-01#section-2.1
:
"    - Uplink(s) to one ISP is a primary (preferred) one.  All other
      uplinks are backup and are not expected to be used while the
      primary one is operational;

   -  If the primary uplink is operational, all Internet traffic should
      flow via that uplink;
"

Section 2 defines the scenarios the draft is applicable to, and then
Section 3 illustrates how the conditional RAs can be used in those
scenarios.

I'll add an additional reference to the Section 2 if you think it
makes the document more clear.

> If load balancing is used, R1 sends packets to an ISP_x based on the source address. This means the host must know which ISP (source address) to select. How does host know this?

It does not. Hosts select source addresses as per RFC6724 so (depends
on the destination addresses) we can not predict which prefix will be
used as a source.
As both (all) uplinks can be used at the same time, that's fine, each
flow will be sent to the correct uplink based on its source address.

Maybe the words 'load balancing' are confusing. Does 'load sharing'
sound better?

> Is this just copy and paste from the NON-load-balancing policy? Why would H1 use 2001:db8:2:1::/64 if "ISP_B_uplink_route is NOT present" and send traffic over R2 - R3 - ISP_A with 2001:db8:2:1::x…

Ah, yeah, typo indeed, should read ' if ISP_B_uplink_route is present
then preferred_lifetime 604800, else preferred_lifetime = 0'.
Good catch, thanks!

> Typo: "trgger"

Thanks, will fix it.

---

From: Jen Linkova <[email-masked]>
Date: Tue, 6 Mar 2018 14:02:44 +1100

On Sat, Mar 3, 2018 at 9:29 AM, Mudric, Dusan (Dusan) <[email-masked]> wrote:
> will not work in the load sharing example. Unlike the router, when the host has two prefixes it does not know which one is 'desired'.

It's not 'desired', it's 'can be used' (== 'communication is
possible'). In the load sharing scenario both uplinks can be used so
hosts are free to select addresses in all prefixes.

Please note that we are not trying to make hosts *full* aware of all
paths properties, traffic engineering is not in scope. All we are
doing is signalling to hosts which addresses would provide network
connectivity (if the ISP uplink is down, there is no network
connectivity for source addresses in that prefix).
If a hosts has received a signal that there is more than one prefix is
available (== network is capable of sending packets with that source
address out), then the host is using RFC6724 to select the addess.

>Possible option is to assign a prefix preference in RA.

Kind of a combination of Rule 5.5 and router preference? Both of them
are not widely implemented, that's the whole point of the draft.

> If so, the draft needs to make it clear.

The introduction says:
"However using IPv6 PA blocks
   in multihoming scenario introduces some challenges, including but not
   limited to:

   o  Selecting the correct uplink based on the packet source address;

   o  Signaling to hosts that some source addresses should or should not
      be used (e.g. an uplink to the ISP went down or became available
      again).
"

>The underlined assumption is that the host selected the right source address and the router uses that address to send a packet to a right ISP.

It's the first bullet point, but the second one addresses your
concern, I believe.

---

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Tue, 6 Mar 2018 14:54:36 +0000

Hi Jen,

Please see inlined. 

Regards,
Dusan.

> From: Jen Linkova [mailto:[email-masked]]
> Sent: Monday, March 05, 2018 10:03 PM
> Cc: Gert Doering; [email-masked]
> > will not work in the load sharing example. Unlike the router, when the host
> has two prefixes it does not know which one is 'desired'.
> 

[[Dusan]] 
[[Dusan]] What I am trying to say is RFC6724 does not know which ISP is preferred, when both prefixes are advertised. If a router will have a new policy to know the ISP preference, the preference should be passed to the host, and RFC6724 should be enhanced to use that preference. This will facilitate a network management where one router policy is configured and all the hosts benefit from that knowledge. When all this is put together, with multiple prefixes, a host will select the preferred source address and a router will use it to forward packet to preferred ISP. If the host part is not addressed, the host will 'randomly' (current RFC6724 will not use the prefix preference) select the source address and packets might go to either ISP1 or ISP2.

> are not widely implemented, that's the whole point of the draft.

[[Dusan]] 
[[Dusan]] Exactly. Since both prefixes are assigned by two next hops, Rule 5.5 should be enhanced to use the preferred prefix. Standards should not address the current deployment state, but provide a better solution. I would not focus on what is not currently implemented but rather on what should be implemented. 

> 
> > "Actually the new text is completely missing the point. Selecting the right
> source address *to select the egress ISP to use* is

[[Dusan]] 
[[Dusan]] These two cases are valid but they don't cover ".Selecting the right source address *to select the egress ISP to use*". This would be a third problem to address. This comes before the first bullet (the source address needs to be chosen first, then the correct uplink). The second bullet talks about the connectivity issues, where a prefix is removed when a link is down. We need to mention the source address selection when both links are up.

> 
> >The underlined assumption is that the host selected the right source address
> and the router uses that address to send a packet to a right ISP.
> 

[[Dusan]] To some extent. It needs to be clarified. And the draft should state if, how and where "Selecting the right source address *to select the egress ISP to use*" is done.

---

From: Jen Linkova <[email-masked]>
Date: Wed, 7 Mar 2018 09:58:23 +1100

On Wed, Mar 7, 2018 at 1:54 AM, Mudric, Dusan (Dusan) <[email-masked]> wrote:
> [[Dusan]] What I am trying to say is RFC6724 does not know which ISP is preferred, when both prefixes are advertised.

Sorry, it looks like I did not explain it very well..

If/when both prefixes are advertised, both ISPs are "preferred" (in
the same way IPv6 address can be in "preferred" state, which means 'it
can be used for outgoing communications".

Again, we are not solving traffic engineering issue, nor we are trying
to make hosts be fully aware of various paths properties (shameless
plug: please come to PANRG  for that :)).
ISP uplink is either "preferred" (hosts are allowed to use it, the
network can send packets there, because the uplink is up and there is
no explict policy which indicates that the uplink should not be used
if other uplinks are available) or not preferred (it's down or there
is a policy to prevent using this uplink ex. for backup).

If an uplink is preferred - the corresponding prefix is advertised
with non-zero preferred lifetime.
It's perfectly fine to have multiple preferred uplinks/ISPs. Hosts can
use addresses from any of them.

>If a router will have a new policy to know the ISP preference, the preference should be passed to the host,

It's binary state. ISP is either "usable" (preferred lifetime > 0) or
not (preferred lifetime == 0). It's how the preference is passed to
the host.

>and RFC6724 should be enhanced to use that preference.

No changes in RFC6724 are needed.
Hosts with Rule 5.5 and default support will do it anyway (if R1 has
is a preferred default router,  then the host selects a source address
from prefixes received from R1).

>This will facilitate a network management where one router policy is configured and all the hosts benefit from that knowledge. When all this is put together, with multiple prefixes, a host will select the preferred source address and a router will use it to forward packet to preferred ISP. If the host part is not addressed, the host will 'randomly' (current RFC6724 will not use the prefix preference) select the source address and packets might go to either ISP1 or ISP2.

If ISP2 should not be used and ISP1 should be preferred (is it the
scenario you are talking about??) then ISP2 prefix should be
advertised to hosts with preferred_lifetime 0. Then hosts will not
select addresses from ISP 2 prefix but use ISP1 prefix instead and
routers will forward packet to preferred ISP1.
Why do we need "prefix preference" for this?

> [[Dusan]] Exactly. Since both prefixes are assigned by two next hops, Rule 5.5 should be enhanced to use the preferred prefix. Standards should not address the current deployment state, but provide…

Have I sent this email to 6man@ by mistake? Ah, does not look like that :)
And I seem to set the document status to 'Informational'.

This draft documents a solution which works right here, right now, the
solution I can deploy today in my network.
Better (more general, covering more use cases, relying on Rule 5.5)
approach to the problem is documented in
https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming-03
but it can not be used in networks with hosts which do not support
5.5.
Also there is mPVD work etc.

> [[Dusan]] These two cases are valid but they don't cover ".Selecting the right source address *to select the egress ISP to use*". This would be a third problem to address. This comes before the fir…

If the both uplinks are up and can be used (so load sharing scenario)
then hosts can use either uplink. ISP1 or ISP2. Why is it a problem?
If one uplink (let;s say ISP2) should not be preferred (low
bandwidth/higher price etc) then it's a 'primary/backup' scenario and
hosts are selecting ISP1 prefix as it's the only prefix with non-zero
preferred lifetime.

---

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Thu, 8 Mar 2018 16:40:10 +0000

Hi Jen,

I believe I understood the intent of the draft. Let me explain my understanding of rule 5.5. Please see inlined.

Thanks,
Dusan.

> Sent: Tuesday, March 06, 2018 5:58 PM
> > [[Dusan]] What I am trying to say is RFC6724 does not know which ISP is
> preferred, when both prefixes are advertised.
> 
 [[Dusan]] 
[[Dusan]] The router knows more than that. For 'usable' prefixes, the router knows which prefix is the most preferred. The router can pass that knowledge in RA PIO option, using 'reserved' bits, for example (like Prf in rfc4191#section-2.2 ). Without this information, if there are two prefixes (ISP1-D and ISP2-D) how will host know if it should talk to D over ISP1 or ISP2? Rule 5.5 cannot determine it:

   Rule 5.5: Prefer addresses in a prefix advertised by the next-hop.
   If SA or SA's prefix is assigned by the selected next-hop that will
   be used to send to D and SB or SB's prefix is assigned by a different
   next-hop, then prefer SA.

You can consider the trick in https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming-03#section-4.2.2 by setting two routes in RA and the Prf from rfc4191#section-2.2, based on the router policy. This would allow a host to select the preferred next hop and use the preferred next hop in Rule 5.5. Using Prf in RIO the router policy is distributed to all hosts and they can select the preferred ISP. A router SHOULD set Prf based on the ISP selection policy. There might be use case when the network would use only one route and advertise one prifix.

> >This will facilitate a network management where one router policy is
> configured and all the hosts benefit from that knowledge. When all this is put
> together, with multiple prefixes, a host will select the preferred source address
> and a router will use it to forward packet to preferred ISP. If the host part is not
> addressed, the host will 'randomly' (current RFC6724 will not use the prefix
> preference) select the source address and packets might go to either ISP1 or
> ISP2.
> 

[[Dusan]] 
[[Dusan]] Because the router should not limit the communication over one ISP by advertising only one prefix. The host should be able to decide which of the two ISPs to use. Both prefixes should be given to the host and they should have the preference assigned.

> > [[Dusan]] Exactly. Since both prefixes are assigned by two next hops, Rule 5.5
> should be enhanced to use the preferred prefix. Standards should not address
> the current deployment state, but provide a better solution. I would not focus
> on what is not currently implemented but rather on what should be
> implemented.
> 

[[Dusan]] 
[[Dusan]] If we change IPO option, it is not informational any more.
> 3A__tools.ietf.org_html_draft-2Dietf-2Drtgwg-2Denterprise-2Dpa-
> 2Dmultihoming-
> 2D03&d=DwIFaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIo
> UWB8YLZU23029sMQGQ2kY&m=gr_lthjRS8_DD-N91eL2ejRNLgUVtJ4c-
> 1rQ69a_-ik&s=3BfJ_Yppp8atCnoX9IjFXP7ud9qoymX0yAj80cyJDDg&e=

[[Dusan]] 
[[Dusan]] You approach to remove the prefix from the host when one link fails will address https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming-03#section-1 PA multihoming problem:

"If the
   packet with source address in prefix-A was sent to ISP-B because the
   uplink to ISP-A failed, then if ISP-B does not drop the packet and
   the packet reaches its destination somewhere on the Internet, the
   return packet will be sent back with a destination address in prefix-
   A.  The return packet will be routed over the Internet to ISP-A, but
   it will not be delivered to the multihomed site because its link with
   ISP-A has failed."

Did you mention this in your draft?

https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming-03#section-4 states:
"The host will control which ISP is used
   for its traffic by selecting one of the addresses configured on the
   host as the source address for outgoing traffic.

the choice of source address
   by the host may be driven ...
by the desire to take advantage of
   certain properties of a particular uplink or ISP.

   How a host should make good decisions about source address selection
   in a multihomed site is not a solved problem.  We do not attempt to
   solve this problem in this document.  Instead we discuss the current
   state of affairs with respect to standardized solutions and
   implementation of those solutions. 

For a session originated by a host internal to the multi-homed site,
   the decision of what source address to select is more complicated.
   We consider three main methods for hosts to get information about the
   network.  The two proactive methods are Neighbor Discovery Router
   Advertisements ..."

Your proposal in this draft is to use https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming-03#section-4.1 
" Rule 3 (Avoid deprecated addresses)
...
 send a Router Advertisement message contaning a
   Prefix Information Option for the source prefix to be discouraged (or
   prohibited) with the preferred lifetime set to zero."
But
" This is a rather blunt tool,"

And this draft has the same issue when two prefixes are advertised:
"   Rule 5.5 (Prefer addresses in a prefix advertised by the next-hop) is
   not useful in the scenario when both R1 and R2 will advertise both
   source prefixes."

Rule 7 has the same issue:
"Rule 7 (Prefer temporary addresses)
...
prefer the two temporary addresses, but it would not break the tie
   between the two source prefixes from ISP-A and ISP-B."
which could be avoided if Rule 5.5 selected one address.

What they did was:
https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming-03#section-4.2.2 . Your draft can use the same approach and set two routes in RA and the Prf from rfc4191#section-2.2, based on the router policy. No protocol change is required.

> > [[Dusan]] These two cases are valid but they don't cover ".Selecting the right
> source address *to select the egress ISP to use*". This would be a third problem
> to address. This comes before the first bullet (the source address needs to be
> chosen first, then the correct uplink). The second bullet talks about the
> connectivity issues, where a prefix is removed when a link is down. We need to
> mention the source address selection when both links are up.
> 
> >> >> >    source address, and the network to select the correct egress ISP based
> on
> >> >> the packet
> >> >> >    source address"
> >> >>

---

From: Jen Linkova <[email-masked]>
Date: Sun, 18 Mar 2018 01:08:43 +1100

>> It's binary state.

> [[Dusan]] The router knows more than that. For 'usable' prefixes, the router knows which prefix is the most preferred.

Would you be able to give an example for 'most preferred'? The network
has two uplinks to two ISPs, two prefixes. Both uplinks can be used.
Each border router has a default route pointing to the uplink.
What prefix is 'most preferred' to reach  Internet?

>The router can pass that knowledge in RA PIO option, using 'reserved' bits, for example (like Prf in rfc4191#section-2.2 ). Without this information, if there are two prefixes (ISP1-D and ISP2-D) how will host know if it should talk to D over ISP1 or ISP2? Rule 5.5 cannot determine it:
>

I'm not sure if you've been following the whole story....When
draft-ietf-rtgwg-enterprise-pa-multihoming was presented to v6ops in
Berlin, we came to conclusion that while the described trick is nice,
it could not be widely used now. So some tactical solution is needed.
That's why this draft was 'forked' from the RTGWG one.
So anything which require protocol changes/new features/rule 5.5
support is out of scope of this draft by the definition.

> [[Dusan]] Because the router should not limit the communication over one ISP by advertising only one prefix. The host should be able to decide which of the two ISPs to use. Both prefixes should be given to the host

In the 'load sharing' scenario it is exactly what happens. All ISP
prefixes are included in PIO.
In the active/backup scenario only active uplinks prefixes are
advertised, because the network policy does not allow hosts to use the
backup uplink so hosts should not select backup uplinks.

> and they should have the preference assigned.

Which hosts would not recognize.

> [[Dusan]] If we change IPO option, it is not informational any more.

Again, the goal of *this particular* draft is to document existing
solution, which can be used right now w/o any changes on the host.

If we have ideas how to improve the protocol - great, but those ideas
should not be in this draft.

---

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Wed, 21 Mar 2018 18:22:06 +0000

Hi Jen,

Please see inlined.

Thanks,
Dusan.

> Sent: Saturday, March 17, 2018 10:09 AM
> > [[Dusan]] The router knows more than that. For 'usable' prefixes, the router
> knows which prefix is the most preferred.
> 

[[Dusan]] 
[[Dusan]] One ISP might be more reliable or can provide better QoS, for example.

> 
> >The router can pass that knowledge in RA PIO option, using 'reserved' bits, for
> example (like Prf in rfc4191#section-2.2 ). Without this information, if there are
> two prefixes (ISP1-D and ISP2-D) how will host know if it should talk to D over
> ISP1 or ISP2? Rule 5.5 cannot determine it:
> >
> > You can consider the trick in
> 2Dmultihoming-2D03-23section-
> 2D4.2.2&d=DwIFaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCr
> hIoUWB8YLZU23029sMQGQ2kY&m=-
> jWKLsdR_Ty6WyW71Q4s9hNbRCBu7cfr6LE5DlJd-
> RE&s=i7CFt5kd63hOe44GmGE3u3pd8OVgZnJxGGsj1JONwrs&e= by setting two
> routes in RA and the Prf from rfc4191#section-2.2, based on the router policy.
> This would allow a host to select the preferred next hop and use the preferred
> next hop in Rule 5.5. Using Prf in RIO the router policy is distributed to all hosts
> and they can select the preferred ISP. A router SHOULD set Prf based on the ISP
> selection policy. There might be use case when the network would use only one
> route and advertise one prifix.
> 

[[Dusan]] 
[[Dusan]] Why the Prf from rfc4191#section-2.2 cannot be set by the router? This is not a protocol change? The flags are already defined in the RFC4191.

> > [[Dusan]] Because the router should not limit the communication over one ISP
> by advertising only one prefix. The host should be able to decide which of the
> two ISPs to use. Both prefixes should be given to the host
> 

[[Dusan]] 
[[Dusan]] This is an implementation aspect. If hosts implement the Prf from rfc4191#section-2.2, they will recognize Rrf. You should not limit the options based on the current implementations.

> should not be in this draft.
[[Dusan]] I think Rrf setting should be in the draft. It might not be used by the hosts today, but can be used by the hosts of tomorrow.

---

From: Lorenzo Colitti <[email-masked]>
Date: Wed, 21 Mar 2018 18:34:14 +0000

On Wed, Mar 21, 2018 at 6:22 PM, Mudric, Dusan (Dusan) <[email-masked]>
wrote:

> [[Dusan]] This is an implementation aspect. If hosts implement the Prf
> from rfc4191#section-2.2, they will recognize Rrf. You should not limit the
> options based on the current implementations.
>

Actually, we should. This is an operational working group, and as such,
needs to document practices that work today.

---

From: 神明達哉 <[email-masked]>
Date: Wed, 21 Mar 2018 12:26:40 -0700

At Wed, 21 Mar 2018 18:34:14 +0000,

> needs to document practices that work today.

I've not closely followed this thread so this may be out of context
(if so, sorry), but if it's about host implementation of the "Prf"
value of RFC4191, BSD variants have been supporting it for more than
10 years (for example, FreeBSD added it in 2006
https://svnweb.freebsd.org/base/head/sys/netinet6/nd6_rtr.c?revision=156871&view=markup).
So I suspect it's also in iOS.  Unlike other features of
the RFC like route information option, implementing Prf is pretty
simple, so I wouldn't be surprised if it's also available in non
BSD-flavor host implementations.

--
JINMEI, Tatuya

---

From: Jen Linkova <[email-masked]>
Date: Thu, 22 Mar 2018 06:40:28 +1100

Hi Dusan,

> [[Dusan]] One ISP might be more reliable or can provide better QoS, for example.

In this case it comes down to 'active/backup uplink' scenario: use the
more reliable uplink, keep the second one as a backup/prefix
deprecated.

> [[Dusan]] Why the Prf from rfc4191#section-2.2 cannot be set by the router? This is not a protocol change? The flags are already defined in the RFC4191.

It can be set by a router but I'm aware of just one host OS which
supports this. So the router preference in RAs would not be recognized
by end devices.
Again, please note, that the draft documents a solution which can be
deployed *today* in SMB/Enterprise environment, where BYOD is common,
w/o upgrading/replacing host OSes.

> [[Dusan]] This is an implementation aspect. If hosts implement the Prf from rfc4191#section-2.2, they will recognize Rrf. You should not limit the options based on the current implementations.

It looks like I'm not explaining it properly.
*This* particular draft is written to document a solution which works
with existing hosts in existing networks today.
Solutions which do not exist right now but may be deployed in 3-5-10
years, when all hosts get new shiny features implemented are out of
scope of this particular document. As I've mentioned, a solution
similar to what you are suggesting (using 5.5 and rfc4191) is
documented in https://tools.ietf.org/html/draft-ietf-rtgwg-enterprise-pa-multihoming
.

> [[Dusan]] I think Rrf setting should be in the draft. It might not be used by the hosts today, but can be used by the hosts of tomorrow.

What would be the difference between what you are suggesting and
draft-ietf-rtgwg-enterprise-pa-multihoming?

---

From: Jen Linkova <[email-masked]>
Date: Thu, 22 Mar 2018 06:45:36 +1100

> BSD-flavor host implementations.

Yeah, you are right, it's reasonably well supported. The issue here is
that just using router preference would not help at all (or may even
make the routing more sub-optimal). Let's say a host honors the router
preference and selects a router  R1 (connected to an ISP1) as a
default gateway but it does not support Rule 5.5, then it may select
ISP2 source prefix and the packet will be send to ISP2 ( R1 -> R2 ->
ISP2). So I'm not sure I can see any value in using the router
preference w/o rule 5.5 support in this scenario..Am I missing
anything?

---

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Thu, 22 Mar 2018 14:55:35 +0000

> Sent: Wednesday, March 21, 2018 3:46 PM
> To: 神明達哉
> Cc: [email-masked]
> 3A__svnweb.freebsd.org_base_head_sys_netinet6_nd6-5Frtr.c-3Frevision-
> 3D156871-26view-
> 3Dmarkup&d=DwIGaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3i
> CrhIoUWB8YLZU23029sMQGQ2kY&m=Bt5Tr3OFzuMg-KD6-
> VnJOYJxuE_o98MuBUuvWheKQYU&s=fKZV7uOZ6tmyWip4Gvh2oTIbeFM-
> 48LejDFEElgSf7o&e=).

[[Dusan]] 
[[Dusan]] https://tools.ietf.org/html/draft-ietf-6man-rfc6434-bis-07 makes Rule 5.5 SHOULD requirement:
"[RFC8028]
   updates rule 5.5 from [RFC6724]; implementations SHOULD implement
   this rule."

That does not mean there are not end points that don't benefit from Rule 5.5. If both Prf and Rule 5.5 are implemented, the endpoints will benefit from Prf preference sent from the router.

Btw, is there any update on this discussion about Linux Rule 5.5: https://www.ietf.org/mail-archive/web/v6ops/current/msg24934.html 

> 3A__www.ietf.org_mailman_listinfo_v6ops&d=DwIGaQ&c=BFpWQw8bsuKpl1
> SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIoUWB8YLZU23029sMQGQ2kY&m=Bt5Tr3
> OFzuMg-KD6-
> VnJOYJxuE_o98MuBUuvWheKQYU&s=fACl19Teeq2xU8evNXsgijmoqbwYJmLXsj
> 85X9EuxGM&e=

---

From: Jen Linkova <[email-masked]>
Date: Fri, 23 Mar 2018 02:12:27 +1100

> That does not mean there are not end points that don't benefit from Rule 5.5. If both Prf and Rule 5.5 are implemented, the endpoints will benefit from Prf preference sent from the router.

Sure. There is draft, draft-ietf-rtgwg-enterprise-pa-multihoming which
discusses in details multihoming for networks, where hosts support
Rule 5.5.
However there is a demand to provide multihoming for networks where
not all endpoints support rule 5.5. The general solution described in
draft-ietf-rtgwg-enterprise-pa-multihoming has some limitations in
those case. So another document (this one) was written and explicitly
scoped to very specific use case:

1) Internet access only;
2) Internet uplinks are used in either 'Active/Backup' or load sharing
(no traffic engineering) modes;
3) hosts are not expected to support rule 5.5 (== the documented
solution should work for hosts which do not support it).

So I'm not sure I understand why do you insist on expanding the scope
of this document to the scope covered by
draft-ietf-rtgwg-enterprise-pa-multihoming.
Doing so will violate the requirement #3 above.

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 22 Mar 2018 16:13:17 +0000

> solution should work for hosts which do not support it).

I think this is the key point here. This is an operational working group
and the goal of this document is to describe operational practices that
work on existing hosts, and the majority of hosts today do *not* support
rule 5.5.

If setting the router preference will help such hosts multihome better,
*and* if the router preference is accepted by the majority of hosts today
(is it?) then that could be in scope for this document. But I think that
practices that will not help the majority of existing hosts aren't in scope
for this document.

---

From: "Mudric, Dusan (Dusan)" <[email-masked]>
Date: Thu, 22 Mar 2018 16:19:01 +0000

> Sent: Thursday, March 22, 2018 11:12 AM
> Cc: 神明達哉; [email-masked]
> > [[Dusan]] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_draft-2Dietf-2D6man-2Drfc6434-2Dbis-
> 2D07&d=DwIBaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=UT3Bk9cbLeaJxhf3iCrhIo
> UWB8YLZU23029sMQGQ2kY&m=3gfnhGvZmyAJsTIcnmjE3rGwNXI9M2PsTBUa
> Z2HMNp4&s=IBo12ASnigyjtF3y9mzUG-jxoLYE5Iof80v3twY3As8&e= makes
> Rule 5.5 SHOULD requirement:
> > "[RFC8028]
> > That does not mean there are not end points that don't benefit from Rule 5.5.
> If both Prf and Rule 5.5 are implemented, the endpoints will benefit from Prf
> preference sent from the router.
> 

[[Dusan]] 
[[Dusan]] This scope is very clear. Can you please put it in the draft?

Please add that when the network administrator configures the policy for load balancing on routers R1 and R2, the same policy might be used to set the default or more specific router preferences from RFC4191.