Subject: Re: [Idr] [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

From: Robert Raszuk <[email-masked]>
Date: Sat, 8 Aug 2015 17:13:53 +0200

/* First this discussion does not belong to BESS WG but to IDR, Readjusting
the mail alias */

DT,

> But this proposal more significant to tier 2 and tier 3 ISPs who have
multi-homing
> with long distance upstream ISPs.

Those tier-2 and tier-3 ISPs today are using BGP policy and based on pretty
simple match
of AS_PATH content can influence best path selection in their ASes without
any need for
new BGP extensions.

BGP tools to be used: weight, local pref or cost community.

The values for the metrics can be coming with offline (server based)
measurements if they
are to be dynamic or can be  set statically. For enforcing that set of
probe data will exit via
required links EPE may come handy.

> I propose, each border router need to check delay to eBGP neighbor every
15 min interval.

Please be consistent. In previous mail you said:

"2. Manually configure  binding to remote AS."

Are you expecting operator to readjust configuration every 15 min ???

> I decide to continue research on this aspect.

You are clearly free to conduct research on any topic. However please be
informed
that perhaps a better target for your research may be IRTF or SIGCOMM.

regards,
R.

On Sat, Aug 8, 2015 at 4:42 PM, Duleep Thilakarathne <[email-masked]>
wrote:

> Raszuk,
>
>
>
> Please find my comments inline.
>
>
>
>
>
> Regards
>
> Duleep
>
>
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Robert
> Raszuk
> *Sent:* Saturday, August 8, 2015 3:05 PM
> *To:* Duleep Thilakarathne
> *Cc:* Richard Li; UTTARO, JAMES; [email-masked]
> *Subject:* Re: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Hi Thilakarathne,
>
>
>
> Point #1:
>
>
>
> What makes you think that 10 ms over 1 GB ethernet peering is any better
> then 20 ms RTT over 100 GB ? I would really prefer to get routed over 100
> GB peerings even if the RTT would be doubled.
>
>
>
> Your preference may correct theoretically or in a different scenario other
> than ISP environment. ISPs need to pay for links and link capacity decided
> based on the traffic requirement and several other factors. There are none
> pay scenarios in ISP peering but I am not going to explain as it is
> irrelevant to this discussion. ISPs setup multiple links with multiple
> capacities in order to achieve several targets. Redundancy, low latency are
> few of them. Further ISPs advertise IPs to all upstream in order to get
> redundancy unless there is no special reason. Therefore there is high
> possibility to select so called 10ms 1GB link than 100 GB 20ms link for
> particular IP blocks in ISP environment.
>
>
>
> I will try to explain the scenario I am trying to address again since you
> have not understand it clearly.
>
>
>
>
>
> Assume particular tier 3 ISP in Asia (say ISP A) has 10 GB link to USA POP
> and 1G link to Asia POP. Web hosting provider  based on Asia advertise its
> IP block through BGP and ISP A receives web hosting provider IP block from
> both paths. Assume in this case AS PATH length is equal. Then how does
> router select best outgoing path. According to you, if router selects 10G
> link, all traffic route to USA and come back to Asia. It is not desired
> path. My proposal is to decide best path in such a scenario and not in
> general BGP scenario. My proposal try to prevent random route selection
> using new end to end BGP attribute.
>
> If router has another end to end BGP attribute other than AS-PATH , router
> can consider it during tie condition. Other attribute I propose is delay
> between AS which is proportional to geographic distance.
>
> Further this proposal may not have more significance, if eBGP peer resides
> short geographic distance. For example same state in USA. But this proposal
> more significant to tier 2 and tier 3 ISPs who have multi-homing with long
> distance upstream ISPs.
>
>
>
> Point #2:
>
>
>
> > There is no reqirment to synchronize different administrative
>
> > domains since router itself automatically calculate value and
>
> > add when routes  advertised similar to AS PATH addition
>
> > operation.
>
>
>
> Sorry to ruin your impression about power and intelligence of routers, but
> they only do what they are programmed to do.
>
>
>
> So Jim's point about synchronizing metrics is still valid. Of course I
> assume that for you the only metric you consider here are milliseconds and
> therefor do not bother.
>
>
>
> Imagine one operator chooses to use physical distance and other RTT. So in
> the new attribute you will get time [ms] vs distance [miles]. Yet one more
> will also use distance buy expressed in kilometers. Please elaborate how
> useful such comparison will turn out to be ?
>
>
>
> I am not oppose to Jim’s point on synchronizing. What I want to highlight
> is that synchronizing is not required for the attribute proposed by me.
> present routers program to select outgoing path based on random parameter
> when AS-PATH length is equal (other conditions after AS length check may
> not have more significance). My suggestion is to change the program in such
> scenario. This will prevent random route selection which router programed
> to do.
>
>
>
> Proposed attribute can use either length in km or delay in milliseconds.
> This need decide and include in the RFC.  but there is a relationship
> between delay and distance to eBGP peer . Practically  south Asian ISP can
> achieve around 30ms two way delay to  Singapore POP and 300ms to USA POP.
> We can find cable distance if required.
>
>
>
>
>
> Point #3:
>
>
>
> As you are suggesting use of ICMP to measure RTT please keep in mind that
> ICMP is not high priority protocol. It may wait in the remote or local
> router for processing much more then the propagation delay of the link it
> arrived on.
>
>
>
> I am open in this comment. ICMP or TCP delay can be used. Based on my
> experience even ICMP is not give big issue as most of border routers are
> carrier grade. 5% to 10% delay variation may not affect to route selection
> decision.
>
>
>
> Point #4:
>
>
>
> How often do you plan to remeasure the eBGP propagation ? Note that today
> many optical long haul transmission is hidden from ASBRs. That means that
> your provider of long distance connection may at will reroute you via his
> own web of fiber which does affect RTT. So it is pretty safe to assume what
> you have measured yesterday today is irrelevant.
>
>
>
> I propose, each border router need to check delay to eBGP neighbor every
> 15 min interval. If routers detects 25% delay difference to previous delay
> value, router triggers BGP update message. if not, no update message
> triggers. Optical long haul transmission will not have big issue in
> practical scenarios as SLAs exists. They are free to change backhaul path
> as long as delay does not change significantly. If delay changes
> significantly,  it address separately and business cannot exists.
>
>
>
> As I described earlier this proposal has much significance when ISPs have
> large geographic separation.
>
>
>
> Point #5:
>
>
>
> eBGP propagation may be few orders of magnitude less relevant as
> propagation within each AS path is traversing. And except the case of few
> ASes under the same administration we do not have a way to express that one
> today except AIGP attribute.
>
>
>
> AIGP RFC clearly mentioned , it is proposed  for single administrative
> domain.  I am talking about different administrative domains and eBGP.
>
>
>
> So if you would like to continue your research perhaps looking at that
> aspect first may be more valuable ....
>
>
>
> Since you have not understood practical issue  and solution I am
> explaining and you have not provided valid reason to change my research
> ,still I decide to continue research on this aspect.
>
>
>
>
>
> Cheers,
> R.
>
>
>
>
>
> On Sat, Aug 8, 2015 at 7:10 AM, Duleep Thilakarathne <[email-masked]>
> wrote:
>
> Richard,
>
>
>
> Yes I am referring eBGP scenario. I suggest distance calculations based on
>
>
>
> 1. ICMP delay between eBGP speakers.
>
> 2. Manually configure  binding to remote AS.
>
>
>
>
>
> Each eBGP speaking routers need to accumulate  distance value when
> advertised routes to external peer.There is no reqirment to synchronize
> different administrative domains since router itself automatically
> calculate value and add when routes  advertised similar to AS PATH addition
> operation.
>
>
>
>
>
>
>
>
>
>
>
> ----- Reply message -----
> From: "Richard Li" <[email-masked]>
> To: "Duleep Thilakarathne" <[email-masked]>, "UTTARO, JAMES" <
> [email-masked]>, "'Robert Raszuk'" <[email-masked]>
> Cc: "[email-masked]'" <[email-masked]>
> Subject: [bess] BGP route selection criteria - geographic distance when
> AS_PATH are equal
>
> Date: Fri, Aug 7, 2015 10:11 PM
>
>
>
> There might be a good point here. RFC 7311 only takes care of the IGP
> metrics. But In Duleep’s example, the metrics between two eBGP speakers are
> not taken into consideration. In order to have AIGP attribute to really
> represent the accumulated one, the metrics on such links should be
> considered as well. However, there might be some challenges or obstacles:
> The way to configure one metrics on the link between two eBGP speakers
> might not be consistent with the way to configure another metrics on the
> another link between two speakers.
>
>
>
> Richard
>
>
>
> *From:* BESS [mailto:[email-masked]] *On Behalf Of *Duleep
> Thilakarathne
> *Sent:* Friday, August 07, 2015 8:41 AM
> *To:* UTTARO, JAMES; 'Robert Raszuk'
> *Cc:* [email-masked]'
> *Subject:* Re: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Jim,
>
>
>
>
>
> What I want to suggest is to insert item 5 (refer below items listed) to
> BGP best path selection algorithm. Once AS-PATH length is equal, next we
> can think on how to select best outgoing interface. If we don’t select
> proper outgoing interface it will affect to latency. I am talking this
> based on practical experience I have in ISP environment. There are several
> options to select best outgoing interface when AS-PATH are equal.  In this
> case I suggest geo distance to destination. Following are options to
> calculate geo distance. Router selects outgoing interface with lowest GEO
> distance to destination.
>
>
>
> 1. BGP speaking router can add distance when advertise to route to
> upstream similar to AS-PATH attribute. For example
>
>
>
> A----B----C-----D
>
>
>
> Router B advertise distance AB to router C. router C advertise accumulated
> distance AB+BC to router D.
>
>
>
> 2. Above distance can be configured as manual interface command or
> dynamically using ICMP or similar mechanism. We can assume ICMP delay
> propositional to geo distance.
>
>
>
> 3. Alternative option is to calculate real geo distance from coordinate
> system. In this case we miss intermediate hops. Accuracy is not much
> accurate since cable paths do not follow real coordinate based distance. In
> this case we should have knowledge on coordinates of upstream router which
> relevant IP block advertise.
>
>
>
> Option 1 can be achieved through BGP protocol itself ,if agreed to
> introduce new attribute.
>
> Option 3 more suitable to SDN based implementation. Calculation can be
> daily or weekly basis as this is not primary criteria.
>
>
>
> Further Consider following scenario.
>
>
>
> I am in Sri Lanka. Assume I have upstream POPS to Singapore, AMS, New
> York.  Assume I need to reach destination IP located at Japan. When I check
> BGP routing table, AS-PATH length is equal from all three upstream. Then I
> have three options. Then router selects any interface randomly if no policy
> configured. I hope you agreed up to this point. In such case I suggest to
> consider GEO distance to destination. In most cases lowest distance path is
> the best path. This may not correct always but better than random outgoing
> interface selection.
>
>
>
>
>
> 1. Discarding the routes with the unreachable Next_Hop.
>
>
>
>
>
> 2. Preferring the route with the highest Local_Pref.
>
>
>
>
>
> 3. Preferring the aggregated route. The preference of an aggregated route
> is higher than the preference of a non-aggregated route.
>
>
>
>
>
> 4. Preferring the route with the shortest AS-Path.
>
>
>
>
>
> *5. If AS-Path finds equal, consider shortest GEO distance. If still
> distance is same follow next steps. *
>
>
>
>
>
> 6. Comparing the Origin attribute and selecting the routes with the Origin
> attribute as IGP, EGP, or Incomplete in order.
>
>
>
>
>
> Regards
>
> Duleept
>
>
>
>
>
>
>
> *From:* UTTARO, JAMES [mailto:[email-masked] <[email-masked]>]
> *Sent:* Friday, August 7, 2015 7:44 PM
> *To:* Duleep Thilakarathne; 'Robert Raszuk'
> *Cc:* [email-masked]'
> *Subject:* RE: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Duleep,
>
>
>
>                 So a bit confused here.
>
>
>
>                 How do want the decision making to go if a path has a
> shorter AS-PATH and longer latency than the alternative?? If latency is the
> prime motivator why do you care about AS-PATH length at all.. Comments
> In-Line..
>
>
>
> Jim Uttaro
>
>
>
> *From:* Duleep Thilakarathne [mailto:[email-masked]
> <[email-masked]>]
> *Sent:* Friday, August 07, 2015 9:31 AM
> *To:* Robert Raszuk
> *Cc:* UTTARO, JAMES; [email-masked]
> *Subject:* RE: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Hi Raszuk,
>
>
>
>
>
> Question 1: How does the router know about user's high latency ?
>
>
>
> Actually I am referring ISP edge router to another ISP edge router delay
> due to transmission distance.
>
> *[Jim U>] The underlying facility and it’s representative transmission
> distance will most likely differ from geographical distance. Which do you
> want to address? To Robert’s point you still need to acquire that knowledge
> and it may be orthogonal to an attribute that is defined as delay.*
>
>
>
>
>
> Question 2: How do you assure Internet stability where you start churning
> paths based on the latency of data plane ?
>
>
>
> It is not required to consider stability in this situation since it is
> unavoidable. What is refer is, router need to select best outgoing path
> considering physical distance whenever possible when AS-PATH length is
> equal. If router selects long distance path randomly, it impacts to
> latency.
>
>
>
> Question 3: What you are after has effectively been solved many years ago
> .. it is called Optimized Edge Routing (OER) / Performance Routing (PFR) -
> I suggest you google for those terms.
>
>
>
> Thank for the suggestion. I gone through these proposals. But what I am
> suggesting is  whether we can address this idea from BGP protocol level.
> For example by introducing new attribute related to physical distance/delay
> similar to AS-PATH. New attribute need to update across the As path. My
> ultimate objective is to prevent router randomly select outgoing path when
> AS-PATH lengths are  equal. Further I am trying SDN based simulation these
> days. Hope I can share output. But this could similar to what you have
> proposed except geo distance calculation mechanism.
>
>
>
> Refer below standard BGP route selection criteria. I suggest item 5.
> Wordings may different from vendor to vendor.
>
>
>
>
>
> 1. Discarding the routes with the unreachable Next_Hop.
>
>
>
>
>
> 2. Preferring the route with the highest Local_Pref.
>
>
>
>
>
> 3. Preferring the aggregated route. The preference of an aggregated route
> is higher than the preference of a non-aggregated route.
>
>
>
>
>
> 4. Preferring the route with the shortest AS-Path.
>
>
>
>
>
> *5. If AS-Path finds equal, consider shortest GEO distance. If still
> distance is same follow next steps. *
>
>
>
>
>
> 6. Comparing the Origin attribute and selecting the routes with the Origin
> attribute as IGP, EGP, or Incomplete in order.
>
>
>
>
>
>
>
>
>
>
>
>
>
> Regsrds
>
> Duleept
>
>
>
> *From:* [email-masked] [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *Robert Raszuk
> *Sent:* Friday, August 7, 2015 6:29 PM
> *To:* Duleep Thilakarathne
> *Cc:* UTTARO, JAMES; [email-masked]
> *Subject:* Re: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Duleep,
>
>
>
> > Then end user experiences high latency to reach destination. In such
>
> > a case, I suggest router need to consider geographic distance to
>
> > destination and select path via NTT to reach destination by default.
>
>
>
> Question 1: How does the router know about user's high latency ?
>
>
>
> Question 2: How do you assure Internet stability where you start churning
> paths based on the latency of data plane ?
>
>
>
> Question 3: What you are after has effectively been solved many years ago
> .. it is called Optimized Edge Routing (OER) / Performace Routing (PFR) - I
> suggest you google for those terms.
>
>
>
> Regards,
>
> R.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Aug 7, 2015 at 2:51 PM, Duleep Thilakarathne <[email-masked]>
> wrote:
>
> Hi Jim,
>
>
>
> Please refer below example.
>
>
>
> Assume destination IP is in Asian region. Particular ISP in a different
>  location (Say India) has upstream peering to US POP (Say AT&T) and Asia
> POP (Say NTT). If we check BGP routing table, assume it shows
>
>
>
> XX.XX.XX.XX/24 ------àAS - AT&T,AS-XX,AS-Destination
>
>                                 ------àAS - NTT,AS-YY,AS-Destination
>
>
>
>
>
> In above case AS-PATH is equal and assume router automatically select path
> via AT&T. Then end user experiences high latency to reach destination. In
> such a case, I suggest router need to consider geographic distance to
> destination and select path via NTT to reach destination by default.
> Deciding geo distance is a challenge but there are options. Here geo
> distance means shortest distance to reach IP destination from upstream POP.
> Current practice is to use community strings, but it depends on upstream
> ISP capability.
>
>
>
> Can you comment my idea.
>
>
>
> Regards
>
> Duleept
>
>
>
>
>
>
>
> *From:* UTTARO, JAMES [mailto:[email-masked]]
> *Sent:* Friday, August 7, 2015 4:09 PM
> *To:* Duleep Thilakarathne; 'Robert Raszuk'
>
>
> *Cc:* [email-masked]'
> *Subject:* Re: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Duleep,
>
>
>
>                 Assuming AS-PATH is equal and AS-Content different how can
> you know that the internal metrics of each AS are consistent and mirror
> actual geographic distances? You have to be assured that each
> administrative domain applies the same metric assignment. I do not believe
> this is possible when there are multiple administrative domains.
>
>
>
> Jim Uttaro
>
>
>
> *From:* BESS [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *Duleep Thilakarathne
> *Sent:* Friday, August 07, 2015 5:19 AM
> *To:* Robert Raszuk
> *Cc:* [email-masked]
> *Subject:* Re: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Hi Raszuk,
>
>
>
> I went through RFC7311 and my concern is different than RFC 7311. I have
> analyzed full BGP routing table (541,199 routes) with two tier 1 ISP
> multi-homing scenario and found nearly 50% of routes have equal AS-PATH
> length. In this analysis It was considered, there was no route policy
> applied to influence local preference. According to BGP best path selection
> algorithm, when AS-PATH lengths  are equal, router breaks tie condition
> based on route internal logic. This does not grantee proper outgoing path
> selection.
>
>
>
> Appreciate your concern on above analysis.
>
>
>
> Regards
>
> Duleept
>
>
>
>
>
>
>
>
>
>
>
> *From:* Robert Raszuk [mailto:[email-masked] <[email-masked]>]
> *Sent:* Monday, July 27, 2015 2:40 AM
> *To:* Duleep Thilakarathne
> *Cc:* [email-masked]
> *Subject:* Re: [bess] BGP route selection criteria - geographic distance
> when AS_PATH are equal
>
>
>
> Hi Duleep,
>
>
>
> Please consider RFC 7311 and provide feedback why you think it is not
> sufficient for your objective.
>
>
>
> https://tools.ietf.org/html/rfc7311
>
>
>
> Best,
>
> R.
>
>
>
>
>
> On Sun, Jul 26, 2015 at 9:15 PM, Duleep Thilakarathne <[email-masked]>
> wrote:
>
> Hi,
>
>
>
> I would like to suggest to consider geographic distance when AS_PATH  are
> equal in BGP route selection criteria. (as tie breaking rule). Can anybody
> comment on my idea.
>
>
>
>
>
> Regards
>
> Duleept
>
>
>
>
>
>
>
> This e-mail and any attachments may contain confidential and
> privileged information. If you are not the intended recipient,
> please notify the sender immediately by return e-mail, delete this
> e-mail and destroy any copies. Any dissemination or use of this
> information by a person other than the intended recipient is
> unauthorized and may be illegal.
> Mobitel (Pvt) Ltd.
>
>
> _______________________________________________
> BESS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bess
>
>
>
> This e-mail and any attachments may contain confidential and privileged
> information. If you are not the intended recipient, please notify the
> sender immediately by return e-mail, delete this e-mail and destroy any
> copies. Any dissemination or use of this information by a person other than
> the intended recipient is unauthorized and may be illegal. Mobitel (Pvt)
> Ltd.
>
> This e-mail and any attachments may contain confidential and privileged
> information. If you are not the intended recipient, please notify the
> sender immediately by return e-mail, delete this e-mail and destroy any
> copies. Any dissemination or use of this information by a person other than
> the intended recipient is unauthorized and may be illegal. Mobitel (Pvt)
> Ltd.
>
>
>
> This e-mail and any attachments may contain confidential and privileged
> information. If you are not the intended recipient, please notify the
> sender immediately by return e-mail, delete this e-mail and destroy any
> copies. Any dissemination or use of this information by a person other than
> the intended recipient is unauthorized and may be illegal. Mobitel (Pvt)
> Ltd.
>
> This e-mail and any attachments may contain confidential and privileged
> information. If you are not the intended recipient, please notify the
> sender immediately by return e-mail, delete this e-mail and destroy any
> copies. Any dissemination or use of this information by a person other than
> the intended recipient is unauthorized and may be illegal. Mobitel (Pvt)
> Ltd.
>
> This e-mail and any attachments may contain confidential and
> privileged information. If you are not the intended recipient,
> please notify the sender immediately by return e-mail, delete this
> e-mail and destroy any copies. Any dissemination or use of this
> information by a person other than the intended recipient is
> unauthorized and may be illegal.
> Mobitel (Pvt) Ltd.
>
>
> This e-mail and any attachments may contain confidential and privileged
> information. If you are not the intended recipient, please notify the
> sender immediately by return e-mail, delete this e-mail and destroy any
> copies. Any dissemination or use of this information by a person other than
> the intended recipient is unauthorized and may be illegal. Mobitel (Pvt)
> Ltd.
>

---

From: Duleep Thilakarathne <[email-masked]>
Date: Sat, 8 Aug 2015 17:10:03 +0000

Hi Raszuk,

> But this proposal more significant to tier 2 and tier 3 ISPs who have multi-homing

Those tier-2 and tier-3 ISPs today are using BGP policy and based on pretty simple match
of AS_PATH content can influence best path selection in their ASes without any need for
new BGP extensions.

The values for the metrics can be coming with offline (server based) measurements if they
are to be dynamic or can be  set statically. For enforcing that set of probe data will exit via
required links EPE may come handy.

Yes. I do accept route policy is the current practice. Actually in my second mail, I have mentioned that BGP full routing table was analyzed without applying any route policy. One purpose of introducing new path attribute is to reduce number of route policies. I am checking the possibility of influencing best path selection through protocol itself without manual route policies.
As per my understanding there is no point of discussing AS-PATH length equal scenario ,if we use route policies.

> I propose, each border router need to check delay to eBGP neighbor every 15 min interval.

If manually configured, periodic checking will not take effect. But we need to analyze this scenario carefully since allowing manual configuration can loss the purpose.

> I decide to continue research on this aspect.

You are clearly free to conduct research on any topic. However please be informed
that perhaps a better target for your research may be IRTF or SIGCOMM.

Thank you very much for your recommendation.

Regards,
Duleep

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Saturday, August 8, 2015 8:44 PM
To: Duleep Thilakarathne
Cc: idr wg
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

/* First this discussion does not belong to BESS WG but to IDR, Readjusting the mail alias */

DT,

> But this proposal more significant to tier 2 and tier 3 ISPs who have multi-homing

Those tier-2 and tier-3 ISPs today are using BGP policy and based on pretty simple match
of AS_PATH content can influence best path selection in their ASes without any need for
new BGP extensions.

The values for the metrics can be coming with offline (server based) measurements if they
are to be dynamic or can be  set statically. For enforcing that set of probe data will exit via
required links EPE may come handy.

> I propose, each border router need to check delay to eBGP neighbor every 15 min interval.

> I decide to continue research on this aspect.

You are clearly free to conduct research on any topic. However please be informed
that perhaps a better target for your research may be IRTF or SIGCOMM.

regards,
R.

On Sat, Aug 8, 2015 at 4:42 PM, Duleep Thilakarathne <[email-masked]<mailto:[email-masked]>> wrote:
Raszuk,

Regards
Duleep

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert Raszuk
Sent: Saturday, August 8, 2015 3:05 PM
To: Duleep Thilakarathne
Cc: Richard Li; UTTARO, JAMES; [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Hi Thilakarathne,

Point #1:

What makes you think that 10 ms over 1 GB ethernet peering is any better then 20 ms RTT over 100 GB ? I would really prefer to get routed over 100 GB peerings even if the RTT would be doubled.

Your preference may correct theoretically or in a different scenario other than ISP environment. ISPs need to pay for links and link capacity decided based on the traffic requirement and several other factors. There are none pay scenarios in ISP peering but I am not going to explain as it is irrelevant to this discussion. ISPs setup multiple links with multiple capacities in order to achieve several targets. Redundancy, low latency are few of them. Further ISPs advertise IPs to all upstream in order to get redundancy unless there is no special reason. Therefore there is high possibility to select so called 10ms 1GB link than 100 GB 20ms link for particular IP blocks in ISP environment.

I will try to explain the scenario I am trying to address again since you have not understand it clearly.

Assume particular tier 3 ISP in Asia (say ISP A) has 10 GB link to USA POP and 1G link to Asia POP. Web hosting provider  based on Asia advertise its IP block through BGP and ISP A receives web hosting provider IP block from both paths. Assume in this case AS PATH length is equal. Then how does router select best outgoing path. According to you, if router selects 10G link, all traffic route to USA and come back to Asia. It is not desired path. My proposal is to decide best path in such a scenario and not in general BGP scenario. My proposal try to prevent random route selection using new end to end BGP attribute.
If router has another end to end BGP attribute other than AS-PATH , router can consider it during tie condition. Other attribute I propose is delay between AS which is proportional to geographic distance.
Further this proposal may not have more significance, if eBGP peer resides short geographic distance. For example same state in USA. But this proposal more significant to tier 2 and tier 3 ISPs who have multi-homing with long distance upstream ISPs.

> add when routes  advertised similar to AS PATH addition

Sorry to ruin your impression about power and intelligence of routers, but they only do what they are programmed to do.

So Jim's point about synchronizing metrics is still valid. Of course I assume that for you the only metric you consider here are milliseconds and therefor do not bother.

Imagine one operator chooses to use physical distance and other RTT. So in the new attribute you will get time [ms] vs distance [miles]. Yet one more will also use distance buy expressed in kilometers. Please elaborate how useful such comparison will turn out to be ?

I am not oppose to Jim’s point on synchronizing. What I want to highlight is that synchronizing is not required for the attribute proposed by me. present routers program to select outgoing path based on random parameter when AS-PATH length is equal (other conditions after AS length check may not have more significance). My suggestion is to change the program in such scenario. This will prevent random route selection which router programed to do.

Proposed attribute can use either length in km or delay in milliseconds. This need decide and include in the RFC.  but there is a relationship between delay and distance to eBGP peer . Practically  south Asian ISP can achieve around 30ms two way delay to  Singapore POP and 300ms to USA POP. We can find cable distance if required.

Point #3:

As you are suggesting use of ICMP to measure RTT please keep in mind that ICMP is not high priority protocol. It may wait in the remote or local router for processing much more then the propagation delay of the link it arrived on.

I am open in this comment. ICMP or TCP delay can be used. Based on my experience even ICMP is not give big issue as most of border routers are carrier grade. 5% to 10% delay variation may not affect to route selection decision.

Point #4:

How often do you plan to remeasure the eBGP propagation ? Note that today many optical long haul transmission is hidden from ASBRs. That means that your provider of long distance connection may at will reroute you via his own web of fiber which does affect RTT. So it is pretty safe to assume what you have measured yesterday today is irrelevant.

I propose, each border router need to check delay to eBGP neighbor every 15 min interval. If routers detects 25% delay difference to previous delay value, router triggers BGP update message. if not, no update message triggers. Optical long haul transmission will not have big issue in practical scenarios as SLAs exists. They are free to change backhaul path as long as delay does not change significantly. If delay changes significantly,  it address separately and business cannot exists.

As I described earlier this proposal has much significance when ISPs have large geographic separation.

Point #5:

eBGP propagation may be few orders of magnitude less relevant as propagation within each AS path is traversing. And except the case of few ASes under the same administration we do not have a way to express that one today except AIGP attribute.

AIGP RFC clearly mentioned , it is proposed  for single administrative domain.  I am talking about different administrative domains and eBGP.

So if you would like to continue your research perhaps looking at that aspect first may be more valuable ....

Since you have not understood practical issue  and solution I am explaining and you have not provided valid reason to change my research ,still I decide to continue research on this aspect.

Cheers,
R.

On Sat, Aug 8, 2015 at 7:10 AM, Duleep Thilakarathne <[email-masked]<mailto:[email-masked]>> wrote:
Richard,

Each eBGP speaking routers need to accumulate  distance value when advertised routes to external peer.There is no reqirment to synchronize different administrative domains since router itself automatically calculate value and add when routes  advertised similar to AS PATH addition operation.

----- Reply message -----
From: "Richard Li" <[email-masked]<mailto:[email-masked]>>
To: "Duleep Thilakarathne" <[email-masked]<mailto:[email-masked]>>, "UTTARO, JAMES" <[email-masked]<mailto:[email-masked]>>, "'Robert Raszuk'" <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>'" <[email-masked]<mailto:[email-masked]>>
Subject: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal
Date: Fri, Aug 7, 2015 10:11 PM

There might be a good point here. RFC 7311 only takes care of the IGP metrics. But In Duleep’s example, the metrics between two eBGP speakers are not taken into consideration. In order to have AIGP attribute to really represent the accumulated one, the metrics on such links should be considered as well. However, there might be some challenges or obstacles: The way to configure one metrics on the link between two eBGP speakers might not be consistent with the way to configure another metrics on the another link between two speakers.

Richard

From: BESS [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Duleep Thilakarathne
Sent: Friday, August 07, 2015 8:41 AM
To: UTTARO, JAMES; 'Robert Raszuk'
Cc: [email-masked]<mailto:[email-masked]>'
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Jim,

What I want to suggest is to insert item 5 (refer below items listed) to BGP best path selection algorithm. Once AS-PATH length is equal, next we can think on how to select best outgoing interface. If we don’t select proper outgoing interface it will affect to latency. I am talking this based on practical experience I have in ISP environment. There are several options to select best outgoing interface when AS-PATH are equal.  In this case I suggest geo distance to destination. Following are options to calculate geo distance. Router selects outgoing interface with lowest GEO distance to destination.

1. BGP speaking router can add distance when advertise to route to upstream similar to AS-PATH attribute. For example

A----B----C-----D

Router B advertise distance AB to router C. router C advertise accumulated distance AB+BC to router D.

2. Above distance can be configured as manual interface command or dynamically using ICMP or similar mechanism. We can assume ICMP delay propositional to geo distance.

3. Alternative option is to calculate real geo distance from coordinate system. In this case we miss intermediate hops. Accuracy is not much accurate since cable paths do not follow real coordinate based distance. In this case we should have knowledge on coordinates of upstream router which relevant IP block advertise.

Option 1 can be achieved through BGP protocol itself ,if agreed to introduce new attribute.
Option 3 more suitable to SDN based implementation. Calculation can be daily or weekly basis as this is not primary criteria.

I am in Sri Lanka. Assume I have upstream POPS to Singapore, AMS, New York.  Assume I need to reach destination IP located at Japan. When I check BGP routing table, AS-PATH length is equal from all three upstream. Then I have three options. Then router selects any interface randomly if no policy configured. I hope you agreed up to this point. In such case I suggest to consider GEO distance to destination. In most cases lowest distance path is the best path. This may not correct always but better than random outgoing interface selection.

3. Preferring the aggregated route. The preference of an aggregated route is higher than the preference of a non-aggregated route.

5. If AS-Path finds equal, consider shortest GEO distance. If still distance is same follow next steps.

6. Comparing the Origin attribute and selecting the routes with the Origin attribute as IGP, EGP, or Incomplete in order.

Regards
Duleept

From: UTTARO, JAMES [mailto:[email-masked]]
Sent: Friday, August 7, 2015 7:44 PM
To: Duleep Thilakarathne; 'Robert Raszuk'
Cc: [email-masked]<mailto:[email-masked]>'
Subject: RE: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Duleep,

                So a bit confused here.

                How do want the decision making to go if a path has a shorter AS-PATH and longer latency than the alternative?? If latency is the prime motivator why do you care about AS-PATH length at all.. Comments In-Line..

Jim Uttaro

From: Duleep Thilakarathne [mailto:[email-masked]]
Sent: Friday, August 07, 2015 9:31 AM
To: Robert Raszuk
Cc: UTTARO, JAMES; [email-masked]<mailto:[email-masked]>
Subject: RE: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Hi Raszuk,

Actually I am referring ISP edge router to another ISP edge router delay due to transmission distance.
[Jim U>] The underlying facility and it’s representative transmission distance will most likely differ from geographical distance. Which do you want to address? To Robert’s point you still need to acquire that knowledge and it may be orthogonal to an attribute that is defined as delay.

Question 2: How do you assure Internet stability where you start churning paths based on the latency of data plane ?

It is not required to consider stability in this situation since it is unavoidable. What is refer is, router need to select best outgoing path considering physical distance whenever possible when AS-PATH length is equal. If router selects long distance path randomly, it impacts to latency.

Question 3: What you are after has effectively been solved many years ago .. it is called Optimized Edge Routing (OER) / Performance Routing (PFR) - I suggest you google for those terms.

Thank for the suggestion. I gone through these proposals. But what I am suggesting is  whether we can address this idea from BGP protocol level. For example by introducing new attribute related to physical distance/delay similar to AS-PATH. New attribute need to update across the As path. My ultimate objective is to prevent router randomly select outgoing path when AS-PATH lengths are  equal. Further I am trying SDN based simulation these days. Hope I can share output. But this could similar to what you have proposed except geo distance calculation mechanism.

Refer below standard BGP route selection criteria. I suggest item 5. Wordings may different from vendor to vendor.

3. Preferring the aggregated route. The preference of an aggregated route is higher than the preference of a non-aggregated route.

5. If AS-Path finds equal, consider shortest GEO distance. If still distance is same follow next steps.

6. Comparing the Origin attribute and selecting the routes with the Origin attribute as IGP, EGP, or Incomplete in order.

Regsrds
Duleept

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, August 7, 2015 6:29 PM
To: Duleep Thilakarathne
Cc: UTTARO, JAMES; [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Duleep,

Question 2: How do you assure Internet stability where you start churning paths based on the latency of data plane ?

Question 3: What you are after has effectively been solved many years ago .. it is called Optimized Edge Routing (OER) / Performace Routing (PFR) - I suggest you google for those terms.

Regards,
R.

On Fri, Aug 7, 2015 at 2:51 PM, Duleep Thilakarathne <[email-masked]<mailto:[email-masked]>> wrote:
Hi Jim,

Assume destination IP is in Asian region. Particular ISP in a different  location (Say India) has upstream peering to US POP (Say AT&T) and Asia POP (Say NTT). If we check BGP routing table, assume it shows

XX.XX.XX.XX/24 -------->AS - AT&T,AS-XX,AS-Destination
                                -------->AS - NTT,AS-YY,AS-Destination

In above case AS-PATH is equal and assume router automatically select path via AT&T. Then end user experiences high latency to reach destination. In such a case, I suggest router need to consider geographic distance to destination and select path via NTT to reach destination by default. Deciding geo distance is a challenge but there are options. Here geo distance means shortest distance to reach IP destination from upstream POP. Current practice is to use community strings, but it depends on upstream ISP capability.

Can you comment my idea.

Regards
Duleept

From: UTTARO, JAMES [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Friday, August 7, 2015 4:09 PM
To: Duleep Thilakarathne; 'Robert Raszuk'

Cc: [email-masked]<mailto:[email-masked]>'
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Duleep,

                Assuming AS-PATH is equal and AS-Content different how can you know that the internal metrics of each AS are consistent and mirror actual geographic distances? You have to be assured that each administrative domain applies the same metric assignment. I do not believe this is possible when there are multiple administrative domains.

Jim Uttaro

From: BESS [mailto:[email-masked]] On Behalf Of Duleep Thilakarathne
Sent: Friday, August 07, 2015 5:19 AM
To: Robert Raszuk
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Hi Raszuk,

I went through RFC7311 and my concern is different than RFC 7311. I have analyzed full BGP routing table (541,199 routes) with two tier 1 ISP multi-homing scenario and found nearly 50% of routes have equal AS-PATH length. In this analysis It was considered, there was no route policy applied to influence local preference. According to BGP best path selection algorithm, when AS-PATH lengths  are equal, router breaks tie condition based on route internal logic. This does not grantee proper outgoing path selection.

Regards
Duleept

From: Robert Raszuk [mailto:[email-masked]]
Sent: Monday, July 27, 2015 2:40 AM
To: Duleep Thilakarathne
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal

Hi Duleep,

Please consider RFC 7311 and provide feedback why you think it is not sufficient for your objective.

Best,
R.

On Sun, Jul 26, 2015 at 9:15 PM, Duleep Thilakarathne <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

I would like to suggest to consider geographic distance when AS_PATH  are equal in BGP route selection criteria. (as tie breaking rule). Can anybody comment on my idea.

Regards
Duleept

_______________________________________________
BESS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/bess

This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd.
This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd.

This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd.
This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd.

This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd.