Subject: Re: [GROW] I-D Action: draft-ietf-grow-filtering-threats-04.txt

From: Greg Skinner <[email-masked]>
Date: Thu, 12 Feb 2015 19:54:52 +0000 (GMT)

Brief comment inline.

On Feb 12, 2015, at 09:58 AM, Rick Casarez <[email-masked]> wrote:

Hello Pierre,

With the hope of not cluttering it up too much I will also respond in line. 

-------------------
Cheers, Rick

Experiences not things.

On Thu, Feb 12, 2015 at 12:15 PM, Pierre Francois <[email-masked]> wrote:

Hello Rick, 

Please see my comments inline.

On Feb 12, 2015, at 8:18 AM, Rick Casarez <[email-masked]> wrote:

I checked the history of the document but did not see this so forgive me if these were discussed before.

No problem. Thank you for your interest and time spent on giving us feedback. 

In section 2.2.1: In the scenario and figure listed I am having trouble finding what the issue is. The forwarding is following the exact path the end AS wants based on policy so I am not sure how this is unexpected. 

The issue is not on the end AS side, the end AS is happy with the situation. 
The issue is that AS64502 configures bgp to not provide connectivity between 64503 and 64504, (typically because they both are peers and 64502 does not want to provide a free ride on its network), 
and yet, it does so due to the game going on. 

Ah, I see where you are going with it now although I think you meant AS64505 which is the Peer AS and not customer AS (64504). 

While I understand it I am still somewhat confused. Since AS64505 does not have a connection to AS64501 it will still use 64502 as a transit AS to get to the destination AS (AS64501). So the problem is not the usage of AS64502 as a transit AS (Since that still occurs if the /34 is withdrawn), nor is it that the routing shown is unexpected given the prefix being leaked and the limitations of propogation set by the originator. It is the fact that the traffic-engineering being done by AS64501 may impact the interconnect or Peering policies between AS64502 and AS64503 adversely. This would especially be important if as part of the Peering relationship between AS64502 and AS64503 there is a ratio requirement or the interconnect is already running near peak.

Perhaps instead of "unexpected routing" it should be "adverse effects of this type of routing". I would even posit that the word "unintentional" might also be added since most peering agreements are locked under NDAs and the originator AS64501 might not know he is having an impact or even the nature of the relationship between AS64502 and AS64503 legally.

Now if you were to shift AS64505 so that it dual homed to both AS64502 AND AS64503 you can demonstrate a scenario of inefficient (and unexpected) routing if I am reading the figure correctly. Since AS64505 only sees the /32 from both AS upstreams from AS64501 but its algorithm selects AS64502. We know that AS64502 forwards to AS64503 and then on to the originating AS (AS64501). This is inefficient since AS64505 has a direct connection to AS64503 it could have saved an AS hop (and consequently line/equipment induced delays) by going direct to that AS had the /34 prefix been allowed to propagate further.   

The very first time we presented this case, we've been invited by an operator telling us the following story:

"It happened to me".

(I put it in context of figure 2.1.1).

Transit to 64502 is expensive and 64501 tends to use its link too much.
64503 is not expensive, nice capacity, and some common business interests with 64501. 

64502 has much better reach to 64504 than 64503 (capacity, delay), because 64503 uses 64505 to reach 64504 and well, 64505 is not the best transit network ever.
So 64503 and 64502 have both interest in having the trick path followed. 

so, to summarize: 

64501 - 64502 - 64504 : best path but expensive and transit link to 64502 tends to saturate on rainy days. 
64501 - 64503 - 64505 - 64504 : the other path 64501 could pick, but it's not a nice one, 64505's reach to 64504 is not good. 

The config trick leads to the following path being established

64501 - 64503 - 64502 - 64504 : Same length as the alternative in AS count, but much much better reach. 

Problem: 64502 does not make a single dime on the flows following that path. 

Message of the draft for operators: it's complicated/inefficient/dangerous to defend against this actively, so adding some checks in monitoring tools, and interaction with customers and peers is probably the way to go…

Let the problem take place and fix it. Some do not like this but well, not many like the proposed alternatives.

I agree with the problem statement for sure. That is the risk when doing settlement-free interconnects. It is also one of the main reasons some ISPs made settlement-free peering requirements so high or did away with it altogether and do 'Paid Peering' instead. I am not sure if either of those are within the scope for this document but those are alternative actions that can be taken as well. In addition to turning down the Peer if they violate their agreement. Those are really business decisions though.
 
Given my understanding of the charter of this WG, it seems reasonable to me to include some of the text in the above paragraph as an advisory note of sorts.  It would be beyond the scope of the document to tell people how to run their networks, but useful to advise them of undesirable side-effects.

I don't regularly participate in operations groups such as NANOG or SANOG, but perhaps some people who do might have some opinions about this issue.

Greg

Section 3.2: I am not sure I agree with this statement especially the abuse part:

"This situation may appear as an abuse to the network resources of other ISPs."

The section says this then goes on to explain all the valid reasons to create unexpected traffic flows. There is no example of an "abuse" situation. I am also at a loss on how anyone would think it "appears like an abuse of resources" when the previous sections outline plenty of real reasons these types of announcements are done. Prefix hijacking I can understand as an abuse but that is not discussed here. Can someone explain the abuse sentence to me?

Because good reasons to some (64501 and 64503) leads to a money loosing situation to another (64502).

That is not abuse though. If we do one of the solutions and force routing back to 64502-64501 then you reverse your statement above. By that definition it is still abuse but now abuse by 64502 (Gains revenue back) against 64503 (Loses revenue) and 64501 (Overrides originating AS traffic engineering). Especially if 64503 is following the peering agreement agreed two by the two network entities and passing along valid NLRI with valid attributes that 64502 is accepting (and as demonstrated acting upon said attributes).

By tweaking the scenario and adding a different /34 from the same /32 being sent to AS64502 and not AS64503 with the same type of attributes (Only advertise to AS64503 and nowhere else)  we would by definition now see "abuse" by AS64501, AS64502, and AS64503? It is not abuse though but normal traffic engineering that has be done for a long, long time.

When I think abuse I think of prefix-hijacking and spam. Traffic engineering really should not fall under the same label. If 64502 feels that either network (64503 or 64505) is abusing the peering/customer agreements then the solutions would fall under a business/legal framework between those companies. It would appear obvious a re-negotiation is in order.

Section 4: If I understand this correctly the originating AS has made a traffic engineering decision based on its internal policies but the upstreams should actively try to counter it? I am not sure I agree with Section 4 in almost its entirety. All the solutions listed seem to be a list of bad ideas except for the very first suggestion of contacting the originating AS to find out why they are doing it and if it can be stopped. 

This is our point. Our submission to GROW had the goal of discussing the best operational practice to solve the problem.

With time, we often heard "yeah but just do that and you're good". As, very much like you, we were not really agreeing, we preferred to discuss why and document it. 

For this scenario the technical solutions add snowflakes into a standardized network and as you even pointed out now have to be maintained and updated periodically every time the originating AS adjust to overcome them (or just make a change without even knowing you are actively opposing the changes). If it falls within the scope of the draft the non-technical solutions (Which are more common) should be addressed. Unless peering agreements and other tactics ISPs have used (That I know I have used) to address those abusing agreements are still a no-no to talk about publicly.

Section 4.2.1 covers actively filtering without thought to the fact that the originating AS may be disconnecting the connection between AS65401 and AS65402 and this more specific leaking is a precursor to that. Further, section 4.2.1 could also violate the agreement between the two networks (AS65402 and AS65403). Finally, it would put the network doing ACL'ing (AS65402) under risk (From their own customers and other networks along with the originating network) since a common traceroute would identify who is filtering the traffic.

Exactly. 

General formatting: Some main chapters are in bold while others are not. The first sub-chapters also have this issue.

I do not seem to have the problem on my side, and the html converted doc is not provided by us, but by the submission tool :-S

Ah, then it is the tool. No worries. 

Best regards, 

Pierre.

-------------------
Cheers, Rick

Experiences not things.

On Wed, Feb 11, 2015 at 6:27 AM, Camilo Cardona <[email-masked]> wrote:
Hello all,
We submitted a new version of draft-ietf-grow-filtering-threats, which includes the latest suggestions made by Wes. Please let us know if any of you have any more recommendations on how to improve this document.

Thank you,

Camilo Cardona

Begin forwarded message:

From: [email-masked] 
Subject: [GROW] I-D Action: draft-ietf-grow-filtering-threats-04.txt 
Date: 11 Feb 2015 11:50:58 GMT+1
To: <[email-masked]>
Cc: [email-masked] 

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Global Routing Operations Working Group of the IETF.

       Title           : Impact of BGP filtering on Inter-Domain Routing Policies
       Authors         : Camilo Cardona
                         Pierre Francois
                         Paolo Lucente
Filename        : draft-ietf-grow-filtering-threats-04.txt
Pages           : 16
Date            : 2015-02-11

Abstract:
  This document describes how unexpected traffic flows can emerge
  across an autonomous system, as the result of other autonomous
  systems filtering, or restricting the propagation of more specific
  prefixes.  We provide a review of the techniques to detect the
  occurrence of this issue and defend against it.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-grow-filtering-threats/ 

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-grow-filtering-threats-04 

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-grow-filtering-threats-04 

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/ 

_______________________________________________
GROW mailing list
[email-masked] 
https://www.ietf.org/mailman/listinfo/grow 

_______________________________________________
GROW mailing list
[email-masked] 
https://www.ietf.org/mailman/listinfo/grow 

_______________________________________________
GROW mailing list
[email-masked] 
https://www.ietf.org/mailman/listinfo/grow 

_______________________________________________
GROW mailing list
[email-masked] 
https://www.ietf.org/mailman/listinfo/grow

---

From: Camilo Cardona <[email-masked]>
Date: Fri, 13 Feb 2015 13:45:39 +0100

Hello Greg,

Thanks for your comment. Comments inline.

Regards,
Camilo

Begin forwarded message:

> From: Greg Skinner <[email-masked]>
> Subject: Re: [GROW] I-D Action: draft-ietf-grow-filtering-threats-04.txt
> Date: 12 Feb 2015 20:54:52 GMT+1
> To: Rick Casarez <[email-masked]>
> Cc: [email-masked]
> 
> Brief comment inline.
> 

I think we agree here. As I just wrote to Rick, the objective of the draft is to document the situation, and recommend operators to be careful, monitor,  and, in case of a problem, take any decision (technical or contractual) based on their specific case..  

> I don't regularly participate in operations groups such as NANOG or SANOG, but perhaps some people who do might have some opinions about this issue.

Pierre presented this work a few years ago at IDR and Grow. We got the advice to go to NOGs meetings to present. We did it, and also visited operators to get their feedback. The final document is the result of this tour.