Subject: [GROW] Dropped Updates in BMP?

From: Job Snijders <[email-masked]>
Date: Tue, 20 Mar 2018 13:52:23 +0000

Hi all,

Reudiger Volk mentioned something interesting at the microphone
yesterday about getting more visiblity into BGP UPDATES that are
rejected/dropped somewhere in the policy chain transitioning from
Adj-RIB-In to Loc-RIB.

To make a crude route-map example:

    ip prefix-list allow-ebgp-in permit 192.0.2.0/24
    !
    route-map ebgp-in permit 10
        match ip address prefix-list allow-ebgp-in
    !
    route-map ebgp-in deny 20
        bmp-log-code 21438

It would be great to see what UPDATEs get dropped in "route-map ebgp-in deny 20".
It would perhaps be quite useful if we can get to the point where you
can even attach custom policy-exit codes to the "Dropped Updates" send
in this new BMP feed. I can see how this makes operational life easier.

RFC 4271 Section 9.1: "The Decision Process selects routes for
    subsequent advertisement by applying the policies in the local
    Policy Information Base (PIB) to the routes stored in its
    Adj-RIBs-In. The output of the Decision Process is the set of
    routes that will be advertised to peers; the selected routes will be
    stored in the local speaker's Adj-RIBs-Out, according to policy."

Perhaps a series of BMP "PIB" drafts are in order?

Is this worthy of a new BMP draft? Are there volunteers?

Kind regards,

Job

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 20 Mar 2018 10:14:02 -0400

Job,

> in this new BMP feed. I can see how this makes operational life easier.

The "exit code" is fairly reasonable.  The "dropped updates" can be
problematic depending on what you meant.

If you meant that the routes were discarded and not kept as inactive,
implementations may need mirror mode support since the route would not be
stored in the RIB.

Routes that are simply rejected but stored in the rib (e.g. "keep all" in
JunOS) can be reported as part of post-policy monitor mode.

> Perhaps a series of BMP "PIB" drafts are in order?

PIB docs were mostly left out of scope except as a general abstraction to
talk about in the BGP standardization effort because no one can agree on The
One True Policy Engine.  See similar conversation occurring right now in the
rtgwg policy yang model.

> Is this worthy of a new BMP draft? Are there volunteers?

I'd suggest keeping the scope to the bmp-log-code.

-- Jeff

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 20 Mar 2018 15:18:07 +0000

Pedantry perhaps, but I think you mean "routes" and not "UPDATEs". No implementation should literally be dropping UPDATE messages under any circumstances (well, unless it closes the session then). For that matter, they shouldn't be dropping routes (in the sense of discarding them unprocessed), so I think you mean "filtered by policy" and not "dropped".

£0.02,

--John

> On Mar 20, 2018, at 1:52 PM, Job Snijders <[email-masked]> wrote:
> 
> Hi all,
> 
> _______________________________________________
> GROW mailing list
> [email-masked]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_grow&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=sDs-eutrXrd8oD5o2L4-hgQq8DGzPAt5gosqFOcsk18&s=XDz9VZoMEh4fZP-HmZroeFV27-iRnvtsLb9SEhfGLfw&e=

---

From: Job Snijders <[email-masked]>
Date: Tue, 20 Mar 2018 15:28:04 +0000

On Tue, Mar 20, 2018 at 03:18:07PM +0000, John G. Scudder wrote:
> Pedantry perhaps, but I think you mean "routes" and not "UPDATEs". No
> implementation should literally be dropping UPDATE messages under any
> circumstances (well, unless it closes the session then). For that
> matter, they shouldn't be dropping routes (in the sense of discarding
> them unprocessed), so I think you mean "filtered by policy" and not
> "dropped".

Dear John,

I don't consider this "pedantry" at all. I think it is very important to
get the terminology correct, this helps creating a meeting of the minds :-)

Thanks for the clarification, I indeed mean "routes".

Kind regards,

Job

---

From: "Tim Evens (tievens)" <[email-masked]>
Date: Tue, 20 Mar 2018 16:44:28 +0000

I +1 that this is covered by post-policy, but there are some caveats.   In order to determine what was "dropped/filtered/rejected/whatever" we do a simple diff between pre-policy and post-policy.  Caveat is that this requires the BMP sender implementation to support both pre & post policy at the same time. Currently JunOS supports this, but others do not.     For BMP senders that do not implement both pre & post for the same neighbor/peer, we have to glean the post-policy by other means.  

I've run into a few support issues with folks that have been comparing Adj-RIB-In pre-policy to what the router shows via CLI (e.g. bgp sum counts).   They understand the idea of post-policy but they were confused when they saw different counts on peers that had no filters/policy applied.   I explained to them that it was likely rejected prefixes, not filtered. They were still unsure and wanted to have a better understanding why there are so many rejected prefixes.   

BMP includes stat counters, such as duplicates, invalidate due to ...   I was able to show "quickly" via these counters that if you subtract the invalidated/duplicate counts from the total, you get the number of RIB entries shown in CLI.   In this case, the issue for them was AS Path loops.  I was able to provide them a query to show the pre-policy prefixes that were rejected by the router due to AS Path loop.  

In the above example, they didn't have post-policy turned on so we couldn't easily do a diff.   Had they had post-policy we could do a diff instead of querying for the specific entries that were rejected (known via the stat counters). 

Even with pre/post diffs and bmp stat counters, we still don't have a reject/filter reason associated per route/prefix.  Instead, we have to "determine" the reason based on the diff. 

I also had discussions with users regarding how they are only interested in the diff output of pre/post and not the post RIB itself.  

I believe there is value in possibly adding a new Adj-RIB-In-PostDiff table/conveyance, which would contain ONLY the NLRI's that are filtered/rejected.  If we had this table, we could add flags or something similar to indicate the reason the NLRI is in the diff table, such as rejected due to AS Path loop.   This might get a bit complicated though. 

Thanks,
Tim

On 3/20/18, 7:14 AM, "GROW on behalf of Jeffrey Haas" <[email-masked] on behalf of [email-masked]> wrote:

    Job,

    

    

    
    -- Jeff
    
    _______________________________________________
    GROW mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/grow

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 20 Mar 2018 13:10:43 -0400

> I believe there is value in possibly adding a new Adj-RIB-In-PostDiff table/conveyance, which would contain ONLY the NLRI's that are filtered/rejected.  If we had this table, we could add flags or…

Even JunOS will have issues depending on types of loops.  In several
circumstances, routes are discarded rather than held as inactive.  E.g. PE
mode for VPN address families and no matching route targets.

This is a circumstance I would simply counsel that routes which are not
stored in the RIB may not be present in BMP output except perhaps in mirror
mode.  (And we don't support mirror mode.)

-- Jeff

---

From: Paolo Lucente <[email-masked]>
Date: Tue, 20 Mar 2018 19:20:35 +0100

Minor comment: my understanding is that Reudiger was interested in getting more visibility in what happens between Adj-RIB-In pre- and post- policy. 

I concur with Tim Evens comment “In order to determine what was "dropped/filtered/rejected/whatever" we do a simple diff between pre-policy and post-policy", which is essentially the same i replied to Reudiger as in what can be done with what we have. Speaking to pmacct users using BMP, the ability to diff pre- and post- policy was found good enough as a starting point to further research what happened to missing routes (through means external to BMP).

It would be nice to get to some sort of increased visibility and have a kind of ‘exit code’, as Jeff Haas described it, when a route is filtered (‘f’ flag to be ported to Adj-RIB-In and Adj-RIB-Out?) and I reckon things may get complicated if trying to stretch the concept too much beyond this point. I’d be willing to contribute effort if it is found that there is enough interest. 

Paolo

> On 20 Mar 2018, at 14:52, Job Snijders <[email-masked]> wrote:
> 
> Hi all,
>

---

From: "Serpil Bayraktar (serpil)" <[email-masked]>
Date: Tue, 20 Mar 2018 18:54:06 +0000

+1

Not knowing which policy is filtering which route has been the #1 reason for operators not wanting to touch existing policy configuration on the routers AFAIK. Hence the enormous size of policy configs. Not to mention not having an easy way to validate whether a new policy is filtering what it is intended to filter or not.
If there is any visibility into why a route was filtered that can be relayed via BMP, it’d be super useful. I do agree that it should probably be another draft as Job suggested.

As for those routes that are dropped before even hitting the policy filters (such as due to errors in the path attributes like AS path loops), I think the operators need to step in here to indicate how valuable this information is to them and whether the vendors can provide them without harming the operation of the router or not.

Serpil

From: GROW <[email-masked]> on behalf of Paolo Lucente <[email-masked]>
Date: Tuesday, March 20, 2018 at 11:20 AM
To: Job Snijders <[email-masked]>
Cc: Grow Mailing List <[email-masked]>
Subject: Re: [GROW] Dropped Updates in BMP?

Paolo

On 20 Mar 2018, at 14:52, Job Snijders <[email-masked]<mailto:[email-masked]>> wrote:
Hi all,
[email-masked]<mailto:[email-masked]>

_______________________________________________
GROW mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/grow

---

From: <[email-masked]>
Date: Thu, 22 Mar 2018 05:24:23 +0000

Hi all,

Thank you very much for the updated drafts and presentation at IETF 101 regarding bmp and Adj-RIB-Out and Local-RIB support. I have been reading the updated draft and fully support them.

Regarding route filtering the comment from Reudiger Volk. From a network operator point of view, I share Serpil's comments. I also share Paolo's concerns bringing in more complexity could potentially lead being less adapted by vendors. I share Job's suggestion that this should if being addressed in a separate draft.

At current state BMP is able to answer the question at which point (Adj-RIB-In, Adj-RIB-Out, Local-RIB) how much and which routes are filtered, passed or when passed where the route attributes were modified. What BMP can't answer is why these routes where passed or when passed, why where the route attributes modified. And as Serpil already explained, for a network operator, if complex and historical evolved route-policy configuration is in place, can be quiet cumbersome to troubleshoot. Especially as route-policy configuration are not standardized and differ between vendors. An there, BMP could bring valuable insights in, which we should not ignore.

How about for instance adding three fields to each route in Adj-Post-RIB-In , Local-RIB and Adj-Post-RIB-Out:

Action:              Passed, Filtered, Modified
Place:               route-policy name or function within router. An example for functions could be: route-target filter
Debug:             free text of what has been modified if modified. An example: modified next-hop to 1.1.1.1

I fully understood that adding these fields has its price and therefore it should be addressed and discussed in a new separate draft, especially since Adj-RIB-In is also affected. From my perspective, where I disagree, is that the a filtered flag alone would be enough to address the wish to get more visbility into the filter behavior since it would not explain why it has been filtered.

Regarding Serpil's question wherever it would be sufficient only to flag routes if filtered by route-policy. From my point of view I can confirm that this is the most interesting part, but as I described above, I fear that it might be not enough when we start going down that road.

Kind regards
Thomas Graf
____________________________________________________________________________
Network Engineer
Tribe IT Clouds
Telefon +41-58-223 84 01
Mobile   +41-79-728 80 12
[email-masked]<mailto:[email-masked]>
____________________________________________________________________________
Swisscom (Schweiz) AG
IT, Network & Infrastructure
Tribe IT Clouds
Binzring 17
8045 Zürich
www.swisscom.com
Postadresse:
Binzring 17
8045 Zürich

From: GROW [mailto:[email-masked]] On Behalf Of Serpil Bayraktar (serpil)
Sent: Tuesday, March 20, 2018 7:54 PM
To: Paolo Lucente <[email-masked]>; Job Snijders <[email-masked]>
Cc: [email-masked]
Subject: Re: [GROW] Dropped Updates in BMP?

+1

Serpil

From: GROW <[email-masked]<mailto:[email-masked]>> on behalf of Paolo Lucente <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, March 20, 2018 at 11:20 AM
To: Job Snijders <[email-masked]<mailto:[email-masked]>>
Cc: Grow Mailing List <[email-masked]<mailto:[email-masked]>>
Subject: Re: [GROW] Dropped Updates in BMP?

Paolo