Subject: [Idr] Use of BGP for Opaque Signaling

From: Robert Raszuk <[email-masked]>
Date: Sat, 12 Mar 2016 16:10:03 -0500

​Petr, Eben,

I have few questions and comments to your proposal.

- The draft ​attempts to reuse BGP mechanism for p2mp distribution of
opaque information. That alone is not a bad thing :) However when you make
reference to BGP-LS it needs to be observed that BGP-LS is not position to
directly impact routing (ie content of RIB and FIBs) unlike one of the use
cases described in your draft Intro paragraph. That means that unlike
BGP-LS it must co-exist with routing BGP on the same BGP speakers.

Here I would like to bring back my proposal to define new port for
Transport Instance BGP https://tools.ietf.org/html/draft-raszuk-ti-bgp-01
which at the time of discussion was supposed to be shortly addressed by
dynamic port allocations of multisessions which to the best of my knowledge
never happened.

So first comment is that if we go with reusing BGP-4 for this type of
signalling let's enforce it's early separation from "classic" BGP.

- Second comment is related to the scope of use. If the scope is private -
let's perhaps enforce it more explicitly such that by the spec it is
illegal AFI/SAFI pair in the public Internet. For example by stating that
it is illegal to have non private AS in the AS_PATH of such UPDATE msg.

- Third the draft states that there is no new security implications .. well
if the scope is non private I am afraid that is no the case. As you have
seen (and still seeing) securing BGP UPDATE MSG is MP_REACH content
dependent. If content is free form I am afraid it will be rather hard to
secure it well :)

- Last as opaque value container you are defining an transitive attribute ?
Is the intention to use this attribute in existing AFI/SAFIs as well ?
Otherwise if not why do you need this to be transitive if capability
negotiation already may implicitly define it's support by new AFI/SAFI.

Cheers,
R.

PS.

> There is an ongoing discussion around avoiding NLRI (key) collisions.

Where is such discussion taking place ? Shouldn't IDR be perhaps in the
loop a bit :)

---

From: "Bertrand Duvivier (bduvivie)" <[email-masked]>
Date: Sat, 12 Mar 2016 23:42:44 +0000

Petr, Eben,

What about L3VPN use-cases ?

If we make this VRF aware,  couldn't we have 2 VPN's sending same / similar opaque updates... and therefor leaking from one VPN to another VPN.

Should we also carry the Route Distinguisher or VPN-id to avoid collision, and of course for non VPN cases, this could be zero.

Best Regards Bertrand Duvivier

---

From: Petr Lapukhov <[email-masked]>
Date: Sun, 13 Mar 2016 00:57:47 +0000

Robert, thanks for your comments!

Re: Impact on routing. I should be more explicit stating that the idea is for BGP to not interpret the key-value mappings in any way. The consumer (running on same device, possibly) may still influence routing based on its interpretation of data, either by accessing the platform SDK or by performing BGP routing injection over the corresponding AFI/SAFI.

Re: Transport instance. I think it's a great idea to run this on separate instance, but right now we typically run just one process, and it adds some operational overhead to maintain two of them. Also, often this extension could be used just with out-of-band reflection. This is why I feel it is rather better to keep this applicable to any type of BGP instance.

Re: Scope of application. The main goal was to use this in data-center, but I can theorize this could be used among few operators that are willing to develop on-device applications and exchange data using the pre-established channels (e.g. the WISER use case). In this situation, limiting scope to just private ASNs could be tricky. Furthermore, non-private ASNs could be used in data-center as well (just not exposed outside...).

Re: Transitive attribute. Ideally, it would be great to carry the value in next-hop field of MP_REACH_NLRI, but the length is limited to 8 bytes, hence the new attribute. I do not have strong arguments to making this transitive, it should be fine to go with non-transitive and corresponding error handling.

Re: Security. Since we do not directly influence path selection with new data, the threats could be limited to those targeting BGPinfra itself e.g. flooding with information, disrupting session etc. There could be questions about information authenticity, but this could be left up to the applications to decide (e.g. signing the key-value bindings).

Regards,

Petr

________________________________
From: [email-masked] [[email-masked]] on behalf of Robert Raszuk [[email-masked]]
Sent: Saturday, March 12, 2016 1:10 PM
To: Petr Lapukhov; [email-masked]
Cc: idr wg
Subject: Use of BGP for Opaque Signaling

​Petr, Eben,

- The draft ​attempts to reuse BGP mechanism for p2mp distribution of opaque information. That alone is not a bad thing :) However when you make reference to BGP-LS it needs to be observed that BGP-LS is not position to directly impact routing (ie content of RIB and FIBs) unlike one of the use cases described in your draft Intro paragraph. That means that unlike BGP-LS it must co-exist with routing BGP on the same BGP speakers.

Here I would like to bring back my proposal to define new port for Transport Instance BGP https://tools.ietf.org/html/draft-raszuk-ti-bgp-01<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Draszuk-2Dti-2Dbgp-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=75Q-nvX2RhhxYm-r2BIxFhtIpKzT9pEgGx9UWjq6fT4&s=kK71eSqWBITalG1Y1iNWaiHaSA7vrMMnL4lkieaq2uY&e=> which at the time of discussion was supposed to be shortly addressed by dynamic port allocations of multisessions which to the best of my knowledge never happened.

So first comment is that if we go with reusing BGP-4 for this type of signalling let's enforce it's early separation from "classic" BGP.

- Second comment is related to the scope of use. If the scope is private - let's perhaps enforce it more explicitly such that by the spec it is illegal AFI/SAFI pair in the public Internet. For example by stating that it is illegal to have non private AS in the AS_PATH of such UPDATE msg.

- Third the draft states that there is no new security implications .. well if the scope is non private I am afraid that is no the case. As you have seen (and still seeing) securing BGP UPDATE MSG is MP_REACH content dependent. If content is free form I am afraid it will be rather hard to secure it well :)

- Last as opaque value container you are defining an transitive attribute ? Is the intention to use this attribute in existing AFI/SAFIs as well ? Otherwise if not why do you need this to be transitive if capability negotiation already may implicitly define it's support by new AFI/SAFI.

Cheers,
R.

PS.

> There is an ongoing discussion around avoiding NLRI (key) collisions.

Where is such discussion taking place ? Shouldn't IDR be perhaps in the loop a bit :)

---

From: Petr Lapukhov <[email-masked]>
Date: Sun, 13 Mar 2016 01:02:15 +0000

Bertrand,

we spent some time discussing whether an explicit multiplexor is needed with the key (NLRI) field. The major problem is agreeing on format and uniqueness of such multiplexor. In the end, the agreement was to make this look more like memcache/redis and let the applications decide on how the data should be properly separated. The content is VPN-agnostic, in the sense that the new AFI/SAFI is completely orthogonal to routing information distribution.

Keyur had some suggestions to make this more "domain-unique", I hope he can comment here as well. I think it's pretty common to feel awkward about flat name-space, but in general there are way to independently generate unique identifiers and distribute them out of band. Since there is so many of them, it is probably better not to include them in specification...

________________________________
From: Bertrand Duvivier (bduvivie) [[email-masked]]
Sent: Saturday, March 12, 2016 3:42 PM
To: Robert Raszuk; Petr Lapukhov; [email-masked]
Cc: idr wg
Subject: Re: [Idr] Use of BGP for Opaque Signaling

Petr, Eben,

If we make this VRF aware,  couldn’t we have 2 VPN's sending same / similar opaque updates… and therefor leaking from one VPN to another VPN.

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 12 Mar 2016 20:25:35 -0500

​Hello Petr,​

> Re: Impact on routing. I should be more explicit stating that the idea is
> for BGP to not interpret the key-value mappings in any way.
>

​That was very clear already. But if you look at number of BGP
implementations recognizing the AFI/SAFI coming over single session is not
done immediately in the BGP message parsing. That is my main point
suggesting transport instance port.

I no longer believe multisessions will do it neither that we could off load
it as implementation specific thing.​

> The consumer (running on same device, possibly) may still influence
> routing based on its interpretation of data, either by accessing the
> platform SDK or by performing BGP routing injection over the corresponding
> AFI/SAFI.
>

​That is exactly the issue. The only way BGP-LS went by was the claim about
their explicit disjointed topology from existing BGP. It was well
positioned that BGP-LS can be completely incongruent with "routing" BGP. ​

​The possible channels the IGP information may impact routing were
classified as out of the band likely coming via some other ways from
controllers (I2RS, APIs, routing BGP SAFI etc ...).

> Re: Transport instance. I think it's a great idea to run this on separate
> instance, but right now we typically run just one process, and it adds some
> operational overhead to maintain two of them. Also, often this extension
> could be used just with out-of-band reflection. This is why I feel it is
> rather better to keep this applicable to any type of BGP instance.
>

​Imagine that you will not get far in this BGP proposal. Then what is left
is nothing else like a new process talking over one of few message buses
today to distribute the set of information you need.

In fact I am not convinced p2mp is the best way to go with here. Pub-sub
without RTC hacks may be much better - don't you honestly think ?

Best,
R.​

> Re: Scope of application. The main goal was to use this in data-center,
> but I can theorize this could be used among few operators that are willing
> to develop on-device applications and exchange data using the
> pre-established channels (e.g. the WISER use case). In this situation,
> limiting scope to just private ASNs could be tricky. Furthermore,
> non-private ASNs could be used in data-center as well (just not exposed
> outside...).
>
> Re: Transitive attribute. Ideally, it would be great to carry the value in
> next-hop field of MP_REACH_NLRI, but the length is limited to 8 bytes,
> hence the new attribute. I do not have strong arguments to making this
> transitive, it should be fine to go with non-transitive and corresponding
> error handling.
>
> Re: Security. Since we do not directly influence path selection with new
> data, the threats could be limited to those targeting BGPinfra itself e.g.
> flooding with information, disrupting session etc. There could be questions
> about information authenticity, but this could be left up to the
> applications to decide (e.g. signing the key-value bindings).
>
> Regards,
>
> Petr
>
> ------------------------------
> *From:* [email-masked] [[email-masked]] on behalf of Robert Raszuk [
> [email-masked]]
> *Sent:* Saturday, March 12, 2016 1:10 PM
> *To:* Petr Lapukhov; [email-masked]
> *Cc:* idr wg
> *Subject:* Use of BGP for Opaque Signaling
>
> ​Petr, Eben,
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Draszuk-2Dti-2Dbgp-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=75Q-nvX2RhhxYm-r2BIxFhtIpKzT9pEgGx9UWjq6fT4&s=kK71eSqWBITalG1Y1iNWaiHaSA7vrMMnL4lkieaq2uY&e=>
> - Third the draft states that there is no new security implications ..
> well if the scope is non private I am afraid that is no the case. As you
> have seen (and still seeing) securing BGP UPDATE MSG is MP_REACH content
> - Last as opaque value container you are defining an transitive attribute
> ? Is the intention to use this attribute in existing AFI/SAFIs as well ?

---

From: Petr Lapukhov <[email-masked]>
Date: Sun, 13 Mar 2016 03:20:28 +0000

> In fact I am not convinced p2mp is the best way to go with here.
> Pub-sub without RTC hacks may be much better - don't you honestly think ?

Not exactly, but maybe I misunderstood what p2mp means :) The main advantage of using BGP (at least in our environment) comes from the fact that it's already bootstrapped when on-switch (or on-server) applications start coming up. And it stays there even when the servers hosting external key-value store may be gone.

Aside from this, I fully agree - it does not offer much more than any key-value store. And it has its own constraints, like ~4K value size. Plus, it's not even a proper pub-sub, since there is no discreet message semantic - only state dissemination :)

________________________________
From: [email-masked] [[email-masked]] on behalf of Robert Raszuk [[email-masked]]
Sent: Saturday, March 12, 2016 5:25 PM
To: Petr Lapukhov
Cc: [email-masked]; idr wg
Subject: Re: Use of BGP for Opaque Signaling

​Hello Petr,​

Re: Impact on routing. I should be more explicit stating that the idea is for BGP to not interpret the key-value mappings in any way.

​That was very clear already. But if you look at number of BGP implementations recognizing the AFI/SAFI coming over single session is not done immediately in the BGP message parsing. That is my main point suggesting transport instance port.

I no longer believe multisessions will do it neither that we could off load it as implementation specific thing.​

The consumer (running on same device, possibly) may still influence routing based on its interpretation of data, either by accessing the platform SDK or by performing BGP routing injection over the corresponding AFI/SAFI.

​That is exactly the issue. The only way BGP-LS went by was the claim about their explicit disjointed topology from existing BGP. It was well positioned that BGP-LS can be completely incongruent with "routing" BGP. ​

​The possible channels the IGP information may impact routing were classified as out of the band likely coming via some other ways from controllers (I2RS, APIs, routing BGP SAFI etc ...).

​Imagine that you will not get far in this BGP proposal. Then what is left is nothing else like a new process talking over one of few message buses today to distribute the set of information you need.

In fact I am not convinced p2mp is the best way to go with here. Pub-sub without RTC hacks may be much better - don't you honestly think ?

Best,
R.​

Regards,

Petr

________________________________
From: [email-masked]<mailto:[email-masked]> [[email-masked]<mailto:[email-masked]>] on behalf of Robert Raszuk [[email-masked]<mailto:[email-masked]>]
Sent: Saturday, March 12, 2016 1:10 PM
To: Petr Lapukhov; [email-masked]<mailto:[email-masked]>
Cc: idr wg
Subject: Use of BGP for Opaque Signaling

​Petr, Eben,

Cheers,
R.

PS.

---

From: Gunter Van De Velde <[email-masked]>
Date: Sun, 13 Mar 2016 10:11:48 +0100

imo the promise of a proposal like this one, is that it allows Application development at DevOps speed and removes slowness of industry bodies for innovation.

I believe it is a valid problem space that is being addressed. Maybe some features/optimisations from EVN could be transplanted? (i.e. Designated router selection, aliasing and mass-withdraw)

G/

> On 13 Mar 2016, at 04:20, Petr Lapukhov <[email-masked]> wrote:
> 
> From: [email-masked] <mailto:[email-masked]> [[email-masked] <mailto:[email-masked]>] on behalf of Robert Raszuk [[email-masked] <mailto:[email-masked]>]
> Cc: [email-masked] <mailto:[email-masked]>; idr wg
> From: [email-masked] <mailto:[email-masked]> [[email-masked] <mailto:[email-masked]>] on behalf of Robert Raszuk [[email-masked] <mailto:[email-masked]>]
> To: Petr Lapukhov; [email-masked] <mailto:[email-masked]>
> Cc: idr wg
> Here I would like to bring back my proposal to define new port for Transport Instance BGP https://tools.ietf.org/html/draft-raszuk-ti-bgp-01 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Draszuk-2Dti-2Dbgp-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=75Q-nvX2RhhxYm-r2BIxFhtIpKzT9pEgGx9UWjq6fT4&s=kK71eSqWBITalG1Y1iNWaiHaSA7vrMMnL4lkieaq2uY&e=> which at the time of discussion was supposed to be shortly addressed by dynamic port allocations of multisessions which to the best of my knowledge never happened. 
> 
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 13 Mar 2016 10:59:48 -0400

Hi Gunter,

For dissemination of information produced by network applications we
already have today various tools be it ansible, chef, puppet etc ...

If there is need to use inter-domain distribution of such applications I
see no reason why not to define registered or private wide bgp community
and use this as transport instead of new SAFI on already established BGP
sessions. Note that wide BGP community precisely like new attribute being
proposed here can carry opaque to BGP data.

Hi Petr,

Do you see any issue with the above wide community approach to address you
needs ? Especially for bootstrapping the transport it is even easier as you
do not need to enable new SAFI. And those are already IDR WG docs.

https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities-01
https://tools.ietf.org/html/draft-ietf-idr-registered-wide-bgp-communities-01

As far as limit of 4K let's all note that we already have a draft
addressing this and moving the limit to 65K:
https://tools.ietf.org/html/draft-ietf-idr-bgp-extended-messages-11

Cheers,
r.

On Sun, Mar 13, 2016 at 5:11 AM, Gunter Van De Velde <
[email-masked]> wrote:

> imo the promise of a proposal like this one, is that it allows Application
> development at DevOps speed and removes slowness of industry bodies for
> innovation.
>
> I believe it is a valid problem space that is being addressed. Maybe some
> features/optimisations from EVN could be transplanted? (i.e. Designated
> router selection, aliasing and mass-withdraw)
>
> G/
>
>
> Not exactly, but maybe I misunderstood what p2mp means :) The main
> advantage of using BGP (at least in our environment) comes from the fact
> that it's already bootstrapped when on-switch (or on-server) applications
> start coming up. And it stays there even when the servers hosting external
> key-value store may be gone.
>
> Aside from this, I fully agree - it does not offer much more than any
> key-value store. And it has its own constraints, like ~4K value size. Plus,
> it's not even a proper pub-sub, since there is no discreet message semantic
> - only state dissemination :)
>
> *Sent:* Saturday, March 12, 2016 5:25 PM
> *To:* Petr Lapukhov
> *Cc:* [email-masked]; idr wg
> *Subject:* Re: Use of BGP for Opaque Signaling
>
> ​Hello Petr,​
>
>
> I no longer believe multisessions will do it neither that we could off
> load it as implementation specific thing.​
>
>
>
> ​That is exactly the issue. The only way BGP-LS went by was the claim
> about their explicit disjointed topology from existing BGP. It was well
>> Re: Transitive attribute. Ideally, it would be great to carry the value
>> in next-hop field of MP_REACH_NLRI, but the length is limited to 8 bytes,
>> *From:* [email-masked] [[email-masked]] on behalf of Robert Raszuk
>> [[email-masked]]
>> - Second comment is related to the scope of use. If the scope is private
>> - let's perhaps enforce it more explicitly such that by the spec it is

---

From: Petr Lapukhov <[email-masked]>
Date: Sun, 13 Mar 2016 16:09:07 +0000

Gunter - that's exactly the point. The idea is to have something that does not require standards body coordination and could be used in ad-hoc fashion. Naturally, there are downsides to this approach (e.g. out-of-band coordination needed), but I feel that keeping it simple is more important. Plus, this simplicity makes implementation really straightforward :)

________________________________
From: Gunter Van De Velde [[email-masked]]
Sent: Sunday, March 13, 2016 1:11 AM
To: Petr Lapukhov
Cc: Robert Raszuk; idr wg
Subject: Re: [Idr] Use of BGP for Opaque Signaling

G/

Cc: [email-masked]<mailto:[email-masked]>; idr wg

​Hello Petr,​

Best,
R.​

Regards,

Petr

​Petr, Eben,

Cheers,
R.

PS.

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=KYJg1DBxc6Ggc4HQyxF3XMMlwslPfWKGarfJDgeWXEU&s=eMk8N44SnsTXls8QN3I-C0sTt3Zv-Be-tCAXrMx0_pw&e=>

---

From: Petr Lapukhov <[email-masked]>
Date: Sun, 13 Mar 2016 16:37:08 +0000

Robert, I totally looked at the wide communities before trying the opaque stuff. However, there was some semantical awkwardness in regard to transporting opaque data. E.g., you need an NLRI of some kind to attach the communities, i.e. every originator needs to inject a prefix - which isn't too bad implementation wise, but ties information distribution to reachability info, requiring a policy to ignore this reachability information. I really don't like adding more policies :)

Furthermore, wide communities could be interpreted by BGP process, and hence their implementation is a bit more involved and requires more interop testing. The opaque AFI simply acts as a pipe to disseminate bits in the domain (e.g. we use thrift to encode values). Implementing it requires accepting the new AFI/SAFI and skipping IP-specific policy code (e.g. filtering only on attributes).

Most importantly, opaque data is "free" of any structure that could require further standardization :) This is on purpose - I was hoping to have this channel defined once, and let it go without any further need of defining additional sub-structures...

PS
The 64K messages would be pretty awesome! :) Having the 4K limit in some cases requires annoying value fragmentation. Ugh.

________________________________
From: [email-masked] [[email-masked]] on behalf of Robert Raszuk [[email-masked]]
Sent: Sunday, March 13, 2016 7:59 AM
To: Gunter Van De Velde
Cc: Petr Lapukhov; idr wg
Subject: Re: [Idr] Use of BGP for Opaque Signaling

Hi Gunter,

For dissemination of information produced by network applications we already have today various tools be it ansible, chef, puppet etc ...

If there is need to use inter-domain distribution of such applications I see no reason why not to define registered or private wide bgp community and use this as transport instead of new SAFI on already established BGP sessions. Note that wide BGP community precisely like new attribute being proposed here can carry opaque to BGP data.

Hi Petr,

Do you see any issue with the above wide community approach to address you needs ? Especially for bootstrapping the transport it is even easier as you do not need to enable new SAFI. And those are already IDR WG docs.

https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities-01<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dwide-2Dbgp-2Dcommunities-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=cGH3bKxTAm1Xiy1qkwo3f69UjqvwTTjKqu2W4nrYhG8&e=>
https://tools.ietf.org/html/draft-ietf-idr-registered-wide-bgp-communities-01<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dregistered-2Dwide-2Dbgp-2Dcommunities-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=bApGAS0Vc65I-5JXPEbRMymvicovHKZYArXUwbZcRaU&e=>

As far as limit of 4K let's all note that we already have a draft addressing this and moving the limit to 65K: https://tools.ietf.org/html/draft-ietf-idr-bgp-extended-messages-11<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dbgp-2Dextended-2Dmessages-2D11&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=44sn2G4s0yMfrbNf7GGtcfd1InmvpRE6axtKhrM3bVU&e=>

Cheers,
r.

On Sun, Mar 13, 2016 at 5:11 AM, Gunter Van De Velde <[email-masked]<mailto:[email-masked]>> wrote:
imo the promise of a proposal like this one, is that it allows Application development at DevOps speed and removes slowness of industry bodies for innovation.

G/

​Hello Petr,​

Best,
R.​

Regards,

Petr

​Petr, Eben,

Cheers,
R.

PS.

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=UEjDtCq5z4uORK2BA9Wqg6I8MnbxxtajRr33DW-dWWM&e=>

---

From: Job Snijders <[email-masked]>
Date: Sun, 13 Mar 2016 17:47:06 +0100

On Sun, Mar 13, 2016 at 04:37:08PM +0000, Petr Lapukhov wrote:
> Most importantly, opaque data is "free" of any structure that could
> require further standardization :) This is on purpose - I was hoping
> to have this channel defined once, and let it go without any further
> need of defining additional sub-structures...

Have you considered using a regular messaging bus such as AMQP or Kafka
instead of bolting this on BGP? 

Kind regards,

Job

---

From: Petr Lapukhov <[email-masked]>
Date: Sun, 13 Mar 2016 17:17:39 +0000

Surely, those work just fine, though I'm more in ZMQ :) As I mentioned before, the major reason to turn to BGP is the "simple bootstrapping" part - once DC network is built up, BGP is available (at least for those who use BGP for routing in DC). Same for inter-AS communications: it would be easier to use existing pre-established channel, rather than agreeing on configuring a shared pub-sub system. Also, co-locating message transport with network allows to be more robust in presence of infra failures - your "messaging" system is as reliable as your network.
________________________________________
From: Job Snijders [[email-masked]]
Sent: Sunday, March 13, 2016 9:47 AM
To: Petr Lapukhov
Cc: Robert Raszuk; Gunter Van De Velde; idr wg
Subject: Re: [Idr] Use of BGP for Opaque Signaling

Kind regards,

Job

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 13 Mar 2016 15:12:38 -0400

Petr,

> E.g., you need an NLRI of some kind to attach the communities, i.e. every
originator
> needs to inject a prefix - which isn't too bad implementation wise, but
ties information
> distribution to reachability info

True about need for NLRI. Not so true about putting equal sign between NLRI
and reachability in existing SAFIs IMHO.

Let's consider that we define a special (v6) prefix which will be used as
an NLRI prefix for your opaque distribution. Part of such prefix (suffix)
can be your key (verbatim) ... values on the other hand can be carried in
wide communities or if you insist in your new attribute. We could even have
one for v4 too - say 240.*.*.*.

It will be much easier to filter or accept such UPDATE messages and policy
required will be pretty much the same as filtering to accept or drop some
keys from your new SAFI. Besides AFI/SAFI 2/1 is already there both on RRs
or ASBRs hence interested parties may not need to upgrade OS running on
those routers.

Besides we have already seen from the other drafts (for example Stephane's
https://tools.ietf.org/html/draft-litkowski-idr-bgp-timestamp-02) the need
to define such special NLRIs to carry opaque information in existing SAFIs
which do not really need to be processed by BGP implementations.

> Furthermore, wide communities could be interpreted by BGP process

Only if you configure them to be processed or they are registered as public
and their types are maintained by IANA. The default is opaque just as you
need it.

Thoughts ?

Best,
Robert.

On Sun, Mar 13, 2016 at 12:37 PM, Petr Lapukhov <[email-masked]> wrote:

> Robert, I totally looked at the wide communities before trying the opaque
> stuff. However, there was some semantical awkwardness in regard to
> transporting opaque data. E.g., you need an NLRI of some kind to attach the
> communities, i.e. every originator needs to inject a prefix - which isn't
> too bad implementation wise, but ties information distribution to
> reachability info, requiring a policy to ignore this reachability
> information. I really don't like adding more policies :)
>
> Furthermore, wide communities could be interpreted by BGP process, and
> hence their implementation is a bit more involved and requires more interop
> testing. The opaque AFI simply acts as a pipe to disseminate bits in the
> domain (e.g. we use thrift to encode values). Implementing it requires
> accepting the new AFI/SAFI and skipping IP-specific policy code (e.g.
> filtering only on attributes).
>
> require further standardization :) This is on purpose - I was hoping to
> have this channel defined once, and let it go without any further need of
> defining additional sub-structures...
>
> PS
> The 64K messages would be pretty awesome! :) Having the 4K limit in some
> cases requires annoying value fragmentation. Ugh.
>
> *Sent:* Sunday, March 13, 2016 7:59 AM
> *To:* Gunter Van De Velde
> *Cc:* Petr Lapukhov; idr wg
> *Subject:* Re: [Idr] Use of BGP for Opaque Signaling
>
> Hi Gunter,
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dwide-2Dbgp-2Dcommunities-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=cGH3bKxTAm1Xiy1qkwo3f69UjqvwTTjKqu2W4nrYhG8&e=>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dregistered-2Dwide-2Dbgp-2Dcommunities-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=bApGAS0Vc65I-5JXPEbRMymvicovHKZYArXUwbZcRaU&e=>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dbgp-2Dextended-2Dmessages-2D11&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=44sn2G4s0yMfrbNf7GGtcfd1InmvpRE6axtKhrM3bVU&e=>
>
> Cheers,
> r.
>
>
>
>
>
>> imo the promise of a proposal like this one, is that it allows
>> Application development at DevOps speed and removes slowness of industry
>> bodies for innovation.
>>
>>> Re: Impact on routing. I should be more explicit stating that the idea
>>> is for BGP to not interpret the key-value mappings in any way.
>>>
>>
>>> Re: Transport instance. I think it's a great idea to run this on
>>> separate instance, but right now we typically run just one process, and it
>>> adds some operational overhead to maintain two of them. Also, often this
>>> extension could be used just with out-of-band reflection. This is why I
>>> feel it is rather better to keep this applicable to any type of BGP
>>> instance.
>>>
>>
>>
>> ​Imagine that you will not get far in this BGP proposal. Then what is
>> left is nothing else like a new process talking over one of few message
>> buses today to distribute the set of information you need.
>>
>>> *From:* [email-masked] [[email-masked]] on behalf of Robert
>>> Raszuk [[email-masked]]
>>> Transport Instance BGP
>>> https://tools.ietf.org/html/draft-raszuk-ti-bgp-01
>>>  which at the time of discussion was supposed to be shortly addressed
>>> by dynamic port allocations of multisessions which to the best of my
>>> knowledge never happened.
>>>
>>> - Last as opaque value container you are defining an transitive
>>> attribute ? Is the intention to use this attribute in existing AFI/SAFIs as
>>> well ? Otherwise if not why do you need this to be transitive if capability
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=UEjDtCq5z4uORK2BA9Wqg6I8MnbxxtajRr33DW-dWWM&e=>
>>
>>
>>
>

---

From: Gunter Van De Velde <[email-masked]>
Date: Sun, 13 Mar 2016 20:42:00 +0100

One of the questions in the equation is if signalling of Opaque information needs potentially modified/different path selection mechanisms compared to a placeholder IPv6 NLRI + wide community.

Ciao,
G/

> On 13 Mar 2016, at 20:12, Robert Raszuk <[email-masked]> wrote:
> 
> Petr,
> 
> > E.g., you need an NLRI of some kind to attach the communities, i.e. every originator 
> > needs to inject a prefix - which isn't too bad implementation wise, but ties information 
> True about need for NLRI. Not so true about putting equal sign between NLRI and reachability in existing SAFIs IMHO. 
> 
> Let's consider that we define a special (v6) prefix which will be used as an NLRI prefix for your opaque distribution. Part of such prefix (suffix) can be your key (verbatim) ... values on the other hand can be carried in wide communities or if you insist in your new attribute. We could even have one for v4 too - say 240.*.*.*. 
> 
> It will be much easier to filter or accept such UPDATE messages and policy required will be pretty much the same as filtering to accept or drop some keys from your new SAFI. Besides AFI/SAFI 2/1 is already there both on RRs or ASBRs hence interested parties may not need to upgrade OS running on those routers. 
> 
> Besides we have already seen from the other drafts (for example Stephane's https://tools.ietf.org/html/draft-litkowski-idr-bgp-timestamp-02 <https://tools.ietf.org/html/draft-litkowski-idr-bgp-timestamp-02>) the need to define such special NLRIs to carry opaque information in existing SAFIs which do not really need to be processed by BGP implementations. 
> 
> Only if you configure them to be processed or they are registered as public and their types are maintained by IANA. The default is opaque just as you need it. 
> 
> Thoughts ? 
> 
> Best,
> Robert.
> 
> 
> On Sun, Mar 13, 2016 at 12:37 PM, Petr Lapukhov <[email-masked] <mailto:[email-masked]>> wrote:
> https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities-01 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dwide-2Dbgp-2Dcommunities-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=cGH3bKxTAm1Xiy1qkwo3f69UjqvwTTjKqu2W4nrYhG8&e=>
> https://tools.ietf.org/html/draft-ietf-idr-registered-wide-bgp-communities-01 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dregistered-2Dwide-2Dbgp-2Dcommunities-2D01&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=bApGAS0Vc65I-5JXPEbRMymvicovHKZYArXUwbZcRaU&e=>
> 
> As far as limit of 4K let's all note that we already have a draft addressing this and moving the limit to 65K: https://tools.ietf.org/html/draft-ietf-idr-bgp-extended-messages-11 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dbgp-2Dextended-2Dmessages-2D11&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=44sn2G4s0yMfrbNf7GGtcfd1InmvpRE6axtKhrM3bVU&e=>
> 
> Cheers,
> r.
> 
> 
> 
> 
> 
> On Sun, Mar 13, 2016 at 5:11 AM, Gunter Van De Velde <[email-masked] <mailto:[email-masked]>> wrote:
>> On 13 Mar 2016, at 04:20, Petr Lapukhov <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/idr <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=CwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=LU_vJaM_EQu1Ssm35j2xlA&m=h-7EJnudbBhYxtlkgN-DvH3EQwxZkBfN9YlkWGqrg8g&s=UEjDtCq5z4uORK2BA9Wqg6I8MnbxxtajRr33DW-dWWM&e=>
> 
>

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 13 Mar 2016 15:53:37 -0400

Gunter,

In BGP path selection happens per prefix. Here prefix is either key alone
(new SAFI) or v4/6+key (existing SAFIs).

So there is rather no difference if you add different set of attributes to
enforce path selection to one or other UPDATE Msgs unless you are talking
about completely new rules for selecting best path in the new SAFI (which
may be rather contr-argument :)

Moreover draft already says that use of add-paths may be useful and that
alone would rather be much simpler to have it enabled for existing SAFIs
(where it is already working) vs in the new ones of quite loosely defined
NLRI semantics.

Cheers,
R.

On Sun, Mar 13, 2016 at 3:42 PM, Gunter Van De Velde <
[email-masked]> wrote:

> One of the questions in the equation is if signalling of Opaque
> information needs potentially modified/different path selection mechanisms
> compared to a placeholder IPv6 NLRI + wide community.
>
> Ciao,
> G/
>
>
>
> True about need for NLRI. Not so true about putting equal sign between
> NLRI and reachability in existing SAFIs IMHO.
>
> https://tools.ietf.org/html/draft-litkowski-idr-bgp-timestamp-02) the
> need to define such special NLRIs to carry opaque information in existing
> SAFIs which do not really need to be processed by BGP implementations.
>
> Only if you configure them to be processed or they are registered as
> public and their types are maintained by IANA. The default is opaque just
> as you need it.
>
> Thoughts ?
>
> Best,
> Robert.
>
>
>> Do you see any issue with the above wide community approach to address
>> you needs ? Especially for bootstrapping the transport it is even easier as
>> you do not need to enable new SAFI. And those are already IDR WG docs.
>>
>>> I believe it is a valid problem space that is being addressed. Maybe
>>> some features/optimisations from EVN could be transplanted? (i.e.
>>> Designated router selection, aliasing and mass-withdraw)
>>>
>>> G/
>>>
>>>
>>>> Re: Security. Since we do not directly influence path selection with
>>>> new data, the threats could be limited to those targeting BGPinfra itself
>>>> e.g. flooding with information, disrupting session etc. There could be
>>>> questions about information authenticity, but this could be left up to the
>>>> - Second comment is related to the scope of use. If the scope is
>>>> private - let's perhaps enforce it more explicitly such that by the spec it
>>>> is illegal AFI/SAFI pair in the public Internet. For example by stating
>>>> that it is illegal to have non private AS in the AS_PATH of such UPDATE
>>>> msg.
>>>>

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 28 Mar 2016 18:30:17 -0400

On Sun, Mar 13, 2016 at 01:02:15AM +0000, Petr Lapukhov wrote:
> Keyur had some suggestions to make this more "domain-unique", I hope he
> can comment here as well. I think it's pretty common to feel awkward about
> flat name-space, but in general there are way to independently generate
> unique identifiers and distribute them out of band. Since there is so many
> of them, it is probably better not to include them in specification...

While I'd echo some of Robert's comments about Wide communities being a
possible fit, I can buy into the argument that it's not quite the right
mechanism for the application.  That said, Wide communities would certainly
help some of the scoping and security considerations.

Comments for the draft:
For key and value lengths, make them two octets in length.  You'll want
something bigger than 255 bytes at some point.

For the key, consider a consistent leading field prior to the key of 4-byte
AS#.  This will let you disambiguate your key space across the Internet at
least.  You might consider a 2 byte "local" value as well, permitting
multiple applications within a given AS to work.  The leading AS# also
provides the ability to easily do some simple policy on key spaces in
"interdomain" applications.

The general purpose distribution graph problem is ugly as noted elsehwere,
but for DC applications I'm not bothered.  This may motivate further
discussion in that problem space at some point.

-- Jeff