Subject: Re: [Pce] Association Source in draft-ietf-pce-segment-routing-policy-cp-01

From: "Mike Koldychev (mkoldych)" <[email-masked]>
Date: Fri, 22 Jan 2021 23:53:13 +0000

Hi WG,

Our latest version of the draft (https://tools.ietf.org/html/draft-ietf-pce-segment-routing-policy-cp-02#section-4.2) addresses the issue that was pointed out in this thread. We are able to avoid the race condition by setting Association Ext ID = <color, endpoint>, so that all the PCEs are able to agree on the same Association ID/ExtID without having to communicate between each other.

Thanks,
Mike.

-----Original Message-----
From: Dhruv Dhody <[email-masked]> 
Sent: Monday, November 23, 2020 11:59 PM
To: Mike Koldychev (mkoldych) <[email-masked]>
Cc: Dhruv Dhody <[email-masked]>; Mike Koldychev (mkoldych) <mkoldych=[email-masked]>; pce-chairs <[email-masked]>; [email-masked]; [email-masked]; Cyril Margaria <[email-masked]>; Stone, Andrew (Nokia - CA/Ottawa) <[email-masked]>
Subject: Re: [Pce] Association Source in draft-ietf-pce-segment-routing-policy-cp-01

Hi Mike,

On Mon, Nov 23, 2020 at 9:04 PM Mike Koldychev (mkoldych) <[email-masked]> wrote:
>
> Hi PCE-CHAIRS,
>
> Would like to understand how does RFC 8697 work in the following scenario:
>
> t=1: PCE_A creates LSP_1 in Association_A (using Source=PCE_A)
> t=2: PCE_B creates LSP_2 and joins Association_A
> t=3: PCE_A deletes LSP_1
>
> Now, PCE_B owns an Association that has the source address of PCE_A. Meanwhile, PCE_A may disconnect from the given PCC and may not be aware that PCE_B is using PCE_A as the Source. So PCE_A can now reuse the same Association ID somewhere else?
>

The stateful PCE model works with the assumption that the LSP state from the network is synchronized to all PCEs in the domain. BTW in the case of a temporary disconnect, synchronization between PCE is the proposed solution. Basically, PCE_A needs to be aware of the continued use of association at the PCC/PCE_B in your example.

Also, when RFC 8697 suggested using virtual IP as a source in case of PCE redundancy, it was with this understanding that the state is synchronized and one PCE would be aware of association created by another, using the virtual IP as association source (this principle can be applied to PCE_A as source as well). Also, in your case above it would be a good implementation principle for PCE_A to not reuse the same association ID immediately if it can help it :)

Anyways as a last resort, there is also a possibility for PCE_B to remove the old association and recreate a new association in an extreme case. Though I would not recommend you do that!

=

RFC 8697 provides a mechanism that is generic and includes the association with LSPs with different headends. PCC based allocation would not work in that case. Isn't it likely that implementations would support other association types as well, and likely implement the generic handling. To me reusing the generic techniques where we can makes sense (that was the very reason for the WG producing RFC 8697).

=

Thanks!
Dhruv (as a WG member)

> Thanks,
> Mike.
>
> -----Original Message-----
> From: Mike Koldychev (mkoldych)
> Sent: Friday, November 6, 2020 2:39 PM
> To: Dhruv Dhody <[email-masked]>
> Cc: Dhruv Dhody <[email-masked]>; Mike Koldychev (mkoldych) 
> <mkoldych=[email-masked]>; pce-chairs 
> <[email-masked]>; [email-masked]; 
> [email-masked]; Cyril Margaria 
> <[email-masked]>; Stone, Andrew (Nokia - CA/Ottawa) 
> <[email-masked]>
> Subject: RE: [Pce] Association Source in 
> draft-ietf-pce-segment-routing-policy-cp-01
>
> Hi Dhruv,
>
> Thanks again for bringing up this topic. Let's think some more about the possible solutions and also let other people provide feedback. Perhaps a side-meeting can be held as well.
>
> Have a good weekend,
> Mike.
>
> -----Original Message-----
> From: Dhruv Dhody <[email-masked]>
> Sent: Friday, November 6, 2020 10:50 AM
> To: Mike Koldychev (mkoldych) <[email-masked]>
> Cc: Dhruv Dhody <[email-masked]>; Mike Koldychev (mkoldych) 
> <mkoldych=[email-masked]>; pce-chairs 
> <[email-masked]>; [email-masked]; 
> [email-masked]; Cyril Margaria 
> <[email-masked]>; Stone, Andrew (Nokia - CA/Ottawa) 
> <[email-masked]>
> Subject: Re: [Pce] Association Source in 
> draft-ietf-pce-segment-routing-policy-cp-01
>
> Hi Mike,
>
> On Fri, Nov 6, 2020 at 7:39 PM Mike Koldychev (mkoldych) <[email-masked]> wrote:
> >
> > Hi Dhruv,
> >
> > -----Original Message-----
> > From: Dhruv Dhody <[email-masked]>
> > Sent: Friday, November 6, 2020 8:27 AM
> > To: Mike Koldychev (mkoldych) <mkoldych=[email-masked]>
> > Cc: Dhruv Dhody <[email-masked]>; pce-chairs 
> > <[email-masked]>; [email-masked]; 
> > [email-masked]; Cyril Margaria 
> > <[email-masked]>; Stone, Andrew (Nokia - CA/Ottawa) 
> > <[email-masked]>
> > Subject: Re: [Pce] Association Source in
> > draft-ietf-pce-segment-routing-policy-cp-01
> >
> > Hi Mike,
> >
> > On Fri, Nov 6, 2020 at 5:08 PM Mike Koldychev (mkoldych) <mkoldych=[email-masked]> wrote:
> > >
> > > Hi Dhruv,
> > >
> > > I don't think it's valid to dismiss race conditions in the protocol because they are "rare". If they can happen at all means that implementations need to have extra logic to handle these race conditions.
> > >
> >
> > Doesn't this "extra" logic exist anyway, as you must make sure there is only one SR policy association with a given set of SR Policy parameters under normal operations.
> >
> > [MK] If the PCC allocates the Association Source/ID, then it's always going to choose a unique value, so there will never be any collision. So no, this "extra" logic won't exist if we go with my proposal.
> >
>
> +1 to what Cyril said. The idea that even when PCE knows about the
> existing association created at the PCC and it wants a new CP to join that association, it cannot do so and it must use association-id as zero is weird - there is a reason we maintain associations at the PCE.
> So my reading was that the use of 0 was only when the SR Policy association was not known.
> Also, you don't mention the PCUpd message in your draft, first an existing SR-TE path (RFC 8664) can join the SR Policy association using the PCUpd message. Also, you may want to update the existing CP and carry the already known association in the PCUpd message. I assume you would want to put "a non-zero PCC allocated ID''!
>
> >
> > > What is rare today in your deployment, may not be rare tomorrow in another deployment. I can think of a case where a PCC connects simultaneously to many PCEs which create many thousands of SR CPs on it. If they all send PCInit messages before the head-end replies to them with PCRpt, then you will have this "rare" race condition repeated thousands of times. Each PCE will choose different Source/ID in PCInit and PCC will have to send PCError back to them.
> > >
> >
> > You make a good point here! What you describe is "possible"!
> >
> > [MK] I would say "very likely" that you would get a flood of PCError messages in that scenario under scale.
> >
> >
> > > Furthermore, you need logic on the PCC to choose the right Association out of many.
> >
> > The logic is the first association created for a given SR policy at the PCC and that's it!
> >
> > > There are also undesirable security/privacy implications of putting PCE/NMS IP address into the Source. It means that if two PCEs are controlling different CPs of the same policy, then one of the PCEs can learn the IP address of the other by reading the Association Source of the PCRpt messages from the PCC. This is especially bad, because this Association Source has no semantic meaning in SR Policy and may be hidden in normal show commands. Furthermore, this IP address of the PCE/NMS that created the policy will be associated to the Policy even after PCE disconnects from the PCC, as long as any CP remains in that Policy.
> > >
> >
> > If any of this is really an issue, we got to update RFC 8697! I am not advocating for that BTW :) Privacy of one PCE from another in an SR domain  (under same administrative control) is quite a stretch IMHO.
> >
> > [MK] The two PCEs may not be under one administrative control. Network A may delegate some tunnels/policies to Network B PCE (for routing through Network B, for example).
>
> Delegation outside of your network domain would open you up to so much more security concern than just knowing another PCE address! Even in the case you mention we would assume there would be a PCEP session between the PCEs across network domains (so hiding PCE address should not be a concern).
>
> > In this case, the internal NMS/PCE addresses of Network A would be leaked. An operator who is not intimate with the PCEP messaging internals might never even realize this leak is happening. I believe we should update RFC 8697, we need to at least state that a privacy violation is possible and that the operator needs to assume that internal NMS/PCE addresses are going to be leaked to other PCEs via the ASSOCIATION object.
> >
> >
> > BTW, what are your thoughts on the operator-configured association in the previous email? Not viable?
> >
> > [MK] You could set AssocExtID=Color, but I’m not sure what you would set Source to? Can we just set it to 0.0.0.0/0::0 and be done? Isn't that also a "special value"?
> >
>
> You can set association-source as head-end IP address which is known as part of SR-Policy tuple <headend, color, endpoint>. This rule can be scoped under the association-type=SR-Policy (and not applicable anywhere else).
>
> >
> > > All of this can be avoided if we just allow Source/ID to be 0 in PCInit messages. Is that really such a big change?
> > >
> >
> > No, but the WG worked on RFC 8697 to make sure all the associations can be handled in a common way as much as possible. When deviating from that, IMHO a higher bar should be met. The WG should ponder if it is met here based on the scenario described above, that's all.
> >
> > [MK] I fully understand, but the value of 0 is reserved in that RFC. Can each application use 0/0.0.0.0/0::0 for its own purpose? Is that allowed/forbidden in the RFC?
> >
>
> The issue for me is not so much the use of 0, it is the idea that it MUST always be a PCC that allocates association information (going against an existing standard). I feel a simple rule of association creation at any PCEP peer such as Assoc-Type=SR-Policy, Assoc-ID=0x0, Ext-Assoc-ID=color, Association-source=headend can solve it!
>
> Have a good weekend everyone!
>
> Thanks!
> Dhruv
>
>
>
> >
> > Thanks!
> > Dhruv
> >
> > PS. Process comment - If the WG decides this is needed (and I am in rough), the procedure needs to be generic and outside the SR policy draft in a separate I-D, so that other associations can also use it.
> >
> > > Thanks,
> > > Mike.
> > >
> > > -----Original Message-----
> > > From: Dhruv Dhody <[email-masked]>
> > > Sent: Friday, November 6, 2020 5:15 AM
> > > To: Mike Koldychev (mkoldych) <[email-masked]>
> > > Cc: Stone, Andrew (Nokia - CA/Ottawa) <[email-masked]>; 
> > > Cyril Margaria <[email-masked]>; [email-masked]; 
> > > [email-masked]; pce-chairs 
> > > <[email-masked]>
> > > Subject: Re: [Pce] Association Source in
> > > draft-ietf-pce-segment-routing-policy-cp-01
> > >
> > > Hi Mike, Andrew, Cyril,
> > >
> > > Thanks for a great discussion, more inline...
> > >
> > > On Fri, Nov 6, 2020 at 7:23 AM Mike Koldychev (mkoldych) <[email-masked]> wrote:
> > > >
> > > > Hi Andrew,
> > > >
> > > > See inline with [MK]
> > > >
> > > > Hi Mike, Dhruv, Cyril:
> > > >
> > > > We do use errors on initiate messages during race conditions, for example, symbolic name uniqueness on pce-initiated vanilla LSPs. So error based protection to enforce uniqueness and protect race conditions is manageable/done today.
> > > >
> > > > [MK] The chances of symbolic-names being the same is much less than the chance of Association Sources being different. Also the symbolic-name actually has an important meaning, the Association Source has no meaning. Getting a flood of PCError messages about a field that has no meaning would be bad. If we can eliminate these error messages completely, why not do that?
> > > >
> > > >
> > >
> > > [DD] First, the flood of errors is a stretch by any means :) And I agree with Andrew about the 'initiate' case.
> > >
> > > >
> > > > However: would it make sense for the SRPAG definition, to be defined by the ‘first’ entity which created the candidate path? when it’s a shared construct with other entities which are now forced to re-use that value? Using a Virtual IP on the PCE(s) would certainly help, but wouldn’t work correctly with mixed use PCC/PCE init candidate paths (would anyone do that?), or different vendor PCE/clusters/different virtual IPs would add more complexity.  The common element I see is the uniqueness on PCC and the fact that SRPAG==SRPolicy. Since Association Source is ‘scoping for the association ID’, and there is no association scoping used/needed, then the value is essentially unused – therefore just a dummy value assigned by PCC?
> > > >
> > > > [MK] Yes it’s unused! I like the idea of PCC choosing it.
> > > >
> > >
> > > [DD] For a dynamic association, the default is for the local PCEP speaker to be the association source unless local policy says otherwise.
> > >
> > > Anyways, based on the requirement that you had in the earlier 
> > > email
> > > -
> > >
> > > Mike> 1. all 3 parties: PCC, PCE1 and PCE2 agree on the same 
> > > Mike> source, AND 2. they agree without talking to each other.
> > >
> > > One can make the SRPolicy association to be considered as an operator-configured association (i.e. association parameters configured by the operator beforehand on the PCEP peers).
> > >
> > > Hear me out, we anyway have the SR Policy configuration happening 
> > > at all peers, could we say that the PCEP association parameters
> > > (type/id/source..) need also be set by the operator. If you are worried that it would be extra configuration, there could be rules set by the operator for filling the association parameters using SRPolicy such as Assoc-type=SR-Policy, Assoc-ID/Extended Association ID=Color, Assoc-source=headend/special value.
> > >
> > > Note that allowing SRPolicy to be both dynamic and operator-configured is also a possiblity.
> > >
> > > >
> > > >
> > > > I think there is a bit of ambiguity as mentioned (PCEP session? Router ID? ), and still run the risk that PCEP is terminating on different addresses towards different PCEs / different view of the ‘PCC address’.  Requesting the PCC to just assign the (unused?) value seems to like a way to avoid all of the above.  With that said, I could be missing other implications/usage of the Association Source field.
> > > >
> > > > [MK] Yes, requesting the PCC to assign a value would resolve this issue. But the question is what value would the PCE send in PCInit message when first creating a policy? I propose that PCE can send just 0.0.0.0 or 0::0 in PCInit messages to indicate to the PCC to pick a value. Alternatively, PCE can send any value of Association Source/ID, but the PCC will not honor it and just choose its own Association Source/ID.
> > > >
> > > >
> > >
> > > I would like us (as a WG) to explore if we can use existing mechanisms first (the very reason we have common association groupings).
> > > As of now, I am not sold that the use of error in a 'rare'
> > > race-condition is such a bad protocol design that we need to 
> > > update
> > > RFC8697 and introduce new rules for dynamic associations, esp when other ways exist.
> > >
> > > What do others think?
> > >
> > > Thanks!
> > > Dhruv
> > >
> > > >
> > > > Cheers
> > > > Andrew
> > > >
> > > >
> > > >
> > > > From: Pce <[email-masked]> on behalf of "Mike Koldychev 
> > > > (mkoldych)" <mkoldych=[email-masked]>
> > > > Date: Thursday, November 5, 2020 at 1:30 PM
> > > > To: Cyril Margaria <[email-masked]>, Dhruv Dhody 
> > > > <[email-masked]>
> > > > Cc: "[email-masked]" <[email-masked]>, 
> > > > "[email-masked]"
> > > > <[email-masked]>, pce-chairs 
> > > > <[email-masked]>
> > > > Subject: Re: [Pce] Association Source in
> > > > draft-ietf-pce-segment-routing-policy-cp-01
> > > >
> > > >
> > > >
> > > > Hi Cyril,
> > > >
> > > >
> > > >
> > > > See inline with [MK]
> > > >
> > > >
> > > >
> > > > From: Cyril Margaria <[email-masked]>
> > > > Sent: Thursday, November 5, 2020 11:35 AM
> > > > To: Dhruv Dhody <[email-masked]>
> > > > Cc: Mike Koldychev (mkoldych) <[email-masked]>; 
> > > > [email-masked]; pce-chairs <[email-masked]>; 
> > > > [email-masked]
> > > > Subject: Re: [Pce] Association Source in
> > > > draft-ietf-pce-segment-routing-policy-cp-01
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I have a related question: how do you define the "PCC address", PCEP session address , one router id?
> > > >
> > > > [MK] By PCC Address, I meant the IP address of the PCEP session. I believe a better approach is actually to set Association Source in PCInitiate message to 0.0.0.0 or 0::0 and let the PCC allocate the correct Source, same as how Association ID allocation is proposed in the draft.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > For the association source race condition, the race condition will result in a "Conflicting SRPAG TLV" from a PCInitiate/PCUpd, the PCE can use the correct SRPAG.
> > > >
> > > > [MK] It’s not a good protocol design to allow PCError messages to appear randomly when all the parties are following the protocol. Would really like to avoid that.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, 5 Nov 2020 at 16:16, Dhruv Dhody <[email-masked]> wrote:
> > > >
> > > > Hi Mike,
> > > >
> > > > On Thu, Nov 5, 2020 at 9:34 PM Mike Koldychev (mkoldych) 
> > > > <[email-masked]> wrote:
> > > > >
> > > > > Hi Dhruv,
> > > > >
> > > > >
> > > > >
> > > > > Perhaps we can avoid this by letting PCE send PCInitiate message with Association Source set to some reserved value, like 0. This can mean that the PCE is basically requesting the PCC to allocate an Association Source and to “own” that Association. We already do this with the Association ID. PCE sets the ID to 0 in PCInitiate and PCC chooses an Association ID and reports it back.
> > > > >
> > > > >
> > > >
> > > > The comment was applicable for association-id as well as 
> > > > association-source! The use of 0 as association ID is being 
> > > > introduced by your draft and not part of the base RFC 8697 and 
> > > > that triggered the original email. Julien and I were 
> > > > uncomfortable with that and wanted to understand what is new 
> > > > here for SR policy association that requires a new procedure and cant be handled by RFC 8697.
> > > >
> > > > Thanks,
> > > > Dhruv
> > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Mike.
> > > > >
> > > > >
> > > > >
> > > > > From: Dhruv Dhody <[email-masked]>
> > > > > Sent: Thursday, November 5, 2020 10:43 AM
> > > > > To: Mike Koldychev (mkoldych) <[email-masked]>
> > > > > Cc: [email-masked];
> > > > > [email-masked]; pce-chairs <[email-masked]>
> > > > > Subject: Re: Association Source in
> > > > > draft-ietf-pce-segment-routing-policy-cp-01
> > > > >
> > > > >
> > > > >
> > > > > Hi Mike,
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Nov 5, 2020 at 7:51 PM Mike Koldychev (mkoldych) <[email-masked]> wrote:
> > > > >
> > > > > Hi Dhruv,
> > > > >
> > > > >
> > > > >
> > > > > Thanks for bringing this up.
> > > > >
> > > > >
> > > > >
> > > > > By setting ASSO_SOURCE = PCC_ADDRESS, we guarantee that:
> > > > >
> > > > > all 3 parties: PCC, PCE1 and PCE2 agree on the same source, 
> > > > > AND they agree without talking to each other.
> > > > >
> > > > >
> > > > >
> > > > > In your proposal below, if we set ASSO_SOURCE = NMS_ADDRESS, it seems that condition 1 may be fulfilled, but it requires exchange of PCRupt/PCUpd messages between the 3 entities, which violates condition 2. Please correct me if I misunderstood something. In the picture that you drew, you say that “Policy Endpoint=X” and “Association Source=X”, are you suggesting to use the policy endpoint as the ASSO_SOURCE? That would satisfy both conditions, but I’m not sure if you intended that?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > No, I did not!
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I believe condition 2 is important to satisfy, because otherwise there could be race conditions where the 3 parties have different ASSOC_SOURCE for the same policy. Consider what happens when all 3 parties try to create the same policy at the same time.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > The SR-Policy association is "dynamic" in nature, and we need to go by the association parameters we receive from the PCEP peer. Condition 2 of talking to each other is the very nature of a dynamic association!
> > > > >
> > > > >
> > > > >
> > > > > If the race condition is the issue to solve, we can use the SR-Policy parameters (color, endpoint, source). And make sure there is only one SR-Policy-association-group with a given set of SR-Policy parameters (and generate an error otherwise). The other PCE would learn about the association and can use it subsequently!
> > > > >
> > > > >
> > > > >
> > > > > I’m open to any proposal, but IMO we should respect the above two requirements.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I feel the requirement 2 is not compatible with a dynamic 
> > > > > association