Subject: Re: [mpls-tp] wg last call on draft-ietf-mpls-tp-fault-03

From: Maarten vissers <[email-masked]>
Date: Tue, 01 Mar 2011 09:09:19 +0000

Hi Eric,

A specification that is based on unspecified magic control isn't a complete specification and will result in non-deterministic behaviour. OAM behaviour must be deterministic, and this implies that any necessary control must be explicitly specified in the standard and not left to the interpretation of individual designers. As such I disagree with your observation that "it is entirely an implementation issue".

"a priori knowledge" does not exist in transport networks. "Knowledge" in a MEP (e.g. MEP3) about a capability in its server (sub)layers and the actual status of such capability is only present when there is a specified means to obtain this "knowledge".

For this example, there must be a specified communication means between W.MEP7/P.MEP9 and associated SNC Protection Switch instance and MEP3 to exchange this "knowledge". There must also be a specified communication means between MEP3 and MEP1 to exchange this "knowledge".

1) If it would be required to exchange this "knowledge" prior to generation of AIS messages, then there is a need to extend the fault draft with a communication message which relays the status from server (sub)layer MEPs and protection/restoration functions to client (sub)layer MEPs, and a specification how the status is determined. This determination is based on local information and on received information from a server (sub)layer. 
The local information is based on the SF/SD status of the working and protection MEP functions, the presence/absence of protection and/or restoration and the external commands which could force protection/restoration into a special mode (e.g. lockout of protection, forced switch to working, etc).

2) If it would be required to exchange the status via AIS messages (e.g. "A node (say, MEP3) could send L=1, then send L=0 when it receieves L=0"), then it must be specified that the L-flag in received AIS messages has to be used as input for the value of the L-flag in generated AIS messages. Furthermore, the behaviour has to be analysed of such forwarding of this information...

Section 2.1 specifies "The AIS message SHOULD be sent as soon as the condition is detected." and section 5.1 adds "The message MUST be refreshed two more times at an interval of one second.  Further refreshes are sent according to the value of the refresh timer." 

"As soon as the condition is detected" does not give an upper bound of the time to insert the first AIS message for an LSP (or PW). If an interface port has e.g. 1000 LSPs active, how long will it take to have the first AIS message for the last LSP inserted? May we assume that AIS generation is done always in hardware? Or, will it be allowed to do this under software control? 
If hardware based insertion is required, then on a 10GE interface port AIS messages can be minimum length messages (i.e. 64 byte long plus 12-byte inter-frame gap), and generating 1000 AIS messages will take (64 + 12) x 8 x 1000 bit periods = 608000 bit periods. This requires about 60 microseconds. 
If insertion is done under software control, then the insertion of the first AIS message for the last LSP can be after e.g. 60 milliseconds.
Fewer active LSPs per interface port will reduce the time to insertion of first AIS message, more active LSPs per interface port will increase the time to insertion of first AIS message. Lower speed interface ports (e.g. 1GE) will increase time to insertion of first AIS message, and higher speed interface ports (e.g. 40GE, 100GE) may decrease time to insertion of first AIS message.

For the case of the fault scenario described in my original email, AIS messages in MEP3 are generated initially with L-flag=1. When MEP3 receives the first AIS message generated by W.MEP7 with L-flag=0, MEP3 can change its LDI-status from "1"to "0" and when generating the next AIS message it can be generated with L-flag=0. The next AIS message can be e.g. 300 ms after the first AIS message.

If MEP1 is on a router that wants to use the LDI information, then MEP1 on this router receives one or more AIS messages with L-flag=1 followed by AIS messages with L-flag=0. The router must still use a hold off timer before it should react on the L-flag information in the received AIS frame(s), as this L-flag value may change after e.g. 300 ms or longer period. The value of this hold-off timer depends on the required maximum time to insert the first AIS message in the last active LSP and the maximum time to insert the second, third, or forth AIS message. The L-flag change will ripple through the chain; e.g. if the section is protected, then Section MEP generates AIS(L-flag=0), and this L-flag=0 indication has to ripple through the transport-LSP MEP, possibly a transport service SPME MEP and the transport service MEP. In this chain the L-flag=0 information may reach the transport service MEP after being relayed by transport-LSP MEP and transport service SPME MEP (e.g. after 300+300=600 ms).

Conclusion
==========
The second method above is not very deterministic in my opinion. So the first method seems better, but this would require the definition of an additional OAM control message.

Regards,
Maarten

> -----Original Message-----
> From: Eric Osborne (eosborne) [mailto:[email-masked]]
> Sent: 28 February 2011 23:24
> To: Maarten vissers; [email-masked]
> Subject: RE: [mpls-tp] wg last call on draft-ietf-mpls-tp-fault-03
> 
> Hi Maarten-
> 
>   It's not my draft, so I can only respond as a possible implementor.
> But what you ask seems like it is entirely an implementation issue, and
> a pretty straightforward one at that.  A node (say, MEP3) could send
> L=1, then send L=0 when it receieves L=0.  Or MEP3 could have been
> configured with a priori knowledge that the failure of SPME B should be
> L=0.  The draft alludes to this latter approach in section 2.1.1.  The
> way I read the draft, the important thing is that a MEP advertises what
> it knows; how it determines that is out of scope, IMO.
> 
> eric
> 
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On
> Behalf
> > Of Maarten vissers
> > Sent: Monday, February 28, 2011 7:49 AM
> > To: [email-masked]
> > Subject: Re: [mpls-tp] wg last call on draft-ietf-mpls-tp-fault-03
> >
> > Comment on LDI operation:
> >
> > LSPs can be nested, and it is possible to have an LSP stack with e.g.
> five
> > LSPs (A,B,C,D,E) as follows:
> >
> > MEP1-----------------------(A)--------------------------MEP2
> >        MEP3----------(B)----------MEP4   MEP5--(C)---MEP6
> >              W.MEP7--(D)--W.MEP8
> >              P.MEP9--(E)--P.MEP10
> >
> > When LSP(D) fails, LSP(B) and LSP(A) fail as well.
> > W.MEP8, MEP4 and MEP2 will detect this failure and will insert AIS.
> >   - W.MEP8 can send AIS(L-flag=0).
> >   - MEP4 and MEP2 have no immediate knowledge of the
> >     server layer/sublayer protection, and send AIS(L-flag=1).
> >
> > As there is protection present at a server (sub)layer and both
> working
> and
> > protection paths are available before the fault occurs, MEP4 and MEP2
> > should have send AIS(L-flag=0) instead of AIS(L-flag=1).
> >
> > How to control that MEP4 and MEP2 set the LDI flag to "clear" when
> working
> > and protection paths are both available in this case?
> >
> > Regards,
> > Maarten
> >
> >
> > (2011/02/03 20:36), Loa Andersson wrote:
> > > Working Group,
> > >
> > > this is to start a four week working group last call on "MPLS Fault
> > > Management OAM" (draft-ietf-mpls-tp-fault-03).
> > >
> > > Please send comments to the [email-masked] mailing list.
> > >
> > > This working group last call ends on February 28, 2011.
> > >
> > > Loa, George and Ross
> > > MPLS wg co-chairs
> > > --
> > > Loa Andersson email: [email-masked] Sr Strategy and
> > > Standards Manager [email-masked] Ericsson Inc phone: +46 10 717 52 13
> > > +46 767 72 92 13
> > _______________________________________________
> > mpls-tp mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/mpls-tp

---

From: <[email-masked]>
Date: Tue, 1 Mar 2011 11:22:45 +0000

Hmmmm.....

Perhaps some folks might now be starting to get a hint here of why Andy Reid and I decided several years ago that AIS OAM messages were a very bad idea for the co-ps mode (we know they make no sense in the cl-ps mode).  The co-ps mode is not like the co-cs mode (where one MUST fill the resource time-slot with something) as we have to consciously generate traffic unit OAM messages and populate their functional fields....this just creates work, scaling problems and is something else that can go wrong.

And it's quite unnecessary anyway.  So all jolly good reasons not to do this

Moreover, to create AIS just so that one can see artificial nested TCM sublayers working in a non-TOS layer network is, quite frankly, just plain madness....

regards, 

Neil Harrison
BT Design

This email contains BT information, which may be privileged or confidential.
It's meant only for the individual(s) or entity named above. If you're not the intended
recipient, note that disclosing, copying, distributing or using this information
is prohibited. If you've received this email in error, please let me know immediately
on the email address above. Thank you.
We monitor our email system, and may record your emails.
British Telecommunications plc
Registered office: 81 Newgate Street London EC1A 7AJ
Registered in England no: 1800000

> Behalf Of Maarten vissers
> Sent: 01 March 2011 09:09
> To: Eric Osborne (eosborne); [email-masked]
> A specification that is based on unspecified magic control isn't a
> complete specification and will result in non-deterministic behaviour.
> OAM behaviour must be deterministic, and this implies that any
> necessary control must be explicitly specified in the standard and not
> left to the interpretation of individual designers. As such I disagree
> with your observation that "it is entirely an implementation issue".
> 
> "a priori knowledge" does not exist in transport networks. "Knowledge"
> in a MEP (e.g. MEP3) about a capability in its server (sub)layers and
> the actual status of such capability is only present when there is a
> specified means to obtain this "knowledge".
> 
> For this example, there must be a specified communication means between
> W.MEP7/P.MEP9 and associated SNC Protection Switch instance and MEP3 to
> exchange this "knowledge". There must also be a specified communication
> means between MEP3 and MEP1 to exchange this "knowledge".
> 
> 1) If it would be required to exchange this "knowledge" prior to
> generation of AIS messages, then there is a need to extend the fault
> draft with a communication message which relays the status from server
> (sub)layer MEPs and protection/restoration functions to client
> (sub)layer MEPs, and a specification how the status is determined. This
> determination is based on local information and on received information
> from a server (sub)layer.
> The local information is based on the SF/SD status of the working and
> protection MEP functions, the presence/absence of protection and/or
> restoration and the external commands which could force
> protection/restoration into a special mode (e.g. lockout of protection,
> forced switch to working, etc).
> 
> 
> 
> 2) If it would be required to exchange the status via AIS messages
> (e.g. "A node (say, MEP3) could send L=1, then send L=0 when it
> receieves L=0"), then it must be specified that the L-flag in received
> AIS messages has to be used as input for the value of the L-flag in
> generated AIS messages. Furthermore, the behaviour has to be analysed
> of such forwarding of this information...
> 
> Section 2.1 specifies "The AIS message SHOULD be sent as soon as the
> condition is detected." and section 5.1 adds "The message MUST be
> refreshed two more times at an interval of one second.  Further
> refreshes are sent according to the value of the refresh timer."
> 
> "As soon as the condition is detected" does not give an upper bound of
> the time to insert the first AIS message for an LSP (or PW). If an
> interface port has e.g. 1000 LSPs active, how long will it take to have
> the first AIS message for the last LSP inserted? May we assume that AIS
> generation is done always in hardware? Or, will it be allowed to do
> this under software control?
> If hardware based insertion is required, then on a 10GE interface port
> AIS messages can be minimum length messages (i.e. 64 byte long plus 12-
> byte inter-frame gap), and generating 1000 AIS messages will take (64 +
> 12) x 8 x 1000 bit periods = 608000 bit periods. This requires about 60
> microseconds.
> If insertion is done under software control, then the insertion of the
> first AIS message for the last LSP can be after e.g. 60 milliseconds.
> Fewer active LSPs per interface port will reduce the time to insertion
> of first AIS message, more active LSPs per interface port will increase
> the time to insertion of first AIS message. Lower speed interface ports
> (e.g. 1GE) will increase time to insertion of first AIS message, and
> higher speed interface ports (e.g. 40GE, 100GE) may decrease time to
> insertion of first AIS message.
> 
> For the case of the fault scenario described in my original email, AIS
> messages in MEP3 are generated initially with L-flag=1. When MEP3
> receives the first AIS message generated by W.MEP7 with L-flag=0, MEP3
> can change its LDI-status from "1"to "0" and when generating the next
> AIS message it can be generated with L-flag=0. The next AIS message can
> be e.g. 300 ms after the first AIS message.
> 
> If MEP1 is on a router that wants to use the LDI information, then MEP1
> on this router receives one or more AIS messages with L-flag=1 followed
> by AIS messages with L-flag=0. The router must still use a hold off
> timer before it should react on the L-flag information in the received
> AIS frame(s), as this L-flag value may change after e.g. 300 ms or
> longer period. The value of this hold-off timer depends on the required
> maximum time to insert the first AIS message in the last active LSP and
> the maximum time to insert the second, third, or forth AIS message. The
> L-flag change will ripple through the chain; e.g. if the section is
> protected, then Section MEP generates AIS(L-flag=0), and this L-flag=0
> indication has to ripple through the transport-LSP MEP, possibly a
> transport service SPME MEP and the transport service MEP. In this chain
> the L-flag=0 information may reach the transport service MEP after
> being relayed by transport-LSP MEP and transport service SPME MEP (e.g.
> after 3
>  00+300=600 ms).
> 
> 
> 
> Conclusion
> ==========
> The second method above is not very deterministic in my opinion. So the
> first method seems better, but this would require the definition of an
> additional OAM control message.
> 
> Regards,
> Maarten
> 
> 
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 1 Mar 2011 13:36:39 +0200

Loa, and all,
I have two LC comments on the draft. in question. They both refer to issues I've raised in private discussions with George and Matthew and which, to the best of my understanding, have not been resolved in the -03 version.

My first issue deals with potential dangers of sending fault OAM messages in conjunction with certain protection mechanisms.

The example I've presented to George and Matthew  deals with the situation when the LSP in question employs Facility FRR for node protection. 

The topology is shown below:

|------|     |-----|     |-----|     |-----|     |-----| 
|  A   |<--->|  B  |<--->|  C  |<--->|  D  |<--->|  E  |
|------|     |-----|     |-----|     |-----|     |-----|
                \                      /
                 \                    /
                  \                  /
                   \                /
                    \              /
                     \            / 
                      \ |------| /
                       \|   C' |/
                        |------|

The LSP runs A-->B-->C-->D-->E with LSP-level MEPs at A and D  and MIPs in the rest of the nodes.
If the link BC is broken, the link-level MEP at C detects that and initiates (I'll skip the details) insertion of AIS (with LDI flag set after some stabilization) into the LSP. These AIS/LDI packets will be captured by the LSP MEP in D.
Meanwhile, B could operate a node protection bypass (B-->C'-->D) for this LSP, starting at B and terminated at D, D would be the merge point. Note that C would not be aware of this action, and D would not aware of AIS insertion undertaken by C.
As a consequence, E would receive both AIS/LDI generated by C and valid traffic coming thru bypass.

This is definitely not healthy, and the draft does not define any ways to avoid that.

The text in the draft that deals with protection seems to refer to link protection rather than to LSP protection.

IMHO the same problem would appear in the case of segment protection if a link in the middle of a segment is broken, so this issue is not limited just to FRR.

My second issue deals with association between links and LSPs that pass thru these links. Ability to insert Fault OAM messages into all the LSPs crossing a certain link may be compromised IMHO in the case LSPs that use labels from the per-platform space. It is not clear from the draft how the LSR that has detected a link failure could decide whether Fault OAM messages should be inserted in a transit LSP (which it perceives as an ILM table entry) or not - because the actual LSP could be running thru a different link. The problem would presumably disappear if per-interface label space were used; but RFC 5960 does not restrict MPLS-TP from using per-platform label space.

I apologize for sending these comments so late in the LC.

Regards,
     Sasha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Loa Andersson
Sent: Thursday, February 03, 2011 1:37 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: [mpls-tp] wg last call on draft-ietf-mpls-tp-fault-03

Working Group,

this is to start a four week working group last call
on "MPLS Fault Management OAM" (draft-ietf-mpls-tp-fault-03).

Loa, George and Ross

MPLS wg co-chairs

--

Loa Andersson                         email: [email-masked]
Sr Strategy and Standards Manager            [email-masked]
Ericsson Inc                          phone: +46 10 717 52 13
                                              +46 767 72 92 13