Subject: Static VPLS MAC withdrawal draft

From: Giles Heron <[email-masked]>
Date: Tue, 08 May 2012 20:05:31 +0100

Hi everyone,

The following draft was presented at IETF in Paris:

http://tools.ietf.org/html/draft-boutros-l2vpn-mpls-tp-mac-wd-00

The authors have asked for it to be adopted as a WG draft, but at this stage
the chairs' view is that it probably hasn't had enough sets of eyes on it.

So this email is to request that those on the list take a read of the draft
and send comments back to the list.  Depending on the response we get we may
then ask the WG if they feel happy adopting this as a WG draft.

Nabil and Giles

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 10 May 2012 14:11:19 +0000

Resending with less addressees...

Regards,
     Sasha

> -----Original Message-----
> From: Alexander Vainshtein
> Sent: Thursday, May 10, 2012 5:08 PM
> To: [email-masked]:'; Sami Boutros ([email-masked]);
> [email-masked]'
> Cc: [email-masked]; 'Giles Heron'; Rotem Cohen; Andrew Sergeev; Mishael
> Wexler; Gideon Agmon
> Subject: RE: Static VPLS MAC withdrawal draft
> 
> Hi all,
> I've read the draft in question and I would like to clarify several issues that
> look either undefined or ambiguous to me.
> 
> 1. To the best of my understanding, the draft requires each PE participating
> in a given VPLS instance to set up two counters for each static PW
> connecting it to a peer PE device in this VPLS instance: one for static MAC
> withdrawal messages it is going to send and another - for static MAC
> withdrawal messages it receives. Is this understanding correct?
> 
> 2. Assuming a positive answer to the previous questions, how are these
> counters initialized? The draft seems to be moot on this point. The more
> interesting question is, of course, about initialization of the counter of
> received static MAC withdrawal messages, since, with static PWs, generally
> speaking, there is no synchronization between setting up one of its end
> points and setting up another one.
> 
> 3. The draft specifies that:
> 	- The transmitter sends the MAC withdrawal message with the same
> sequence number until it receives an ACK for it (Section 4.1.1)
> 	- The receiver, after having acknowledged a MAC withdrawal
> message, ignores refresh messages with the same sequence number
> (Section 4.1.2)
>   What is supposed to happen in the following scenario:
> 	- Transmitter finds out that some MACs have to be withdrawn (e.g.,
> because the AC from which they have been learned fails)
> 	   and sends a MAC withdrawal message to its peers with sequence
> number N
> 	- Immediately after that (i.e., before it receives an ACK for this
> message) it finds out that yet another set of MAC addresses has to be
> withdrawn
> 	- Retransmission timer expires without ACK received.
> 
> 4. Suppose that only one end point of a static PW between a given pair of
> peers has been set up. In this case transmitting the static MAC withdrawal
> messages via such a PW is possible, but ACKs will never be received. Should
> retransmission of these messages go on "ad infinitum"?
> 
> 5. The draft does not specify any default value for the retransmit time.
> 
> 6. A nit:  Heading of Section 4 is followed by headings for Sections 4.1.1 and
> 4.1.2 without any heading for 4.1.
> 
> Hopefully these questions will be useful.
> 
> Regards,
>      Sasha
> 
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Giles Heron
> > Sent: Tuesday, May 08, 2012 10:06 PM
> > To: [email-masked]
> > Subject: Static VPLS MAC withdrawal draft
> >
> > Hi everyone,
> >
> > and send comments back to the list.  Depending on the response we get
> we
> > may

This e-mail message is intended for the recipient only and contains information which is CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this transmission in error, please inform us by e-mail, phone or fax, and then delete the original and all copies thereof.

---

From: Sami Boutros <[email-masked]>
Date: Thu, 10 May 2012 08:34:39 -0700

Hi Saha,

Thanks for your comments, Please see responses inline..

>> withdrawal messages it receives. Is this understanding correct?

Sami: This is correct.

>> counters initialized? The draft seems to be moot on this point.

Sami: This is what the draft has for setting up the sequence # in section 3.
   Only half of the sequence number space is used. Modular arithmetic
   is used to detect wrapping of sequence number. When sequence number
   wraps (i.e., when it becomes 0), all MAC addresses are flushed and
   the sequence number is reset.
Sami: Can you please explain what's vague in the above?

>> The more

Sami: Initially the rx sequence # can be assumed to be 0.

>> 	- Retransmission timer expires without ACK received.

Sami: A new sequence # will be transmitted that will include both sets of MAC addresses 
Sami: that need to be flushed, and the transmitter will wait for an ack for this new sequence #.
Sami: We can add this to the draft. 

>> retransmission of these messages go on "ad infinitum"?

Sami: Yes, this will be the case, the mechanism will be no different then the static PW status one for this case. 

>> 5. The draft does not specify any default value for the retransmit time.

Sami: Sure will add.

>> 4.1.2 without any heading for 4.1.

Sami: Sure will fix.

>> Hopefully these questions will be useful.

Thanks,

Sami

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 10 May 2012 19:34:42 +0000

Sami,
Lots of thanks for a prompt response.
Please see additional comments/questions inline (marked [[Sasha]]).

Regards,
     Sasha

> From: Sami Boutros [mailto:[email-masked]]
> Sent: Thursday, May 10, 2012 6:35 PM
> To: Alexander Vainshtein
> Cc: [email-masked]; [email-masked]; [email-masked]; Giles Heron
> Subject: Re: Static VPLS MAC withdrawal draft
> 
> Hi Saha,
> 
[[[Sasha]]]  This does not say too much about initialization (at least, not to me).
And it would be nice if the draft was more specific here.
> Sami: Initially the rx sequence # can be assumed to be 0.
[[[Sasha]]] But you've said that Rx number 0 results in flushing all MAC addresses?
> Sami: We can add this to the draft.
[[[Sasha]]] First of all, I think something like that MUST be added to the draft. 
And I am not sure what you say is really OK: consider the case when ACK for the first withdrawal message is received after the 2nd message has been sent.
If one of the ACKed (and withdrawn) MAC addresses has been then re-learned they will be flushed again...
> static PW status one for this case.
[[[Sasha]]] But ACK is not mandatory in the PW status message 9indeed, it is not clear if it is ever needed).
Here you may end with multiple retransmitting messages that are sent to a black hole.
> > This e-mail message is intended for the recipient only and contains
> information which is CONFIDENTIAL and which may be proprietary to ECI
> Telecom. If you have received this transmission in error, please inform us by
> e-mail, phone or fax, and then delete the original and all copies thereof.
> >

---

From: Sami Boutros <[email-masked]>
Date: Thu, 10 May 2012 18:21:47 -0700

Hi Sasha,

> And it would be nice if the draft was more specific here.

Sami: Agreed, we need to add the initialization value to the text.
 
> [[[Sasha]]] But you've said that Rx number 0 results in flushing all MAC addresses?

Sami: Good point, but this is why the receiving PE should always flush on receiving 0.
Sami: The Transmitter PE would then increment the sequence # and subsequent 
Sami: Transmission will be higher than 0. 

> If one of the ACKed (and withdrawn) MAC addresses has been then re-learned they will be flushed again…

Sami: Yes, this will lead to deleting again, and then the macs will be learnt again, keep in mind that we are talking about using a mac list (not an empty mac list), and deleting 2 sets of macs and we didn't Sami: get the ack for the 1st set in time before we ask to delete the 2nd set. I would think this should be ok, however am open to better schemes.

> Here you may end with multiple retransmitting messages that are sent to a black hole.

Sami: Wouldn't that be TRUE too for PW status messages? however is there a scheme you have in mind to solve this?

Thanks,

Sami

---

From: Alexander Vainshtein <[email-masked]>
Date: Fri, 11 May 2012 05:11:31 +0000

Sami,
Lots of thanks for prompt responses.

My general gut feeling is that the draft tries to compensate for lack of reliable delivery provided by TCP in the case of signaled PWs between the VPLS peers, but leaves aside some other issues that stem from the static nature of the PWs. These include at least the following:

- Synchronized setup of the PW endpoints.   Life would be simpler if the two endpoints could guarantee that they have both come up and both deal with the same values of Rx and Tx sequence numbers (cross-matched).

- Synchronized reset for the sequence numbers: consider the case when one of the PEs undergoes restart in the middle of operation and starts with a new pair of Rx and Tx numbers while its peers remain unaware of the fact.

- Ordered (in addition to reliable) delivery of messages: consider the case with two messages following each other (from my original example), but the first one being lost. The 2nd one (including all MAC addresses from the 1st one) is delivered and ACKed, then the 1st one is retransmitted and delivered...

One thing that the draft does not mention is management of the list of MAC addresses that should be withdrawn but have not be ACKed for withdrawal (an analog o the TCP socket Tx buffer). Such a list is required per PW, and it would be nice to understand how it behaves.

I must admit that I do not have a better scheme in mind at the moment (that is, if you do not count using TCP as a valid better scheme :-).
But using some non-TCP mechanism for synchronized, ordered and reliable delivery (e.g., one used in OSPF?) may be worth consideration.

IMHO and FWIW, if your colleagues and you prefer to keep it as simple as possible, the draft should at least describe possible pitfalls and their consequences (say in an Applicability section).

Last but not least:
I may be exceptionally dumb these days, but could you please clarify which "half of the sequence number space" do you have in mind, and which modulo do you use in the "modular arithmetic"?

Regards,
     Sasha

> Sent: Friday, May 11, 2012 4:22 AM
> To: Alexander Vainshtein
> > And I am not sure what you say is really OK: consider the case when ACK
> for the first withdrawal message is received after the 2nd message has been
> sent.
> Sami: Yes, this will lead to deleting again, and then the macs will be learnt
> again, keep in mind that we are talking about using a mac list (not an empty
> mac list), and deleting 2 sets of macs and we didn't Sami: get the ack for the
> 1st set in time before we ask to delete the 2nd set. I would think this should
> be ok, however am open to better schemes.
> 
> 
> >>
> >> Sami: Yes, this will be the case, the mechanism will be no different then
> the
> >> static PW status one for this case.

---

From: Neil McGill <[email-masked]>
Date: Fri, 11 May 2012 07:24:40 -0700 (PDT)

> - Ordered (in addition to reliable) delivery of messages: consider the case with two messages following each other (from my original example), but the first one being lost. The 2nd one (including a…

I'm concerned that if we are concatenating MAC addresses for the 2nd
message then we are going to hit size limits and will need to deal with
fragmentation. And we have the double flush potential too.

A lot of this complexity could be avoided if we simply did a flush-all
with no MACs. Are other vendors actually flushing MACs on a per basis ?
At least until now all I've seen is flush-all behaviour.

If we're going for 'as simple as possible' I'd opt to take out the
option to flush individual MACs and go for 'flush all'

If we are going to continue with per MAC flushing I do agree that we
need a more reliable delivery. My preference is that we only ack what
we have received and we transmit until acked. So if msg 1 is lost and
msg 2 received then the peer will not ack both, but would continue to
request msg 1; just like tcp/ospf/lap*/l2tpv3 would do.

Thanks,

Neil.