Subject: [pim] pim-state mailing list shutting down

From: Tom Pusateri <[email-masked]>
Date: Fri, 11 Nov 2005 15:48:16 -0500

The pim-state mailing list was created for a small design team to
work on pim state refresh reductions. That team now needs more input
from the PIM working group and L3VPN working group. To encourage this
and allow others to stay in touch with the continued effort, I have
decided to close down the pim-state mailing list and move all of its
traffic over to the main [email-masked] list.

All further discussion about the PIM State Refresh should take place on the
PIM mailing list ([email-masked]).

The archives will remain available for a long while in case anyone
wants to peruse them. They can be found at:

http://www.mountain2sea.com/pipermail/pim-state/

Thanks,
Tom

_______________________________________________
pim mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pim

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 12 Nov 2005 13:35:50 -0800

>> traffic over to the main [email-masked] list.

    Enclosed is the current thinking for the JP-Ack proposal I summarized
    at the PIM working group meeting.

    And I would like the working group to judge if this should be an extension
    to PIMv2 or an addition to PIMv3. I think the later so we can get PIMv2
    to Draft Standard.

Dino

-------------------------------------------------------------------------------

      JP-Ack Proposal (for reducing periodic JP messages in PIM)
      ----------------------------------------------------------

[1] Introduce a JP-Ack PIM message. It is always multicasted to 224.0.0.13 or
    ff02::d. Contains same format as JP message.

[2] JP-Ack-Capable routers indicate they can send and receive JP-Acks by
    including JP-Ack Hello Option in Hello messages.

[3] What JP senders do:

    o When they send JPs to an upstream router which is JP-Ack-Capable, they
      set a Ack-Received flag to FALSE for each route that is inserted in the 
      join- or prune-list portion of the JP.

    o Then start a 1 second jittered retransmission timer. Retransmission 
      occur until the upstream router disappears from the neighbor cache. The 
      retransmission timer can optionally be altered but cannot exceed 60
      seconds (the default periodic interval).

    o When the timer expires, build JP for all routes with the Ack-Received
      flag set to FALSE.

    o When the 1 minute periodic JP interval expires, only include routes
      which have the Ack-Received flag set to FALSE.

[4] What JP-Ack senders do:

    o After a received JP is processed, modify the type field to turn the
      message into a JP-Ack. No other fields change in the PIM payload message
      other than the type and checksum fields.

    o For each route in the JP, set Ack-Sent flag to TRUE. This signifies that
      any oif-list entry will not time out regardless of the holdtime value
      sent in the JP. This allows the JP transmitter to not worry about 
      refreshing the route.
 
    o If the PIM checksum check fails, drop the JP message and do not return
      a JP-Ack.

[5] What do receivers of JP-Acks do:

    o Regardless if you sent an JP or not, for each entry in the JP which
      you have state for, set the Ack-Received flag to TRUE so at the next 
      periodic interval, the route is not included in a periodic JP. During 
      steady-state, there will be no routes with Ack-Received equal to FALSE, 
      so no JPs are sent.

[6] What about triggered JPs?

    o When new state is created, the Ack-Received flag is set to FALSE, so a 
      JP message is triggered. And will later be JP-Ack'ed.

    o When an RPF change occurs, for each route that is affected, set the
      Ack-Received flag to FALSE and trigger a JP.

[7] What happens when a joiner goes down?

    o When a single joiner goes down, you want the upstream router to start
      timing out routes.

    o When there is more than one joiner, and a joiner goes down, you want to
      make sure timeout-suppression still occurs.

    o Whenever a PIM neighbor goes down, upstream routers will set the Ack-Sent
      flag to FALSE which will cause timeout processing to start.

    o Whenever a PIM neighbor goes down, downstream routers will set the
      Ack-Received flag to FALSE which will cause periodic processing to 
      start. Which means when the 1 minute periodic interval expires, JPs 
      will be sent. This will scatter the JPs across a jittered 1 minute
      periodic interval.

    o If, from an upstream router's perspective, a downstream neighbor times 
      out but the downstream neighbor thinks the upstream router is alive
      (an issue with one-way connectivity), the upstream router should
      send a triggered Hello with a new GENID value. This will cause the
      downstream router to detect an adjacency change. In this case, the
      downstream router will send a full set of JPs. This is necessary so
      upstream router does not time out oifs when the downstream neighbor
      thinks it doesn't need to send a JP message.

      It is important to note that the Hello be unicast with a GENID value of
      0. The Hello is unicast so other neighbors on the LAN don't get involved
      in sending JPs. The reason for a GENID value of 0, is so the downstream
      router can get solicited to build a full set of JP messages. When the
      next hello interval expires, the upstream router will transmit a Hello
      with a GENID equal to it's last value. This way other neighbors don't
      think there is a neighbor up/down transition.

[8] Changing an entry for join-state to prune-state:

    o When a prune-entry (negative cache state entry) has an empty oif-list
      which goes to non-empty, a Join will be triggered, in this case the
      Ack-Received flag is set to FALSE so the Join can be sent reliably and
      the now join-entry can be suppressed from future JPs.

    o When a join-entry which has an non-empty oif-list goes to empty, a Prune
      will be triggered, in this case the Ack-Received flag is set to FALSE so
      the Prune can be sent reliably and the now prune-entry can be suppressed
      from future JPs.

    o In the case an entry changes from a join-entry to a prune-entry or
      vice versa before an JP-Ack is received for the previous state should
      cause no problem since the JP sender only cares about the latest state
      it has. If a JP-Ack is returned for a entry that is in the Join-list
      but the cached state is a prune-entry (or in the Prune-list for a
      cached join-entry), the Ack-Received flag is not touched (this is an old
      Ack).

    o In the case a JP sequence (3 separate JP messages) consists of a {Join, 
      Prune, Join} for the same multicast route, if the an JP-Ack for the first
      JP-Join message is received by the downstream router after it had
      sent the third JP Join message, the downstream router would not ever
      retransmit a JP-Join message. In the case, the third JP-Join message
      was lost, the upstream router would have prune-state where the downstream
      router would have join-state. To avoid this situation, an implementation
      can delay sending the third JP-Join awaiting the JP-Ack for the first 
      JP-Join.

      The same situation occurs with a JP sequence of {Prune, Join, Prune} for
      the same multicast route. In this case, the upstream router keeps 
      forwarding data since it has join-state but the downstream router has
      prune-state.

      This situation should be rare since the rate of a multicast route flap
      would most likely be greater than a JP retransmission interval.

[9] Duplicate JP-Acks and JPs:

    o If a JP-Ack is received for a set of routes which already have the
      Ack-Received flag set to TRUE, no action is taken. This is known as
      a duplicate JP-Ack. This is the case where a router on a multi-access 
      LAN either didn't received the JP-Ack or it's JP was not received by the
      upstream router. So a retransmission of either had occurred.

    o If a JP-Ack is received for a route where the JP-Ack has a prune-list
      entry for the route but the route has join-state (oif-list not empty),
      then the Ack-Received flag should be set to FALSE so retransmission
      of a Join can happen. The converse is true if the JP-Ack had a join-list
      entry for a route in prune-state (oif-list that is empty). This can
      happen when a route transition from join-state to prune-state (or vice-
      versa) occurs and either the JPs or the JP-Acks were reordered.

    o If an upstream router receives a JP for a set of routes which already
      have the Ack-Sent flag set to TRUE, a JP-Ack should be returned. This
      is a case where a downstream router didn't not receive the previously
      sent JP-Ack or has triggered a JP it wants an JP-Ack for.

[10] RPF Changes:

    o When an RPF neighbor change occurs, a JP with join-list entries is sent
      to the newly determined RPF neighbor. It is optional to also send a
      JP with prune-list entries to the previous RPF neighbor. In the later
      case, while the JP-prune-list sender is waiting for a JP-Ack from the
      previous RPF neighbor, other RPF change could occur. An implementation 
      has to remember the previous RPF neighbors so it can make sure the 
      prune-list JP gets reliably sent.

[11] Dealing with Prune-Override on multi-access LANs:

    o On multi-access LAN, a downstream router that is pruning for a multicast
      route will multicast the JP message on the LAN. The upstream router
      will delay removing the oif from the multicast route waiting for other
      downstream routers to override the Prune with a Join message. To make
      this sequence reliable would require a total of 4 messages. The initial
      JP-Prune triggers the sequence from the downstream router, the upstream
      router responds with a JP-Ack multicast to the LAN, then an overriding
      router (if one exists) will send a multicast JP-Join which causes the
      upstream router to return a JP-Ack for the Join.

    o The reliability problem occurs because an overriding router may not see
      the triggering JP-Prune or the upstream router's JP-Ack. Even though 
      the overriding router has 2 chances to send the Join to override the
      Prune, both messages could be lost during a congestion period. Since
      the join-state in the downstream router had previously been acknowledged
      by the upstream router, the downstream router would never send the Join
      again.

    o There are two solutions to the problem, (1) Upstream Router Join
      Solicitation or (2) Explicit Tracking.

    o Upstream Router Join Solicitation is when the upstream router will
      forward the received JP-Prune back out the multi-access LAN and require
      JP-Acks from each downstream neighbor regardless if they want to Prune-
      override or not. This assures that all neighbors have the chance to
      send their Joins. In this case, not all neighbors have to send Joins.
      The Join suppression mechanism from the base PIM spec can be used.

    o Explicit Tracking is when the upstream router keeps track of each
      neighbor who has Joined for a particular multicast route. So when a 
      JP-Prune is sent to the upstream router, it knows when the last Joiner
      stops joining. In this case, only the JP-Prune needs to be JP-Acked
      and the oif is removed from the multicast route only when the last 
      Joiner sends a Prune. This will trade-off memory for less messages on
      the LAN.

    o We propose when Explicit Tracking is used, that each router on the LAN
      advertise the fact in a PIM Hello option. This allows the downstream
      routers to tell the upstream router not to expect Prune-override
      messages. Also it tells the upstream router that the downstream routers
      will not be doing Join Suppression (each downstream router must send
      Joins so the upstream router can explicitly track each one).

-------------------------------------------------------------------------------

---

From: Marshall Eubanks <[email-masked]>
Date: Sat, 12 Nov 2005 17:14:56 -0500

I think this should be in "VPN Extensions to PIM v2", instead of a PIMv3.

Regards
Marshall

---

From: [email-masked]
Date: Sun, 13 Nov 2005 22:51:33 -0500 (EST)

I think this work should be an extension of PIMv2.

I think we should use a 32-bit sequence number in the JP messages.
The ACKs for these messages would be a simple message containing
the sequence number for JP message that it is ACKing.

I can write up a proposal with more details on this...

-Bob

---

From: Tom Pusateri <[email-masked]>
Date: Sun, 13 Nov 2005 23:04:02 -0500

Its really a matter of practicality. You won't be able to deploy a new
version that doesn't interoperate with the existing version so any change
that requires all routers on a lan segment or both ends of a p2p link to
change at the same time is a non-starter. This means the changes will
need to be negotiated during hellos.

But I'm definintely not convinced that we should try to duplicate the
features of TCP or SCTP. That is way out of the scope of our working
group. I would rather stand on the shoulders of the giants before us
and just use one of the proven reliable transports that exist already.

Tom

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 14 Nov 2005 14:36:28 -0800

>> I think this should be in "VPN Extensions to PIM v2", instead of a PIMv3.

    But the implications of these changes go way beyond VPN usage.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 14 Nov 2005 14:46:11 -0800

>> need to be negotiated during hellos.

    And hence complexity in the design will result. 

>> and just use one of the proven reliable transports that exist already.

    Agree. So now the question is should a full-mesh of TCP connections be
    used across a LAN to simulate the upper-PIM machinery of perserving
    the multi-access/multi-neighbor part of the protocol?

    I do know there are many multi-vendor hardware out there that optimizes
    on the fact that a downstream router can hear another downstream router's
    Join to an upstream router to create (S,G) state. Plus, the fundanmental
    PIM machinery causes loops to get suppress due to this (S,G) state
    creation.

    So we have to be careful to not jump in and just say "require explicit
    neighbor tracking" and "just remove join suppression and prune override
    logic" not to mention Assert logic.

    And what if the number of neighbors (across a large switched network) is
    in the 100s or 1000s, we will then have to put in Route Reflectors. ;-)

Dino

P.S. Seen this movie before.

---

From: Hugo Santos <[email-masked]>
Date: Tue, 15 Nov 2005 03:38:51 +0000

>     the multi-access/multi-neighbor part of the protocol?

  Not for an extension of PIMv2. Also, is it clear that PIM would
  benefict with TCP? The fact that PIM messages can be processed out of
  order allows for a faster convergence in general. I believe the
  J/P-Ack mechanism is simple enough that the solution you proposed
  works well as long as route state integrity is guaranteed
  (multi-access networks issues apart).

  Some comments/stuff to think about regarding 'Upstream Router Join
  Solicitation'. You mention that:

    ``JP-Acks are required from each downtream neighbor regardless if
    they want to Prune-override or not. This assures that all neighbors
    have the chance to send their Joins.''

  This means that each Prune must be individually ACKed by each router
  in the same interface. Would the potential downstream Joins in this
  case work as implicit ACKs? Also, will the JP-Prune be periodicly
  re-sent (up to 60 seconds) until all routers ACK it? If the JP-Prune
  is re-transmitted, do the downstream routers that aren't interested in
  the (S,G)s keep state or just ACK them again?  This is only triggered
  for Prunes though. Can we have some kind of robustness value
  instituted here that would dictate the number of re-transmissions
  required or we can't be sure since this could be a temporary and short
  connection loss and some neighbor could have lost all JP-Prunes, or if
  a router crashed/lost full connectivity and has just sent an Hello
  prior to the Prune (it will be around for more ~105 secs using default
  timer values).

  We could just go with Explicit Tracking, but it also suffers with
  large number of joiners if state is kept per route.

Hugo

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 09:11:18 -0800

>>   (multi-access networks issues apart).

    I tend to agree for p2p links. It is pretty trivial. It's the LAN cases
    that will add most of the complexity. Maybe we should just make some
    hard calls.

>>   in the same interface. Would the potential downstream Joins in this

   It is saying that if the Prune gets to the upstream router *only* the
   other (downstream) routers don't have a chance to send a Join to override
   the Prune (if they have oif-list state). The JP-Ack from the upstream
   router gives them a chance to see it. 

   In the legacy machinery, a retransmitted Join would allow the other
   downstream routers to get another chance to see (grant it at 1 minute
   later).

>>   re-sent (up to 60 seconds) until all routers ACK it? If the JP-Prune

   I hadn't planned it to work that way. This could work but I wanted to
   remove all periodic nature. Plus 60 seconds is too long.

>>   the (S,G)s keep state or just ACK them again?  This is only triggered

    They do what they do today. I don't propose they change this. I have seen
    in several implementations that it is good to err on creating (S,G) state
    on a LAN so you can do fast drops in hardware and to avoid sending packets
    to the control-plane CPU.

>>   prior to the Prune (it will be around for more ~105 secs using default

    Bottom-line if you exceed the number of transmissions what do you do? It
    simply has to be reliable. Flapping the neighbor to try to restart doesn't
    make the situation go away. It just makes the network unstable longer for
    the network operator.

>>   large number of joiners if state is kept per route.

    Right. If the working group decides if TCP is the way to go but doesn't
    want a full-mesh of TCP connections on a LAN, the other approach is
    to have a TCP reflector so you have N TCP connections per LAN versus
    N^2 TCP connections per LAN. 

    I particularly don't like this centralized approach because if the box
    goes down, you have to reestablish N connections to a new TCP-reflector
    and what happens the reliability of packets in the meantime.

Dino

---

From: Tom Pusateri <[email-masked]>
Date: Tue, 15 Nov 2005 12:19:30 -0500

>    and what happens the reliability of packets in the meantime.

So then you have to have a connection to a DR and also an active
connection to a BDR. ;-)

2N < N^2

Tom

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 09:20:35 -0800

>> So then you have to have a connection to a DR and also an active

    Yes, that could follow.

Dino

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 12:09:37 -0800

On Tue, Nov 15, 2005 at 03:38:51AM +0000, Hugo Santos wrote:
>   order allows for a faster convergence in general.

That's kinda bogus, is it not ? If you have a receive queue of PIM
messages you can process them in any order you want, however yo got
them (datgagram or TCP). 

What you are arguing is the case where for example a packet is
lost in transit but you got the packet afterwards already, so if you
now use TCP you may consider yourself forced not to deliver it
to PIM already because that would constitute a layer violation. 
I would argue that a) sometimes one could live with layer violations,
and b) it is probably a very good thing not to try to converge fast
under such a condition anyhow: After all, the most important thing is
reliable convergence, and the fact that you missed a packet is already
an indication of congestion. Given that we should always use internet
priority TOS for PIM messages (whether in TCP or not), the most likely
reason for packet loss is input queue overrun, and that's exactly a
ype of cngestion that i think TCP does actually work very well against.

Chers
	Toerless

---

From: Hugo Santos <[email-masked]>
Date: Wed, 16 Nov 2005 01:00:28 +0000

> to PIM already because that would constitute a layer violation. 

  Yes, i was refering to this case and not to a queue of received
  complete PIM messages.

> I would argue that a) sometimes one could live with layer violations,

  Well, i would prefer to avoid that.

> an indication of congestion. Given that we should always use internet

  Sure, but consider a case for instance of a Join(S,G) followed by a
  Prune(S,G) or even Prune(S,G) and Join(S,G) [for same S and G]. With
  the current specification, when you are overriding previous state, if
  the first message is lost you can still act safely as the next one is
  more relevant to the state. With TCP you'll have to wait for the
  retransmission. Not being able to Prune immediatly may even mean
  prolonging the congestion. With a solution like Dino proposed,
  retransmission is done per state, so we guarantee consistency while
  still being able for instance of overriding messages pending ACKs,
  e.g:

    Join, no ACK received, want to Prune, don't need to wait for Join's
    ACK, wait for Prune's ACK.

  This could also be somewhat achieved using SCTP and having a different
  SCTP flow for each state (not sure if it scales). Although i have no
  desire to reinvent the wheel, Dino's solution seems simpler to me.

  Since with a full TCP connection mesh we would do Explicit Tracking
  anyway .. we might as well push it to solve J/P-Acks problems in
  multi-access LANs.

Hugo

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 21:52:32 -0800

>>   still being able for instance of overriding messages pending ACKs,

    Right. You allow suppression of old events because the state is kept in
    PIM rather than in a TCP send buffer.

>>     ACK, wait for Prune's ACK.

    The difference between PIM-over-TCP from BGP and LDP is that the state
    transition are much more dynamic. That was one of the arguments to not
    put PIM functionality in BGP.

>>   anyway .. we might as well push it to solve J/P-Acks problems in

    Right. Agree.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 16 Nov 2005 02:27:13 -0800

Dino,

>     PIM rather than in a TCP send buffer.

The amount of "old events" that could end up in a TCP send buffer
is bounded by the size of the buffer. Since the size of the TCP send
buffer could be (locally) controlled by the sender, it means
that the sender has a way to impose an upper bound on the amount 
of "old events" in a TCP send buffer.

> >>   anyway .. we might as well push it to solve J/P-Acks problems in

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 16 Nov 2005 11:03:29 -0800

>> is bounded by the size of the buffer. Since the size of the TCP send

    And the size of the buffer is also a function of the receiver's window
    size.

    And the amount of time the "old events" sit in the send buffer is a 
    function of the RTT latency as well as the loss rate of the network.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Tue, 22 Nov 2005 06:13:13 -0800

Marshall,

> I think this should be in "VPN Extensions to PIM v2", instead of a PIMv3.

Are these extensions just for 2547 VPNs, or for VPLS as well ?

Yakov.

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 25 Nov 2005 10:47:07 -0500

Dear Yakov;

> Are these extensions just for 2547 VPNs, or for VPLS as well ?

That is a good question. On the one hand, I know that people are talking about
multicast transport over VPLS :

http://www.alcatel.com/com/en/appcontent/opgss/18847_VPLSBased_Multicas_an_tcm228-160151635.pdf

The following presentation (I can send a PDF copy  of anyone needs it) indicates
that Join  Acks or something similar would be usefule for
draft-raggarwa-l2vpn-vpls-mcast-01.txt type VPLS's.

http://www3.ietf.org/proceedings/05mar/slides/l2vpn-4/l2vpn-5.ppt

Do you agree (I think you are a co-author on that) ?

On the other hand, I have heard negative things about the prospects for
 L2 VPN multicast, but I have nothing  written to critique.

Regards
Marshall

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 30 Nov 2005 06:42:43 -0800

Marshall,

> That is a good question. 

Thanks !!!

Still waiting for an answer...

> On the one hand, I know that people are talking about multicast 
> transport over VPLS :
> 
> The following presentation (I can send a PDF copy  of anyone needs it) 
> indicates that Join  Acks or something similar would be usefule for

The overhead of doing PIM snooping on pseudo-wires is a serious
concern. Therefore, a mechanism that would reduce this overhead
would certainly be useful. There is a document,
draft-qiu-serbest-l2vpn-vpls-mcast-ldp-00.txt, that describes how
to reduce this overhead by carrying PIM information in LDP.
  
> L2 VPN multicast, but I have nothing  written to critique.

Could you please elaborate on these "negative things".

Yakov.