Subject: Re: [pim] Join Prune Ack Proposal

From: Marshall Eubanks <[email-masked]>
Date: Sun, 13 Nov 2005 23:16:08 -0500

Note : Subject change

Dear Bob;

Since the JP ack  is supposed to be a JP message, where would you put
this in the message ? In place of the first Joined source / prune
address ?

In that case, will it be 128 bits in IPv6, or will the rest be all zeros ?

I am not quite sure  why 32 bits, but I assume the writeup will cover this.

Regards;
Marshall

On 11/13/05, [email-masked] <[email-masked]> wrote:
> I think this work should be an extension of PIMv2.
>
> I think we should use a 32-bit sequence number in the JP messages.
> The ACKs for these messages would be a simple message containing
> the sequence number for JP message that it is ACKing.
>
> I can write up a proposal with more details on this...
>
> -Bob
>
>
>
>
>
>
> >>> The pim-state mailing list was created for a small design team to
> >>> work on pim state refresh reductions. That team now needs more input
> >>> from the PIM working group and L3VPN working group. To encourage this
> >>> and allow others to stay in touch with the continued effort, I have
> >>> decided to close down the pim-state mailing list and move all of its
> >>> traffic over to the main [email-masked] list.
> >
> >     Enclosed is the current thinking for the JP-Ack proposal I summarized
> >     at the PIM working group meeting.
> >
> >     And I would like the working group to judge if this should be an
> > extension
> >     to PIMv2 or an addition to PIMv3. I think the later so we can get
> > PIMv2
> >     to Draft Standard.
> >
> > Dino
> >
> > -------------------------------------------------------------------------------
> >
> >
> >       JP-Ack Proposal (for reducing periodic JP messages in PIM)
> >       ----------------------------------------------------------
> >
> > [1] Introduce a JP-Ack PIM message. It is always multicasted to 224.0.0.13
> > or
> >     ff02::d. Contains same format as JP message.
> >
> > [2] JP-Ack-Capable routers indicate they can send and receive JP-Acks by
> >     including JP-Ack Hello Option in Hello messages.
> >
> > [3] What JP senders do:
> >
> >     o When they send JPs to an upstream router which is JP-Ack-Capable,
> > they
> >       set a Ack-Received flag to FALSE for each route that is inserted in
> > the
> >       join- or prune-list portion of the JP.
> >
> >     o Then start a 1 second jittered retransmission timer. Retransmission
> >       occur until the upstream router disappears from the neighbor cache.
> > The
> >       retransmission timer can optionally be altered but cannot exceed 60
> >       seconds (the default periodic interval).
> >
> >     o When the timer expires, build JP for all routes with the
> > Ack-Received
> >       flag set to FALSE.
> >
> >     o When the 1 minute periodic JP interval expires, only include routes
> >       which have the Ack-Received flag set to FALSE.
> >
> > [4] What JP-Ack senders do:
> >
> >     o After a received JP is processed, modify the type field to turn the
> >       message into a JP-Ack. No other fields change in the PIM payload
> > message
> >       other than the type and checksum fields.
> >
> >     o For each route in the JP, set Ack-Sent flag to TRUE. This signifies
> > that
> >       any oif-list entry will not time out regardless of the holdtime
> > value
> >       sent in the JP. This allows the JP transmitter to not worry about
> >       refreshing the route.
> >
> >     o If the PIM checksum check fails, drop the JP message and do not
> > return
> >       a JP-Ack.
> >
> > [5] What do receivers of JP-Acks do:
> >
> >     o Regardless if you sent an JP or not, for each entry in the JP which
> >       you have state for, set the Ack-Received flag to TRUE so at the next
> >       periodic interval, the route is not included in a periodic JP.
> > During
> >       steady-state, there will be no routes with Ack-Received equal to
> > FALSE,
> >       so no JPs are sent.
> >
> > [6] What about triggered JPs?
> >
> >     o When new state is created, the Ack-Received flag is set to FALSE, so
> > a
> >       JP message is triggered. And will later be JP-Ack'ed.
> >
> >     o When an RPF change occurs, for each route that is affected, set the
> >       Ack-Received flag to FALSE and trigger a JP.
> >
> > [7] What happens when a joiner goes down?
> >
> >     o When a single joiner goes down, you want the upstream router to
> > start
> >       timing out routes.
> >
> >     o When there is more than one joiner, and a joiner goes down, you want
> > to
> >       make sure timeout-suppression still occurs.
> >
> >     o Whenever a PIM neighbor goes down, upstream routers will set the
> > Ack-Sent
> >       flag to FALSE which will cause timeout processing to start.
> >
> >     o Whenever a PIM neighbor goes down, downstream routers will set the
> >       Ack-Received flag to FALSE which will cause periodic processing to
> >       start. Which means when the 1 minute periodic interval expires, JPs
> >       will be sent. This will scatter the JPs across a jittered 1 minute
> >       periodic interval.
> >
> >     o If, from an upstream router's perspective, a downstream neighbor
> > times
> >       out but the downstream neighbor thinks the upstream router is alive
> >       (an issue with one-way connectivity), the upstream router should
> >       send a triggered Hello with a new GENID value. This will cause the
> >       downstream router to detect an adjacency change. In this case, the
> >       downstream router will send a full set of JPs. This is necessary so
> >       upstream router does not time out oifs when the downstream neighbor
> >       thinks it doesn't need to send a JP message.
> >
> >       It is important to note that the Hello be unicast with a GENID value
> > of
> >       0. The Hello is unicast so other neighbors on the LAN don't get
> > involved
> >       in sending JPs. The reason for a GENID value of 0, is so the
> > downstream
> >       router can get solicited to build a full set of JP messages. When
> > the
> >       next hello interval expires, the upstream router will transmit a
> > Hello
> >       with a GENID equal to it's last value. This way other neighbors
> > don't
> >       think there is a neighbor up/down transition.
> >
> > [8] Changing an entry for join-state to prune-state:
> >
> >     o When a prune-entry (negative cache state entry) has an empty
> > oif-list
> >       which goes to non-empty, a Join will be triggered, in this case the
> >       Ack-Received flag is set to FALSE so the Join can be sent reliably
> > and
> >       the now join-entry can be suppressed from future JPs.
> >
> >     o When a join-entry which has an non-empty oif-list goes to empty, a
> > Prune
> >       will be triggered, in this case the Ack-Received flag is set to
> > FALSE so
> >       the Prune can be sent reliably and the now prune-entry can be
> > suppressed
> >       from future JPs.
> >
> >     o In the case an entry changes from a join-entry to a prune-entry or
> >       vice versa before an JP-Ack is received for the previous state
> > should
> >       cause no problem since the JP sender only cares about the latest
> > state
> >       it has. If a JP-Ack is returned for a entry that is in the Join-list
> >       but the cached state is a prune-entry (or in the Prune-list for a
> >       cached join-entry), the Ack-Received flag is not touched (this is an
> > old
> >       Ack).
> >
> >     o In the case a JP sequence (3 separate JP messages) consists of a
> > {Join,
> >       Prune, Join} for the same multicast route, if the an JP-Ack for the
> > first
> >       JP-Join message is received by the downstream router after it had
> >       sent the third JP Join message, the downstream router would not ever
> >       retransmit a JP-Join message. In the case, the third JP-Join message
> >       was lost, the upstream router would have prune-state where the
> > downstream
> >       router would have join-state. To avoid this situation, an
> > implementation
> >       can delay sending the third JP-Join awaiting the JP-Ack for the
> > first
> >       JP-Join.
> >
> >       The same situation occurs with a JP sequence of {Prune, Join, Prune}
> > for
> >       the same multicast route. In this case, the upstream router keeps
> >       forwarding data since it has join-state but the downstream router
> > has
> >       prune-state.
> >
> >       This situation should be rare since the rate of a multicast route
> > flap
> >       would most likely be greater than a JP retransmission interval.
> >
> > [9] Duplicate JP-Acks and JPs:
> >
> >     o If a JP-Ack is received for a set of routes which already have the
> >       Ack-Received flag set to TRUE, no action is taken. This is known as
> >       a duplicate JP-Ack. This is the case where a router on a
> > multi-access
> >       LAN either didn't received the JP-Ack or it's JP was not received by
> > the
> >       upstream router. So a retransmission of either had occurred.
> >
> >     o If a JP-Ack is received for a route where the JP-Ack has a
> > prune-list
> >       entry for the route but the route has join-state (oif-list not
> > empty),
> >       then the Ack-Received flag should be set to FALSE so retransmission
> >       of a Join can happen. The converse is true if the JP-Ack had a
> > join-list
> >       entry for a route in prune-state (oif-list that is empty). This can
> >       happen when a route transition from join-state to prune-state (or
> > vice-
> >       versa) occurs and either the JPs or the JP-Acks were reordered.
> >
> >     o If an upstream router receives a JP for a set of routes which
> > already
> >       have the Ack-Sent flag set to TRUE, a JP-Ack should be returned.
> > This
> >       is a case where a downstream router didn't not receive the
> > previously
> >       sent JP-Ack or has triggered a JP it wants an JP-Ack for.
> >
> > [10] RPF Changes:
> >
> >     o When an RPF neighbor change occurs, a JP with join-list entries is
> > sent
> >       to the newly determined RPF neighbor. It is optional to also send a
> >       JP with prune-list entries to the previous RPF neighbor. In the
> > later
> >       case, while the JP-prune-list sender is waiting for a JP-Ack from
> > the
> >       previous RPF neighbor, other RPF change could occur. An
> > implementation
> >       has to remember the previous RPF neighbors so it can make sure the
> >       prune-list JP gets reliably sent.
> >
> > [11] Dealing with Prune-Override on multi-access LANs:
> >
> >     o On multi-access LAN, a downstream router that is pruning for a
> > multicast
> >       route will multicast the JP message on the LAN. The upstream router
> >       will delay removing the oif from the multicast route waiting for
> > other
> >       downstream routers to override the Prune with a Join message. To
> > make
> >       this sequence reliable would require a total of 4 messages. The
> > initial
> >       JP-Prune triggers the sequence from the downstream router, the
> > upstream
> >       router responds with a JP-Ack multicast to the LAN, then an
> > overriding
> >       router (if one exists) will send a multicast JP-Join which causes
> > the
> >       upstream router to return a JP-Ack for the Join.
> >
> >     o The reliability problem occurs because an overriding router may not
> > see
> >       the triggering JP-Prune or the upstream router's JP-Ack. Even though
> >       the overriding router has 2 chances to send the Join to override the
> >       Prune, both messages could be lost during a congestion period. Since
> >       the join-state in the downstream router had previously been
> > acknowledged
> >       by the upstream router, the downstream router would never send the
> > Join
> >       again.
> >
> >     o There are two solutions to the problem, (1) Upstream Router Join
> >       Solicitation or (2) Explicit Tracking.
> >
> >     o Upstream Router Join Solicitation is when the upstream router will
> >       forward the received JP-Prune back out the multi-access LAN and
> > require
> >       JP-Acks from each downstream neighbor regardless if they want to
> > Prune-
> >       override or not. This assures that all neighbors have the chance to
> >       send their Joins. In this case, not all neighbors have to send
> > Joins.
> >       The Join suppression mechanism from the base PIM spec can be used.
> >
> >     o Explicit Tracking is when the upstream router keeps track of each
> >       neighbor who has Joined for a particular multicast route. So when a
> >       JP-Prune is sent to the upstream router, it knows when the last
> > Joiner
> >       stops joining. In this case, only the JP-Prune needs to be JP-Acked
> >       and the oif is removed from the multicast route only when the last
> >       Joiner sends a Prune. This will trade-off memory for less messages
> > on
> >       the LAN.
> >
> >     o We propose when Explicit Tracking is used, that each router on the
> > LAN
> >       advertise the fact in a PIM Hello option. This allows the downstream
> >       routers to tell the upstream router not to expect Prune-override
> >       messages. Also it tells the upstream router that the downstream
> > routers
> >       will not be doing Join Suppression (each downstream router must send
> >       Joins so the upstream router can explicitly track each one).
> >
> > -------------------------------------------------------------------------------
> >
> > _______________________________________________
> > pim mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/pim
> >
>
>
> _______________________________________________
> pim mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pim
>

_______________________________________________
pim mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pim

---

From: Hugo Santos <[email-masked]>
Date: Mon, 14 Nov 2005 14:40:07 +0000

I also believe this work should be done in extension to PIMv2 instead of
waiting for PIMv3 (it would only delay the deployment of these features
even further). Also, you mentioned that we could call these 'VPN
extensions for PIMv2' but i think these changes could be used in general
and not only in VPN setups (although some of the requirements of L3VPNs
have been discussed lately).

Regarding the J/P-Ack format, i also favor the sequenced-ack instead of
per-source-ack (and message replication). Maybe Dino could provide us
with more details on why he picked the format he described versus a
sequence-based one. The difference i see involves that while in the
per-source approach you only have to keep the Acked flag for the source
in the TIB and then you changed the Acked flag by matching the
(S,G)-state, for the sequence based one you have to keep the sequence
number the source state is waiting to be acked (and possibly additional
state to map seq to states, for performance).

To include the sequence number in the J/P messages, a new Encoded
Unicast address could be defined that includes the upstream address and
a sequence number (and in PIMv3 change the J/P message itself perhaps).
Only a single sequence should be required by update (J/P message) as the
message integrity is guaranteed by the checksum.

The J/P-Ack could be a new PIM message type since it is only used by
J/P-Ack understanding routers which simply includes the sequence number
being ACKed.

Obviously there is still a lot to be discussed on the functionality and
protocol behaviour.

Hugo

---

From: Hugo Santos <[email-masked]>
Date: Mon, 14 Nov 2005 14:59:35 +0000

> sequence-based one. The difference i see involves that while in the

To answer myself, Dino's idea allows exactly the same J/P message format
to be used, allowing non J/P-ack understanding routers that are on the
same link to interpret the messages while for instance my suggestion
doesn't. Still, there is a 1-byte reserved field in the J/P messages
that could be used to signal the presence or not of a sequence number
somewhere, although this would require a checking the the current
implementations' behaviour.

Hugo

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 14 Nov 2005 14:32:21 -0800

>> implementations' behaviour.

    At the end of the day, we have to decide what the requirements are for
    LAN procedures.

Dino

---

From: [email-masked]
Date: Mon, 14 Nov 2005 18:48:41 -0500 (EST)

We could create a new JP message type that includes the sequence number.
These new messages would be sent to only upstream neighbors which
advertised the capability.  Legacy routers on the same LAN would not
be able to do Join Suppression and Prune Overrides if we use a new
message type.

On the pim-state mailing list there was a proposal to use a TLV at the
end of the JP message after all the encoded groups.  If we
wanted to put the sequence number in the existing JP message, then this
might be a way to not effect existing implementations.

Creating a new address encoding I think will have issues for legacy
routers trying to parse the message (unless we unicast the message to
the upstream router, and then again Join Suppression and Prune Overrides
will not be possible).

-Bob

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 14 Nov 2005 18:50:21 -0500

I still don't see why the sequence number  has to be so large.
It seems to me with the timing of these ACKs that even a one
byte (256 choices) would be enough; even 4 bits might suffice. Isn't
all you need is to

- detect drops while
- being sure that you haven't wrapped ?

One byte or less would fit in the reserved field, thereby unifying
your and Dino's suggestions.

I don't see what 4 billion sequence numbers buys  you. Are you
thinking of hashing some information in there ? If so, what ?

Regards
Marshall

On 11/14/05, Hugo Santos <[email-masked]> wrote:
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.5 (GNU/Linux)
>
> iD8DBQFDeKZX7asb/itUNKwRAq+VAJwPhWGkwBQI3jBFZmNzKGg9ZNVmRgCeLNjn
> xZsKwwB1P/E1cxGr80QSpiE=
> =Wlq3
> -----END PGP SIGNATURE-----
>
>

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 14 Nov 2005 18:54:11 -0500

Oh, and I now think this should be called something like

Extension to PIMv2

Dino is right, this is not just applicable to L3VPN's.

The key think is to avoid PIMv3, which I think everyone agrees with.

Marshall

---

From: Hugo Santos <[email-masked]>
Date: Tue, 15 Nov 2005 00:27:01 +0000

> I still don't see why the sequence number  has to be so large.

Although 256 might seem a large number, this means you could only have
256 'transactions' in flight. 256 isn't a big number if you consider
that not all implementations might properly aggregate J/P information.

But assuming they do, for IPv6, for single sourced ASM groups or SSM
states (1 group and 1 source), you'll get 44 octets per group, ~30
(S,G)s per J/P on ~1500 MTU, which puts the limit on 30 * 256 = 7680
(S,G)s in flight. For non-aggregated triggered J/Ps that's 256 (S,G)
pending transactions. If this is an acceptable number (which i don't
think it is), the reserved octet would be enough for the sequence
number.

If the sequence number if global to the link instead of local to the
target upstream it would limit even more the number of possible
pending-acks.

Hugo

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 14 Nov 2005 19:11:29 -0800

>> The key think is to avoid PIMv3, which I think everyone agrees with.

    You are probably right, but we also want to avoid reopening PIMv2 since
    the spec is so far along.

Choosing poison,
Dino

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 14 Nov 2005 22:54:21 -0500

I agree. Thus, Extensions to PIM V2.

I regard the PIM V2 bis spec as written on tablets of stone,
unchangable and unstoppable by us mere mortals.

Marshall

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 07:01:54 -0800

> The key think is to avoid PIMv3, which I think everyone agrees with.

Not really ;-) I think i said we shouldn't do anything that would
slow down the PIMv2 RFC, but maybe that actually is now really
out the door.

It's also entirely unclear what PIMv3 would/should be. After all,
there is no effort underway to identify issues with PIMv2 in general -
eg: beyond JP agregation. PIMv2 does in general work just fine. 

If we add backward compatible or optional extensions to PIMv2 and
stronger operational practices (avoid asserts, use p2p links...),
then PIMv3 could as well just be the best (simplest) possible profile:

  SSM mode only (subset of existing PIM-SM), only p2p links used 
  (operational practice), reliable hard state signaling (based on
  some future extension).

The one place where i think we can be more aggressively insert some 
of the lessons learned is for tree building with label switching,
and i think we had that dicussion during the MPLS working group. 

Cheers
	Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 07:07:58 -0800

>     At the end of the day, we have to decide what the requirements are for

Indeed:
  -> access LANs with sources/receivers. NO TRANSIT PIM TRAFFIC.
     No need to optimize PIM here
  -> Backbone interfaces:
     NO sources/receivers on them, ONLY TRANSIT.
     DR function should really be disabled here..
     - real p2p links
       use simple p2p signaling (eg: TCP ?)
     - p2p ethernet links (2 routers)
       need to decide whether automatically or explicitly configured.
       use simple p2p signaling (eg: TCP ?)
     - multi-access LANs
       - should really only be used for 802.17/DPT
       - this is the difficult case. my favoured way to best opeate
	 in this environment is to avoid asserts completely by
	 mean of strong rpf-check, not sure if this has a chance
	 in ietf religion/politics/hw-requirements..

Cheers
	Toerless

       
  -> 

-- 
Thanks
   Toerless Eckert

---

From: Marshall Eubanks <[email-masked]>
Date: Tue, 15 Nov 2005 10:14:48 -0500

Dear Toerless;

That is an interesting point. I have heard  rumours that there were
other  changes for PIMv3 waiting in the wings (presumably for PIMv2 to
publish). That would be the real reason  not to go to PIMv3 immediately -
if it opened a  5 year plus can of worms. People are assuming this, but
maybe it's just an urban legend.

So, I think it's time  to at least find out what's out there : Does anyone have
a PIM wish list beyond state / signalling reduction in a "L3VPN type"
environment ?

If so, what would  it be ?

Regards
Marshall

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 07:30:44 -0800

> So, I think it's time  to at least find out what's out there : Does anyone
> have a PIM wish list beyond state / signalling reduction in a "L3VPN type"
> environment ?
>

Dino's proposal for joi-counters, which is somewhat more generalized ?

But that doesn't really try to improve things from PIMv2, it rather
extends it to solve another issue. 

Cheers
	Toerless

---

From: Pekka Savola <[email-masked]>
Date: Tue, 15 Nov 2005 18:02:13 +0200 (EET)

>     No need to optimize PIM here

How does a router know whether an Ethernet is an access LAN or a 
transit LAN ?

>     DR function should really be disabled here..

How would a router join to a group (e.g., SAP) ?
(not sure how it's done now)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 08:48:02 -0800

>> and i think we had that dicussion during the MPLS working group. 

    I definitely agree. The risk of PIMv3 is feature creep. Unless we say
    PIMv3 is only a change of transport, period. But as Tom said, in either
    case we will have to worry about PIMv2 and PIMv3 interoperability on
    a LAN. So I guess it doesn't matter what you call it, you have to deal
    with the deployment issues.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 08:51:58 -0800

>>        - should really only be used for 802.17/DPT

    This presupposes that the switch realizing the multi-access LAN participate
    in PIM so the topology now looks like a start topology of p2p links.
    Are you actually suggesting this?
 
>> 	 in ietf religion/politics/hw-requirements..

    If you do my presupposition above, you got it. But if you don't, how will
    you be able to carry an upstream IP address in the data packet or do you
    do a strong RPF-check on the MAC address? And you surely know this 
    requires hardware changes to every existing box on the planet already
    running multicast.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 08:54:18 -0800

>> Dino's proposal for joi-counters, which is somewhat more generalized ?

    And guess what, if you get rid of periodic JPs, then they can't scale
    well anymore.

    It appears that this design discussion is going towards an explicit
    tracking approach. And I think it won't scale for large default MDT
    and CE-based MGRE scenarios. We are trading one set of problems for 
    another.

Dino

---

From: Marshall Eubanks <[email-masked]>
Date: Tue, 15 Nov 2005 11:59:00 -0500

If nothing else, if we could clearly describe the
trade-offs, we could involve the L3-VPN  folks in the
discussion of which problems are best to solve.

Regards
Marshall

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 11:17:13 -0800

> How does a router know whether an Ethernet is an access LAN or a 

It's an access LAN if IGMP is enabled, but then it has to be
passive in the IGPs so that there's no transit traffic. Or some
explicit config or the like.

> link scope groups are joined to on some loopback interface.

Cheers
	Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 11:33:37 -0800

>     And guess what, if you get rid of periodic JPs, then they can't scale

Why not ?

>     and CE-based MGRE scenarios. We are trading one set of problems for 

Explicit tracking should work as well as the current PIM LAN procedures
on a big LAN (Default-MDT or not) as long as we have hard state signaling
when we do explicit tracking:  The only thing the curent PIM LAN procedures
do is to suppress periodic joins when multiple joiners are present, and
there are no periodic joins in hard-state anyhow. The trigered PIM joins
of new joiners are the same with current PIM LAN procedures as well as
with hard-state/explicit-tracking.

Cheers
	Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 11:45:47 -0800

>     Are you actually suggesting this?

The easiest solution is to enable L3 routing on the switch
and just don't have any L2 switches in Backbone LANs. We have been
pretty successfull with that proposition in any mayor enterprise where
multicast is business critical.

The question is really under which circumstances you can not avoid
using an L2 switch within a backbone network (not on access LANs, there
L2 switching is fine, it "just" requires IGMP snooping, but in backbone
LANs it's PIM snooping).

The place i fear is obviously L2VPN, where PIM snooping will be
happening on PEs. I quite frankly think that that should be excluded
from any IETF L2VPN architecture, because it breaks the basic expectation
of L2VPN customers of protocol transparency in the SP network, but
hey: nowadays the IEF just walks he lemmings route.

So yes, if we fail to argue that L2VPN-PEs always need to really just
do broadcasting towards PIM-CEs, i think it is necessary to consider
some more explicit signaling in PIM to include the PE as the (only)
PIM neighbor. This will have L2VPN customes cry wolf on non-transparency,
but it was never transparent anyhow.

>     If you do my presupposition above, you got it.

No, not really. I was asking for strong RPF-check for LANs where
you really don't have any active L2 switches, like WLAN, 802.17, DPT.

>     But if you don't, how will

I wouldn't want to change packet encoding, so yes, i would favour
source MAC address based filtering.

The only products that would have real problems are AIC based ones.
Microcode based outer should be fine with this. But yes, it would be
a mayor change, so the best approach would be to make it an
optional per ip-subi feature first. Like i think any kind of 
improved PIM signaling(hard-state/explicit tracking) should be as well.

Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 11:59:08 -0800

>     with the deployment issues.

Define "feature creep". That sounds to me like name calling on a good
thing (features ;-).  Features are never bad by themselves, there
are just bad features and unnecessary feaures (like: IGMPv3 is a bad
feature, because it's so much more complex han what we needed for SSM).

The problem is that PIM wasn't specified in a clear modular and
layered fashion: transport procedures, subinterface procedures,
tree-building. I think we can and should work on improvements
for both transport and subinterface procedures, but consider the
tree building to be the holy grail to be left untouched:

Both transport and subinterface procedures can be enhanced on a 
per-subinterface basis without impacting the tree building aka:
interoperability beyond a single LAN. Eg: asking for explicit
tracking and for strong-RPF-check to avoid asserts and to be able to
only rely on a p2p transport procedure (TCP et. al.) is such an
improved subinterface procedure which i think would make much sense.

Come to think of it, i guess that even the long time implemented
NBMA subinterface procedures aren't specified anywhere in the IETF..

Cheers
	Toerless

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 13:44:43 -0800

>> >     And guess what, if you get rid of periodic JPs, then they can't scale

    We don't want a counter change to cause a triggered update. In my PIM
    pop count proposal, the new values would be sent at the next periodic
    JP time.

>> when we do explicit tracking:  The only thing the curent PIM LAN procedures

    But what if you have a mix of old and new routers? You can't really do
    ET and Prune-Override.

>> with hard-state/explicit-tracking.

    A triggered JP now is a single packet that is sent. Pretty much received
    by all routers roughly around the same time. When run over TCP, the sending
    box has to replicate to *each* TCP connection.

    So this isn't straight-forward.  ;-)

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 15 Nov 2005 13:47:27 -0800

>> multicast is business critical.

    That turns the problem into PIM-over-TCP on point-to-point links only.
    You saw a private note I had on this. But don't think it's realistic in
    general.

Dino

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 14:07:39 -0800

>     pop count proposal, the new values would be sent at the next periodic

It's always dangerous to tie undependent functions together in such a
way because that makes it harder in the future to change then any single
one of them independently n the future without affecting the other.

The first thing to consider (i think) is how to specify the accuracy
of accounting in such a way that it can be aggregated easily in a router.

If we define for example two parameters: threshold [%] and  throttle [sec],
then we would send an update if the membership has changed by more than
+- <threshold> percent, unless the last update was less than <throttle>
seconds ago. We then signal these two parameters down the tree to get
measurements the way we want them on the source side (wih policy
override on each router).

I'd argue that including such a threshold based throttling would cause
much better efficiency in reduction of updates than the simple binding
to periodic updates.

>     But what if you have a mix of old and new routers? You can't really do

It's a ip subnet local thing, so i don't care. We just need to make
sure that the extension is done in such a way that a set of router
who agree to a certain minimum of featuers doesn't get DoS'ed by an
obsolete old router. We've already got the same problem in IGMP: Put
an IGMPv1 host on an access LAN and screw up leaveand SSM procedures
(just ignore the bugger).

>     by all routers roughly around the same time. When run over TCP,
>     the sending box has to replicate to *each* TCP connection.

Why ? It only needs to send the triggered PIM to the upstream router!
hard-state p2p signal (TCP, ...), explicit tracking, strong rpf-check,
use only if all routers on a LAN support it.

>     So this isn't straight-forward.  ;-)

"i beg to differ" ;-) ((c) 2005 dino).

Toerless

---

From: Beau Williamson <[email-masked]>
Date: Tue, 15 Nov 2005 23:24:21 -0600

>     You saw a private note I had on this. But don't think it's realistic in

That's why I think the only real approach is the Multiaccess PIM DR/BDR ala 
OSPF as you and Tom were discussing.  I just don't see the TCP approach 
scaling well.

Beau
P.S. Remember the words of the great philosopher who said, "Nothing 
difficult was ever easy." I think that was Yogi Berra, wasn't it?

---

From: Pekka Savola <[email-masked]>
Date: Wed, 16 Nov 2005 08:17:09 +0200 (EET)

> explicit config or the like.

I think most router vendors enable IGMP by default on all interfaces, 
or rather it's a global on/off toggle.  Fine-grained configuration is 
often possible as well.  My point is, it may not be easy to make this 
determination automatically in a reliable fashion.

> link scope groups are joined to on some loopback interface.

But what about global scope groups joined by the router?

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 15 Nov 2005 23:54:30 -0800

> determination automatically in a reliable fashion.

Didn't claim to. But you could provide some easy CLI that couple
things like passive IGP config with IGMP enabling 
("ip interface-mode [ access | core]") - or the like ;-)

> But what about global scope groups joined by the router?

Where did the line of my original mail go that said that globabl
groups have to be joined via a loopback interface ?

Cheers
	Toerless

---

From: Pekka Savola <[email-masked]>
Date: Wed, 16 Nov 2005 16:09:02 +0200 (EET)

> groups have to be joined via a loopback interface ?

In some dark corner of your brain?  Seriously, it was never sent on 
the list :)

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 16 Nov 2005 10:45:03 -0500

Isn't the real point here that PIM speaking routers do not need
to run IGMP to join any groups it might be convenient for them to
join ?

Regards
Marshall

---

From: Stig Venaas <[email-masked]>
Date: Sun, 27 Nov 2005 12:53:19 +0100

[...]
> If so, what would  it be ?

Careful what you ask for (:

One of my pet ideas, is to use shared tree and RP only for signalling,
basically only using it for source discovery, and only use SPTs for
data.

This could be done by doing only null-registers, and also basically
sending something like null-registers down the shared tree.

This is getting close to SSM, but not all that different from ASM.
For some applications I think using shared trees for data is better,
but they could perhaps use bidir instead...

You could also have a combined approach, where somehow the RP
decides whether to send data or only control packets down the
tree. Maybe the application could somehow make the choice.

Stig

---

From: Stig Venaas <[email-masked]>
Date: Sun, 27 Nov 2005 13:12:49 +0100

> OSPF as you and Tom were discussing.  I just don't see the TCP approach 

I don't understand why some of you are suggesting TCP. We don't need
congestion control, do we? I'm worried that the signalling may be
bursty and that TCP wouldn't work all that well...

Stig

---

From: Toerless Eckert <[email-masked]>
Date: Sun, 27 Nov 2005 06:31:04 -0800

> bursty and that TCP wouldn't work all that well...

Of course we do need congestion control. Every time reconvergence happens,
a PIM router  that happens to be the new PIM upstream router for a bunch
of downstream router, preferrably on multiple interfaces will ge bombarded
with PIM joins. There's really no telling to how many joins need to be
sustained except worst case maximum number of PIM states in the network
times number of downstream neighbors.

If TCP isn't the right choice for this, then why do LDP and BGP use it ?
Remember that OSPF and ISIS are NOT in the same league as PIM with respect
to the total number of LSAs they ususally need to carry.

And really: I don't care. If TCP is supposedly not the right protocol
to do reliable and congestion safe transport for PIM, then propose something
else, but propose it as a transport protocol below PIM and not as an
extension to PIM. PIM should just be doing tree building an rely on 
some transport protocol to reliably and congestion controlled transfer
of PIMs messages. We really should stop mushing these things together into
a complex state machinery. And no, i have not yet seen good proof that
out-of-order transmission (which would be problematic with TCP but possible
with other transport protocols) is even important under congestion.

And primarily we should get rid of the PIM LAN procedure and asserts 
completely, so that we can easily rely on a simple downstream->upstream
p2p signaling and thus transport protocol.

Cheers
	Toerless

---

From: Marshall Eubanks <[email-masked]>
Date: Sun, 27 Nov 2005 11:12:22 -0500

Dear Toerless;

Are you worried about

- lots of PIM joins from one neighbor or
- lots of PIM joins from many neighbors or
- possibly both ?

And are you worried about

- interface level congestion (just too many packets for the wire) or
- higher level congestion (of, say, the CPU) ?

I would suggest that TCP carriage would only help the former, not the latter.

Let me try and put numbers here. Please correct me if I  am off-base.

I have heard IETF hall discussions about scaling into the millions of groups
(thousands of VLANS, each with a thousand groups or so). I will assume 1 million
groups for  now as a performance spec.

Say that the average number of sources / group is 3 (typical of interdomain
ASM recently), and that pruning is rare. Then each IPv4 J/P takes on average
5 x 32 bits, or 20 bytes (plus 6 x 32 bits or 24 bytes overhead per
message), so we are talking about something over 20 megabytes for all
of this information, and as much as 44 MB if every group gets its own
J/P. (12 to 36 MB for an all SSM scenario.)

20 - 40 Mbytes does not seem especially congestion worthy amount of
data for a core network. So, I wonder if TCP congestion control is
really needed.

I also wonder if sending  a  20 MB J/P message is a good idea, even
with TCP. That seems like asking  for trouble if there is anything
sub-optimal about the network. If PIM is going to have to break
messages apart if they get "too big," it sounds like the same trouble
as supporting a MTU. (And, there is also the question of, can the
early data be used if the TCP goes into a CC or even a time-out state
?)

Also, would there be a significant Router CPU hit for a modern router
in opening up 1000 TCP sessions all at once ? or 10,000 ? That would
seem to be impied by these specs and the use of TCP.

Regards
Marshall

---

From: Marshall Eubanks <[email-masked]>
Date: Sun, 27 Nov 2005 11:20:14 -0500

> 5 x 32 bits, or 20 bytes (plus 6 x 32 bits or 24 bytes overhead per

Sorry, 5 plus  3 is 8 x 32 bits or 32 bytes of overhead per message.
Must have... more... coffee...

> of this information, and as much as 44 MB if every group gets its own

44 -> 52 MB. Still not that much  for a burst.

Marshall

---

From: Toerless Eckert <[email-masked]>
Date: Sun, 27 Nov 2005 18:21:38 -0800

> - lots of PIM joins from many neighbors or

More abourt the second one: As soon as i've got two or more downstream
neighbors, the downstream neighbor can not make assumptions anymore about the
speed with which the upstream neighbor can accept PIM-JP packets because
that depends on whether the topology change causing the triggered messages
did also affect other branches of trees converging in the upstream neighbor,
aka: the only solution for congestion control under lots of PIM joins from 
multiple neighbors is an active one controlled by the upstream router
(receiver).

> - higher level congestion (of, say, the CPU) ?

Both

> I would suggest that TCP carriage would only help the former, not the latter.

Wy not ?

> (thousands of VLANS, each with a thousand groups or so). I will assume 1 million  groups for  now as a performance spec.
> 

Wouldn't transfers of less han 20 MByte with TCP contribute to
the overwhelming mayority of traffic on the Internet ? And that's a bad idea ?

I don't think that anyone would ever claim TCP only becomes
useful and/or necessary beyond 20 MBytes. I think you need congestion
control way in before that.

> sub-optimal about the network.

BGP ?

> If PIM is going to have to break messages apart if they get "too big,"
> it sounds like the same trouble as supporting a MTU. 

I don't think this is necessary.

> (And, there is also the question of, can the early data be used if the
> TCP goes into a CC or even a time-out state ?)

Too much complexity. If TCP is in time-out state, then there's somehing
broken badly enough to consider resyncing state.

> seem to be impied by these specs and the use of TCP.

Why ? When i asked our BGP folks, hey told me that the time spent by
TCP in eg: a BGP route reflector is neglectible, all the real CPU time
is spent by BGP activity. It's not as if 1000 TCP sessions is anywhere
close to what web servers do, right ?

Cheers
    Toerless

---

From: Dino Farinacci <[email-masked]>
Date: Sun, 27 Nov 2005 20:32:01 -0800

>> to the total number of LSAs they ususally need to carry.

    I think PIM is in the same league as OSPF and IS-IS.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Sun, 27 Nov 2005 20:43:51 -0800

>> multiple neighbors is an active one controlled by the upstream router

    Only in the cases where Join Suppression is disabled on the LAN. There
    can be a lot of message suppression if implementations would do Join
    Suppression. Plus when a downstream router hears Joins from other
    downstream neighbors, and it still has the join messages in it's control,
    then it is able to be suppress messages.

    This is a crucial part of message scaling. That is, you lose when you
    pass it off to an independent transport layer.

Dino

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 28 Nov 2005 01:45:55 -0500

Dear Toerless;

([email-masked] removed from cc list)

> > I would suggest that TCP carriage would only help the former, not the latter.

Because TCP does not view router CPU load as a congestion control signal.

> the overwhelming mayority of traffic on the Internet ? And that's a bad idea ?

That's not quite what I mean. Of course, congestion control is a good thing.
However, you are talking about changing the protocol in use for router
- router communication. Now, I would expect that a core router
handling 1 million groups
would have fast connections, and 20 MB could go in a few seconds at most.

How often will the full state have to be transmitted ? I thought this
was only when bad things happened, such as when a router comes up or
is rebooted. Is it worth changing the protocol to  use TCP to avoid a
few seconds of possible congestion on an infrequent basis ?

(The answer to that may be "yes", I am just trying to figure out what
going to TCP will actually gain.)

> > sub-optimal about the network.

I don't get the reference. I had in mind trying to deal with packet
loss. Suppose that there is congestion, and so loss; it could take a
long time to send 20 MB with TCP time out, and the router has no state
until it all comes. However, in the desired scenario, it should still
get there relatively quickly and having to wait 10 or even 100 seconds
every once in a great while doesn't sound too bad. It still might be
an optimization to break this big J/P into smaller pieces sent
separately.

> broken badly enough to consider resyncing state.

ACK.

> close to what web servers do, right ?

That's good to know.

With the pentium based servers that I use, I find I can support many
100's of simultaneous sessions and something less than 100 Mbps of
aggregate transfer per box. You are correct, it's not the TCP, it's
what's in the TCP that's the limit.

Marshall

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Nov 2005 05:16:43 -0800

> Because TCP does not view router CPU load as a congestion control signal.

Doesn't matter. Because TCP uses positive acknowledgements, a CPU
congestion will cause a delay in sending acknowledgments, which in
turn throttles the window. This is why web-servers and other systems
can run nicely at 100% CPU load with lots of TCP sessions, and everything
works, but it just gets slower. NAK based protocols of course are much
more tricky to implement in the face of varying amount of CPU available to
a receive process !

> would have fast connections, and 20 MB could go in a few seconds at most.

Doesn't matter. At the point in time when reconvergence happens, routers
are likely active with IGP reconvergence, and even worse BGP reconvergence
as well, and so the performance you have to handle multicast joins will
vary widely and quickly over time based on the amount of other activities
going on.

> few seconds of possible congestion on an infrequent basis ?

Full state update will also happen if this router just so happens to be
a backup for some failed link and a downstream router converges onto it.

And yes, from our customer side we do see fast and reliable/reproducable
convergence behavior especially today in the face of IPTV type applications
as a primary requirement, and i am particularily concerned about the
reliability aspect, because whenever you just inadvertently start to
drop a PIM join, then you get what's considered to be unreliability, aka:
blackholing some groups for up to 60 seconds. And that's worst if it's
only reproducably one out of 100 times or so !

> going to TCP will actually gain.)

Reliability of convergence and option of convergence speed to the actual
CPU available on all ends.

> loss.

> Suppose that there is congestion, and so loss; it could take a
> until it all comes.

That's not really true, right ? if you transfer 20 MByte, the receiving
side would start to build state immediately when the first acknowledged
data arrives. 

> However, in the desired scenario, it should still
> every once in a great while doesn't sound too bad.

The 10 seconds blackout you're just talking about is by the way your
favourite TV program during primetime.

And don't tell me we don't need TCP for PIM because you're one
of those freaks that don't watch TV ;-)))

But let me ask the other way around: Why not TCP ?

> It still might be an optimization to break this big J/P into smaller
> pieces sent separately.

Not sure what that means.

> what's in the TCP that's the limit.

And again: TCP is just the obvious choice. The real change of attitute
i think needed by PIM is to let go of taking responsibility for transport
of it's message elements and let that be done by whatever transport
protocol we deem fit for the task. Modularize the bloody design to
reduce the complexity of each component.

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Nov 2005 05:21:14 -0800

>     I think PIM is in the same league as OSPF and IS-IS.

-> I think PIM should be capable to maintain far more state than OSPF
   and IS-IS are design to. That's why i think it's closer to BGP in
   terms of scalability requirements.
-> LDP is also in the same league as OSPF and IS-IS and still uses TCP.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Nov 2005 05:27:17 -0800

On Sun, Nov 27, 2005 at 08:43:51PM -0800, Dino Farinacci wrote:
>     Suppression.

Not with the current PIM state machinery that we took more than 10 years
to define. In this state machinery, join suppression only happens for
periodic joins, and we will not have periodic joins with a hard state
based approach anyhow.

>     Plus when a downstream router hears Joins from other

These are all proposals that will change the PIM state machinery beyond
simply taking the tast of reliable transport away from it. remembering
all join-state from other neighbors even when we are not interested in
it is trading some message reduction against a potentially large amount of
additional memory in routers. After all, this is PIM "Sparse-Mode", eg:
the more routers you have on a LAN the more likely it should be that 
every single one of them is not interested in all states.

Also:
The amount of buffering in the independent transport layer is something
that can nicely be scaled to the requirements of the underlying connection:
On L2 connections you can make it quite small to avoid unnecessary
buffering, and across L3 (eg: Default-MDT) you make it larger to achieve
better performance.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 28 Nov 2005 10:23:47 -0800

>> periodic joins, and we will not have periodic joins with a hard state

    It doesn't have to. And the original RFC indicated that you could do
    Join Suppression anytime. I remember, I was there.  ;-)

>> additional memory in routers. After all, this is PIM "Sparse-Mode", eg:

    Yes, of course, Memory/bandwidth tradeoff.

>> every single one of them is not interested in all states.

    Right, but you are arguing the other way now. The discussion started
    with multiple downstream routers swittching to an upstream router which
    caused a bombardment of JP messages.

Dino

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Nov 2005 10:55:37 -0800

>     caused a bombardment of JP messages.

I think i am arguing quite consistently. I am happy to accpt he bombardement
of an upstream router with triggered joins and without join-suppression,
and i want to have TCP for it ;-))

See:  think if i have 100 downstream routers, it's not a bad thing if
the behavior messagng and memory consumption is the same whether all
neighbors are on their own interface or whether they are all on their
shared LAN - that's why i am not interested in join suppression to protect
the upstream router - after all that router likely would need to support
as many neighbors on p2p interface as well.

On the other hand, if i am a downstream router, and i need to start
remembering state i am not interested in to make life easier for the
upstream router:  that does create additional effort that the downstream
router needs not to sustain when we wouldn't have a large LAN but just
p2p links, so i wonder why i wanted to do that shift of load...

Cheers
	Toerless

---

From: Eric Rosen <[email-masked]>
Date: Mon, 28 Nov 2005 16:13:37 -0500

Dino>  I think PIM is in the same league as OSPF and IS-IS. 

Do you mean  that the overhead of PIM is roughly  proportional to the number
of nodes in the network, and is independent of enduser activity?

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 28 Nov 2005 13:32:00 -0800

>> of nodes in the network, and is independent of enduser activity?

    I was not comparing the functional characteristics among them, but
    the requirement aspects. But I do agree with Toerless, it is probably
    more in between an IGP and BGP requirement (even though he didn't say
    that).

Dino

---

From: Eric Rosen <[email-masked]>
Date: Tue, 29 Nov 2005 10:11:58 -0500

> I don't understand why some of you are suggesting TCP. 

This  discussion has been  all over  the place,  but I  think the  answer is
simple enough.   (N.B.: I  am not  offering an opinion  here for  or against
running PIM over TCP.)  When folks  suggest TCP, it's not because they think
there's a  requirement for guaranteed  sequential delivery, it's  because of
the following factors:

- A TCP connection requires a fixed  amount of state to ensure that its data
  stream gets delivered.   If you multiplex n control  streams over a single
  TCP connection, this  amount of state does not  increase.  However, if you
  run those  n control streams  as n independent transport  connections, the
  state needed to ensure delivery increases by a factor of n.  E.g., TCP has
  only one retransmission timer per  connection, not one for every multicast
  stream. 

- TCP provides  flow control, which means  that the rate at  which data gets
  sent will  not exceed  the rate at  which data  is being processed  at the
  other end.  It's well  known that if the remote end is  slow, the best way
  to kill it entirely is to  keep sending it frequent retransmissions of the
  messages  it is unable  to handle.   So soft  state protocols  usually use
  exponential backoff of the retransmission timers to provide a form of flow
  control.  But that form  of flow control isn't as good, as  it tends to be
  initially too fast and then  quickly becomes too slow.  TCP's flow control
  algorithms have been carefully tweaked  over several decades, it's hard to
  reproduce their effect with something else. 

  I suppose one could argue that  flow control isn't needed because the load
  of PIM  control messages  could never exceed  a node's capacity  to handle
  them  promptly.   In  my  experience,  this  sort  of  argument  generally
  presages a  disaster ;-)  And people  do keep telling  me that  there's no
  bound on the rate at which PIM messages may need to get generated.  

  (Well, people  also tell me that not  only is the PIM  load unbounded, but
  it has strict latency requirements  which must never be violated no matter
  what  the  load.   Obviously the  laws  of  mathematics  do not  apply  to
  multicast ;-))

- TCP also eliminates the need for an  app to fit each of its control messages
  into  a single  network packet,  which is  generally an  advantage, though
  usually not as important as the above. 

That's it really.  

The price  for this is that everything  needs to get processed  in the order
sent,  so   inessential  ordering  relations  get   introduced.   And  while
complexity  is saved  in some  areas,  it is  added in  others.  Putting  an
arbitrary control protocol on top of TCP is not necessarily trivial.  At the
sending end, the app  may be ready to give something to  TCP when TCP is not
ready to take it,  and this needs to be handled by  the app's state machine.
At the  receiving end,  the app  needs to be  able to  take things  from TCP
whether or not it is really ready for them, or else you can get head of line
blocking.

TCP also provides congestion control, i.e., it slows down when a packet gets
lost, and then speeds up only gradually,  in the hope of achieving a sort of
network-wide  equilibrium where packets  don't get  lost due  to congestion.
That's very important when TCP is carrying data, but when it's only carrying
control  messages, it  shouldn't be  too important.   If there  are  so many
multicast control messages  that they have an effect  on network congestion,
the multicast system is hosed anyway. 

One  can raise  the question  of whether  the prospect  of thousands  of TCP
connections doesn't  present its own  scalability issues, but  just remember
that the  alternative may  be hundreds of  thousands of  independent control
streams.   On  the  other  hand,  if  you  end  up  with  thousands  of  TCP
connections, each  carrying one control  stream, you haven't gained  much by
going to TCP.  One of the advantages of the "route reflector" mechanism used
by BGP is that  it allows you to limit the number  of TCP connections, while
ensuring that each carries a large aggregate of the control data.

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 29 Nov 2005 07:31:08 -0800

> ready to take it,  and this needs to be handled by  the app's state machine.

True. But that's not a TCP specific problem. That's a problem of making
PIM hard state.  Today PIM simply sends packets into some interface send 
queue, and when it get's dropped there - so what - we've got soft state.
If we want to avoid the periodic refresh, we need to go hard state, and
if we want to go hard state we have this issue. Whatever transport protocol
or PIM extension we build to provide the hard state.

> whether or not it is really ready for them, or else you can get head of line

You can only get head of line blocking if you multiplex on either end
traffic with different priorities. At this time in PIMs life, there is no
such thing. All triggered joins and prunes are equal. If we had different
priority messages, we should use something better than TCP (eg: SCTP).

> network-wide  equilibrium where packets  don't get  lost due  to congestion.
  ^^^^^^^^^^^^ universe-wide ? Or is that only in latter work of Vince ? ;-)

> the multicast system is hosed anyway. 

I'd rather see it as a useful throttling in case topology changes in the
network do create a lot of control plane load on router.

You are right though, the original reasons for TCPs congestion control are
outside of the scope of what we're trying to achieve, but they are nevertheless
useful and i think necessary in the context where we want to use it.

Aka: I assume we sill see congestion not because we loose packets in the
network (can protect the banddwidth we need with DiffServ), but in the
participating routers.

Whether the slow-start is all that useful in that context is of course the
biggest open question. I would have hoped someone with BGP experience would
chime in and say why BGP hasn't moved on to a different protocol, because
BGP should have the same situation.

> going to TCP.

You mean as opposed to running "simply" PIM ? Oh yes i have. I've
avoided all the periodic refresh and i have congestion control. That's all
i want ;-) (that and opbviously a p2p peering type model to avoid the
need of multicast PIM messages).

> One of the advantages of the "route reflector" mechanism used

#TCP sessions i think is a red herring as far as practice is concerned.
The reasons why RRs are successfull has nothing to do with number of
TCP control blocks in router, but is rather stuff like centralization
of policy, configuration simplification on PE, saving CPU by running
BGP best path selection only once, ... -> None of these arguments
by the way would necessarily be the same for multicast, which is why
i am somewhat annoyed by the RR argument for putting PIM joins into BGP...
because we can have RRs!

Cheers
	Toerless

---

From: Eric Rosen <[email-masked]>
Date: Tue, 29 Nov 2005 11:07:49 -0500

> traffic with different priorities. 

Not so. 

Typically HoL blocking  occurs in a situation like  the following.  Your TCP
receive buffer (from C, say) contains  two messages, the first of which must
be  forwarded over  a  TCP connection  to A,  the  second of  which must  be
forwarded to  B.  However, your TCP connection  to A is blocked  due to flow
control,  while your  TCP connection  to  B is  not blocked.   To avoid  HoL
blocking you  have to keep reading your  connection from C, so  that you can
extract all the stuff going to B, which means that you have to have some way
of holding onto all the stuff from C which needs to go to A. 

Hardly an unsolvable problem, but not necessarily a trivial one. 

> I assume we sill see congestion not because we lose packets in the network
> ... but in the participating routers

This would be a situation in which you cannot even get the data into the TCP
receive buffer  and/or can't get the TCP  ack to go out.   This is possible,
but  in  this  situation I'm  not  sure  that  TCP's carefully  crafted  and
complicated  congestion   control  mechanisms   will  do  any   better  than
exponential backoff of independent retransmission timers would. 

> #TCP sessions i think is a red herring as far as practice is concerned. 

I don't think that consideration can be dismissed out of hand, especially if
one is looking for something that  will work in a large variety of different
environments.  If one is trying to  reduce the total amount of state that is
needed, one has to take all factors into account.

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 29 Nov 2005 08:22:31 -0800

> of holding onto all the stuff from C which needs to go to A. 

Ok, i shouldn't have said "priorities" ;-) I just meant multiplexing.

If we use a single TCP connection to pass PIM j/p from one downstream
PIM router to one upstream PIM router, where do we get multiplexing ?
The j/p themselves are just one flow.

> Hardly an unsolvable problem, but not necessarily a trivial one. 

Sure. See SCTP. But again. I don't see how it relates to the PIM over TCP
application.

> exponential backoff of independent retransmission timers would. 

If you send PIM j/p for 100,000 (S,G) states to one upstream PIM neighbor,
you don't really want to create 100,000 retransmission timers, that really
doesn't make sense.

Sure, the retransmission policies in TCP are probably more complex than needed
for this, but not by much. The sliding window scheme for example is a good
scheme for congestion control not only in the network but also in the end
nodes.

> needed, one has to take all factors into account. 

Ok. Any proof for #TCP session issues ... except for the well known
old IOS versions TCP performance problems we had during i think the second
half of the last decade of the previous millenium ? Because that's the
one thing we always hear in this discussion... Together with other similar
intelligent notions like "RSVP does not scale", and "MPLS is better because
it does not have PIM" ;-)

So, how about BGP and TCP ? A love match made in hell ?
Why does the Internet still work ?

Cheers
	Toerless

---

From: Eric Rosen <[email-masked]>
Date: Tue, 29 Nov 2005 12:42:33 -0500

> The j/p themselves are just one flow. 

No, the control flow for each  multicast group is independent of the control
flow for any other multicast group, and with TCP we multiplex them together,
thereby introducing unintended  side effects.  This may be  worth doing, but
let's not  pretend we're not  doing it. To  avoid HoL blocking, you  need to
design your app  so that it is always  able to read from TCP  and to process
whatever it gets from TCP, no matter what state it is in.  No need for SCTP.

> The  sliding window scheme  for example  is a  good scheme  for congestion
> control

The sliding  window scheme  is a "flow  control" (end-to-end  rate matching)
scheme, nothing to do with "congestion control" (response to loss of traffic
in network).

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 29 Nov 2005 10:00:50 -0800

On Tue, Nov 29, 2005 at 12:42:33PM -0500, Eric Rosen wrote:
> flow for any other multicast group

Yes, but you agree that you're making a theoretical argument, right ?
Even HTTP is trying to reuse TCP connections instead of using one per
transaction (URL). Not to talk about the transport level of IGPs or BGP...

> and with TCP we multiplex them together, thereby introducing unintended 
> side effects.

Yes, but i also only have one physical/logical link across which i can
send my triggeed 100,000 (S,G) joins, so i already have L2 and L2 serialization.

> This may be  worth doing, but let's not  pretend we're not  doing it.

> To  avoid HoL blocking, you  need to

Sure, but te argument was that the HoL blocking in the case of normal PIM
is not an issue and the fact that it will trottle processing likely more
than such optimized receiver side processing is likely more helpfull in
case of congestion anyhow.

> scheme, nothing to do with "congestion control" (response to loss of traffic

Well, as explained in some initial mail of this long thread, the fact that
the transfer speed/volume is also implicitly controlled through the positive
ACKs does contribute to the way congestion is handled. And as said: Congestion
is not only loss of traffic in the network but could as well be overload on
one of the end systems due to which you need to reduce the speed of transmission.

Cheers
	Toerless

---

From: Beau Williamson <[email-masked]>
Date: Tue, 29 Nov 2005 12:07:33 -0600

>whatever it gets from TCP, no matter what state it is in.  No need for SCTP.

But this is no different from what we have today.  If I'm not mistaken, all 
PIM control packets go into an ingress PIM control queue and thus are 
already "multiplexed" together.  So we aren't any worse off with TCP in 
that regard than we are today.

The issue may be that the normal congestion mgmt of TCP may result in the 
send queues growing quite large on the downstream router under certain 
topology changes with LOTS of state.

Beau

---

From: Eric Rosen <[email-masked]>
Date: Tue, 29 Nov 2005 13:53:28 -0500

> that regard than we are today. 

Suppose that  you cannot  process the  first message on  the queue  for some
reason.  E.g., perhaps processing it would  require you to send a message on
a TCP connection  which is blocked.  And suppose that  you could process the
second message  if only the first message  were not in the  way.  Today, you
can throw  away the first message, knowing  that it will show  up again, and
proceed  directly to  the second  message.  With  a TCP-based  protocol, you
can't do this.  You have to process the first message.

Using TCP to  transport multicast routing info may well be  a good idea, but
the point is that there's more to it than simply "running PIM over TCP".

---

From: Beau Williamson <[email-masked]>
Date: Tue, 29 Nov 2005 13:06:04 -0600

>can't do this.  You have to process the first message.

Eric,

This is simply just not applicable to PIM control plane operations.  You're 
thinking from a data "forwarding" point of view.  This isn't the case in 
PIM since all signalling is hop-by-hop.  Hence, you never "forward" an 
incoming control packet.  You process it and create state.  If for some 
reason due to resource exhaustion or PIM state limits you can't process 
it.  You just discard it and go to the next packet in the queue.  Thus, 
there is NEVER any HoL blocking in this path.

>the point is that there's more to it than simply "running PIM over TCP".

There may be but the arguments you give above just aren't an issue wrt PIM 
control packets.

Beau