Subject: Re: [pim] Some questions and suggestions on PIM-PORT draft

From: Stig Venaas <[email-masked]>
Date: Tue, 01 Feb 2011 11:27:35 -0800

Hi Bharat

I think the main remaining point we need to discuss is whether you can
switch back and forth between datagram and PORT mode.

The below should summarize my thoughts... I'm curious what others
think too.

Assume that we do datagrams until the PORT connection is established.
Once it is established we need to ignore datagram J/Ps. This is needed
in case a datagram J/P was sent by one peer before connection got
established, and received by the other right after.

Else you might get a PORT (S,G)-join and a delayed datagram (S,G)-prune
removing the (S,G) state and never recover.

It might also make an attack like sending a single spoofed (S,G)-prune
efficient...

Assume now that we have routers A and B with a TCP connection between
them. Assume that B send an (S,G)-join to A at some point, but that now
only A sends joins to B. If the TCP packets are dropped between A and B,
then A will eventually realize the connection is down (because it does
enough retries), while B will not (unless it uses TCP keep-alive)
because it does not try to send.

Then A will revert to datagrams and send J/Ps that B will ignore. Also,
the (S,G)-join state on A will expire because it does not receive any
joins from B. B does not send datagram joins since it thinks the
connection is up.

Using keep-alives would certainly help here.

However, if the connection goes down, I think it is better to try to
reestablish it, than to immediately switch to datagram mode. It's
bad to send a lot of datagrams J/Ps if the connection is coming up
almost right after, and then have to send all the J/P state again.

If the TCP connection fails repeatedly, why not instead have the
router stop sending Hello's with the PORT option?

What do you think?

Stig

---

From: Bharat Joshi <[email-masked]>
Date: Sun, 6 Feb 2011 10:16:43 +0530

Hi Stig,

     Sorry for a delayed reply. Please see my response in line.

Regards,
Bharat

> It might also make an attack like sending a single spoofed (S,G)-prune

Ok.
 
> because it does not try to send.

Ok.

> joins from B. B does not send datagram joins since it thinks the

But 'B' was anyway not sending anything so I did not understand the (S,G) join state on A expiring because of that.

> almost right after, and then have to send all the J/P state again.

Ok. I agree that we can retry couple of times here before declaring that it will not come up.
 
> router stop sending Hello's with the PORT option?

Yes. This looks fine. We can stop using the PORT option in the hello messages.

One more thing I would like to clarify. Can a neighbour start/stop using PORT option dynamically i.e. without changing the GEN-ID?

I am now thinking that restricting native J/P if PORT option is enabled looks better. Let us see what others think about this.

**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely 
for the use of the addressee(s). If you are not the intended recipient, please 
notify the sender by e-mail and delete the original message. Further, you are not 
to copy, disclose, or distribute this e-mail or its contents to any other person and 
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken 
every reasonable precaution to minimize this risk, but is not liable for any damage 
you may sustain as a result of any virus in this e-mail. You should carry out your 
own virus checks before opening the e-mail or attachment. Infosys reserves the 
right to monitor and review the content of all messages sent to or from this e-mail 
address. Messages sent to or from this e-mail address may be stored on the 
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***

---

From: Stig Venaas <[email-masked]>
Date: Mon, 07 Feb 2011 10:47:38 -0800

> But 'B' was anyway not sending anything so I did not understand the (S,G) join state on A expiring because of that.

B sent a PORT (S,G)-join before the connection stopped working.

> One more thing I would like to clarify. Can a neighbour start/stop using PORT option dynamically i.e. without changing the GEN-ID?

I would say that a neighbor can stop announcing the PORT option and not
change the GENID. If any PORT connections were up and in use, they will
then be taken down, and they will start sending periodic updates. When
the connection goes down, timers should be started for the OIL entries,
so that state expires unless they are refreshed by regular J/Ps.

You only want to change the GEN-ID if you for some reason need to get
a full update quickly.

Stig

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Tue, 8 Feb 2011 06:09:28 +0530

> > But 'B' was anyway not sending anything so I did not 
> understand the (S,G) join state on A expiring because of that.
> 

I repeat - I don't think it's a good idea mixing PORT joins/prunes with datagram joins/prunes. If a router was running PIM in the datagram mode and switches to the PORT mode (or vice-versa) then all state MUST be refreshed again. Any join not refreshed should get timed out naturally.

If PORT connection dies, then we MUST remove all (S,G) and (*,G) state from the routers.

> > Yes. This looks fine. We can stop using the PORT option in 

I think it should require operator intervention and the operator MUST deconfigure the PORT mode before we start sending native joins/prunes. We MUST not automatically start sending Hellos without the PORT option!

Cheers, Manav

---

From: Stig Venaas <[email-masked]>
Date: Mon, 07 Feb 2011 19:30:14 -0800

> I think it should require operator intervention and the operator MUST deconfigure the PORT mode before we start sending native joins/prunes. We MUST not automatically start sending Hellos without t…

So it seems I will keep the text in the current draft on
this. That is my preferred option.

I don't want to add text saying that an implementation could stop
announcing the option. But I think an implementation could choose
to do that (think of it as a configuration change) and still be
following the spec. Whether operators would like that or not is
another matter.

I'll be submitting a new version shortly. It seems to be that the
only open issue is regarding keep-alives. And perhaps the TTL
issue. I'll respond to some of your other emails on the keep-alives.
I'm also hoping others will join the discussion. I want to come to
a conclusion quickly and be done with this document.

Stig

---

From: Stig Venaas <[email-masked]>
Date: Tue, 08 Feb 2011 11:23:16 -0800

It's nice to see all this activity on the wg list :)

One remaining issue, and which you may have thoughts on,
is whether keep-alives should be TCP keep-alives or in
PORT itself.

I hope we have some kind of agreement that keep-alives
can be useful to detect error conditions, and that it is
good to allow some kind of keep-alive to be enabled. I
prefer for it to be off by default though.

Do you see good reasons for doing one rather than the
other? I think a benefit of doing it in PORT is that
you check that the communication path is working all
the way from PIM process to PIM process, rather than
just TCP. I'm also wondering whether it is useful to
detect that the neighbor has failed to send keep-alives.

If we choose to do keep-alives in PORT, then I think
it should only be in the absence of other traffic.
I.e. you restart a timer every time you send a PORT
message. If the timer expires, you send a keep-alive.

Stig

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 8 Feb 2011 11:52:36 -0800

Any reason why the BFD faction shouldn't jump up and declare
further keepalives obsolete ?

Btw: when it comes to all these environmental factors of PORT,
i think it would be good to consider taking guidance from LDP.

When we designed MPLS support for multicast, we originally
started thinking it should be over TCP directly, but we just
got tired of figuring out neighbor discovery, graceful
restart, addressing, and the like. So we put it over LDP and
called it mLDP. Of course, PORT should not use LDP, but if you
remove all the label stuff from LDP, the rest of the protocol can
give good guidance on how to do things in PORT if in doubt.
Definitely closer to PORT than BGP (which is the other widely
deployed unicast TCP based protocol).

Just saying ;-)) I know it's always dangerous to mention
label protocols in the non-label world or vice versa ;-))

Cheers
    Toerless

On Tue, Feb 08, 2011 at 11:23:16AM -0800, Stig Venaas wrote:
> _______________________________________________
> pim mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pim

-- 
---
Toerless Eckert, [email-masked]
Cisco NSSTG Systems & Technology Architecture
Network Agnostic: Applications that work best on cheap, stupid networks.
Suicide: Devaluating the intelligent network with network agnostic apps.

---

From: Stig Venaas <[email-masked]>
Date: Tue, 08 Feb 2011 12:14:49 -0800

> further keepalives obsolete ?

Certainly. Which is one reason for not always doing keep-alives.
Even though I see some benefit with keep-alives (if not doing
BFD), I'm also fine with not defining keep-alives now, and
adding it later if real life experience with PORT shows that
it is needed.

The options are:

No keep-alive
PORT keep-alive
TCP keep-alive

Stig

---

From: Eric Rosen <[email-masked]>
Date: Tue, 08 Feb 2011 15:59:30 -0500

> if you remove all the label stuff from LDP, the rest of the protocol can
> give good guidance on how to do things in PORT

LDP does have a keepalive timer, and an LDP speaker expects to receive some
LDP PDU from its peer before the timer expires.  There is a keepalive
message, but that is only sent if nothing else has been sent during the
keepalive interval.  BGP has a similar keepalive, I believe.  I don't think
anyone really knows for sure whether these are valuable or not, but they
don't seem to do any harm.

If an application level keepalive is defined, I don't think there's much
point to having it off by default.

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Wed, 9 Feb 2011 03:52:54 +0530

Hi Stig, 

I would prefer PORT keep-alives, with a CLI knob to turn them off if not required.

Using TCP keep-alive only ensures that the TCP connection (at the TCP socket layer) is up. It does not indicate if the PORT session is up. Thus TCP keepalives will NOT detect a PORT task that is say deadlocked with some other task in the system. Its thus imperative that we have this connectivity check at the application layer.

Cheers, Manav

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Stig Venaas
> Sent: Wednesday, February 09, 2011 1.45 AM
> To: Toerless Eckert
> Cc: [email-masked]
> Subject: Re: [pim] PORT and keep-alives
>

---

From: Stig Venaas <[email-masked]>
Date: Tue, 08 Feb 2011 14:44:28 -0800

> Using TCP keep-alive only ensures that the TCP connection (at the TCP socket layer) is up. It does not indicate if the PORT session is up. Thus TCP keepalives will NOT detect a PORT task that is sa…

Yes, I think doing it with PORT messages provides more info.
Note though that in our case we have PIM Hello's, so if say
the PIM process stopped responding, or whatever, the neighborship
would expire. Of course it is possible that the PIM process
keeps sending Hello's and something else goes wrong.

Another thing, how helpful is it to notice that PORT is not
working, if all you can do is establish a new connection. I
think it might be helpful, I'm trying to get a better
understanding of why. Main thing I can think of is alerting
operators, logging/debugging.

Sending keep-alives would make the session eventually reset
if there are no ACKs, but you probably also want to reset
the connection if you don't receive any messages from your
neighbor then? Assuming that either both or none of the
peers are doing keep-alive.

Stig

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Wed, 9 Feb 2011 04:21:26 +0530

> peers are doing keep-alive.

The reason most protocols use keep-alives is to inform to the remote peer about their liveliness. So its symmetrical and is turned on on both sides. 

Since we have PIM hellos that also pretty much does the same thing we should imo define the keep-alives in PORT and put in some text in the draft that PORT neighbors MAY want to use it for liveliness detection.

Cheers, Manav

---

From: Stig Venaas <[email-masked]>
Date: Wed, 09 Feb 2011 09:44:10 -0800

> Since we have PIM hellos that also pretty much does the same thing we should imo define the keep-alives in PORT and put in some text in the draft that PORT neighbors MAY want to use it for liveline…

Do others agree that this is a good plan? I'll take out the current
text on TCP keep-alives, define a keep-alive PORT message type, say
that implementations MAY (or SHOULD?) support it? I would also prefer
to say that it MUST be off by default (I think it's bad if different
implementations have different defaults since it would probably make
the connection go down if you just use default between two neighbors).

I'm thinking the keep-alives should be sent if there are more than 60s
since the last PORT message, which means you only send them if there
are no other PORT messages being sent. On the receiving end you can
take some action if you don't receive them, do we want to describe a
particular action? Is 60s a reasonable default interval?

Stig

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Wed, 9 Feb 2011 23:20:01 +0530

> particular action? Is 60s a reasonable default interval?

You'll have to define a Holdtime which would typically be 3 times of the Keepalive. Thus a router will have to miss 3 Keepalives, after which the neighbor session is torn down.

Cheers, Manav

---

From: Stig Venaas <[email-masked]>
Date: Wed, 09 Feb 2011 09:56:30 -0800

> You'll have to define a Holdtime which would typically be 3 times of the Keepalive. Thus a router will have to miss 3 Keepalives, after which the neighbor session is torn down.

I think the typical "3 times" rule is due to possible packet loss. Is
it still reasonable with a reliable transport?

Stig

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Wed, 9 Feb 2011 23:31:50 +0530

> it still reasonable with a reliable transport?

Most TCP based applications still use 3x factor because it would also indicate if the application/process is in a sane condition (not deadlocked, CPU too busy to give this application any CPU cycles, etc) and is able to respond to your KA packets. The router may be too busy to respond to your keepalives and you would not want to bring down your session based on one delayed packet. If you however find that its not responding to the last three keepalives that you have sent then its probably a good idea to bring down the session.

Cheers, Manav

---

From: Stig Venaas <[email-masked]>
Date: Wed, 09 Feb 2011 10:08:22 -0800

> Most TCP based applications still use 3x factor because it would also indicate if the application/process is in a sane condition (not deadlocked, CPU too busy to give this application any CPU cycle…

Yes, you may be right. Note that it would not be about reacting to the
keep-alives, but rather that it is too busy to send you keep-alives.

Stig

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Wed, 9 Feb 2011 23:41:46 +0530

> keep-alives, but rather that it is too busy to send you keep-alives.

Yes, which is why I said CPU too busy to give this application any CPU cycles. If this application is not getting any CPU cycles it cannot respond to your keep-alives. Another scenario when CPU may be giving it cycles and yet it doesn't respond to keep-alives is when its (i) either blocked on some resource OR (ii) is deadlocked with some other task.

So, we need to define a HoldTime which is 3x Keep-Alive.

Cheers, Manav