Subject: [OPSEC] AD Review of draft-ietf-opsec-protect-control-plane-03

From: Ronald Bonica <[email-masked]>
Date: Fri, 22 Oct 2010 15:00:01 -0400

Folks,

Having reviewed draft-ietf-opsec-protect-control-plane-03, I find that it is nearly ready for publication. However, I have the following comments:

- editorial: the document editor might want to take one more scan through the document looking for grammatical errors (word duplication, missing words, etc.) Also, please make sure that all acronyms and expanded the first time that they are used.

- technical: You apply the following filter on the interface between the control and forwarding planes:

     - drop all fragmented packets
     - allow all OSPF from known peers
     - allow all iBGP from known peers
     - allow all eBGP from known peers
     - allow all SNMP from known managers
     - allow all NTP  from known NTPs
     - allow all SSH  from know managers
     - rate limit ICMP
     - rate limit non-IP
     - rate limit everything else

First, do you want to drop all IPv4 optioned packets? IPv6 packets that include hop-by-hop options? 
This filter is simple considering that the example doesn't include any protocols that rely on IP options. If we were running a protocol like RSVP, the filter would be slightly more complex.

Filtering optioned packets protects you from "packet-o-death attacks" where a single a single crafted packet with malformed options causes the router control plane to crash. Your router may not be vulnerable to such attacks, but why find out the hard way?

Second, do you want to rate limit or drop all packets that are not specifically identified by your filter. If you rate limit them, you protect yourself from DoS attacks. You also protect yourself from your own mistakes. If you have forgotten to allow some class of required traffic, it will get through.

If you drop all non-specified traffic, you protect yourself from some "packet-o-death" attacks, where a single crafted packet causes the control plane to fall over. However, you don't protect yourself from your own mistakes.

Which way do we want to go?

                                                 Ron

                                                         Ron

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Sat, 23 Oct 2010 02:53:30 +0530

Hi,

> - technical: You apply the following filter on the interface 
> between the control and forwarding planes:
> 

A question - When we say allow, are we saying that there is no user configured rate limiting done? Wouldn't we also want to include FTP and TFTP for image upgrade/large config load above?

>      - rate limit non-IP

What is non-IP - L2 traffic like STP, LACP, dot1AG, CFM, etc? If Yes, then would be helpful to point that out.

> First, do you want to drop all IPv4 optioned packets? IPv6 

I couldn't find any text that says this.

> packets that include hop-by-hop options? 
> This filter is simple considering that the example doesn't 
> include any protocols that rely on IP options. If we were 
> running a protocol like RSVP, the filter would be slightly 
> more complex.

The draft can mention that packets with IPv4 options MUST be allowed if MPLS is enabled.

Manav

> 
> Filtering optioned packets protects you from "packet-o-death 
> attacks" where a single a single crafted packet with 
> malformed options causes the router control plane to crash. 
> Your router may not be vulnerable to such attacks, but why 
> find out the hard way?
> 
> Second, do you want to rate limit or drop all packets that 
> are not specifically identified by your filter. If you rate 
> limit them, you protect yourself from DoS attacks. You also 
> protect yourself from your own mistakes. If you have 
> forgotten to allow some class of required traffic, it will 
> get through.
> 
> If you drop all non-specified traffic, you protect yourself 
> from some "packet-o-death" attacks, where a single crafted 
> packet causes the control plane to fall over. However, you 
> don't protect yourself from your own mistakes.
> 
> _______________________________________________
> OPSEC mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/opsec
>

---

From: Ronald Bonica <[email-masked]>
Date: Fri, 22 Oct 2010 17:45:30 -0400

> -----Original Message-----
> From: Bhatia, Manav (Manav) [mailto:[email-masked]]
> Sent: Friday, October 22, 2010 5:24 PM
> To: Ronald Bonica; [email-masked]
> Subject: RE: AD Review of draft-ietf-opsec-protect-control-plane-03
> 
> Hi,
> 
> A question - When we say allow, are we saying that there is no user
> configured rate limiting done? Wouldn't we also want to include FTP and
> TFTP for image upgrade/large config load above?

We probably don't want to allow FTP at all. SCP is safer.

> then would be helpful to point that out.

I was thinking of PPP, HDLC, stuff like that. The authors might want to point that out.

> The draft can mention that packets with IPv4 options MUST be allowed if

Currently, they are allowed. That's exactly my point. They shouldn't be!

                              Ron

---

From: Carlos Pignataro <[email-masked]>
Date: Mon, 25 Oct 2010 20:30:45 -0400

Hi Ron,

Many thanks for reviewing the document. We have submitted a new revision
addressing (we believe) your comments, please see inline.

On 10/22/2010 3:00 PM, Ronald Bonica wrote:
> Folks,
> 
> Having reviewed draft-ietf-opsec-protect-control-plane-03, I find
> that it is nearly ready for publication.

Thanks !

> However, I have the following comments:
> 
> - editorial: the document editor might want to take one more scan
> through the document looking for grammatical errors (word
> duplication, missing words, etc.) Also, please make sure that all
> acronyms and expanded the first time that they are used.

Ack -- we've gone through the document with the fine comb, a couple of
times, and did find grammos to fix. Thanks for pointing out, hopefully
readability improved now.

> 
> - technical: You apply the following filter on the interface between
> the control and forwarding planes:>
>
> First, do you want to drop all IPv4 optioned packets? IPv6 packets
> that include hop-by-hop options? This filter is simple considering
> that the example doesn't include any protocols that rely on IP
> options. If we were running a protocol like RSVP, the filter would be
> slightly more complex.

We had discussed this, and agreed that since there are documents with
more specific matches to the subject, that we would take a general
pros/cons description on the topic and no configs; please see next comment.

> 
> Filtering optioned packets protects you from "packet-o-death attacks"
> where a single a single crafted packet with malformed options causes
> the router control plane to crash. Your router may not be vulnerable
> to such attacks, but why find out the hard way?

Agreed there's this tradeoff. That's why we had the existing text
already in the document:

3.3.  Design Trade-offs

   Two types of traffic that need special consideration are IP fragments
   and IP optioned packets.
   [...]
   Similarly, some deployments may chose to drop all IP Optioned
   Packets.  Others may need to losen the constraint to allow for
   protocols that require IP Optioned packets such as RSVP.

and then:

4.  Security Considerations

   Additional considerations pertaining to the usage and handling of
   traffic that utilizes the IP Router Alert Options can be found at
   [I-D.ietf-intarea-router-alert-considerations], and additional IP
   Options filtering explanations can be found at
   [I-D.gont-opsec-ip-options-filtering].

But to your point, we have expanded the "S3.3.  Design Trade-offs"
paragraph significantly in this new rev -04.

Please do let us know if you feel the additions do not address your
concern explicitly enough.

> 
> Second, do you want to rate limit or drop all packets that are not
> specifically identified by your filter. If you rate limit them, you
> protect yourself from DoS attacks. You also protect yourself from
> your own mistakes. If you have forgotten to allow some class of
> required traffic, it will get through.
> 
> If you drop all non-specified traffic, you protect yourself from some
> "packet-o-death" attacks, where a single crafted packet causes the
> control plane to fall over. However, you don't protect yourself from
> your own mistakes.
> 

Similarly, we do agree and we do discussed this topic extensively. The
position we took is that we protect the operator from "allow omissions"
in the example and we describe in great detail the process of selecting
what to do here.

In fact, we had a final/default "drop all" and we changed it and add the
description based on feedback.

On the document you will find this:

3.2.  Filter Design

   In an actual production environment, predicting a complete and
   exhaustive list of traffic necessary to reach the router's control
   plane for day-to-day operation may not be this obvious.  One
   recommended method to gauge this set of traffic is to allow all
   traffic initially, and audit the traffic that reaches the router
   control plane before applying any explicit filters or rate limits.
   See the Section 3.3 section below for more discussion of this topic.

3.3.  Design Trade-offs

   In addition to the traffic matching explicit classes, care should be
   taken on the policy decision that governs the handling of traffic
   that would fall through the classification.  Typically that traffic
   is referred to as traffic that gets matched in a default class.  It
   may also be traffic that matches a blanket protocol specific class
   where previous classes that have more granular classification did not
   match all packets for that specific protocol.  The ideal policy would
   have explicit classes to match only the traffic specifically required
   at the control plane and drop all other traffic that does not mach a
   predefined class.  As most vendor implementations permit all traffic
   hitting the default class, an explicit drop action would need to be
   configured in the policy such that the traffic hitting that default
   class would be dropped versus permitted and delivered to the control
   plane.  This approach requires rigorous traffic pattern
   identification such that a default drop policy does not break
   existing functionality of the device.  The approach defined in this
   document allows the default traffic and rate limits it rather than
   just dropping it.  This approach was highlighted as a way to give
   time for the implementer to evaluate and characterize traffic in a
   production scenario prior to dropping all traffic not explicitly
   matched and permitted.  However, it is highly recommended that after
   monitoring the traffic matching the default class that explicit
   classes be defined to catch the legitimate traffic.  After all
   legitimate traffic is being explicitly allowed the default class
   should be configured to drop any remaining traffic.

There are also other related passages.

Do you think that this addresses your concern?

Thanks again, Ron !

-- Carlos.

> 
> Ron
> 
> 
> 
> Ron
> 
> _______________________________________________ OPSEC mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/opsec
>

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 26 Oct 2010 17:30:15 -0400

> From: Carlos Pignataro [mailto:[email-masked]]
> Sent: Monday, October 25, 2010 8:31 PM
> To: Ronald Bonica
> Cc: [email-masked]
> Subject: Re: [OPSEC] AD Review of draft-ietf-opsec-protect-control-
> plane-03
> 
> Hi Ron,
> 

OK.. I'll go along with that.

> Do you think that this addresses your concern?

I am OK with rate-limiting other for a short time to discover what traffic has been ignored by the filter. When you discover something that you should have allowed, you add it to the filter and continue the experiment. At some point, when the rate-limit other clause hasn't caught anything for a while, you should drop other.

Rate-limit other should be a temporary thing

                          Ron

---

From: Rodney Dunn <[email-masked]>
Date: Tue, 26 Oct 2010 19:40:28 -0400

Ron,

We 100% agree. We had added verbage in Section 3.3 directly targeted at 
this exact deployment point.

Is it comprehensive and direct enough?

<snip>
    at the control plane and drop all other traffic that does not match a

</snip>

Rodney

---

From: Ronald Bonica <[email-masked]>
Date: Wed, 27 Oct 2010 14:16:56 -0400

Rodney,

I think that we are very close to converging. We agree that:

- while an operator is in the process of identifying legitimate traffic, the default action is to rate-limit
- when the operator has identified all legitimate traffic, the default action is to drop

We agree that both states need to be mentioned in the document. But which should be demonstrated? I think that later/permanent state.

                                             Ron

> From: Rodney Dunn [mailto:[email-masked]]
> Sent: Tuesday, October 26, 2010 7:40 PM
> To: Ronald Bonica
> Cc: Carlos Pignataro; [email-masked]
> > I am OK with rate-limiting other for a short time to discover what
> traffic has been ignored by the filter. When you discover something
> that you should have allowed, you add it to the filter and continue the
> experiment. At some point, when the rate-limit other clause hasn't
> caught anything for a while, you should drop other.
> >
> >

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 27 Oct 2010 13:56:41 -0500

Ron,

Good summary, thanks.

Our thought is that we want to demonstrate the _initial_ state, not the
permanent; the rational is to minimize potential self-blackholing when
first implementing, and to give time to get to the permanent state.

The spirit and philosophy of this document is that it present a
minimalistic and initial state in the demonstration in the Appendix,
that needs to evolve from there.

Do you think this safer approach is OK? and if so that we need to
further clarify the intent beyond the excerpt included below?

Thanks,

-- Carlos.

-----Original Message-----
From: Ronald Bonica [mailto:[email-masked]] 
Sent: Wednesday, October 27, 2010 2:17 PM
To: Rodney Dunn (rodunn)
Cc: Carlos Pignataro (cpignata); [email-masked]
Subject: RE: [OPSEC] AD Review of
draft-ietf-opsec-protect-control-plane-03

Rodney,

- while an operator is in the process of identifying legitimate traffic,
the default action is to rate-limit
- when the operator has identified all legitimate traffic, the default
action is to drop

We agree that both states need to be mentioned in the document. But
which should be demonstrated? I think that later/permanent state.

                                             Ron

>     at the control plane and drop all other traffic that does not
match
> a

---

From: "Smith, Donald" <[email-masked]>
Date: Wed, 27 Oct 2010 13:00:42 -0600

+1

(coffee != sleep) & (!coffee == sleep)
[email-masked] gcia

> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Ronald Bonica
> Sent: Wednesday, October 27, 2010 12:17 PM
> To: [email-masked]
> Subject: Re: [OPSEC] AD Review of
> - while an operator is in the process of identifying
> legitimate traffic, the default action is to rate-limit
> We agree that both states need to be mentioned in the
> document. But which should be demonstrated? I think that
> later/permanent state.
>
>                                              Ron
>
>
> >     where previous classes that have more granular
> classification did
> > not
> >     at the control plane and drop all other traffic that
> does not match
> > a

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

---

From: Ronald Bonica <[email-masked]>
Date: Wed, 27 Oct 2010 15:14:57 -0400

Carlos,

This is really a hard call. Whichever way you present the document, the reader is prone to mistakes. If you present the initial state only, some readers will stay in the initial state forever. They will not be protected from packets-o-death. If you present the final state only, some readers will progress to the final state prematurely. They will cause their own death.

Either way......

Why don't we let a few others weigh in. I will go along with WG consensus.

                                                      Ron

> From: Carlos Pignataro (cpignata) [mailto:[email-masked]]
> Sent: Wednesday, October 27, 2010 2:57 PM
> To: Ronald Bonica; Rodney Dunn (rodunn)
> Cc: [email-masked]
> Subject: RE: [OPSEC] AD Review of draft-ietf-opsec-protect-control-
> plane-03
> 
> Ron,
> 
> Good summary, thanks.
>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 27 Oct 2010 14:24:10 -0500

Ron,

Sounds good.
[I too see both sides (and both cons), and we can go with what makes
more sense to the group.]

Thanks,

-- Carlos.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Ronald Bonica
Sent: Wednesday, October 27, 2010 3:15 PM
To: Carlos Pignataro (cpignata); Rodney Dunn (rodunn)
Cc: [email-masked]
Subject: Re: [OPSEC] AD Review of
draft-ietf-opsec-protect-control-plane-03

Carlos,

This is really a hard call. Whichever way you present the document, the
reader is prone to mistakes. If you present the initial state only, some
readers will stay in the initial state forever. They will not be
protected from packets-o-death. If you present the final state only,
some readers will progress to the final state prematurely. They will
cause their own death.

Either way......

Why don't we let a few others weigh in. I will go along with WG
consensus.

                                                      Ron

> >     where previous classes that have more granular classification
did

---

From: Warren Kumari <[email-masked]>
Date: Wed, 27 Oct 2010 19:18:31 -0400

> Why don't we let a few others weigh in. I will go along with WG consensus.

I guess it depends on which death is preferable -- personally I think the latter one. I'd much rather have folk progress to the final state, realize that they have just shot themselves in the foot and revert the offending bit than have folk get sidetracked while in the "fiddling" stage and not be fully protected (or, worse yet, rate-limiting at a level that works much of the time, but not quite at busy times, which leads to debugging entertainment...)

Warren "No Hats" Kumari.