Subject: [Detnet] DetNet Security draft update strategy for tomorrow's update (3rd try, plain text format)

From: "Grossman, Ethan A." <[email-masked]>
Date: Thu, 9 Jan 2020 21:37:31 +0000

Hi All (and calling out Lou specifically below),
I will have some time tomorrow to update the Security draft, but so far I am lacking any input from the WG regarding the notes from our last meet-up call (at bottom). So if you have any comments, now would be a good time to voice them. 
The plan for the Security draft (as of IETF 106, i.e. modulo the results of the meeting below) was to make the very minor changes we knew of (limiting scope to MPLS and IP) and then do WG LC and simultaneously start a SecDir review. At this point I am inclined to make a few additional changes (specified below, based on some of the meeting notes) but not do any major surgery such as "removing tutorial material" (as suggested at that call); at this point, lacking other WG input, I would rather leave that in there and see if the SecDir reviewer likes it or not; I feel that the draft as written reflects various individual authors' contributions, and I don't want to toss any out wholesale unless there is a clear mandate to do so. 
There is one question I would like to see answered today if possible: 

1. Is the suggestion of use of IPSec valid (or do we need to explain specific constraints) given the need for flow ID based on 6-tuple. Can anyone comment further on that, e.g. Lou? 

Apart from that (if I get an answer) here is what I plan to change for this revision: 

2. Limit scope to IP and MPLS. (i.e. cut TSN and references to future data planes). 

3. Pull out the appendix with the "security statements from use case draft" and replace it with a statement that the reader is assumed to be familiar with the Use Cases draft (as suggested below). 

4. Incorporate comment from IETF 106 that flow ID and OAM are relevant significant differentiators between MPLS and IP data planes. 

5. Note that for our purpose MPLS is inherently more secure than IP since it is internal to routers and it is well-known how to protect it from outside influence. 

6. Add statements to the effect that we assume a "very well managed network (both data plane and control plane)" as a starting place for this draft. 

We have another DetNet Security meet-up call scheduled for Tues Jan 21 at 10am PST. That would be within a 2-week WG LC period, if I were to start WG LC tomorrow, and also request the SecDir early review. At this point I am inclined to go that way. Any other (timely) input is welcome.

Thanks,
Ethan (as Editor of DetNet Security Draft)
--------------------------------------------------------------------------------
From: detnet <mailto:[email-masked]> On Behalf Of Grossman, Ethan A.
Sent: Wednesday, December 18, 2019 3:12 PM
To: mailto:[email-masked]; Le Boudec Jean-Yves (mailto:[email-masked]) <mailto:[email-masked]>
Subject: [Detnet] DetNet Security Considerations 17Dec19 call notes

Hi All, 
Here are my notes on the DetNet Security call we had last week (attendees: Stewart Bryant, David Black and me). Thus far I have not made any changes to the draft, because the implications of the discussions below, if implemented, could result in a significant revision of the draft, and I would like to hear what y'all think about the proposed direction. So please put in your $0..02. 
Jean-Yves, if you're out there, I would be particularly interested in your input since I know you have experience with observing non-DetNet-WG peoples' reactions to the draft in its current form. 
Reminder that we have scheduled another call for Tuesday Jan 21, 2020, 10am. 
Thanks,
Ethan (as DetNet Security Draft editor) 
-----------------------------------------------------------------------
. On recommendation for using IPSec for DetNet IP - problem: Can't see enough of the flow to do flow differentiation; e.g. need 6tuple but is always hidden in IPSec. Lou wrote that section, need to check with him. 
. We want to set the initial expectation that the DetNet network to be secured is already a well-designed and well-managed network (other possible phrasing: "a traffic-managed, controlled environment" or "a traffic-engineered network") following existing best practices. We assume that all such relevant characteristics are in place as a starting place for this DetNet Security Considerations draft/discussion. We can thus simplify our draft by setting this assumption. Having the draft be shorter is a positive step. 
. Regarding control plane - we can state that every Control Plane has to be well managed and secured by industry standard practice - this removes the need to say a lot of the specific things we say in the draft.
. We want to the initial expectation that the reader will be familiar with the DetNet Use Cases draft, because that is where the "expectations" (not saying "requirements") regarding security are stated, since there is no DetNet Security Requirements ("Expectations") draft. (As a result, our intention is to remove Sec 8. Appendix A: DetNet Draft Security-Related Statements from the draft - any objections to that?) In review if someone says something about lack of specific security "requirements" to review the draft against, we can point to the Use Cases drafts. (This is normal in IETF specs - each RFC is a chapter in the book, assumed reader will read all of chapters. Contrast with other organization specifications which are more self-contained).
. An MPLS network is inherently more secure than an IP network, because with MPLS it is difficult to introduce a rogue LSR or PE into the network, compared with IP, in which a random device is less difficult to introduce onto the network (and from there send packets into the network). An MPLS network is highly managed by its provider, e.g. BT, ATT, etc. Nothing gets in there.
. In contrast to a PW model for MPLS, an IP model can be thought of as an overlay model, so the goal is to protect the overlay forwarding resources. To do that, we need to manage the network as tightly as an MPLS network, equivalent to MPLS-TE. Need to have that as a starting point - should be "this is a mega tight network, nothing gets in that we don't allow in". 
. Another way to phrase this is like "with a standard IT network, we manage it at least as well as a VPN". We have a lot of experience with VPNs running through networks with other VPNs, we know how to do that. Perhaps this could be pointed out somewhere near the text that says that legacy OT networks are "isolated" (meaning "air gapped") (from e.g. the IT network) and are thus less susceptible to attack.  Assume VPN characteristics, but note that any "bumping into" (interference) of one packet with another can have an effect.
. Network slicing - could this be used for DetNet? Or is it that DetNet could be used to implement slicing? This is a meta-problem at architecture level; Perhaps the world has changed since Arch doc was written. Lots of people working on slicing now, e.g. for 3GPP. Perhaps Lou's work in TEAS on TE'd VPN is relevant? Two strategies for this: Classical MPLS model vs SR model. Issue of stealing resources - we could describe this as an issue, or maybe could even refer to slice drafts, but we don't want to be held up by those drafts. Point is that others are thinking about this.
. Draft needs to set out principles, good quality abstract models, rather than details; currently the draft has too many details. Not enough focus on delay and packet loss considerations, which are the core. Current draft focus is explicit on "now" technologies, as opposed to setting considerations for any technology. Need to establish  what they need to do to conform to security considerations. Current draft is drilled down into classical existing world, perhaps not the world we will find in the future. Need to review what we can assume is known, and what we have to do to encompass various technologies.
. Section 3 - seems like a basic tutorial on security threats? Is that level of tutorial needed, given the expectations set above? For example Internal vs external, access to segments, all standard in secured traffic engineered MPLS networks. Can assume that is taken care of. But delaying a packet is more significant, can be very subtle, need to focus more on that. 
. Regarding DetNet to transfer clocking - could be beneficial to clock distribution, however if using PREOF then it could be detrimental to clock sync and ranging. Need to teach people that there are limitations - is this the place for that? We are clear that we are not addressing how time is to be distributed in a DetNet network, but maybe a caveat is in order? Clock xfer is subtle, few understand it.
. Not discussing what a data plane needs to do or how it would do it, but describe properties it needs to have in order to conform to security properties or expectations or needs. That is, we need to state expectations of DP, how it would meet requirements, but if it can't, how it would mitigate them. For example, consider a requirement for "IT traffic must not affect DetNet traffic". So a possible implementation, as one would do in a non-time-sensitive network is to put them in their own VPNs or MPLS tunnels to ensure they run independently. However DetNet special since packets "bumping into" other packets can be a problem.  We need to say that they have to provide this isolation, but not how to provide it. So we can say something like "forwarding bandwidth (and related resources) must be protected so that they are available to detnet traffic". How they do it is up to the dataplane designers. Otherwise we end up committing to solve an infinite number of problems, and possibly limit options of designers to innovate or otherwise solve problems.
-----------------------------------------------------------------------

---

From: Henrik Austad <[email-masked]>
Date: Fri, 10 Jan 2020 10:08:05 +0100

Hi all,

Some points/thoughts on the summary of last confcall. I added a number
to each point from Ethans mail (in an attempt) to appear more coherent.

tl;dr: I agree with the sentiment, but I'm not fully on board with
stripping out as much as what I understood the proposal wanted.

Detailed comments below.

1. "On recommendation for using IPSec for DetNet IP - problem: Can't see
    enough of the flow to do flow differentiation; e.g. need 6tuple but is
    always hidden in IPSec. Lou wrote that section, need to check with him."

I think I missed this discussion. AFAIK, the only parts where we discuss
IPSec is potentional ways an end-user can protect the content of its
stream. Is the concern here that IPSec will hide too much of the header
for DetNet bridges to be able to properly prioritize them?  I must
confess I am not familiar with exactly how the headers of IPSec is
constructed.

2. "We want to set the initial expectation that the DetNet network to be
    secured is already a well-designed and well-managed network (other
    possible phrasing: "a traffic-managed, controlled environment" or "a
    traffic-engineered network") following existing best practices. We
    assume that all such relevant characteristics are in place as a
    starting place for this DetNet Security Considerations
    draft/discussion. We can thus simplify our draft by setting this
    assumption. Having the draft be shorter is a positive step.

If we assume that the network is already well designed and well managed,
then what does this document really provide? I.e. if I were to set up a
DetNet network, I would come to *this* document to see what is relevant
security-wise. I understand that this document should try to not dictate
specific steps, but painting a threat landscape with a broad brush is
not something I think we should leave out.

3. "Regarding control plane - we can state that every Control Plane has
    to be well managed and secured by industry standard practice - this
    removes the need to say a lot of the specific things we say in the
    draft."

But this does not make sense, industry standard practice, would this not
vary depending on what type of network you are setting up? I am
reluctant to cut out things just because we can hide behind "industry
standard practice", *this* document is supposed to help guide what is
industry best practice wrt to DetNet.

4. "We want to the initial expectation that the reader will be familiar
    with the DetNet Use Cases draft, because that is where the
    "expectations" (not saying "requirements") regarding security are
    stated, since there is no DetNet Security Requirements
    ("Expectations") draft. (As a result, our intention is to remove Sec
    8. Appendix A: DetNet Draft Security-Related Statements from the
    draft - any objections to that?) In review if someone says something
    about lack of specific security "requirements" to review the draft
    against, we can point to the Use Cases drafts. (This is normal in
    IETF specs - each RFC is a chapter in the book, assumed reader will
    read all of chapters. Contrast with other organization
    specifications which are more self-contained)."

I agree that we should expect a reader to be familiar with RFC 8578
Determinic Networking Use Cases.

*However*, in the Abstract of detnet-security we state that
 determinsitic networks have been running "real-time operational
 technology (OT) applications", and as of now, the only part of
 detnet-security where we actually raise the concern for the impact a
 disrupted DetNet service will have for RT-apps, is in appendix
 A. Before dropping this, I would like to go through sec. 4 and see if
 we stress this point enough.

5. "An MPLS network is inherently more secure than an IP network,
    because with MPLS it is difficult to introduce a rogue LSR or PE
    into the network, compared with IP, in which a random device is less
    difficult to introduce onto the network (and from there send packets
    into the network). An MPLS network is highly managed by its
    provider, e.g. BT, ATT, etc. Nothing gets in there."

Not very familiar with MPLS, but I assume this comment is aimed at Sec
7.2 "MPLS"?

6. "In contrast to a PW model for MPLS, an IP model can be thought of as
    an overlay model, so the goal is to protect the overlay forwarding
    resources. To do that, we need to manage the network as tightly as
    an MPLS network, equivalent to MPLS-TE. Need to have that as a
    starting point - should be "this is a mega tight network, nothing
    gets in that we don't allow in".

Yes, agree. But does this not conflict with the desire to leave more out
from #2 (industry standard practice)? Or did I misunderstand the
intention behind the cleanup in #2?

7. "Another way to phrase this is like "with a standard IT network, we
    manage it at least as well as a VPN". We have a lot of experience
    with VPNs running through networks with other VPNs, we know how to
    do that. Perhaps this could be pointed out somewhere near the text
    that says that legacy OT networks are "isolated" (meaning "air
    gapped") (from e.g. the IT network) and are thus less susceptible to
    attack.  Assume VPN characteristics, but note that any "bumping
    into" (interference) of one packet with another can have an effect.

What part of the document does this apply to? Not sure if I fully grok
this section.

8. "Network slicing - could this be used for DetNet? Or is it that
    DetNet could be used to implement slicing? This is a meta-problem at
    architecture level; Perhaps the world has changed since Arch doc was
    written. Lots of people working on slicing now, e.g. for
    3GPP. Perhaps Lou's work in TEAS on TE'd VPN is relevant? Two
    strategies for this: Classical MPLS model vs SR model.. Issue of
    stealing resources - we could describe this as an issue, or maybe
    could even refer to slice drafts, but we don't want to be held up by
    those drafts. Point is that others are thinking about this.

If a bridge allows an actor to inject so much traffic that it
affects DetNet flows, it is an issue, so a bridge must adequately handle
flooding. This also goes for control-traffic, the network must be
configured to handle sudden floods without loosing DetNet flows *or*
control frames as that is a pretty likely attack scenario.

9. "Draft needs to set out principles, good quality abstract models,
    rather than details; currently the draft has too many details. Not
    enough focus on delay and packet loss considerations, which are the
    core. Current draft focus is explicit on "now" technologies, as
    opposed to setting considerations for any technology. Need to
    establish what they need to do to conform to security
    considerations. Current draft is drilled down into classical
    existing world, perhaps not the world we will find in the
    future. Need to review what we can assume is known, and what we have
    to do to encompass various technologies.

Yes, I agree to this. I think we discussed this way back, we had a
pretty lengthy discussion about likely threat vectors to a DetNet setup,
some of them are a bit heavy on the details in an effort to explain the
gist of them.

10. "Section 3 - seems like a basic tutorial on security threats? Is
     that level of tutorial needed, given the expectations set above?
     For example Internal vs external, access to segments, all standard
     in secured traffic engineered MPLS networks. Can assume that is
     taken care of. But delaying a packet is more significant, can be
     very subtle, need to focus more on that.

I'd like to keep a point about the significance of a successful
injection attack. For an IT network, a replay-attack can do bad things
to your RDBS, for DetNet you could harm physical things, physically
harming humans.

I agree that we could focus more on delay-attacks, but we *should* keep
some points about the other attacks simply to ensure that they are
taken into consideration.

11. "Regarding DetNet to transfer clocking - could be beneficial to
     clock distribution, however if using PREOF then it could be
     detrimental to clock sync and ranging. Need to teach people that
     there are limitations - is this the place for that? We are clear
     that we are not addressing how time is to be distributed in a
     DetNet network, but maybe a caveat is in order? Clock xfer is
     subtle, few understand it.

Perhaps this could be used as an example for why delay-attacks can be
detrimental, help elaborate in #10? Apart from that, the finer details
of clock distribution is best done elsewhere. I was not under the
impression that we discussed clock distribution much though.

12. "Not discussing what a data plane needs to do or how it would do it,
     but describe properties it needs to have in order to conform to
     security properties or expectations or needs. That is, we need to
     state expectations of DP, how it would meet requirements, but if it
     can't, how it would mitigate them. For example, consider a
     requirement for "IT traffic must not affect DetNet traffic". So a
     possible implementation, as one would do in a non-time-sensitive
     network is to put them in their own VPNs or MPLS tunnels to ensure
     they run independently. However DetNet special since packets
     "bumping into" other packets can be a problem.  We need to say that
     they have to provide this isolation, but not how to provide it. So
     we can say something like "forwarding bandwidth (and related
     resources) must be protected so that they are available to detnet
     traffic". How they do it is up to the dataplane
     designers. Otherwise we end up committing to solve an infinite
     number of problems, and possibly limit options of designers to
     innovate or otherwise solve problems.

Yes, we've discussed this and I was under the impression that that is
what we were aiming at.

I read this as a summary of all the above points and infer that we need
to clean up and strip down a bit in order to meet this goal. Was that
the intention of #12, or did I read that wrong?

-- 
Best regards,
Henrik Austad

---

From: "Black, David" <[email-masked]>
Date: Fri, 10 Jan 2020 23:53:24 +0000

Hi Henrik,

Comments inline on some of your points/thoughts ...

Thanks, --David
----------------------------------------------------------------
David L. Black, Senior Distinguished Engineer
Dell EMC, 176 South St., Hopkinton, MA  01748
+1 (774) 350-9323 New    Mobile: +1 (978) 394-7754
[email-masked]
----------------------------------------------------------------

> -----Original Message-----
> From: detnet <[email-masked]> On Behalf Of Henrik Austad
> Sent: Friday, January 10, 2020 4:08 AM
> To: Grossman, Ethan A.
> Cc: [email-masked]; Le Boudec Jean-Yves ([email-masked]);
> [email-masked]; [email-masked]
> Subject: Re: [Detnet] DetNet Security draft update strategy for tomorrow's
> update (3rd try, plain text format)
> 
> 
> [EXTERNAL EMAIL]
> 
> Hi all,
> 

[David>] And the IPsec headers is where the problems are to be found.   The DetNet IP data plane identifies flows via a 6-tuple that consists of two IP addresses, the transport protocol ID, two transport protocol port numbers and the DSCP in the IP header.   When IPsec is used, the transport header is encrypted and the next protocol ID is an IPsec protocol, usually ESP, and not a transport protocol  (e.g., neither TCP nor UDP nor ...), leaving only three components of the 6-tuple, the two IP addresses and the DSCP.  This assumes that DetNet nodes cannot decrypt IPsec traffic.
 
> not something I think we should leave out.

[David>] Well-designed and well-managed is primarily about denial-of service prevention by ensuring that DetNet traffic goes where it's supposed to and that an external attacker can't inject traffic that disrupts DetNet's delivery timing assurance.  Security involves a *LOT* more ...
> industry best practice wrt to DetNet.

[David>] Actually it does, see RFC 7510 (MPLS/UDP) and the TMCE (Traffic Managed Controlled Environment) scenario in RFC 8086 (GRE/UDP) for a couple of worked examples of this sort of approach.  RFC 8086 is also a good place to understand what differs between a scenario that makes this sort of assumption and a scenario that does not.

> intention behind the cleanup in #2?

[David>]  This may be somewhat overstated, as many IP networks are not managed exactly as tightly as an MPLS network, but also see my comment on #2 above.
 
> What part of the document does this apply to? Not sure if I fully grok

[David>] VPN seems to be a better way to think about this than matching MPLS, and again, see my comment on #2 above.

> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet

---

From: Henrik Austad <[email-masked]>
Date: Mon, 13 Jan 2020 23:18:36 +0100

> Comments inline on some of your points/thoughts ...

Hi David,

Thanks for the reply!

I replied inline below, but tl;dr; I got my concerns answered.

-Henrik

> [David>] And the IPsec headers is where the problems are to be found.  
> The DetNet IP data plane identifies flows via a 6-tuple that consists of 
> two IP addresses, the transport protocol ID, two transport protocol port 
> numbers and the DSCP in the IP header.  When IPsec is used, the transport 
> header is encrypted and the next protocol ID is an IPsec protocol, 
> usually ESP, and not a transport protocol (e.g., neither TCP nor UDP nor 
> ...), leaving only three components of the 6-tuple, the two IP addresses 
> and the DSCP.  This assumes that DetNet nodes cannot decrypt IPsec 
> traffic.

Ah, yes. Having each node decrypt the IPSec stream is not a valid approach! 
I see Ethan added the clarification in 2dbbf18d, so I have no further 
argument here.

> [David>] Well-designed and well-managed is primarily about denial-of 
> service prevention by ensuring that DetNet traffic goes where it's 
> supposed to and that an external attacker can't inject traffic that 
> disrupts DetNet's delivery timing assurance.  Security involves a *LOT* 
> more ...

Ok, I think I misunderstood your point there then, my bad, or perhaps I did 
not make my opinions clear. I do not disagree with this.

> [David>] Actually it does, see RFC 7510 (MPLS/UDP) and the TMCE (Traffic 
> Managed Controlled Environment) scenario in RFC 8086 (GRE/UDP) for a 
> couple of worked examples of this sort of approach.  RFC 8086 is also a 
> good place to understand what differs between a scenario that makes this 
> sort of assumption and a scenario that does not.

Ok, I have some reading to do it seems :)

> [David>] This may be somewhat overstated, as many IP networks are not 
> managed exactly as tightly as an MPLS network, but also see my comment on 
> #2 above.

Ok

> [David>] VPN seems to be a better way to think about this than matching 
> MPLS, and again, see my comment on #2 above.

Ok

-- 
Henrik Austad