Subject: Re: [Detnet] Benjamin Kaduk's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

From: János Farkas <[email-masked]>
Date: Sat, 2 Mar 2019 19:22:15 +0100

Hi Benjamin,

Thank you for your review!

Focusing on the main point now: Section 5 Security Considerations. We 
could consolidate the section to address your comment. Would it be 
helpful to replace the section with the following text?:

Security considerations for DetNet are described in detail in 
[I-D.ietf-detnet-security]. This section considers exclusively security 
considerations which are specific to the DetNet Architecture.

Security aspects which are unique to DetNet are those whose aim is to 
secure the specific quality of service aspects of DetNet, which are 
primarily to deliver data flows with extremely low packet loss rates and 
bounded end-to-end delivery latency. A DetNet is implemented using MPLS 
and IP (including both v4 and v6) technologies, and thus inherits the 
security properties of those technologies at both the control plane and 
the data plane.

Security considerations for DetNet are constrained (compared to, for 
example, the open Internet) because DetNet is defined to operate only 
within a single administrative domain (see Section 1).  The primary 
considerations are to secure the request and control of DetNet 
resources, maintain confidentiality of data traversing the DetNet, and 
provide uninterrupted availability of the DetNet quality of service.

To secure the request and control of DetNet resources, authentication 
and authorization must be provided for each device connected to a DetNet 
domain, most importantly to network controller devices. Control of a 
DetNet network may be centralized or distributed (within a single 
administrative domain). In the case of centralized control, precedent 
for security considerations as defined for ACTN can be found in RFC8453, 
Section 9. In the case of a distributed control protocols, DetNet 
security is expected to be provided by the security properties of the 
protocols in use. In any case, the result must be that manipulation of 
administratively configurable parameters is limited to authorized entities.

To maintain confidentiality of data traversing the DetNet, application 
flows must be protected through whatever means is provided by the 
underlying technology. For example, encryption may be used, such as that 
provided by IPSec for IP (Layer 3) flows and by MACSec for Ethernet 
(Layer 2) flows. DetNet flows are identified on a per-flow basis, which 
may provide attackers with additional information about the data flows 
(when compared to networks that do not include per-flow identification); 
this is an inherent property of DetNet which has security implications 
which must be considered when determining if DetNet is a suitable 
technology for any given use case. Similar consideration may need to be 
given to the use of replicated packet flows, which result in increased 
surface for Reconnaissance attacks. In any case, the result must be that 
the confidentiality of the data is maintained to a degree sufficient for 
the use cases addressed by the given DetNet implementation.

To provide uninterrupted availability of the DetNet quality of service, 
provisions must be made against DOS attacks and delay attacks. To 
protect against DOS attacks, excess traffic due to malicious or 
malfunctioning devices must be prevented or mitigated, for example 
through the use of traffic admission control applied at the input of a 
DetNet domain, as described in Section 3.2.1.1, and through the fault 
mitigation methods described in Section 3.3.2.

To prevent DetNet packets from being delayed, protection must be 
provided against Man-In-The-Middle attacks, for example through use of 
authentication and authorization of devices within the DetNet domain.

Time synchronization methods are not specified by DetNet, however any 
time synchronization method used with DetNet must also be secured, using 
the security methods available for the given time synchronization 
mechanism, for example as described in [RFC7384].
In any case, the result must be that the DetNet is protected from DOS, 
delay, and time manipulation attacks to the extent sufficient for the 
use cases addressed by the given DetNet implementation.

DetNet use cases are known to have widely divergent security 
requirements. The intent of this section is to provide a baseline for 
security considerations which are common to all DetNet designs and 
implementations, without burdening individual designs with specifics of 
security infrastructure which may not be germane to the given use case. 
Designers and Implementers of DetNet systems are expected to take use 
case specific considerations into account in their DetNet designs and 
implementations.

Regards,
Janos

On 2/21/2019 9:32 PM, Lou Berger wrote:
> Hi Benjamin,
>
> On 2/20/2019 4:21 PM, Benjamin Kaduk wrote:
>> Hi Lou,
>>
>> On Wed, Feb 20, 2019 at 11:34:11AM -0500, Lou Berger wrote:
>>> On 2/20/2019 11:07 AM, Benjamin Kaduk wrote:
>>>> On Wed, Feb 20, 2019 at 09:33:34AM -0500, Lou Berger wrote:
>>>>> On 2/19/2019 10:44 PM, Benjamin Kaduk wrote:
>>>>>> I note that the DETNET WG is explicitly chartered with a work 
>>>>>> item for the
>>>>>> "overall architecture: This work encompasses ... and security 
>>>>>> aspects".
>>>>>> It seems incomplete to specify an architecture for a topic such as
>>>>>> deterministic networking without specifically considering what 
>>>>>> threats are
>>>>>> and are not in scope to be protected against.
>>>>> The working group certainly recognizes that security is an important
>>>>> topic and necessary part of the work. The WG's  approach to ensure 
>>>>> that
>>>>> this significant topic is sufficiently covered is to have a  document
>>>>> dedicated to the topic.  As you note below this work,
>>>>> draft-ietf-detnet-security, is not at the same level maturity - so
>>>>> please stay tuned.  Of course, the WG would welcome any input on this
>>>>> security draft now or even a full early-review or security area 
>>>>> advisor.
>>>> I do appreciate that the WG has taken the time to work on a dedicated
>>>> security document, that goes through the (tedious!) effort to 
>>>> tabulate the
>>>> various known attacker capabilities, threats, and their interactions.
>>>>
>>>> What I think is missing from the high-level architecture document is a
>>>> sense for what security goals the architecture intends to achieve.  
>>>> It's
>>>> fine to have this be separate from the nitty-gritty stuff in the 
>>>> separate
>>>> document, but these high-level questions can have an impact on the
>>>> high-level design of the protocol ecosystem as a whole.  We have 
>>>> mounds of
>>>> examples of "come up with a design; bolt security on as an 
>>>> afterthought"
>>>> working poorly or being nigh-impossible, and I haven't seen any 
>>>> attempt at
>>>> justification for why doing so here is likely to be any different.
>>>> Including the security concepts from the start and wholly 
>>>> integrating them
>>>> into the system tends to produce a much more elegant design and 
>>>> smoother
>>>> outcomes all around.
>>> We went back and forth on how much of the security topic to include in
>>> the architecture document and settled on the current text of section 5
>>> where just high level security requirements are identified. From your
>>> list below, the architecture covers the points you raise (non-detnet
>>> traffic impacting detnet traffic, and detnet traffic trying to use more
>>> than its allocation) mainly from a service delivery perspective.  It
>>> does mention "shaping" as a security concern, the document could expand
>>> these if helpful. But I think the real discussion point is how much of
>>> the security details should be in the base (vs dedicated) document  --
>>> and that you disagree with where the WG has drawn the line. Is this
>>> correct?
>> Well, it's not as much that I disagree about where the line is drawn, as
>> that I disagree with the characterization as "high level security
>> requirements".
>>
>> The first paragraph covers topics worth including here, certainly -- an
>> attacker that can introduce delay can inherently break the DetNet
>> guarantees, and there is no further defense.  But the text could 
>> probably
>> go further than its current version and explicitly call out that no
>> technical measures can defend against such delay, other than by
>> rerouting/avoiding the compromised position.
>>
>> The second paragraph also covers a topic worth noting, that entropy and
>> Murphy as as important of risks to consider as active attacks.
>>
>> The rest of the section/list don't seem to match up with the high-level
>> view, though -- "protection of the signaling protocol", but protection
>> against what?  Authentication and authorization are important parts of
>> security, but authorization is tied to actions and entities, not just
>> entities in vacuo.  For "identification and shaping of DetNet flows", if
>> security must cover them, what security properties are needed. Do we 
>> need
>> to provide integrity protection for the marking, or confidentiality
>> protection, too?  Does authentication/integrity need to be provided for
>> in-line or are out-of-band considerations in play?
>>
>> The rest of the document presents a good high level picture of what 
>> pieces
>> will be involved in a system, and how they fit together.  I'm hoping 
>> to see
>> a high-level picture of what attacks/failures the architecture does and
>> does not aim to protect against.  It's somewhat implied that we need to
>> protect against the failure of any single node or link (except at the
>> boundary) due to random faults; explicitly saying this would be a 
>> perfect
>> hting to include in the security considerations here!  Once we get 
>> beyond
>> entropy as an attack, into more active attack scenarios, though, we 
>> need to
>> consider what capabilities an attacker might have.  Do we protect 
>> against
>> an entity with the ability to inject various forms of traffic? Modify or
>> drop valid traffic?  Are there any (classes of) credentials that we 
>> assume
>> must be securely held, or will we attempt to protect against 
>> attackers that
>> have some privilege within the system?  (Note that misbehaving 
>> hardware can
>> sometimes present as similar or identical to such an attacker.)
>>
>> Does this give a better picture of what I'm thinking of as "high-level
>> security requirements"?  I don't necessarily need details of specific
>> hardware or credentials at play, and certainly not for the solutions, 
>> but I
>> want to have a clear picture of what the scope of future security 
>> analysis
>> will be.  "How will I be able to tell if the actual protocols meet the
>> security requirements of the architecture?"
>
> I'll get with authors (of both the architecture and security docs) and 
> see how much they think can be reasonably be added to this document -- 
> to be clear you *are* asking for a change in how the WG decide to 
> address this topic and what would go in which document.
>
> Lou
>
>> -Benjamin
>>
>> _______________________________________________
>> detnet mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/detnet
>>
>
> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet

---

From: János Farkas <[email-masked]>
Date: Sat, 2 Mar 2019 21:12:23 +0100

Hi Benjamin,

Coming to your detailed comments.
Please see in-line.

Thank you,
Janos

On 2/20/2019 4:44 AM, Benjamin Kaduk wrote:
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
>
>
> Abstract
>
>                                   DetNet operates at the IP layer and
>     delivers service over sub-network technologies such as MPLS and IEEE
>     802.1 Time-Sensitive Networking (TSN).
>
> I don't know what "sub-network technologies" means.  (Should I?  Is it
> defined somewhere we can reference?)
>
> More generally, is DetNet supposed to be a "sub-layer" and/or "sub-network"
> that lies between specific layers or classes of layer?  Does DetNet
> itself have component "sub-layers" that provide distinct DetNet
> functionality?  These are good questions to address early on in the
> document so the reader is familiar with the concepts as they progress
> through the document.
Please note that the WG has reached consensus on introducing the term 
"sub-layer", in particular using the terms "DetNet Service sub-layer" 
and DetNet Forwarding sub-layer", see: 
https://mailarchive.ietf.org/arch/msg/detnet/TR6oXu7IMWhelrH_I-tOuHGfDqo.
These are explained in detail in Section 4. DetNet Architecture. I do 
not see how to bring it earlier in the document.

Section 4. DetNet Architecture also explains what is meant by 
sub-network. Figure 3 shows it and the text above Figure 3 explains.
I don't see how could be this explanation earlier in the document.

As for the abstract, would it help to replace?:

OLD:
DetNet operates at the IP layer and
    delivers service over sub-network technologies such as MPLS and IEEE
    802.1 Time-Sensitive Networking (TSN).

with

NEW:
DetNet operates at the IP layer and
    delivers service over lower layer technologies such as MPLS and IEEE
    802.1 Time-Sensitive Networking (TSN).

Similar replacement could be done in the introduction:

OLD:
    DetNet operates at the IP layer and delivers service over sub-network
    technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
    (TSN).

NEW:
    DetNet operates at the IP layer and delivers service over lower layer
    technologies such as MPLS and IEEE 802.1 Time-Sensitive Networking
    (TSN).

The next occurrence of sub-network is with a cross-reference to the 
section with Figure 3 and description. I think it is fine this way as 
the reader can jump there and check the details.

>
> Section 1
>
>                            DetNet is for networks that are under a single
>     administrative control or within a closed group of administrative
>     control; these include campus-wide networks and private WANs.  DetNet
>     is not for large groups of domains such as the Internet.
>
> side note: Campus-wide networks at educational institutions are basically
> guaranteed to have untrusted entities participating in them, just as a
> backdrop for security considerations.
>
> Section 3.1
>
>                          This mechanism distributes the contents of
>     DetNet flows over multiple paths in time and/or space, so that the
>     loss of some of the paths does need not cause the loss of any
>
> The failure models for which this statement is absolutely true as opposed
> to probabilistically true seem rather unrealistic models of real physical
> systems.
>
> Section 3.2.1.1
>
>     The primary means by which DetNet achieves its QoS assurances is to
>     reduce, or even completely eliminate packet loss due to output packet
>     contention within a DetNet node as a cause of packet loss.  [...]
>
> editing error?
>
>                                       Note that App-flows are generally
>     not expected to be responsive to implicit [RFC2914] or explicit
>     congestion notification [RFC3168].
>
> I note that the word "implicit" does not appear in RFC 2914; it may be
> worth a bit more detailed of a mapping from concept to reference.
> (This text/reference also appears in Section 4.3.2.)
Would it help to change the text?:

OLD:
    There is no expectation in DetNet for App-flows to be responsive to
    implicit [RFC2914] or explicit congestion notification [RFC3168].

NEW:
    There is no expectation in DetNet for App-flows to be responsive to
    congestion control [RFC2914] or explicit congestion notification 
[RFC3168].

>
> Section 3.2.1.2
>
>                                                                     In
>     general, users are encouraged to use, instead of, "do this when you
>     get the packet," a combination of:
>
> It seems that an architecture would be within its rights to *mandate* such
> application design, rather than just encourage it.  What sorts of
> exceptions would cause us to not want to mandate this design?
This is not mandatory.
For instance, it is not mandatory to use time synchronization. There may 
be DetNet services that do not require time synchronization.

>
> Section 3.2.2.2
>
> Please expand SRLG (it is only used once, so the abbreviation itself may
> not be needed at all).
Replace "SRLG" with "Shared Risk Link Group (SRLG)"

>
> Section 3.2.3
>
>     Out-of-order packet delivery can be a side effect of distributing a
>     single flow over multiple paths especially when there is a change
>     from one path to another when combining the flow.  [...]
>
> nit: comma before "especially".
Add comma.

>
>     Resource allocation
>             The DetNet forwarding sub-layer provides resource allocation.
>             See Section 4.5.  The actual queuing and shaping mechanisms
>             are typically provided by underlying subnet, these can be
>
> nit: is this usage of "subnet" common?
Figure 3 shows it and the text above Figure 3 explains sub-net.

> Also, this comma looks to be a comma splice.
Make it two sentences.

>
>             closely associated with the means of providing paths for
>             DetNet flows, the path and the resource allocation are
>             conflated in this figure.
>
> nit: Hmm, actually, is this comma *also* a comma splice?
Make it two sentences.

>
>     Operations, Administration, and Maintenance (OAM) leverages in-band
>     and out-of-band signaling that validates whether the service is
>     effectively obtained within QoS constraints.  [...]
>
> nit: is there a singular/plural mismatch here ("the service"/"service" vs.
> "effectively within"/"effectively obtained within")?
Seems good to me.

>
> Section 4.1.1
>
> This figure would have helped me a lot several sections earlier.
>
> Section 4.1.2
>
>     A "Deterministic Network" will be composed of DetNet enabled end
>     systems, DetNet edge nodes, DetNet relay nodes and collectively
>     deliver DetNet services.  DetNet relay and edge nodes are
>
> Nit: I think this is intended to be:
> A "Deterministic Network" will be composed of DetNet-enabled end
> systems, DetNet edge nodes, and DetNet relay nodes, which collectively
> deliver DetNet services.  DetNet relay and edge nodes are
Fine by me to make the cahnge.

>
>                                                               Examples of
>     sub-networks include MPLS TE, IEEE 802.1 TSN and OTN.  [...]
>
> nit: are these sub-networks or protocols used by sub-networks?
These are sub-network technologies. They may include/use multiple protocols.

>
>     Distinguishing the function of two DetNet data plane sub-layers, the
>     DetNet service sub-layer and the DetNet forwarding sub-layer, helps
>     to explore and evaluate various combinations of the data plane
>     solutions available, some are illustrated in Figure 4.  This
>
> nit: this last comma is a comma splice.
Suggest to replace with:
"DetNet data plane is divided into two sub-layers: the
DetNet service sub-layer and the DetNet forwarding sub-layer.
This helps to explore and evaluate various combinations of the data plane
solutions available. Some of them are illustrated in Figure 4."
(https://mailarchive.ietf.org/arch/msg/detnet/JvPTAFVTJwxjhzZYEF9uCnUTFBA)

>
>     There are many valid options to create a data plane solution for
>     DetNet traffic by selecting a technology approach for the DetNet
>     service sub-layer and also selecting a technology approach for the
>     DetNet forwarding sub-layer.  There are a high number of valid
>     combinations.
>
> nit: I think "large number" is more conventional prose.
Replace "high number" with "large number".

>
> Section 4.3.1
>
> I think I'm confused about how, for these flows that "require the <foo>
> feature", whether that means that the DetNet implementation must provide
> <foo>, or that it is required for the application to have implemented the
> <foo> feature.  A mapping (if it makes sense) to the categorization of end
> systems in Section 4.2.1 would be a big help.
The notation is crafted such that to provide mapping.

>
> Section 4.3.2
>
>     Asynchronous DetNet flows are characterized by:
>
>     o  A maximum packet size;
>
>     o  An observation interval; and
>
>     o  A maximum number of transmissions during that observation
>        interval.
>
> Is there necessarily only a single tier of observation interval/rate?
> (E.g., could there be a burst cap in a small interval and then a lower
> overall baseline rate over large intervals?)
We intentionally have a simple model here.

>
>                       That is, while any useful application is written to
>     expect a certain number of lost packets, the real-time applications
>     of interest to DetNet demand that the loss of data due to the network
>     is a rare event.
>
> (I might even go with "vanishingly rare".)
Yes, there are extremely low loss requirements in certain use cases. 
That's why we have PREOF.

>
> Section 4.4.1
>
> (Is there a standard reference for "Northbound"?  I know we're all used to
> it, but it's probably best to have a reference if we can.)
RFC7426 is a reference. It is cited at the very beginning of the section.

>
> Section 4.4.2
>
>                                           The deterministic sequence can
>     typically be more complex than a direct sequence and include
>     redundancy path, with one or more packet replication and elimination
>     points.  [...]
>
> nit: "redundancy paths", plural?

Replace "redundancy paths", "redundant paths".

>
> Section 4.8
>
> How does *provisioning* require knowledge of *dynamic* state?
I guess the idea was that the resource may change dynamically.

Would it help to delete "dynamic"?
Thus get: "The state of a DetNet node's DetNet resources."

>
> Section 4.9
>
> Does aggregation like this pose a risk of all the aggregatees getting
> affected when one exceeds their allocation substantially (so as to also
> cause the aggregate to exceed the aggregate's allocation)?
One way could be to apply rate limitation for each member flow being 
aggregated at the input of the aggregate flow. The need to apply rate 
limitation is explained at multiple places in the document.

>
> Section 6
>
> The ability for an attacker to use QoS markings as part of traffic
> correlation/inspection is not new with DetNet, but is probably still worth
> mentioning explicitly.
Do you mean mention it in Section 5 Security Considerations?

The text in my previous mail 
(https://mailarchive.ietf.org/arch/msg/detnet/mOzgdXOJ5lGr4gLBGAapYt6Azl8) 
has a paragraph starting:
"To provide uninterrupted availability of the DetNet quality of service, 
provisions must be made against DOS attacks and delay attacks."

Would it be good to extend that paragraph with a sentence, e.g.,?: 
"Malicious QoS marking should be also considered".

---

From: Benjamin Kaduk <[email-masked]>
Date: Sun, 3 Mar 2019 20:45:57 -0600

Hi Ethan,

On Mon, Feb 25, 2019 at 06:31:16AM +0000, Grossman, Ethan A. wrote:
> Hi Benjamin,
> As editor and co-author of the DetNet Security draft, I thank you very much for your comments regarding the security aspects of the DetNet Architecture and Security draft. Indeed it is our intent to ensure that security is considered throughout the design process and not "bolted on" at the end; this is why we have been developing the Security draft in parallel with the Architecture, Data Plane and other drafts.  Here are a couple of my initial thoughts on your comments; I present them for discussion, not as a defense of a pre-determined outcome. 

Understood.

> I follow your concern that we have not made a distinction between the laundry list of possible threats versus those threats that designs based on the architecture must address in order to satisfy the security requirements of the architecture. Excellent point, in that I for one have not considered the topic in that light before, and I appreciate the need to do so now. 
> 
> Up until this point, i.e. prior to your comments, we have been working
> based on at least two guiding principles. One is that we are only
> addressing security considerations which are specific to time-sensitive,
> routed, centrally-administered networks, so anything that could be
> considered to be "covered elsewhere" we have intentionally omitted. For
> example you cite "using HMAC for message integrity protection without
> mention of infrastructure for distributing the keys for keying the HMAC";
> I would expect that this topic could be regarded as "covered elsewhere",
> but please correct me if I'm missing something here. 

I think it's fine to say that DetNet only needs to provide solutions for
addressing security considerations that are specific to these
time-sensitive networks, but that doesn't necessarily mean that we can
completely ignore the more mundane "generic" considerations.  That is, we
can and should still be aware of what security properties we expect of the
traffic flowing through detnet installations, and choose the other
technologies that are used to build solutions accordingly.  As a complete
example, not necessarily even relevant to detnet, we can say that if we
want confidentiality protection for IP traffic, IPsec is an option, but we
also would need to know that to actually use IPsec we'd either need to pull
in IKE or have pre-established shared keys to use.  Similarly for HMAC to
provide integrity protection -- the underlying technology and its usage is
well-understood, but comes with the caveat that the participants need some
way to share the secret keys it uses.  The idea is not necessarily to say
that detnet must specify a single way of doing things, but rather to be
clear about what it does (and does not!) require, and to have pointers for
how a solution *could* be built using existing technologies, without
requiring it to be done in that way.

> Another main thread running through almost every aspect of DetNet is that as noted in the Use Cases draft, there are many and diverse applications for DetNet, spanning networks contained within a single machine all the way through networks spanning a nation. As such it is difficult to make generalizations about almost anything in DetNet - everything has to be framed as "scalable" so that we don't mandate something that makes sense for one use case but not for another. 
> 
> So the intent of the Security draft (in conjunction with security text in the other DetNet drafts) thus far has been to provide some insights into the aspects of security that are introduced with time-sensitivity to routed networks, with a nod as to how they might be addressed, while being almost completely non-prescriptive about it, leaving the details as an exercise for the reader based on their particular use case. 
> 
> In other words, as soon as we say "a DetNet must include [any specific security property]", someone will inevitably come up with a use case for which that didn't make sense (insufficient time, insufficient CPU cycles, not required in this case because blah, etc.) For example when I look at your questions like "Do we need to provide integrity protection for the marking, or confidentiality protection, too?  Does  authentication/integrity need to be provided for in-line or are out-of-band considerations in play?" my thought would be to say "well it depends on the use case, i.e. the specific implementation". 

In a similar vein to what I note above, I think that at the architectural
level the questions we want to ask are things like "does the architecture
need to support use cases where the deterministic traffic is given
confidentiality protection by the network?"  We don't need to require that
it be used, and potentially not even require that every instantiation of
the architecture have the ability to provide it (though for confidentiality
protection in particular that will definitely be a discussion to have), but
we establish it as something that we need to think about in the protocol
design.  The flip side of this, of course, is that we also have the ability
to explicitly disclaim responsibility for some types of protection, and not
have to worry about that in the protocol design, thus leaving any
protections for that threat to be at a different layer.

> So at this point I don't know whether this stance is valid, or whether it is an abdication of our responsibility, or whether there are some stakes in the ground that we can actually claim will fit all use cases, and thus we should be stating as mandatory. Certainly we would like to avoid the outcome that the architecture leaves open questions as to whether a given implementation satisfies its security requirements. We also want to avoid being overly prescriptive such that "exceptions" would need to be made to accommodate diverse use cases. 

At this point I'm not necessarily looking for hard security requirements,
but rather a requirement for things that we include in the security
analysis of the system.  The answer may well be that (e.g.) it's hard to
defend against an attacker that can delay packets, so all we can do is
document the threat and move on; there could be other cases where a
technical defense is possible, but expensive, so we can provide an optional
mechanism and document the tradeoffs, so that individual use cases can make
their own decisions.

Hope this helps,

Benjamin

> Please let us know how this line of thinking affects your assessment of our situation, if at all. My short term goal here is to unblock publication of the Architecture draft, but not at the expense of adequate coverage and integration of security. 
> 
> Thanks again for your review, 
> Sincerely,
> Ethan (as editor and co-author of the DetNet Security draft and the DetNet Use Cases draft). 
> 
> 
> -----Original Message-----
> From: detnet <[email-masked]> On Behalf Of Lou Berger
> Sent: Thursday, February 21, 2019 12:32 PM
> To: Benjamin Kaduk <[email-masked]>
> Cc: [email-masked]; [email-masked]; The IESG <[email-masked]>; [email-masked]
> Subject: Re: [Detnet] Benjamin Kaduk's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)
> 
> Hi Benjamin,
> 
> >>>>> I note that the DETNET WG is explicitly chartered with a work item 
> >>>>> for the "overall architecture: This work encompasses ... and security aspects".
> >>>>> threats are and are not in scope to be protected against.
> >>>> The working group certainly recognizes that security is an 
> >>>> important topic and necessary part of the work. The WG's  approach 
> >>>> to ensure that this significant topic is sufficiently covered is to 
> >>>> have a  document dedicated to the topic.  As you note below this 
> >>>> work, draft-ietf-detnet-security, is not at the same level maturity 
> >>>> - so please stay tuned.  Of course, the WG would welcome any input 
> >>>> on this security draft now or even a full early-review or security area advisor.
> >>> I do appreciate that the WG has taken the time to work on a 
> >>> dedicated security document, that goes through the (tedious!) effort 
> >>> to tabulate the various known attacker capabilities, threats, and their interactions.
> >>>
> >>> What I think is missing from the high-level architecture document is 
> >>> a sense for what security goals the architecture intends to achieve.  
> >>> It's fine to have this be separate from the nitty-gritty stuff in 
> >>> the separate document, but these high-level questions can have an 
> >>> impact on the high-level design of the protocol ecosystem as a 
> >>> whole.  We have mounds of examples of "come up with a design; bolt security on as an afterthought"
> >>> attempt at justification for why doing so here is likely to be any different.
> >>> integrating them into the system tends to produce a much more 
> >>> elegant design and smoother outcomes all around.
> >> We went back and forth on how much of the security topic to include 
> >> in the architecture document and settled on the current text of 
> >> section 5 where just high level security requirements are identified.  
> >> From your list below, the architecture covers the points you raise 
> >> (non-detnet traffic impacting detnet traffic, and detnet traffic 
> >> trying to use more than its allocation) mainly from a service 
> >> delivery perspective.  It does mention "shaping" as a security 
> >> concern, the document could expand these if helpful. But I think the 
> >> real discussion point is how much of the security details should be 
> >> in the base (vs dedicated) document  -- and that you disagree with 
> >> where the WG has drawn the line.  Is this correct?
> > Well, it's not as much that I disagree about where the line is drawn, 
> > as that I disagree with the characterization as "high level security 
> > requirements".
> >
> > The first paragraph covers topics worth including here, certainly -- 
> > an attacker that can introduce delay can inherently break the DetNet 
> > probably go further than its current version and explicitly call out 
> > that no technical measures can defend against such delay, other than 
> > by rerouting/avoiding the compromised position.
> >
> > The second paragraph also covers a topic worth noting, that entropy 
> > and Murphy as as important of risks to consider as active attacks.
> >
> > The rest of the section/list don't seem to match up with the 
> > high-level view, though -- "protection of the signaling protocol", but 
> > protection against what?  Authentication and authorization are 
> > important parts of security, but authorization is tied to actions and 
> > entities, not just entities in vacuo.  For "identification and shaping 
> > of DetNet flows", if security must cover them, what security 
> > properties are needed.  Do we need to provide integrity protection for 
> > the marking, or confidentiality protection, too?  Does 
> > authentication/integrity need to be provided for in-line or are out-of-band considerations in play?
> >
> > pieces will be involved in a system, and how they fit together.  I'm 
> > hoping to see a high-level picture of what attacks/failures the 
> > architecture does and does not aim to protect against.  It's somewhat 
> > implied that we need to protect against the failure of any single node 
> > or link (except at the
> > perfect hting to include in the security considerations here!  Once we 
> > get beyond entropy as an attack, into more active attack scenarios, 
> > though, we need to consider what capabilities an attacker might have.  
> > Do we protect against an entity with the ability to inject various 
> > forms of traffic?  Modify or drop valid traffic?  Are there any 
> > (classes of) credentials that we assume must be securely held, or will 
> > we attempt to protect against attackers that have some privilege 
> > within the system?  (Note that misbehaving hardware can sometimes 
> > present as similar or identical to such an attacker.)
> >
> > but I want to have a clear picture of what the scope of future 
> > security analysis will be.  "How will I be able to tell if the actual 
> > protocols meet the security requirements of the architecture?"
> 
> I'll get with authors (of both the architecture and security docs) and see how much they think can be reasonably be added to this document -- to be clear you *are* asking for a change in how the WG decide to address this topic and what would go in which document.
> 
> Lou
> 
> > -Benjamin
> >

---

From: Benjamin Kaduk <[email-masked]>
Date: Sun, 3 Mar 2019 21:25:24 -0600

Hi János,

Thanks for the new text.  These are some great things to be thinking about,
but I'm a little worried that it's just specifying mechanism without
describing the motiviations; in fact, it may even be more detailed than we
need for this document.  More inline...

> provide uninterrupted availability of the DetNet quality of service.

"secure" (verb) can mean many different things in different contexts.  So
when we want to "secure" the request and control of resources, that would
presumably be something like denying unauthorized entities the ability to
use the resources, and probably also to limit the consumption of any
given entity.  This probably also means that the ability of resources
external to DetNet to disrupt the DetNet resources should be limited in
some fashion, but the exact details are still unclear to me.
Similarly, the "uninterrupted availability" is not without qualifications:
an EMP or nuclear blast may not be something in the threat model that we
need to defend against.  But if we just say "uninterrupted availability",
the reader has no way to know that.

To be clear, I'm not saying that this introductory text has to change, but
if we keep it, the follow-on paragraphs need to provide the
motivation/reasoning that is missing here.

> domain, most importantly to network controller devices. Control of a 

We require authn/authz because there is/are some risk(s) that we are trying to
protect against.  What is/are that risk?

> administratively configurable parameters is limited to authorized entities.

Deferring on control protocol security is a fine thing to do.

> (Layer 2) flows. DetNet flows are identified on a per-flow basis, which 

(I wouldn't care whether the "For example" sentence is present or not; we
aren't necessarily obligated to provide it.)

> the use cases addressed by the given DetNet implementation.

I could consider putting this in a new paragraph.  I'd also consider
trimming it down: the key point at this level is that detnet provides a new
type of information on packets that can be used by an attacker in the
network looking at metadata.  The specific details of that could be shunted
off to the other document with no real harm.

> mitigation methods described in Section 3.3.2.

This is getting closer to the sort of thing I was looking for -- we need to
provide availability *in the face of DOS attacks and delay attacks*.  We
could probably say more about where the boundary is between "external"
entities that could threaten DOS and "internal" entities that we assume
behave properly -- is it the boundary of the DetNet domain, or will we
consider the risk of misbehaving/compromised equipment in the domain?

> authentication and authorization of devices within the DetNet domain.

(nit: maybe join this into the previous paragraph, since that paragraph's
topic sentence also mentions delay)
This is also pretty close to what I hoped for: it implies that we need to
keep the traffic inside our trusted elements (by avoiding the MITM), since
the MITM is untrusted and could introduce delay, but the internal elements
are trusted to not delay.  So this could be "prevent DetNet packets from
being delayed by external network entities", which implicitly disclaims an
attempt to protect against a compromised DetNet router.

> use cases addressed by the given DetNet implementation.

I'd consider rephrasing this more like "Because DetNet mechanisms can make
heavy use of methods that require precise time synchronization, the
accuracy, availability, and integrity of time synchronization is of
critical importance.  Extensive discussion of this topic can be found in
[RFC7384]."  We may not need the rest of the details in this document.

> case specific considerations into account in their DetNet designs and 

This is good.

-Benjamin

---

From: Benjamin Kaduk <[email-masked]>
Date: Sun, 3 Mar 2019 21:41:31 -0600

> not see how to bring it earlier in the document.

That's a fair point.  I guess I would consider adding a Terminology entry
for "Sub-layer" that notes that "The DetNet architecture divides the DetNet
functionality into two sub-layers, the forwarding sub-layer and the service
sub-layer.", but use your judgement.

> the reader can jump there and check the details.

I agree; these two replacements in abstract+introduction would alleviate
the main question of whether this usage is some existing well-known thing
that the reader should be familiar with, and the rest of the document is
well-contained.

>     congestion control [RFC2914] or explicit congestion notification 

Sure, thanks.

> be DetNet services that do not require time synchronization.

I was basically asking if you could list an example for me (here in email,
not necessarily in the document).

> These are sub-network technologies. They may include/use multiple protocols.

Should this be "examples of sub-networking technologies" then?

> (https://mailarchive.ietf.org/arch/msg/detnet/JvPTAFVTJwxjhzZYEF9uCnUTFBA)

Thanks.

> The notation is crafted such that to provide mapping.

I inferred that intent, yet.  I'm claiming that "It only requires" confused
me about which entity required what.  Changing to "It only uses" would
remove that confusion (but would implicitly deny the ability for
opportunistic use of other features, so is not a perfect replacement).

> We intentionally have a simple model here.

OK.

> Thus get: "The state of a DetNet node's DetNet resources."

That would probably help, yes.  Maybe even "The current state", but I
didn't think about it very hard.

> limitation is explained at multiple places in the document.

OK

> Do you mean mention it in Section 5 Security Considerations?

That's not what I had in mind, no. I was thinking

OLD:
   DetNet is provides a Quality of Service (QoS), and as such, does not
   directly raise any new privacy considerations.

NEW:
   DetNet provides a Quality of Service (QoS) mechanism, and the generic
   considerations for such mechanisms apply.  In particular, such markings
   allow for an attacker to correlate flows or to select particular types
   of flow for more detailed inspection.

> "Malicious QoS marking should be also considered".

I wouldn't oppose something like that, but it may not be needed in the
architecture document (as opposed to the documents that actually describe
the QoS markings).

-Benjamin

---

From: János Farkas <[email-masked]>
Date: Tue, 5 Mar 2019 22:40:04 +0100

Hi Benjamin,

I'm glad that we are progressing. :-)
Thank you!

Please see in-line.

> some fashion, but the exact details are still unclear to me.
"secure is not the right word in this sentence. I suggest to replace it 
with "provide" to get:

           Security aspects which are unique to DetNet are those whose 
aim is to
           provide the specific quality of service aspects of DetNet, 
which are
           primarily to deliver data flows with extremely low packet 
loss rates
           and bounded end-to-end delivery latency.

> the reader has no way to know that.
"uninterrupted availability" is not a good phrase here, can be confused 
with zero loss.
Suggest to replace "uninterrupted availability" with "the" to get:

           Security considerations for DetNet are constrained (compared 
to, for
           example, the open Internet) because DetNet is defined to 
operate only
           within a single administrative domain (see Section 1). The 
primary
           considerations are to secure the request and control of DetNet
           resources, maintain confidentiality of data traversing the 
DetNet,
           and provide the availability of the DetNet quality of service.

> protect against.  What is/are that risk?

What we want to prevent is having unauthorized device placed within a 
DetNet domain, or attached to the DetNet domain; or unauthorized person 
accessing DetNet domain. This helps preventing some of the attacks, 
e.g., man-in-the-middle, diverging the routes of DetNet flows.

> aren't necessarily obligated to provide it.)
I think the examples are useful to make it clearer what is meant.

> off to the other document with no real harm.
OK, make it a new paragraph.

A shorter version:

OLD:
           DetNet flows are identified on a per-flow basis, which may 
provide
           attackers with additional information about the data flows (when
           compared to networks that do not include per-flow 
identification);
           this is an inherent property of DetNet which has security
           implications which must be considered when determining if 
DetNet is
           a suitable technology for any given use case. Similar 
consideration
           may need to be given to the use of replicated packet flows, which
           result in increased surface for Reconnaissance attacks. In 
any case,
           the result must be that the confidentiality of the data is
           maintained to a degree sufficient for the use cases addressed 
by the
           given DetNet implementation.

NEW:
           DetNet flows are identified on a per-flow basis, which may 
provide
           attackers with additional information about the data flows (when
           compared to networks that do not include per-flow 
identification).
           This is an inherent property of DetNet which has security
           implications which should be considered when determining if 
DetNet is
           a suitable technology for any given use case.

> topic sentence also mentions delay)
OK to merge it.

> attempt to protect against a compromised DetNet router.
I like your proposal.
to make it clear that external is relative to the DetNet domain, I 
suggest to have:
"To prevent DetNet packets from being delayed by an entity external to a 
DetNet domain, "

Got another comment suggesting not to use "protection" here. With that 
the suggested new text:

                                                                "To 
prevent DetNet packets from
           being delayed by an entity external to a DetNet domain, DetNet
           technology definition must allow for the mitigation of
           Man-In-The-Middle attacks, for example through use of
           authentication and authorization of devices within the DetNet 
domain."

> [RFC7384]."  We may not need the rest of the details in this document.
I like your proposal.

I suggest to replace

OLD:
"Time synchronization methods are not specified by DetNet, however any
time synchronization method used with DetNet must also be secured, using
the security methods available for the given time synchronization
mechanism, for example as described in [RFC7384]."

with:

NEW:
           Because DetNet mechanisms or applications that rely on DetNet can
           make heavy use of methods that require precise time 
synchronization,
           the accuracy, availability, and integrity of time 
synchronization is
           of critical importance. Extensive discussion of this topic can be
           found in [RFC7384].

That is, I suggest to add "or applications that rely on DetNet" to the 
text you proposed.
I suggest to capture that time synchronization can be important to the 
application running on an end system. There may be cases when a 
particular DetNet deployment does not rely on time synchronization, but 
it is still needed for an application.

Based on your proposal delete:
"          In any case, the result must be that the DetNet is protected 
from DOS,
           delay, and time manipulation attacks to the extent sufficient 
for the
           use cases addressed by the given DetNet implementation."

>> case specific considerations into account in their DetNet designs and

We've got a few more comments. The main one is to remove "must" from the 
security section because the use of security mechanisms is a deployment 
choice, not that of the IETF.

A diff is attached to show the changes described above.

Thank you,
Janos

---

From: János Farkas <[email-masked]>
Date: Tue, 5 Mar 2019 23:16:42 +0100

Hi Benjamin,

Please see in-line some further suggested changes based on your comments.

> sub-layer.", but use your judgement.
I suggest to extend the definitions of the two sub-layers instead of 
introducing a new one:

OLD:

    DetNet forwarding sub-layer
            The DetNet layer that optionally provides resource allocation
            for DetNet flows over paths provided by the underlying
            network.

    DetNet service sub-layer
            The DetNet sub-layer at which A DetNet service, e.g., service
            protection is provided.

NEW:

    DetNet forwarding sub-layer:
                   DetNet functionality is divided into two sub-layers. 
One of
                   them is the DetNet forwarding sub-layer, which optionally
                   provides resource allocation for DetNet flows over paths
                   provided by the underlying network.

    DetNet service sub-layer:
                   DetNet functionality is divided into two sub-layers. 
One of
                   them is the DetNet service sub-layer, at which a 
DetNet service,
                   e.g., service protection is provided.

> not necessarily in the document).
There may be a DetNet service with very strict loss requirement, but not 
that strict delay and jitter requirement. In this case, we may not need 
any fine tuned QoS technique to address delay. However, we need PREOF to 
address loss. Time synchronization is not needed in this example.

> Should this be "examples of sub-networking technologies" then?
That would be clearer phrasing.
Replace
"Examples of sub-networks include MPLS TE, IEEE 802.1 TSN and OTN."
with
"Examples of sub-network technologies include MPLS TE, IEEE 802.1 TSN 
and OTN."

(sub-networking sounds a bit strange to me, not used in the document)

> opportunistic use of other features, so is not a perfect replacement).
We can make it clearer. What about the following update:

OLD:

    A DetNet flow can have different formats while its packets are
    forwarded between the peer end systems.  Therefore, the following
    possible types / formats of a DetNet flow are distinguished in this
    document:

    o  App-flow: native format of the data carried over a DetNet flow.
       It does not contain any DetNet related attributes.

NEW:

    A DetNet flow can have different formats while its packets are
    forwarded between the peer end systems depending on the type
    of the end systems.  Corresponding to the end system types,
    the following possible types / formats of a DetNet flow are
    distinguished in this document. The different flow types have
    different requirements to DetNet nodes.

    o  App-flow: native format of the data carried over a DetNet flow
       between DetNet unaware end systems. An app-flow does not
       contain any DetNet related attributes and does not imply any
       specific requirement on DetNet nodes.

>     of flow for more detailed inspection.
Looks good to me. (the word "attacker" made me unsure whether it goes to 
security)
Combining with the change in 
https://mailarchive.ietf.org/arch/msg/detnet/QqUWlFnG9iJl5qXnuprPC6J4rDo, 
we'd get:

NEW:

    DetNet provides a Quality of Service (QoS), and as such, is not 
expected to
    directly raise any new privacy considerations, the generic 
considerations
    for such mechanisms apply.  In particular, such markings allow for an
   attacker to correlate flows or to select particular types of flow for 
more
   detailed inspection.

> architecture document (as opposed to the documents that actually describe
OK, leave it then out from the architecture document.

Thank you,
Janos

---

From: János Farkas <[email-masked]>
Date: Mon, 25 Mar 2019 17:17:59 +0100

Hi Benjamin,

We believe that we have addressed your comments in the most recent 
revision: https://tools.ietf.org/html/draft-ietf-detnet-architecture-12. 
(https://mailarchive.ietf.org/arch/msg/detnet/utVL9ZVGcOeGtRIASRFx5WT_ErM)

Please let us know what else you would like to see done before you clear 
your DISCUSS.

I/we would be happy to meet with you this week if there is anything you 
would like to discuss.

Regards,
Janos

On 3/5/2019 10:40 PM, János Farkas wrote:
> "secure is not the right word in this sentence. I suggest to replace 
> it with "provide" to get:
>
> "uninterrupted availability" is not a good phrase here, can be 
> confused with zero loss.
> DetNet domain, or attached to the DetNet domain; or unauthorized 
> person accessing DetNet domain. This helps preventing some of the 
> attacks, e.g., man-in-the-middle, diverging the routes of DetNet flows.
>
>>
>           maintained to a degree sufficient for the use cases 
> addressed by the
> "To prevent DetNet packets from being delayed by an entity external to 
> a DetNet domain, "
>
>           authentication and authorization of devices within the 
> DetNet domain."
>
>>
> particular DetNet deployment does not rely on time synchronization, 
> but it is still needed for an application.
>
> "          In any case, the result must be that the DetNet is 
> protected from DOS,
> We've got a few more comments. The main one is to remove "must" from 
> the security section because the use of security mechanisms is a 
> deployment choice, not that of the IETF.
>