Subject: what is a threat analysis?

From: Michael Thomas <[email-masked]>
Date: Wed, 10 Aug 2005 12:41:56 -0700

Having a "threat analysis" was brought up at the plenary by Steve
Bellovin as being a Good Thing(tm). At the MASS/DKIM BOF we are
being required to produce such a thing as a prerequisite to even
getting chartered as a working group. The problem that I have (and
Dave Crocker at the plenary) is that there doesn't seem to be
any definition of what a "threat analysis" is. Contrary to what
it seems many people demanding such a thing suppose, the term
isn't self evident. Maybe I've missed it but I'm not sure that
I've ever seen one. Worse, I'm not very sure that the people who
were telling us that we needed one could easily be able to come to
consensus on what constitutes a threat analysis.

So, if this is going to be yet another hoop that the IESG and IAB
sends working groups through like problem statements, requirements
documents and the like, I think it ought to be incumbent on
those people demanding such things to actually both agree and
document what it is that they are demanding. This is not just
annoyance at yet more process on my part, but a real desire to
not have people waste a lot of time producing documents that
fail to meet a definition that is otherwise only determined by
multiple iterations of "that's not what we want". This is, in
fact, what happened this time around, and has happened in the
past with the SIP wg.

		Mike

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Dave Crocker <[email-masked]>
Date: Wed, 10 Aug 2005 14:56:21 -0700

> any definition of what a "threat analysis" is. 

As I posted on the DKIM mailing list on Monday 
<http://mipassoc.org/pipermail/ietf-dkim/2005q3/000033.html> our AD, Russ 
Housely has provided us with a rather straight-forward, 3-question template 
for discussing DKIM's threat analysis:

   * Who are the bad actors?
   * Where do they fit into the protocol environment (eg, middle of net)?
   * What are we trying to prevent them from doing?

I think Russ' list is quite reasonable and he has been clear as to the reason 
he views the development of the threat analysis (TA) as a pre-requisite. 
Further, given the history both of previous anti-spam standards efforts and 
previous IETF security efforts, I am hard-pressed to disagree with him.  It's 
a pain in the ass to have to develop this stuff prior to chartering, but the 
project manager in me knows it will be Very Good Thing for ensuring coherence 
and focus of the effort.

(In fact my own real concern about the TA requirement is noting how little 
followup there has been on the DKIM list, since my original posting there, 48 
hours ago. Some folks are having great fun, there, debating other details 
about DKIM concepts and DKIM documentation, but there has been essentially no 
followup on threat analysis.)

At any rate, my point at the plenary was a concern that the Security 
community, itself, does not yet appear to have adequate consensus description 
of, and requirements for, TA to give the rest of us consistent guidance. From 
what I have seen over the years, the requirement for doing a TA up-front, in 
the IETF, is recent.  So it is not surprising that the method of doing it is 
unfamiliar to the rest of us.

> documents and the like, I think it ought to be incumbent on

No question about it.  Recruiting a larger community to support an effort 
certainly involves tasks that aren't fun.  Especially since it should be so 
obvious why the effort is wonderful...

In any event, Russ has been clear and consistent.  Although general discussion 
about threat analysis shows community variations, the DKIM effort has not 
(yet) been given inconsistent guidance (that I am aware of.)

d/

---

From: Stephen Kent <[email-masked]>
Date: Wed, 10 Aug 2005 18:24:07 -0400

Dave & Michael,

In the DoD environment, a threat analysis for a system identifies the 
classes of adversaries that the author believes are of concern, and 
describes their capabilities and motivations. Russ's three questions 
are a concise way of stating this:
	- The "bad actors" are adversaries.
	- Their capabilities allude to where the adversaries "fit 
into the system" and what sorts of attacks they may employ of effect 
their goals.
	- Their motivations indicate what they are trying to do, the 
flip side of "what are we trying to prevent them from doing."

The term is often used more broadly in the commercial world today, 
encompassing activities that might be better termed "threat 
assessment" "risk analysis" etc.

Steve

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Wed, 10 Aug 2005 21:19:44 -0400

>he views the development of the threat analysis (TA) as a pre-requisite. 

The only thing I'd add is a clarification of the first point: are they 
on links, on nodes, or both?  One of the points of my talk is that in 
multiparty protocols, you don't know who runs remote protocol 
participants, even in the absence of hacking.  

		--Steven M. Bellovin, http://www.cs.columbia.edu/~smb

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 11 Aug 2005 11:11:28 +0200

Michael, you've had some quite concrete responses which I hope
have clarified things, but I really want to say that making
Internet protocols secure isn't a hoop jumping exercise; it's
more like a survival requirement, and has been for ten years
at least.

    Brian

---

From: Michael Thomas <[email-masked]>
Date: Thu, 11 Aug 2005 07:52:58 -0700

> more like a survival requirement, and has been for ten years

Where did I say that? My issue is that if people are going
to invoke process, they should be prepared to define what
the process is. And not just hand waving; concrete pointers
to documents that have been through the rough consensus
mechanism so that all parties can shoot for a common
goal.

And to answer your question, no it hasn't been answered
because I've yet to hear from the people -- and there were
many on both the IESG and IAB -- who were asking for it.
Do you seriously think you could write a "threat analysis"
given the definition in 2828?

		Mike

---

From: Dave Crocker <[email-masked]>
Date: Thu, 11 Aug 2005 10:57:04 -0700

Michael,

Your view of requirements for IETF process do not match mine and your 
assessment of the current situation with the DKIM effort does not match mine.

In fact I do not know what specific behaviors, and by whom, you are 
complaining about.

> My issue is that if people are going

The requirement for a threat analysis, prior to chartering a DKIM working 
group, was stated by our cognizant area director.  He provided concrete, 
written guidance, in the form of the 3-point questionnaire that I've cited. It 
would be delightful to have a tutorial, but what he gave us is a long way from 
hand-waving.

Current IETF rules require a sponsoring AD. Whether to do the sponsoring is 
their own decision and, therefore, must satisfy their own professional 
criteria.  The rest of the IESG also needs to be satisfied, well enough to 
pass the working group approval vote.

The process certainly can be subject to whimsy, but it is quite clear that 
this is not the case here.

My own comment at the plenary was a very long way from a complaint about the 
handling of the DKIM effort. It was, rather, a request for more assistance, in 
the form of a) work by the security community to develop better, 
community-wide consistency about the term, and b) more tutorial guidance for 
completing this thing being called a threat analysis.

It would be great to have rough consensus about all things at all time, but if 
you think about that for more than a millisecond you will see that it is a 
good way to ensure never doing anything new, and more likely never doing anything.

Steve Bellovin's plenary presentation did quite a good job of explaining the 
challenges in the security area.  The area has been searching for ways to make 
tools with better, more predictable, and longer-lasting efficacy.  That 
requires exploring planning and design techniques.  A threat analysis seems 
like a pretty reasonable path to explore, especially since it apparently has 
some history in the security community.

So, I am sorry, but I really do not understand what you are complaining about, 
that has any application to the DKIM effort.

d/

ps. On the DKIM mailing list, I have twice posted requests for discussion 
about some draft text describing the threats to which DKIM responds.  As 
nearly as I can tell, your own postings, there, have been about an aspect of 
the DKIM specification, rather than about the threat analysis.  We are, 
finally, starting to get some focus on the topic, so I encourage you to help 
us try to develop rough consensus on our own threat analysis effort.

---

From: Ned Freed <[email-masked]>
Date: Thu, 11 Aug 2005 10:47:50 -0700 (PDT)

> Where did I say that?

Of course you didn't, and the implication that you did say that was nothing but
a strawman, a tactic I'm sad to say often seems to crop up in discussions on
the IETF list.

> the process is.

Absolutely right.

> And not just hand waving; concrete pointers

I agree, although some degree of bootstrapping is permissible.

> many on both the IESG and IAB -- who were asking for it.

And what we have heard has, for me at least, confused rather than clarified.

> given the definition in 2828?

I certainly could write something that conformed to that (very vague)
definition. Whether or not it would actually be useful, and whether or not it
would be what's being asked for here, are different matters entirely.

I actually think that, having read several threat analyses and even contributed
to a couple more, I know what a threat analysis is. The problem as I see it is
that there are several different sorts of threat analsysis that coupld be done
in the MASS/DKIM space, and it isn't clear what one we're being asked to
provide.

For example, one sort would be to look at how DKIM can be directly attacked:
Private key theft, replays, taking advantage of weaknesses in the delsp
canonicalization algorithm and/or line count limits, hash function exploits,
that sort of thing. This would be a highly technical analsysis of the DKIM
proposal itself.

Another, somewhat overlapping, analysis would be of likely responses by
spammers to widespread use of DKIM. This would include creation of bogus but
similar domains, exploiting the gaps between the signature identity and message
originator information, and so on. This is more a gaming exercise than anything
else (and the obvious tool to use to describe the result would be an attack
tree).

And yet another would be to look at threats to email in general and attempt to
figure out where DKIM fits in the overall email threat picture. This will have
more the flavor of a justification for the DKIM approach and a specification of
what DKIM is intended to do.

My guess is that the third of these is what's being asked for. But that's just
a guess on my part. And maybe I'm just being dense, but the list of questions
to be answered provided by Russ Housely did NOT help clarify this at all:

In any case, I for one am completely unwilling to spend time working on this
until I know what the goal is.

				Ned

---

From: Stephen Kent <[email-masked]>
Date: Thu, 11 Aug 2005 15:40:07 -0400

Folks,

I thought that what Russ asked for was not a threat analysis for 
DKIM, but a threat analysis for Internet e-mail, the system that DKIM 
proposes to protect. The idea is that only if we start with a 
characterization of how and why we believe adversaries attack e-mail, 
can we evaluate whether any proposed security mechanism, e.g., DKIM, 
is appropriate, relative to that threat analysis.

Steve

---

From: Michael Thomas <[email-masked]>
Date: Thu, 11 Aug 2005 13:43:40 -0700

Stephen Kent wrote:
> Folks,
> 
> I thought that what Russ asked for was not a threat analysis for DKIM, 
> but a threat analysis for Internet e-mail, the system that DKIM proposes 
> to protect. The idea is that only if we start with a characterization of 
> how and why we believe adversaries attack e-mail, can we evaluate 
> whether any proposed security mechanism, e.g., DKIM, is appropriate, 
> relative to that threat analysis.

That's pretty much my guess, but at least 2 other people
guessed completely wrong. Which is really why I'm vexed
by this -- why should we have to be guessing in the first
place? As I said, at least 5 members of the IESG or IAB
piped up about this as if it were self evident.

Here's something I wrote a while back which I intend to
dust off and propose as the basis for a formal requirements
draft that we have promised in our charter. Sections 4 and
5 are kind of what I would think of as being asked here as
a threat analysis, but as I've stated, I'm not really sure
that this jibes with what other people think is being asked
for.

http://www.mtcc.com/standards/draft-thomas-mass-req-00.txt

		Mike

PS: this still needs some editing, especially in the requirements
     section, so don't take this as my current position...

---

From: Ned Freed <[email-masked]>
Date: Thu, 11 Aug 2005 14:38:22 -0700 (PDT)

> assessment of the current situation with the DKIM effort does not match mine.

They match mine pretty well, however.

> In fact I do not know what specific behaviors, and by whom, you are

Actually, I think Michael has been quite specific in his complaint, and I'm
behind him 100%.

> group, was stated by our cognizant area director.

Nobody is saying that having such an analysis is a bad idea.

> He provided concrete,

The three points that were given are simply another way of describing what goes
into a generic threat analysis. So I guess it is fair to say they aren't
handwaving.

However, as far as I can tell they offer no guidance whatsoever about the scope
and direction of the analysis, which is vital lest we work on the wrong thing.

As I said before, perhaps I'm just being dense and not seeing how this tells us
what to do. But simply reiteraling that these points have been provided isn't
telling what I'm missing.

> completing this thing being called a threat analysis.

Dave, there are at least five RFCs (4016, 3837, 3833, 3756, 3694) that analyze
threats in onw way or another. There are also plenty of other examples in the
security literature, so examples, at least, are fairly abundant.

However, if you look at these you'll find that some of them look at threats to
entire services, others look at attacks on specific protocols, and still others
try and model various sorts of attacks. There are lots of ways this particular
tool can be used.

While it would be really nice to have a "guide to performing threat analysis"
and I think the increased calls for such analyses (which did not start with
MASS/DKIM) mean the production of such a guide should be a work item for the
security area, I don't think the lack of such a guide is the real issue here.

> good way to ensure never doing anything new, and more likely never doing anything.

Again, there is nothing new about calling for threat analysis as part of the
IETF process. What is new, or different, in my experience at least, is having a
call for such an analysis with no guidance as to scoping or direction.

> us try to develop rough consensus on our own threat analysis effort.

I will again repeat that I have zero interesting in discussing the analysis you
have started writing until I have some assurance it is headed in the right
direction. I will also add that my best guess is that what you have is fairly
wide of the mark. More specifically, what you have is focused fairly
narrowly on spam and forgery, whereas my guess as to what is being
asked for is a much more general analysis of email threats.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 11 Aug 2005 15:08:50 -0700 (PDT)

> is appropriate, relative to that threat analysis.

This is more less my guess as to what's being asked for, although I
disagree with the implication that DKIM proposes to protect email in
its entirety. Regardless, others do not appear to agree and instead
apppear to be doing very different sorts of analyses.

				Ned

---

From: Bill Sommerfeld <[email-masked]>
Date: Thu, 11 Aug 2005 18:10:00 -0400

> is appropriate, relative to that threat analysis.

I wasn't at the MASS BoF so I'm likely missing context here.

It's been quite clear for a while that many of the proposals for
securing Internet e-mail have fallen flat specifically when they've run
into disagreements about the threat model.

So an effort to come up with a consensus threat analysis sounds like a
very good idea.  It might even be worthy of a working group of its own
as it would likely be useful as a base for more than just the MASS/DKIM
work.

						- Bill

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 12 Aug 2005 10:16:05 +0200

One small point.....

> mechanism so that all parties can shoot for a common

I did not hear at any stage Russ claiming that asking for a threat analysis 
was "invoking process". He was asking for information that would allow him 
to make up his mind about whether or not to support DKIM becoming a WG.

As far as I know, there is no formal process called "ask for a threat 
analysis". Some people would argue that there should be, and if that 
argument were to be adopted, it should certainly have guidance attached to 
it.

But in this case, I believe all the formal process there is is "AD, using 
his or her best judgment, will decide".
This is contained in RFC 2418 section 2.1 (quoted below). The threat 
analysis asked for seems to be intended to improve the basis for judgment 
on the first 3 points....
-----------------------------------------------------------------------
2.1. Criteria for formation

   When determining whether it is appropriate to create a working group,
   the Area Director(s) and the IESG will consider several issues:

    - Are the issues that the working group plans to address clear and
      relevant to the Internet community?

    - Are the goals specific and reasonably achievable, and achievable
      within a reasonable time frame?

    - What are the risks and urgency of the work, to determine the level
      of effort required?

    - Do the working group's activities overlap with those of another
      working group?  If so, it may still be appropriate to create the
      working group, but this question must be considered carefully by
      the Area Directors as subdividing efforts often dilutes the
      available technical expertise.

    - Is there sufficient interest within the IETF in the working
      group's topic with enough people willing to expend the effort to
      produce the desired result (e.g., a protocol specification)?
      Working groups require considerable effort, including management
      of the working group process, editing of working group documents,
      and contributing to the document text.  IETF experience suggests
      that these roles typically cannot all be handled by one person; a
      minimum of four or five active participants in the management
      positions are typically required in addition to a minimum of one
      or two dozen people that will attend the working group meetings
      and contribute on the mailing list.  NOTE: The interest must be
      broad enough that a working group would not be seen as merely the
      activity of a single vendor.

    - Is there enough expertise within the IETF in the working group's
      topic, and are those people interested in contributing in the
      working group?

    - Does a base of interested consumers (end-users) appear to exist
      for the planned work?  Consumer interest can be measured by
      participation of end-users within the IETF process, as well as by
      less direct means.

    - Does the IETF have a reasonable role to play in the determination
      of the technology?  There are many Internet-related technologies
      that may be interesting to IETF members but in some cases the IETF
      may not be in a position to effect the course of the technology in
      the "real world".  This can happen, for example, if the technology
      is being developed by another standards body or an industry
      consortium.

    - Are all known intellectual property rights relevant to the
      proposed working group's efforts issues understood?

    - Is the proposed work plan an open IETF effort or is it an attempt
      to "bless" non-IETF technology where the effect of input from IETF
      participants may be limited?

    - Is there a good understanding of any existing work that is
      relevant to the topics that the proposed working group is to
      pursue?  This includes work within the IETF and elsewhere.

    - Do the working group's goals overlap with known work in another
      standards body, and if so is adequate liaison in place?

   Considering the above criteria, the Area Director(s), using his or
   her best judgement, will decide whether to pursue the formation of
   the group through the chartering process.

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Fri, 12 Aug 2005 11:07:12 +0200

>as it would likely be useful as a base for more than just the MASS/DKIM

Agree. After 9/11 ICANN changed all its priorities towards security. I 
proposed the BC (Business Constituency) to work on an "ICP-4 Network 
Security" proposition (ICPs are the ICANN documents). Several large 
operators, manufacturers contributed to the project which went to an end 
rather quickly for Layer 8/9 reasons. The first things we tried to do was:

- to list all the possible types of threads on the conceptual network (this 
does not include the user security by itself).
- all the motivations for attacks - from war down to employee retaliation 
or flooding, etc.

We saw it was an iterative process. Motivations helping identifying 
threats, and vice versa. I am not sure I still have a copy of the work we 
carried (it was privately made in the paranoia of security which was the 
mood), but I established a method to help it. Anyway from this experience I 
would suggest the Charter of such a group should be discussed with an 
enlarged committee including Steve Crocker's ICANN security committee, 
people from Govs, Military and Commercial Intelligence, people from GAC, 
people from targeted industries (Banks, Airlines, critical infrastructures, 
life depending solutions).

I used this experience when considering the network security aspects in the 
dot-root test-bed, carried in 2002/2003, and for the resulting "national 
vulnerability to the internet" meetings I organised for France in 
2003/2004. This lead to the identification of the need of a 
compartmentalisation of the internet for risk containment, intelligence 
protection, community development, service innovation, spaces of exhanges, 
trust and services, etc. and of the need to analyse, organise and run its 
intergovernance. We incorporated the AFRAC as a national internet community 
effort to work and test on the matter.

We observed that it was the best way to achieve the necessary virtual 
partitionning of the internet while preventing the grassroots balkanisation 
which develops. Basically it means that the unicity and the unity of the 
global network will be supported and stenghtened by a myriad of well 
established, suited and protected externets (virtual external network 
look-alike). Security of a distributed network is to be distributed.

Also, one has to consider the external implications of the security 
failures and understand that threats can be to second, third etc. degree, 
the implication can be major and concern (due to the today implication of 
the Internet in the world's life) the life of millions. This was the first 
study, which served as basis for the thinking, and the most important one 
todate due to its implications: http://whitehouse.gov/pcipb and its 
preparation is certainly a basic input to consider in that area. Question 
is: is that IETF? IETF is probably more interested in the resulting 
specifications? The implications on the current internet structure and 
vision are important.

jfc

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Fri, 12 Aug 2005 11:46:21 +0200

>     - Are the issues that the working group plans to address clear...
>
>     - Are the goals specific...

If you cannot tell us what the threat is that you are trying to address, 
and in particular if you cannot answer Russ's three questions, then the 
answer to these criteria is "no".

Anti-spam and related efforts in the IETF have a history of the answer to 
this question being "no".  That is a big part of why MARID concluded 
without reaching consensus -- the participants had never really agreed on 
what the problem was.  That same problem was evident in the first MASS BOF.

Due to this history, it is entirely reasonable for Russ to insist on making 
sure the answer is "yes" before chartering the work, since otherwise it's 
likely to be a huge waste of everyone's time.  He is free to do this in any 
way he chooses.  The approach he's chosen is pretty straightforward - he 
asked you to define the problem you are trying to solve.

Frankly, it amazes me that so many people start to do significant work 
coming up with proposals in this area without bothering to first figure out 
what they are trying to accomplish.  That's bad management practice _and_ 
bad engineering.

I would suggest that if you really want this work to go forward in the 
IETF, you should quit arguing about whether some glossary defines the term 
"threat analysis" well enough, and get on with the business of answering 
the specific questions Russ actually asked.  I expect that having coherent, 
meaningful answers to those questions will go a long way toward actually 
getting a working group chartered, and will also make the group 
considerably more effective once it is chartered.

-- Jeff

---

From: Stephen Kent <[email-masked]>
Date: Fri, 12 Aug 2005 09:29:00 -0400

>apppear to be doing very different sorts of analyses.

I agree that DKIM need not protect e-mail in all security dimensions. 
My definition of threat analysis for this context does not require 
that, although I admit the wording could have been clearer.

In any threat analysis, the author decides what threats he/she wants 
to address. The reader decides if the author has omitted any that the 
reader believes are important (to the reader), and thus may reject 
the analysis if threats of interest to the reader were not addresses.

In this case, I believe the informal discussion centered on 
adversaries who wish to inject spam into the Internet e-mail system, 
or who wish to engage in phishing attacks via e-mail. If so, then the 
author merely states that, and proceeds to discuss the motivations 
for such adversaries (what constitutes success for them) and by what 
means they can/do carry out attacks.

With this as background, the author then explains how a proposed set 
of countermeasures prevents such attacks, or makes them harder, etc. 
The reader then evaluates the claims of the author re the 
effectiveness of the proposed countermeasures, given an agreed upon 
threat model.

Steve

---

From: Michael Thomas <[email-masked]>
Date: Fri, 12 Aug 2005 07:24:17 -0700

Harald Tveit Alvestrand wrote:
> I did not hear at any stage Russ claiming that asking for a threat 
> analysis was "invoking process". He was asking for information that 
> would allow him to make up his mind about whether or not to support DKIM 
> becoming a WG.

Then maybe we can call it a "process gate", or something else
entirely. My main point still remains: if you're going to
impose conditions, it seems only fair that the conditions
be understandable by all concerned, and most especially if
multiple people are calling for the same process gate that
they actually _agree_ on what constitutes at least the form
of fulfillment. This conversation here has thus far not
relieved any of my unease that there really is any such
agreement across all those who think this is a good process
gate.

> argument were to be adopted, it should certainly have guidance attached 
> to it.

My feeling is that it's probably a good thing for requirements
drafts to talk about. In the case of a security protocol, it
seems very reasonable that requirements are derived at least in
part from the threats they are intended to address. Non-security
protocols would probably do very well to consider up front what
the security requirements are in any solution space.

		Mike

---

From: Dave Crocker <[email-masked]>
Date: Fri, 12 Aug 2005 07:29:56 -0700

>> proposes to protect.
> its entirety. 

The phrasing that I took from interactions with Russ Housely was: What are the 
threats that DKIM responds to?

That's in the arena of protecting email, albeit stated perhaps with more focus.

The 'threats' document on the dkim web page definitely attends to threats TO 
dkim.  Russ was clear the day after the bof that that was not what he had 
meant.  So, it's a useful bit of writing, but not what we need to focus on 
right now.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 12 Aug 2005 07:34:18 -0700

> IETF, you should quit arguing about whether some glossary defines the 
> term "threat analysis" well enough, and get on with the business of 
> answering the specific questions Russ actually asked. 

We are, on the dkim list.

> I expect that 
> having coherent, meaningful answers to those questions will go a long 
> way toward actually getting a working group chartered, and will also 
> make the group considerably more effective once it is chartered.

So do I.

The dkim list discussion is going through the usual amount of thrashing around 
the topic, but we seem to (finally) have some productive consideration underway.

d/

---

From: "Nicholas Staff" <[email-masked]>
Date: Mon, 15 Aug 2005 21:52:25 -0700

Why doesn't someone just ask Russ what he meant and be done with it? 

-Nick

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Michael Thomas
> Sent: Friday, August 12, 2005 7:24 AM
> To: Harald Tveit Alvestrand
> Cc: Michael Thomas; [email-masked]
> Subject: Re: what is a threat analysis?
>

---

From: Dave Crocker <[email-masked]>
Date: Tue, 16 Aug 2005 00:19:51 -0700

>  Why doesn't someone just ask Russ what he meant and be done with it? 

Someone did:

   1.  <http://www1.ietf.org/mail-archive/web/ietf/current/msg36954.html>

   2.  <http://www1.ietf.org/mail-archive/web/ietf/current/msg36919.html>

   3.  <http://mipassoc.org/pipermail/ietf-dkim/2005q3/000086.html>

d/

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 16 Aug 2005 14:24:59 +0200

Excuse *me* but Mike's note that I was responding to said (in part):

> document what it is that they are demanding.

He explicitly raised the question of hoop jumping, which for me at
least carries a strong implication of pointlessness. That's what
I was responding to.

> given the definition in 2828?

which reads
   "$ threat analysis
        (I) An analysis of the probability of occurrences and consequences
        of damaging actions to a system."

As a glossary definition, that seems admirably clear. For a complex case,
I'd expect to ask some experts for help in determining the type of
threats to be considered in particular. And I would study 3552 carefully,
warts and all. But the bottom line is that this is hard work to get
right - compare the Security Considerations of RFC 3056 with RFC 3964
for example.

     Brian

---

From: Michael Thomas <[email-masked]>
Date: Tue, 16 Aug 2005 06:23:03 -0700

> least carries a strong implication of pointlessness. That's what

So you've fixated on the word "hoop". Fine. Delete it
and the original question still stands.

> As a glossary definition, that seems admirably clear. 

I repeat: could you write a threat analysis based upon the
definition in 2828? That was suggested by somebody as being
adequately defined for my original question.

> For a complex case,

The reason I brought this to this list is because there's
no clarity about what is meant by a "threat analysis", though
it seems to be cropping up more and more in the IETF process
(look ma! no hoops!). If it's going to be part of our process,
then I think its incumbent on those who want to impose the
process to be clear about what they're asking for, and for
that process to not be an idiosyncratic. The waste of time
here is not the process per se, but the work on drafts,
etc, that are not what the person making the demand is asking
for. Do you have an objection to clarity in process?

		Mike

---

From: Ned Freed <[email-masked]>
Date: Tue, 16 Aug 2005 08:02:40 -0700 (PDT)

> least carries a strong implication of pointlessness. That's what

It carries no such connotation for me. www.dictionary.com says the phrase means
"To undergo a rigorous trial or examination." Encarta says it means "to go to
extreme lengths to gain favor with somebody or to carry out somebody's wishes
(informal)". Nothing "pointless" in either definition as far as i can see.

But let's assume that's what Mike meant. So what? It doesn't change the thrust
of his argument in any way, which was and is that there are a variety of ways a
threat analysis can written in this space and we're not sure which one of these
is needed.

> As a glossary definition, that seems admirably clear.

I disagree and think as definitions go it is pretty poor, but let's assume for
the moment it is clear. Again, so what? It's still just a definition, not a
guide for how to write these things. Are you seriously going to claim that this
is all you need to know to write a threat analysis? You say below that getting
these things right is difficult. And in another part of this thread we have
people asserting that some amount of training is needed to write the things.

> right - compare the Security Considerations of RFC 3056 with RFC 3964

All the more reason for those in charge to be quite specific about what it is
they're after, which still hasn't happened. All we get is more fixation on the
words Mike happened to use in one message rather than responding to his issue
with the lack of clarity in what's being asked for.

You really need to start looking at the points people are trying to make rather
than focusing on the words people happen to choose make them. Not everything is
an attack on some dearly-held IETF principle or other, you know.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Tue, 16 Aug 2005 09:53:59 -0700

> All the more reason for those in charge to be quite specific about what 
> it is they're after, which still hasn't happened. All we get is more fixation 
> on the words Mike happened to use in one message rather than responding to his 
> issue with the lack of clarity in what's being asked for.

Minor comment about the "still":

      It's a very short time after the IETF meeting. And it's August.  Area 
Directors are allowed to take a vacation, and even have no Internet 
connectivity during the time.

Not-so-minor comment about guidance:

      I am hard-pressed to believe anyone will argue against the utility of a 
  guidance document. In fact that's why some of us suggested it to EKR during 
IETF week and it is why I made my IETF plenary comment.  Since EKR has a track 
record of producing security-related document for non-security geeks, I'm 
hopeful we will get something useful.

      But most of the IETF works has been done without benefit of such 
guidance documents.  We have relied on the communication of guidance and 
criteria through other means.  So I think the real question is whether 
reasonable, clear, stable criteria are being communicated?  Since the "threat 
analysis" requirement is  newly-imposed on the rest of us, one can expect a 
period of time where the communication is iterative and maybe even looks like 
a negotiation.  The question, for this particular requirement, is whether 
there is a process to converge on clarity and stability for the requirement. 
Although it is early days, I am hopeful. (It helps that the construct involved 
has substantial history within the security community.)

      So the question about the threat analysis requirement is whether an 
effort that has this requirement imposed on it gets the help it needs to 
satisfy it?

      Noting that, in reality, an IETF effort always has quite a few 
requirements imposed on it, my own view is that most early-stage IETF efforts 
gets vastly less help than is needed. The truth of that matter is that we 
mostly rely on a self-forming group to figure things out on its own.  At most, 
we assign a relatively experienced IETFer to help, but I'm not convinced that 
that helps as much as we would like.

      For DKIM, our AD met with the design team at the beginning of IETF week 
and met with me at the end.  He tried to give us pragmatic guidance at the 
beginning of the week.  Some of seem to have understood it and some of us 
clearly did not.  Also, EKR sent an extended note about threat analysis to the 
MASSS mailing list, prior to our BOF, rather than just showing up at the 
microphone and lobbing a bomb at us.  I see two questions leading from the 
fact of our own confusion about the requirement:

           1. Is there something useful we can do on our own about this?

           2. Are we (going to get) assistance from our AD?

The answer to #1 *can be* yes.  As is clear from a number of the threads on 
the DKIM list, the group is not all that cohesive or clear about the 
functional requirements DKIM is intended to satisfy.  An analysis of the 
threats that DKIM is intended to respond to can significantly help resolve the 
confusion, separate from whether that analysis is exactly what we have been 
asked for. That is, getting the group to be clear and cohesive (rough 
consensus) about the goals, motivations, concerns, whatever of DKIM is an 
inherent good. And, indeed, there is some potentially useful discussion 
proceeding.

The answer to #2 is simply yes.  Russ (and, by the way, Sam) are trying to be 
helpful.  As someone with lots of practice complaining about pretty much 
everything, I could express all sorts of wishes about how things could be 
better. However the fact of the matter is that they are looking for ways to 
improve the likelihood of success and they are clearly willing to iterate with 
us to get there.  It is difficult to reasonably ask for more.

That said, I hope we keep the pressure on, to get a document that specifies 
the threat analysis requirement in a way that is both clear and useful for 
working group productivity.

-- 
 
   d/

  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

---

From: Ned Freed <[email-masked]>
Date: Tue, 16 Aug 2005 10:19:46 -0700 (PDT)

> connectivity during the time.

It seems a bit unlikely all of dozen or so relevant people are simultaneously
on vacation, but perhaps that's indeed the case. If so, that's fine, but (a) A
simple statement to that effect would sure have been nice and (b) We should
then refrain from attempting to construct something that's likely to be the
wrong thing in the meantime. (It should be obvious why getting far down a
specific path, only to be told it is the wrong path later, is a bad idea.)

>   guidance document.

And indeed, nobody is making such an argument. Can we finally put this strawman
to bed, or light it on fire, or do whatever it is you do to get rid of
strawmen?

> In fact that's why some of us suggested it to EKR during

> guidance documents.

This has also been pointed out repeatedly.

> We have relied on the communication of guidance and
> reasonable, clear, stable criteria are being communicated?

That's the question in a nutshell. And it is my belief that this is not
happening.

> Since the "threat
> a negotiation.

I would not characterize it as such. An iterative clarifying process might
serve to crystallize management thinking as to what's needed, but ideally it
should not result in the sorts of changes implied by a give-and-take
negotiation process.

> The question, for this particular requirement, is whether

Numerious recent events compell me not to share in your enthusiasm. As John
Klensin has remarked in a separate thread, it now seems that the stock response
to any suggestion or criticism whatsoever is either some sort of strawman or
reductio ad absurdum argument. 

> effort that has this requirement imposed on it gets the help it needs to

I confess to a bit of amusement in seeing that you yourself have characterized
this in terms of "imposed requirements". If I were Brian no doubt i could read
volumes into this choice of words.

As it happens I don't think of this as an imposed requirement, although the way
this has been handled may make it seem that way. Rather, I see the need for
careful analysis as an unavoidable consequence of the specific space we're
working in. The security folks are merely pointing this out.

> gets vastly less help than is needed.

I have believed this ever since some years back the chair of a WG-to-be flew
down from the bay area to literally knock on my door here in LA to ask for help
getting his group chartered. (No, I am not making this up.)

> The truth of that matter is that we

> The answer to #1 *can be* yes.

Of course it can. There is always a chance that if I spin around and shoot a
gun at random I'll hit the target. It just seems like we could do better.

> As is clear from a number of the threads on

Maybe. Around half dozen different documents or document fragments have been
put forward as possible candidate threat analyses. I don't know about anyone
else, but I find it next to impossible to keep them all straight. Nor is it
clear to me which one is the best one to use as a basis for further work.

> us to get there.  It is difficult to reasonably ask for more.

I think I have been very clear as to what I'm asking for. And various people,
most notably Steve Kent, have tried to be helpful. But guidance really has to
come from the people tasked with managing and approving this effort.

> working group productivity.

As I have stated previously, writing and reviewing these sorts of
specifications is really hard work, work I see no point in starting in the
absence of a clear statement of what's actually needed.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Wed, 17 Aug 2005 11:31:11 -0700

> on vacation, but perhaps that's indeed the case. If so, that's fine, but 

And therein lies much of my confusion.

This thread began as a complaint against a particular requirement being 
imposed on a particular pre-working group effort.  It has mostly continued to 
focus on that.  There are only two ADs for the Area under discussion. As far 
as I know, only one has asserted the requirement for a threats analysis and he 
had done it with respect to a single working-group-to-be. That one AD is on 
vacation.

In my view, that one AD is very much trying to be helpful, even as he erected 
this barrier.

(Given the diversity of goals and views on the mass and now the dkim mailing 
lists, his requirement strikes as both timely and useful, for getting group 
consensus about focus, and I am inclined to believe that we really *do* need 
to establish that before chartering rather than seek it after.)

All other discussion about threat analysis concerns trends, preferences, 
needs, eventual requirements and the like.  Hence, they are abstract.  That 
does not at all make them irrelevant or unproductive to pursue, but it strikes 
me that it *does* mean the impact of the discussion is, at best, somewhere in 
the future.  While that certainly means we need to press the security 
community for clear and pragmatic guidance, I do not see how it justifies 
complaining about the need/requirement.

>>       I am hard-pressed to believe anyone will argue against the 
>> utility of a guidance document.
> 
> And indeed, nobody is making such an argument. Can we finally put this 
> strawman to bed, 

Right.  I was merely trying to establish a base of agreement, not to claim you 
had been claiming otherwise.  (Again, I'm just a tad sensitized to diversity 
of views, given the range of directions the DKIM list discussion is going, so 
I want to be careful not to make implicit assumptions.)

> That's the question in a nutshell. And it is my belief that this is not

I posted a note citing the guidance details we've been given by our AD.  I 
(we) would have loved more notice of the requirement as well as a written, 
pedagogical cookbook for satisfying it.

As for the timing of getting the requirement (only a few days before the BOF) 
I'll take that sort of "sudden" imposition at the *beginning* of a group 
process, over the more-typical late-stage surprise any day of the week.

Since I'm the one that has had the most interaction with our AD on this 
requirement for DKIM, I can only say that I view the sequence as reasonable, 
although obviously needing more work.  Yes, it's unfortunate that some of us 
who heard him impart the requirement misunderstood it.  But some of us did not.

>> Since the "threat analysis" requirement is  newly-imposed on the rest of us, one can 
>> expect a  period of time where the communication is iterative and maybe even 
>> looks like a negotiation.
> serve to crystallize management thinking as to what's needed, but 
> ideally it should not result in the sorts of changes implied by a give-and-take
> negotiation process.

If I thought that a) our AD and the security community were entirely clear 
about the requirement as it applies to an Internet standards process, and b) 
those of us who will be doing the working group work were automatically able 
to perform whatever clarified requirement they settle on, then I would agree 
that it is a clarification process, rather than a negotiation.  I think, 
however, that things are rather more fluid.  Hence "negotiation".

That said, we probably don't have to agree on the terminology and I'm only 
commenting because you and I seem to have such different perspectives on this 
and my comment might help, ummmm, clarify the difference.

> Klensin has remarked in a separate thread, it now seems that the stock 
> response to any suggestion or criticism whatsoever is either some sort of 
> strawman or reductio ad absurdum argument.

Somehow, I suspect this is where the real tension lies in this exchange.  And 
I've probably made enough public comments to make clear that this larger, IETF 
management problem is one that you and I (and John and a host of others) agree on.

And it is certainly true that the threat analysis pre-requisite for dkim 
chartering could become an exemplar of this problem.

My point, however, is that my own experience is that... so far... it isn't 
even close.  It feels nothing at all to me like an example of the historical 
(and very serious) problem.

> I confess to a bit of amusement in seeing that you yourself have 
> characterized this in terms of "imposed requirements". If I were Brian no doubt i 
> could read volumes into this choice of words.

You probably won't be surprised to hear that I chose the word carefully.  The 
only surprise might be that I'm not complaining about it.

The reasons are pretty simple:  We are asking an AD to support the activity. 
It is entirely the AD's choice whether to do this. So the question is what he 
needs to be satisfied.  He's imposed this requirement as a basis for gaining 
his support.  This is entirely reasonable, within current IETF rules.  I 
happen to think that it is also reasonable given the highly problematic 
histories (yes, plural) of the domains (yes, plural) dkim is working in and 
the obviously problematic pattern on its mailing list.

It would be a very different matter, indeed, if there were an IETF-wide (or 
even a Security Area-wide) formal rule about all this.  At that point, yes, we 
damn well better get clear, reasonable documentation of how to satisfy it.

> As it happens I don't think of this as an imposed requirement, although 
> the way this has been handled may make it seem that way. Rather, I see the need for

The "imposition" is that it is pre-chartering.

>  But guidance really has to

DKIM is the only, immediate exemplar I know about.  I believe DKIM is in a 
process that is getting the guidance it needs from the AD who is imposing the 
requirement at this stage.  He needs to keep giving it.  I've no doubt he will.

Let me pound this into the ground a bit more: Like many of us, I have quite a 
bit of experience with ADs who are unreasonable and I have a fair amount of 
experience pushing back on them.  What is happening with DKIM, in my view, is 
absolutely NOTHING like those other experiences.

> absence of a clear statement of what's actually needed.

I've noted the comments folks have made, about the effort needed to do a 
thorough threat analysis. However it is not at all clear to me that we are be 
required to produce anything that extensive.

At the moment, we do not even have agreement about the basic threats to which 
we are responding.  I posted some draft text that attempts to do that.  It is 
in a form similar to one I did informally for Russ some months ago, for a 
different specification.  He said it was in the right direction. So I am 
hopeful that my current text, or some other text of a similar style, will also 
be a useful base.

The effort needed to produce that text was not trivial, but it also wasn't large.

-- 
 
   d/

  Dave Crocker

---

From: Michael Thomas <[email-masked]>
Date: Wed, 17 Aug 2005 11:40:45 -0700

> imposed on a particular pre-working group effort. 

No it did not. Stop imputing my motives.

		Mike

---

From: Dave Crocker <[email-masked]>
Date: Wed, 17 Aug 2005 14:47:12 -0700

Michael Thomas wrote:
>> This thread began as a complaint against a particular requirement 
>> being imposed on a particular pre-working group effort. 
> 

Michael,

I readily admit to having no clue as to what your motives are.  I can't 
guess... and I haven't tried.

What I DID do was to characterize the semantics of the original posting that 
you sent, creating this thread:

 > any definition of what a "threat analysis" is.

Whatever your intent, the text you sent complains that the DKIM effort was 
given a requirement but was not given guidance, notably that we were not given 
a definition of the term that describes the requirement.

Yet you were present at the lunch where our area director gave us both the 
requirement and the definition.

That some of us understood what he said and some of us did not is certainly 
important, of course.  It probably at least means we need more/better guidance.

But it does not change the fact that we were given explicit guidance and a 
definition to back it up.

-- 
 
   d/

  Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 19 Aug 2005 16:46:52 +0200

....
> it is

I'm not fixated on his words; I simply pointed out that I didn't make
them up.

I'm not the relevant AD, who is now back from a short vacattion and
has sent additional clarification.

> an attack on some dearly-held IETF principle or other, you know.

I'm not sure where that came from, but I will leave this conversation to
the BOF Chairs and the AD now.

    Brian