Subject: Rough consensus? #425 3.5

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 19 Jan 2005 11:47:25 +0100

Trying to close this item, which is not resolved in the -04 draft:

I believe that the list discussion has converged on very rough consensus 
(Sam and Avri being the people who worry that we're building a DoS attack 
defense that we don't need, but Brian, Scott and John Klensin, at least, 
strongly arguing that we need that mechanism) on the following text, which 
I suggested on Jan 13, replacing the last 3 paragraphs of section 3.4:
------------------------------------------------------
3.5 Decision review

In the case where someone questions a decision of the IAD or the
IAOC, he or she may ask for a formal review of the decision.

The request for review is addressed to the person or body that made
the decision. It is up to that body to decide to make a response,
and on the form of a response.

The IAD is required to respond to requests for a review from the
IAOC, and the IAOC is required to respond to requests for a review
of a decision from the IAB or from the IESG.

If members of the community feel that they are unjustly denied a
response to a request for review, they may ask the IAB or the IESG
to make the request on their behalf.

Answered requests for review and their responses are made public.
-------------------------------------------------------
Can we live with this?

                         Harald

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Margaret Wasserman <[email-masked]>
Date: Wed, 19 Jan 2005 07:12:03 -0500

Is this intended to replace the section on appeals?  In other words, 
will we have this review mechanism _instead_ of the ability to 
actually appeal a decision of the IAOC?

If so, I'm not comfortable with that.  I understand that there needs 
to be some limit on the action that is taken in the event of an 
appeal (we can't undo a signed contract, for instance), but I don't 
understand why we should eliminate the appeal mechanism altogether. 
We already have limits on the actions that can be taken in appeals 
(the IAB or ISOC can't approve a document on an appeal of an IESG 
decision, and no one can up-publish an RFC).  So, why can't we just 
include a clause like that and use the existing appeals mechanism?

Although we haven't had very many of them rise to the formal level, I 
think that a formal appeals process is valuable, as it underscores 
the fact that a group is accountable to the community and it gives a 
person who is unhappy with a decision some leverage to raise the 
issue and get it resolved without recourse to a formal appeal.  So, 
I'd be sorry to see us lose that leverage with the IAOC.

It seems strange, IMO, to be so worried about DoS attacks through the 
appeal process we've been using this process for several years for 
IESG and WG decisions and haven't experienced that sort of problem...

Margaret

At 11:47 AM +0100 1/19/05, Harald Tveit Alvestrand wrote:
>I believe that the list discussion has converged on very rough 
>consensus (Sam and Avri being the people who worry that we're 
>building a DoS attack defense that we don't need, but Brian, Scott 
>and John Klensin, at least, strongly arguing that we need that 
>mechanism) on the following text, which I suggested on Jan 13, 
>replacing the last 3 paragraphs of section 3.4:

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 19 Jan 2005 13:42:23 +0100

Harald Tveit Alvestrand wrote:
> (Sam and Avri being the people who worry that we're building a DoS 
> attack defense that we don't need, but Brian, Scott and John Klensin, at 
> least, strongly arguing that we need that mechanism) on the following 
> text, which I suggested on Jan 13, replacing the last 3 paragraphs of 
> section 3.4:

I can
     Brian

---

From: Scott W Brim <[email-masked]>
Date: Wed, 19 Jan 2005 09:00:26 -0500

On 1/19/2005 05:47, Harald Tveit Alvestrand allegedly wrote:

> consensus (Sam and Avri being the people who worry that we're building 
> a DoS attack defense that we don't need, but Brian, Scott and John 
> Klensin, at least, strongly arguing that we need that mechanism) on 
> the following text, which I suggested on Jan 13, replacing the last 3 
> paragraphs of section 3.4:

I think you should just replace the 3rd from last paragraph (on appeals) 
and keep the last two.  The middle paragraph gives guidance for what 
happens after a review, and I recall the last paragraph was wanted by 
someone for clarification.

If an individual requests a review, receives a response, and in the 
individual's opinion there is a problem, what does the individual do?  
Let's finish the state machine.  Add a sentence -- I suggest we say the 
individual asks the IAB/IESG for "action", where "action" is undefined 
here, but probably means that they make a request for a review themselves. 

I like not requiring that all requests for review be made public, in 
line with allowing IAD/IAOC to shake off DoS attacks.

Putting it all together, I propose:

  3.5 Decision review

  In the case where someone questions a decision of the IAD or the IAOC,
  he or she may ask for a formal review of the decision.

  The request for review is addressed to the person or body that made
  the decision. It is up to that body to decide to make a response, and
  on the form of a response.

  The IAD is required to respond to requests for a review from the IAOC,
  and the IAOC is required to respond to requests for a review of a
  decision from the IAB or from the IESG.

  If the IESG, IAB or the ISOC Board of Trustees find that procedures
  have been violated, they may advise the IAOC, but do not have
  authority to overturn or change a decision of the IAOC.

  If members of the community feel that they are unjustly denied a
  response to a request for review, they may ask the IAB or the IESG to
  make the request on their behalf.  If members of the community receive
  a response to a request for review, and believe that procedures have
  been violated, they may ask the IAB or the IESG to take further
  action.

  The IAOC plays no role in appeals of WG Chair, IESG, or IAB decisions.

---

From: Margaret Wasserman <[email-masked]>
Date: Wed, 19 Jan 2005 09:38:01 -0500

Okay, Harald indicated to me privately that I should be more specific 
about my objections to the current wording and offer some 
alternative, so here goes...

I do not object to the use of the term "review" instead of "appeal".

However, I do object to the current wording proposed by Harald for two reasons:

(1) I think that there should be an effective way for members of the 
community (not just members of the I*) to question the decisions of 
the IAOC and receive some response.  If a wrong decision was made, it 
may not always be possible to reverse the decisions of the IAOC 
(contracts, etc.), but it could be possible to consider the situation 
and create new rules or guidelines to prevent a similar situation 
from occurring in the future.

(2) I don't think that the mechanism is appropriately specified.  If 
we used the appeals mechanism in 2026, there is already a definition 
and some practical history.  I understand there is some objection to 
using that mechanism, but if we want to invent a new one, then I 
think we need to specify it so that a member of he community (not 
just I* members) could actually use it.

Here is a stab at some alternative wording...

In the case where someone believes that a decision of the IAD or the IAOC
he or she may ask for a formal review of the decision by sending e-mail
to the IAOC chair.  The request for review is addressed to the IAOC, and
should include details of the decision that is being reviewed, an explanation
of why the decision should be reviewed, and a suggestion for what action
should be taken to rectify the situation.  All requests for review will be
published publicly on the IAOC web site.

It is up to the IAOC to determine what level of formal review is required
based on the specifics of the request for review.  However, the IAOC is
expected to make some public response to a request for review within
90 days of the request, indicating the findings of the review.

If the IAOC finds that an incorrect or unfair decision was made, it will be
up to  the IAOC to decide what type of action, if any, makes sense as a
result.  In many cases, it may not be possible or practical to change the
decision (due to signed contracts or business implications), but the IAOC
may choose to make changes to its policies or practices to avoid similar
mistakes in the future or may simply wish to acknowledge that  a mistake
was made and learn from the error.

If a person believes that his or her request for review was not handled
properly or fairly by the IAOC, he or she may escalate the request to the
IESG by sending mail to the IETF chair.  The IESG will consider the IAOC's
response and may take one of three actions:  (1) indicate that the decision
was properly reviewed and the IAOC's response was fair, (2) state why
the review was improper or unfair and offer advice to the IAOC
regarding what type of response or action would be justified, or (3)
determine that there is a problem with the rules governing the IAOC and
propose changes to this document (or other BCPs) to the IETF
community.  In no case, may the IESG reverse or change a decision of
the IAOC or make a direct change to the IAOC's operating policies.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 19 Jan 2005 17:02:58 +0100

Hmm. I think this bothers me a lot unless
a) unsuccessful bidders and their agents
and
b) unsuccessful job candidates

are explicitly excluded. Otherwise, every time
the IASA awards a contract or hires somebody, they are
exposed to public attack by the unsuccessful.

    Brian

Margaret Wasserman wrote:
> about my objections to the current wording and offer some alternative, 
> so here goes...
> 
> community (not just members of the I*) to question the decisions of the 
> IAOC and receive some response.  If a wrong decision was made, it may 
> not always be possible to reverse the decisions of the IAOC (contracts, 
> etc.), but it could be possible to consider the situation and create new 
> rules or guidelines to prevent a similar situation from occurring in the 
> future.
> 
> (2) I don't think that the mechanism is appropriately specified.  If we 
> used the appeals mechanism in 2026, there is already a definition and 
> some practical history.  I understand there is some objection to using 
> that mechanism, but if we want to invent a new one, then I think we need 
> to specify it so that a member of he community (not just I* members) 
> could actually use it.
>

---

From: Steve Crocker <[email-masked]>
Date: Wed, 19 Jan 2005 11:03:17 -0500

I have not been paying close attention to the debate over this section 
of the BCP before, so I may be covering a point that's been made before.

I think there will necessarily be a mixture of formal and informal 
processes at work once the IASA is in operation.  The IAOC is intended 
to be at once both independent of the day to day operation of the IESG 
and IAB so it can relieve them of the burden of managing the details on 
a day to day basis and at the same time responsive to the community.  No 
matter what formal mechanisms are put in place, the IAOC needs to keep 
its eyes and ears open to understand how well it is serving the 
community's needs.  Inevitably, there will be some decisions or actions 
that some will complain about.  When things are working well, the IAOC 
will find useful ways of respoding to such complaints, either by 
explaining the situation more fully, adjusting its decisions and 
actions, or, when the complaints simply represent a small minority with 
an unresolvable difference of opinion, standing firm.

Formal means for resolving disputes do need to exist.  I haven't studied 
Margaret's formulation carefully, but on first glance it looks fine to 
me.  Other formulations will also work.  As we all know, if there are 
very many formal disputes, then something larger is probably broken and 
needs to be fixed.  I'm confident the community will raise the noise 
level in that case and we'll be re-engaged in a full, open, community 
review of the IASA, IAOC, etc.

The bottom line on this for me is that everyone should expect the IAOC 
to report regularly and substantively to the community and to listen 
carefully to the community, and that form of communication will be the 
primary safety valve.

Steve

---

From: Sam Hartman <[email-masked]>
Date: Wed, 19 Jan 2005 11:26:13 -0500

I prefer Margaret's wording but could live with Harld's wording.

---

From: Margaret Wasserman <[email-masked]>
Date: Wed, 19 Jan 2005 11:27:55 -0500

>exposed to public attack by the unsuccessful.

In general, people do not choose to raise a public stink when they 
are not hired for a job or are not chosen as a contractor, perhaps 
due to a lack of desire to air those facts in public.

But, if we do have someone who wants to raise a stink and/or waste 
IETF resources over that type of issue, I don't think that the lack 
of a formal review process would stop them.  In fact, the lack of a 
reasonable way of dealing with this type of disagreement within the 
IETF context might lead people to take legal action, which would be 
even worse (more time consuming, expensive, damaging).

Margaret

---

From: Michael StJohns <[email-masked]>
Date: Wed, 19 Jan 2005 11:40:42 -0500

Hi Harald et al -

I apologize for chiming in on this so late, but I had hopes it would get 
worked out without me pushing over apple carts.

I can't support this and I recommend deleting this section in its entirety.

My cut on this:

The decisions of the IAD should be subject to review (and in some cases 
ratification) ONLY by the IAOC.

The decisions of the IAOC should not be subject to further review by the 
IETF at large.  The proper venue for expressing tangible displeasure with a 
decision is during the appointment and reappointment process.  (Note, I'm 
not precluding pre-decision comment by the community at large, and I 
encourage the IAOC to seek such comment where appropriate but once the 
decision is made its time to stop whining and get on with things)

The decisions of the IAOC must be publicly documented to include voting 
records for each formal action.

The IAOC and IAD must accept public or private comment but there is no 
requirement to either respond or comment on such missives.

The IAOC and IAD should not be subject to the IETF appeals process.  The 
appropriate venue for egregious enough complaints on the commercial side is 
the legal system or the recall process.

My reasoning:

The IAD and IAOC are making commercial (as opposed to standards) decisions 
and the result of that may be contracts or other commercial relationships. 
Its inappropriate in the extreme to insert a third (or fourth or fifth) 
party into that relationship.

The IAD/IAOC relationship is going to be somewhat one of employee/employer 
and its inappropriate to insert external parties into that relationship.

The documentation requirement is so that when the appointment process 
happens there will be some audit trail as to who did what to whom.

The IETF appeals process is not appropriate for a commercial action.  A 
standards action may adversely affect competitors across a broad spectrum 
of companies.  This commercial action only affects the bidders or winners.

Please, let's get the IETF out of the metaphorical administrative back seat 
and get them back to doing what they do well - technology.

---

From: Leslie Daigle <[email-masked]>
Date: Wed, 19 Jan 2005 18:21:15 -0500

Interesting...

To the extent that the IAD and IAOC are dealing with
decisions about implementing requirements, I agree.

To the extent that the IAD and IAOC are applying judgement
to interpret the "best needs of the IETF" (i.e., determining
those requirements), I disagree.  I think it's a little
heavy-handed to have to instigate a recall procedure if the
IAD (or IAOC) seem not to have heard the *community's* requirements
for meeting location.

So, (how) can we make the distinction without creating a
decision tree of epic proportions?

Leslie.

Michael StJohns wrote:
> IETF at large.  The proper venue for expressing tangible displeasure 
> with a decision is during the appointment and reappointment process.  
> (Note, I'm not precluding pre-decision comment by the community at 
> large, and I encourage the IAOC to seek such comment where appropriate 
> but once the decision is made its time to stop whining and get on with 
> things)
> 
> appropriate venue for egregious enough complaints on the commercial side 
> is the legal system or the recall process.
> 
> My reasoning:
> 
> The IAD and IAOC are making commercial (as opposed to standards) 
> decisions and the result of that may be contracts or other commercial 
> relationships. Its inappropriate in the extreme to insert a third (or 
> fourth or fifth) party into that relationship.
> 
> The IAD/IAOC relationship is going to be somewhat one of 
> employee/employer and its inappropriate to insert external parties into 
> that relationship.
> 
> standards action may adversely affect competitors across a broad 
> spectrum of companies.  This commercial action only affects the bidders 
> or winners.
> 
> 
> Please, let's get the IETF out of the metaphorical administrative back 
> seat and get them back to doing what they do well - technology.
> 
>

---

From: Michael StJohns <[email-masked]>
Date: Thu, 20 Jan 2005 00:00:36 -0500

>decision tree of epic proportions?

Two things I learned in the military:  1) Pick good subordinates, give them 
a clear goal and get out of the way. 2) Systems only work when authority 
and responsibility are balanced.

What does that mean for the IAOC/IAD?

1)  It means that we need to pick good people and then trust they will do 
the right thing.  We shouldn't attempt or expect to be able to manage them 
on a retail basis (aka micro management).  We shouldn't be second guessing 
them.  If we can't trust them we need to get rid of them (the nuclear option).

2) The problem with giving the IETF (for a loosely defined value of IETF) 
direct review authority over the IAOC/IAD is that there isn't a concomitant 
assignment of responsibility.  For the selection of the IAOC, we can hope 
the "members" (e.g. through the nomcom/IAB/IESG) will work in their own 
self interest (which translates accepting responsibility for bad selections 
for the committee).  For the day to day business of the IAOC there really 
isn't a way to ensure the people who would have the authority (e.g. to 
force a review of the decisions, to second guess, etc, to cause various 
disruptions) would also be saddled with the results of their bad 
decisions.  (Membership organizations at least have the benefit that the 
member pays something to belong and so takes financial responsibility for 
the org - maybe we require someone to post a bond as part of forcing a review?)

With respect to selecting where we go - well, that comes under the heading 
of setting clear goals, but then getting out of the way.

It all comes down to who gets to decide and that's what's being argued 
here.  You?  Me?  The IAOC?  The IAD?  Everyone who ever read an email on 
the IETF mailing list?  The IAB?  The IESG? *sigh*  If it's the IETF then 
lets not go through the farce of appointing an IAOC.  If its the IAB or 
IESG why do we need an IAOC?

For the specific issue of the meeting location I think it needs to be the 
IAD with the advice and consent of the IAOC, with input from the IAB, IESG 
and the community.    And I think that's how most decisions will need to be 
made if we want the IAOC to be an effective organization.

If you (general plural) really feel this section needs to stand I think you 
need to address at least two issues and narrow them substantially:  who has 
standing to ask for a formal review? and on what specific issues can the 
IAOC/IAD be reviewed?   If you can't this section needs to go.

For standing you've currently got "someone"... *sigh*
For issues you've got "a decision of the IAD or IAOC" ... *sigh*  At a 
minimum, I'd explicitly prohibit review of the IADs actions by any body 
except the IAOC - direct the review to the IAOC only.

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Thu, 20 Jan 2005 02:03:58 +0100

>decision tree of epic proportions?

Just say that they are to consult the IETF when they do not feel sure about 
the "best needs of the IETF". A recall procedure would probably not be 
called the first time, even if the issue is important (preserving 
stability), but can be called even on a small issue if they repeatedly do 
not consult the IETF when a disagreement/uncertainity is obvious.

So, the recall procedure is not on a possibly disputed case - the dispute 
would harm the whole IETF - but on a repeated Management poor practice 
where accumulated displeasure will probably make the case less disputed. It 
also permits IETF to vote warnings.

jfc

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 20 Jan 2005 10:10:57 +0100

Michael StJohns wrote:
> At a minimum, I'd explicitly prohibit review of the IADs actions 
 > by any body except the IAOC - direct the review to the IAOC only.

I think this is correct, managerially. That way the IAD knows who
his or her boss is, and that is important. But there is nothing
in draft-ietf-iasa-bcp-04 suggesting that anybody except the IAOC
can review the IAD's actions, and the stuff about decision review
in section 3.4 is explicitly about the IAOC. It's also very clear
from sections 3.1 and 3.2 that IAD reports to the IAOC. So I
see no need for a text change.

    Brian

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 20 Jan 2005 12:25:05 +0100

--On torsdag, januar 20, 2005 00:00:36 -0500 Michael StJohns 
<[email-masked]> wrote:

> If you (general plural) really feel this section needs to stand I think
> you need to address at least two issues and narrow them substantially:
> who has standing to ask for a formal review? and on what specific issues
> can the IAOC/IAD be reviewed?   If you can't this section needs to go.
>

Mike,

the text that I started off this thread with had:

- Standing: Anyone can *ask*, but only the IESG and IAB can demand an 
answer.
- Issues: Any decision. I think this is right - enumerating the issues that 
can be reviewed is simply the wrong level of detail for this document.
- Who reviews: IAD reviews IAD decisions, IAOC reviews IAOC decisions. The 
read "review" as "explain why", not "I override you".

The fact that IAOC has oversight of the IAD is actually not mentioned here; 
it's mentioned in enough other places.

Of course, Margaret's proposed text is different. But I believe you were 
starting off by objecting to the one I wrote - and I'm not sure I 
understand how you interpreted what I wrote.

                 Harald

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 20 Jan 2005 14:06:40 +0100

Margaret Wasserman wrote:
> In general, people do not choose to raise a public stink when they are 
> not hired for a job or are not chosen as a contractor, perhaps due to a 
> lack of desire to air those facts in public.
> 
> But, if we do have someone who wants to raise a stink and/or waste IETF 
> resources over that type of issue, I don't think that the lack of a 
> formal review process would stop them.  In fact, the lack of a 
> reasonable way of dealing with this type of disagreement within the IETF 
> context might lead people to take legal action, which would be even 
> worse (more time consuming, expensive, damaging).

I think that is not really a concern. If someone has a grievance that
is serious enough for them to hire a lawyer to make a complaint, no
words in an RFC will stop them. But the right words in an RFC will allow
the IAD to say:

"If you have any complaints, please contact the IAOC."
[I believe the current words allow this.]

and the IAOC to say, very rapidly,

"We've looked at your complaint and concluded that it is out of scope
of RFC xxxx."

I don't think the current words allow that, because they are very broad.
We could fix this concern quite easily, by adding
    Contract awards and employment decisions may not be appealed
    using this process.

        Brian

---

From: Michael StJohns <[email-masked]>
Date: Thu, 20 Jan 2005 10:38:35 -0500

> and on the form of a response.

Yes, but the stuff about decision review in Harald's text is pretty clear 
that its the "person" or "body" that's subject to review.

At 04:10 AM 1/20/2005, Brian E Carpenter wrote:
>  But there is nothing
>in section 3.4 is explicitly about the IAOC.

>  It's also very clear

---

From: [email-masked]
Date: Thu, 20 Jan 2005 11:01:57 -0500

Hi,

In general I am happy with this formulation.  Some comments below.

either need an extra phrase here or need to change 'believes that' to 
something like
'objects to'

or add the phrase: 'runs counter to the provisions of RFC ..." or "... 
provisions of this BCP ..."

or maybe: 'is not in the best interests of the IETF'

> community.

In this formulation, I would like to see one further possible step

or (4) refer the issue to the ISOC BoT for further review.

> In no case, may the IESG reverse or change a decision of

a.

---

From: Michael StJohns <[email-masked]>
Date: Thu, 20 Jan 2005 11:09:34 -0500

At 06:25 AM 1/20/2005, Harald Tveit Alvestrand wrote:

>- Standing: Anyone can *ask*, but only the IESG and IAB can demand an answer.

Ok - this has two interesting and potential annoying outcomes.  First 
outcome is that the IAOC just ignores any non-IESG/IAB query, second 
outcome is that the ignoree petitions the IAB/IESG to get their point to be 
made and then appeals when the IAB/IESG says no.

>- Issues: Any decision. I think this is right - enumerating the issues 
>that can be reviewed is simply the wrong level of detail for this document.

Let's try it from the other end.  Try and list 3 things that can/should be 
amenable to review and why.  I think you're going to find you can't or that 
you're listing things that are so trivial in the extreme that they really 
shouldn't be reviewable.  If you can list 3 and can show there are 
potentially a lot more I'll shut up on this point.

>read "review" as "explain why", not "I override you".

Sorry - that's not what "review" means.  In the context, its 2 : to examine 
or study again; especially : to reexamine judicially.   A body does not 
generally "review" its own decisions.  It "reconsiders" them.  If you mean 
explain, then say explain.

>The fact that IAOC has oversight of the IAD is actually not mentioned 
>here; it's mentioned in enough other places.
>

Yes - I'm looking at your text.  That was the last consensus text not hers.

>                 Harald

On the general topic of reconsider vs review (and partially on Margaret's 
note).  If you want to provide a mechanism for "someone" to petition for 
reconsideration of a decision that mechanism needs to require the "someone" 
to a) describe why they believe the decision was made in error, b) a 
proposed remedy (might be as simple as say "yes" instead of "no" but may be 
much more complex.).  It also needs to set a sunset  or statute of 
limitations on when these petitions can be submitted.

---

From: Sam Hartman <[email-masked]>
Date: Thu, 20 Jan 2005 14:43:51 -0500

>>>>> "Brian" == Brian E Carpenter <[email-masked]> writes:

    Brian> I think that is not really a concern. If someone has a
    Brian> grievance that is serious enough for them to hire a lawyer
    Brian> to make a complaint, no words in an RFC will stop them. But
    Brian> the right words in an RFC will allow the IAD to say:

    Brian> "If you have any complaints, please contact the IAOC."  [I
    Brian> believe the current words allow this.]

    Brian> and the IAOC to say, very rapidly,

    Brian> "We've looked at your complaint and concluded that it is
    Brian> out of scope of RFC xxxx."

    Brian> I don't think the current words allow that, because they
    Brian> are very broad.  We could fix this concern quite easily, by
    Brian> adding Contract awards and employment decisions may not be
    Brian> appealed using this process.

I could not support that change.  I think it is entirely reasonable to
review a contract decision and conclude that procedures need to be
changed.  I think doing so regularly would be very problematic.

--Sam

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 21 Jan 2005 11:47:24 +0100

> changed.  I think doing so regularly would be very problematic.

Reviewing procedures is fine. Reviewing specific awards isn't, IMHO,
which is all I intended my words to exclude.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 21 Jan 2005 11:49:04 +0100

Michael StJohns wrote:
> Yes, but the stuff about decision review in Harald's text is pretty 
> clear that its the "person" or "body" that's subject to review.

Mike, yes, this text is problematic. But it isn't in the draft.
I don't find the text in the actual -04 draft anywhere near
as problematic, apart from the point I am arguing with Sam in
my previous note.

    Brian

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 21 Jan 2005 12:00:38 +0100

> as problematic, apart from the point I am arguing with Sam in

I'm afraid I'm beating a dead horse even deader, but.....

> Answered requests for review and their responses are made public.

What part of that text makes it "pretty clear that its the "person" or 
"body" that's subject to review", as Mike says?

                         Harald

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 21 Jan 2005 12:44:46 +0100

> "body" that's subject to review", as Mike says?

That's how I read the 2nd sentence.

But I can live with this text, even though I prefer what is in the -04
draft. The point about exempting contract awards and employment
decisions is still on my list, though.

    Brian

---

From: Sam Hartman <[email-masked]>
Date: Fri, 21 Jan 2005 11:30:45 -0500

>>>>> "Brian" == Brian E Carpenter <[email-masked]> writes:

    Brian> Reviewing procedures is fine. Reviewing specific awards
    Brian> isn't, IMHO, which is all I intended my words to exclude.

Attempting to undo a specific award once things are signed (or
delaying signing) is generally unacceptable.  Reviewing a specific
award to come to conclusions like "we failed to follow our
procedures," is definitely problematic but IMHO sometimes necessary.
Such reviews need to be handled with great care: some of the data used
to make the decision may not be available to the reviewing body and
much of the data must not become public.  Also, if you conclude that
you did follow the wrong procedures in a specific incident but are
stuck with the contract because it is already signed, that creates all
sorts of bad feelings and potential liability.

--Sam

---

From: Leslie Daigle <[email-masked]>
Date: Fri, 21 Jan 2005 19:37:27 -0500

Following up the point I made in response to Mike St.Johns
a couple days ago, I went back through the document to see if/how
it distinguishes between being adequately responsive and
accountable to the community, from having appropriate
chains of accountability for contractual purposes (and
no micromanagement of the business affairs of the IASA).

It seems to me that we should:

1/  Change this section:

3.3 Relationship of the IAOC to Existing IETF Leadership

to "3.6 Responsiveness of IASA to the IETF"

and include the original text plus Harald's text adjusted to
be about the general processes.  And a point about objective
process metrics.

2/  Add a section (3.5) specifically about business decisions --
which, as Mike St.Johns pointed out, should remain within
the IAD/IAOC.

That would make:

3.5  Business Decisions

Decisions made by the IAD in the course of carrying out IASA business
activities are subject to review by the IAOC.

3.6  Responsiveness of IASA to the IETF

The IAOC is directly accountable to the IETF  community for the
performance of the IASA.  However, the nature of the IAOC's work
involves  treating the IESG and IAB as major internal  customers of the
administrative support services.  The IAOC and the IAD should not
consider their  work successful unless the IESG and IAB are also
satisfied with the administrative support that the  IETF is receiving.
In order to achieve this, the IASA should ensure there are
reported objective performance metrics for all IETF process
supporting activities.

In the case where someone questions an action of the IAD or the
IAOC in meeting the IETF requirements as outlined above, he or she
may ask for a formal review of the decision.

Leslie.

Leslie Daigle wrote:
>> I apologize for chiming in on this so late, but I had hopes it would 
>> get worked out without me pushing over apple carts.
>>
>> The decisions of the IAD should be subject to review (and in some 
>> cases ratification) ONLY by the IAOC.
>>
>> The decisions of the IAOC should not be subject to further review by 
>> the IETF at large.  The proper venue for expressing tangible 
>> displeasure with a decision is during the appointment and 
>> reappointment process.  (Note, I'm not precluding pre-decision comment 
>> by the community at large, and I encourage the IAOC to seek such 
>> comment where appropriate but once the decision is made its time to 
>> stop whining and get on with things)
>>
>> The decisions of the IAOC must be publicly documented to include 
>> voting records for each formal action.
>>
>> The IAOC and IAD should not be subject to the IETF appeals process.  
>> The appropriate venue for egregious enough complaints on the 
>> commercial side is the legal system or the recall process.
>>
>> My reasoning:
>>
>> employee/employer and its inappropriate to insert external parties 
>> into that relationship.
>>
>> The IETF appeals process is not appropriate for a commercial action.  
>> A standards action may adversely affect competitors across a broad 
>> spectrum of companies.  This commercial action only affects the 
>> bidders or winners.
>>
>>

---

From: John C Klensin <[email-masked]>
Date: Sat, 22 Jan 2005 09:43:39 -0500

Leslie,

I think this is a huge improvement, and a large step in the
right direction.  Two observations:

(i) In the revised 3.5, it would be good to get a slightly
better handle/ definition on what is, or is not, a "business
decision".  Since the IAD and IAOC are ultimately all about
business decisions (and management decisions that are arguably a
subset of management decisions), an IAOC acting in poor faith
could easily argue that any decision fell within that category.
I'd consider saying "contractual or personnel decisions", but
don't know if that is sufficient.  _However_, in practice, if we
couldn't find satisfactory text, it probably don't make much
difference: an IAOC that resorts to that sort of hair-splitting
would probably need a much more drastic course correction than
any quibbling about the text of the BCP would provide.

(ii) In deference to one of the points Mike made, I think the
first "review" paragraph could usefully be modified to read
"...ask for a formal review (including reconsideration if that
is appropriate) of the decision."   I don't think that needs to
be repeated further on, but will leave that to editorial
judgment.  Another way to accomplish the same thing, also
subject to editorial preference, would be to add a sentence
toward the bottom, that indicates that outcome of a review could
include either a decision to do things differently in the future
or reconsideration, if it is not an already-committed business
decision, of the pending action.

    john

---

From: [email-masked]
Date: Sat, 22 Jan 2005 11:51:54 -0500

Hi,

> to make the request on their behalf.

I still find the lack of direct accountability to the IETF community 
problematic, though I do realize that this does leave two options open 
to the community in the case that the IAB or IESG does not decide to 
honor someone's request for a review:

- Protest on the IETF list and at plenary and review in the court of 
public opinion; i.e resort to public confrontation.
- Appeal of a decision by the IESG to not ask the IAOC for a review.

I am also assuming that since these appeals would always be procedural 
they would always be appealable through to the ISOC BoT.  So while a 
bit contorted, there is a path for accountability for the public.

> and on the form of a response.

I do not understand what happens if the IESG/IAB do ask for a review 
and the IAOC gives what they consider an inadequate response.  Is the 
only recourse at that point to initiate a recall? Or to ask the ISOC 
BoT to fire the IAD?
I am troubled that at the end of the day recall is the only way to 
redress an error that those in office may commit.  Since this is such a 
large hammer to use, I expect that it will leave the IAOC and IAD free 
to act with relative impunity.

I know we will be reviewing the procedures over the years, but 
initiating the restructuring without proper mechanisms for 
accountability seem, to me, to be a mistake.

a.

---

From: Michael StJohns <[email-masked]>
Date: Sat, 22 Jan 2005 23:52:40 -0500

John/Leslie et al - this is a good improvement, and Leslie's 3.5 now reads 
in a way I can support.  3.6 still has some sticking points.

After the last round of comments I went away and thought and came up with 
the following:

There are three separate things that I think were meant by the original 3.5 
phrase "review".  They were IAOC review of IAD decisions, IAOC 
reconsideration of its own actions and public review of the IAOC and IAD 
actions.

"Review" above I took to have a meaning in line with "judicial 
reexamination" with all the nuances that follow from that. In other words, 
review can result in a revision of the action (i.e. "overturned").

Considering IAOC review of IAD decisions, this is probably proper and not 
overly burdensome to the process, and probably the most appropriate check 
and balance on the IAD.  Its the latter two that are sticky.

The questions I still have that none of the language has answered:

What's the goal of the reconsideration or review?  E.g. can this action 
result in the IAOC or IAD reversing or revising a decision?  If not, why do it?

What's the statute of limitations for a demand for reconsideration or 
review? Is an action or decision subject to reexamination 6 months after 
the decision?  Does the IAD or IAOC need to announce such decisions with 
enough of a waiting period that the reexamination process can take place 
before the decision becomes final?

What is the exact set of decisions subject to this 
process?  Everything?  Meeting sites?  Meeting fees?  A delay in 
publication of an RFC?  Cost of paper needed to support an IETF 
meeting?  The date of the public meeting?  Etc.  If we can't come up with a 
description of this that isn't "everything" I'm very concerned, but I'm 
having problems figuring out what smaller set is appropriate.

Who gets to kick this process into starting (e.g. who gets to file a 
complaint)?

I'm going to suggest this section still needs radical reworking.  And, 
since we pay more attention to people who suggest solutions to problems I'm 
going to suggest some approaches that may make sense.  (Note - I'm using 
"semi-annual" here - could me annual, could be monthly)

"The normal channel for public review of the actions of the IAOC and the 
IAD is during the semi-annual public meeting.  Beginning 60 days prior to 
the meeting and ending 30 days prior the IAD will accept proposals for 
public hearings on past activities of the IAOC.  The IAD in consultation 
with the IAOC, the IESG and the IAB will select no more than 3 topics for 
hearing and will publish such list no later than 21 days prior to the 
meeting including as well those topics not selected.   Each topic will be 
heard during a 2 hour (pick a length) session.  Anyone may submit questions 
for the IAOC and IAD on the selected topic, and the IAD will select no more 
than 10 to provide written answers no later than 5 days prior to the 
meeting.  The complete list of submitted questions will be published at 
that time as well.  Anyone may request time to speak during a session and 
will make that request via email no later than 5 days prior to the 
meeting.  No more than 75% of the session time will be allocated to ...."

(OK - I'm getting way too detailed here for a comment - but you see where 
I'm going)

"The IAD shall advise the IAOC of and shall require consent from them for 
any decision which commits the IETF to an expenditure of funds.  The IAOC 
may from time to time specify other decisions for which its approval shall 
be required.  A decision for which consent is given by the IAOC is not 
subject to further review or revision by the IAOC.  A decision made by the 
IAD shall be considered final and not reviewable by the IAOC 30 days after 
the IAOC is informed of such decision.  Such notifications shall be placed 
on the web site once the 30 days expires or after the IAOC ratifies the 
related decision."

"The IAOC and IAD shall not be required to reconsider their decisions once 
made.  However, IAOC and IAD shall request public comment for those 
decisions which will have a major change in the visible support available 
to the IETF and shall request such comments beginning at least 45 days 
prior to the date of the decision and ending no later than 7 days 
prior.  All such comments shall be public and will be made available on the 
IAOC web site."

"The actual decision of where to hold any individual IETF meeting is not 
subject to the above process as the complex relationship between timing, 
budget, hosting proposals, and site availability does not lend itself well 
to public comment.  However, the IAOC will solicit feedback on selected 
sites, proposals for future sites, needs of the participants, and general 
commentary on the balance of meeting sites across the various areas of the 
globe."

"Contractual issues by the ISOC on behalf of the IAOC/IAD/IETF are also not 
subject to the above process due to the legal constraints of the 
contracting process.  The IAOC/IAD shall request public input for any 
contemplated solicitation for IETF support services prior to the issuance 
of such solicitation.  [Are there any classes of this where asking the 
public might not be appropriate?]"

As you can see, I'm a big fan of getting input up front, but much less so 
of post-decision whining. I also can't figure out a reasonable way of 
constraining the "public" with respect to getting the IAD/IAOC to change 
their mind.  This provides a mechanism post-decision to provide the needed 
feedback, a pre-decision mechanism to solicit input without turning the 
IETF into a monkey on the back of the IAOC.

  I think the appropriate venue for IAD/IAOC public review is during the 
public meeting.  State and local governments have used this to good effect 
over the years.  Note that I also am suggesting a much more structure bitch 
session - the structure we use at the IETF plenaries isn't going to be 
efficient enough or documented enough to provide the level of oversight 
that people seem to be asking for.

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sun, 23 Jan 2005 17:10:20 +0100

Michael,
I see you come - in still a too detailed manner - the real life way I 
suggested. That IAOC and IAD are under the obligation to ask the IETF 
approval for their decisions. But that they may decide there is a consensus 
(there might be a formula coined to that end, they should use when signing 
a decision so it is clear). That IETF may decide any time to blame or 
remove them for abusing that formula. In one big case or in regular way.

Considering each "decision" as a separate case will lead to easy DoS 
blocking of the IETF. Considering the way they take decisions (using 
challenged decision as examples to document it), makes only one case and 
may be watched by independent watchdogs. These watchdogs could make a 
report at each IETF meetings. If a positive report is not challenged, the 
way the IAOC and IAD managed during that period is deemed approved (not 
what they did, but the way they did it).
jfc

At 05:52 23/01/2005, Michael StJohns wrote:

>There are three separate things that I think were meant by the original 
>3.5 phrase "review".  They were IAOC review of IAD decisions, IAOC 
>meeting?  The date of the public meeting?  Etc.  If we can't come up with 
>a description of this that isn't "everything" I'm very concerned, but I'm 
>since we pay more attention to people who suggest solutions to problems 
>I'm going to suggest some approaches that may make sense.  (Note - I'm 
>using "semi-annual" here - could me annual, could be monthly)
>
>heard during a 2 hour (pick a length) session.  Anyone may submit 
>questions for the IAOC and IAD on the selected topic, and the IAD will 
>select no more than 10 to provide written answers no later than 5 days 
>prior to the meeting.  The complete list of submitted questions will be 
>published at that time as well.  Anyone may request time to speak during a 
>session and will make that request via email no later than 5 days prior to 
>the meeting.  No more than 75% of the session time will be allocated to ...."
>
>prior.  All such comments shall be public and will be made available on 
>the IAOC web site."
>
>"Contractual issues by the ISOC on behalf of the IAOC/IAD/IETF are also 
>not subject to the above process due to the legal constraints of the 
> public meeting.  State and local governments have used this to good 
> effect over the years.  Note that I also am suggesting a much more 
> structure bitch session - the structure we use at the IETF plenaries 
> isn't going to be efficient enough or documented enough to provide the 
> level of oversight that people seem to be asking for.
>
>

---

From: John C Klensin <[email-masked]>
Date: Sun, 23 Jan 2005 11:50:37 -0500

--On Saturday, 22 January, 2005 23:52 -0500 Michael StJohns
<[email-masked]> wrote:

> John/Leslie et al - this is a good improvement, and Leslie's
> 3.5 now reads in a way I can support.  3.6 still has some
> sticking points.
> 
> After the last round of comments I went away and thought and
> came up with the following:
> 
> There are three separate things that I think were meant by the
> original 3.5 phrase "review".  They were IAOC review of IAD
> decisions, IAOC reconsideration of its own actions and public
> review of the IAOC and IAD actions.
>....

Mike,

I've read, and reread, your note several times and want to
comment on it from a slightly higher level.   First, and most
important, I think we all need to understand and accept that
there is real disagreement in the community on this subject.  

If I have correctly understood, you are at, or close to, one
extreme.  Your view, as I understand it, is that the IASA should
be treated as a semi-independent body with periodic "reviews" of
its performance and little or no ability of the IETF community
(including the IAB and IESG) to interfere in any individual
decisions either before or after the fact.  "Interfere" might
include micromanagement, specific direction about decisions, and
the ability to overturn decisions or even to force the IAOC to
reconsider decisions.   And you believe that, without that much
autonomy, the IASA will simply be unable to function, or at
least function efficiently.

My personal, historical, position is close to that.  I believe
that the IASA needs considerable autonomy and to be treated by
the IETF largely as an entity.  While I have modified my
positions in the hope of reaching some sort of consensus (see
below), I have argued from time to time that the majority of the
IAOC membership should come from outside the IETF process, for
the presence of the IETF and IAB Chairs on the IAOC only as
observers or liaisons, for "reviews" of performance only and no
reviews of individual decisions, and for the ability of the IETF
community to "fire the IAOC" --remove them as a group-- rather
than relying on recall of individuals or incremental
nomcom-based phaseout.   I've stated each of those positions
separately, usually in terms of either preventing interference
or abuse by those who are not chosen for their skills in the
types of management that IASA will require or preventing the
appearance of conflicts of interest between the administrative
entity and the standards process.   I have now realized, after
reading your notes, that they are really all part of the same
underlying theme, which is that the IAOC and IAD must be able to
function without concern about having individual decisions
--contractual or not-- regularly second-guessed.

At the other extreme are those who believe that the IASA is
really a functional subsidiary of the IETF, that all (or as many
as possible) of its decisions should be able to be overturned by
the IESG (or IAB, or both), that the community should be able to
appeal individual IAD decisions, with the potential of having
those decisions reversed and with the appeal chain running
though the IESG and IAB to, potentially, the ISOC Board as is
the case with standards-related decisions.  Many of the
advocates of those positions see them as democratic, with the
IETF membership being in ultimate and immediate control.

And there are various people and positions "in the middle".
While I think the "democratic" position is logically consistent,
however impractical, I can't see logical or operational sense in
some of the "middle" positions.  But maybe I just lack
sufficient imagination.

Until we can somehow resolve these rather different models of
how to proceed, I don't think new and more specific text helps
except insofar as it clarifies the differences in positions.

Where I, and some others, have tried to go in the interest of
finding a position that everyone can live with is well short of
what I (and I think you) would like.  I suspect we may still end
up pretty close to your position or mine in practice, but that
the community will need to learn, experimentally, that frequent
and detailed interactions with the IAOC does not serve either
the IASA or the IETF well.   The base of that idea is that the
text in the BCP needs to reflect the following model, more or
less (the current draft text is close to this, but may need
additional editorial or even substantive tuning):

	* The IAOC may establish the conditions, restrictions,
	and permissions under which the IAD may make decisions
	that commit the IASA without prior IAOC approval.
	Subject to commercial and contractual constraints, the
	IAOC may also establish the rules by which they can
	reverse IAD decisions or actions after they are made.
	
	* While lots of things may happen informally, only the
	IAOC can demand that the IAD reconsider a decision or
	hold the IAD directly accountable for such decisions.
	Anything else creates a management-chain mess.  However,
	this statement is at variance with the way I read the
	current text in the draft BCP.
	
	* Any member of the community may provide free advice to
	the IAOC (or IAD) by sending them notes.  And any member
	of the community can question a decision with a note,
	preferably one that has some content along the lines of
	"I think you should reconsider because you may not have
	had the following information".  The IAOC may respond to
	such input with a changed decision, a decision to do
	things differently in the future, an explanation as to
	why the decision or direction are appropriate, or some
	combination of them. But they are under no obligation to
	do so because needing to respond to every input, no
	matter how well intentioned, can easily prevent anything
	else from getting done.  While that concern has been
	coded as "DoS attack", such attacks are really not the
	problem I've been concerned about.   Instead, I've been
	concerned about an excess of free advice, at a micro
	level, from perfectly well-intentioned members of the
	community.    I've also assumed that an IAOC which
	decided to ignore all such input (and review/
	reconsideration requests), regardless of content, would
	find itself retired in due course.
	
	* The IAB and/or IESG can ask the IAOC to "review" a
	decision and can insist on a response.   I/we used the
	term "review", rather than "reconsider" deliberately,
	although putting them together with an "or" is probably
	desirable.    The outcome of a review could be a
	decision to reconsider (when contractually feasible,
	etc.).  It could be a decision that the decision was
	correct, even after new information was considered.   It
	could be a decision to go ahead with the original
	decision, but to adjust procedures to permit timely
	consideration or weighting of different types of
	information in the future.  Or it could be something
	else.  The IAB and IESG get to do this, not because they
	are important, or even because they are going to be the
	members of the community most familiar with the impact
	of IASA operations and decisions (although that will
	almost certainly be the case), but because they can be
	held accountable if they overload the IAOC with "review"
	requests to the point that it interferes with IASA
	functioning.  Random members of the community, or even
	collections of such members, cannot be held accountable
	in the same way.

Looking at your specific questions in this context:

> What's the goal of the reconsideration or review?  E.g. can
> this action result in the IAOC or IAD reversing or revising a
> decision?  If not, why do it?

It can result in a reversal or revision of a decision.  Whether
it does, even if  the IAOC agrees with the complaint, is up to
the IAOC.   And, even if that "even if we agree, we aren't going
to change the decision" result can be anticipated, we should do
it because it is almost inevitable that we are going to learn
about what should be done here as we go along.  That makes "hmm.
good point, we will do it differently next time" a valuable
result -- perhaps the most valuable result.

> What's the statute of limitations for a demand for
> reconsideration or review? Is an action or decision subject to
> reexamination 6 months after the decision?  Does the IAD or
> IAOC need to announce such decisions with enough of a waiting
> period that the reexamination process can take place before
> the decision becomes final?

I hope we can avoid the waiting period, precisely because I
don't ever want to see the potential of that much
micromanagement from outside the IAOC (as should be obvious from
the more general comments above).  But there is no statute of
limitations.  The only real limitation is that, the longer one
waits, the higher the odds that the IAOC will respond "will
consider this the next time an issue like this arises" rather
than "we will change the decision".   And, if the times start to
get really long the odds rise that the IAOC will respond with
"waste of time to consider this" (or not respond at all.

> What is the exact set of decisions subject to this process?
> Everything?  Meeting sites?  Meeting fees?  A delay in
> publication of an RFC?  Cost of paper needed to support an
> IETF meeting?  The date of the public meeting?  Etc.  If we
> can't come up with a description of this that isn't
> "everything" I'm very concerned, but I'm having problems
> figuring out what smaller set is appropriate.

Everything, although I'd expect some of the things you list to
get very brief consideration.  And, to give an example I'm sure
won't arise in practice, if the IAB or IESG were to demand that
the IAOC review explain the cost of paper at frequent intervals,
I'd expect the IAOC members to make that, and its impact on
getting work done, known to the community, the nomcom, etc.

> Who gets to kick this process into starting (e.g. who gets to
> file a complaint)?

Anyone, but only the IAB or IESG can demand a response.

Is that what I'd like in a perfect world?  Nope.  Do I think it
will need tuning over time, probably in the direction of more
IASA autonomy?  Yes, but maybe not in ways that change the BCP.
But I think that, given the divisions in the community over this
issue,  it is probably about the best that any of us will be
able to get.

regards,
     john

---

From: Michael StJohns <[email-masked]>
Date: Sun, 23 Jan 2005 14:20:16 -0500

At 11:50 AM 1/23/2005, John C Klensin wrote:

I deleted all the stuff I agreed with - e.g. most of it.  I'm afraid I 
agree with you that there a vast chasm of differences on this topic.  I 
also agree with you that the body should be independent and that others 
disagree.  *sigh*  In any event I'm pretty close to my last comment on this 

>result -- perhaps the most valuable result.

The "hmm good point" is agreed.  I just think that doing it as a process 
review (e.g. during the public meeting) instead of a decision review is the 
right level of oversight for the broader community.  In other words the 
"public" doesn't get to cause the IAOC to begin a random fire drill, but it 
can cause them to do an in depth analysis of any specific decision during a 
public hearing.  The IETF may choose after that hearing to twiddle with the 
BCP, but not with the decision.

>"waste of time to consider this" (or not respond at all.

Goes back to whether something is reversible/revisable or not.  If its not, 
then the complaints are just so much make work.  If it is, then theres the 
possibility the IAOC will halt implementation of something important just 
because of a complaint (e.g. IAOC make decision, starts process to 
implement, complaint comes in, IAOC pauses implementation until the 
complaint process completes; 5 years later IAOC is still arguing with two 
very vocal IETF members on the proper way to implement).

>getting work done, known to the community, the nomcom, etc.

I'm confused - I thought you were in the hands-off camp.  OK - let's 
consider one very specific case - meeting site selection.  Please describe 
all the actions possible around this decision listing the IAOC, ISOC, IAD, 
IASA, IAB, IESG and IETF in general.  Please explain what should happen if 
we announce we're going to Beijing and 10 IETF participants complain for 
various reasons and cite the fact the meeting cost would be less in 
Minneapolis.  Does the IAOC *really* need to respond to this formally?  By 
the time things are announced they are contractual and are generally not 
subject to revision without great expense.

Another case.  Selection of the IASO.  The IAOC/IAD writes and issues an 
RFP.  They receive bids, evaluate them and with the ISOC issue a contract 
for services.  Does the public in general get to challenge the signing of 
the contract with org A?  Can the IAB/IESG force the IAOC to reconsider the 
award if they don't like the result?  Is that even legal?

Last case.  IAOC/IAD take to heart the section that only the IAB/IESG can 
compel response.  The IAOC/IAD keep getting 10's of complaints about small 
things which they decline to spend anytime on.  The 10's of complaints come 
from the same 10-20 people.  The IETF body as a whole says - "why are you 
ignoring our rightful complaints?".  What happens then?  Does the IAB/IESG 
start compelling responses?  Why not just have the complaints go to the 
IAB/IESG in the first place and have them filter them to the IAOC/IAD?

>Anyone, but only the IAB or IESG can demand a response.

I'm pretty sure that distinction won't work very well.  See above.

>issue,  it is probably about the best that any of us will be

John - all I'm looking for are specific worked examples.  E.g. take various 
cases and work through how they should be handled. Document each step.  See 
if there are common processes for each case. Determine the cost/benefit 
analysis from those processes (e.g. does doing X actually improve the 
behavior of the IAOC?) Derive the BCP from those processes.  Right now the 
BCP describes a process that hasn't even been validated on paper or on a 
white board.  We have neither rough consensus nor running code.

---

From: Sam Hartman <[email-masked]>
Date: Sun, 23 Jan 2005 15:25:59 -0500

>>>>> "Leslie" == Leslie Daigle <[email-masked]> writes:

    Leslie> 3.5 Business Decisions

    Leslie> Decisions made by the IAD in the course of carrying out
    Leslie> IASA business activities are subject to review by the
    Leslie> IAOC.

    Leslie> The decisions of the IAOC must be publicly documented to
    Leslie> include voting records for each formal action.

I object to formal voting records being public because this conflicts
with the requirement that the IAOC make decisions by consensus.

I am uncomfortable with the general approach you suggest.

In decreasing preference order, I prefer:

1) margaret's text on decision  review

2) Something like your 3.5 with margaret's text on decision review.  I'd need to understand what the 3.5 was trying to accomplish better.

3) Harald's text without your 3.5

4) Harald's text with 3.5.

5) Mike's original text.

The point at which I could no longer support the document is either
before option4 or before option 5.

I'm uncomfortable with the 3.5 text because I don't fully understand
what it is trying to accomplish or why that is a good idea.  If the
intent is to prevent some class of business decisions from being
reviewed, then I think I am very uncomfortable.

I can accept that business decisions should not be overturned by an
outside body (although I'm not actually sure I agree), but I will find
it difficult to accept that they should not be reviewed.

--Sam

---

From: Sam Hartman <[email-masked]>
Date: Sun, 23 Jan 2005 17:42:44 -0500

>>>>> "John" == John C Klensin <[email-masked]> writes:

    John> Where I, and some others, have tried to go in the interest
    John> of finding a position that everyone can live with is well
    John> short of what I (and I think you) would like.  I suspect we
    John> may still end up pretty close to your position or mine in
    John> practice, but that the community will need to learn,
    John> experimentally, that frequent and detailed interactions with
    John> the IAOC does not serve either the IASA or the IETF well.

I completely agree that frequent and detailed interactions do not
serve the IETF or IASA well.  

My position stems from two assumptions.  First, while frequent
second-guessing of the IASA is problematic, there are exceptional
circumstances where asking to review a specific decision will improve
the quality of that decision or the underlying procedures; recalling
the IAOC or removing members in these circumstances is too heavy
weight of a solution.  Secondly, I assume that only very limited
procedural safeguards are required to make sure these interactions do
not become too frequent: I believe the IETF community understands the
value in rarely using exception paths and in selecting carefully
decisions for which exception paths should be used.  Moreover I
believe that the cost of trying to design procedural limits up front
is higher than the cost of figuring out if any procedural limits are
needed and putting them in place later.

I do have operational familiarity with being in a management role
where my decisions were subject to appeal.  Overall I actually prefer
working in a situation where there are sanity checks on my actions.  I
would find it unacceptable if my decisions were often appealed.
Similarly, if certain decisions were overturned I would find that I
needed to move on and find other responsibilities.  Provided that the
appeals/review body is aware of the implications of both these limits,
review procedures are a good thing.

--Sam

---

From: John C Klensin <[email-masked]>
Date: Sun, 23 Jan 2005 19:18:22 -0500

--On Sunday, 23 January, 2005 14:20 -0500 Michael StJohns
<[email-masked]> wrote:

Responding in the areas where you are unclear about what I'm
suggesting (or see it differently)....

>> get very brief consideration.  And, to give an example I'm
>> sure won't arise in practice, if the IAB or IESG were to
>> demand that the IAOC review explain the cost of paper at
>> frequent intervals, I'd expect the IAOC members to make that,
>> and its impact on getting work done, known to the community,
>> the nomcom, etc.
> 
> I'm confused - I thought you were in the hands-off camp.  

I consider myself to be in that camp, but believe that the best
way to enforce parts of it is to allow some flexibility in the
system and then hold the various actors accountable for how they
handle that flexibility.   "You can never, never, do this" rules
make me too anxious, especially with a body we've never seen in
action.

> OK -
> let's consider one very specific case - meeting site
> selection.  Please describe all the actions possible around
> this decision listing the IAOC, ISOC, IAD, IASA, IAB, IESG and
> IETF in general.  Please explain what should happen if we
> announce we're going to Beijing and 10 IETF participants
> complain for various reasons and cite the fact the meeting
> cost would be less in Minneapolis.  Does the IAOC *really*
> need to respond to this formally?  By the time things are
> announced they are contractual and are generally not subject
> to revision without great expense.

To respond to the specific example: No.  The IAOC should tell
the ten people to go away.  Or, depending on the objections, it
should remember them for consideration in future meeting site
selections.  And, if it is ten people who have raised this type
of objection in the past, or if the objection lacks substance,
I'd expect the IAOC to ignore it or give it very short shrift.

Now, suppose the ten people then go to to the IESG and say "we
don't think we should go to Beijing and the IAOC didn't change
the meeting or give us an explanation we found satisfactory as
to why we should".   I'd expect the IESG to say "too late, so
sorry, probably we should review criteria for meetings within
the IETF and make some suggestions for future action to the
IAOC".  Or, depending on the objection, I'd expect the IESG to
say "you are wasting everyone's time, go pound sand".  If the
IESG went to the IAOC and said "ok, we think Beijing is maybe
not such a good idea, please review your decision and change
it", I'd expect the IAOC to take the time to explain to the IESG
why the decision was made, that they presumably had an
opportunity for input when the IAOC was considering the decision
(after all, the IETF Chair is an IAOC member), and reminding
them about commercial realities.   And, if the IESG made that
sort of request very often, I'd expect the IAOC members to try
to identify the offenders and to have a discussion with the
Nomcom at an appropriate time.

> Another case.  Selection of the IASO.  The IAOC/IAD writes and
> issues an RFP.  They receive bids, evaluate them and with the
> ISOC issue a contract for services.  Does the public in
> general get to challenge the signing of the contract with org
> A?  Can the IAB/IESG force the IAOC to reconsider the award if
> they don't like the result?  Is that even legal?

First of all, if I correctly read the intent of the RFP, the
IAOC/IAD write that RFP and make it available for comment from
the community before bids are evaluated and contracts are
awarded (and probably before it is issued).  If they get
reasonable input and ignore it, we have a problem, but a rather
different one from the issue you raise above.  So let's assume
that a contract is awarded within the parameters of an RFP that
had general sign-off in the community.  The IESG/IAB can force
the IAOC to review the decision for any improprieties,
especially if they think they have information the IAOC didn't.
They can force the IAOC to report to them on that review.  But
they cannot force reconsideration of the decision/award itself.
Now, I would suggest that the IESG or IAB asking for that sort
of review in the absence of allegations of fraud or other
activities sufficient to invalidate the contract would be a
waste of everyone's time.  But the community has decided that
wasting time is dealt with by individual recalls or nomcom
action, not by trying to write rigid rules.

> Last case.  IAOC/IAD take to heart the section that only the
> IAB/IESG can compel response.  The IAOC/IAD keep getting 10's
> of complaints about small things which they decline to spend
> anytime on.  The 10's of complaints come from the same 10-20
> people.  The IETF body as a whole says - "why are you ignoring
> our rightful complaints?".  What happens then?  Does the
> IAB/IESG start compelling responses?  Why not just have the
> complaints go to the IAB/IESG in the first place and have them
> filter them to the IAOC/IAD?

There are two cases.  In the first, the 10-20 people a
generating trivial or spurious complaints.  The IAOC comments,
probably no more than once or twice, on the inappropriateness of
those complaints to the IASA process and model, then ignores the
complaints.  If the IAB/IESG then compels a response, I'd expect
the response to be much the same.  And, of course, if the
IAB/IESG asks for such a response multiple times, I'd expect
them to get the same answer.  The implied deadlock would then
need to be worked out in the nomcom or by recall.

Why not to the IAB/IESG first?  Because I think this whole
business will succeed to the extent that it unloads work from
the IAB and IESG, rather than increasing their workload.   And
because I think there will be cases in which a member of the
community will want to send a semi-confidential note to the IAD
or IAOC that says "I have some information you don't, and I
think you should review that decision with that information
available to you" without involving the IAB or IESG.  Moreover,
I'm sympathetic to the desire of many members of the community
to be able to communicate with/ complain to the IAOC: I just
thing the burden should be on them, and not on the procedures,
to demonstrate that their complaints are worth paying any
attention to.

>> > Who gets to kick this process into starting (e.g. who gets
>> > to file a complaint)?
>> 

Yes, see above for the reasons for it.  Note that the
"everything goes to the IAB or IESG first" is just a degenerate
case so, if the distinction doesn't work, the BCP-level
procedure doesn't need changing.

>> Is that what I'd like in a perfect world?  Nope.  Do I think
>> it will need tuning over time, probably in the direction of
>> more IASA autonomy?  Yes, but maybe not in ways that change
>> the BCP. But I think that, given the divisions in the
>> community over this issue,  it is probably about the best
>> that any of us will be able to get.
> 
> John - all I'm looking for are specific worked examples.  E.g.
> take various cases and work through how they should be
> handled. Document each step.  See if there are common
> processes for each case. Determine the cost/benefit analysis
> from those processes (e.g. does doing X actually improve the
> behavior of the IAOC?) Derive the BCP from those processes.
> Right now the BCP describes a process that hasn't even been
> validated on paper or on a white board.  We have neither rough
> consensus nor running code.

I have been trying to make variations of that point in more
instances than this case, so we agree.  But the community
doesn't seem to want to do that case analysis, or believes that
the IESG and IAB have done it for them and are taking
responsibility for it, and I don't see any way to make that
particular horse sip the water.  I do hope that, if there are
members of the community are taking the "trust the IESG and/or
IAB" approach to this (based on assurance they presumably think
they have gotten), that the IESG and IAB have actually done the
analysis and that, if they have not done it and done it
adequately, those community members will find the recall
procedures and use them.

    john

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 23 Jan 2005 20:35:33 -0600

I agree with the idea that there are extremes when we talk about our 
ideas on "review", but please don't assume that JohnK and Michael are 
the only people at that end of the pool...

I had assumed that the IETF would let IASA run things with periodic 
general feedback and rare specific feedback (only in situations when 
the feedback begins "we can't imagine why you ..."). The further we 
move from that end of the spectrum, the less I understand why we need 
an IASA in the first place.

Spencer

> decisions either before or after the fact.
>
deleted down to
> the case with standards-related decisions.

---

From: Sam Hartman <[email-masked]>
Date: Sun, 23 Jan 2005 22:39:23 -0500

>>>>> "John" == John C Klensin <[email-masked]> writes:

    John> --On Sunday, 23 January, 2005 14:20 -0500 Michael StJohns
    John> <[email-masked]> wrote:

    >>> > Who gets to kick this process into starting (e.g. who gets >
    >>> to file a complaint)?
    >>> 

    John> Yes, see above for the reasons for it.  Note that the
    John> "everything goes to the IAB or IESG first" is just a
    John> degenerate case so, if the distinction doesn't work, the
    John> BCP-level procedure doesn't need changing.

    >>> community over this issue, it is probably about the best that
    >>> any of us will be able to get.
    >> handled. Document each step.  See if there are common processes
    >> for each case. Determine the cost/benefit analysis from those
    >> processes (e.g. does doing X actually improve the behavior of
    >> the IAOC?) Derive the BCP from those processes.  Right now the
    >> BCP describes a process that hasn't even been validated on
    >> paper or on a white board.  We have neither rough consensus nor
    >> running code.

    John> I have been trying to make variations of that point in more
    John> instances than this case, so we agree.  But the community
    John> doesn't seem to want to do that case analysis, or believes
    John> that the IESG and IAB have done it for them and are taking
    John> responsibility for it, and I don't see any way to make that
    John> particular horse sip the water.  

John, thanks for doing such an excellent job of explaining why someone
might believe that a loosely constrained review process could be a
reasonable thing.  I realize that we disagree on this issue, but I
think your message is a useful tool in explaining the part of this
spectrum in which I find myself.  I agree with the sorts of
explanations you are bringing up and they do serve to explain why I
believe that Margaret or Harald's text is reasonable.

I don't think particularly extensive case analysis has been done; if
people in the community are assuming that has been done and believe
that is important then they should speak up.

I'd like to present one other example that motivates why I think
having the review process is important.  Say that the IASA has decided
to pursue a meeting in Beijing.  No contract has been signed yet, but
that's getting close.

Many contributors indicate they will be unable to  attend.  The IAOc has been focusing on the business aspects of the meeting: how affordable is it, are the necessary facilities available.  

The IESG and/or IAB formally asks for a review, arguing that whether
the right set of people will attend a particular meeting is an
important factor to consider in meeting site selection.  Regardless of
whether the IAOC ends up deciding to reverse its decision, having this review be available is important.

--Sam

---

From: [email-masked]
Date: Mon, 24 Jan 2005 08:39:05 -0500

I suppose that I am one of those at the other end of the scale, looking 
for a solution that allows full and direct IETF community 
review/appeal.

> an IASA in the first place.

My main reason for supporting the creation of the IAOC specifically, is 
to offload most of the administrative effort from the IESG/IAB.  That 
is also, incidentally one of my reasons for not wanting the IESg/IAB to 
be involved in every review/appeal, but only in those that get 
escalated.  The other reason is that it is the IAOC that will be the 
IETF community's representatives in administration and therefore they 
should be directly responsible to that community.

I agree with the role of IAD becaue I think that job needs to focused 
in one person.  But that does not mean that the person should not be 
accountable to the community.  I know it is harder to do a job with 
accountability and transparency, but I see any other solution as being 
problematic for reasons that have outlined earlier.

I remain in favor of Margaret's formulation as the compromise position 
from what I would truly prefer which is for all IAD/IAOC actions and 
decisions being reviewable in the same manner that the actions and 
decisions of the ADs, IESG and IAB are currently appealable.

a.

---

From: John C Klensin <[email-masked]>
Date: Mon, 24 Jan 2005 09:15:16 -0500

Sam,

Let's take another look at your example, from my point of view
(I can't speak for Mike).

--On Sunday, 23 January, 2005 22:39 -0500 Sam Hartman
<[email-masked]> wrote:

> 
> I'd like to present one other example that motivates why I
> think having the review process is important.  Say that the
> IASA has decided to pursue a meeting in Beijing.  No contract
> has been signed yet, but that's getting close.
> 
> Many contributors indicate they will be unable to  attend.
> The IAOc has been focusing on the business aspects of the
> meeting: how affordable is it, are the necessary facilities
> available.  
> 
> The IESG and/or IAB formally asks for a review, arguing that
> whether the right set of people will attend a particular
> meeting is an important factor to consider in meeting site
> selection.  Regardless of whether the IAOC ends up deciding to
> reverse its decision, having this review be available is
> important.

Ok.  If we are doing meeting planning the way we have been doing
meeting planning, there is no announcement to the community
before a contract is signed.  The IETF Chair knows, but the IAB
and IESG generally don't, and random IETF participants certainly
don't know.  If we are going to change that, then changes need
to be made elsewhere than in this section.    Under current
practice, the window you assume simply doesn't exist.   If we
want it to exist, we need to make procedural changes elsewhere
-- probably not in the BCP, but in some way that makes
expectations extremely clear to the IAOC and IAD.

If, via those changes, the window is opened sufficiently that
the IESG and/or IAB can complain, then, under the scenario you
are describing, the IESG/IAB aren't involved with any of what I
(and I think Mike) are worried about.    What you describe is
providing input to the IAD and IAOC _before_ a decision is made.
I think the IAOC should be open to pre-decision community input,
formal or otherwise.  I think it would be seriously bad news if
the IAOC closed its collective ears to such input.  I think
that, if the IAD stops listening to community input, and
especially IAB and IESG input, it would be a good sign that the
individual in that job should be re-educated or forced out of
the position.

But that isn't what I (and probably Mike, Spencer, and others)
are concerned about.  We are concerned about the "decision gets
made and then someone tries to second-guess it" scenarios, on
which we want to place extreme limits.

So, from my perspective, if the case you describe is the one
that is of interest, you should be concentrating less on the
"review" (or appeal) procedures and other mechanisms for
questioning or correcting decisions already made and more on
being certain that, in areas where it is appropriate, the IAOC
is sufficiently open about what it is considering that
pre-decision/ pre-contract input is feasible.

Now, for some cases, that won't be practical for particular
situations or potential contracts.  But there too, I think there
is a useful "make a window for comments" process.  

Taking your meeting case as an example, I personally believe
that the IAOC should be setting criteria for meeting site
selection, that those criteria should be public, and they should
be set independent of particular sites and with full IETF
community involvement (a radical change from either "the IETF
Chair approves" or "the IETF Chair decides").  "The right set of
people needs to be able to go" is an important criterion which
has nothing specific to do with possible meetings in Beijing
(or, for that matter, in Fiji or Washington).  If that is an
established criterion, than an IAD who doesn't do due diligence
on who could or could not attend prior to making a decision is
failing in the job.  IAD job failure problems aren't managed by
reviews or appeals; that is what we have the IAOC for.  And, if
they can't manage the job failures in an effective way... well,
that gets us back to discussions of group firing, but it isn't,
IMO, a post-decision review issue either.

         john

---

From: Sam Hartman <[email-masked]>
Date: Mon, 24 Jan 2005 10:18:27 -0500

I agree with you that pre-decision comments are preferable and that
processes and procedures should allow these comments.

I also agree that the example I proposed cannot happen under current
procedures because there is not a comment window for meeting
locations.  I do not intend to speak to whether such a window would be
a good idea.

However, failure to take adequate comments before making a decision
seems like a reasonable justification from my standpoint for reviewing
that decision.  Depending on the consequences of doing so it may even
be appropriate to reverse such decisions.  There is significant but
*not infinite* cost to reversing a decision.  There can also be
significant cost to having a bad decision.  There is also a cost to
the review process itself.

My expectation is that the review process will be used infrequently
enough that the cost of reviewing and reversing decisions will be less
than the cost of bad decisions.

I fully understand that I may be wrong; we may need to add procedural
safeguards if the cost of review ends up being too high in practice.

Question: do you see cases where if a problem developed we'd be unable
to deploy safeguards fast enough or unwilling to deploy the safeguards
even given an actual instead of theoretical problem?  How likely do
you see these situations?

--Sam

---

From: John C Klensin <[email-masked]>
Date: Mon, 24 Jan 2005 13:26:44 -0500

--On Monday, 24 January, 2005 10:18 -0500 Sam Hartman
<[email-masked]> wrote:

> I agree with you that pre-decision comments are preferable and
> that processes and procedures should allow these comments.
> 
> I also agree that the example I proposed cannot happen under
> current procedures because there is not a comment window for
> meeting locations.  I do not intend to speak to whether such a
> window would be a good idea.

Understood.  Let me only make the observation that I think it
would be better to have clear criteria that the IASA is expected
to follow than to have discussions of individual decisions,
either before or after the fact.  One of the reasons is that,
for some sites that might otherwise be completely reasonable,
having it be known that the site was considered and then
rejected might turn out to be an embarrassment for the site
and/or the IETF and such things should be avoided when possible.

> However, failure to take adequate comments before making a
> decision seems like a reasonable justification from my
> standpoint for reviewing that decision.  Depending on the
> consequences of doing so it may even be appropriate to reverse
> such decisions.  There is significant but *not infinite* cost
> to reversing a decision.  There can also be significant cost
> to having a bad decision.  There is also a cost to the review
> process itself.

This may go to the core of the difference in my position and
yours.   Let's review the composition of the IAOC.   The IETF
and IAB Chairs are members.  If the are doing their jobs there,
they are aware of decisions before they are made, can argue the
IAOC to establish appropriate rules for the IAD to engage in
consultation before a decision is made, and so on.  If they are
dissatisfied with the amount of review a pending decision is
getting, they can request reviews or reconsideration internally,
take the issue up with the IAB or IESG which might then demand a
review, or even initiate an IETF-wide discussion about recalls.
Conversely, if they fail in those regards and decisions are made
without adequate comments and that fact takes the IETF community
by surprise, then perhaps they should be the first people
recalled.  

I'm making an assumption here which might not be valid.  I'm
assuming that, generally, possible IAOCs will fall into one of
two categories.  One --the one we want-- will be open and
transparent whenever possible, will try to design things to
allow for community input before decisions are made whenever
possible, and will try to establish principles, in conjunction
with the community, about how things should be done and then
follow them.  At the other extreme, one might imagine an IAOC
that tried to do everything in secrecy, that was relatively
insensitive to community input, and that tried to arrange its
decision-making processes (and that of the IAD) so that there
was as little opportunity for distracting input or comment as
possible.    I think it is extremely unlikely that we would end
up with an IAOC that would be open about some things but as
secretive as possible about others, at least unless the openness
was used as a deliberate cover for the secrecy.

Now, in my view, the problem of the second or third styles of
IAOC operation is not solved by better techniques for reviewing
and/or changing individual decisions.  The solution is replacing
the IAOC membership (serially if needed) with a group that will
"get" the notion of being open and responsive to the IETF
community.   If we all understand that the expectation of the
IAOC is as much openness and opportunity for comment as is
possible and sensible given the particular issue, and we hold
the IETF-appointed members of the IAOC (including especially the
two Chairs) responsible for reminding the IAOC about that norm,
then we won't need post-decision reconsideration procedures for
individual decisions.  Conversely, if the IAOC doesn't act
according to that norm, we should focus on that fact and fixing
it (in a hurry) and not on the particular decisions that
result... if only because a good decision that is made without
adequate opportunity for contact is as much of a problem as a
bad decision made under the same circumstances.

> Question: do you see cases where if a problem developed we'd
> be unable to deploy safeguards fast enough or unwilling to
> deploy the safeguards even given an actual instead of
> theoretical problem?  How likely do you see these situations?

I think it is up to the Nomcom and to the IAB.  The IETF
community's main insurance against IAOC misbehavior with regard
to decision-making is the presence of the two Chairs on the IAOC
and how those people behave, not provisions of the BCP.  Were
the two Chairs to start behaving as if they took their
instructions and guidance direct from some deity, rather than
from the community, the IAB and the IESG, and to encourage the
IAOC to behave on a similar basis, then I can imagine all sorts
of problems that none of the safeguards we have discussed would
help solve quickly.  But it is unlikely that such behavior would
catch up by surprise and, if the members of the IESG and IAB
remember their responsibilities to the community if some of
their members start misbehaving seriously, we presumably don't
need additional mechanisms.

      john

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 24 Jan 2005 19:55:33 -0600

> which we want to place extreme limits.

This is exactly what Spencer is concerned about... this, and the "bozo 
factor" that results from us making and remaking decisions ("is this 
the last zig, or will we be zagging again?").

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 25 Jan 2005 11:48:11 +0100

> sorts of bad feelings and potential liability.

Exactly. And we need to be sure that the "appeals" text allows for
review of procedures, including the kind of "case study" you suggest,
without allowing the appeal procedure to be used for commercial
food-fights. It's tricky to get that exactly right.

    Brian

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 25 Jan 2005 07:29:58 -0500

Hi Brian,

>food-fights. It's tricky to get that exactly right.

Are you sure that we have consensus on this?  It seems to me that 
some people would like the IAB/IESG to be able to force the 
re-consideration of an IAOC decision, but not for other IETF 
participants to have any way to effectively question the IAOC.  At 
least, that is how most of the recent suggestions read to me.  People 
don't even seem to have picked up on my concern that the current text 
doesn't say who to contact if you do want a decision reviewed, making 
the process difficult and intimidating for anyone who isn't embroiled 
in IETF politics.

Maybe it would help (in getting this issue closed) if we could figure 
out what we want in general terms and then try to draft the text? 
Mixing basic conceptual arguments with text tuning within multiple 
(ever changing) camps doesn't seem to be getting us anywhere.

So, I'll take a shot at a few things attributes that I think would be 
good in a review process:

(1) I agree with you that we do not want a review process (whether 
invoked by an individual or by the IAB and IESG) that can overturn a 
contract award or hiring decision after that decision is made.  The 
current proposed text (I think that the latest was from Leslie) makes 
the community impotent, without properly restricting the review 
requests from the IAB/IESG, IMO.

[N.B.  I do hope that the IAOC will run an open RFP process for most 
(or all?) contracts, and that the process will include a public 
comment stage after the IAOC chooses a potential winner, so that 
there will be an opportunity for the community to raise concerns (if 
any) before it is too late.  Given recent posts, I don't think that 
the IASA-TT/IAOC is likely to take us in this direction within the 
next year, but I do hope that we get there eventually.]

(2) I think that the review process should be well-enough specified 
that a person who is not  a (past or present) member of the I* could 
use it.  This means that it needs to say where you send a review 
request, how you unambiguously identify a formal review request and 
what a review request should contain.  This should be at least at the 
level of detail of the RFC 2026 appeals process (or even a bit more 
detailed, as folks seem to find that process confusing enough that 
they don't often get it right).

(3) I think that review requests should be limited to situations 
where the IAOC violates written procedures (their own or the IASA 
BCP) and/or makes a decision that is against the best interests of 
the IETF.  The request for review should be specific about what 
procedure was violated and/or how a specific decision runs against 
the IETF's interests.

(4) Personally, I think that any member of the community (and yes, I 
understand that means the general public) should be able to make a 
formal review request and expect to get a response from the IAOC 
within a reasonable time period (~90 days).  I do not think the 
response needs to be a lengthy hearing, or a complex legal document. 
But, I think that we should have a review process, open to everyone, 
where a response is mandated.  The response could be:  "We looked 
into this decision, and we didn't find anything irregular about the 
decision or about how it was reached".

(5) I think that there should be at least one level of escalation 
possible if the person requesting a review does not receive a 
satisfactory response from the IAOC (I had suggested that this would 
go to the IESG).  I don't think that the person should have to 
persuade the IAB or IESG to act on his/her behalf (which is another 
way in which the current process is really only open to political 
insiders), I think that the IESG (or whoever we use as the next level 
lf escalation) should be required to consider the IAOC's response and 
respond to the escalated review within a reasonable timeframe.

(6) I do not think that we want the IAB or IESG (or anyone else) to 
be able to overturn a decision of the IAOC, only to advise the IAOC 
that they believe that an incorrect decision was made.

Do folks agree with these thoughts?  Are there other basic ideas that 
should be included?

Margaret

---

From: [email-masked]
Date: Tue, 25 Jan 2005 07:54:51 -0500

hi,

I generally agree with these principles with some comments:

in (2) I think that the review request need to be addressed to the 
chair of the respective body.  I think the language of 2026 can be 
adapted as to contents.

in (5), I think the appeals should have the full chain of appeals.  I 
know 'at least one level' does include the chain, but I think it is 
important to be able to appeal all the way to the ISOC BoT.

in (6), I agree that decisions that involve contractual obligations 
mustn't be overturned, but I have a problem with saying that there are 
no decisions that can be overturned.

a.

On 25 jan 2005, at 07.29, Margaret Wasserman wrote:
> comment stage after the IAOC chooses a potential winner, so that there 
> will be an opportunity for the community to raise concerns (if any) 
> before it is too late.  Given recent posts, I don't think that the 
> IASA-TT/IAOC is likely to take us in this direction within the next 
> year, but I do hope that we get there eventually.]
>
> (3) I think that review requests should be limited to situations where 
> the IAOC violates written procedures (their own or the IASA BCP) 
> and/or makes a decision that is against the best interests of the 
> IETF.  The request for review should be specific about what procedure 
> was violated and/or how a specific decision runs against the IETF's 
> interests.
>
> where a response is mandated.  The response could be:  "We looked into 
> this decision, and we didn't find anything irregular about the 
> (6) I do not think that we want the IAB or IESG (or anyone else) to be 
> able to overturn a decision of the IAOC, only to advise the IAOC that 
> they believe that an incorrect decision was made.
>

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 25 Jan 2005 08:33:47 -0500

>chair of the respective body.  I think the language of 2026 can be 

Yes, I agree.  That is what I included in my proposed wording (lo 
these many moons ago), but would also accept other alternatives, as 
long as it is clear when someone would send a review request and it 
is clear who is responsible for making sure that it is considered.

>in (5), I think the appeals should have the full chain of appeals. 
>I know 'at least one level' does include the chain, but I think it 
>is important to be able to appeal all the way to the ISOC BoT.

Why do you think so?  If it is the ISOC BoT that you want in the loop 
specifically, I suppose that the first level could go there...   It 
seems to me that multi-level appeals in the IETF have not worked very 
well...

>mustn't be overturned, but I have a problem with saying that there 
>are no decisions that can be overturned.

So, how would you make the distinction?  One of the reasons why I'd 
rather see no decisions subject to being overturned is that I don't 
want the IAOC to have an incentive to hide their contract or hiring 
plans from us until after they are signed (and can't be overturned).

Margaret

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 25 Jan 2005 15:37:21 +0100

Margaret Wasserman wrote:
> Are you sure that we have consensus on this?

Between me and Sam, apparently :-)

> ... It seems to me that some 
> people would like the IAB/IESG to be able to force the re-consideration 
> of an IAOC decision, but not for other IETF participants to have any way 
> to effectively question the IAOC.  At least, that is how most of the 
> recent suggestions read to me.  People don't even seem to have picked up 
> on my concern that the current text doesn't say who to contact if you do 
> want a decision reviewed, making the process difficult and intimidating 
> for anyone who isn't embroiled in IETF politics.

I thought it was fairly clear that you should start with the IAOC (just
like you start with the WG for a standards appeal).

> out what we want in general terms and then try to draft the text? Mixing 
> basic conceptual arguments with text tuning within multiple (ever 
> changing) camps doesn't seem to be getting us anywhere.
> 
> the community impotent, without properly restricting the review requests 
> from the IAB/IESG, IMO.

I agree with that intent. IMO the text in draft-ietf-iasa-bcp-04.txt
has some defects, but is mainly consistent with that intent.

> 
> [N.B.  I do hope that the IAOC will run an open RFP process for most (or 
> all?) contracts, and that the process will include a public comment 
> stage after the IAOC chooses a potential winner, so that there will be 
> an opportunity for the community to raise concerns (if any) before it is 
> too late.  Given recent posts, I don't think that the IASA-TT/IAOC is 
> likely to take us in this direction within the next year, but I do hope 
> that we get there eventually.]
> 
> (2) I think that the review process should be well-enough specified that 
> a person who is not  a (past or present) member of the I* could use it.  
> This means that it needs to say where you send a review request, how you 
> unambiguously identify a formal review request and what a review request 
> should contain.  This should be at least at the level of detail of the 
> RFC 2026 appeals process (or even a bit more detailed, as folks seem to 
> find that process confusing enough that they don't often get it right).

[email-masked]? draft-ietf-iasa-bcp-04.txt says 'starting with the IESG'
which actually seems wrong - the review should start right with the
IAOC and then be escalated if necessary.

> the IAOC violates written procedures (their own or the IASA BCP) and/or 
> makes a decision that is against the best interests of the IETF.  The 
> request for review should be specific about what procedure was violated 
> and/or how a specific decision runs against the IETF's interests.

Yes, but we need to make sure that the text more or less excludes
appeals against the wrong kind of cookies as well as against
commercial decisions.

> formal review request and expect to get a response from the IAOC within 
> a reasonable time period (~90 days).  I do not think the response needs 
> to be a lengthy hearing, or a complex legal document. But, I think that 
> we should have a review process, open to everyone, where a response is 
> mandated.  The response could be:  "We looked into this decision, and we 
> didn't find anything irregular about the decision or about how it was 
> reached".

Sure, that is analagous to appeals under 2026

> satisfactory response from the IAOC (I had suggested that this would go 
> to the IESG).  I don't think that the person should have to persuade the 
> IAB or IESG to act on his/her behalf (which is another way in which the 
> current process is really only open to political insiders), I think that 
> the IESG (or whoever we use as the next level lf escalation) should be 
> required to consider the IAOC's response and respond to the escalated 
> review within a reasonable timeframe.

I see no reason not to allow the whole appeal chain, again by analogy
with 2026.

> they believe that an incorrect decision was made.

Exactly right.

And since fewer words are better, maybe we should look for the minimum
changes to the words in draft-ietf-iasa-bcp-04.txt to achieve these
goals.

    Brian

---

From: [email-masked]
Date: Tue, 25 Jan 2005 11:11:23 -0500

On 25 jan 2005, at 08.33, Margaret Wasserman wrote:

> long as it is clear when someone would send a review request and it is 
> clear who is responsible for making sure that it is considered.

I think that the Chair is always ultimately responsible for seeing to 
it that the work of the IAOC, or the IAB or IESG for that matter, gets 
done.

> seems to me that multi-level appeals in the IETF have not worked very 

I think the multi level escalation works by making sure that the 
correct level of the organizational layering gets to handle the review. 
  It would be impossible, or at least very complex to say, reviews for 
topic X go to the IAB while reviews for y go to the the IESG and for Z 
to the ISOC BoT.  And while I think that the ISOC BoT is the reviewer 
of last resort (as they are today for process issues), I don't believe 
they necessarily need to be the reviewer of first resort.

If multi-level reviews are not working well, I think this is a 
different problem to be explored separately, but my view is that the 
IAOC should be slipped into the processes in as compatible a method as 
possible, and I think that following the same escalation procedures as 
are currently in existence is an appropriate approach.

> plans from us until after they are signed (and can't be overturned).

I think it is relatively easy to distinguish a contractual obligation 
from a decision that is not contractual, there is a contract.

Now you bring up a good point, what if the IAC uses that as a loophole, 
and make a decision to not be transparent about contractual plans so 
that their decisions cannot be altered.  This would certainly, to my 
mind, qualify as a decision where a successful appeal would force a 
change.  I can imagine many other decision that they could make that 
might also be subject to revision.

My problem with making all decisions unchangeable is that it makes 
appeals toothless, they never can have any real effect.  And I believe 
that this is almost, but not quite, as bad as not having an appeals 
process as it leave no recourse other then recall or public discontent.

a.

---

From: Sam Hartman <[email-masked]>
Date: Mon, 24 Jan 2005 17:54:30 -0500

>> to reversing a decision.  There can also be significant cost to
    >> having a bad decision.  There is also a cost to the review
    >> process itself.

    John> I'm making an assumption here which might not be valid.  I'm
    John> assuming that, generally, possible IAOCs will fall into one
    John> of two categories.  

I think I disagree with this assumption and I think that is the core
of our disagreement.

    John> One --the one we want-- will be open and
    John> transparent whenever possible, will try to design things to
    John> allow for community input before decisions are made whenever
    John> possible, and will try to establish principles, in
    John> conjunction with the community, about how things should be
    John> done and then follow them.  
I agree with you that if the IAOC  is interested in secrecy then
    John> recall is the best and only solution.

however I suspect that even a desirable IAOC will not match the
platonic ideal of your first category.  They will end up cutting
corners, making decisions that people disagree with, etc.  We hope
they are generally interested in openness, consensus-building and
establishing principles.  They will also be interested in getting work
done and will have limited resources.

I suspect that some times, even the best of people will make the wrong
decision or use the wrong process.  When such a group of people is
faced with an appeal or review, they will honestly sit down and
reconsider their decision and sometimes even decide they were wrong.

Infrequent use of an appeals process--even one leading to successful
appeals does not always mean there is a process problem or even that
someone needs replacing.

--Sam

---

From: Sam Hartman <[email-masked]>
Date: Tue, 25 Jan 2005 15:50:01 -0500

I agree with Margaret's general principles with a few comments.

(4) is desirable to me but not critical.

I am ambivalent on (6); I don't think it is particularly problematic
but do not think it is required.  I understand others disagree with me
strongly on this point.

The rest of the principles for a review process seem very important to
me.  Also, I believe Margaret's principles are sufficient.

---

From: Leslie Daigle <[email-masked]>
Date: Tue, 25 Jan 2005 21:15:31 -0500

With apologies for having posted & disappeared (ISP & other
unexpected connectivity challenges), I'd like to try another
cut at what I was getting at, based on the discussion since.

On Friday, I tried a minimal edit on words that had flown
around the list and seemed to have some consensus. Here's
more of an extensive rewrite.

Apart from the distinction I had tried to make about

	. business decisions (implementation) versus
	. performance

I heard these requirements expressed:

	. DoS concerns -- don't create a system that begs for
	  denial of IASA service through the appeal/review process
	. second-guessing -- don't create an environment where
	  some or all of the community is second-guessing IAD/IAOC
	  at every move
	. community involvement -- adequate and appropriate
	  community involvement in the IASA decision making process

And I heard various theories about distinguishing between

	. before decisions are made
	. during the decision-making process
	. after decisions have been made

particularly in terms of whether we are discussing

	. appeal (review of executed action by an univolved
	  body)
	. review (further discussion of an action or proposal)
	. recall of IAOC members

Separately from those considerations, there is the question of
implementation -- what people/body(ies) invoke an action, and
so on.

So, generally speaking, I think the important things to capture
in the BCP are:

	. business decisions remain within the IASA
	  in terms of review mechanisms (i.e., contracts, etc)

	. the IASA should be explicitly pressed to publicize
	  and seek input on guidelines for making those decisions

	. public information should include objective measurements
	  of system performance (e.g., document processing
	  times)

	. there should be a crisp review process to address
	  the situation when (some subset of the IETF believes)
	  the IASA has not followed its guideliness in
	  carrying out an action -- and that includes the expectation
	  of having public guidelines.

To the meeting location example -- that would mean (IMO) that
the IASA should have some public guidelines for how it picks
meeting sites, and if a site is picked that appears to be at
odds with those guidelines, then there is a process for reviewing
the IASA's actions in selecting that site.  NB - that is different
than appealing the site itself.

So, with all that in mind, I propose the following revised
text for the 2 sections I suggested on Friday:

--------

3.5  Business Decisions

The decisions of the IAOC must be publicly documented for each formal
action.

The IAOC is directly accountable to the IETF  community for the
performance of the IASA.  In order to achieve this, the
IAOC and IAD will ensure that guidelines are developed
for regular operation decision making.  Where appropriate, these
guidelines should be developed with public input.  In all
cases, they must be made public.

Additionally, the IASA should ensure there are reported objective
performance metrics for all IETF process supporting activities.

In the case where someone questions that an action of the IAD or the
IAOC has been undertaken in accordance with this document
or those operational guidelines (including the creation of an
appropriate set of such guidelines), he or she may ask for a formal
review of the action.

The request for review is addressed to the person or body that took
the action. It is up to that body to decide to make a response,
and on the form of a response.

Answered requests for review and their responses are made public.
Reviews of the IAD's actions will be considered at his or her following
performance review.  Reviews of the IAOC's actions may be considered
when IAOC members are subsequently being seated.

-----------

Note that this deletes much of the text that is currently
the "Responsiveness of the IASA to the IETF Leadership"
section:

> However, the nature of the IAOC's work

I wondered if this was still particularly relevant/appropriate or
needed...

Where it does leave the IAB and IESG in priviledged position is
that they act as channelers.  I am not personally strongly wedded
to this -- it's a leftover from previous discussion.

Leslie.

Leslie Daigle wrote:
>>> reappointment process.  (Note, I'm not precluding pre-decision 
>>> comment by the community at large, and I encourage the IAOC to seek 
>>> such comment where appropriate but once the decision is made its time 
>>> to stop whining and get on with things)
>>>
>>> The IAOC and IAD must accept public or private comment but there is 
>>> no requirement to either respond or comment on such missives.
>>>
>>> Please, let's get the IETF out of the metaphorical administrative 
>>> back seat and get them back to doing what they do well - technology.
>>>
>>>

---

From: [email-masked]
Date: Tue, 25 Jan 2005 23:44:09 -0500

Hi Leslie,

This formulation is still of the form that does not give the IETF 
community a direct voice in the review and appeal mechanisms for the 
IAOC.

I, personally see not reason why the IAOC is not directly addressable 
by the community and does not have a direct obligation to the IETF 
community.  While I am comfortable with the IESG and IAB being the 
appeal path for the IAOC, I am not comfortable with them being a 
firewall for the IAOC.

I think this is a fundamental question that differentiates Margaret's 
formulation from yours.  I also think it is a fundamental question that 
goes back to issues in the problem statement about the current 
leadership model:  too much influence is focused in one leadership 
group.  One benefit of the creation of the IAOC is that it spreads the 
task of running of the IETF to another group of people.  As such, I 
think the IAOC must be required to respond directly to the community.

a.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 26 Jan 2005 08:23:57 +0100

Avri,

--On tirsdag, januar 25, 2005 23:44:09 -0500 [email-masked] wrote:

> community a direct voice in the review and appeal mechanisms for the IAOC.

I do not understand what you mean by "direct voice". Could you explain?

If what you mean is that the community should have representatives involved 
in the consideration of the issues, and do not think that the 
nomcom-selected members, the IESG-selected members and the IAB-selected 
members of the IAOC are appropriate community representation, I do not see 
any mechanism short of the way we constitute recall committees that will 
give you what you want.

If you think that the community should have the right of complaint, then I 
think you need to accept some limitation by human judgment on how much 
effort each complaint can cause. If that judgment is to lie outside of the 
IAOC, it has to be invoked for all complaints to the IAOC (making the 
system more formalistic); if it is inside the IAOC, it seems reasonable to 
have some means of overriding it.

> I, personally see not reason why the IAOC is not directly addressable by
> the community and does not have a direct obligation to the IETF
> community.  While I am comfortable with the IESG and IAB being the appeal
> path for the IAOC, I am not comfortable with them being a firewall for
> the IAOC.

I do have a problem with seeing the words that Leslie proposed as fitting 
your description. As described, it isn't a firewall - it's an override of a 
safeguard.

> goes back to issues in the problem statement about the current leadership
> model:  too much influence is focused in one leadership group.  One
> benefit of the creation of the IAOC is that it spreads the task of
> running of the IETF to another group of people.  As such, I think the
> IAOC must be required to respond directly to the community.

I don't quite see the logic here - we take tasks that are currently 
performed in an undocumented and unaccountable fashion and move them into a 
body that has oversight over them, is selected by the community, is 
removable by the community, and is (as I see it) normally expected to 
respond to the community.

Question: My reading of Leslie's words is that "It is up to that body to 
decide to make a response" should be read by the IAOC as "you'd better have 
a good reason not to make a response".

Is what you're really looking for a way to make that "bias" in judgment 
explicit?

                       Harald

---

From: [email-masked]
Date: Wed, 26 Jan 2005 08:03:18 -0500

Hi Harald,

> I do not understand what you mean by "direct voice". Could you explain?

As I understand Leslie's formulation, the IAOC has no requirement to 
process a review from a normal member of the IETF Community unless that 
request comes from the IAB or IESG.  To my mind, this means that the 
IAOC is answerable to the IAB or IESG and not directly answerable to 
the IETF Community.

When an individual IETF participant makes a review request, it may be 
ignored.
If someone is unhappy at being ignored they may make a request to the 
IAB or IESG for recognition.  This request may be also ignored, with 
the only recourse to that being an appeal of the IAB or IESG decision 
to ignore their request.

That is, it is only if someone interests the IAB or IESG in their issue 
that it forces a review.  Also the only decision that can really be 
appealed is the IAB or IESG handling of the request for a review not 
the decision of the IAOC. I am defining that this as not having direct 
voice.

>
> If what you mean is that the community should have representatives 
> involved in the consideration of the issues, and do not think that the 
> nomcom-selected members, the IESG-selected members and the 
> IAB-selected members of the IAOC are appropriate community 
> representation, I do not see any mechanism short of the way we 
> constitute recall committees that will give you what you want.

My issue is not with how the members are appointed to the IAOC.  I am 
fine with that.  My issue is whether they are accountable to the 
community or the community's representatives.  As written they are 
accountable only to the the community's representatives and are thus 
one step removed from direct accountability to the community.

>
> If you think that the community should have the right of complaint, 
> then I think you need to accept some limitation by human judgment on 
> how much effort each complaint can cause.

I have not seen any argument that convinces me that those limits should 
be any different then the limits to judgment that currently exist to 
complaints, i.e. appeals, against the IAB or IESG.  I am basically 
using the 'running code' argument and asking that the appeal process we 
currently have be extended to this new IETF management group.

> If that judgment is to lie outside of the IAOC, it has to be invoked 
> for all complaints to the IAOC (making the system more formalistic); 
> if it is inside the IAOC, it seems reasonable to have some means of 
> overriding it.
>
>> by
>> appeal
> I do have a problem with seeing the words that Leslie proposed as 
> fitting your description. As described, it isn't a firewall - it's an 
> override of a safeguard.

A firewall protects.  As written the IESG or IAB protects the IAOC from 
the IETF community, which to some extent is being assumed to be a 
sometimes malicious DOS'ing environment that the IAOC needs to be 
isolated from.

>> leadership
> performed in an undocumented and unaccountable fashion and move them 
> into a body that has oversight over them, is selected by the 
> community, is removable by the community, and is (as I see it) 
> normally expected to respond to the community.

To some extent those tasks were performed in an unaccountable fashion, 
and to some extent the Chair's of the IAB and IESG (and maybe the 
groups themselves) have been the only ones who had any visibility, for 
some degree of visibility, into them.

But that is not really the point.  If as you say this is oversight that 
never occurred before, then I see this formulation as adding more 
responsibilities to the IAB/IESG, i.e. acting as the oversight body and 
as the arbiter of the community's voice.  And to refer back to the 
Problem process this is adding responsibility to a group that is 
already overloaded and which has a scope of responsibility that some 
feel is already overly large.

I guess I dispute, and that really is a fundamental point, that the 
IAOC in this formulation is normally expected to respond to the 
community.  I see them as normally expected to respond to the IAB and 
IESG.

>
> Question: My reading of Leslie's words is that "It is up to that body 
> to decide to make a response" should be read by the IAOC as "you'd 
> better have a good reason not to make a response".

I don't read it that way.  I read it as: "you don't have any need to 
respond to anyone who is not in a senior leadership position."  And 
given that the Chair's of both the IAB and IESG will be on the IAOC, it 
puts them in the position of saying "we can ignore it, I don't believe 
the IAB or IESG will back up this request for a review even if they get 
that far".

>
> Is what you're really looking for a way to make that "bias" in 
> judgment explicit?

I am not sure I understand this question.  But if by '"bias" in 
judgment' you mean: am I looking for a formulation that forces the IAOC 
to respond to every request for a review by members of the IETF 
community, the answer is yes.  And to go further I am looking for the 
appeal to start with the IAOC and proceed from there through the normal 
IESG-IAB-ISOC BoT chain of appeal.

a.

---

From: Leslie Daigle <[email-masked]>
Date: Wed, 26 Jan 2005 12:30:36 -0500

Avri,

I hear what you are saying.   I retained the proposed text
for being obliged to respond only when direct by IAB/IESG
because people seemed to want it for rate limiting (i.e.,
preventing DoS).  So, we can't just throw it out.  We can
change it (entirely), but the empty set option does
not seem to meet other requirements expressed here.

The one further adjustment I think I would suggest to
my text of yesterday is to change "Answered requests...
made public" to "All requests...made public".  I.e., it
should not be possible for the IAD/IAOC to sweep issues
under a carpet.

Personally, I believe that does make the IASA ultimately
accountable to the IETF community as a whole, even if it
is not *instantaneously* accountable to the IETF community
for every decision.

And I think it is important that we not lose sight of
the other things my proposed text attempted to capture
and distinguish.

Leslie.

---

From: Sam Hartman <[email-masked]>
Date: Wed, 26 Jan 2005 18:01:25 -0500

I don't think I can support your proposed text.  I still don't
understand what your proposed section 3.5 does and don't think I could
go along with the plausable readings I'm coming up with for that text.

I don't think your text does a good job of meeting the principles
Margaret tried to outline; in particular it seems to fail to meet the
principle on escalation.  I'd be very interested in an explanation of
how you think it compares to those principles.

--Sam

---

From: Leslie Daigle <[email-masked]>
Date: Wed, 26 Jan 2005 20:26:02 -0500

Sam,

Let me first take another stab at recap'ing the discussion that
lead to my proposal for 3.5 and 3.6, and clarifying what I
intend as a distinction between them.

As I understood them, John Klensin, Mike St.Johns, and others
were concerned about creating an IASA that could not or
operate without constant checking by the IETF community (having
their feet shot at, in the worst case).  That makes sense
to me -- the IASA, as a separate activity, should have clear
boundaries of responsibility.  The IETF community
as a whole should not become the invisible hands driving
the IASA actions.  This is the intent of section 3.5.

At the same time, the IASA is meant to be responsive to the IETF,
and should not be able to lose track of that.   We should have a
mechanism for expressing when we think the IASA is not behaving
according to its rules (the BCP, and operational guidelines it
develops to carry out those functions).  This is the intent
of section 3.6

So, I don't (personally) expect a future where individual
IETF participants can derail a proposed meeting site because
they don't agree with it.  However, individual IETF members
should be able to point out that a proposed meeting site
selection is not in line with state operational guidelines
for picking meeting sites (which might include proposing them
publicly for 2 weeks before finalizing, for eg).

To your specific question of how my proposed 3.5 and 3.6
(reproduced below for those who didn't memorize them) fit
with Margaret's notes:

[Margaret wrote:]
 > the community impotent, without properly restricting the review requests

Well, I disagree that it makes the community impotent.  See my
note to Avri today.  My text does attempt to make clear what level
individual IETF members should get involved at.

So, the intent of my proposed text is to not only prevent
undoing of signed contracts, but also say that the IETF in
general should not be focused on every action that leads to
such contracts.  I believe this is a point where Margaret and
I disagree.

 > that a person who is not a (past or present) member of the I* could use
 > it. This means that it needs to say where you send a review request, how
 > you unambiguously identify a formal review request and what a review
 > request should contain. This should be at least at the level of detail
 > seem to find that process confusing enough that they don't often get it
 > right).

Section 3.6 attempts to be clear about its review process. It
may not be sufficiently detailed.  I think it is probably detailed
enough for the RFC. (Further detail could be... operational guidelines).

> where

I believe my text agrees with that.  I'm positing that "best interests
of the ietf" are captured in the BCP and the operational guidelines;
to the extent that they do not, then it would be hard for the IASA
to know what it was supposed to have done.  This may mean that
operational guidelines need to be created or updated for future
situations.

 > didn't find anything irregular about the decision or about how it was

I think the latter is achievable under 3.6, or a very slight
modification thereof.  Earlier today I suggested that I thought
the last "Answered requests..." should be modified to "All requests...",
and it's not a big step from there to saying that all requests
are answered even with a short statement like hte last.  I
don't have an issue with that.  Others concerned about DoS
attacks might.

 > review within a reasonable timeframe.

I don't have an opinion here. There have been cogent arguments against
making the IAB/IESG the next level of escalation, or the ISOC BoT.
Others who had stronger feelings should comment.

> be

I don't have an issue wiht this, and I don't think the text
suggests any.

So -- what was the problem with the proposed text and these
principles (apart from the one noted disagreement, above)?

Leslie.

[I had proposed 3.5 and 3.6:]

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 26 Jan 2005 17:40:43 -0800

> how you think it compares to those principles.

I'm not sure I necessarily agree with Margaret's principles. Here's
a cut at a set of my own:

Any set of rules for how the IASA makes decisions and how the
IETF can influence them needs to satisfy two somewhat conflicting
constraints:

1. Preserve the openness of the process and allow the IETF to
   supervise IASA in order to ensure that it is serving the
   interests of the IETF.

2. Allow the IASA to make decisions without being second guessed
   at every turn by individual IETFers who happen not to like
   them.

This is always a tricky problem, but we have a model to fall
back on, which is RFC 2026 and 2727. We have three major
mechanisms for the community to provide oversight for the
IESG and IAB:

1. Appeals up the chain (S 6.5 of RFC 2026) for mistaken decisions
   by IESG.

2. The nomcom and recall process (RFC 2727) for misconduct by
   IESG/IAB members.

3. Creating new RFCs which constrain the behavior of IESG and IAB. 

It's important to note two things about these mechanism. First, each has
its own role. Individual appeals, even if successful, do not really
constrain the IESG's behavior in the future. Conversely, we generally
don't attempt to write new RFCs to reverse specific IESG decisions, nor
would we expect that they would succeed. 

Second, 2026 grants both the IESG and IAB a fair amount of latitude
about how to structure their internal procedures. In particular,
the IESG and IAB rules about how they reach decisions are largely
self-made, with only very modest amounts of guidance from the 
RFCs. I think it's generally considered that that's a good thing.

With that in mind, I would like to suggest the following principles:

1. The IETF community should have input on the internal rules
   set by the IASA and the IASA should be required to respond
   to comments by the community on said rules.

2. While the IASA's behavior should be constrained by BCPs (as it will
   be constrained by the first one) we should in general refrain from
   enacting specific internal IASA rules changes by BCP. If it is believed
   that the IAOC is making bad decisions we have a mechanism for unseating
   them.

3. Decisions of the IAOC should be appealable (following the
   usual 2026 appeal chain) on the sole grounds that the IASA's
   processes were not followed. Those decisions should NOT be
   appealable on the grounds that the decisions were simply bad ones.

I recognize that point (3) above is somewhat controversial, so 
I'll attempt to justify it a bit. The simple fact is that the 
IASA (both IAD and IAOC) will be constantly making business decisions
and if we are to keep the job manageable (and in the case of IAD
keep the costs down) they have to know that they will not be
constantly second-guessed by the community on what kind of cookies
they purchased. If the community feels very strongly that the 
wrong kind of cookies were purchased then they can ask for a
rule demanding chocolate chip, and in the worst case pass
a chocolate chip cookie RFC.

There's a parallel here in the RFC 2026 appeals process. While
the IAB and IESG can review a decision both for process violations
AND for technical merit, the ISOC BOT is extremely constrained in 
that it can only provide limited procedural review (S 6.5.3)

   Further recourse is available only in cases in which the procedures
   themselves (i.e., the procedures described in this document) are
   claimed to be inadequate or insufficient to the protection of the
   rights of all parties in a fair and open Internet Standards Process.
   Claims on this basis may be made to the Internet Society Board of
   Trustees.  The President of the Internet Society shall acknowledge
   such an appeal within two weeks, and shall at the time of
   acknowledgment advise the petitioner of the expected duration of the
   Trustees' review of the appeal.  The Trustees shall review the
   situation in a manner of its own choosing and report to the IETF on
   the outcome of its review.

   The Trustees' decision upon completion of their review shall be final
   with respect to all aspects of the dispute.

It seems to me that this is a good model to follow for IASA.

I've deliberately avoided trying to write text because I think
it's important to agree on principles first.

-Ekr

---

From: [email-masked]
Date: Wed, 26 Jan 2005 23:29:09 -0500

This set of principles works for me.

a.

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 26 Jan 2005 21:39:03 -0800

Sam Hartman <[email-masked]> writes:

>>>>>> "Eric" == Eric Rescorla <[email-masked]> writes:
>
>     Eric> bad decisions we have a mechanism for unseating them.
>
>     Eric> 3. Decisions of the IAOC should be appealable (following the
>     Eric> usual 2026 appeal chain) on the sole grounds that the IASA's
>     Eric> processes were not followed. Those decisions should NOT be
>     Eric> appealable on the grounds that the decisions were simply bad
>     Eric> ones.
>
>     Eric> I recognize that point (3) above is somewhat controversial,
>     Eric> so I'll attempt to justify it a bit. The simple fact is that
>     Eric> the IASA (both IAD and IAOC) will be constantly making
>     Eric> business decisions and if we are to keep the job manageable
>     Eric> (and in the case of IAD keep the costs down) they have to
>     Eric> know that they will not be constantly second-guessed by the
>     Eric> community on what kind of cookies they purchased. 
>
> I agree so far.
>
> However people seem to be making a jump I don't agree with from they
> should not be constantly second-guessed to they should never be
> second-guessed.  The problem with constant second-guessing is that it
> takes up too much time and resources, not that it doesn't sometimes
> improve quality of decisions.

Well, sometimes it improves the quality of decisions and sometimes
it makes them worse. One common pathological case is what's known
in the economics literature as rent seeking.

>  So, my goal is to find the minimal
> restriction that prevents a resource consumption attack against the
> IASA in practice.  I'd prefer to err on the side of allowing a
> resource consumption attack and fixing it later than on the side of
> being too restrictive in what review we allow.

I'm not sure why you think this is the best approach.  It's precisely
in the current moment where things are in flux and, frankly, not
functioning that smoothly, that it's critical to be able to move
effectively. In my view, it makes sense at this point to give the IASA
a fair amount of latitude to form its own processes (under the
supervision of the IAB, IESG, and ISOC board, of course.) So, why not
err on the side of effectiveness now and then fix it if we find
good decisions aren't being made?

>     Eric>    The Trustees' decision upon completion of their review
>     Eric> shall be final with respect to all aspects of the dispute.
>
>     Eric> It seems to me that this is a good model to follow for IASA.
>
> OK.  I don't like this model because I don't believe it is minimal
> restrictive.  I'd like to cite another parallel from RFC 2026.  Under
> RFC 2026 section 6.5, the IAB does not have the power to assert a
> specific decision, only to void an existing decision.  I'd rather see
> limits on what the results of a review can be than on the subject
> matter of reviews.  I think that is a more minimal restriction and
> believe it will be sufficient to avoid DOS concerns.

I'm not convinced that this will do the job. The goal of this kind
of influence behavior is often not to get any particular decision
reversed but merely to make the cost of making the decision one
doesn't like excessive. That said, it seems to me that having a 
decision one doesn't like voided, even if not reversed, is a 
pretty dramatic outcome when we're talking about cancelling a 
proposed contract. 

-Ekr

---

From: Sam Hartman <[email-masked]>
Date: Thu, 27 Jan 2005 00:15:13 -0500

>>>>> "Leslie" == Leslie Daigle <[email-masked]> writes:

    Leslie> Sam,

    Leslie> Let me first take another stab at recap'ing the discussion
    Leslie> that lead to my proposal for 3.5 and 3.6, and clarifying
    Leslie> what I intend as a distinction between them.

    Leslie> As I understood them, John Klensin, Mike St.Johns, and
    Leslie> others were concerned about creating an IASA that could
    Leslie> not or operate without constant checking by the IETF
    Leslie> community (having their feet shot at, in the worst case).
    Leslie> That makes sense to me -- the IASA, as a separate
    Leslie> activity, should have clear boundaries of responsibility.
    Leslie> The IETF community as a whole should not become the
    Leslie> invisible hands driving the IASA actions.  This is the
    Leslie> intent of section 3.5.

Thanks so much for this note.  It was useful in helping me figure out
where we disagreed.

    Leslie> So, I don't (personally) expect a future where individual
    Leslie> IETF participants can derail a proposed meeting site
    Leslie> because they don't agree with it.  However, individual
    Leslie> IETF members should be able to point out that a proposed
    Leslie> meeting site selection is not in line with state
    Leslie> operational guidelines for picking meeting sites (which
    Leslie> might include proposing them publicly for 2 weeks before
    Leslie> finalizing, for eg).

I think we disagree on this point.  I think it is appropriate for an
individual, the IESG or IAB to ask for such a review, arguing that the
decision was not in the best interests of the IETF for some reason.  I
think that such requests for review should meet a much higher standard
than claims procedures or guidelines are violated in order to be
considered seriously.

As such, I disagree with your proposed text.

    Leslie> [Margaret wrote:]
    >> (1) I agree with you that we do not want a review process
    >> (whether invoked by an individual or by the IAB and IESG) that
    >> can overturn a contract award or hiring decision after that
    >> decision is made. The current proposed text (I think that the
    >> latest was from Leslie) makes the community impotent, without
    >> properly restricting the review requests from the IAB/IESG,
    >> IMO.

    Leslie> Well, I disagree that it makes the community impotent.
    Leslie> See my note to Avri today.  My text does attempt to make
    Leslie> clear what level individual IETF members should get
    Leslie> involved at.

    Leslie> So, the intent of my proposed text is to not only prevent
    Leslie> undoing of signed contracts, but also say that the IETF in
    Leslie> general should not be focused on every action that leads
    Leslie> to such contracts.  I believe this is a point where
    Leslie> Margaret and I disagree.

I agree with Margaret.
That said, I recognize that something will have to be done if the review/appeal process is often used.

    >> (3) I think that review requests should be limited to
    >> situations where the IAOC violates written procedures (their
    >> own or the IASA BCP) and/or makes a decision that is against
    >> the best interests of the IETF. The request for review should
    >> be specific about what procedure was violated and/or how a
    >> specific decision runs against the IETF's interests.

    Leslie> I believe my text agrees with that.  I'm positing that
    Leslie> "best interests of the ietf" are captured in the BCP and
    Leslie> the operational guidelines; to the extent that they do
    Leslie> not, then it would be hard for the IASA to know what it
    Leslie> was supposed to have done.  This may mean that operational
    Leslie> guidelines need to be created or updated for future
    Leslie> situations.

I believe your definition of best interests of the IETF is too narrow; I believe this is the same issues as point 1 above.
    Leslie> So -- what was the problem with the proposed text and
    Leslie> these principles (apart from the one noted disagreement,
    Leslie> above)?

I think we're down to that one core disagreement.  Thanks for helping
me realize that.

---

From: Sam Hartman <[email-masked]>
Date: Thu, 27 Jan 2005 00:26:42 -0500

I agree so far.

improve quality of decisions.  So, my goal is to find the minimal

    Eric> If the
    Eric> community feels very strongly that the wrong kind of cookies
    Eric> were purchased then they can ask for a rule demanding
    Eric> chocolate chip, and in the worst case pass a chocolate chip
    Eric> cookie RFC.

That's one way to look at it.  And if that's the only way we can make
forward progress it's better than not making forward progress.

    Eric> There's a parallel here in the RFC 2026 appeals
    Eric> process. While the IAB and IESG can review a decision both
    Eric> for process violations AND for technical merit, the ISOC BOT
    Eric> is extremely constrained in that it can only provide limited
    Eric> procedural review (S 6.5.3)

    Eric>    Further recourse is available only in cases in which the
    Eric> procedures themselves (i.e., the procedures described in
    Eric> this document) are claimed to be inadequate or insufficient
    Eric> to the protection of the rights of all parties in a fair and
    Eric> open Internet Standards Process.  Claims on this basis may
    Eric> be made to the Internet Society Board of Trustees.  The
    Eric> President of the Internet Society shall acknowledge such an
    Eric> appeal within two weeks, and shall at the time of
    Eric> acknowledgment advise the petitioner of the expected
    Eric> duration of the Trustees' review of the appeal.  The
    Eric> Trustees shall review the situation in a manner of its own
    Eric> choosing and report to the IETF on the outcome of its
    Eric> review.

--Sam

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 27 Jan 2005 08:36:56 -0500

Hi Eric,

>    appealable on the grounds that the decisions were simply bad ones.

I think that something along these lines could meet all of my 
"principles".  BTW, I don't feel that I have any special right to set 
the principles for this mechanism, so folks should feel free to add 
new ones or argue with the ones I put forward...

Your proposal certainly meets the following ones:

(2) It is well-enough specified (here and in RFC 2026) that person 
who is not a (past or present) member of the I* could use it.

(4) Any member of the community can make a review request (or appeal) 
and get a response.

(5) There is at least one level of escalation possible.  (In fact, 
there are three.)

Your criteria are different than mine, and therefore there may be 
some disagreement on:

(3) Review requests should be limited to situations where the IAOC 
violates written procedures (their own or a BCP) and/or makes a 
decisions that is against the best interests of the IETF.

You only want to allow appeals based on the fact that a published 
rule was not followed, and I would like to allow appeals based on 
both of the above criteria.  I don't know how close or how far we are 
on this...  My idea is that the IAOC should not be able to make 
decisions that are damaging to the IETF without any community 
recourse, just because there is no written rule that prevents it.

If we are going to use the full appeals chain (which I totally 
support), I would like to see appeals based on the best interests of 
the IETF constrained to being escalated to the IESG and IAB (since 
they are chosen by the IETF) and only allow process-based appeals to 
be escalated to the ISOC BoT -- much as we do today for technical vs. 
process appeals of WG decisions.  I do not think that it is the ISOC 
BoT's place to be the ultimate arbiter what is in the best interests 
of the IETF.

We also seem to have some disagreement on:

(1) A review request cannot overturn a signed contract or hiring decision.

And

(6) The IAB, IESG and ISOC BoT (in hearing an appeal) cannot overturn 
a decision of the IAOC, only advise the IAOC that they believe that 
an incorrect decision was made.

I feel quite strongly about principle (1), as I don't know how the 
IAOC could negotiate and administer contracts if they might, at any 
time, be overturned by someone else.  And, I don't even know what it 
would mean to "overturn" a signed contract, since it would then be 
outside the control of IASA to change it.  I do think that the IAOC's 
contract negotiation rules should include a period of public comment 
on major decisions before they sign binding contracts or MOUs, but 
that is a subject for future IAOC operating procedures, not for this 
document, I guess.

I am actually not strongly in favor of principle (6) myself.  I think 
that the IAB, IESG and ISOC BoT could be trusted to decide whether 
overturning a particular (non-binding) decision is appropriate in a 
particular situation.  But, others seemed to feel strongly that 
allowing anyone else to overturn any decision of the IAOC would be 
very bad.  I can't say that I fully understand why.

Please note that principle (6) does allow the IAOC to overturn an IAD 
or IAOC decision based on a review request/appeal.  It also allows 
the IAOC to overturn a decision based on advice from the IAB, IESG or 
ISOC BoT that it would be best to do so.  So,  I am comfortable with 
this one either way.

Do you have a strong opinion on this one way or the other?

Margaret

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 27 Jan 2005 08:10:12 -0600

> I am actually not strongly in favor of principle (6) myself.  I 
> think that the IAB, IESG and ISOC BoT could be trusted to decide 
> whether overturning a particular (non-binding) decision is 
> appropriate in a particular situation.  But, others seemed to feel 
> strongly that allowing anyone else to overturn any decision of the 
> IAOC would be very bad.  I can't say that I fully understand why.

Hi, Margaret,

There are probably a variety of reasons why people might have this 
view, but mine are:

- if we have people on IAB/IESG who are better at making IAOC 
decisions than the IAOC is, we have the wrong people on the IAOC (or 
perhaps the wrong people on IAB/IESG, but that's another story), and

- some decisions are always bad ("let's start a land war in Asia"), 
but most bad decisions are more obviously bad when looking back, so 
we're usually talking about second-guessing based on hindsight.

So it's not that I don't trust the IAB/IESG to decide whether 
overturning a decision is appropriate in a particular situation, it's 
that I don't like what that says about our structure and processes.

Saying, "you guys didn't think about X when you made this decision, 
and you should think about it the next time this decision matters" is 
fine with me.

Spencer

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 27 Jan 2005 06:47:25 -0800

Margaret Wasserman <[email-masked]> writes:
> (2) It is well-enough specified (here and in RFC 2026) that person who
> is not a (past or present) member of the I* could use it.
>
> Your criteria are different than mine, and therefore there may be some
> disagreement on:
>
> You only want to allow appeals based on the fact that a published rule
> was not followed, and I would like to allow appeals based on both of
> the above criteria.  I don't know how close or how far we are on
> this...  My idea is that the IAOC should not be able to make decisions
> that are damaging to the IETF without any community recourse, just
> because there is no written rule that prevents it.
The problem is that "best interests of the IETF " is a completely
amorophous standard (In my view, chocolate helps people think
better so we need chocolate chip cookies in order to produce
better standards), so I don't seee how this rules out any appeals
at all. 

> a decision of the IAOC, only advise the IAOC that they believe that an
> incorrect decision was made.
>
> outside the control of IASA to change it.
Yes, I think this is absolutely correct.

> or IAOC decision based on a review request/appeal.  It also allows the
> IAOC to overturn a decision based on advice from the IAB, IESG or ISOC
> BoT that it would be best to do so.  So,  I am comfortable with this
> one either way.
>

I could live either way, but my original thought was that in the
case of allegations that the IAOC had not followed its internal
rules, the IESG and IAB could overturn its decisions. Certainly,
they can impose new rules on it through the RFC process....

-Ekr