Subject: Re: Updating BCP 10 -- NomCom

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 10 Feb 2015 23:40:08 -0800

Cycling back around since the threads died off:

On Fri, Jan 9, 2015 at 10:40 AM, Allison Mankin <[email-masked]>
wrote:

> 3. A bit fuzzier, but I think important, would be to include some guiding
> language for Nomcoms about their task as hiring managers recruiting and
> selecting leadership for the IETF.  I found it very important to advise my
> Nomcom to use "slow thinking"  (in Kahneman's sense) and make sure to
> really digest the resumes, questionnaires and feedback, because the pace of
> the Nomcom and human nature tend to result in "fast" or intuitive dismissal
> of some really good candidates based on e.g. their having less fame.
> Similarly, without constraining the exact procedures, it is important to
> explicitly find ways to make interviews consistent across candidates and to
> have enough of a panel of interviewers.  The pace and stress of scheduling
> interviews means it's important to put these goals front and center,
> because they are actually quite challenging to accomplish.
>
> I can propose some text for these.
>

Hi Allison,

I think I've nailed down the other two, but I would really like some text
for this last one.  You probably have a much better idea of what you'd like
to see here than I do.

I may post -02 tonight so people can see the other proposed changes.  If I
do that, I can catch your text in -03.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Feb 2015 00:50:41 -0800

This thread died off without reaching consensus on how eligibility would be
measured beyond the 3/5 rule.  In particular:

On Fri, Jan 9, 2015 at 11:03 AM, Michael Richardson <[email-masked]>
wrote:

>
> I would like to change the nomcom eligibility criteria.
> SM has proposed some things awhile ago in:
>         draft-moonesamy-nomcom-eligibility-01
>
> kept the current rules of 3/5, but added options where
> the "3rd" meeting could really be in the form either having
> been to a lot of meetings, or having used day-passes..
>
> I don't think SM's proposal does the right thing.
> My concern is primarily about people who enter our culture,
> and then for some reason are unable to travel. (Could be health,
> could be inability to get VISAs, could be funding, could be children)
>
> So I would keep the 3/5 in-person meetings to *become* nomcom
> eligible.
>
> Once eligible, the rules for remaining eligible would be different.
> I would propose something like having *contributed* to at least two
> meetings in the past four.  We could come up with complex or simple
> rules on what it means to contribute, we could automated it, and
> we can discuss all the ways that various rules could be gamed.
> My ideas for contribution would include:
>   0) attend the meeting in person.
>   1) be a document shepherd or working group chair on a document
>      that entered AUTH48.
>   2) be the document uploader (pressed submit) on a document that
>      was scheduled into a WG session. (A document authors that has
>      never been to a meeting would never have become eligible. If
>      document authors want to rotate who submits, that actually
>      seems like a good idea if it keeps their hand in, as I've had to
> almost
>      stalk some co-authors during AUTH48 who seem to have fallen off the
>      planet)
>   3) opened a ticket on a document that was scheduled into a WG session.
>   4) scribed for the I* telechats.
>

I definitely get the intent here and I see that a lot of people like the
idea, but the details seem to fall short to me so far.  Can we bang on it a
little more so I can come up with text to include?

Setting aside for the moment what exactly we consider "contributed", here's
a hypothetical:

For IETF 92, one would become eligible by attending at least three of IETFs
87, 88, 89, 90, and 91, just like it has been all along.  Now I'm
"eligible".  Does that last forever, or should it expire after some period,
requiring another 3/5 in-person to restore eligibility?  (I'm reminded of
non-airline pilot re-certification, which at least in the US has to be done
every two years regardless of how much you have or have not been flying.)

Let's suppose it's perpetual, and then I disappear for three years (nine
meetings).  Now I reappear at IETF 104, which will probably be in
Minneapolis.  Since IETF 91, I opened a single ticket on a document in
DNSOP around the time of IETF 100, and I acted as IAOC scribe once around
the time of IETF 103.  I have paid no attention whatsoever in the
intervening years to ietf@, to any administrative or technical plenary,
gone to any of the working groups or training sessions (even remotely),
participated in no hallway track discussions, and not otherwise engaged in
any way.  Should I be eligible to serve on the NomCom as a selecting member?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Feb 2015 00:54:52 -0800

On Sun, Jan 11, 2015 at 12:11 PM, Allison Mankin <[email-masked]>
wrote:

> I like Michael's original suggestion that there should be two meetings in
> the last five and if there is not a third, then meeting some other
> requirement, a choice of things like attended lots of past meetings, has a
> defined role in a wg, ...
>
> Others have noted this, but the two important goals of these criteria are:
>
> Has attained a (minimal)  level of familiarity with IETF
>
> Has a probability of attending in person for the Nomcom activities
>
> Michael has crystallized in other posts the fact that Nomcom really needs
> the in person time in order to work together, and IETF teally, critically
> needs full attendance if Nomcom during the meeting(s) with interviews,
> especially November, for the candidate interviews.
>
Having served on the last two NomComs, I have to say I agree with these
points, especially the in-person one.  NomCom would be pretty hard to do
remotely.  I know we're making great strides in remote participation in
working groups, but those are very open things while NomCom is not.
There's a lot of value in being available just walking around the hallways
when serving as a NomCom member, something that can't be done quite as
easily when you're far away.

-MSK

---

From: Allison Mankin <[email-masked]>
Date: Wed, 11 Feb 2015 04:24:04 -0500

Murray,  see if these paragraphs work in the draft.  I've covered the
points that I suggested in my point 3 that you quoted.  Thanks very much
for the ping!!

Section ??  - Hiring Manager Responsibilities of the Nomcom

The voting members of the Nomcom serve as hiring managers for the
leadership of the IETF.  That responsibility includes not only assessing
whether those nominated are technically capable of the leadership roles in
question but also whether they will serve well to lead others and work
synergistically in the bodies, the IESG, the IAB, and the IAOC.  To make
good judgements on these axes, the Nomcom needs to cultivate very good
listening (to interviews, to feedback).  Even more so, the Nomcom should
cultivate their skills of  "slow thinking" as they evaluate, interview and
deliberate.  The Nomcom should be careful to really digest the resumes,
feedback and other input about candidates, because otherwise it is too easy
to pick a familiar or "famous" candidate when a less well-known candidate
may have a great deal more to offer.  This term "slow thinking" comes from
the 2011 book "Thinking Fast and Slow" by the economics Nobelist Daniel
Kahneman.  Slow thinking refers to careful, thorough, deliberative
processes of thought, in contrast to rapid judgements, intuition, gut
feelings, all of which make up fast thinking.  During fast thinking,
unconscious biases have extra sway so that equal or more competent nominees
may be dismissed too quickly compared with nominees who are "known
quantities."   Fast thinking is likely to result in not truly digesting and
perceiving every candidate's skills - based on gut feelings, nominating
panels give less time to the unfamiliar resumes, are less conscious of all
the qualifications of the less familiar candidates.  It is very important
for the IETF as an organization that excellent nominees not be
inadvertently overlooked.

Section ?? - A Note about Interviewing

Interviews that the Nomcom decides to conduct need to be carefully planned
and organized to emphasize fairness and consistency to the candidates.
While the specific procedures will be determined by each Nomcom aided by
the Nomcom Chair, here are some overarching principals:  it is advisable to
prepare a starting slate of questions that each nominee in particular
positions will all be asked; this ensures a basic fairness and also helps
to make interviews complete.  Each interview should have sufficient
participants, including a set minimum number of voting members.  One
approach to organizing interviews is for the Chair to assign a Lead, a
Scribe, and one or more Observers for each.  The outcome of having
consistently sized and organized interview panels is that all nominees will
receive fair amounts of attention and sufficient efforts on interview
reports to be shared with the rest of the Nomcom.

---

From: John C Klensin <[email-masked]>
Date: Wed, 11 Feb 2015 07:40:41 -0500

--On Wednesday, February 11, 2015 00:50 -0800 "Murray S.
Kucherawy" <[email-masked]> wrote:

> Let's suppose it's perpetual, and then I disappear for three
> years (nine meetings).  Now I reappear at IETF 104, which will
> probably be in Minneapolis.  Since IETF 91, I opened a single
> ticket on a document in DNSOP around the time of IETF 100, and
> I acted as IAOC scribe once around the time of IETF 103.  I
> have paid no attention whatsoever in the intervening years to
> ietf@, to any administrative or technical plenary, gone to any
> of the working groups or training sessions (even remotely),
> participated in no hallway track discussions, and not
> otherwise engaged in any way.  Should I be eligible to serve
> on the NomCom as a selecting member?

Let me turn this question around, introducing a possibility that
we have no way to measure but that anyone who watches meetings
carefully knows happens.  Suppose a hypothetical individual,
Elmer, attended all of IETF 89, 90, and 91.  His definition of
"attended" consists of signing up, paying the registration fee,
showing up at the registration desk to collect a badge, and then
attending the social and/or bits-and-bytes if either is held.
In IETF 89 and 90, he sat in on a few WG meetings and signed the
blue sheets, but spent the time reading email, text-chatting
with friends, and contributed absolutely nothing to the
discussion, not even paying enough attention to hums.  At IETF
91, he spent the week on the beach.  He has never posted
anything substantive to the IETF list or any WG list, although
he has gotten caught up in some of the threads most charitably
described as comedy and contributed to the noise.

I may be exaggerating -- I don't personally know of cases that
bad.  But I have reason to believe that there are
near-approximations out there.

I don't think we want him as a selecting member of the Nomcom
either.  I don't know how to make easy measurements that would
identify him and keep him off if his company decided he'd be
useful to have on the Nomcom to get more of their employees
selected to key roles.  But let's at least try to remember that
3/5 is nothing more than an easy-to-measure but very weak
surrogate for "has a clue about what is going on".   If I had to
make a choice, I'd prefer the scenario Murray outlines to Elmer
although I'm not wild about either.

One other observation: please remember that we've entangled
"Nomcom eligibility" with a number of other things including,
IIR, eligibility to apply for other positions and the ability to
participate in attempts to remedy gross injustice or malfeasance
by recalling members of various bodies.  We either need to
decouple those things or we need to consider how changing (or
not changing) Nomcom eligibility might affect them.

     john

---

From: Loa Andersson <[email-masked]>
Date: Wed, 11 Feb 2015 20:48:35 +0800

John,

> not changing) Nomcom eligibility might affect them.

Why would Elmer ever put his name into the NomCom election? Wouldn't
that be committing to do real work?

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: Dave Crocker <[email-masked]>
Date: Wed, 11 Feb 2015 05:44:30 -0800

> I don't know how to make easy measurements that would
> selected to key roles. 

Swap the focus.  Rather than seeking to characterize the clueless, let's
characterize folk who are much more likely to be clueful:

     Ensure that at least a portion of the Nomcom has contributed in
formal, documented way.  AD, Chair, author.. whatever set of activities
we feel makes it likely they will have clue.

As for Loa's question about why someone who hasn't done real IETF work
would want to volunteer, the answer is politics and/or ego.  They or
their company might want to hold sway over nomcom or the person might
just want to add this to their resume.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Loa Andersson <[email-masked]>
Date: Wed, 11 Feb 2015 22:26:40 +0800

Dave,

> characterize folk who are much more likely to be clueful:

Would be a blessing :)
> we feel makes it likely they will have clue.

I guess I'm old-time minded here, what is wrong with 3 meetings out
of 5, and if you want co-authored document.
> just want to add this to their resume.

With due respect - I don't think this is how it work, do we have any
running code? Our rules so far has not stopped anyone form paying and
register, sit at the back of a a couple of wg meetings and three IETF
meetings later drop his/her name into the hat. But has it happened?

/Loa

--

---

From: Ted Lemon <[email-masked]>
Date: Wed, 11 Feb 2015 09:35:09 -0500

> of 5, and if you want co-authored document.

The main issue with three out of five is that it discriminates against people who can't attend that often but are actively involved in the IETF.   E.g., you have kids, or you have a family member with an illness who needs care, so you stay home for a year, and suddenly you're disenfranchised, even though you were actively involved in the IETF that whole time.

> meetings later drop his/her name into the hat. But has it happened?

The operation of each nomcom are pretty opaque to those who are not on it.   For those who have interacted with a nomcom as candidates, such an impression might exist.   It's possible that nomcom liaisons or chairs could speak to this.   However, since nomcom proceedings are supposed to be confidential, I don't know how much they could really say.   Because these properties of the nomcom are intentional and useful, it does make sense to be particularly careful about how nomcom eligibility is determined and not just trust to peoples' good natures.

---

From: Loa Andersson <[email-masked]>
Date: Wed, 11 Feb 2015 23:43:29 +0800

Ted,

> The main issue with three out of five is that it discriminates against people who can't attend that often but are actively involved in the IETF.   E.g., you have kids, or you have a family member w…

Well a nice try - do we have hard figures? Every rule discriminates
against someone, some of those people I want to discriminate against
others I don't want to discriminate against.

Let us say that we have 1200 attendees per meeting, the group you are
talking is 1% of the 1200, we are talking about 12 people.

But this could be calculated, go back 15-20 meetings and see how many
has been disenfranchised by a 3 out of 5 rule. ANd after that picked
uo a regular attendance. If it is in the order of 10 I think the
improvement we see in NomCom would be worth it. If we are talking
several times that number we have a problem with such a rule.

> The operation of each nomcom are pretty opaque to those who are not on it.   For those who have interacted with a nomcom as candidates, such an impression might exist.   It's possible that nomcom l…

Sorry I was not talking about the operations of the NomCom, but how
many drones we have in the pool. Looking at a few of the last pools
(the +100) I dare say that the figure is low, if we regardless of that
have problems in the NomCom operations no rule whatsoever will help.

/Loa

--

---

From: Dave Crocker <[email-masked]>
Date: Wed, 11 Feb 2015 07:59:37 -0800

> of 5, and if you want co-authored document.

That's a version of what I'm suggesting.  And I think that requiring
some percentage of nomcom to have that qualification would be better
than our current requirement.  (I'm not commenting on the debate about
registration numerics.  I think it has merit and I think good points are
being made, but don't have a strong opinion about it.)

However I think it would be better still to count other IETF roles too.
 In reality, authors often have little direct contact with the processes
that Nomcom selects people to perform.  On the other hand, most (all?)
folk who do have that direct experience with IETF management processes
have also been authors. (That is, I'm suspecting that the single, simple
criterion you cite might really be sufficient as a superior
clue-correlate...)

> running code?

Unfortunately, yes.  One or more occurrences.  Perceived clearly by a
variety of IETF management folk.  But alas, no, it can't be discussed in
public.

Happily, this sort of thing is rare.  But a position on Nomcom is
disproportionately high leverage and, therefore, needs meaningful
protection against problematic membership, IMO.

d/

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 11:23:51 -0500

> of 5, and if you want co-authored document.

Have a baby (as a mother or father) and see if you can remain nomcom
eligible.  Try this as a single parent. Or if your partner works a job with
rigid hours (doctor, nurse, RT, lawyer, firefighter, police...).
For many of us in technology, we are the "flexible" parent in the team.
Remember that nomcom eligibility also matters for recall.

(I didn't say that you'd want to volunteer for nomcom in the year that you
gave birth; but consider if weren't eligible in year X, you probably won't be
eligible until year X+2)

Be someone with a limited travel budget, who can only attend meetings in
person when they are "near".  Sometimes it's not the dollars in the budget,
but the travel time that is the problem.
We give this as the reason why the IETF travels to people rather than
repeating Minneapolis/Vancouver all the time.  So someone who finds a happy
medium where they can attend 1 meeting/year in person, and they read mailing
lists, etc.  Assuming that they get over the hurdle of 3/5 to become
eligible, why would we want to throw away all that training?

    > meetings later drop his/her name into the hat. But has it happened?

The current system can be gamed if you like, there is no question.

I'm not interested in how we can make the current system harder, or about
this hypothetical Elmer.  If s/he was one of my selecting members, and didn't
come to meetings, then there would be an issue, and the nomcom has ways to
deal with that.

I'm interested in how we can include more people in the nomcom from a more
diverse background.  It's a FIFO problem: if we want diversity at the top,
you have to have diversity at the bottom.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 11:32:38 -0500

> meetings).  Now I reappear at IETF 104, which will probably be in

Assuming that you did not contribute for three years in anyway:
   wrote/submitted no documents, never presented remotely in a meeting, etc.
then at IETF97 (5 meetings after IETF92, assuming you attended that one),
you would become ineligible, and you'd have to attend 3/5 again to become
eligible.  You'd have to come to IETF104,105, and 106.

    > any way.  Should I be eligible to serve on the NomCom as a selecting member?

So, I would claim that you were actually more active than some people who
have in fact attended nearly every meeting.  (and certainly more active than
our hypothetical Elmer)

Alas, everyone reading this thread is probably 10x more active than the IETF
average, so we may have a skewed notion about what it means.

(I wrote contributed twice in the past four meetings. So out of
100,101,102,103, you got 100 and 103.)

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 12:02:20 -0500

Ted Lemon <[email-masked]> wrote:
    > The operation of each nomcom are pretty opaque to those who are not on
    > it.   For those who have interacted with a nomcom as candidates, such
    > an impression might exist.   It's possible that nomcom liaisons or
    > chairs could speak to this.   However, since nomcom proceedings are
    > supposed to be confidential, I don't know how much they could really
    > say.   Because these properties of the nomcom are intentional and
    > useful, it does make sense to be particularly careful about how nomcom
    > eligibility is determined and not just trust to peoples' good natures.

The 2013/2014 had someone who had to be removed because they could not attend
any meetings, and were never able to organize themselves to attend calls.
I think that they weren't Elmer; my poor recollection is that a family member
got ill, and they simply couldn't do much other than their 9-5.  I never met
this person, didn't know who they were.

The take home if that if one does select Elmer, and s/he sits on the beach
rather than coming to the nomcom meetings, the nomcom can boot them out. If
it happens early enough, the nomcom chair can replace them, or the nomcom can
operate with 9 rather than 10.

Allison has suggesting selecting 11 people, with the 11th being a
participating, but non-voting spare.  I'm undecided if this would be a good
thing.  In 2014/2015 I did select an 11th from the pool, and confirmed that
selection with others in case we needed someone else.

---

From: Mary Barnes <[email-masked]>
Date: Wed, 11 Feb 2015 11:18:30 -0600

> selection with others in case we needed someone else.
[MB] I actually really like this idea as it seems to be more the rule than
the exception that one person has to leave the nomcom or just isn't engaged
(I had the latter on the Nomcom I chaired and the former on the one for
which I was past-chair advisor).  So, I think having a backup is a really
good idea.  I would suggest if that happens that each Nomcom should agree
at the start the criteria under which they would add the 11th as a 10th
voting member.   I had a voting member that just wasn't participating at
all for an extended period of time.  I was almost at the point of going
through the process of having them removed as a voting member, but finally
I was able to get some response. But, this situation wasted a lot of time
and does a disservice to the process.
[/MB]

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 13:56:54 -0500

> and does a disservice to the process.

The issue is whether the 11th member (the spare), sits through the
proceedings, goes to the interviews, etc.  If they don't, then they aren't of
much use.... If they *do* it seems like a large burden to do that, and then
not get to vote unless someone gets hit by a bus.

---

From: Mary Barnes <[email-masked]>
Date: Wed, 11 Feb 2015 13:31:01 -0600

> not get to vote unless someone gets hit by a bus.
[MB] It wouldn't be particularly useful if the 11th wasn't involved in the
process as a voting
member would be.  The process has a number of people that invest a lot of
time and effort
that don't get to vote.   Also, as we've seen it's not just getting hit by
the bus - from what I've
seen it is more the rule than the exception that at least one voting member
finds they can no longer serve or
they just aren't investing the time.  I think it would also be possible for
the 11th to participate
in discussions, provide input, etc. and just not vote.  Although, I think
that would be up to the
particular nomcom, as is a number of other decisions in terms of how
engaged anyone that isn't a
voting member is in the process - e.g., some nomcoms actually have the past
chair in interviews.
As chair, I didn't even feel it was a necessity for me to be involved in
all interviews.  I did sit
in on some where we did not have sufficient voting members available.
[/MB]

---

From: Ted Lemon <[email-masked]>
Date: Wed, 11 Feb 2015 14:49:28 -0500

> talking is 1% of the 1200, we are talking about 12 people.

I suspect that what you will find is that of the remote attendees, zero are there to read email in a meeting room.   So I think this 1% figure you've pulled out of your hat is probably not a good one to use: it's likely that a regular remote attendee is very involved in the IETF.

For my own part, now that I'm no longer an AD as of March, I'm seriously questioning whether I want to keep flying to IETF meetings every time.   It's a huge carbon footprint, expensive for my company, and really disruptive to my life.   IETF has been getting better and better at supporting remote attendees.   I would love to be able to skip a couple of IETFs a year, and the nomcom eligibility requirement is a major impediment to being able to do that in good conscience.

(And yes, I recognize that if I wound up on nomcom, attendance during that period would not be optional.)

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 12 Feb 2015 08:57:01 +1300

Being geeks, we have a tendency to invent solutions that
are theoretically wonderful but quite impractical. The current
3/5 rule avoids that and is fairly easy to check algorithmically.

I think we need to keep that simplicity as much as possible. So
something like the following might work.

Either
 3 out of the last 5 meetings
or
 3 out of the last 10 meetings and (RFC author or WG Chair or AD in the last 5 years)

   Brian

---

From: Joel Halpern <[email-masked]>
Date: Wed, 11 Feb 2015 15:24:31 -0500

Given that such an extra person will be inherently included in the 
conversations, I believe that this change is better described as:

Increase the size of the nomcom to 11 volunteers, but keep the minimum 
size and Quorum as they are today.

Now, that may be a good idea.  But we should not pretend it is really 
just a hot standby.

Yours,
Joel

On 2/11/15 2:31 PM, Mary Barnes wrote:
>
>
> On Wed, Feb 11, 2015 at 12:56 PM, Michael Richardson <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>     Mary Barnes <[email-masked]
>     <mailto:[email-masked]>> wrote:
> [MB] It wouldn't be particularly useful if the 11th wasn't involved in
> the process as a voting
> member would be.  The process has a number of people that invest a lot
> of time and effort
> that don't get to vote.   Also, as we've seen it's not just getting hit
> by the bus - from what I've
> seen it is more the rule than the exception that at least one voting
> member finds they can no longer serve or
> they just aren't investing the time.  I think it would also be possible
> for the 11th to participate
> in discussions, provide input, etc. and just not vote.  Although, I
> think that would be up to the
> voting member is in the process - e.g., some nomcoms actually have the
> past chair in interviews.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 15:29:11 -0500

> in discussions, provide input, etc. and just not vote.  Although, I think

I agree.
The other liasons to the process are doing this at the request of their
respective organizations...

    > voting member is in the process - e.g., some nomcoms actually have the past

I made use of all of the liasons and past-chair as observers.

    > in on some where we did not have sufficient voting members available.

The same for me; I tried to sit through at least the first interview for each
time I had a "green" lead person... but I certainly didn't feel I had to be
there the whole time.    In fact, I got rather giddy with (parental?)
satisfaction as I found that the nomcom members were completely exceeding my
expectations in the way they were conducting the interviews.

So, the question is: should the selection of a spare, and the process (that
they be treated as a non-selection-voting observer) be written into BCP10?

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 15:31:35 -0500

> I suspect that what you will find is that of the remote attendees, zero
    > are there to read email in a meeting room.   So I think this 1% figure
    > you've pulled out of your hat is probably not a good one to use: it's
    > likely that a regular remote attendee is very involved in the IETF.

Actually, that's a really good point: anyone who bothers to show up at a bad
timezone to be a remote attendee for a session is probably twice as committed
as someone who is just in the room.

the first two rows are for people who've read the drafts... we should get rid
of the rest of the chairs :-)

---

From: Michael StJohns <[email-masked]>
Date: Wed, 11 Feb 2015 15:45:06 -0500

A couple of things.  

The nomcom is not either individually nor collectively the "hiring manager" for the IETF.  It neither makes the final decision (that's the CB's call), nor does it direct the work of the "hired" entity.

The Nomcom is and always has been a "search committee" and as such it gathers and winnows candidates before proposing them for confirmation.  Indeed, there is no bar to the Nomcom proposing multiple candidates for a position and allowing the CB to choose its preference - that hasn't actually been done, but its not barred.

The interview should be used to fill in details not clear from the questionnaire, so matters of fairness should be addressed when crafting the questions, not later when you get around to talking with the candidate.  Generally, the interview should be used to evaluate the BS factor of the answers from the questionnaire and not delve into new fields or hobby horses of the interviewers.

The digression to "thinking slow" is interesting, but somewhat trendy.  Next year there will be another interview and decision method that will be trendy and its unclear why this "thinking slow" would be a better choice than next year's flavor and why it should be immortalized in the next version of 3777.  I would instead focus on the virtues of clarity, completeness and fairness and leave the rest of it to the best abilities of the chosen Nomcom.  In any event, attempting to "program" the Nomcom members to a specific behavior pattern will be unsuccessful.

Allison - your desire to have 3+ interviewers per interviewee is going to contribute to slowing down the process even more.  At this point I see us starting to try and begin the process a full two years before confirmation. (Yup - hyperbole, but seriously, we're spending WAY too much calendar time from first indication of candidacy to selection).

Mike

At 04:24 AM 2/11/2015, Allison Mankin wrote:

>Murray, Â see if these paragraphs work in the draft.Â  I've covered the points that I suggested in my point 3 that you quoted.Â  Thanks very much for the ping!!
>
>Section ?? Â - Hiring Manager Responsibilities of the Nomcom
>
>The voting members of the Nomcom serve as hiring managers for the leadership of the IETF.Â  That responsibility includes not only assessing whether those nominated are technically capable of the leadership roles in question but also whether they will serve well to lead others and work synergistically in the bodies, the IESG, the IAB, and the IAOC.Â  To make good judgements on these axes, the Nomcom needs to cultivate very good listening (to interviews, to feedback).Â  Even more so, the Nomcom should cultivate their skills of Â "slow thinking" as they evaluate, interview and deliberate.Â  The Nomcom should be careful to really digest the resumes, feedback and other input about candidates, because otherwise it is too easy to pick a familiar or "famous" candidate when a less well-known candidate may have a great deal more to offer.Â  This term "slow thinking" comes from the 2011 book "Thinking Fast and Slow" by the economics Nobelist Daniel Kahneman.Â  Slow thinking refers to careful, thorough, deliberative processes of thought, in contrast to rapid judgements, intuition, gut feelings, all of which make up fast thinking.Â  During fast thinking, unconscious biases have extra sway so that equal or more competent nominees may be dismissed too quickly compared with nominees who are "known quantities." Â  Fast thinking is likely to result in not truly digesting and perceiving every candidate's skills - based on gut feelings, nominating panels give less time to the unfamiliar resumes, are less conscious of all the qualifications of the less familiar candidates.Â  It is very important for the IETF as an organization that excellent nominees not be inadvertently overlooked.Â 
>
>Interviews that the Nomcom decides to conduct need to be carefully planned and organized to emphasize fairness and consistency to the candidates.Â  While the specific procedures will be determined by each Nomcom aided by the Nomcom Chair, here are some overarching principals: Â it is advisable to prepare a starting slate of questions that each nominee in particular positions will all be asked; this ensures a basic fairness and also helps to make interviews complete.Â  Each interview should have sufficient participants, including a set minimum number of voting members.Â  One approach to organizing interviews is for the Chair to assign a Lead, a Scribe, and one or more Observers for each.Â  The outcome of having consistently sized and organized interview panels is that all nominees will receive fair amounts of attention and sufficient efforts on interview reports to be shared with the rest of the Nomcom. Â Â 
>
>On 11 February 2015 at 02:40, Murray S. Kucherawy <<mailto:[email-masked]>[email-masked]> wrote:
>On Fri, Jan 9, 2015 at 10:40 AM, Allison Mankin <<mailto:[email-masked]>[email-masked]> wrote:
>3. A bit fuzzier, but I think important, would be to include some guiding language for Nomcoms about their task as hiring managers recruiting and selecting leadership for the IETF.Â  I found it very important to advise my Nomcom to use "slow thinking" Â (in Kahneman's sense) and make sure to really digest the resumes, questionnaires and feedback, because the pace of the Nomcom and human nature tend to result in "fast" or intuitive dismissal of some really good candidates based on e.g. their having less fame.Â  Similarly, without constraining the exact procedures, it is important to explicitly find ways to make interviews consistent across candidates and to have enough of a panel of interviewers.Â  The pace and stress of scheduling interviews means it's important to put these goals front and center, because they are actually quite challenging to accomplish.
>
>I think I've nailed down the other two, but I would really like some text for this last one.Â  You probably have a much better idea of what you'd like to see here than I do.
>
>I may post -02 tonight so people can see the other proposed changes.Â  If I do that, I can catch your text in -03.
>
>-MSK
>

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 15:48:52 -0500

> 3 out of the last 10 meetings and (RFC author or WG Chair or AD in the

When you say "RFC author", I guess you mean something which went through the
process to get a number.  I guess that means that having your name on the
front page now has more significant value now.   If two IDs merge, now it
matters who is an editor, and who is in "Contributors" at the end...

I would like to know why my previous list was too long or unclear?
I specifically listed:

"WG chair" was listed above, and I added "that entered AUTH48", because I
wanted to deal with the possible way to game things where a WG is created as
a sinecure. No dead WG.  It's the act of going through AUTH48 that I care
about, not who you were. [The authors of the document get caught by 2]

And I included shepherd, because I want to provide more opportunities for
more people; if you attend meetings rarely, you are unlikely to be visible to
ADs for selection as an WG chair, but shepherds are in short supply.
And all the above things are trivially checked in the datatracker.

  2) be the document uploader (pressed submit) on a document that
     was scheduled into a WG session.

So anyone can upload at any time.  If the WG takes your document on,
then the datatracker clearly knows this, so that part is easy to check.
If you draft-author- is scheduled, this will require some additional
datatracker effort for the WG chairs to indicate that your document goes into
the agenda... but, I we wanted to be able to track which documents were
discussed into which WG anyway for all sorts of reasons, so I think that his
work will already happen.

While the Issue tracker is not well integrated into the datatracker, I know
that many want it to be, and also that many want many other improvements.
Again, knowing that an issue was discussed at a WG session is pretty
important thing to track, I think... so really the datatracker OUGHT to know this.

This requires secretariat/I* action to confirm, and is frankly probably my
least interesting way.

Now, I can see how "AD" could make one eligible, and how an AD after a 6
year term, could easily become non-eligible having taken a break. But:

     I don't like AD this way because it looks too much like the previous
     IESG is selecting the new IESG. Yeah, random selection.

     Since former ADs are well known, often well spoken (thus causing them to
     write IDs), and are often asked to join directorates and do reviews
     (therefore opening issues), and might even be really really good
     shepherds...  I prefer rules about what you do, not who you are/were.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Michael StJohns <[email-masked]>
Date: Wed, 11 Feb 2015 15:58:50 -0500

>have problems in the NomCom operations no rule whatsoever will help.

It would be interesting to take a look at the "large company" pool volunteers over say the past 20 years and figure out a way of giving them an activity factor (e.g. attendee vs contributor) - but finding an objective scale that we could all agree with to assign such activity factor would be difficult.

I say large company because about 55-60% (rough number - just from scanning) or so of the voting members of the Nomcoms over the last 20 years appear have come from companies with large numbers of attendees at the IETF - said companies tending to be what I think of as "big".   

The other number to look at might be the number of attendees (percentage wise) per company per meeting vs number of volunteers (percentage wise) per company per nomcom.  All things being equal I would expect those percentages to be close to identical.  I would expect where the nomcom volunteer percentage exceeds the attendee percentage to maybe be indicative of a desire by the company to place members on the Nomcom.

Later, Mike

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 16:14:59 -0500

Michael StJohns <[email-masked]> wrote:
    > Allison - your desire to have 3+ interviewers per interviewee is going
    > to contribute to slowing down the process even more.  At this point I
    > see us starting to try and begin the process a full two years before
    > confirmation. (Yup - hyperbole, but seriously, we're spending WAY too
    > much calendar time from first indication of candidacy to selection).

I can not agree with you.

If anything, slow thinking and good interviews contribute to *FASTER*
decision making, because you don't have two people who jumped to a conclusion
fighting each other for hours/days.

My Nomcom finished all it's work a week before Xmas, the rest of the delay is
due to CB discussions, and just simple CB scheduling.
The previous one (Allisons) was about the same.

The one before (Matt) took much longer because the transport issue.

As for your notion that the nomcom could provide the CB with two from which
it could select: ha.  That's fiction.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 16:19:31 -0500

Michael StJohns <[email-masked]> wrote:
    > It would be interesting to take a look at the "large company" pool
    > volunteers over say the past 20 years and figure out a way of giving
    > them an activity factor (e.g. attendee vs contributor) - but finding an
    > objective scale that we could all agree with to assign such activity
    > factor would be difficult.

Agreed.
There is also an interesting scale at which you will get *NO* volunteers
from.  Those are companies just large enough to support an AD or an IAB
member, but not a lot of other in-person participation.  Since you can't
stand for a position if you are on the nomcom, those people won't voluneer.

so, when you do the analysis of activity, you have to also include all the
nominees into the pool.

    > The other number to look at might be the number of attendees
    > (percentage wise) per company per meeting vs number of volunteers
    > (percentage wise) per company per nomcom.  All things being equal I
    > would expect those percentages to be close to identical.

I anecdotally observe that I think that Google has a lower number of
volunteers for a "big" company.

    > I would
    > expect where the nomcom volunteer percentage exceeds the attendee
    > percentage to maybe be indicative of a desire by the company to place
    > members on the Nomcom.

It would also be useful to take, rather than attendance, instead, draft
authorship for the numerator.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 16:23:12 -0500

On this topic, I asked the secretariat:

> Putting aside the problems of different emails used to register,  is it a
> simple query against the registration database (which I think is not in
> anyway connected to the datatracker), to determine how many people are
> nomcom eligible at a given time?  (I don't care who they are)

> Can this run from a historical point of view?

> Regardless, if we can run this, could we run it three times a year (after
> each meeting), and capture that data?

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 12 Feb 2015 10:29:57 +1300

> I would like to know why my previous list was too long or unclear?

For me there is an over-arching consideration: a relatively simple
script must be able to generate the list of all qualified volunteers
from existing databases. Otherwise the whole process will be
impractical.

Inevitably that results in a somewhat blunt instrument.

   Brian

---

From: John C Klensin <[email-masked]>
Date: Wed, 11 Feb 2015 16:53:07 -0500

--On Wednesday, February 11, 2015 20:48 +0800 Loa Andersson
<[email-masked]> wrote:

>...
 
>> Let me turn this question around, introducing a possibility
>> that we have no way to measure but that anyone who watches
>> meetings carefully knows happens.  Suppose a hypothetical
>> individual, Elmer, attended all of IETF 89, 90, and 91.  His
>> definition of "attended" consists of signing up, paying the
>> registration fee, showing up at the registration desk to
>...

Remember there is an epidemic disease in a number of standards
bodies in which the people whom some companies send to those
bodies are not only the group whom Marshall Rose famously
referred to a "go-ers" but are chosen from  the ranks of folks
who aren't much good for anything else but are too hard to get
rid of.   Those folks are often expected to serve the function
of acting to protect the company against Something Bad happening
in the standards body and are doing their jobs as long as Bad
things don't occur or they provide early warning. Some of the
companies who play those games have not yet figured out that the
IETF isn't one of those standards bodies (or have forgotten, or
think the IETF should be made into one of them).

That role is entirely consistent with "Elmer" -- loosely
watching a WG or two of interest but not actually doing much of
anything, certainly not contributing technically (which might be
outside his skill set anyway).

But, if the company has decided that it is desirable to have
people on the Nomcom who will reliably support the company's
agenda (which might be to get company employees or others
sympathetic to the company and its products on the IESG or IAB),
then telling Elmer to volunteer would be an entirely reasonable
move.   He can presumably follow directions, he is useless for
any of the company's actual technical work (so the opportunity
cost of committing him to the Nomcom is zero), and, if he isn't
terribly helpful to the Nomcom's work, that might be a problem
for the Nomcom and the community, but isn't a problem for either
Elmer or his organization.

     john

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Feb 2015 14:10:01 -0800

> Assuming that you did not contribute for three years in anyway:

Not three but four, right?

> eligible.  You'd have to come to IETF104,105, and 106.

OK, so just to confirm:

(a) Attend 3/5 to become eligible.  That's easy to verify from the records
we keep; you either did or you didn't.

(b) For maintaining eligibility, we need to nail down how this gets
evaluated.  Is it something like this?

After attaining initial eligibility, eligibility is maintained by
"contributing" at least twice during every calendar four-year period.  This
is essentially a sliding window four years wide, during which there must be
two recorded contributions for every consecutive four-year window in order
for eligibility to continue.

-MSK

---

From: Joel Halpern <[email-masked]>
Date: Wed, 11 Feb 2015 17:26:36 -0500

I am starting to sympathize with Mike.  What is our goal here.
Expanding the nomcom pool with folks who may be contributing but do not 
have visibility to how things are working does not seem to help us.

Given that most companies that appear to be trying to get a lot of 
nomcom members also send a varying range of people, and that much of the 
constraint on nomcom volunteering seems to be willingness, I do not see 
any indication that a larger available pool will reduce the proportion 
each of the 5 or so large companies offer up.

And while contributing is a good indicator of caring, it is an even 
worse indicator than attendance of having any visibility to the 
community or the management challenges, or any of the other criteria I 
think each of us have in our heads for good nomcom members.

To paraphrase my earlier note, if we can expand the set of qualified and 
highly suitable nomcom members (and get them to volunteer) that does 
seem desirable.  I am trying to relate the conditions to the goal.

Yours,
Joel

On 2/11/15 5:10 PM, Murray S. Kucherawy wrote:
> On Wed, Feb 11, 2015 at 8:32 AM, Michael Richardson
> <[email-masked] <mailto:[email-masked]>> wrote:
>
> (a) Attend 3/5 to become eligible.  That's easy to verify from the
> records we keep; you either did or you didn't.
>
> "contributing" at least twice during every calendar four-year period.
> This is essentially a sliding window four years wide, during which there
> must be two recorded contributions for every consecutive four-year
> window in order for eligibility to continue.
>
> -MSK

---

From: Ted Lemon <[email-masked]>
Date: Wed, 11 Feb 2015 17:42:05 -0500

> work will already happen.

I worry that this criterion will result in people being even more aggressive about scheduling talks on documents that were never discussed on the mailing list.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Feb 2015 16:05:21 -0800

On Wed, Feb 11, 2015 at 2:26 PM, Joel Halpern <[email-masked]> wrote:

> [...]

There was plenty of support for, and no objection to, the idea of relaxing
how one remains eligible after becoming eligible.  It's the mechanics of
this that seem to be the problem.  It's exacerbated by the fact that it's
hard to evaluate programmatically one's contributions other than purely by
counting them.

I could be WG chair of a WG that exists for five years, but never meets and
rarely produces anything.  The datatracker shows me as a co-chair all that
time.

I could be a WG co-chair for ten years where my co-chair does all the work
and I am but a well-liked slacker.  The datatracker still shows me as a
co-chair all that time.

I could be editor of a major RFC along with others, one that takes years to
get to publication, and my only contribution was to write one major
paragraph that's unchanged since the -00 version years ago.  Still, my name
is front page on a standards track RFC.

I could scribe for the IAOC every time and do absolutely nothing else.
This gives me depth on one topic, but no breadth on any other, yet I can
still claim scribe credits.

I could have the same count of WG chairships, documents, scribings, etc.,
as someone else, but still have a frighteningly poor understanding of how
the IETF operates compared to that other person because I skate by in terms
of my contribution while the other person is fully involved.  However,
numerically, we are equivalent.

etc. etc.

As has been observed, any rule we define here can be gamed.  If we want to
solve for this, the problem then is that evaluating each case claiming
qualification now puts a burden on the Chair, or the Secretariat, or
someone else, to decide which claims "count" and which don't, and be able
to support that evaluation.  If I may invoke some grade school rhetoric:
"Not it!"

On the other hand, maybe we should remember that these are merely the
criteria to get into the pool of potential volunteers.  It doesn't land you
a NomCom slot; you still have to get randomly selected.  So maybe the
percentage of people that skate by as described above will generally be low
enough that we shouldn't let perfect be the enemy of the good.

And if it seriously doesn't work, we can always change it again.  I'll bet
it would be at most one NomCom cycle before a terrible selection process
gets replaced.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Feb 2015 16:46:46 -0800

On Wed, Feb 11, 2015 at 12:45 PM, Michael StJohns <[email-masked]>
wrote:

>  A couple of things.
>
> The nomcom is not either individually nor collectively the "hiring
> manager" for the IETF.  It neither makes the final decision (that's the
> CB's call), nor does it direct the work of the "hired" entity.
>
> The Nomcom is and always has been a "search committee" and as such it
> gathers and winnows candidates before proposing them for confirmation.
> Indeed, there is no bar to the Nomcom proposing multiple candidates for a
> position and allowing the CB to choose its preference - that hasn't
> actually been done, but its not barred.
>

I think that's a fair point.  I've adjusted her suggested text accordingly.

> The interview should be used to fill in details not clear from the
> questionnaire, so matters of fairness should be addressed when crafting the
> questions, not later when you get around to talking with the candidate.
> Generally, the interview should be used to evaluate the BS factor of the
> answers from the questionnaire and not delve into new fields or hobby
> horses of the interviewers.
>

The questionnaire is actually not codified in BCP 10 anywhere.  Does anyone
think it needs to be?

> The digression to "thinking slow" is interesting, but somewhat trendy.
> Next year there will be another interview and decision method that will be
> trendy and its unclear why this "thinking slow" would be a better choice
> than next year's flavor and why it should be immortalized in the next
> version of 3777.  I would instead focus on the virtues of clarity,
> completeness and fairness and leave the rest of it to the best abilities of
> the chosen Nomcom.  In any event, attempting to "program" the Nomcom
> members to a specific behavior pattern will be unsuccessful.
>

I tend to agree with this, though since we do have an appendix that talks
about "oral tradition", there's probably little harm in including this
stuff in a similar appendix.  We could even label it clearly as something
that worked well for NomCom 2013.

> Allison - your desire to have 3+ interviewers per interviewee is going to
> contribute to slowing down the process even more.  At this point I see us
> starting to try and begin the process a full two years before confirmation.
> (Yup - hyperbole, but seriously, we're spending WAY too much calendar time
> from first indication of candidacy to selection).
>

I also disagree with this.  I served on NomComs 2013 and 2014, we did it
this way, and it worked rather well.

-MSK

---

From: John C Klensin <[email-masked]>
Date: Wed, 11 Feb 2015 20:30:22 -0500

--On Wednesday, February 11, 2015 09:35 -0500 Ted Lemon
<[email-masked]> wrote:

> The operation of each nomcom are pretty opaque to those who
> are not on it.   For those who have interacted with a nomcom
> as candidates, such an impression might exist.   It's possible
> that nomcom liaisons or chairs could speak to this.   However,
> since nomcom proceedings are supposed to be confidential, I
> don't know how much they could really say.   Because these
> properties of the nomcom are intentional and useful, it does
> make sense to be particularly careful about how nomcom

+1

And, again, unless we start changing other things, we really
aren't just talking about the Nomcom.  We use Nomcom eligibility
for some other things where "disenfranchised" is more direct
than being ineligible to volunteer for a pool from which Nomcom
members are selected at random.

    john

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 21:56:37 -0500

> from existing databases. Otherwise the whole process will be

I note that we have no such script today.

The nomcom chair takes a list of volunteers, and submits them to a web site
one at a time.  Then if the person isn't eligible, contacts them to find out
if they have any other emails that they registered with (such as because they
changed companies), adds that, and iterates.

Except that I wrote some perl to drive this from a CSV file, and screen
scrape the result, and I've asked that it be made into an API.

The registration database is presently not integrated into the datatracker;
the datatracker actually knows a reasonable history of people's email
addresses, or at least, if we integrated things better, then the registration
system could link things better.

One of the reasons I wrote the list the way I did is because I know that the
datatracker can generate that list; the list of attendees is actually the
hardest part to automate.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 22:10:37 -0500

> records we keep; you either did or you didn't.

Agreed.  Recording that somewhere is actually useful for use with recall
processes that need nomcom eligible to sign.  Then people can actually find
out if they are on the list.

    > "contributing" at least twice during every calendar four-year period.

I would write that one should contribute at least three times during every
calendar year, and say that that attendance counts.
To remain eligible, I had that one needed to attend once a year; it might be
worth making that some kind of sliding window of three.  While I am open to
rates of attendance less than 1/3, I don't think the community is.

The situation where we might like to think a bit about is where someone
attends meeting IETF92 (in March 2015), and then misses four meetings in a
row (contributing in other ways the whole time), attending IETF97 and
IETF98.
That's being away for 1.5 years: a difficult pregnancy (such as enforced bed
             rest), plus a year of infant that can't travel.

Just before IETF99, picking for nomcom 2017, one ought to be eligible,
provided that one never became in-eligible.

The question is, just prior to IETF96 (the one in Summer 2016), when the 2016
nomcom is picked, would one be eligible?  One hasn't attended in the past
three meetings, but a) did attend four meetings ago, b) intend to attend
IETF97.   I'd like to think that one could be eligible, but in the case, I
care more that the person doesn't get made in-eligible for 2017.

    > This is essentially a sliding window four years wide, during which
    > there must be
    > two recorded contributions for every consecutive four-year window in
    > order for eligibility to continue.

I'm confused how you came up with four years.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 11 Feb 2015 22:48:54 -0500

mcr> So anyone can upload at any time.  If the WG takes your document on,
   mcr> then the datatracker clearly knows this, so that part is easy to check.
   mcr> If you draft-author- is scheduled, this will require some additional
   mcr> datatracker effort for the WG chairs to indicate that your document
   mcr> goes into
   mcr> the agenda... but, I we wanted to be able to track which documents were
   mcr> discussed into which WG anyway for all sorts of reasons, so I think
   mcr> that his
   mcr> work will already happen.

Ted Lemon <[email-masked]> wrote:
    > I worry that this criterion will result in people being even more
    > aggressive about scheduling talks on documents that were never
    > discussed on the mailing list.

I too worried about that when I wrote it.

I then considered that this is a form of gaming the system, and if the chairs
are participating in it, then this is a problem.  This gaming occurs now
without any (nomcom-eligible) reward, and I think it has other root causes,
which if we could discover we could address.

Or to put it differently: we already have this problem, and we already need
to fix it.

---

From: Ted Lemon <[email-masked]>
Date: Wed, 11 Feb 2015 22:55:08 -0500

> Or to put it differently: we already have this problem, and we already need

True.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Feb 2015 22:23:31 -0800

> I'm confused how you came up with four years.

You said:

Once eligible, the rules for remaining eligible would be different.
I would propose something like having *contributed* to at least two
meetings in the past four

...and I remembered "years" instead of "meetings" for some reason.  Still,
the things you suggested as qualifying contributions don't account for some
other things that I would argue should qualify.  For instance:  Requiring
that I be the submitting author on a document that got WG time disregards
valuable documents that never needed in-person WG time.  The same goes for
the point about opening a ticket against a document that got WG time.

Even better, contributing to the IETF in meaningful ways is not always tied
to meetings.  Some working group chairs and editors do great work for WGs
that never even meet.  Nothing ever gets agenda time in those.

The speed with which something gets to AUTH48 isn't always reliable.  I'm
not sure making that a gating factor would work so well; I could lose my
eligibility just because some directorate review took way too long to come
in.

To repeat: I recognize that we like the general idea here, and I agree.
I'm just having trouble coming up with ways to qualify that aren't
difficult to describe precisely and completely and don't needlessly exclude
people that probably should qualify.

-MSK

---

From: Russ Housley <[email-masked]>
Date: Thu, 12 Feb 2015 01:57:22 -0500

On Feb 11, 2015, at 5:10 PM, Murray S. Kucherawy wrote:

> On Wed, Feb 11, 2015 at 8:32 AM, Michael Richardson <[email-masked]> wrote:
> (a) Attend 3/5 to become eligible.  That's easy to verify from the records we keep; you either did or you didn't. 
> 
> (b) For maintaining eligibility, we need to nail down how this gets evaluated.  Is it something like this?
> 
> After attaining initial eligibility, eligibility is maintained by "contributing" at least twice during every calendar four-year period.  This is essentially a sliding window four years wide, during which there must be two recorded contributions for every consecutive four-year window in order for eligibility to continue.

I'm not accepting that this approach is making things better.

I see the 3/5 attendance record as a means to ensure that the potential NomCom voting member is familiar with the culture of the IETF and that they are aware of the current issues that are facing the IETF, at leas in the portion of the IETF where they are active.  I see (b) as a way to determine continued participation, but it can be done in a fashion that is quite isolated.  In my view, we want to encourage participation in the IETF community.

Russ

---

From: Ted Lemon <[email-masked]>
Date: Thu, 12 Feb 2015 08:02:14 -0500

> I see the 3/5 attendance record as a means to ensure that the potential NomCom voting member is familiar with the culture of the IETF and that they are aware of the current issues that are facing t…

That's why the 3/5 is how you get in, and continued participation is how you stay.   I think your point about IETF culture is right, but it's ridiculous to think that an organization that invented and that continues to invent the Internet has to have in-person carbonfests every four months in order to be effective, and I don't think it's true.   Requiring mostly in-person attendance to maintain cultural relevance is not necessary, and goes against what we have been trying to achieve.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 12 Feb 2015 13:44:19 +0000

> members are selected at random.

I hesitate to offer my opinion, because I'm not a proper IETF participant,
of course.

I've tended to spend my career working for smaller companies, which simply
cannot send that many people to week long meetings around the world three
times a year, so despite that being the important part of the IETF, I have
had to restrict my participation to minor activities, like engaging on the
mailing lists, authoring standards track documents, serving as a working
group chair, and so on. Obviously such insignificant contributions mean I
cannot possibly understand the implications of being on the NomCom.

Nonetheless, and despite not having any idea of the workings of the IETF, I
would think that John is right in his implication that my having a voice in
the recall of an area director - whatever one of those is - would seem to
have some use.

Dave.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 12 Feb 2015 08:44:38 -0500

Murray S. Kucherawy <[email-masked]> wrote:
    > valuable documents that never needed in-person WG time.

I take your point; the reason I wrote it that way is because I am sure that
a document that gets 'air-time' must in some way be important.  But, if your
document goes straight to publication, I guess we could change things in
point (1) about AUTH48, and include authors in that list.

    > Even better, contributing to the IETF in meaningful ways is not always
    > tied to meetings.  Some working group chairs and editors do great work
    > for WGs that never even meet.  Nothing ever gets agenda time in those.

I was trying to capture this into the AUTH48 criteria. The reason I picked
AUTH48 is that the document can sit for months in misref after that point.

    > eligibility just because some directorate review took way too long to
    > come in.

I see your point;  we value writing documents, reviewing them, and
progressing them.  Progressing them is the thing that brings a person most
into contact with the bureaucracy, and leads (in my opinion) to the most
knowledge of how things work (short of being on the nomcom...)

hmm. I guess perhaps being on the nomcom needs to be included as a
contribution...

    > difficult to describe precisely and completely and don't needlessly
    > exclude people that probably should qualify.

One reason I wondered if the list should be contained in BCP10 is that I
wondered if we would find ways to evolve the list.  Of course, we can write a
document that updates BCP10.  I'm thinking that starting by including some
people is better than including none, and that if there is needless
exclusion, we should figure that out, and adjust things.

[having spent way too much time writing emails on this topic in the last 24
hours, including a few emails I decided not to send, I now wonder if I'll be
near the top of Thomas' posting summary.  Then it occured to me just now,
given how few people who came to the plenary actually read ietf@ietf, one
wonders if being on Thomas' list is a + or - :-)

---

From: Michael Richardson <[email-masked]>
Date: Thu, 12 Feb 2015 08:50:19 -0500

> I see the 3/5 attendance record as a means to ensure that the potential
    > NomCom voting member is familiar with the culture of the IETF and that
    > they are aware of the current issues that are facing the IETF, at leas
    > in the portion of the IETF where they are active.  I see (b) as a way

Yes, and it is not proposed to eliminate that.
We need the "indoctrination" phase to get the "pledges" up to speed.
After which one becomes Alumni, and the requirements are less.

Can you see how this is an issue for many participants?

One of the things that seems to get in the way of improving the remote
attendance options/technologies is the question of what this will do to
nomcom eligibility.  (The other thing is what it would do to revenue)
We have an innovator's dilemma and an old-boys club problem here: the people
who pick the people who make the decisions, are self-selected from the people
who can travel.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 12 Feb 2015 15:16:54 -0500

> I am starting to sympathize with Mike.  What is our goal here.

I wrote a long email last night, but I did not press send, because I wanted
to try to be clear, and I know that I tend to be long winded and
rambly. {"Oops, I did it again" http://goo.gl/sLebz}

The goal is, I think, to have an I* which is make up competent people, who
spend a reasonable amount of time, but not 100%, making the Internet a better
place for everyone on the planet.  That means not just having a diverse group
of people in charge, but making sure that it's clear to everyone on the
planet that we welcome them: that we desire their participation.

A major feedback from the confirming bodies into the nomcom is about
diversity of all sorts: gender, age, culture, geography.   The ISOC board
really cares about this (and that discussion is one reason the IAB
announcement took longer, btw).  How can we have more diversity without
sacrificing any of the other things that are also important: like being able
to work well together.

People have been upset during the past two nomcoms (both of which I was on),
about the resulting diversity coming out of the selection process.  The
reality is that the nomcom can't produce diversity; it only filter the input
for the best people.  That means we if we want diversity, we have to feed the
pipeline at the bottom.  That presents a bit of an innovator's dilemma
because the current group of people have been doing things this way for a
long time, and it has supposedly been "working"...

Yet, notice how hard it is to get younger people in?
How has the average age of IETF participants grown?

The nomcom eligibility is about as close as we have to a membership process.
The nomcom eligibility really says to most people: this the minimal amount of
resources we expect a person to have (and expend) in order to participate.

Right now that criteria significantly excludes people for whom:
   - travel is expensive for them and/or their company

   - obtaining a US visa is hard. *this is really a huge diversity killer*

   - being away from home and/or office for ~10 days is a challenge
     (remember that operators have other events, and if we want people
     who write the code to be enfranchised, we have to remember that they
     have other *code* related events)

   - travel is just physically difficult for all sorts of reasons
     (we have a regular and recurring thread on this)

Furthermore, the current criteria, essentially leads to a kind of oligarchy.
People who are present are the people who decide who shall make the
decisions, and can essentially only select among those who are present.

The remote participation process is poor in part because nobody of
significance uses it regularly (and it doesn't enfranchise you).  Since the
people who make the decisions are never remote, and the people who make the
decisions about who makes the decisions are also not remote participants.

We claim that we are an open, no-membership required, inclusive organization.

But, the "cabal" that runs the IETF is the group of ~600 people who attend
essentially every meeting, and I think that about 100 of those people are
possibly "Elmers".  Of the other 600, I suspect 300 attend regularly, but
below the 3/5 level, are sometimes above/sometimes below thus too busy
to volunteer, and 300 were proximal to the meeting, so they came (even if it
wasn't their first meeting).

But, I note that the nomcom doesn't get 600 volunteers, and of the 233 names
that I did get, about 80 were rejected: half were very new and surprised at
the criteria, and the other half were surprised to realize that they were not
eligible.  Perhaps Joel, you still have your list from when you were chair;
it would be interesting to compile those statistics into a graph.

I also want to repeat what Ted said this morning about "carbonfest".

---

From: Joel Halpern <[email-masked]>
Date: Thu, 12 Feb 2015 15:56:59 -0500

Taking this out of order, I think I had one person fail qualification, 
and one person withdraw because they should not have been allowed, 
although the rules at the time would have allowed him.  So your having 
to reject 1/3 of your volunteers is interesting.  I wonder if this is a 
consequence of our pushing so hard for folks to volunteer?

With regard to nomcom qualification and diversity, I wonder if this is 
in part a consequence of our conflating two things.  On one side we have 
actual eligibility to serve on the nominating committee.

On the other hand we have a (small, but significant) number of processes 
which treat nomcom eligibility as essentially membership for having 
standing of various kinds.  This update is probably not the place to 
disentangle the two.  But I think trying to relax nomcom criteria in 
order to solve that problem is putting things the wrong way around.  We 
may well have taken too much of a shortcut in other cases.

I really don't think that relaxing nomcom membership rules is an 
effective way to address our (very real) leadership diversity issue. 
That being said diversity is clearly of value for the nomcom itself, in 
addition to the value for our leadership bodies.  I have no idea whether 
the proposed rules would actually qualify a more diverse set of people. 
  Larger, yes.  More diverse?  Maybe.  But given that all of these rules 
are very rough approximations for what we need, I am concerned that 
relaxing them without sufficient relationship to our needs has a too 
high a probability of making things worse in important ways.

All of which is why I want to see a specific proposal.  And why I have 
said that in the abstract I would like to see improvement.

Yours,
Joel

---

From: Michael Richardson <[email-masked]>
Date: Thu, 12 Feb 2015 16:15:43 -0500

Joel Halpern <[email-masked]> wrote:
    > effective way to address our (very real) leadership diversity
    > issue. That being said diversity is clearly of value for the nomcom
    > itself, in addition to the value for our leadership bodies.  I have no
    > idea whether the proposed rules would actually qualify a more diverse
    > set of people. Larger, yes.  More diverse?  Maybe.  But given that all
    > of these rules are very rough approximations for what we need, I am
    > concerned that relaxing them without sufficient relationship to our
    > needs has a too high a probability of making things worse in important
    > ways.

If we can find a way to get the registration database imported into the
datatracker, that would permit one to run some experiments.  We need that
data, because we need to know if someone is 3/5 qualified already.

    > said that in the abstract I would like to see improvement.

I have written some specific ideas (close to, but not specific wording for
BCP10) on this thread.  If that was insufficient for you to evaluate
conceptually (parameters can be tweaked); can you tell me in what form you
think it needs to be presented?

Or is this really a continuation of the above paragraph; and really you are
saying you'd like to be able apply the process against real data, and observe
the results?  (running code)

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 12 Feb 2015 16:20:03 -0500

With regard to the proposal, the original proposal was sufficiently far 
back in the thread, and there ere enough ideas about modifying it, that 
I am not sure I can construct exactly what you intended.  Your recent 
message in this regard in which you restated the general shape helped. 
I have seen some of the ideas for "contribution" and some discussion of 
same, but ...

being able to look at real data would definitely help.  I understand 
that the lack is not your doing.  And the secretariat and tools team 
folks have other things on their plate.  Just hard to judge.

Yours,
Joel

---

From: Russ Housley <[email-masked]>
Date: Thu, 12 Feb 2015 23:40:20 -0500

> That's why the 3/5 is how you get in, and continued participation is how you stay.   I think your point about IETF culture is right, but it's ridiculous to think that an organization that invented…

We disagree.  Many people tell us (on the IAOC surveys) that the most important thing that happens at IETF meetings is the hallway conversations.  It is a significant part of the culture.

Russ

---

From: Loa Andersson <[email-masked]>
Date: Fri, 13 Feb 2015 13:05:03 +0800

Folks,

+1

/Loa

--

---

From: Dave Cridland <[email-masked]>
Date: Fri, 13 Feb 2015 07:44:40 +0000

On 13 February 2015 at 04:40, Russ Housley <[email-masked]> wrote:

> We disagree.  Many people tell us (on the IAOC surveys) that the most
> important thing that happens at IETF meetings is the hallway
> conversations.  It is a significant part of the culture.
>

This makes me really quite angry.

Maybe my last note wasn't clear; I wrote it as gently mocking sarcasm, but
clearly this is too subtle, since you've written something so diametrically
opposed to what I was hinting at that it may as well be a strawman. Maybe
it is, and maybe you're simply trolling for a more obvious reaction from
me, and similar folk. In which case, bravo, you have it. But if this is a
genuine viewpoint, then it epitomises everything that's wrong with the
IETF, and represents an appalling and untenable position for any IAB
member, let alone the IAB Chair to take.

Sure, I appreciate that human contact is important. I've been to two IETF
meetings in the flesh, and I enjoy, and have had significant benefit from,
hallway conversations.

But to claim it's "the most important thing", and to further imply that no
other IETF participation or activity should count for anything is just
astonishing.

What you're saying is that my contributions don't count for shit.

I'm not going to claim that my track record is in the same league as yours,
but I don't think it's entirely insignificant, and moreover, I'm absolutely
certain that this is not the message you should be conveying to new people
(or people who've have documented contributions over two decades either,
but fuck them if they don't turn up to hallway conversations, eh?)

Really, if that's the case, we should abandon having working group meetings
at the IETF meetings, and not bother with RFCs, or any of the other
tiresome things which get in the way of hallway conversations.

But I should point out that the stated position of the IETF is that the
work of the IETF happens on mailing lists, not in hallway conversations.

Now if we want a NomCom which only works as an old boy's club, and just
puts people into positions where the committee members get along fine with
them, and share a few rounds, then this is a perfect setup. Well done, if
that's your aim. And why the hell not, given it's a status quo which serves
you just fine? Actually, let's back this up with a self-selecting survey of
people who attend meetings, because that's definitely not biased at all.

If you want a NomCom (and I'm ignoring the frankly far more important
details of recall etc) that represents the working core of the IETF,
though, you might want to capture a little more diversity in the volunteer
pool. My position is, and remains, that there are more ways to
significantly contribute to the IETF than idle chats in hallways.

But just as I find it frankly insulting that I still qualify to have a
mentor as a "newbie", because obviously I'm so totally clueless, it's also
astonishingly clear to me now that the stated position of the IAB Chair is
that nothing I have done within the IETF matters.

Thanks, and be glad that RFC 3777 gives me no voice in your recall. Good
move, best keep it that way.

Dave.

---

From: Melinda Shore <[email-masked]>
Date: Thu, 12 Feb 2015 22:59:46 -0900

> conversations.  It is a significant part of the culture.

I think there's a cultural thing at work - it appears to be increasingly
the case that IETF culture requires meetings and face-to-face
discussion.  There are companies that have telecommuting cultures and
in which it's possible to have "hallway" discussions online, and ones
in which remote work is not part of the culture and if there are
people offsite they're remembered as an afterthought, if at all.  I've
been irritated recently by the extent to which we've moved away from
being an organization that makes useful things towards one which
publishes requirements documents and problem statements and which
sinks a lot of effort into proposals which isn't driven by actual
product or deployment needs.  I think there's a productivity
cost, because we get big pulses of work three times/year and not
enough gets done between.  The increasing valuation of meetings
doesn't really do much to discourage that.  (My real beef is with
being underresourced and oversludged and the amount of time it takes
to complete work as a result; if we had plenty of bandwidth and
tons of work was being done between meetings, less-than-useful
work wouldn't be a problem).

Melinda

---

From: Dave Cridland <[email-masked]>
Date: Fri, 13 Feb 2015 08:05:33 +0000

On 13 February 2015 at 07:59, Melinda Shore <[email-masked]> wrote:

> discussion.

And what do you expect when we make it absolutely clear that this is the
only thing that counts?

---

From: Ted Lemon <[email-masked]>
Date: Fri, 13 Feb 2015 07:45:36 -0500

> We disagree.  Many people tell us (on the IAOC surveys) that the most important thing that happens at IETF meetings is the hallway conversations.  It is a significant part of the culture.

We don't actually disagree that much.   I agree that the most important thing that happens at IETF meetings is hallway conversations.   Where I appear to disagree with you is that I think that someone who has been participating in IETF for some time will do what it takes to get the same effect some other way if they are not able to attend in person.

---

From: John Leslie <[email-masked]>
Date: Fri, 13 Feb 2015 10:12:29 -0500

> conversations.  It is a significant part of the culture.

   Since Russ is no longer my boss, I'll pick a bone here.

   I have participated in these "hallway conversations" -- and while
they are most helpful to relative newbies, I have found them less
necessary as I became more familiar with IETF lore. They always had
a certain amount of "blind-men-and-the-elehant" to them anyway. :^(

   But regardless, if "hallway conversations" are important, why
aren't we finding a way to accomplish this value over the 'net (and
more than three times a year)?

   IMHO, the "most important thing" about IETF weeks is that they
impose deadlines for _something_ to happen.

--
John Leslie <[email-masked]>

---

From: Ted Lemon <[email-masked]>
Date: Fri, 13 Feb 2015 10:16:36 -0500

On Feb 13, 2015, at 3:05 AM, Dave Cridland <[email-masked]> wrote:
> And what do you expect when we make it absolutely clear that this is the only thing that counts?

Dave, to be fair, what Russ actually said was that some people say the most important thing that happens at IETF _meetings_ is hallway conversation.   I tend to agree with that in a sense: I think that the discussions that happen in the hallways are actually triggered by people thinking about presentations and discussions that have happened in meetings, so I don't think you can have the one without the other.

One of the things that I obsess about when I'm thinking about attending remotely is how to get the same effect, because I agree with Russ that it's important.   But I look at it kind of the opposite of the way you expressed.   I think it's important for offsite attendees to be able to participate meaningfully, so what I want is to figure out how we can export the hallway conversations, not how we can exclude people who can't currently access them.

I think Melinda is on to something with her discussion about telecommute-friendly versus in-person environments.   Rather than just getting discouraged, I think we should try to actively work on this.

---

From: Mary Barnes <[email-masked]>
Date: Fri, 13 Feb 2015 09:43:18 -0600

On Fri, Feb 13, 2015 at 9:16 AM, Ted Lemon <[email-masked]> wrote:

> Dave, to be fair, what Russ actually said was that some people say the
> most important thing that happens at IETF _meetings_ is hallway
> conversation.   I tend to agree with that in a sense: I think that the
> discussions that happen in the hallways are actually triggered by people
> thinking about presentations and discussions that have happened in
> meetings, so I don't think you can have the one without the other.
>
> One of the things that I obsess about when I'm thinking about attending
> remotely is how to get the same effect, because I agree with Russ that it's
> important.   But I look at it kind of the opposite of the way you
> expressed.   I think it's important for offsite attendees to be able to
> participate meaningfully, so what I want is to figure out how we can export
> the hallway conversations, not how we can exclude people who can't
> currently access them.
>
> I think Melinda is on to something with her discussion about
> telecommute-friendly versus in-person environments.   Rather than just
> getting discouraged, I think we should try to actively work on this.
>
> [MB] I totally agree with Melinda's points.  I worked in any environment
where it was more the norm than the exception to have folks working from
home for eons.  In  my last job, video calls were also the norm. Those can
dramatically improve the experience since so much of our communication is
visual, which is one of the reasons why hallway interactions and small
group discussions can be so effective.   I now work in an environment where
I am the only one that works from home and the difference in my day to day
interactions is startling.  People do forgot about you. Audio calls in
particular are only about 10-20% effective due to poor audio setup in
meeting rooms and whomever is running the meeting not adequately taking
into consideration the remote participations.

So, personally, I think it's time to consider whether we can do everything
on mailing lists between meetings.  In one WG I chaired, we had weekly
audio calls for our design team, which was comprised of anyone that was
willing to contribute to the work (i.e., text, detailed document reviews,
etc.). Those can work fine when everyone knows everyone as you've learned
to pick up on tones of voice and you know one another fairly well from the
face to face meetings.  We carefully documented decisions and produced
publicly available minutes, etc.   There was one participant and key
contributor that has only been to one or two of the face to face meetings
for the WG.    I would posit that these meetings can be equally as
effective as face to face meetings in making progress. Once we started
those meetings, we didn't find that we needed f2f interim meetings.
 There of course, the timezone issue with these sorts of meetings, but that
can be managed.    But, we do have all the tools to make work between
meetings more effective and more open to remote participants that I don't
think we're taking advantage of.

Of course, I still believe that we need face to face meetings for these
hallway discussions.  I believe that there are, however, some decisions
that might be made amongst a smaller group of contributors that ought to be
better communicated to other interested parties.  I don't think we always
do this.  And, in the spirit of the transparency that was the objective
behind the changes in the email notifications recently, I think folks
should ensure, for example that discussions that are related to WG
activities are communicated to the WG.  This does happen sometimes - just
not frequently enough IMHO.   I think this touches on Dave Cridland's point.

Regards,
Mary.
[/MB]

---

From: Russ Housley <[email-masked]>
Date: Fri, 13 Feb 2015 10:58:58 -0500

> This makes me really quite angry.

I'm sorry that you are angry.

> Maybe my last note wasn't clear; I wrote it as gently mocking sarcasm, but clearly this is too subtle, since you've written something so diametrically opposed to what I was hinting at that it may as well be a strawman. Maybe it is, and maybe you're simply trolling for a more obvious reaction from me, and similar folk. In which case, bravo, you have it. But if this is a genuine viewpoint, then it epitomises everything that's wrong with the IETF, and represents an appalling and untenable position for any IAB member, let alone the IAB Chair to take.

I was not replying to your note.  I was replying to Ted Lemon.  Based on his follow-up message, I think he understood my point.

> Sure, I appreciate that human contact is important. I've been to two IETF meetings in the flesh, and I enjoy, and have had significant benefit from, hallway conversations.
> 
> But to claim it's "the most important thing", and to further imply that no other IETF participation or activity should count for anything is just astonishing.

I said no such thing.  I said that NomCom members need to understand the culture, and that participation in the meetings is an important aspect of learning that culture.  In my view, this is confirmed by the survey results.

I would love for remote participation tools to offer the same experience.  Today they do not.  Maybe some day, remote participation tools will become good enough, and when that happens we can deemphasize the meeting participation in the rules for NomCom eligibility.

Russ

---

From: Mary Barnes <[email-masked]>
Date: Fri, 13 Feb 2015 10:00:22 -0600

And, talking to myself...

I should have updated the subject since it's getting a tad OT. But, to
bring this back to Nomcom, I do think we have a way in which remote
individuals can effectively participate in Nomcom.  We have video
technology and as I said below that is extremely effective for
communicating.  And, I think it can be effective for Nomcom interviews. In
the last job I had, the hiring manager was in India and we did the
interview over Skype.  I worked in his organization for 4 years and never
met him in person.   This hiring manager in India had shifted his work
schedule to be more compatible with North America as the majority of his
team was here.  It does require some flexibility on the part of the
individuals involved.   But, I honestly believe that if someone who has
participated primarily remotely is a good listener and does due diligence
in terms of reading community input and using that input as well as that
gathered from the interviewee (i.e., it should be expected that the
interviewers have read the nominees questionnaire - that hasn't always
happened in  my experience) and asks the right questions during interviews,
that this Nomcom voting member could be equally as effective as the average
regular IETF attendee.  So, I do think there is some truth in Dave's
concerns, in terms of the perception of those that aren't IETF regulars.

Regards,
Mary.

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 13 Feb 2015 16:11:19 +0000

> not get to vote unless someone gets hit by a bus.
Michael

Unless they are required to be silent unless spoken to, the spare member 
will
have some influence in the outcome, through the debate that takes place.
They may not have as much influence as someone with a vote but they
have significantly more influence than anyone outside the Nomcom.

If you take the contra view that they should not speak, there is a danger
that they will not support the work so far, if and when they do get the 
vote,
and some back tracking of the process will be required.

Stewart

---

From: Sam Hartman <[email-masked]>
Date: Thu, 12 Feb 2015 15:27:54 -0500

I support what michael claims as a goal.
I believe that it is an important goal for the IETF.

In the past I've been nervous about giving remote participation too much
power in part because I'm worried about how that impacts meeting fees
and in part because I value cross-area involvement.

However, I think the goal Michael outlines is important enough that we
need to find some way of improving the extent to which we meet that
goal.

---

From: Dave Cridland <[email-masked]>
Date: Fri, 13 Feb 2015 17:44:22 +0000

On 13 February 2015 at 15:58, Russ Housley <[email-masked]> wrote:

> I said no such thing.  I said that NomCom members need to understand the
> culture, and that participation in the meetings is an important aspect of
> learning that culture.  In my view, this is confirmed by the survey results.
>
>
I'll accept that you didn't intend to suggest that, but comments like those
in this thread about the overarching importance of meeting attendance and
hallway discussions send a clear signal to people who "only" do
specification work that their opinion is not as important as those backed
by large companies or who can otherwise afford to attend meetings and buy
rounds of drinks.

Moreover, if you accept that the word "culture" is effectively
indistinguishable to outsiders from the term "status quo" (though the
intent is obviously different), it's really quite revealing. All this
"preserving the culture" talk comes out in an entirely different light.

> I would love for remote participation tools to offer the same experience.
> Today they do not.  Maybe some day, remote participation tools will become
> good enough, and when that happens we can deemphasize the meeting
> participation in the rules for NomCom eligibility.
>
>
a) The NomCom eligibility rules do not emphasize meeting *participation*,
but meeting attendance.

b) The NomCom eligibility rules do not *emphasize* meeting attendance, they
*are* meeting attendance.

c) The "NomCom eligibility" rules govern essentially any say in the
leadership of the IETF.

So you need to say "... when that happens we can change the rules for
having any say in the leadership of the IETF to something other than purely
meeting attendance."

Moreover, speaking as someone who has worked remotely for years, and who
currently works for a distributed company, I would argue that your
assertion that remote participation tools are not good enough is incorrect
- they're extremely close for everything aside from hallway discussions,
and even for those, the tools are pretty good -- if deployed. What kills
them without fail is if you have a mixture of remote and local participants
- it's really hard to balance those two.

But anyway - what would the incentive be to develop and deploy such tools
for the IETF?

Your own position is predicated on the "culture", and a significant portion
of IETF funding occurs due to meetings.

If we, as a community -- sorry, my mistake.

If you, as a community, were to aim to replace one meeting per year with a
virtual one, we'd push the quality of online meetings through the roof
almost instantly. (Preventing physical interim meetings would be a sensible
short-term step). I fondly imagine that the bulk of the costs for the IETF
are in holding the meetings; I also similarly imagine that the bulk of the
attendee costs are hotels and flights, so maybe the funding problem is
somewhat soluble.

If you, as a community, were to enfranchise remote participants, it would
have the inevitable effect of causing their views to be represented amongst
the I* leadership.

And yes, this would change IETF culture -- that is the point of diversity.

A sensible way of managing this change to avoid disruption would be to
provide several distinct criteria for eligibility, and select from each
pool proportionally, changing the proportions over time. So recall, for
example, might need 15 meeting attendees and 5 people qualifying under
active participation rules to initiate, gradually changing to emphasize
active participation.

Dave.

---

From: Joel Halpern <[email-masked]>
Date: Fri, 13 Feb 2015 12:51:45 -0500

The fact that the nomcom eligibility rules are attendance rather than 
participation is a bug.  It was known to be a bug when it was written.
Okay, if one wants to be picky, it was known to be a mediocre heuristic 
approximation for what was desired.)

The challenge has been to find rules that capture the range of exposure 
/ participation that would actually provide good background for 
leadership selection.

We could not do so when we wrote the rules.
Maybe we can now.  I would love to see better rules.
The challenge for me is telling that a proposed rule set has a 
reasonable chance (there is no sure thing here) of being better.  That 
is part of what prompted the exchange with Michael about data.

Yours,
Joel

On 2/13/15 12:44 PM, Dave Cridland wrote:
> On 13 February 2015 at 15:58, Russ Housley <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>>     Sure, I appreciate that human contact is important. I've been to
>>     two IETF meetings in the flesh, and I enjoy, and have had
>>     significant benefit from, hallway conversations.
>>
>>     But to claim it's "the most important thing", and to further imply
>>     that no other IETF participation or activity should count for
>>     anything is just astonishing.
>
>     I said no such thing.  I said that NomCom members need to understand
>     the culture, and that participation in the meetings is an important
>     aspect of learning that culture.  In my view, this is confirmed by
>     the survey results.
>
...
> a) The NomCom eligibility rules do not emphasize meeting
> *participation*, but meeting attendance.
>
> b) The NomCom eligibility rules do not *emphasize* meeting attendance,
> they *are* meeting attendance.
...

---

From: Michael Richardson <[email-masked]>
Date: Fri, 13 Feb 2015 12:52:48 -0500

Mary Barnes <[email-masked]> wrote:
    > or two of the face to face meetings for the WG.  I would posit that
    > these meetings can be equally as effective as face to face meetings in
    > making progress. Once we started those meetings, we didn't find that we
    > needed f2f interim meetings.  There of course, the timezone issue with
    > these sorts of meetings, but that can be managed.  But, we do have all
    > the tools to make work between meetings more effective and more open to
    > remote participants that I don't think we're taking advantage of.

For the remote participant, the most important thing that the physical
meeting does it to establish a time zone, and force everyone else to conform
to that.  Additionally, by removing people from their office, it removes a
whole host of "distractions": bosses, spouses, kids, commutes,...

One of the things that I think we could try is to have a meeting with dozens
to hundreds of locations. 

It would still be 5 days long.
It would have a primary location in order to establish a time zone for it. 
It would have a schedule, and agenda and conflict resolution.
It's just that I would go stay in a nearby hotel and/or conference center,
and I'd have to have lunch or do social events with nearby people.
It would an experiment.

The hardest thing for me, when attending remotely, is convincing my kid (who
wakes up at 5am regularly), is that I'm "at work" at that time, and that
when it's time for me to sleep (in remote time zone), that I really do need
to sleep.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 13 Feb 2015 12:56:56 -0500

Stewart Bryant <[email-masked]> wrote:
    >> proceedings, goes to the interviews, etc.  If they don't, then they
    >> aren't of much use.... If they *do* it seems like a large burden to do
    >> that, and then not get to vote unless someone gets hit by a bus.

    > Unless they are required to be silent unless spoken to, the spare
    > member will have some influence in the outcome, through the debate that
    > takes place.  They may not have as much influence as someone with a
    > vote but they have significantly more influence than anyone outside the
    > Nomcom.

I would not make them be silent; and so I do agree with you: they will have
some influence.

I think, though, that it would be a very frustrating experience myself,
and the committee would be most likely to loose that member.

---

From: Melinda Shore <[email-masked]>
Date: Fri, 13 Feb 2015 09:10:36 -0900

> "preserving the culture" talk comes out in an entirely different light.

I think this is a really important comment.  I mean,
*really* important comment.

But it also seems to me pretty clear that the culture
is changing, anyway, and it's one of those things that
I expect most people know without addressing it directly.
I don't think meetings were so heavily emphasized 10
years ago, although that's subjective and could be wrong.
We've now got a very large number of people participating
whose primary job function is to create standards, and
that's caused some changes because their incentives are
different from those whose job it is to create products
or technology.  I don't know how long it's been since
running code was a significant adjunct to the work being
done in the IETF, but I think it's been quite awhile.
So these cultural shifts are taking place anyway, and
they are not being "managed."  Some are good, some are not.
I do think that the increased significance of meetings
in IETF participation (and here, I'm not talking about
things like nomcom but about significance to our technical
work) is a problem, both because it tends to marginalize
people who can't come to meetings and because it slows
work down.

Melinda

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 13 Feb 2015 10:55:57 -0800

> c) The "NomCom eligibility" rules govern essentially any say in the

I disagree.  "Eligibility for NomCom" and "having a say in IETF leadership"
are not synonyms.

Your claim is right only if you presuppose that those who become selecting
members of the NomCom typically disregard the feedback they receive about
candidates and instead assert their own agendas.  I'd be comfortable
claiming that at least a large majority of the selecting members of the
last two NomComs, both of which included me, gave proper consideration to
feedback received from people in the community irrespective of who was
submitting the feedback or what their attendance or contributions were.

Remember, they're volunteers.  They volunteer to provide a service, which
is the donation of time to do interviews, participate in conference calls,
accept and sift through mountains of feedback, deliberate, argue, and
enforce their own propriety.  At least from my own experience, it's been
anything but a power play.

> active participation rules to initiate, gradually changing to emphasize

This might be a reasonable starting point.  What are your thoughts on
Michael's proposals for measuring active participation?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 13 Feb 2015 11:00:52 -0800

> years ago, although that's subjective and could be wrong.

I'm not so sure I agree that there's active effort to preserve the culture,
or emphasize the importance of same when selecting NomCom members.  Rather,
it's important to have people on the NomCom that at least understand the
culture as it is now; an understanding of the past and present is important
to identify what we need for the future.  Sometimes we need to select
candidates to continue work along the current path, and sometimes we need
to select candidates that can take us in a better direction when it's
obvious that improvement is needed.

-MSK

---

From: Mary Barnes <[email-masked]>
Date: Fri, 13 Feb 2015 13:10:20 -0600

> and I'd have to have lunch or do social events with nearby people.
[MB] Right. That's a choice you make because you feel it's important to
optimize
your participation.  I'm making the same choice for the Dallas meeting, but
I don't think
that sort of thing is a general factor for planning a meeting (i.e., how
many people have to
stay at a hotel even with this model).  [/MB]

It would an experiment.

 [MB]  Personally, I don't think the logistics of arranging this are at all
worth the efforts.
That all said, maybe we could setup a wiki for remote participants that are
willing to make
local arrangements for the meetings for folks in the area to get together
for various sessions.
Although, with our remote participation tools, I don't see a huge advantage
other than of
course enabling informal discussions.
[/MB]

> when it's time for me to sleep (in remote time zone), that I really do need

[MB] That's well understood. But, again not something that applies to all
remote
participants.  Also, you need to consider the fact that at every meeting we
have the
majority of the attendees not in their own timezone - just look around the
meeting rooms
and see the folks that are falling asleep or sleeping in various chairs and
couches around
the meeting venue.   Just because we're all physically in the same timezone
doesn't mean
we all physiologically in the same time zone.

I think we need to focus on the things we can change.  We can't totally
eliminate the timezone
issue (even for face to face meetings) and we certainly can't make
adjustments that address everyone's
personal life challenges.  We all have to balance those - even when we are
at f2f meetings, we
usually create another set of personal life challenges.
[/MB]

---

From: John C Klensin <[email-masked]>
Date: Fri, 13 Feb 2015 14:13:37 -0500

--On Friday, February 13, 2015 09:10 -0900 Melinda Shore
<[email-masked]> wrote:

>> indistinguishable to outsiders from the term "status quo"
>> (though the intent is obviously different), it's really quite
>> revealing. All this "preserving the culture" talk comes out
>> in an entirely different light.
> 

I completely agree with all of the above.  I think we are also,
as a community, frequently in denial about those changes and
their consequences.  To take a pair of examples that has been
discussed before but without either conclusions or actions, if a
Nomcom cannot distinguish between those whose "primary job
function is to create standards" and those who function is
primarily to "create products or technology" --either because
they have been told the difference isn't important or because
most of them fall into the first group (perhaps because that is
where the resources are to support Nomcom work), then the odds
are increased that we end up with an IESG that is mostly the
former group as well (again, perhaps in part because it is
easier for professional standards developers to find support for
the time and resources the IESG takes than it is for an
organization to take a product designer off the job for a few
years).  Perhaps that is ok, but we are not discussing its
implications, implications that experience in some other SDOs
suggest are significant.

    john

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 14 Feb 2015 08:43:45 +1300

Hi Melinda,
> *really* important comment.

It is. We need to preserve the useful parts of our culture, of course.
But corridor conversations at meetings is only one aspect of that.
The ability to hear from, and respond to, anybody with email
is another one.

> years ago, although that's subjective and could be wrong.

I think it's probably wrong. Whatever other changes have occurred
(such as noticeable improvement in remote participation technology,
for those who recall the occasional burps of audio from the audio
multicast setup in the 1990's, and very great improvement in the
on-line tools), the underlying human beings are still running the
same software. As I type this, I can visualise Melinda and think
of her as a real person. That is significant, and only possible
because we've attended quite a few meetings together. Humans
work that way.

High quality audio/video can help with this, but my own experience
in a large company heavily dependent on remote interaction was
clear: it helps enormously if you have real, in the room, contact
with the remote people once or twice a year. Eating a meal with
somebody creates a bond.

Have we diluted the principle that decisions are taken on the
mailing list? I don't think so. Have we diluted the principle
that details are debated on the mailing list? I don't think so.
But we haven't changed the fundamental nature of human beings
as social animals, either.

> done in the IETF, but I think it's been quite awhile.

That isn't true everywhere in the IETF. For certain types
of protocol & product, it is clearly a real problem
due to long development cycles. I don't know how to fix that,
because it seems to be a feature of the real world, not
a social artefact.

> people who can't come to meetings and because it slows

It is very clear that things have slowed down compared, let's
say, to 20 years ago. It's harder to introduce new things
than it used to be, because of the size of the network and
the inertia of operational practices. So running code at an
early stage is definitely harder, and (as I said above)
product development cycles are longer. I think that's outside
our control.

Thank goodness we have three hard deadlines a year, set by
the meetings. Otherwise, I think we'd be even slower.

    Brian

---

From: Dave Cridland <[email-masked]>
Date: Fri, 13 Feb 2015 20:53:00 +0000

On 13 February 2015 at 18:55, Murray S. Kucherawy <[email-masked]>
wrote:

> I disagree.  "Eligibility for NomCom" and "having a say in IETF
> leadership" are not synonyms.
>
>
That's true to the extent that a sitting NomCom listens to feedback without
considering the "NomCom eligibility" of the person providing it, yes.

However it's considerably harder for me to, for example, get an RFC 3777
Recall action going, since my say explicitly does not count there.

So I admit I'm not wholly right, but I'm not wholly wrong either.

> Michael's proposals for measuring active participation?
Essentially on the right track, but not right.

0) I'm happy to include "attended meeting", though I think there'd be
significant value in expressing it as a percentage of working group
sessions over the past one/two years, rather than just attendance to the
meeting in general. Again, I hear what people say about hallway discussions
and I don't disagree about the value there, but my gut feeling is that this
would be a metric more closely approximating what is sought here. But maybe
this doesn't matter.

1) AUTH48 is interesting as a metric; it's clearly an indicator of a
document actually getting published, and one that doesn't get delayed
outside the author's control unlike publication. Moreover, anyone going
through AUTH48 has had to deal with an IESG telechat on their document;
that's extremely useful experience in leadership selection. Possibly "last
call" and "telechat" are also good enough - and interesting in as much as a
document that "fails" last call is arguably as important a contribution as
one that goes on to publication.

Document shepherds and working group chairs are both clearly contributing.

I'd say that authors should be included here (and I'd go for "active"
authors in the document uploader sense), but perhaps that's covered by the
next item.

2) I think the document uploader concept is good, but limiting the
documents to only those which get scheduled time in a WG session isn't so
good; some WGs don't meet often, or even at all, and a document that's so
well crafted that it doesn't need discussion in a face to face meeting is a
really good document in my opinion. Clearly we don't want arbitrary
documents either. Any WG document would seem a "good" contribution, and I
suspect any document with a shepherd assigned should be safe, too, since
that effectively implies an expectation to publish.

3) This works for WGs that use a ticketing system and have meetings. Not
sure what percentage that actually is.

I understand that Michael is trying to propose a metric for adequately
measuring working group participation, but I don't think such a metric
exists.

We could have each active working group just submit up to X names agreed by
rough consensus any time an eligible pool is needed (ie, during NomCom
calls for volunteers). This is harder to deal with for Recall actions,
though.

4) I currently do this for the XSF Board, so it seems reasonable to me.
Again, this would provide very good experience of the incumbent leadership,
which'd seem useful to have in the core of NomCom.

Other observations:

a) I think that (0), (4), and (1-3) fit in different pools. You don't want,
I suspect, a NomCom filled with only I* meeting scribes, for example. To
put it another way, applying quotas (minimal and maximal) to the make-up of
the NomCom is fine, and quite interesting. (Thought experiment: What would
happen if the NomCom was exclusively made up from Asian, African, and
female members? How would that affect the nominations?)

I suspect that pools/quotas also make the system much harder to game in
practise.

b) I think WG Chairs might be a pool as well.

c) I suspect that whichever way we do this, the potential influence of a WG
Chair increases as a result, and I'm unaware of an appeal action for WG
Chair selection. This might be my own ignorance, but either way we'll need
one.

Dave.

---

From: Ted Lemon <[email-masked]>
Date: Fri, 13 Feb 2015 15:58:55 -0500

> and in part because I value cross-area involvement.

It's possible that we could collect meeting fees from remote attendees, offering a hardship exemption for those who can't afford it.   That would depend on remote attendance working better than it does now, I think, but it would be unfortunate if the main impediment to making remote attendance work well were that we didn't want to lose meeting revenue.

---

From: Mary Barnes <[email-masked]>
Date: Fri, 13 Feb 2015 15:30:34 -0600

On Fri, Feb 13, 2015 at 2:58 PM, Ted Lemon <[email-masked]> wrote:

> It's possible that we could collect meeting fees from remote attendees,
> offering a hardship exemption for those who can't afford it.   That would
> depend on remote attendance working better than it does now, I think, but
> it would be unfortunate if the main impediment to making remote attendance
> work well were that we didn't want to lose meeting revenue.
>
> [MB] I totally agree on this latter point.  I'm very conflicted about
charging for remote participation, but perhaps something nominal.  It's
also quite possible that if we improve the quality, we will get more remote
participants.    And, I obviously, would be perfectly happy for IETF to cut
back (or do away with) all the food and possibly beverage and not worry,
for example about providing IETF breakfast in cases where it's not part of
the room rate.    I know that the model is complex in terms of cost of
meeting rooms being based frequently on  the amount of food and beverage
provided, so I'm not suggesting this is simple or would have a huge impact
on overall cost.  And, yes, I know that there would be a huge uprising
about this, but we could request the hotels to setup carts, etc. where
people can purchase snacks and drinks during breaks as they often do for
lunches and perhaps the potential revenue that they can get from that could
be factored into the contract to offset the reduced catering request.
[/MB]

---

From: Brian Trammell <[email-masked]>
Date: Fri, 13 Feb 2015 22:50:53 +0100

hi Mary, all,

> On 13 Feb 2015, at 22:30, Mary Barnes <[email-masked]> wrote:
> 
> [MB] I totally agree on this latter point.  I'm very conflicted about charging for remote participation, but perhaps something nominal.  It's also quite possible that if we improve the quality, we will get more remote participants.

A requirement (at least at first) to allocate n% of remote participation fees directly to expenses related to the improvement of remote participation would make this a lot more feasible.

Cheers,

Brian

>  And, I obviously, would be perfectly happy for IETF to cut back (or do away with) all the food and possibly beverage and not worry, for example about providing IETF breakfast in cases where it's not part of the room rate.    I know that the model is complex in terms of cost of meeting rooms being based frequently on  the amount of food and beverage provided, so I'm not suggesting this is simple or would have a huge impact on overall cost.  And, yes, I know that there would be a huge uprising about this, but we could request the hotels to setup carts, etc. where people can purchase snacks and drinks during breaks as they often do for lunches and perhaps the potential revenue that they can get from that could be factored into the contract to offset the reduced catering request.
> [/MB]
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 14 Feb 2015 11:47:37 +1300

> A requirement (at least at first) to allocate n% of remote participation fees directly to expenses related to the improvement of remote participation would make this a lot more feasible.

But it begins to smell like a poll tax. Some people participate remotely
because they simply can't justify the travel expenditure; if it costs (say)
$200 to participate remotely, that would be enough to keep some people out.
How the Secretariat could possibly validate hardship cases remotely
is beyond me.

Also, does particpate mean "watch and listen" or "watch, listen and speak"?
I find it hard to imagine paying $200 just to watch and listen.

(Of course, I made up "$200" but it does need to be an amount of money
that's worth collecting, and in that case it will be a significant issue
for, say, a student in a developing country.)

    Brian C

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 14 Feb 2015 11:53:14 +1300

On 14/02/2015 09:53, Dave Cridland wrote:
...
> Chair selection. This might be my own ignorance, but either way we'll need

I don't know if it's happened, but the IESG would IMHO be very foolish
to refuse to consider an appeal against WG Chair selection. Any action
by an AD should be appealable in principle, IMHO, even if not explictly
mentioned in RFC 2026 or RFC 2418.

   Brian

---

From: Ted Lemon <[email-masked]>
Date: Fri, 13 Feb 2015 17:53:52 -0500

> How the Secretariat could possibly validate hardship cases remotely

Hence the suggestion that we have a hardship exemption.   My expense report for IETF including airfare is typically higher than $2500, so $200 for remote participation would be a substantial savings, if that were the fee.

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 13 Feb 2015 18:52:51 -0500

I guess I miss something. Some "smart" initiative to get money from participants?

Internet Society Ecuador
www.isoc.org.ec
Síguenos @isocec

> El 13/2/2015, a las 17:47, Brian E Carpenter <[email-masked]> escribió:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 14 Feb 2015 13:03:18 +1300

> I guess I miss something. Some "smart" initiative to get money from participants?

No. A discussion how to make remote participation a serious alternative
to travelling to meetings, without breaking the budget.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 14 Feb 2015 13:35:16 +1300

Carlos,

That isn't the point. Somebody has to pay for the things paid for
by the existing meeting fees. Suppose that we improve the remote
participation technology such that, say, 500 people who would normally
attend a meeting stay at home. That's a direct reduction of income by
say $350000, three times a year. So the IETF is out of pocket by $1M/year.
The actual reduction in meeting costs would be very slight. The money has
to come from somewhere.

Does this bother me? Yes, a lot. But it's reality.

   Brian

On 14/02/2015 13:13, Carlos Vera Quintana wrote:
> Oh I see. Free is not serious enough..
> 
> Carlos Vera Quintana
> 0988141143
> Sígueme @cveraq
> 
>> El 13/2/2015, a las 19:03, Brian E Carpenter <[email-masked]> escribió:
>>

---

From: Lixia Zhang <[email-masked]>
Date: Fri, 13 Feb 2015 20:04:50 -0800

> On Feb 12, 2015, at 12:16 PM, Michael Richardson <[email-masked]> wrote:
> 
> ......
> pipeline at the bottom.  

I noticed this subject line has been a long running thread and I have not read it all, but I'd really like to repeat Michael's comment here: the nomcom can't produce diversity; it only filter the input for the best people.  Thus the solution of getting diversity is not at nomcom, but the overall IETF process.

Lixia (member of nomcom 2014-15)

---

From: Melinda Shore <[email-masked]>
Date: Fri, 13 Feb 2015 19:27:30 -0900

On 2/13/15 7:04 PM, Lixia Zhang wrote:
> I noticed this subject line has been a long running thread and I have
> not read it all, but I'd really like to repeat Michael's comment
> here: the nomcom can't produce diversity; it only filter the input
> for the best people.  Thus the solution of getting diversity is not
> at nomcom, but the overall IETF process.

I think that's largely true but I don't think it's the entire story.
I do think the nomcom could be weighting criteria differently or
changing parameters in ways that might get make things easier to
get candidates who don't fit the current mold.

Melinda

---

From: Lixia Zhang <[email-masked]>
Date: Fri, 13 Feb 2015 21:41:10 -0800

> On Feb 13, 2015, at 8:27 PM, Melinda Shore <[email-masked]> wrote:
> 

not sure what's the thought behind "changing parameters"; some examples would be helpful.
weighting criteria differently: again, could this be stated more explicitly?  How should we weigh different criteria, in particular competency vs diversity?

Lixia

---

From: Larry Masinter <[email-masked]>
Date: Sat, 14 Feb 2015 06:04:04 +0000

creating effective standards is a community activity to avoid the http://en.wikipedia.org/wiki/Tragedy_of_the_commons

that would result if individuals and organizations all went their own way.
The common good is “the Internet works consistently for everyone” needs to compete against “enough of the Internet works ok for my friends” where everyone has different friends.

For voluntary standards to happen, you need rough consensus — enough people agree to force the remainder to go along.

It’s a community activity, and for that to work there has to be a sense of community. And video links with remote participation aren’t enough to create a sense of community.

There are groups that purport t to manage with minimal face-to-face meetings, but I think those are mainly narrow scope and a small number of relevant players, or an already established community, and they regularly rely heavily on 24/7 online chat, social media, open source tools, wikis which are requirements for full participation.

The “hallway conversations” are not a nice-to-have, they’re how the IETF preserves community with open participation.

One negative aspect of IETF “culture” (loosely, the way in which the IETF community interacts) is that it isn’t friendly or easy to match and negotiate with other SDOs, so we see the WHATWG / W3C / IETF unnecessary forking of URL / URI / IRI, encodings, MIME sniffing, and the https://www.tbray.org/ongoing/When/201x/2014/03/.../RFC7159-JSON JSON competing specs based at least partly on cultural misunderstandings.

The main thing nomcom  needs to select for is  technical leadership (the skill of getting people to follow)  in service of the common good). And nomcom members should have enough experience to have witnessed successful leadership. One hopes there might be some chance of that just by attending 3 meetings, although the most effective leadership is often exercised in those private hallway conversations where compromises are made.

Larry
—
http://larry.masinter.net

---

From: Ted Lemon <[email-masked]>
Date: Sat, 14 Feb 2015 07:32:40 -0500

> It’s a community activity, and for that to work there has to be a sense of community. And video links with remote participation aren’t enough to create a sense of community.

In my rather long experience in the IETF, what creates a sense of community is working together on things.

> There are groups that purport t to manage with minimal face-to-face meetings, but I think those are mainly narrow scope and a small number of relevant players, or an already established community,…

We are an already-established community.

> The “hallway conversations” are not a nice-to-have, they’re how the IETF preserves community with open participation.

I think the hallway conversations are great for getting things done, but I don't agree that they are essential to preserving community.

> One negative aspect of IETF “culture” (loosely, the way in which the IETF community interacts) is that it isn’t friendly or easy to match and negotiate with other SDOs, so we see the WHATWG / W3C / IETF unnecessary forking of URL / URI / IRI, encodings, MIME sniffing, and thehttps://www.tbray.org/ongoing/When/201x/2014/03/.../RFC7159-JSON JSON competing specs based at least partly on cultural misunderstandings.

The unfriendliness of IETF culture is to a large extent a bug, not a feature.   I say to a large extent because the willingness to say unpleasant things is necessary for technical excellence: you have to be willing to say "you are wrong, and here's why."   But we often don't even try to say it pleasantly, and we often don't bother to say "and here's why," and this creates unnecessary difficulties for newcomers and for people who aren't good at taking criticism (which is most people).

> The main thing nomcom  needs to select for is  technical leadership (the skill of getting people to follow)  in service of the common good). And nomcom members should have enough experience to have…

One of the things that I tried to do during my tenure as AD was to communicate more than is typical on the IETF mailing list.   This is a double-edged sword: it's more public and more memorable than the other contexts in which ADs attempt to exhibit leadership, and you are less likely to get people to actually follow you.   That's okay, because leadership isn't actually about telling people what to do--it's about figuring out where the consensus should go, and trying to get it to go in one of the good directions it could go rather than a bad direction.   But when you do that in public and on the record it doesn't always look like leadership, and of course you don't always understand the consensus well enough or pick the right direction.

Nevertheless, the idea that nomcom people wouldn't be exposed to examples of leadership on mailing lists is completely wrong, IMHO.   If a candidate hasn't shown leadership in that context, they aren't a good candidate.

---

From: Ted Lemon <[email-masked]>
Date: Sat, 14 Feb 2015 07:37:55 -0500

On Feb 14, 2015, at 7:32 AM, Ted Lemon <[email-masked]> wrote:
> That's okay, because leadership isn't actually about telling people what to do--it's about figuring out where the consensus should go, and trying to get it to go in one of the good directions it could go rather than a bad direction.

s/should/could/

---

From: "[email-masked]" <[email-masked]>
Date: Sat, 14 Feb 2015 10:46:13 -0500

Savings are welcomed instead fees.

What if you put all that gold bureaucracy in 3 stars hotels and not in 5? 
Fly economy, have offices in cheap places and countries..

Being smart saves money and it's fun..  A lot more that looking at the users's pocket.. But off course it's not as easy 

Carlos
Internet Society Ecuador
www.isoc.org.ec
Síguenos @isocec

> El 13/2/2015, a las 19:35, Brian E Carpenter <[email-masked]> escribió:
> 
> Carlos,
>

---

From: Yoav Nir <[email-masked]>
Date: Sat, 14 Feb 2015 18:43:34 +0200

Uhhm, who are that “gold bureaucracy" who stay at 5-star hotels at the IETF’s expense?

All participants, including area directors pay their own way (or their sponsor pays their way). But 3 star hotels don’t typically have good enough conference centers.

Also, supporting remote participation in a better way than it works today costs more money than is being expended today. That money has to come from somewhere.

Yoav

> On Feb 14, 2015, at 5:46 PM, [email-masked] wrote:
>

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Feb 2015 13:14:31 -0500

--On Saturday, February 14, 2015 18:43 +0200 Yoav Nir
<[email-masked]> wrote:

> Also, supporting remote participation in a better way than it
> works today costs more money than is being expended today.
> That money has to come from somewhere.

As a periodic remote participant, some observations:

First, the big costs to me of a f2f meetin are associated with
being away from home, getting on airplanes (for some medical
reasons, especially bad news when the meeting is outside North
America) and staying in a hotel.  The registration fees have
crept up well beyond the historical nuisance and cookie charge,
but are still close to the noise of overall expenses.

Second, I would actually prefer to be formally registered and
paying some reasonable remote participate registration fee.  I'm
prefer to be recorded as attending sessions I attend and
participate in, whether by my name going on the blue sheet with
an asterisk or by some other mechanism.  I don't like the idea
of others (or even their companies) subsidizing me and would
prefer to be in a situation in which there were established
conventions about what, as a remote participant, I have the
right to expect.  In general, people have been _very_ good about
it, but, when the audio isn't working for the first session on
Monday morning (from my observations, a common problem) I
believe that I, and remote participants who are more shy about
complaining than I am, should feel that we are entitled to have
that situation treated as a major, probably session-stopping,
problem, on a par with the in-room lights or projector not
working or no one in the room being able to hear a speaker.
Similar comments apply to not being able to make a comment or
ask a question during a meeting because of the way the
microphone lines are being managed.

Now I do think that having some fee waiver systems for hardships
is important, but actually no more important than having similar
arrangements for hardship waivers for in-person attendees.  And
I don't think people who just want to listen (or watch) remotely
should be charged or asked to identify themselves as the price
for doing so.   Nothing prevents someone who doesn't way to pay
the fee or ask from a waiver fro listening and then taking
issues up on the relevant mailing list either.  But, if nothing
else, in the interest of openness and fairness, those who are
_participating_ remotely ought to be registered (like everyone
else), identified as participating in specific WG sessions when
they do so (like everyone else), and that it is entirely
reasonable that there be a corresponding registration fee (as
for everyone else).

    john

---

From: Mary Barnes <[email-masked]>
Date: Sat, 14 Feb 2015 12:21:50 -0600

On Sat, Feb 14, 2015 at 10:43 AM, Yoav Nir <[email-masked]> wrote:

> All participants, including area directors pay their own way (or their
> sponsor pays their way). But 3 star hotels don’t typically have good enough
> conference centers.
>
[MB] True. But, perhaps considering other sorts of conference facilities
with nearby hotels could be an option.  In particular, if we really do
improve remote participation to the point that we reduce the number of
onsite participants, the size requirement for the conference facilities
goes down. [/MB]

>
> Also, supporting remote participation in a better way than it works today
> costs more money than is being expended today. That money has to come from
> somewhere.
>
[MB] In one sense yes. But, given some of the work in RAI, this cost should
be going down.  The ability to participate in a meeting remotely with a
very rich multimedia experience is something that we certainly ought to be
able to do with the protocols we're developing.  I have worked extensively
in an environment where these technologies are essential to business (as I
imagine many of us having) and you no longer need an expensive dedicated
video unit to have a high quality experience.  We are developing these
technologies in IETF in the RTCWEB and CLUE WG.  If we can't leverage those
protocols for our own meetings, then we've not done something right in the
IETF.  And, actually this is already happening with Meetecho.   Also,
considering that much of what we need is built by vendors who have
significant participation in the IETF, I would think that the net financial
impact could be optimized.
I think the biggest problem that high quality remote participation will
introduce is that companies will become even more reluctant to send people
to the face to face meetings.  I do still see value in people attending
face to face IETF meetings with some regularity, I strongly believe that
IETF moving to a model that doesn't require so many people to travel to get
the work done is a good thing and ought to be a longterm objective.  I'm
not a financial expert so I can't posit that this will make sense from a
business model, but IETF is a non-profit and in that respect doing the
right thing for the community should be the overarching objective.   [/MB]

> > Being smart saves money and it's fun..  A lot more that looking at the
> users's pocket.. But off course it's not as easy
> >
> > Carlos
> > Internet Society Ecuador
> > www.isoc.org.ec
> > Síguenos @isocec
> >
> >>>>>>>> It's possible that we could collect meeting fees from remote
> attendees, offering a hardship exemption for those who can't afford it.
>  That would depend on remote attendance working better than it does now, I
> think, but it would be unfortunate if the main impediment to making remote
> attendance work well were that we didn't want to lose meeting revenue.
> >>>>>>>>
> >>>>>>>> [MB] I totally agree on this latter point.  I'm very conflicted
> about charging for remote participation, but perhaps something nominal.
> It's also quite possible that if we improve the quality, we will get more
> remote participants.
> >>>>>>>
> >>>>>>> A requirement (at least at first) to allocate n% of remote
> participation fees directly to expenses related to the improvement of
> remote participation would make this a lot more feasible.
> >>>>>>

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 14 Feb 2015 21:15:34 +0000

On 14/02/15 18:21, Mary Barnes wrote:
> And, actually this is already happening with Meetecho.

I think we ought forget about charging for remote attendance until
remote attendance is much better. Remotely attending IETF-91 via
meetecho was a good bit better than I expected but is nowhere near
the point where we could charge. Let's make it work first, and then
see how that affects attendance and then figure out charging models
and not try do that backwards by starting to discuss charging models
for something that doesn't yet exist.

S.

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Feb 2015 16:16:41 -0500

--On Saturday, February 14, 2015 12:21 -0600 Mary Barnes
<[email-masked]> wrote:

> [MB] True. But, perhaps considering other sorts of conference
> facilities with nearby hotels could be an option.  In
> particular, if we really do improve remote participation to
> the point that we reduce the number of onsite participants,
> the size requirement for the conference facilities goes down.
> [/MB]

Yes and no.  I haven't done site selections or evaluations for
large meetings in many years but, unless things have changed
significantly, the number of bigger-than-a-suite rooms we now
seem to "need" (IAB office, IESG office, IAOC office, a Nomcom
office and meeting room, an ISOC office or two, Secretariat work
space, terminal room, NOC, ...) could turn into as much a
barrier to moving to a smaller or less pretentious HQ hotel as
the required number and size of WG and plenary spaces.

> [MB] In one sense yes. But, given some of the work in RAI,
> this cost should be going down.  The ability to participate in
> a meeting remotely with a very rich multimedia experience is
>...

Yes, but there is an issue that more and fancier protocols
and/or hardware doesn't solve, which is that running these
things well and at high quality tends to need serious
operational commitments, which means more staff.  One can reduce
the staff requirement somewhat with _really_ fancy and expensive
technology, but the tradeoff may not be wonderful.  I'm not
talking about the complex technical stuff here, I'm talking
about things closer to "camera gives good view of carpet" and
"if that speaker is going to pace the floor while talking,
either the camera needs to follow or someone needs to apply a
short leash" to say nothing of the perennial microphone
announcement, "MY NAME IS <mumble>".

Similarly, very high quality remote participation with lots of
participants at lots of different locations tends to either put
a premium on participant training and/or typing speed and/or a
requirement for trained moderators who can control both in-room
and remote conversation flow.  Again, not really technical
issues, but not so easy to resolve, at least without cultural
changes.

> I think the biggest problem that high quality remote
> participation will introduce is that companies will become
> even more reluctant to send people to the face to face
> meetings.

Yes, although, borrowing from other threads, the proportion of
design-level people to professional standardizers might start to
improve again if participation didn't require members of the
former group to give up whole days or a whole week.  Of course,
that also has disadvantages in terms of competent cross-area
review.  Not simple.

>  I do still see value in people attending face to
> face IETF meetings with some regularity, I strongly believe
> that IETF moving to a model that doesn't require so many
> people to travel to get the work done is a good thing and
> ought to be a longterm objective.  I'm not a financial expert
> so I can't posit that this will make sense from a business
> model, but IETF is a non-profit and in that respect doing the

Again, be careful what you wish for, lest trying to optimize for
people attending face to face meetings while not requiring so
much travel, bring a situation in which almost all of the people
at a meeting in Region X are from Region X, almost all of those
at a meeting in Region Y are from Region Y, etc.  That loss of
diversity in individual f2f meetings, even if it improved
statistical diversity over a year or two, would not, IMO, be a
desirable outcome.

    john

---

From: Ted Lemon <[email-masked]>
Date: Sat, 14 Feb 2015 16:17:05 -0500

On Feb 14, 2015, at 1:21 PM, Mary Barnes <[email-masked]> wrote:
> I think the biggest problem that high quality remote participation will introduce is that companies will become even more reluctant to send people to the face to face meetings.  I do still see value in people attending face to face IETF meetings with some regularity, I strongly believe that IETF moving to a model that doesn't require so many people to travel to get the work done is a good thing and ought to be a longterm objective.

Fewer F2F meetings per year would be one way to accomplish this: encourage more interim virtual meetings, and then cut back to two or one f2f meeting per year.   Of course now we have strayed a very long way from the original topic, but I guess that was inevitable... :)

---

From: Ted Lemon <[email-masked]>
Date: Sat, 14 Feb 2015 16:20:30 -0500

On Feb 14, 2015, at 4:16 PM, John C Klensin <[email-masked]> wrote:
> Of course,

I question whether there is a real opportunity for cross-area review at F2F meetings.   The schedules are jam-packed.   As an AD, there was never an opportunity to see what other areas were doing, and as an individual participant it's hard to know where to go, and there are often conflicts.

---

From: Ray Pelletier <[email-masked]>
Date: Sat, 14 Feb 2015 16:36:50 -0500

+1

Ray

> On Feb 14, 2015, at 4:15 PM, Stephen Farrell <[email-masked]> wrote:
> 
> 
>

---

From: Mary Barnes <[email-masked]>
Date: Sat, 14 Feb 2015 16:44:00 -0600

Ray,

Please read my response to Stephen.  I didn't at all suggest what he seems
to have interpreted me
to have done.

Regards,
Mary.

---

From: Mary Barnes <[email-masked]>
Date: Sat, 14 Feb 2015 16:46:47 -0600

Rather than respond inline, I'll respond generally as I personally haven't
gone down the road towards looking at the impacts
which rightly should be considered.   I'm not saying improving remote
participation is just a piece of cake and there
are no barriers to doing so.  I'm just suggesting it be a consideration and
not be tossed aside or be given
low priority  just because some folks are concerned about the impact on
"IETF culture" and the financial bottom line.

Regards,
Mary.

---

From: Mary Barnes <[email-masked]>
Date: Sat, 14 Feb 2015 16:47:04 -0600

> for something that doesn't yet exist.
[MB] As I said in an earlier thread, I'm personally very conflicted as to
whether this should be
considered at all.   I'm not at all suggesting that we do this now - I'm
just suggesting that improving
the experience and changing how we work a tad might be possible - that's
all.  Others seem to think that
we would negatively impact, the IETF culture, which is primarily what I
disagree with. .  I do recognize that
quality isn't quite there yet, but it's pretty darn good and maybe good
enough.  The bigger issue might well end
up being whether remote participants have a high enough quality Internet
connection for this to work
well for enough people.  I certainly do and most folks in large
metropolitan areas do.  The good news
is that ISOC has an initiative to try to improve the situation:
http://www.internetsociety.org/access
[/MB]

---

From: John Leslie <[email-masked]>
Date: Sat, 14 Feb 2015 17:51:28 -0500

Ray Pelletier <[email-masked]> wrote:
>> the point where we could charge.

   I disagree.

   Meetecho _may_ be still at the point where folks would demand their
money back ;^) but the biggest problems were simply it being hard to tell
the difference between a microphone turned off and the sound not working.

   This could be fixed rather quickly, if we had anybody on site who
would communicate this difference.

   Obviously (and the Meetecho folks would be the first to agree) there
is room for improvement. But I didn't find Meetecho to be the source
of my frustrations.

   The big frustration is a five-minute lag between when I want to ask
a question and when I _might_ get that question repeated at the mike.
By then, the question may no longer be appropriate!

   Fixing this alone would be worth $50 a day to me. I don't think I'm
alone in this...

>> Let's make it work first, and then see how that affects attendance
>> and then figure out charging models

   Is there anybody besides the Meetecho folks whose task it is to
"make it work"? Is there anybody _including_ the Meetecho folks who
has the ability to arrange similar priority at the mike to that of
on-site participants?

   If not, we're simply not trying to make it work. :^(

>> for something that doesn't yet exist.

   There are _many_ products that _won't_ be developed if the folks
who control the resources don't believe there _is_ a workable charging
model.

   :^( :^( :^(

---

From: John Leslie <[email-masked]>
Date: Sat, 14 Feb 2015 18:06:10 -0500

Mostly, this is just "+1" -- but I'll expand on a few things...

   +1

   Note also that visa problems are real! Even those able to clear
the time and cover the expense may find themselves prohibited from
attending in person.

> right to expect.i

   +1

> In general, people have been _very_ good about it, but, when the
> audio isn't working for the first session on Monday morning
> (from my observations, a common problem) I believe that I,
> and remote participants who are more shy about complaining than
> I am, should feel that we are entitled to have that situation
> treated as a major, probably session-stopping, problem, on a par
> with the in-room lights or projector not working or no one in
> the room being able to hear a speaker.

   +1

   This problem is endemic with the first Monday sessions, and
would be _so_ easy to fix!

   +1

   Being prevented from asking timely questions _is_ the difference
between attendance and participation.

> arrangements for hardship waivers for in-person attendees.

   +1

> And I don't think people who just want to listen (or watch)
> remotely should be charged or asked to identify themselves as
> the price for doing so.

   +1

>...
> But, if nothing else, in the interest of openness and fairness,
> those who are _participating_ remotely ought to be registered
> (like everyone else), identified as participating in specific
> WG sessions when they do so (like everyone else), and that it
> is entirely reasonable that there be a corresponding registration
> fee (as for everyone else).
   +1

   (but that fee need not cover the cookies... ;^)

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Feb 2015 18:08:12 -0500

--On Saturday, February 14, 2015 16:20 -0500 Ted Lemon
<[email-masked]> wrote:

> I question whether there is a real opportunity for cross-area
> review at F2F meetings.   The schedules are jam-packed.   As
> an AD, there was never an opportunity to see what other areas
> were doing, and as an individual participant it's hard to know
> where to go, and there are often conflicts.

I agree as far as actual reviews are concerned.  But, at f2f
meetings, at least some of us (almost certainly not including
ADs) take the opportunity to drop in on WGs we don't follow
carefully and do other things that facilitates reviews on other
occasions.  Without that, expecting a careful and in-depth
review of a document that is completely unfamiliar by someone
from a different area of work and perspective (and not a
designated area reviewer or equivalent) show is, I suspect,
fairly unrealistic.  

One could get most of that same sort of exposure remotely, but
it requires more effort.  It would also not be consistent with
my hypothesis of a technical expert/ designer remotely
participating in a single WG discussion or two without being
tied up for the week.

It would also help remote participants stay in touch if our
minutes contained real WG and Area status reviews, but doing
that well is hard work and another kind of expense.

    john

---

From: John Leslie <[email-masked]>
Date: Sat, 14 Feb 2015 18:19:32 -0500

+1

> [MB] ... The ability to participate in a meeting remotely with a
> very rich multimedia experience is something that we certainly ought
> to be able to do with the protocols we're developing.

   If we're not making progress, there's something wrong with the WG
charters.

> I have worked extensively in an environment where these technologies
> are essential to business (as I imagine many of us having) and you
> no longer need an expensive dedicated video unit to have a high
> quality experience. We are developing these technologies in IETF
> in the RTCWEB and CLUE WG. If we can't leverage those protocols for
> our own meetings, then we've not done something right in the IETF.

   +1

> ... I think the biggest problem that high quality remote participation
> will introduce is that companies will become even more reluctant to
> send people to the face to face meetings.

   Many companies are _already_ reluctant...

> I do still see value in people attending face to face IETF meetings
> with some regularity, I strongly believe that IETF moving to a model
> that doesn't require so many people to travel to get the work done
> is a good thing and ought to be a longterm objective.

   Hasn't it _been_ one?

   Those of us who can't arrange the time and/or money are simply
asking, "If not now, when?"

---

From: Dave Cridland <[email-masked]>
Date: Sat, 14 Feb 2015 23:25:54 +0000

On 14 February 2015 at 21:20, Ted Lemon <[email-masked]> wrote:

> I question whether there is a real opportunity for cross-area review at
> F2F meetings.   The schedules are jam-packed.   As an AD, there was never
> an opportunity to see what other areas were doing, and as an individual
> participant it's hard to know where to go, and there are often conflicts.
>
>
Well, you could always scrap the WG meetings at the IETF weeks and make it
all broad topic meetings, or make the WG meetings actually aimed at
presenting the work to outsiders (ie, deliberately court the people who
don't read the drafts that we normally moan about).

I am, you'll appreciate, thinking aloud here, but if the real value in
face-to-face meetings is in personal contact (agreed) and cross-area review
(wishful thinking), then why not optimize for those cases and use online,
remote, technology for the stuff where it works well?

Dave.

---

From: Rich Kulawiec <[email-masked]>
Date: Sat, 14 Feb 2015 18:30:00 -0500

On Sat, Feb 14, 2015 at 04:17:05PM -0500, Ted Lemon wrote:
> Fewer F2F meetings per year would be one way to accomplish this:
> encourage more interim virtual meetings, and then cut back to two or
> one f2f meeting per year.

At most.  Of all possible organizations, the IETF should be
best-positioned to figure out how to enable broad/deep participation in
virtual-only meetings.  Travel is very expensive (not just in terms of
money, but in terms of time) and increasingly hazardous to the security
and privacy of anyone crossing nearly any border or even just getting
on an airplane.  Harrassment, detainment, and invasive searches are
now commonplace, as are the confiscation and copying of disk drives,
thumb drives, phones and anything else that can hold data. [1]

This may not be a big deal to those whose income and/or support make
the cost inconsequential, or to those whose nationality reduces the
risks at borders, or to those with the technical savvy to avoid having
their private data offloaded.  But to those who aren't so wealthy
(or supported), and to those whose nationality almost guarantees them
unlimited hassles, it may well be.  Yet those are the very people
whose diversity may well enrich the IETF and who might potentially
stand to benefit the most from extensive interaction.

---rsk

[1] Not to mention thefts, assaults, and false arrests on completely
fabricated charges, e.g.:

	Lawsuit: TSA Supervisor Got Traveler Arrested For Bogus 'Terroristic Threat' Charge, Lied About Incident In Court
	https://www.techdirt.com/articles/20150206/10504129938/lawsuit-tsa-supervisor-got-traveler-arrested-bogus-terroristic-threat-charge-lied-about-incident-court.shtml

---

From: John Leslie <[email-masked]>
Date: Sat, 14 Feb 2015 18:33:48 -0500

(I do generally try to avoid high Narten scores, but...)
 
John C Klensin <[email-masked]> wrote:
> 
>... but there is an issue that more and fancier protocols and/or
> hardware doesn't solve, which is that running these things well
> and at high quality tends to need serious operational commitments,
> which means more staff.

   There will be costs, but they _could_ be quite minimal:

> One can reduce the staff requirement somewhat with _really_ fancy
> and expensive technology, but the tradeoff may not be wonderful.
> I'm not talking about the complex technical stuff here, I'm talking
> short leash"

   These don't require on-site staff to notice, so probably these
could be _entirely_ covered by volunteers. Doing something about it
probably would require on-site staff -- but that's needed to make
the sound+picture useful to archive the session anyway...

> to say nothing of the perennial microphone announcement, "MY NAME
> IS <mumble>".

   That's really no worse for remote participants.

> issues, but not so easy to resolve, at least without cultural

   Actually, there are many conferencing systems which already
have solutions to these problems. It's just a learning-curve
issue -- and so far we've avoided learning them.

>... Again, be careful what you wish for, lest trying to optimize
> for people attending face to face meetings while not requiring so

   +1

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Feb 2015 18:45:54 -0500

--On Saturday, February 14, 2015 16:46 -0600 Mary Barnes
<[email-masked]> wrote:

> Rather than respond inline, I'll respond generally as I
> personally haven't gone down the road towards looking at the
> impacts
> which rightly should be considered.   I'm not saying improving
> remote participation is just a piece of cake and there
> are no barriers to doing so.  I'm just suggesting it be a
> consideration and not be tossed aside or be given
> low priority  just because some folks are concerned about the
> impact on "IETF culture" and the financial bottom line.

Mary,

I completely agree with the above.  I've just been thinking a
lot about remote participation since starting to do it fairly
frequently a couple of years ago.    The thing I've been saying
all along is, I think, consistent with your comment above: I
think improved arrangements for remote participation would be
helpful to the IETF and would improve overall technical
diversity and competence as well as whatever benefits it would
have for geographical, demographic, or sociological diversity.
But I think there is a key question about whether the IETF is
really serious about such participation.  If we are, it has
implications for a lot of ways in which we measure and evaluate
participation (including, but not restricted to, Nomcom
eligibility) as well as requiring clear responsibility and
accountability for having effective tools and mechanisms and
having them work.  

This is why I disagree slightly with Stephen (and Ray): I'm
willing to pay a registration fee even before all of the tools
are completely in place because I think that collecting such a
fee represents an IETF (or at least IAOC) commitment that remote
participants are "real" and need to be supported, not just
people who are nice to have around the periphery but who don't
really count.   From my point of view, Ray's endorsing a "don't
charge until everything is working well" position is fine, but,
if we (as a community) are serious, I'd expect to see a
comprehensive plan from the IAOC about what the steps are going
to be to get to an appropriate level of service and what the
planned schedule is for those steps.  And I'd expect to see it
at the IETF 92 plenary.    

Of course, it is also possible that we are not serious yet or,
as John Levine put it, simply not trying to make it work.

best,
    john

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 15 Feb 2015 00:16:21 +0000

On 14/02/15 22:47, Mary Barnes wrote:
> I do recognize that
> enough. 

My experience of being remote was that it was pretty good,
but for a lot of the WGs where I listened in, I was able to
recognise voices of folks I know, and was also able to use
jabber to ask various people there about the goings-on. I
don't think that'd be that common for remote participants,
and without it, the experience would have been much worse I
think. (But it'd be interesting to hear otherwise if that's
the case.)

And while remote listening (and the occasional prod via
jabber) was fairly workable for me, as others have noted
I don't think we're anywhere near solving the problem of
workable remote input in general.

I fully agree that we ought be working more on all of this
though.

S.

---

From: Randy Bush <[email-masked]>
Date: Sun, 15 Feb 2015 10:10:28 +0900

> remote attendance is much better.

given the current state, we should pay people to endure it.
'participate' is not an appropriate term.  it's like watching tv;
though today, your tv listens to you [0].

randy

--

[0] http://www.cnet.com/news/samsung-changes-smarttv-privacy-policy-in-wake-of-spying-fears/

---

From: Ted Lemon <[email-masked]>
Date: Sat, 14 Feb 2015 21:11:18 -0500

> has the ability to arrange similar priority at the mike to that of

A tremendous amount of work goes on behind the scenes to make this work, not just the meetecho folks.   I remember reporting a problem in a meeting and having Alexa show up five minutes later checking to see if it was fixed, and I know that other folks from AMS and from the NOC work hard on this.

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Sun, 15 Feb 2015 02:33:27 +0000

The one thing I haven't heard raised is the risk of disengagement by folks who only participate and contribute in particular working groups and are not "IETFers" in the sense that many of you are. You may find that many remote contributors to working groups that are not sponsored by enterprises will find other venues to contribute their time and energy. 

> -----Original Message-----
> From: ietf [mailto:[email-masked]] On Behalf Of Randy Bush
> Sent: Saturday, February 14, 2015 8:10 PM
> To: Stephen Farrell
> Cc: ietf
> Subject: Re: Remote participation fees [Re: Updating BCP 10 -- NomCom
> ELEGIBILITY]
> 
> 'participate' is not an appropriate term.  it's like watching tv; though today,
> your tv listens to you [0].
> 
> randy
> 
> --
>

---

From: Ted Lemon <[email-masked]>
Date: Sat, 14 Feb 2015 21:50:45 -0500

On Feb 14, 2015, at 9:33 PM, MH Michael Hammer (5304) <[email-masked]> wrote:
> The one thing I haven't heard raised is the risk of disengagement by folks who only participate and contribute in particular working groups and are not "IETFers" in the sense that many of you are.

This is pretty easy to do even if you attend in person.

---

From: Randy Bush <[email-masked]>
Date: Sun, 15 Feb 2015 11:59:48 +0900

hi 5304,

> The one thing I haven't heard raised is the risk of disengagement by
> folks who only participate and contribute in particular working groups
> and are not "IETFers" in the sense that many of you are.

you are replying to someone who goes to the meetings and hides in the
noc to avoid the social scene etc.  i go out to the very few wgs where i
have concerns; though have taken to just listening to the audio and
going to the wg if i really need to say something.  it's a form of
self-regulation.

real, N way (for N>1) [0], remote participation would do me just fine.

randy

---

[0] - we have all been here before -- crosby, stills, nash, and young

From: Randy Bush <[email-masked]>
Subject: Re: [vmeet] draft-ietf-genarea-rps-reqs comments
To: [email-masked]
Date: Sat, 07 Jan 2012 01:36:35 +0900

putting my money where my mouth is ...

for personal and somewhat superstitious reasons, i do not go to meetings
in the southeastern united states.  so for atlanta, i would like
  o to be able to see and hear any of the wg/bof meetings and plenaries
  o when there are slides or movies, i want to be able to see them or
    even pull a pdf to view locally
  o i would like semi-pro video where i can see the speaker, floor
    speaker, etc.  merit used to do a quite good job of this for nanog
  o i would like to be able to make a presentation, with my face visible
    and show slides and hear and answer questions from the floor and
    other remote participants
  o i would like to be able to speak at the virtual floor mic and, if
    possible, have my face shown to the meeting room and to other remote
    folk

i do not want to constrain or over-specify how this works.  i.e. how the
speaking stone is handed around, ...  it's gonna be hard enough to do
without requiring 42 kinky sub-features and have know-it-all amateurs
micro-specifying it.

as much as possible the protocols and the software at the remote end
should be non-proprietary.  and it must run on unix, mac, linux, and
optionally ms-windows. :)

randy

ps: shoot the jabber/im as a formal channel.  it's fine for an informal
    ad hoc social s[n]ide channel

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 15 Feb 2015 00:22:02 -0500

On February 14, 2015 9:33:27 PM EST, "MH Michael Hammer (5304)" <[email-masked]> wrote:
>and are not "IETFers" in the sense that many of you are. You may find
>that many remote contributors to working groups that are not sponsored
>by enterprises will find other venues to contribute their time and
>energy. 

That and it would seem odd to me to periodically move the work of an open standards group completely behind a paywall. 

Personally, I'm glad to participate here and there as time and interest allow. There's plenty of other things I could contribute to though. I think it's not very likely I'd pay to participate remotely. 

Scott K

---

From: Dave Cridland <[email-masked]>
Date: Sun, 15 Feb 2015 09:26:59 +0000

> think. (But it'd be interesting to hear otherwise if that's
I attended IETF 63 in Paris, and IETF 68 in Prague. I've attended two
interim meetings (one in London for Lemonade, one in Brussels for XMPP). I
popped my head through the door of London for literally 15 minutes or so at
the end of one evening.

I recognise most voices in the working groups I frequent, including those
I've never met; I've only ever used the audio feeds and XMPP chatrooms.
This includes both the WGs I'm active in, and most of those that I've
looked in on over the years.

I find it somewhat bemusing that the reverse is not true. Not only do
people obviously not know my voice or face, but they sometimes don't know
my name either; even when we've worked in the same working group and had
lengthy conversations on mailing lists.

> workable remote input in general.
Yeah, this. The number of times I've wanted to "say" something only to find
it's said late, and quite often hasn't translated from text.

Seriously, I've often wished that Cyrus Daboo (who sounds very like me;
I've never met him) would read my questions and comments out; I'm sure
that'd work better,

---

From: Dave Cridland <[email-masked]>
Date: Sun, 15 Feb 2015 09:30:57 +0000

On 15 February 2015 at 05:22, Scott Kitterman <[email-masked]> wrote:

> That and it would seem odd to me to periodically move the work of an open
> standards group completely behind a paywall.
>

Oh, I'm glad that someone else said this - I was thinking it. I would
rather figure out a way to have the meetings themselves free to attend.

I was getting worried that it'd end up with the IETF charging to
participate on mailing lists or something. $150 per draft upload, anyone?

Charging the people we want to be contributing just doesn't seem logical to
me.

Dave.

---

From: Christer Holmberg <[email-masked]>
Date: Sun, 15 Feb 2015 11:40:16 +0000

Hi,

It can't be that difficult to make a tool that everyone (including those participating f2f) use to "line up" behind the microphone.

Many virtual meeting tools already provide a raise-your-hand feature.

Regards,

Christer

---

From: John C Klensin <[email-masked]>
Date: Sun, 15 Feb 2015 08:31:09 -0500

--On Saturday, February 14, 2015 21:11 -0500 Ted Lemon
<[email-masked]> wrote:

>>   Is there anybody besides the Meetecho folks whose task it
>>   is to "make it work"? Is there anybody _including_ the
>> Meetecho folks who has the ability to arrange similar
>> priority at the mike to that of on-site participants?
> 
> A tremendous amount of work goes on behind the scenes to make
> this work, not just the meetecho folks.   I remember reporting
> a problem in a meeting and having Alexa show up five minutes
> later checking to see if it was fixed, and I know that other
> folks from AMS and from the NOC work hard on this.

Ted,

I agree about the efforts and have said that before.  I think we
would be much worse off without those huge, and largely
successful, efforts to remedy or patch things when problems are
noticed.

However, the example you note is part of my concern about
whether the IETF is serious about remote participation.  As John
Leslie and I have noted, "something doesn't work in some 9AM
Monday WG session" seems so common as to be routine.  Sometimes
those things get fixed quickly, sometimes those of us who are
remote are basically told that a fix cannot be made without
disrupting the meeting so things will be fixed and tested after
11:30.  

If we were serious, we'd have these things up and tested before
the sessions start.  If we were serious, we'd promise
availability of remote participation (or at least remote
observation) of Sunday afternoon tutorial sessions and we would
test things in advance for them. Most of those sessions are as
valuable to those who are remote as to those who happen to be
present, and, especially for newcomers, some may be more
important for the remote people who cannot ask questions in the
halls.    If we were serious, it wouldn't be left to remote
participants to report that a camera was giving a lovely view of
the carpet in the meeting room or that speakers were otherwise
off-camera, and, when the problem was detected, fixing it would
be a priority, not something deferred to avoid interrupting the
speaker (or we'd be using remote-controllable cameras).   

If we were serious, we would establish a system that works and
use it and stop trying to half-support three systems (Meetecho,
WebEx, and the streamed audio/Jabber combination in parallel).
Of that list, Meetecho has been quote close to what we need
although I've got scaling concerns; WebEx is notoriously
unreliable, especially for unmoderated sessions with many
participants and many locations; and, if only because of the
combination of audio lag, typing lag, and microphone access lag,
we should stop pretending the last combination is worth the
trouble (there are good reasons for recording and probably
streaming audio, but they aren't related to remote
_participation_)   I don't believe that all of the items on the
list Randy provided three years ago and quoted in a recent note
are entirely necessary, but I'd certainly like to have them too.
If we were serious, we would have made some progress on that
list in three years.  As far as I can tell, we haven't.

And, coming back to the topic of this thread, if charging a
remote participation fee is a component of a serious commitment
by the IETF (rather than the dedication of a handful of
individuals), then let's charge a registration fee.   If it is
better to set a schedule for quality and well-supported remote
participation services, imposing a fee only when things are up
and running, that is fine but let's see the schedule.  By
contrast, if "there is no fee, remote participants get the best
service volunteers can deliver when they have time and without
disrupting meetings, and we have no incentive to announce a
schedule" is the IETF's policy in practice, then let's admit
that, acknowledge whatever it means for diversity of
participation, and move on.

     john

---

From: Ted Lemon <[email-masked]>
Date: Sun, 15 Feb 2015 08:43:02 -0500

On Feb 15, 2015, at 4:26 AM, Dave Cridland <[email-masked]> wrote:
> Seriously, I've often wished that Cyrus Daboo (who sounds very like me; I've never met him) would read my questions and comments out; I'm sure that'd work better,

What would really help with this problem, and I think would actually result in better line discipline, would be if the chairs ran a queue of commenters and anybody, on-site or remote, could put themselves on the queue, but you couldn't talk if you weren't on the queue.   This would give off-site commenters parity with on-site commenters.  Meetecho already allows you to present with video, so I don't see why it couldn't allow participants to comment using video as well.   We'd want cameras at each mic line that are on when it's that person's turn to speak, so that off-site people could see them.

I don't think this is particularly hard to do if we decide to do it; the problem is that right now meetecho support, which is really great compared to what we had before, is still not really taken seriously enough.

---

From: Ted Lemon <[email-masked]>
Date: Sun, 15 Feb 2015 08:48:20 -0500

On Feb 15, 2015, at 4:30 AM, Dave Cridland <[email-masked]> wrote:
> Charging the people we want to be contributing just doesn't seem logical to me.

This isn't really what's happening, though.   What's happening is that corporations tend to have budgets for things like conference attendance, and IETF is getting participants to tap those budgets to partially fund the organization.   The idea isn't really for participants who are contributing to have to pay, and we do have scholarship programs, although it would be unusual for a non-student to be sponsored in one of these programs.

Trying to turn the funding scheme into something "fair" isn't going to work, because the people who really benefit from this are users of the internet, and they don't even know who we are.   They are not going to fund us.   If we really think what we are doing is important, it's not unreasonable to expect us to try to figure out how to fund it.

The idea of a fee for off-site attendees is not to make them pay, but to provide a way for them to get their employers to pay, or for them to pay.   I realize this is a subtle distinction, but the point is that if you think what the IETF does is important, providing funding from an available budget is a good way to support it, and we have not yet identified any better way to support it.   If you think that asking attendees to take on this responsibility is inappropriate, then you should probably be thinking seriously about an alternative proposal for how to fund the organization.

---

From: Ted Lemon <[email-masked]>
Date: Sun, 15 Feb 2015 08:52:40 -0500

On Feb 15, 2015, at 8:31 AM, John C Klensin <[email-masked]> wrote:
> whether the IETF is serious about remote participation.
> [...]

In the interests of brevity I won't include the entire essay you wrote here, but I agree with everything you said in it.   My reason for pointing out the hard work of the people who make the meetecho go is to emphasize their hard work and that fact that that even more such hard work would be required to increase the level of service.

---

From: t.p. <[email-masked]>
Date: Sun, 15 Feb 2015 13:57:13 +0000

There has been a marked increase in the number of  interim meetings.
Using
http://www.ietf.org/meeting/interim/proceedings.html
as a guide, there were
18 in 2011
35 in 2012
45 in 2013
84 in 2014
13 in January 2015 alone.

With them comes a change in the way of working, perhaps rendering some
of our practices historic.

Of the 84 meetings listed for 2014, 21 left no other trace on the IETF
web site, no Agenda, no Minutes, no Proceedings.  Perhaps the WG
provided no materials, perhaps they did not happen; sometimes a
cancellation notice is apparent in the WG List Archives, other times
not.

Of the 63 that have left a trace, 6 produced no Minutes but did produce
slides or recordings and so presumably happened.

Of the 57 that produced Minutes, 18 produced no Agenda while in 13
cases, the Minutes contained no list of Attendees (goodbye Blue
Sheets?).

Only 26 meetings left a complete record, of Agenda, Minutes and
Attendees.

The meetings encompassed 30 Working Groups, of which 16 met once, 14
more than once, with one WG meeting 8 times.

What is more subjective is that, with Virtual Interims, increasingly the
only kind, there is a tendency for the WG Mailing List to no longer
provide a record of discussions, choices, consensus.  For example, they
may make greater use of github so that the minutes record a discussion
of options 1, 2 and 3 for Issue 29 with no indication of what the issue
or options are; a while later, they may record an update to option 3 so
it would now seem impossible to know what was discussed at the earlier
meeting.

Even with the better minutes, they never give the same sense as posts to
a mailing list of who was or was not in favour and how strong their view
was.

Of course, we still have WG Last Calls on the list but if at a future
date, an AD or GenArt reviewer wants to look back and see what options
were discussed and  how rough the consensus was, well, it may be
impossible.

A post in another thread recently said

> people who can't come to meetings and because it slows

Well, I disagree about slowing the work down but certainly agree with
the marginalisation, that WGs holding multiple Interims may tend to
develop an in-crowd of those that can participate with the world at
large only seeing the end result without knowing how it was arrived at
by whom.

Tom Petch

---

From: Michael Richardson <[email-masked]>
Date: Sun, 15 Feb 2015 10:34:44 -0500

Dave Cridland <[email-masked]> wrote:
    > that's extremely useful experience in leadership selection. Possibly
    > "last call" and "telechat" are also good enough - and interesting in as
    > much as a document that "fails" last call is arguably as important a
    > contribution as one that goes on to publication.

So, instead of AUTH48, you'd pick an earlier state like IESG Review.
    (see https://datatracker.ietf.org/doc/help/state/draft-iesg/, but
    actually I think write-up and the diagram are old and missing some states...)

That (perhaps correctly) skips things that go through the Independent
Submission Stream.

I agree that including the authors into the state above solves the problem of
a document which is so good that it needs no review and no tickets.
    
    > documents to only those which get scheduled time in a WG session isn't
    > so good; some WGs don't meet often, or even at all, and a document
    > that's so well crafted that it doesn't need discussion in a face to
    > face meeting is a really good document in my opinion. Clearly we don't
    > want arbitrary documents either. Any WG document would seem a "good"
    > contribution, and I suspect any document with a shepherd assigned
    > should be safe, too, since that effectively implies an expectation to
    > publish.

The problem with shephard assigned is that it could be very late in the
process, and a document which *does* get a lot of discussion and revision,
represents a lot of contribution of effort.   It is not unreasonable to me
that a single document occupies the entire IETF "life" of 3-4 persons.

    > 3) This works for WGs that use a ticketing system and have
    > meetings. Not sure what percentage that actually is.

A number of WG chairs don't like the current ticketing system, but increasing
I think that WGs are going to use some ticket system.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 15 Feb 2015 10:46:03 -0500

> meetecho was a good bit better than I expected but is nowhere near the
    > point where we could charge. Let's make it work first, and then see how

I feel we need to split the difference.

a) we need registration for remote attendance first.  It can come with
   a variety of fee levels, including "buy 1 get 1 free" (if you paid for in-person
   last time....).
   
b) the fee could initially be $20, and could increase as the technology gets
   better.

If we wait for the technology to get better (what exactly does that mean?),
there will be no clear incentive as to how to make it better, and no money to
actually make it better.

---

From: Dave Cridland <[email-masked]>
Date: Sun, 15 Feb 2015 18:27:58 +0000

On 15 Feb 2015 13:48, "Ted Lemon" <[email-masked]> wrote:
> This isn't really what's happening, though.   What's happening is that
corporations tend to have budgets for things like conference attendance,
and IETF is getting participants to tap those budgets to partially fund the
organization.   The idea isn't really for participants who are contributing
to have to pay, and we do have scholarship programs, although it would be
unusual for a non-student to be sponsored in one of these programs.
>
> Trying to turn the funding scheme into something "fair" isn't going to
work, because the people who really benefit from this are users of the
internet, and they don't even know who we are.   They are not going to fund
us.   If we really think what we are doing is important, it's not
unreasonable to expect us to try to figure out how to fund it.
>
> The idea of a fee for off-site attendees is not to make them pay, but to
provide a way for them to get their employers to pay, or for them to pay.
 I realize this is a subtle distinction, but the point is that if you think
what the IETF does is important, providing funding from an available budget
is a good way to support it, and we have not yet identified any better way
to support it.   If you think that asking attendees to take on this
responsibility is inappropriate, then you should probably be thinking
seriously about an alternative proposal for how to fund the organization.

I agree with everything you've written there. I don't think the IETF is pay
to play, as such, and I think funding this kind of thing is particularly
challenging at this scale.

But still, I'd rather avoid seeing participants as a revenue stream, so
yes, seriously looking at an alternative funding structure that doesn't
involve neither any non existent benefactors nor charging contributors
would be interesting.

Dave.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 16 Feb 2015 08:19:56 +1300

Tom,

That analysis is shocking and documents a significant breach of RFC 2418
(BCP 25) Section 3.1. I think the IESG needs to toughen up on this.

  "Interim meetings are subject to the
   same rules for advance notification, reporting, open participation,
   and process, which apply to other working group meetings.

   All working group sessions (including those held outside of the IETF
   meetings) shall be reported by making minutes available.  These
   minutes should include the agenda for the session, an account of the
   discussion including any decisions made, and a list of attendees. The
   Working Group Chair is responsible for insuring that session minutes
   are written and distributed,..."

      Brian

---

From: Dave Cridland <[email-masked]>
Date: Sun, 15 Feb 2015 19:38:11 +0000

> a document which is so good that it needs no review and no tickets.

That all sounds good. And yes, I think an independent stream submission is
reasonable to skip. I did consider suggesting that only IETF consensus
documents should count, but that is probably too narrow.

> that a single document occupies the entire IETF "life" of 3-4 persons.

Yes, true. As I said, I think measuring contribution is a very hard metric
to find, and there's an implicit suggestion lurking here that we're after
long-term, quality contributions - I'm not sure I disagree with that
incidentally.

But in general terms, any document that has any of those characteristics -
it's WG document, one that's getting WG "time" by some measurement, or one
that has a shepherd assigned - seems likely to be good for a document
uploader metric.

It might be interesting to have the tools adapted so the uploader can note
which of the authors have actively contributed this time around, too, to
avoid the situation where two co-authors have to share uploading.
Similarly, it might be useful for WG chairs to be able to tag a document as
having been discussed within their WG.

> I think that WGs are going to use some ticket system.

And, thinking about it further, we probably want to promote the use of a
ticket system anyway. In addition, I'm more comfortable with measurable
metrics.

Again, it might be useful for the ticket system to be able to record
whether the ticket resulted in useful discussion, and note who was
involved. Again, this is at the whim of somebody, but I really don't think
excluding whims is entirely a positive thing.

Dave.

---

From: Simon Pietro Romano <[email-masked]>
Date: Sun, 15 Feb 2015 21:15:01 +0100

Hello Christer, 

we have discussed this multiple times in the past. Ftom the technical standpoint, moderation (involving both locals and remotes) can indeed be implemented in many different ways (some of which are based on IETF standards, btw). We (Meetecho) have proposed and run an experiment called UMPIRE, whose details can be found, e. g., on the vmeet mailing list.

Cheers,

Simon

Il 15 Febbraio 2015 12:40:16 CET, Christer Holmberg <[email-masked]> ha scritto:
>Hi,
>
>It can't be that difficult to make a tool that everyone (including
>those participating f2f) use to "line up" behind the microphone.
>
>A tremendous amount of work goes on behind the scenes to make this
>work, not just the meetecho folks.   I remember reporting a problem in
>a meeting and having Alexa show up five minutes later checking to see
>if it was fixed, and I know that other folks from AMS and from the NOC
>work hard on this.

---

From: Simon Pietro Romano <[email-masked]>
Date: Sun, 15 Feb 2015 21:24:47 +0100

Hello John,

If we were serious (as we are) we'd keep on doing our best to improve remote participation support. Most of the functions you cite are, in my view, if not already there ay least VERY close to what can be defined a reliable service suite for remote participation. I encourage you to look back at past meetings and compare the remote experience you had from one meeting to the other in the last three years. Please don't tell me we didn't do giant steps towards a good service. And when I say 'we', I mean the IETF leadership, the secretariat,  the NOC, some people like you who have provided invaluable feedback on end users experience...and the Meetecho team.

Cheers,

Simon

---

From: John C Klensin <[email-masked]>
Date: Sun, 15 Feb 2015 17:49:23 -0500

--On Sunday, February 15, 2015 21:24 +0100 Simon Pietro Romano
<[email-masked]> wrote:

> If we were serious (as we are) we'd keep on doing our best to
> improve remote participation support. Most of the functions
> you cite are, in my view, if not already there ay least VERY
> close to what can be defined a reliable service suite for
> remote participation. I encourage you to look back at past
> meetings and compare the remote experience you had from one
> meeting to the other in the last three years. Please don't
> tell me we didn't do giant steps towards a good service. And
> when I say 'we', I mean the IETF leadership, the secretariat,
> the NOC, some people like you who have provided invaluable
> feedback on end users experience...and the Meetecho team.

Simon,

I know you are serious and that you and your colleagues have
been making incredible efforts to make things work.  I've said
that several times, tried to say it in this thread, and want to
say it again now.  I feel the same way about the Secretariat and
NOC staff. 

I agree that giant steps have been made, largely because of your
collective efforts.

At the same time, Monday morning problem patterns continue and
are dealt with as emergencies rather than something that can be
eliminated by adequate testing.   I've had you and your
colleagues tell me that a particular problem can't be fixed
during a WG session because it would be disruptive to the
session.  We've had a brief discussion or two about what you
could do with high-end remotely controllable cameras, especially
with a few more people helping out your team.  We still don't
have a regular plan (defined as something I can know about and
count on when I'm deciding whether to travel to a meeting) for
coverage of Sunday tutorial sessions.  We've also had a bit of a
discussion about whether it would be reasonable to interrupt
speakers to remind them not to pace the floor out of the camera
frame.

On a different dimension, several people, notably SM, have
pushed for better recognition of remote participants in the
process --including but not limited to concerns about binding a
Nomcom eligibility formula that stresses in-person meeting
attendance to several other things where rights to participate
in various ways are tied to Nomcom eligibility.  I don't know if
he would agree, but my sense has been that his ideas haven't
gotten off the ground --not because they were the wrong
solutions, but because there is a shortage of general
recognition that the issues are important.

I think there is a difference between "the best it is possible
to do with volunteer efforts (even nearly superhuman efforts)
and some help from the Secretariat" and "what we could be doing
if we really believed that effective and reliable remote
participation was critical to the IETF".  I agree that we
(again, due in very large measure to your efforts and that of
your colleagues) are getting quite close to that first
objective.  I think we would be even closer if the community
stopped wasting time, facilities, and bandwidth on WebEx and
concentrated those resources on Meetecho.  But what we are not
doing -- and what I think the community and its formal
leadership are not serious enough about-- is that second
category.  

As examples, should your colleagues and the NOC be told by the
IESG that, if there are technical problems with remote feeds or
input, you are entirely justified in stopping a WG meeting to
get those problems resolved and should do so?  If there is an
active discussion in a WG and a remote participant cannot
contribute to it do to lag, some technical problem, or how the
mic line is being managed, does she have the right to protest
and, if so, how should that protest be expressed?   As someone
else suggested, should we have a "put hand up" or "put me in the
queue"  function that applied to both those in the room and
remote participants on an equal (or fair) basis?   Should we
really move toward moderation of sessions and, if so, how and
under what rules?    If I'm remote and asking a question at the
virtual mic (or having it asked on my behalf) should my picture
be on the screen in the meeting room and projected to others
and, if so, how would be do that?

I'm sure you have a list of your own.  I also note that the
items on the list above are at least partially non-technical.

"Not serious" was a comment entirely about things like that
second list and the amount of serious (sic) attention it seems
(or, more accurately, does not seem) to be getting, despite your
efforts, my efforts, and the efforts of several others.

Thanks again for your efforts, without while remote
participation would be, IMO, hopeless rather than just about
good enough from a technical standpoint (and that in spite of
some factors working against you).

     john

---

From: Simon Pietro Romano <[email-masked]>
Date: Mon, 16 Feb 2015 00:28:58 +0100

John,

I share all of your wise comments. And I do know that the most challenging issues associated with remote participation (and with Meetecho as well) are of non-technical nature.  We're working hard, together with the IETF leadership and tech team, on further improving remote participation in view of the meeting in Dallas.  Hope you'll keep on helping us with your active participation, your feedbacks...and your patience!

Cheers,

Simon

---

From: Melinda Shore <[email-masked]>
Date: Sun, 15 Feb 2015 21:13:11 -0900

On 2/15/15 4:43 AM, Ted Lemon wrote:
> What would really help with this problem, and I think would actually
> result in better line discipline, would be if the chairs ran a queue
> of commenters and anybody, on-site or remote, could put themselves on
> the queue, but you couldn't talk if you weren't on the queue.  

I'm not going to pretend to be expert in the ways of the W3C, but
I did go to the recent TPAC meeting and have participated in some
calls, and they use IRC channels for this (and for other tasks),
with what appears to be considerable success.  So, it's not just
possible, but attainable, I think.

Melinda

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 16 Feb 2015 11:05:23 +0000

> A tremendous amount of work goes on behind the scenes to make this work, not just the meetecho folks.   I remember reporting a problem in a meeting and having Alexa show up five minutes later check…
Ted

That regrettable and is because the technology that we deploy is still
too immature.

As engineers our goal should be to make this sort of technology
just work so that no one is doing any significant work behind the
scenes to make it work.

Stewart

---

From: Ted Lemon <[email-masked]>
Date: Mon, 16 Feb 2015 09:07:07 -0500

> possible, but attainable, I think.

Yes, absolutely.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 16 Feb 2015 09:14:05 -0500

> just work so that no one is doing any significant work behind the

Yup.   This needs to be reduced to practice, and I think we ought to be making that an IETF objective, not just something that we're noticing some nice people doing for us.   Of course, this may or may not actually be possible.

---

From: James Gannon <[email-masked]>
Date: Sun, 15 Feb 2015 16:19:34 +0000

> If we wait for the technology to get better (what exactly does that mean?), there will be no clear incentive as to how to make it better, and no money to actually make it better.

Speaking purely about the remote participation aspect, I think one of the issues is that yes there are full featured solutions out there (Adobe Connect for example as used in ICANN) but they can be complex and not cheap.
The decision would need to be made whether remote participation is of a high enough priority to IETF to go after a full featured solution first, before then considering means of funding it. As Stephen says I think if remote participation fees are introduced without the remote experience to match it would not be a success.

I'll play devil's advocate for a moment, if you're really serious about having an immersive remote participation I'd would reach out to ICANN staff for some feedback, as personally I've found their remote sessions to work extremely well. I'm sure there are other organization's as well that could provide feedback just speaking from personal experience with remote attending ICNN meetings.

-----Original Message-----
From: ietf [mailto:[email-masked]] On Behalf Of Michael Richardson
Sent: Sunday, February 15, 2015 3:46 PM
To: Stephen Farrell
Cc: Sam Hartman; Carlos Vera Quintana; ietf
Subject: Re: Remote participation fees [Re: Updating BCP 10 -- NomCom ELEGIBILITY]

   

If we wait for the technology to get better (what exactly does that mean?), there will be no clear incentive as to how to make it better, and no money to actually make it better.

---

From: Carlos Vera Quintana <[email-masked]>
Date: Fri, 13 Feb 2015 19:13:55 -0500

Carlos Vera Quintana
0988141143
Sígueme @cveraq

---

From: Carlos Vera Quintana <[email-masked]>
Date: Sat, 14 Feb 2015 14:29:15 -0500

We always can work smart. Lot of countries are cheaper than the always selected ones.

The ICANN staff should work better to save and void work with the users's pocket to finance their jet set luxury conference's style. 

Carlos Vera Quintana
0988141143
Sígueme @cveraq

> El 14/2/2015, a las 13:21, Mary Barnes <[email-masked]> escribió:
> 
> 
> 
> [MB] True. But, perhaps considering other sorts of conference facilities with nearby hotels could be an option.  In particular, if we really do improve remote participation to the point that we reduce the number of onsite participants, the size requirement for the conference facilities goes down. [/MB] 
>> 
> [MB] In one sense yes. But, given some of the work in RAI, this cost should be going down.  The ability to participate in a meeting remotely with a very rich multimedia experience is something that we certainly ought to be able to do with the protocols we're developing.  I have worked extensively in an environment where these technologies are essential to business (as I imagine many of us having) and you no longer need an expensive dedicated video unit to have a high quality experience.  We are developing these technologies in IETF in the RTCWEB and CLUE WG.  If we can't leverage those protocols for our own meetings, then we've not done something right in the IETF.  And, actually this is already happening with Meetecho.   Also, considering that much of what we need is built by vendors who have significant participation in the IETF, I would think that the net financial impact could be optimized.  
> I think the biggest problem that high quality remote participation will introduce is that companies will become even more reluctant to send people to the face to face meetings.  I do still see value in people attending face to face IETF meetings with some regularity, I strongly believe that IETF moving to a model that doesn't require so many people to travel to get the work done is a good thing and ought to be a longterm objective.  I'm not a financial expert so I can't posit that this will make sense from a business model, but IETF is a non-profit and in that respect doing the right thing for the community should be the overarching objective.   [/MB] 
>> 
>> Yoav
>>

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 17 Feb 2015 13:16:32 -0800

> *really* important comment.

Bringing us back to the draft under discussion:

It sounds to me like our policies and tools around remote participation
haven't evolved yet to the point where we can set down some serious NomCom
eligibility criteria different from what's there now.  By that I'm not in
any way saying that these aren't important things to sort out, just that
it's not possible at the moment to come to consensus on what exact changes
we should make.

I suppose another way to look at that is: I have no idea what to write in
terms of replacement text for the current Section 4.14 of BCP 10 that
reflects a consensus here.  Does anyone else want to take a stab at it?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 17 Feb 2015 15:46:35 -0800

Hi Sam,

On Tue, Feb 17, 2015 at 1:42 PM, Sam Hartman <[email-masked]> wrote:

> I actually think the discussion is still ongoing, and I I'm unsure that
> all the threads have really resolved.
>

I think that's exactly my point, plus maybe trying to converge on text to
change in the BCP might help us toward resolution.

Your summary points also match my understanding, so I won't repeat them
here.

So, I think we're left with a number of questions:
>
> 1) Do we want to write forward-looking requirements for nomcom.  That
> is, do we want to be more open than our tools would support today in
> writing nomcom eligibility?
>

I think that would be a good idea at a minimum, if we can't come up with
new volunteer requirements right away.

> 2)  Do we want to allow folks  who are participants already to have more
> relaxed participation requirements (some remotely) for nomcom
> eligibility?
>

The consensus for that seems to be yes, though the challenge is nailing
down a new set of relaxed participation requirements on which there's
consensus.  In many cases the metrics I've seen are related to meetings, or
getting time on meeting agendas, etc., but that seems rather arbitrary
given that there's plenty of WG activity that never lands on agendas
because they don't need face time.  There were some attempts to tie
participation to a document reaching AUTH48, but that too is arbitrary
because a document might not even pass the IESG because of laggy
directorate or expert reviews.  On the flipside, there are some WGs that go
inactive for long periods of time, so perhaps they shouldn't count as
"participation" in between meetings.

> 3) If the answer to two is yes, should they still have to attend
> in-person meetings from time to time?  If so how often?
>

Good question.  Clearly the intent is for it to be less than 3/5, at least
for some period of time.  Michael suggested you qualify initially by
attending 3/5, but then as long as you keep up some lesser criteria, you
remain qualified.  Attending meetings is among those, but it's not the only
way to stay qualified.  Again, though, we need to hammer out the continued
qualification details.

> 4)  Do we want to allow those who have become members of our community
> remotely--chaired working groups, met some criteria--who have done the
> work to become significantly involved remotely even though it is hard to
> be eligible for nomcom?
>

More precisely: Should they be able to become selecting members of the
NomCom?  They will have to be able to participate in the multiple
interviews and other meetings that tend to take place during meetings.

> In conclusion, I think we've had a good wide-ranging discussion here,
> but I think it's time for the doc shepherd (not someone writing text,
> but someone chairing the discussion) to actually come along and chair.
> I've taken a loose stab above but would be happy to step aside for
> whoever actually has that responsibility.
>

Absent the Sergeant-at-Arms or an IESG member stepping up to moderate this
thread, I think that falls to me as the editor, and that's exactly what I
was trying to do here.

I propose that if we can't come up with a new set of criteria right away, a
very clear statement ought to be added to Section 4.14 saying that this
needs to be revisited "soon", as remote participation tools improve, so
that there isn't a continued bias toward well-funded participants among
those selecting the IETF's leadership.

-MSK

---

From: Michael Richardson <[email-masked]>
Date: Tue, 17 Feb 2015 20:35:37 -0500

Murray S. Kucherawy <[email-masked]> wrote:
    > On Fri, Feb 13, 2015 at 10:10 AM, Melinda Shore

    >> On 2/13/15 8:44 AM, Dave Cridland wrote: > Moreover, if you accept
    >> that the word "culture" is effectively > indistinguishable to
    >> outsiders from the term "status quo" (though the > intent is obviously
    >> different), it's really quite revealing. All this > "preserving the
    >> culture" talk comes out in an entirely different light.
    >>
    >> I think this is a really important comment.  I mean, *really*
    >> important comment.
    >>

    > haven't evolved yet to the point where we can set down some serious
    > NomCom eligibility criteria different from what's there now.  By that
    > I'm not in any way saying that these aren't important things to sort
    > out, just that it's not possible at the moment to come to consensus on
    > what exact changes we should make.

    > I suppose another way to look at that is: I have no idea what to write
    > in terms of replacement text for the current Section 4.14 of BCP 10
    > that reflects a consensus here.  Does anyone else want to take a stab
    > at it?

I've been super-busy on a dozen fronts, but I've been reading every message
on this thread.  I don't know if it is urgent get this text done this week.

I might propose that we ask for a BOF; but an in-person discussion would be
self-selecting.  It might be that having a virtual interim meeting after
Dallas would make more sense.   What if we had some kind of remote attendee summit?

I don't think that the quality of the tools or the question of fee or not
should prevent us from considering if there is a way to change the manner in
which ones remains eligible.  As written, I don't think that Dave Cridland
would ever become eligible again (since that would still be 3/5), nor, I
think were he already eligible, that he'd remain so, because you still have
to attend 1 meeting/year to remain eligible.

{It has become clear to me in thinking about this, that we need to make sure
we figure out when we measure properly}

---

From: Sam Hartman <[email-masked]>
Date: Tue, 17 Feb 2015 16:42:15 -0500

I actually think the discussion is still ongoing, and I I'm unsure that
all the threads have really resolved.
Let me try and summarize bits of the discussion I've seen:

* We'd like to encourage remote participation to grow our community;
  lots of discussion of fees and tool quality

* It's not clear that  this has reached a level where we'd know what to
  say about the BCP.

* A lot of people seem to believe that our current tools are good enough
  that members of our community can meaningfully participate for some
  meetings of the year remotely.

* I've not seen a claim that those members cannot follow the IETF well
  enough to participate in the nomcom.

* It's not clear how well enabling those folks to continue participating
  in the nomcom furthers the goal of diversity of nomcom.  It's not
  clear whether we care for the discussion at hand; there may be other
  reasons to relax the nomcom criteria that apply, haven't really seen
  enough discussion to tell what our consensus is there.

It sounds like we're probably not in a position with the tools today
that you could join the ietf community and participate well enough to be
a good nomcom member purely remotely.

---

From: Allison Mankin <[email-masked]>
Date: Wed, 18 Feb 2015 14:07:51 -0500

Working back through the mail threads on this, I realize that I didn't see
this when it first came out.​  My comments are inline.

On Feb 11, 2015, at 19:46, "Murray S. Kucherawy" <[email-masked]>
wrote:

On Wed, Feb 11, 2015 at 12:45 PM, Michael StJohns <[email-masked]>
wrote:

> ​

> actually been done, but its not barred.
​AJM:  Mike, I have always wanted the Nomcom to be like a search committee,
but on the other hand, search committees do produce short lists as you say,
and every time I've suggested this as a way to offset some of the problems
with the unevenness of the Nomcom participants' skills, in every hallway
chat I've ever had, I've received massive pushback.  So while the
confirming bodies approve the selection, the selection is made by the
Nomcom.  When confirming bodies request an alternative selection, this is
not something that can be seen by us outside the process, but the hallways
tell me also that Nomcoms who have experienced this are frustrated and
angry.  Therefore, we should perhaps add something to clarify that the
selection/the hire as it were, is really made by the confirming body, and
see how that fares in discussion.

AJM:  Also, hiring manager in my part of the world does not imply that the
person becomes a supervisor of the selectee - the term is used for the
person who makes the hire happen.

I think that's a fair point.  I've adjusted her suggested text accordingly.
​AJM: What do you think about explaining what a search committee is, in
that case, and noting the option to send short lists rather than single
selections.  ​

​AJM: I think it needs not to be.  But principles for fairness of
conducting the interview are reasonable to have as Best Practice (as per my
suggested text), IMO, because as I'll elaborate below: we shouldn't take
for granted that each collection of 10 voting members and 5 non-voting
members will have even such simple principles clearly in mind.

​AJM:   I disagree with the idea that advising people to be deliberative in
their review of candidates is trendy and could easily be superseded by a
next trend.  I don't care if the text needs to not reference a social
science expert on this, which I did in part because IETFers are rather
contrarian, and can be hard to make a plain and simple human resources
point.   I also think that the previous nomcom, where I was Past Chair,
practiced slow thinking and/or deliberative, thorough review, though
sometimes I found I had to help out.  My observation: the candidates who
are not famous enough to be recognized by most of the voting members what
their varied usually were summarized incompletely in the initial review
discussions - relevant accomplishments were omitted from discussion until
the thorough process (and some reminders) got those back onto the table.
Maybe text that says this more directly rather than by means of a social
science reference could be included.  I feel strongly that it is not a
given that Nomcoms will take their time and give enough diligence, given
the other pressures on them, especially time pressures.

​AJM: Thanks, Murray.

AJM: Final note: there's a truism that you need the great "staff" to do
great work.  A lot of people already are willing to devote many cycles to
our Nomcom process and this is worthy and necessary for the IETF.  ​

-MSK

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Feb 2015 14:19:23 -0500

--On Tuesday, February 17, 2015 16:42 -0500 Sam Hartman
<[email-masked]> wrote:

> I actually think the discussion is still ongoing, and I I'm
> unsure that all the threads have really resolved.

I generally agree with Sam's summary; let me add two questions
to his four:

(5)  Independent of Nomcom eligibility issues, do we need to be
explicit that those who volunteer to serve are making a
commitment to be physically present for specific meetings during
their term and, if so, which meetings?

(6) Whatever Nomcom eligibility requirements emerge from this
process, are they appropriate for all of the other things for
which we use Nomcom eligibility as a surrogate for
"participating adequately to do whatever this is"?  
 
> In conclusion, I think we've had a good wide-ranging
> discussion here, but I think it's time for the doc shepherd
> (not someone writing text, but someone chairing the
> discussion) to actually come along and chair.

+1

     john

---

From: Michael Richardson <[email-masked]>
Date: Wed, 18 Feb 2015 22:47:12 -0500

Sam Hartman <[email-masked]> wrote:
    > that you could join the ietf community and participate well enough to
    > be a good nomcom member purely remotely.

That is not being proposed.
This is not about initial eligibility.

Nor do I propose someone can be completely remote; you need to attend a bit,
just not as many as 2 meeting/year. (the 3/5 rule really means 2/3, because
otherwise, you'd wind up some years with only 2/5 and become ineligible)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Feb 2015 23:56:12 -0500

On Wed, Feb 18, 2015 at 2:07 PM, Allison Mankin <[email-masked]>
wrote:

> ​AJM:  Mike, I have always wanted the Nomcom to be like a search
> committee, but on the other hand, search committees do produce short lists
> as you say, and every time I've suggested this as a way to offset some of
> the problems with the unevenness of the Nomcom participants' skills, in
> every hallway chat I've ever had, I've received massive pushback.  So while
> the confirming bodies approve the selection, the selection is made by the
> Actually as I read Section 5.14, a NomCom can't present the confirming
body with a short list.  It specifically says "a single candidate for each
open position" and then goes on to include what backing materials have to
be provided.

> It appears to be moot in the face of what 5.14 says.  Are we looking to
open discussion of changing that?

> ​AJM:   I disagree with the idea that advising people to be deliberative
> in their review of candidates is trendy and could easily be superseded by a
I think removing the specific references to "slow thinking" and doing some
text massaging still captures the point you're trying to make.  I'll
include something in -03, which I'll post shortly.

-MSK

---

From: Eric Burger <[email-masked]>
Date: Tue, 24 Feb 2015 20:32:47 -0500

On Feb 15, 2015, at 1:27 PM, Dave Cridland <[email-masked]> wrote:
> On 15 Feb 2015 13:48, "Ted Lemon" <[email-masked] <mailto:[email-masked]>> wrote:
> >
> > On Feb 15, 2015, at 4:30 AM, Dave Cridland <[email-masked] <mailto:[email-masked]>> wrote:
> I agree with everything you've written there. I don't think the IETF is pay to play, as such, and I think funding this kind of thing is particularly challenging at this scale.
> 
> But still, I'd rather avoid seeing participants as a revenue stream, so yes, seriously looking at an alternative funding structure that doesn't involve neither any non existent benefactors nor charging contributors would be interesting.
> 
> Dave.
> 
I think Dave nails it. What sets the IETF apart from government-driven or corporate-driven SDOs is if you have an Internet connection, you can be a vibrant contributor to the IETF. You don’t need $5,000/year minimum to play (ITU-T) or ~$8,000/year to play (IETF if you physically go to all three IETF meetings + various face-to-face interims). The IETF works hard to remove barriers to participation. The last thing we need as we are just beginning to have success reaching out beyond North American, European, Japanese, S. Korean, and Australian mid-size to large corporations is to toss up a paywall, some as much as a month’s salary or more, for the ‘privilege’ of contributing to the IETF.

My guess is the Internet Society might weigh in, not with just words but deeds (=funding), to make sure the IETF does not close its doors to the global Internet community.

- Eric

---

From: Ted Lemon <[email-masked]>
Date: Tue, 24 Feb 2015 21:24:58 -0500

On Feb 24, 2015, at 8:32 PM, Eric Burger <[email-masked]> wrote:
> The last thing we need as we are just beginning to have success reaching out beyond North American, European, Japanese, S. Korean, and Australian mid-size to large corporations is to toss up a paywall, some as much as a month’s salary or more, for the ‘privilege’ of contributing to the IETF.

You know, it's really frustrating when you participate in a discussion, try to contribute helpfully, and then essentially get accused of being a blithering idiot by someone who didn't bother to consider the possibility that you might not be.   I'm sure you've had that experience too.   Heck, I've been the one who assumed the other person was an idiot too, so I can relate.

Anyway, if you think I was proposing a paywall, please go back and re-read what I actually wrote, and the rest of the discussion that followed, with the presumption in mind that I did _not_ mean to propose any such thing (because I didn't!), and see if the discussion still works, or if you find something I or someone else said that contradicts that assumption.

Thanks.

---

From: Benoit Claise <[email-masked]>
Date: Wed, 25 Feb 2015 11:40:23 +0100

Hi Tom,

In the WGs for which interim meetings are hold, there is really an 
increased speed of development. I can see this in NETMOD for example. So 
I've been encouraging more interim meetings, when we have the right 
audience in the call.

While we transition to more interim meetings, we have to admit that 
different WGs uses different tools (webex, github, svn, etherpad, irc). 
I don't believe that imposing the interim meeting tools is right. We 
should let each community chose.
Coming back to NETMOD (which is missing some entries at 
http://www.ietf.org/meeting/interim/proceedings.html), it works with 
SVN. Everything is documented at 
https://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/, as communicated on 
the mailing list. However, you are right, we should follow the process, 
and go the extra mile to complete the information at 
http://www.ietf.org/meeting/interim/proceedings.html

Final thought. I believe that some of our tooling will need to adapt 
anyway for interim meeting management. For example, I missed some 
interim meetings in my WGs based on the simple rule: if it's not in my 
calendar, it doesn't exist. The 
https://www.ietf.org/meeting/interim.html format is not very helpful for 
that. We need a way to easily schedule, evaluate conflicts, and download 
the .ics ... in a type of calendar view.

Regards, Benoit
> .

---

From: Yoav Nir <[email-masked]>
Date: Wed, 25 Feb 2015 13:11:04 +0200

I think this conversation is lumping together physical interim meetings (such as httpbis held while working on HTTP/2, and such as the TLS working group has been holding for the past half year) and virtual interim meetings (anything from conf calls to conf calls with shared presentations)

Both are productive, the physical meetings perhaps more so, but I am not happy with the proliferation of those meetings. Attending an IETF week is difficult enough. Flying all over the world to the various physical meetings excludes all but the most dedicated participants. And those who choose not to fly are very much excluded. All the meetings that I considered attending remotely either had no audio transmission, or totally lacked microphone discipline, and the jabber channel was ignored. For all intents and purposes they were a private meeting of the “in crowd”.  Remote participation in IETF week may have its faults, but remote participation in physical interims is unworkable.

Pair that with the large amount of progress that can be made in a full-day or a two-day meeting, and by the time the meeting is over, the conversation has progressed to such an extent that those who were not there are left behind, and the conversation continues on the mailing list even while the things that had been hashed out in the interim meeting are being “taken to the list”.

I think it is fine to encourage virtual interim meetings, but unless we can find a way to provide consistent remote participation for physical interims, we should not encourage them.

Yoav

---

From: t.p. <[email-masked]>
Date: Wed, 25 Feb 2015 12:28:01 +0000

----- Original Message -----
From: "Benoit Claise" <[email-masked]>
To: "t.p." <[email-masked]>; "ietf" <[email-masked]>;
<[email-masked]>
Sent: Wednesday, February 25, 2015 10:40 AM

> the .ics ... in a type of calendar view.

Benoit

Thanks for that.

In a sense, netmod is the worst offender in my analysis in that most of
the interim meetings that were announced and for which nothing has
appeared were netmod.  I think it safe to assume that they did not
happen but it would be better if that web page recorded this, perhaps by
a proceedings or minutes link that says simply that the meeting did not
happen.

Coupled with this, since several of the missing minutes (for me the
worst omission) were in the Routing Area, I flagged this to the Routing
Area list, and the minutes have since appeared.  The WG Chairs commented
that it would be good if tools generated an e-mail reminder a week later
if nothing had been filed.  This could then be either minutes or a
notfication that the meeting did not happen.  With virtual interims
happening a week apart, even a week's delay in the appearance of the
minutes is a serious handicap and I suspect that once the next meeting
has happened, then WG Chairs may lose interest in earlier ones.
Something to take up on the Tools list.

Again linked to this, but this is one for the IESG IMO, is that on some
lists, the view is that without an attendee list the minutes are
incomplete.  In Routing, I found the view that attendees should not be
listed in the minutes. Um, policy decision needed.  Technology makes the
collection of a Blue sheet equivalent very straightforward, at least for
many virtual interims, but is it a breach of privacy (in Germany, at
least) to put that on the web site?  On the other hand, we would regard
as deficient a string of posts to the mailing list with no indication
who was making them; which minutes without an identification of at least
who 'spoke' would seem to be equivalent.  I can guess who Alia is but
not Erik or Tom, which is all I have to go on (nvo3 2014/10/2 - netmod
and netconf minutes are very good in this regard unless, of course, you
regard the list of names as a breach of privacy!).  As I said, I think
IESG guidance is needed on what should appear in the minutes by way of
who took part or who contributed.

Tom Petch

---

From: Tim Chown <[email-masked]>
Date: Wed, 25 Feb 2015 13:45:32 +0000

Hi,

> On 15 Feb 2015, at 16:19, James Gannon <[email-masked]> wrote:
> 

So I don’t think there’s a one size fits all requirement here.

Who might want to participate remotely? The list may include:

a) people who have followed a mail list for a while and want to casually join a specific WG session at a meeting to get a feel for what a session is like
For this, streamed audio is probably fine, with some form of chatroom.
If a question is to be asked, it might be nice to be able to identify the person asking.

b) people who are very active in a WG, but can’t get to a given meeting, but needs high quality participation (along the lines Randy described earlier)
Here the ability to see slides, and perhaps to present, is likely to be needed. Reliable audio is more important.

c) perhaps a WG chair who can’t make a meeting, but would be willing timezone-permitting to chair remotely. 
Unless the solution for this is very good, we would want at least one chair present in person (because of the various administravia to deal with)
Needs are similar to (b), except a way to chat discretely with the co-chair would certainly be useful, and to see the room, and maybe to hear hums.

What other cases are there? How do we do a virtual Ted?

Then there’s the question of whether the participant is interested in just one WG, and thus minimal attendance, or attendance for multiple WGs in the week.

Perhaps charging is introduced for higher quality access (cases b, c), while casual ‘best effort’ remote participation is kept open and free (case a).

BTW a focus on the meeting room experience misses out on a big piece of value in attending - the corridor and bar room discussions.

Tim

---

From: Ted Lemon <[email-masked]>
Date: Wed, 25 Feb 2015 09:05:33 -0500

> Perhaps charging is introduced for higher quality access (cases b, c), while casual ‘best effort’ remote participation is kept open and free (case a).

If I were setting it up, I'd give everyone the same access whether they can afford to pay or not, and incentivize paying by listing people who pay differently in the proceedings.   If you're an amateur participant who isn't being paid to attend, you shouldn't have to pay, period.

If you are being paid to attend, then we should just say that your company is expected to pay the remote attendee fee.  If they don't, it should be a bit embarrassing for them (not for you!).   In practice I would expect that people who are being paid to attend would just pay the remote attendee fee, because they support the IETF, and there's no reason for them not to.

---

From: Eric Burger <[email-masked]>
Date: Wed, 25 Feb 2015 09:10:56 -0500

Mea culpa!

My fault for not being articulate. I was responding to the discussion thread, and happened to chose this one to staple the reply to. I in no way meant to infer that Ted (or Dave or anyone else) was advocating that the IETF should be excluding people. In this case, the impression (which Ted pointed out to me) was that I was saying that he wanted to exclude people based on ability to pay. Ted and I discussed this off-list, and we are in violent agreement. I did not think he was advocating for that, and I do not want anyone to advocate for that.

Moving forward, what I was hoping to avoid was for people to think that because the IETF conference fees defer the costs of operating the IETF (the meeting itself, the secretariat, and a portion of the RFC Editor), that we have to try to squeeze every penny from all sources. That is not necessarily a bad mindset to have: we should be striving to be independent on the largesse of the Internet Society and their contributors if we can manage it. That is not an infinite pot of cash, and no one wants to be beholden to a single funding source. However, what I wanted to get out to the community is the message that the Internet Society believes deeply in expanding access to the IETF and the IETF process. If charging for remote access inhibits participation (the unfortunate ‘paywall’ comment), then I would have no problem at all suggesting the IETF (IAOC in specific) ask the Internet Society to fund remote participation. I think the Board (speaking as an individual, NOT in my role as an Internet Society Trustee) would treat such a request sympathetically.

I can see this could be a dynamic situation. I can envision a time when we as the IETF are truly successful and develop fantastic real-time communication protocols that are easy to deploy, cost almost nothing, and are secure. At that point, one would *hope* in-person IETF meetings become a relic of history. Maybe we would meet once per year or every other year to reminisce about how the only way to get work done was to spend thousands of dollars of cash per year and an uncountable amount of cost for travel time to physically meet in the same location. How 20th Century! At that point, I would expect surpluses from meeting fees would be nonexistent, and we would need to figure alternate means of funding. However, that day seems to be far enough away that charging for remote participation should be a remote possibility for the foreseeable future.

> On Feb 24, 2015, at 9:24 PM, Ted Lemon <[email-masked]> wrote:
>

---

From: Ted Lemon <[email-masked]>
Date: Wed, 25 Feb 2015 09:17:16 -0500

Erik, I think you have aptly expressed what I suspect is the general consensus from the discussion.   I would love it if we could get to the end state that you've described.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 25 Feb 2015 08:10:28 -0800

On 2/25/2015 6:10 AM, Eric Burger wrote:
> Moving forward, what I was hoping to avoid was for people to think that because the IETF conference fees defer the costs of operating the IETF (the meeting itself, the secretariat, and a portion of the RFC Editor), that we have to try to squeeze every penny from all sources. That is not necessarily a bad mindset to have: we should be striving to be independent on the largesse of the Internet Society and their contributors if we can manage it. That is not an infinite pot of cash, and no one wants to be beholden to a single funding source. However, what I wanted to get out to the community is the message that the Internet Society believes deeply in expanding access to the IETF and the IETF process. If charging for remote access inhibits participation (the unfortunate ‘paywall’ comment), then I would have no problem at all suggesting the IETF (IAOC in specific) ask the Internet Society to fund remote participation. I think the Board (speaking as an individual, NOT in my role !
 as an In
ternet Society Trustee) would treat such a request sympathetically.

(To achieve a sufficiently broad base of funding that our loss of any
single source of revenue is tolerable would be particularly healthy.
However there really is nothing concrete in the IETF's current efforts
that will move us in that direction.  Some statements of desire, but
nothing that looks like an orchestrated plan or even a consistent
direction.)

Note that our current participation model is:

  Free:  Email list discussion.
         Remote attendance, with poor-to-mediocre access

  Pay:   IETF Meeting f2f Attendance

What's being discussed here is presumably far better remote attendance
capabilities.

As it approaches adequacy as a replacement for the basic uses of f2f
meeting attendance, it's not automatically clear that it should be
excluded from having a fee.  And I say that in spite of my continuing
concern that the IETF effectively restrict serious participation by
those with constrained budgets (and/or time.)

We already have a model with a free/pay range of participation.  If we
consider the model independent of technical details, we might find
choices that make sense but still have charging for at least some forms
of remote attendance or at least some remote attendees.

d/

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 25 Feb 2015 11:50:59 -0500

Is there really a demand for interim meetings that cannot be supported in
IETF regular meetings? We meet 3 times a year, why isn't that enough? I
don't know of any W3C working group meeting more than three times a year.

Rather than trying to pile requirements onto interim meetings, can we work
out a way to accommodate these needs in regular meetings?

I would like to suggest we go a stage further and change the way the IETF
meetings work.

People are having interim meetings because a one or two day meeting on one
topic is a lot more productive than an IETF WG session. For certain types
of work and at certain stages, a concentrated meeting is the only
productive approach.

W3C has always taken this approach, they have one annual meeting and a
series of interims. The annual meetings are structured like an academic
conference with a plenary track and a series of breakout tracks on one
topic.

Now we can't and should not change every IETF meeting structure because a
lot of IETF work is not of the type that benefits from this mode of work
and those that do do not necessarily benefit all the time.

But we could arrange one IETF on the W3C format as an experiment.

There are pros and cos to both ways of working. I don't see that we have to
limit ourselves just to one approach.

Early in a WG lifecycle I just want to spend two days doing nothing apart
from working through the issues list and kill off all the bike shed issues.
Later on when getting close to completion you might want to have another
bootcamp to close off all the open issues. In between you probably want
more of an update.

---

From: Benson Schliesser <[email-masked]>
Date: Wed, 25 Feb 2015 13:04:10 -0500

t.p. wrote:
> not Erik or Tom, which is all I have to go on (nvo32014/10/2  - netmod

There seems to be two different (perhaps related) issues here.

First, completeness of names in the minutes:

Yes, we should strive to get full names in the minutes rather than short 
names. This is tactically difficult for so many reasons, many of which 
have already been talked about on this list in the past. This includes 
non-uniform familiarity of the note-taker with all the contributors' 
names, speed of the conversation versus ability to write quickly, etc. I 
don't know of any sure-fire way to fix this. A recording of the meeting 
can be useful, but isn't a substitute for minutes and isn't always 
helpful in filling out missing details in the minutes after the fact. If 
anybody has suggestions for improving note-taking then I'm eager to hear 
them.

Otherwise, all I can say is that we'll try harder. Specifically, we can 
spend some effort reviewing the draft minutes and filling in names etc. 
For NVO3, I'll work with my co-chair and secretary to do this. I suppose 
other chairs should be encouraged to do the same.

Second, whether Blue Sheets are posted:

Currently the Blue Sheets for regular meetings are scanned and posted in 
the proceedings. For example see the "Blue Sheets" link at 
http://www.ietf.org/proceedings/89/nvo3.html. There is a mechanism in 
the Meeting Materials Manager tool for chairs and secretaries to upload 
Blue Sheets for interim meetings, too. Though I note that most of the 
interim proceedings seem to be missing them. One positive example I 
found was 
http://www.ietf.org/proceedings/interim/2015/02/09/idr/proceedings.html.

It is clear to me that chairs (myself included) need to do a better job 
collecting Blue Sheets for interims and posting them to the proceedings. 
This is already mandated by 
http://www.ietf.org/iesg/statement/interim-meetings.html along with 
minutes etc.

The subsequent question is "how" to collect names for Blue Sheets at 
virtual interim meetings. NVO3 has experimented with using an etherpad 
for this purpose. (Thanks to Sue Hares for suggesting this to me.) One 
could also imagine using a Google Docs form, Eventbrite, or whatever 
other tool was convenient. It might be nice if we could create an IETF 
tool for such a purpose, but we can certainly manage otherwise.

During today's Routing Chairs meeting there was a discussion about 
whether it is mandatory to fill out Blue Sheets, and the general 
consensus seemed to be "no, not really" given that even the regular 
in-person meetings have imperfect Blue Sheet records. So making it a 
gating step, e.g. in order to register to receive meeting logistics 
information etc, would be inappropriate. Likewise I suspect that it's 
more consistent to make it a voluntary step rather than simply 
collecting the names of people e.g. on a WebEx session. But I'll take 
this up with the IAOC to make sure that my current thinking is 
consistent with the legal needs of the IETF.

Cheers,
-Benson

---

From: Nico Williams <[email-masked]>
Date: Wed, 25 Feb 2015 12:24:08 -0600

On Wed, Feb 25, 2015 at 09:05:33AM -0500, Ted Lemon wrote:
> > Perhaps charging is introduced for higher quality access (cases b,
> > c), while casual ‘best effort’ remote participation is kept open and
> > free (case a).
> 
> If I were setting it up, I'd give everyone the same access whether
> they can afford to pay or not, and incentivize paying by listing
> people who pay differently in the proceedings.   If you're an amateur
> participant who isn't being paid to attend, you shouldn't have to pay,
> period.
> 
> If you are being paid to attend, then we should just say that your
> company is expected to pay the remote attendee fee.  If they don't, it
> should be a bit embarrassing for them (not for you!).   In practice I
> would expect that people who are being paid to attend would just pay
> the remote attendee fee, because they support the IETF, and there's no
> reason for them not to.

Yes, the honor system will do.

Improvements that could be made for remote participation:

 - Virtual mic queuing that's better than the current "get someone in
   the room *and* the jabber room to represent you".

   I'm not sure how important this is.  In practice the current solution
   works well enough for the WGs I participate in.  But it does rely on
   volunteers, and it does put them out somewhat.  It's also not
   guaranteed to work every time, in every WG.  We've been limping along
   without a solution here, so it's not urgent, but it's worth thinking
   about.

   One thing that has worked well is to have two projected displays: one
   for slideware, and one for the jabber room, which the chairs and/or
   others in the room can use to help manage access to the mic.  Perhaps
   two jabber rooms would help (one for chatting, one for requesting mic
   access).  Perhaps a bot would help (maybe even with text-to-speech
   functionality, though a lot of jargon and acronyms would probably
   stump TTS).

 - Follow-along slide viewing.

   Again, we've been limping along with this one, with the aid of a
   volunteer indicating slide changes on the jabber room.  This works
   well enough and with low enough a burden on volunteers that a
   solution may not be needed.

 - Lower-latency audio.

   One of the first things I do when I join a WG remotely is: attempt a
   guestimate of audio latency using the jabber room.  If latency is
   much more than a second then the jabber room can get very
   distracting, but the jabber room is often a great aid to following
   along.

   For webex-style remote participation, where we get to have our voices
   carried into the room, latency is utterly critical.

   For me this is the top priority.  Low-latency audio feeds or bust.

   I will happily pay a very large fraction of the full participation
   fee if the IETF can deliver this one feature, honor system or not.

 - Remote MIC, a la webex.

   I'd like more of this.

I can think of other features that might help, particularly help in
participating in hallway conversations (which surely are out of scope),
but the above is a start.  For example: always leave the room MICs on
(with a warning sign?) so that people can have post-meeting
conversations that should be heard by remote participants (this is
already the case, usually, and it is risky, from a privacy point of
view).  Participation in hallway conversations (and bar BoFs) are a key
justification for on-site attendance, and it should continue being so.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 25 Feb 2015 12:29:41 -0600

On Wed, Feb 25, 2015 at 12:24:08PM -0600, Nico Williams wrote:
>         Participation in hallway conversations (and bar BoFs) are a key

I should add that being on site also forces one to make time for the
meeting, and that's another major justification for on-site attendance.

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 Feb 2015 15:15:29 -0500

--On Wednesday, February 25, 2015 09:05 -0500 Ted Lemon
<[email-masked]> wrote:

>> Perhaps charging is introduced for higher quality access
>> (cases b, c), while casual 'best effort' remote
>> participation is kept open and free (case a).
> 
> If I were setting it up, I'd give everyone the same access
> whether they can afford to pay or not, and incentivize paying
> by listing people who pay differently in the proceedings.   If
> you're an amateur participant who isn't being paid to attend,
> you shouldn't have to pay, period.

Ted,

Just for calibration, I don't know what your definition of
"amateur" is but, if, as the above suggests, it is the opposite
of "someone whose job description includes IETF attendance or
participation" and/or "someone who is paid a salary while
attending IETF meetings" and/or "someone whose consulting or
equivalent work includes the expectation of IETF participation,
perhaps on behalf of the client", then I have been an "amateur
participant" for well over a dozen years now, including when I
was on the IAB from 2009-2011 and when I was IAB Chair in the
first part of 2002.

That makes, IMO, a rather strange definition of "amateur"
(perhaps a different term is needed), but I look forward to not
paying registration fees in the future whether I attend remotely
or in person.  If anyone would like to offer to refund the
registration fees I've paid over the last decade (to keep things
in round numbers), the odds of my showing up in Dallas would
increase significantly.  :-)

    john

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 26 Feb 2015 14:24:49 +1300

On 26/02/2015 07:04, Benson Schliesser wrote:

...
> During today's Routing Chairs meeting there was a discussion about whether it is mandatory to fill out Blue Sheets, and the
> general consensus seemed to be "no, not really" given that even the regular in-person meetings have imperfect Blue Sheet
> records. 

According to BCP 25 (RFC 2418) minutes are mandatory ("shall") and the
list of attendees is a lower-case "should":

   All working group sessions (including those held outside of the IETF
   meetings) shall be reported by making minutes available.  These
   minutes should include the agenda for the session, an account of the
   discussion including any decisions made, and a list of attendees.

Personally I think it should have been a MUST because of verifying
IPR disclosure obligations.

    Brian

---

From: Benson Schliesser <[email-masked]>
Date: Wed, 25 Feb 2015 23:53:01 -0500

> IPR disclosure obligations.

It's worth noting that the IESG guidance at 
http://www.ietf.org/iesg/statement/interim-meetings.html says the 
attendee list "must" be submitted. And I'm happy to follow that guidance.

Of course, chairs being required to submit blue sheets is not the same 
as attendees being required to sign them.

On the other hand, there doesn't seem to be any reason why a chair would 
be prohibited from collecting names for the blue sheets in whatever way 
is most convenient. (E.g. WebEx logs for virtual interim meetings) If 
that conflicts with an attendee's desire to avoid signing the blue 
sheets, then I suppose the chair's prerogative takes precedence.

Cheers,
-Benson

---

From: Abdussalam Baryun <[email-masked]>
Date: Wed, 25 Feb 2015 21:59:02 -0800

Usually, ietf-draft authors that pay in ietf-meetings will get the
best outcome from their efforts and get more interests from the world.
Remote participants need to be encouraged/increased so making it free
is an excellent ietf-strategy currently. IMHO, the utilization&fee
issue is not in the side of remote participants but is in the side of
the f2f participants, because it is still not totally-managed by
session/meeting chairs within IETF meetings (which may be a very good
approach for flexibility). IETF meetings are productive and efficient
but could we make more effort to increase that? we may think to add
fees on draft-authors that need more than 10 minutes to present
their-work/wg-draft?

Is it possible to chair a session and you get 20 participants lined up
while that draft presentation was scheduled for 15 minutes. Or  may I
"line up" for a long time with no much expectation of how long it will
take while the queue has 3 only. Could we have in IETF a determination
of best practice maximum input time per f2f participant per draft.

 I suggest that IETF management guide session chairs to announce
maximum input duration time per draft per meeting-participant, so we
can get the highest efficiency per session/WG-meeting.

AB

On 2/15/15, Christer Holmberg <[email-masked]> wrote:
> Hi,
>
> It can't be that difficult to make a tool that everyone (including those
> participating f2f) use to "line up" behind the microphone.
>
> A tremendous amount of work goes on behind the scenes to make this work, not
> just the meetecho folks.   I remember reporting a problem in a meeting and
> having Alexa show up five minutes later checking to see if it was fixed, and
> I know that other folks from AMS and from the NOC work hard on this.
>
>

---

From: Yoav Nir <[email-masked]>
Date: Thu, 26 Feb 2015 11:17:04 +0200

> On Feb 26, 2015, at 6:53 AM, Benson Schliesser <[email-masked]> wrote:
> 
> Brian E Carpenter wrote:
> It's worth noting that the IESG guidance at http://www.ietf.org/iesg/statement/interim-meetings.html says the attendee list "must" be submitted. And I'm happy to follow that guidance.

I had always been told that blue sheets are required for legal reasons so that attendees cannot claim to not having been present where potentially encumbered technology was discussed.

If this is a legal requirements, why would it not apply to interim meetings? 

> Of course, chairs being required to submit blue sheets is not the same as attendees being required to sign them.

They’re required to sign them in physical meetings. How are virtual interims different?

> On the other hand, there doesn't seem to be any reason why a chair would be prohibited from collecting names for the blue sheets in whatever way is most convenient. (E.g. WebEx logs for virtual interim meetings) If that conflicts with an attendee's desire to avoid signing the blue sheets, then I suppose the chair's prerogative takes precedence.

Depending on the tool, the logs may contain “screen names” that are difficult or impossible to relate to real people. If we mean to require this, we should do this in a more elaborate way. I don’t mean that we need to add strong authentication, but something like a roll call where everyone says who they are would be good.

Yoav

---

From: t.p. <[email-masked]>
Date: Thu, 26 Feb 2015 10:07:39 +0000

---- Original Message -----
From: "Benson Schliesser" <[email-masked]>
To: "Brian E Carpenter" <[email-masked]>
Sent: Thursday, February 26, 2015 4:53 AM

> sheets, then I suppose the chair's prerogative takes precedence.

I see two related requirements.  One is 'Blue Sheets', the desire to
have a complete record of participants, perhaps for IPR reasons, but
that was not the  one that triggered my initial post.

Rather, what drives me is the increased difficulty of knowing what is
going on if all one has to go on is the WG e-mail list (changing the way
we work).  Contributions from unidentified or unidentifiable people at a
virtual interim is a part of this (perhaps giving a sense that the WG is
becoming more of a closed shop and if you do not participate in these
many virtual interims, then you can no longer be part of things).

So whether or not there is a public 'Blue Sheet' for an interim,
recording all those who 'dialled in', whether or not they remained
silent thereafter, I think there should be an identifier associated with
contributions as recorded in the minutes.  If you are there, you will
likely know who 'Tom' is (Nadeau, Ritter, Thorogood, Taylor, Sanders,
...).  After all, at physical meetings the chairs frequently ask
speakers at the mike to identify themselves and this is then recorded in
the minutes.

I believe we need the same, probably more so, for virtual interims.  As
I mentioned before, netmod and netconf WGs manage it, I would like
others to do the same.

Tom Petch

---

From: Benoit Claise <[email-masked]>
Date: Thu, 26 Feb 2015 12:01:15 +0100

Philip,
> Is there really a demand for interim meetings that cannot be supported 
> in IETF regular meetings? We meet 3 times a year, why isn't that 
> enough? I don't know of any W3C working group meeting more than three 
> times a year.
>
> Rather than trying to pile requirements onto interim meetings, can we 
> work out a way to accommodate these needs in regular meetings?
>
>
> I would like to suggest we go a stage further and change the way the 
> IETF meetings work.
>
> People are having interim meetings because a one or two day meeting on 
> one topic is a lot more productive than an IETF WG session.
Or because weekly meetings are more efficient.

Regards, Benoit
> For certain types of work and at certain stages, a concentrated 
> meeting is the only productive approach.
>
> series of interims. The annual meetings are structured like an 
> academic conference with a plenary track and a series of breakout 
> tracks on one topic.
>
>
> Now we can't and should not change every IETF meeting structure 
> because a lot of IETF work is not of the type that benefits from this 
> mode of work and those that do do not necessarily benefit all the time.
>
> There are pros and cos to both ways of working. I don't see that we 
> have to limit ourselves just to one approach.
>
>
> Early in a WG lifecycle I just want to spend two days doing nothing 
> apart from working through the issues list and kill off all the bike 
> shed issues. Later on when getting close to completion you might want 
> to have another bootcamp to close off all the open issues. In between 
> you probably want more of an update.

---

From: Lou Berger <[email-masked]>
Date: Thu, 26 Feb 2015 06:57:59 -0500

Tom,
    I think you got it exactly right.  See below.

> that was not the  one that triggered my initial post.

Certainly all are asked to sign the blue sheet, but if someone chooses
not to, it's never been policy to "out them", i.e., record their
presence without their consent.  I think this is no less true for
interims than for physical meetings.  Also, just like for Jabber, I do
think the name one uses when they enter a webex should be fair game (and
noted).

> speakers at the mike to identify themselves and this is then recorded in

This really is the important point.  For transparency and probably IPR
traceability, but I'm not a lawyer, the minutes need to reflect full
names of anyone contributing/speaking in the session. 

I may be mistaken, but don't recall seeing this stated as a requirement
in any RFC -- it probably should be.

> I mentioned before, netmod and netconf WGs manage it, I would like

100% agreed.

Lou

---

From: Lou Berger <[email-masked]>
Date: Thu, 26 Feb 2015 07:04:53 -0500

Benoit,

On 2/26/2015 6:01 AM, Benoit Claise wrote:
> Philip,
>> Is there really a demand for interim meetings that cannot be
>> supported in IETF regular meetings? We meet 3 times a year, why isn't
>> that enough? I don't know of any W3C working group meeting more than
>> three times a year.
>>
>> People are having interim meetings because a one or two day meeting
>> on one topic is a lot more productive than an IETF WG session.
But are these really author/design team meetings or are they really
interims?

A meeting amongst authors doesn't need to be an interim, it's more like
a design team -- in both cases contribution of the individuals is
reflected in a draft. 

A meeting that is taking the place of discussion on a list, with
non-authors contributing, certainly does qualify as an interim.

Lou

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Feb 2015 07:09:00 -0500

On Feb 26, 2015, at 6:01 AM, Benoit Claise <[email-masked]> wrote:
>> People are having interim meetings because a one or two day meeting on one topic is a lot more productive than an IETF WG session.

Well, they certainly get more done.   I think 6tisch does bi-weekly meetings because it allows them to keep momentum and make continuous forward progress between IETFs.   I think this is a really good thing, although I agree with the observation that such meetings make things harder for people who are not able to be full-time contributors.

I think there is a real tension between a high clock rate enabling a good rate of progress and a too-high clock rate excluding participants.   I don't think there's an easy answer to this: I think that participants who are less able to attend than they would like need to have conversations with their working group chairs and/or ADs to see if the clock rate can be slowed somewhat without damaging the effectiveness of the working group.   I don't think there will ever be one answer to this that applies to all working groups.

---

From: Benoit Claise <[email-masked]>
Date: Thu, 26 Feb 2015 14:08:03 +0100

On 26/02/2015 13:09, Ted Lemon wrote:
> I think there is a real tension between a high clock rate enabling a good rate of progress and a too-high clock rate excluding participants.
I think it depends on well the status is documented (to come back to 
Tom's initial point).
For NETMOD, https://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/, see in 
particular the latest status on all open points: 
https://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/issues.html. Kudos to 
Jürgen Schönwälder for maintaining this.
That being said, (bi-)weekly meetings are more suitable with a well 
documented issue list.

Regards, Benoit

>    I don't think there's an easy answer to this: I think that participants who are less able to attend than they would like need to have conversations with their working group chairs and/or ADs to see if the clock rate can be slowed somewhat without damaging the effectiveness of the working group.   I don't think there will ever be one answer to this that applies to all working groups.
>
>

---

From: Benoit Claise <[email-masked]>
Date: Thu, 26 Feb 2015 14:12:23 +0100

> non-authors contributing, certainly does qualify as an interim.
NETMOD meetings are really interim meetings.
However, point taken, we don't always need official interim meetings 
when authors or design team speak among themselves.

Regards, Benoit

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Feb 2015 09:37:40 -0500

--On Thursday, February 26, 2015 07:09 -0500 Ted Lemon
<[email-masked]> wrote:

>..
> Well, they certainly get more done.   I think 6tisch does
> bi-weekly meetings because it allows them to keep momentum and
> make continuous forward progress between IETFs.   I think this
> is a really good thing, although I agree with the observation
> that such meetings make things harder for people who are not
> able to be full-time contributors.

Such meetings also make things harder for those whose time zones
make the meetings difficult or incompatible with day job
schedules.  Of course, that might not have significant effect
anyone who is really a full-time IETF contributor (rather than
having other day job responsibilities) and who doesn't have a
personal life, but I suspect those people are fairly few.

That is also at least one big argument for doing things on
mailing lists rather than frequent interim meetings.  No matter
how much time they take up, mailing lists are intrinsically
asynchronous relative to time zones and non-IETF work and
personal schedules.  

> I think there is a real tension between a high clock rate
> enabling a good rate of progress and a too-high clock rate
> excluding participants.   I don't think there's an easy answer
> to this: 
>...

I agree, but I think it goes beyond that.  The more we shift
from doing almost all of our work on mailing lists to doing a
significant proportion of it in high-frequency interim meetings,
the more we tend to narrow effective participation to
vendor-supported people with dedicated time in convenient (for
the WG majority) time zones and reduce some of the diversity we
have claimed is important.

Reducing diversity in that way implies another risk: many
private-sector SDOs have been advised by counsel (or threatened/
forced by regulators) to establish measurable categories of
participants and then to not hold meetings or other discussions
unless minimum numbers or percentages of people from each
category were present.  I really hope we never need to go there,
especially if we are evolving in directions that reduce
diversity of interests and backgrounds.

     john

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Feb 2015 09:53:38 -0500

On Feb 26, 2015, at 9:37 AM, John C Klensin <[email-masked]> wrote:
> The more we shift

Actually, my experience is the opposite: mailing lists are incredibly time consuming, because there are a few participants who feel the need to repeat themselves over and over again in any given discussion, and people aren't concise in their responses, nor considerate of the burden their responses will impose on readers, so there is a lot of reading, much of which is completely redundant.   Being restricted to the low shared bandwidth of voice in an online meeting substantially mitigates that, if the working group chair is doing a good job of disciplining the discussion.

There is a reason why we do f2f meetings, and it's not because we like the cookies (although we do).

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Feb 2015 10:05:40 -0500

I need to agree with John here.  There are several WGs I try to monitor 
that started having frequent interim conference calls.  There is no way 
I can reliably make time for that.  The advantage of email is that I can 
fit it in around the work I need to do (including reading it during 
corporate conference calls.)  In one case I have had to dramatically 
reduce my effective participation in the WG because most of the work 
moved to the conference calls.

One of the other standards bodies I have had to work with did all of its 
work in weekly conference calls.  This made it next to impossible for me 
to contribute to most of the topics, as I could not make most of the calls.

Yours,
Joel

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Feb 2015 10:15:46 -0500

On Feb 26, 2015, at 10:05 AM, Joel M. Halpern <[email-masked]> wrote:
> I need to agree with John here.  There are several WGs I try to monitor that started having frequent interim conference calls.  There is no way I can reliably make time for that.  The advantage of email is that I can fit it in around the work I need to do (including reading it during corporate conference calls.)  In one case I have had to dramatically reduce my effective participation in the WG because most of the work moved to the conference calls.

If you "try to monitor" these working groups, it sounds like you aren't an active participant.   The meetings are supposed to be minuted, so you ought to be able to monitor them by reading the minutes.

Do you think we should optimize working groups for getting work done, or for being monitored?   Or have I misunderstood what you mean when you say "try to monitor"?

---

From: Alia Atlas <[email-masked]>
Date: Thu, 26 Feb 2015 10:19:52 -0500

Interim conference calls shouldn't take the place of conversations on the
mailing list, but that is obvious.
I agree about the issues around time-zones and scheduling.   However, high
bandwidth conversations
frequently happen better in voice as does presenting and clarifying
questions or concerns.

On the one hand, it would be good in many cases if the IETF standardization
process could go faster.
On the other, that does leave some people needing to contribute only
sometimes.

I would strongly urge pushing conversations on the list more.  One of the
advantages of the interim
conference calls is that those who have attended tend to have the current
state swapped in and be
able to reply more rapidly - at least in theory.

Regards,
Alia

On Thu, Feb 26, 2015 at 10:05 AM, Joel M. Halpern <[email-masked]>
wrote:

> that started having frequent interim conference calls.  There is no way I
> can reliably make time for that.  The advantage of email is that I can fit
> it in around the work I need to do (including reading it during corporate
> conference calls.)  In one case I have had to dramatically reduce my
> effective participation in the WG because most of the work moved to the
> conference calls.
>
> work in weekly conference calls.  This made it next to impossible for me to
> contribute to most of the topics, as I could not make most of the calls.
>
> Yours,
> Joel
>
>
>> Actually, my experience is the opposite: mailing lists are incredibly
>> time consuming, because there are a few participants who feel the need to
>> repeat themselves over and over again in any given discussion, and people
>> aren't concise in their responses, nor considerate of the burden their
>> responses will impose on readers, so there is a lot of reading, much of
>> which is completely redundant.   Being restricted to the low shared
>> bandwidth of voice in an online meeting substantially mitigates that, if
>> the working group chair is doing a good job of disciplining the discussion.
>>

---

From: Joel Halpern Direct <[email-masked]>
Date: Thu, 26 Feb 2015 10:57:30 -0500

One of the working groups where I have observed this is one where I am a 
document author and was an active contributor.  I am still trying to 
contribute.  Minutes don't cut it.  (I looked at the minutes from the 
one session I participated in.  While they were formally correct, I 
doubt that they would have helped anyone not on the call actually engage 
in the discussion.  As evidence I point out that the discussions do not 
get followed up on the list.

One of the other groups I follow, and no I don't expect the work to be 
optimized for a follower.  But judging from what I see, even an active 
participant and author would have trouble if they could not make the 
phone calls.

We claim that we do our work on the email list.  I do understand that 
phone calls and face to face meetings are useful for resolving hard 
issues.  I am not saying "don't have interims".  But if one is having a 
phone call every two to three weeks, then the working group is NOT 
conducting its work on the mailing list.  If we want to throw in the 
towel and say that you need a higher engagement level to participate, 
then we should own up to that.  It will severely harm 
cross-fertilization and participation in multiple working groups.  But 
maybe that is what we need to give up.

But pretending that frequent working group (not design team, working 
group) conference calls are a good way to work and consistent with our 
ethos does not match what I have seen.

Yours,
Joel

On 2/26/15 10:15 AM, Ted Lemon wrote:
>> I need to agree with John here.  There are several WGs I try to
>> monitor that started having frequent interim conference calls.
>> There is no way I can reliably make time for that.  The advantage
>> of email is that I can fit it in around the work I need to do
>> (including reading it during corporate conference calls.)  In one
>> case I have had to dramatically reduce my effective participation
>> in the WG because most of the work moved to the conference calls.
>
> If you "try to monitor" these working groups, it sounds like you
> aren't an active participant.   The meetings are supposed to be
> minuted, so you ought to be able to monitor them by reading the
> minutes.
>
> Do you think we should optimize working groups for getting work done,
> or for being monitored?   Or have I misunderstood what you mean when
> you say "try to monitor"?
>

---

From: Mary Barnes <[email-masked]>
Date: Thu, 26 Feb 2015 10:13:18 -0600

On Thu, Feb 26, 2015 at 9:57 AM, Joel Halpern Direct <
[email-masked]> wrote:

> contribute.  Minutes don't cut it.  (I looked at the minutes from the one
> session I participated in.  While they were formally correct, I doubt that
> they would have helped anyone not on the call actually engage in the
> discussion.  As evidence I point out that the discussions do not get
> followed up on the list.
>
[MB] Aren't these all management issues?  I would think the WG chairs would
ensure that all the key contributors are available. And, of course, there
should be adequate minutes produced along with action items identified and
I would assume those are taken to the mailing list and/or added to an issue
tracker.  As others have noted email is not the best way to resolve some of
the more complex problems introduced in our technical work.  I totally
agree about the timezone issue.  For CLUE WG, folks were flexible about
shifting our meetings to ensure an attendee in Australia could attend when
we were discussing issues to which he had input.  Note, that we identified
ahead of time on our WG wiki what the topic for that meeting was.  We were
flexible about re-arranging those for the key contributors.  [/MB]

> participant and author would have trouble if they could not make the phone
> calls.
>
> conducting its work on the mailing list.  If we want to throw in the towel
> and say that you need a higher engagement level to participate, then we
> should own up to that.  It will severely harm cross-fertilization and
> participation in multiple working groups.  But maybe that is what we need
> to give up.
>
[MB] In CLUE WG, we had weekly calls (if we had a topic identified that we
felt benefitted from a verbal discussion.  Again, I think it's a management
issue if things are not documented and what is deemed to be consensus is
not taken to the WG mailing list for confirmation and any additional
discussion as necessary.  I still consider the work having been conducted
on the mailing list in that we posted links (or directly the minutes) to
the WG mailing list and when we added issues to the tracker, the WG gets
notified. [/MB]

> ethos does not match what I have seen.
[MB] I, of course, totally disagree. We should use all the communication
tools available to progress our work.  I totally agree of the importance of
traceability in the email archives, which is why links to minutes, issues
in the tracker, etc. ought all be posted to the WG mailing list.  IMHO, we
would actually benefit from WGs actually using the wikis to more carefully
document decisions - it's a heck of lot easier in some cases than trying to
dig through WG or personal email archives.  [/MB]

---

From: Dave Crocker <[email-masked]>
Date: Thu, 26 Feb 2015 08:15:49 -0800

> On 2/26/2015 6:53 AM, Ted Lemon wrote:
> Actually, my experience is  the opposite: mailing lists are
> incredibly time consuming, because there are a few participants who
> feel the need to repeat themselves over and over again in any given
> discussion, and people aren't concise in their responses, nor
> considerate of the burden their responses will impose on readers, so
> there is a lot of reading, much of which is completely redundant.

On 2/26/2015 7:15 AM, Ted Lemon wrote:
> or for being monitored?

Folks,

The above two comments capture essential views shared by many, namely
that mailing lists are too inefficient and noisy for doing serious work
and that less-active participants aren't useful or that they are even
generally problematic.

Certainly the gist of both comments have been uttered many times over
the years, by many people.

And indeed there is an understandable basis for both points, and of
course, some legitimacy.

Mailing lists are unparalleled in their ability to support open
participation:

   *  Anyone with an email account can participate, and at their own
      schedule.

   *  The asynchrony also can permit thoughtful consideration of ideas
      that real-time simply cannot.

   *  Email is good for posting proposals, because of the thoughtful,
      focused review it permits.

But they also suck for /discussion/ of deep issues. The inherently slow
(and even cumbersome) interactions get in the way of constructively
conducting protracted and/or complicated discussion. Real-time
voice/video (f2f or teleconferenced) is vastly better for anything that
looks like a protracted 'negotiation' or exploration of a difficult
topic -- that is, for problem-solving.[1]

And there often are mailing list participants who are especially noisy.
In general, they are more subject to moderation during a real-time session.

However there are two essential flaws with the resulting bias towards
real-time -- and especially f2f.

   *  Real-time -- and especially f2f -- is exclusionary.  It
      significantly restricts who can participate.

   *  Less-active participants occasionally offer insight and
      suggestions that the more-active participants have missed. They
      also provide a wider base of support for post-approval deployment
      and use.

The formal means of resolving these tradeoffs is /supposed/ to be that
we always MUST rely on the mailing list for documenting proposed choices
and documenting rough consensus.  This permits /everyone/ to comment.

   *  The formal IETF model says that real-time is essentially an
      efficiency hack for getting foundational convergence on
      understanding and choice, BUT NOT FOR FORMAL DECISION-MAKING.

Design teams were an explicit construct introduced in the IETF about 20
years ago, to note the benefits of small-group efficiencies that can
complement the full-group formalities.[2]

Unfortunately, IETF culture has shifted towards dismissing folk who are
less active, increasing the bias towards more narrow and/or complex
design thinking. And decreasing the breadth of community understanding
and support.

The enthusiasm of an active core is essential.  But so it the
contribution and support of a much broader community.

This is the essential distinguishing factor, between the original
development culture in the IETF, versus other, classic standards bodies.

d/

[1] http://www.ietf.org/wg/agenda-minutes-procedures.html
    and
    Chapanis, A.,"Interactive Human Communication," Scientific American,
vol. 232, pp. 36-49, March 1975.

    The Chapanis study observed that a real-time voice channel was the
most important for problem-solving.

[2] I think that RFC 1603 was the first documentation of IETF design
teams, capturing a practice of self-organizing core teams, that had
emerged in the IETF; as I recall that bit of the document produced some
controversy, specifically because they DTs are exclusionary.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Feb 2015 11:27:36 -0500

If the calls are produce action items and proposed resolutions that are 
taken to the working group email list in a clear fashion, then it is 
probably working well enough.

I am not seeing that from most of these groups.

If there are a lot of open issues (which would seem to go with having 
value in multiple calls) then a wiki or other easily used tracker to 
keep current information accessible is understandable.  I personally 
find that using those trackers as a place to have the conversations,as 
some WG chairs want to do, is very difficult.  But I can't say it is 
against the rules.

If there is going to be noticable time lag between issues being raised 
and their being addressed in the document, using the document tracker to 
record all the issues and their resolutions is also very helpful.

And I expect that design teams have more frequent calls than working 
groups.  That is why you have a design team.

So I am not saying we should never use conference calls.  And I am not 
saying we should not use good tooling.
But I am seeing more and more working groups having biweekly conference 
calls.  It is quite understandable that the discussion may continue from 
call to call.  But that means that the WG is NOT seeing the discussion 
on the list.  And that the resolution, when it comes to the list, will 
be missing a LOT of context.

Yours,
Joel

On 2/26/15 11:13 AM, Mary Barnes wrote:
> On Thu, Feb 26, 2015 at 9:57 AM, Joel Halpern Direct
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     One of the working groups where I have observed this is one where I
>     am a document author and was an active contributor.  I am still
>     trying to contribute.  Minutes don't cut it.  (I looked at the
>     minutes from the one session I participated in.  While they were
>     formally correct, I doubt that they would have helped anyone not on
>     the call actually engage in the discussion.  As evidence I point out
>     that the discussions do not get followed up on the list.
>
> [MB] Aren't these all management issues?  I would think the WG chairs
> would ensure that all the key contributors are available. And, of
> course, there should be adequate minutes produced along with action
> items identified and I would assume those are taken to the mailing list
> and/or added to an issue tracker.  As others have noted email is not the
> best way to resolve some of the more complex problems introduced in our
> technical work.  I totally agree about the timezone issue.  For CLUE WG,
> folks were flexible about shifting our meetings to ensure an attendee in
> Australia could attend when we were discussing issues to which he had
> input.  Note, that we identified ahead of time on our WG wiki what the
> topic for that meeting was.  We were flexible about re-arranging those
> for the key contributors.  [/MB]
>
>
>     One of the other groups I follow, and no I don't expect the work to
>     be optimized for a follower.  But judging from what I see, even an
>     active participant and author would have trouble if they could not
>     make the phone calls.
>
>     We claim that we do our work on the email list.  I do understand
>     that phone calls and face to face meetings are useful for resolving
>     hard issues.  I am not saying "don't have interims".  But if one is
>     having a phone call every two to three weeks, then the working group
>     is NOT conducting its work on the mailing list.  If we want to throw
>     in the towel and say that you need a higher engagement level to
>     participate, then we should own up to that.  It will severely harm
>     cross-fertilization and participation in multiple working groups.
>     But maybe that is what we need to give up.
>
> [MB] In CLUE WG, we had weekly calls (if we had a topic identified that
> we felt benefitted from a verbal discussion.  Again, I think it's a
> management issue if things are not documented and what is deemed to be
> consensus is not taken to the WG mailing list for confirmation and any
> additional discussion as necessary.  I still consider the work having
> been conducted on the mailing list in that we posted links (or directly
> the minutes) to the WG mailing list and when we added issues to the
> tracker, the WG gets notified. [/MB]
>
>
>     group) conference calls are a good way to work and consistent with
>     our ethos does not match what I have seen.
>
> tools available to progress our work.  I totally agree of the importance
> of traceability in the email archives, which is why links to minutes,
> issues in the tracker, etc. ought all be posted to the WG mailing list.
> IMHO, we would actually benefit from WGs actually using the wikis to
> more carefully document decisions - it's a heck of lot easier in some
> cases than trying to dig through WG or personal email archives.  [/MB]
>
>     Yours,
>     Joel
>
>
>         On Feb 26, 2015, at 10:05 AM, Joel M. Halpern
>         <[email-masked] <mailto:[email-masked]>>
>         wrote:
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Feb 2015 11:52:14 -0500

On Feb 26, 2015, at 10:57 AM, Joel Halpern Direct <[email-masked]> wrote:
> One of the working groups where I have observed this is one where I am a document author and was an active contributor. I am still trying to contribute.  Minutes don't cut it. 

It seems to me that if an active participant can't participate, that's a problem that the chairs need to address.   If the chairs aren't responsive, you should appeal to your AD, and if that fails to the IESG as a whole.

> But pretending that frequent working group (not design team, working group) conference calls are a good way to work and consistent with our ethos does not match what I have seen.

Hm, I suppose that if "our" ethos is that all work occurs on mailing lists, then you are right, but I have two objections to this: first, it's not necessary correct that every working group should follow the same practices in this regard.  Second, we in fact do do work during f2f meetings, and the rule is "confirm on the mailing list."   If working groups are not following that same practice during interim meetings, they are doing it wrong.

As a participant, it's your obligation to be part of the feedback loop that corrects this, and not just accept it as a fait accomplis.   Despite what may appear to be the case, ADs are not omniscient, nor are working group chairs.   Well, I guess actually that's not the appearance either... :)

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 26 Feb 2015 12:43:05 -0500

On Thu, Feb 26, 2015 at 6:01 AM, Benoit Claise <[email-masked]> wrote:

> in IETF regular meetings? We meet 3 times a year, why isn't that enough? I

I consider weekly con-calls a different class. But again, they are often
useful for certain types of WG.

---

From: t.p. <[email-masked]>
Date: Thu, 26 Feb 2015 18:32:58 +0000

----- Original Message -----
From: "Mary Barnes" <[email-masked]>
To: "Joel Halpern Direct" <[email-masked]>
Sent: Thursday, February 26, 2015 4:13 PM
> > One of the working groups where I have observed this is one where I
am a
> [MB] Aren't these all management issues?  I would think the WG chairs
would
> tracker.

Mary

I have flagged netmod as both the WG that has produced the most
comprehensive documentation and the one which held the most Interims in
2014.  It is also a WG I have tracked since it was formed (but no longer
can).

The most recent Virtual Interim minutes,
http://www.ietf.org/mail-archive/web/netmod/current/msg11939.html
(which do give full names of the participants)  include

"  MB: I believe C1 summarizes how things work today.
  AB: Are we trying to come up with tighter rules for humans or for
      tools to take advantage of the tighter rules?
  AB: I am not sure YANG 1.0 specifies C1 explicitly somewhere.
  JS: Does A3 not follow from A2?
  KW: A3 is more a corollary of A2.
  AB: The high-level problem is how to create and maintain the
      information needed to achieve A4. "

and I have no idea what A1 to C1 mean; while the mailing list posts
http://www.ietf.org/mail-archive/web/netmod/current/maillist.html
have subjects such as

Y25 new solution Y25-02
Y59 action
VRFY :Y16: module advertisement optimization
DEAD :Y54: remove the advertisement of conformance information ..

There is a 60 strong list of issues identified by Yxx (not A1 to C1),
often with multiple solutions, and a finite state machine (DEAD, EDIT,
VRFY ...) with seven states.

The WG is progressing and will produce, but whether anyone outside the
core of about six participants, or even they in a year's time (the WG
tends to revisit the same issues every year or two), can tell you how a
decision was arrived at by whom, I doubt.  The mailing list is archived
but what else is?

as Joel said earlier

> >                                             But judging from what I
see, even an active
> > participant and author would have trouble if they could not make the
phone
> > calls.

This is, for better or worse, the changing way of working I have in mind

Tom Petch

>         As others have noted email is not the best way to resolve some
of
> > One of the other groups I follow, and no I don't expect the work to
be
> > participant and author would have trouble if they could not make the
phone
> > We claim that we do our work on the email list.  I do understand
that
> > conducting its work on the mailing list.  If we want to throw in the
towel
> > group) conference calls are a good way to work and consistent with
our

---

From: Benson Schliesser <[email-masked]>
Date: Thu, 26 Feb 2015 14:21:13 -0500

Ted Lemon wrote:
> we in fact do do work during f2f meetings, and the rule is "confirm
> on the mailing list."   If working groups are not following that same
> practice during interim meetings, they are doing it wrong.

I agree with this. In particular, I think the best mindset is that a 
WG's mailing list is the authoritative record. Decisions that happen in 
any sort of meeting are tentative until they find their way to the 
mailing list. (BTW: I think this helps avoid the dependence on having 
great minutes taken by volunteers, and the inevitable disappointment 
that brings.)

Cheers,
-Benson

---

From: Nico Williams <[email-masked]>
Date: Thu, 26 Feb 2015 13:26:18 -0600

On Thu, Feb 26, 2015 at 1:21 PM, Benson Schliesser
<[email-masked]> wrote:
> I agree with this. In particular, I think the best mindset is that a WG's
> mailing list is the authoritative record. Decisions that happen in any sort
> of meeting are tentative until they find their way to the mailing list.
> (BTW: I think this helps avoid the dependence on having great minutes taken
> by volunteers, and the inevitable disappointment that brings.)

Yes, but a record that a concall or other interim meeting took place,
and who attended, even if there are incomplete or missing minutes, is
important for IPR reasons.  Ensuring that such meetings are NOTE WELL
meetings is (should be) a priority, and that includes ensuring that a
record of that much exists.

Ideally the concalls and other interims would be recorded.

(Occasionally I've had to go back and listen to the recording of a WG
meeting at an IETF meeting.  It's very convenient that they exist at
all.  But they are also useful from a legal point of view, I'm sure,
and surely the same would be true of concalls and other interims, but
perhaps a recording requirement would be too onerous today?)

Nico
--

---

From: Benson Schliesser <[email-masked]>
Date: Thu, 26 Feb 2015 14:29:39 -0500

Yoav Nir wrote:
>> On Feb 26, 2015, at 6:53 AM, Benson Schliesser<[email-masked]>  wrote:

This *feels* like it should be true. But I can't find any definitive 
statement that requires attendees to sign blue sheets. Maybe I'm just 
overlooking it?

Cheers,
-Benson

---

From: Andy Bierman <[email-masked]>
Date: Thu, 26 Feb 2015 11:39:50 -0800

On Thu, Feb 26, 2015 at 8:27 AM, Joel M. Halpern <[email-masked]> wrote:
> If there are a lot of open issues (which would seem to go with having value
> in multiple calls) then a wiki or other easily used tracker to keep current
> information accessible is understandable.  I personally find that using
> those trackers as a place to have the conversations,as some WG chairs want
> to do, is very difficult.  But I can't say it is against the rules.
>
> If there is going to be noticable time lag between issues being raised and
> their being addressed in the document, using the document tracker to record
> all the issues and their resolutions is also very helpful.
>
> And I expect that design teams have more frequent calls than working groups.
> That is why you have a design team.
>
> call to call.  But that means that the WG is NOT seeing the discussion on
> the list.  And that the resolution, when it comes to the list, will be
> missing a LOT of context.
>

I agree with your concerns.
IMO virtual interim meetings should be used for discussing
big issues where the WG is stuck.  They should not be for
minor issues that could be handled on the mailing list.
They should not be status meetings either.

If work shifts away from the mailing lists to bi-weekly meetings,
then this is somewhat unfair to the people who live in timezones
where the meeting time is outside normal business hours.

> Joel

Andy

---

From: Benson Schliesser <[email-masked]>
Date: Thu, 26 Feb 2015 14:42:36 -0500

> Ideally the concalls and other interims would be recorded.

I agree completely. My point was that meeting records (including 
minutes) will inevitably be incomplete, or possibly inaccurate, and that 
relying on the mailing list as an authoritative record is more effective.

Of course it is disappointing that we can't meaningfully translate voice 
discussions into text, in the minutes or in mailing list threads. If 
there were some magic tool e.g. that took better minutes then I'd be 
happy to use it. But otherwise, I think we just have to trust chairs to 
manage WG collaboration in whatever way is most effective for their WG's 
collaborators.

This will inevitably be suboptimal for some part of the population. (For 
instance, I've never been able to find an interim meeting time that fits 
the schedules of all attendees.) But if they (we) always revert to the 
mailing list for decision making then I suspect our work can remain open 
and transparent.

Cheers,
-Benson

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 27 Feb 2015 08:52:53 +1300

On 27/02/2015 08:21, Benson Schliesser wrote:
> I agree with this. In particular, I think the best mindset is that a WG's mailing list is the authoritative record. Decisions
> that happen in any sort of meeting are tentative until they find their way to the mailing list. (BTW: I think this helps avoid
> the dependence on having great minutes taken by volunteers, and the inevitable disappointment that brings.)

Sure. But that means that the WG chairs (or secretary) have a strong
obligation to send *detailed* explanations of issues raised and proposed
resolutions to the mailing list, promptly, i.e. the same day or the
next day, asking for discussion and opinions. I'd think (from experience)
that for each hour of teleconference, that would take a half hour of
drafting a careful email. If you don't do that, you are disenfranchising
the people who could not attend the teleconference, which is not OK.

> They should not be status meetings either.

Nor should they be design team meetings masquerading as WG
meetings. If there's a core of active designers who want to
meet weekly, that's fine as long as we don't pretend it's
something else.

> where the meeting time is outside normal business hours.

Delete "somewhat" and I would agree with you.

    Brian

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 26 Feb 2015 15:05:43 -0500

> On Feb 26, 2015:2:39 PM, at 2:39 PM, Andy Bierman <[email-masked]> wrote:
> 

	I would argue that all of the IETF's WGs should move to some form of more granular meetings. This one key way how other organizations have accelerated their production rate. In the various open source organizations my teams and I 
participate often meet weekly. These meetings sometimes are not even using an A/V method - some use IRC. In other
cases all we do is record the Hangout or WebEx and make it available later with some meetbot notes. Another case in point is how we've been able to put our foot down on the gas in NETMOD with periodically scheduled (weekly) meetings. We have witnessed a significant increase in progressing both minor and major issues as well as closing down milestones that in the past were taking literally over a year, down to a few months. But those specific meetings incur significant and frankly a lot of unnecessary overhead due to IETF requirements.  We need to stop considering any form of officially scheduled meeting to have to comply with all of the requirements of a full-on in person meeting.

	My personal observation is that this is also why the IETF's leadership needs to carefully review the overhead associated with interim meetings to ensure they do not discourage/hamper/slow interim meetings.

	--Tom

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 26 Feb 2015 15:08:08 -0500

> On Feb 26, 2015:2:42 PM, at 2:42 PM, Benson Schliesser <[email-masked]> wrote:
> 
> Nico Williams wrote:
> I agree completely. My point was that meeting records (including minutes) will inevitably be incomplete, or possibly inaccurate, and that relying on the mailing list as an authoritative record is more effective.
> 
> Of course it is disappointing that we can't meaningfully translate voice discussions into text, in the minutes or in mailing list threads. If there were some magic tool e.g. that took better minutes then I'd be happy to use it. But otherwise, I think we just have to trust chairs to manage WG collaboration in whatever way is most effective for their WG's collaborators.

	The first step is to agree that an A/V recording is record enough. Perhaps having meetbot/txt notes that at a min include actions/decisions like we do in the issue tracker we've used for NETMOD's Yang 1.1's issues. 

	--Tom

> This will inevitably be suboptimal for some part of the population. (For instance, I've never been able to find an interim meeting time that fits the schedules of all attendees.) But if they (we) always revert to the mailing list for decision making then I suspect our work can remain open and transparent.
> 
> Cheers,
> -Benson
> 
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Feb 2015 15:37:36 -0500

> the people who could not attend the teleconference, which is not OK.

Exactly.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Feb 2015 15:47:50 -0500

But I have seen nothing in our procedures, or even in emails from ADs to 
the wg-chairs list, or to the IETF list, indicating that our leadership 
expects chairs to live up to this standard.
Yes, careful analysis suggests that this is really implicit in our 
current rules.  But it is not happening.

Yours,
Joel

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 27 Feb 2015 10:16:17 +1300

On 27/02/2015 09:08, Thomas D. Nadeau wrote:
> 	The first step is to agree that an A/V recording is record enough. 

It absolutely is not enough. Please see my previous message,
and the relevant rules in RFC 2418.

   Brian

> Perhaps having meetbot/txt notes that at a min include actions/decisions like we do in the issue tracker we've used for NETMOD's Yang 1.1's issues. 
> 
> 	--Tom
> 
> 
>

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 26 Feb 2015 16:21:50 -0500

> On Feb 26, 2015:4:16 PM, at 4:16 PM, Brian E Carpenter <[email-masked]> wrote:
> 

	You are missing my point. RFC or not, the IETF needs to evolve.

	--Tom

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Feb 2015 16:30:40 -0500

Thomas, if participants who can not make the conference calls are 
obliged to listen to the full recordings to get the key points of what 
has happened, why, what are the questions, and similar issues that need 
to be visible to the WG, then we are not running an inclusive process 
that allows for participation by the range of individuals we need.

There is a reason that the IETF distinguishes between design teams 
meetings, where the design team has to explain their work carefully to 
the WG, and working group meetings.  There has always been a problem of 
not getting as much context as we would like from the WG minutes.  But 
since we explicitly take all resolutions to the list, this is 
ameliorated by folks being able to ask for explanations, by those issues 
being taken to the list promptly, and by the fact that we only met 3 
times a year.  If you have bi-weekly calls and the WG can not tell what 
is going on with those calls, then what you have is a design team.  And 
then the folks involved need to own up to it as a design team, 
understand that they need to explain to the list what they have 
analyzed, their reasoning, and their conclusions.

Yours,
Joel

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Feb 2015 16:36:32 -0500

Tom,

The point I think Dave, Joel, Brian, myself, and others are
trying to make is that we don't see the direction you are
arguing for as positive evolution.  Instead, perhaps to
differing degreed and for different reasons, we see it as a step
backwards to an environment in which standards are created and
set in small meetings by representatives of vendors who can
provide a lot of support and under circumstances that provides
little diversity of views and perspectives.

We know standards bodies that do things that way and the wildly
successful network protocols they have produced.

     john

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 26 Feb 2015 16:48:56 -0500

> On Feb 26, 2015:4:30 PM, at 4:30 PM, Joel M. Halpern <[email-masked]> wrote:
> 
> Thomas, if participants who can not make the conference calls are obliged to listen to the full recordings to get the key points of what has happened, why, what are the questions, and similar issues that need to be visible to the WG, then we are not running an inclusive process that allows for participation by the range of individuals we need.

	That is the problem. We're really only geared to have full-on, in person meetings all the time. That does not lend itself to being flexible/agile.  That is not to say that I want to exclude anyone, but to be fair, if a subset of people want to move a pile of work forward, we shouldn't ENCOURAGE that behavior not stifle.

> There is a reason that the IETF distinguishes between design teams meetings, where the design team has to explain their work carefully to the WG, and working group meetings.  There has always been a problem of not getting as much context as we would like from the WG minutes.  But since we explicitly take all resolutions to the list, this is ameliorated by folks being able to ask for explanations, by those issues being taken to the list promptly, and by the fact that we only met 3 times a year.  If you have bi-weekly calls and the WG can not tell what is going on with those calls, then what you have is a design team.  And then the folks involved need to own up to it as a design team, understand that they need to explain to the list what they have analyzed, their reasoning, and their conclusions.

	Design teams should be able to work asynchronously, but with fixed schedules and not have to have everything explicitly documented at every step. If someone else is curious, they can get involved but not in order to slow things down or throw a monkey wrench into the works. If people want to keep leaning back on 10 year old process RFCs and arguing "well thats just the way we've always done things around here" then this organization is going to continue to slow its progress even more - and its descent into irrelevancy.  There are a lot of people here (myself included) that want to evolve things because they think the IETF still has a lot to offer the industry. But if the organization won't evolve, people will take the path of least resistance and go elsewhere as they have been doing if you haven't noticed. 

	If you want a real example of how this can actually work, watch Anees explain how Open Config has done this with just weekly phone calls and a bunch of people typing on keyboards. They've done this in less than a year, and have rough consensus and (production) running code.  This is how the IETF used to operate: people got together, hacked code and got things working.  The goal was not having meetings, but producing code with rough consensus.

	https://code.facebook.com/posts/1421954598097990/networking-scale-recap

	--Tom

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 26 Feb 2015 17:05:23 -0500

The OpenConfig team you are pointing at is a design team, not a working 
group.  And they do understand that all the decisions they have made 
need to be reviewed by a working group if they want IETF publication.

Yours,
Joel

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Feb 2015 17:11:32 -0500

On Feb 26, 2015, at 3:47 PM, Joel M. Halpern <[email-masked]> wrote:
> But I have seen nothing in our procedures, or even in emails from ADs to the wg-chairs list, or to the IETF list, indicating that our leadership expects chairs to live up to this standard.
> Yes, careful analysis suggests that this is really implicit in our current rules.  But it is not happening.

I think this is true, and we should do something about it.

---

From: Lou Berger <[email-masked]>
Date: Thu, 26 Feb 2015 17:12:06 -0500

Tom,
> 	That is the problem. We're really only geared to have full-on, in person meetings all the time. That does not lend itself to being flexible/agile.  That is not to say that I want to exclude anyone…

It sounds to that you are describing design team meetings (or, in some
cases, an authors meetings).  I'm not sure what makes folks think that
the only way folks can collaborate is via the mail list or full on
interim meetings. 

> 	Design teams should be able to work asynchronously, but with fixed schedules and not have to have everything explicitly documented at every step. 
100% agree.

I've been involved in many of such meetings (conf calls, webex, etc.)
and the key is that DT/authors' meetings need to be explained and
discussed on the list *at a time of their choosing*, which in my
experience is usually when a draft is published/updated.

> If someone else is curious, they can get involved but not in order to slow things down or throw a monkey wrench into the works. If people want to keep leaning back on 10 year old process RFCs and arguing "well thats just the way we've always done things around here" then this organization is going to continue to slow its progress even more - and its descent into irrelevancy.  There are a lot of people here (myself included) that want to evolve things because they think the IETF still has a lot to offer the industry. But if the organization won't evolve, people will take the path of least resistance and go elsewhere as they have been doing if you haven't noticed. 

This statement just confuses me as you note below we've always had ways
for folks to make progress fast -- when there's interest in doing so. 
It just seems to me that many are enamored with interims and think it's
the sole/best way of demonstrating progress between full meetings. 
Perhaps you're just saying that they're mistaken...

Lou

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 26 Feb 2015 17:23:22 -0500

> On Feb 26, 2015:5:12 PM, at 5:12 PM, Lou Berger <[email-masked]> wrote:
> 
> Tom,

	Yes.

> Perhaps you're just saying that they're mistaken...

	Its not being enamored as much as it being one of the only obvious/acceptable vehicles to progress WG-level 
work forward - at least if the management is involved. In NETMOD for example, we've broken the interims into two "themes": Yang 1.1 work and modeling.  The former is like its own design team, and the latter is like many design teams coming to one place every other week.  The former not only meets every other week, but discusses issues on the list. But to the latter - that is more like a touch point for those subteams. Those subteams go off on their own for weeks at a time and iterate as needed. And they work without all of the overhead of a formal meeting. They may or may not discuss progress on the list until issues come up.

	--Tom

---

From: Lou Berger <[email-masked]>
Date: Thu, 26 Feb 2015 17:34:25 -0500

On 2/26/2015 5:23 PM, Thomas D. Nadeau wrote:
> Those subteams go off on their own for weeks at a time and iterate as needed. And they work without all of the overhead of a formal meeting. They may or may not discuss progress on the list until issues come up.

There's nothing wrong/out of process with this. The key process
requirement is that they do review & discuss changes with the WG at some
point (which sometimes *is* sometimes a challenge to ensure that it
happens), and certainly no later than when the changes are published. 

Lou

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 27 Feb 2015 12:13:52 +1300

> happens), and certainly no later than when the changes are published. 

And *any* decision taken that way, right down to the choice to use binary
arithmetic, may be challenged and changed by the WG as a whole, and for
that matter by the IETF as a whole when it gets to Last Call. And saying
"go listen to the MP3 file" is not an answer to a technical challenge.

Nobody is arguing against efficient discussion between active designers
and authors. That's been going on since before the IETF existed. Nobody
is arguing against using new technology for such discussions, or against
making them as inclusive as possible.

   Brian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 26 Feb 2015 16:52:07 -0800

On Feb 26, 2015, at 3:13 PM, Brian E Carpenter wrote:
> that matter by the IETF as a whole when it gets to Last Call.

FWIW, that certainly wasn't the case for HTTP/2.  The IETF is currently
very flexible in how chairs are allowed to apply its processes.

....Roy

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 27 Feb 2015 15:40:02 +1300

> very flexible in how chairs are allowed to apply its processes.

Roy, if I had come up with a rational argument why HTTP/2 should have been
based on ternary arithmetic, during IETF Last Call, and the IESG had thought
my argument had merit (and some support in the community), the document
would have got held up until the matter was resolved.

     Brian

---

From: Abdussalam Baryun <[email-masked]>
Date: Thu, 26 Feb 2015 21:12:20 -0800

Thanks. IMHO participants  in IETF are in two categories: individuals and
companies, both needs to have diversity. I think IETF has shortage in
remote participants and in diversity, so the strategy is that no fees
because IETF needs more diversity.

However, the issue in my thoughts is not remote or non-remote or ability to
pay or not (diversity items), but the issue can be about benefits, cost and
payment return/outcome. For example, usually remote participants attend but
the outcome is less than others. In business and organisation what matters
is utilization of time, money, and attendance. People in business meet for
many reasons and different methods. All IETF participants attending meeting
physically have different utilization but pay the same fee. I think that
makes limited attendance. Now days Internet services changing to you pay
for what you use only, so could IETF offer that?

Let us focus on participant outcome (individual or organisation) from the
IETF meeting and then think about the fees to be added or increased per
category. IETF may need to look into that to manage participants'
satisfactions and expectations. I think companies that use and benefit from
IETF services should not be treated similar to individual participant. I
suggest that companies participating (i.e. company name is written in the
WG draft) should/may pay fees specific for that session utilization of
IETF.

AB

IETF Participant from Africa

On Wednesday, February 25, 2015, Eric Burger <[email-masked]>
wrote:

> Mea culpa!
>
> My fault for not being articulate. I was responding to the discussion
> thread, and happened to chose this one to staple the reply to. I in no way
> meant to infer that Ted (or Dave or anyone else) was advocating that the
> IETF should be excluding people. In this case, the impression (which Ted
> pointed out to me) was that I was saying that he wanted to exclude people
> based on ability to pay. Ted and I discussed this off-list, and we are in
> violent agreement. I did not think he was advocating for that, and I do not
> want anyone to advocate for that.
>
> Moving forward, what I was hoping to avoid was for people to think that
> because the IETF conference fees defer the costs of operating the IETF (the
> meeting itself, the secretariat, and a portion of the RFC Editor), that we
> have to try to squeeze every penny from all sources. That is not
> necessarily a bad mindset to have: we should be striving to be independent
> on the largesse of the Internet Society and their contributors if we can
> manage it. That is not an infinite pot of cash, and no one wants to be
> beholden to a single funding source. However, what I wanted to get out to
> the community is the message that the Internet Society believes deeply in
> expanding access to the IETF and the IETF process. If charging for remote
> access inhibits participation (the unfortunate ‘paywall’ comment), then I
> would have no problem at all suggesting the IETF (IAOC in specific) ask the
> Internet Society to fund remote participation. I think the Board (speaking
> as an individual, NOT in my role as an Internet Society Trustee) would
> treat such a request sympathetically.
>
> I can see this could be a dynamic situation. I can envision a time when we
> as the IETF are truly successful and develop fantastic real-time
> communication protocols that are easy to deploy, cost almost nothing, and
> are secure. At that point, one would *hope* in-person IETF meetings become
> a relic of history. Maybe we would meet once per year or every other year
> to reminisce about how the only way to get work done was to spend thousands
> of dollars of cash per year and an uncountable amount of cost for travel
> time to physically meet in the same location. How 20th Century! At that
> point, I would expect surpluses from meeting fees would be nonexistent, and
> we would need to figure alternate means of funding. However, that day seems
> to be far enough away that charging for remote participation should be a
> remote possibility for the foreseeable future.
>
> > On Feb 24, 2015, at 9:24 PM, Ted Lemon <[email-masked]
> <javascript:;>> wrote:
> >
> > On Feb 24, 2015, at 8:32 PM, Eric Burger <[email-masked]
> <javascript:;>> wrote:
> >> The last thing we need as we are just beginning to have success
> reaching out beyond North American, European, Japanese, S. Korean, and
> Australian mid-size to large corporations is to toss up a paywall, some as
> much as a month’s salary or more, for the ‘privilege’ of contributing to
> the IETF.
> >
> > You know, it's really frustrating when you participate in a discussion,
> try to contribute helpfully, and then essentially get accused of being a
> blithering idiot by someone who didn't bother to consider the possibility
> that you might not be.   I'm sure you've had that experience too.   Heck,
> I've been the one who assumed the other person was an idiot too, so I can
> relate.
> >
> > Anyway, if you think I was proposing a paywall, please go back and
> re-read what I actually wrote, and the rest of the discussion that
> followed, with the presumption in mind that I did _not_ mean to propose any
> such thing (because I didn't!), and see if the discussion still works, or
> if you find something I or someone else said that contradicts that
> assumption.
> >
> > Thanks.
> >
>
>

---

From: Nico Williams <[email-masked]>
Date: Fri, 27 Feb 2015 00:08:36 -0600

> companies, both needs to have diversity. I think IETF has shortage in

It may seem like a fiction, but all IETF participants are individuals.

Many a participant's participation is funded by their employer, and in
the process faithfully represent the employer's interests.  But the
participants are individual persons.

> because IETF needs more diversity.

I doubt it, though I don't know.  Remote *meeting* participation tools
have to improve in order to be able to charge much at all for *meeting*
participation.  But we'd pay if the tools were good enough and the IETF
charged for it, or even just if the IETF chaged for it.

Participation on mailing lists should continue to be without charge for
all.  No one is proposing otherwise.

> for what you use only, so could IETF offer that?

Fees are the least of the *meeting* participation costs.  Flights and
hotels are the large majority of the cost (ignoring cost of opportunity,
which may or may not be positive, though I'd assume it's negative in
general, else there would be very few meetings indeed).

> Let us focus on participant outcome (individual or organisation) from the

We measure IETF effectiveness and meeting quality in many ways, but we
don't rate *meeting* *outcomes* for *individual* participants.  (E.g.,
if I don't get my way on a consensus call, that may [or may not!] be a
negative outcome for me, but no one is going to cry me a river for it,
or even take much notice, and I don't expect any different, nor should
anyone.)

Nico
--

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 27 Feb 2015 06:56:05 +0000

I suggest we get rid of cookie neutrality. The more you pay, the faster access you have to the cookies...

Regards,

Christer

Sent from my Windows Phone
________________________________
From: Nico Williams<mailto:[email-masked]>
Sent: ‎27/‎02/‎2015 08:08
To: Abdussalam Baryun<mailto:[email-masked]>
Cc: [email-masked] Discussion<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: 'Paywall, ' IETF self-sufficiency, increasing participation (was Re: Remote participation fees)

Nico
--

---

From: Alia Atlas <[email-masked]>
Date: Fri, 27 Feb 2015 09:57:12 -0500

Christer,

I think we should start with an experiment where it clearly counts -
getting rid of ice cream neutrality.
That's something where speed really matters ;-)

Alia

On Fri, Feb 27, 2015 at 1:56 AM, Christer Holmberg <
[email-masked]> wrote:

>  ------------------------------
> From: Nico Williams <[email-masked]>
> To: Abdussalam Baryun <[email-masked]>
> Cc: [email-masked] Discussion <[email-masked]>; [email-masked]

---

From: "Scott O. Bradner" <[email-masked]>
Date: Fri, 27 Feb 2015 10:02:25 -0500

the requirement in rf. 2418 is that a list of attendees needs to be provided to the Secretariat

the mechanism by which the list is created is not specified

Scott

> On Feb 26, 2015, at 2:29 PM, Benson Schliesser <[email-masked]> wrote:
> 
> 
> 
> Yoav Nir wrote:
> This *feels* like it should be true. But I can't find any definitive statement that requires attendees to sign blue sheets. Maybe I'm just overlooking it?
> 
> Cheers,
> -Benson
>

---

From: t.p. <[email-masked]>
Date: Fri, 27 Feb 2015 14:44:19 +0000

----- Original Message -----
From: "Benson Schliesser" <[email-masked]>
To: "Yoav Nir" <[email-masked]>
Sent: Thursday, February 26, 2015 7:29 PM
> >> Of course, chairs being required to submit blue sheets is not the
same as attendees being required to sign them.
> statement that requires attendees to sign blue sheets. Maybe I'm just

Benson

As Brian pointed out earlier, RFC2418 makes the reporting of attendees a
'should'.  The most recent IESG statement
http://www.ietf.org/iesg/statement/interim-meetings.html
says
"The minutes, including a list of attendees, must be sent to
[email-masked] within 10 days after the meeting, conference call
or jabber session concludes."

I note the 'must'.

And there is a webpage

http://www.ietf.org/documents/interim-meeting-blue-sheet-f2f.pdf

although there is no statement as to whether the use of this document is
a must, should, or something else!  So in terms of participants signing,
I do not see a requirement, but in terms of participants being reported,
I do.  And I would link this to 'Note Well' which warns participants
that their participation may be a matter of public record and so
whatever technology is used for a virtual interim, it should ensure that
participants see the 'Note Well' in some form.

And I would like to see that '10 days' modified, to be either 10 days or
three days prior to the next meeting to cover the case I see of weekly
meetings, when 10 days later is too late IMO.

Tom Petch

---

From: t.p. <[email-masked]>
Date: Fri, 27 Feb 2015 15:03:01 +0000

Inline <tp>

----- Original Message -----
From: "Thomas D. Nadeau" <[email-masked]>
To: "Berger Lou" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 26, 2015 10:23 PM
>> That is the problem. We're really only geared to have full-on, in
person meetings all the time. That does not lend itself to being
flexible/agile.  That is not to say that I want to exclude anyone, but
to be fair, if a subset of people want to move a pile of work forward,
we shouldn't ENCOURAGE that behavior not stifle.

Yes.
>> Design teams should be able to work asynchronously, but with fixed
schedules and not have to have everything explicitly documented at every
step.
>> If someone else is curious, they can get involved but not in order to
slow things down or throw a monkey wrench into the works. If people want
to keep leaning back on 10 year old process RFCs and arguing "well thats
just the way we've always done things around here" then this
organization is going to continue to slow its progress even more - and
its descent into irrelevancy.  There are a lot of people here (myself
included) that want to evolve things because they think the IETF still
has a lot to offer the industry. But if the organization won't evolve,
people will take the path of least resistance and go elsewhere as they
have been doing if you haven't noticed.
> Perhaps you're just saying that they're mistaken...

Its not being enamored as much as it being one of the only
obvious/acceptable vehicles to progress WG-level
work forward - at least if the management is involved. In NETMOD for
example, we've broken the interims into two "themes": Yang 1.1 work and
modeling.  The former is like its own design team, and the latter is
like many design teams coming to one place every other week.  The former
not only meets every other week, but discusses issues on the list. But
to the latter - that is more like a touch point for those subteams.
Those subteams go off on their own for weeks at a time and iterate as
needed. And they work without all of the overhead of a formal meeting.
They may or may not discuss progress on the list until issues come up.

<tp>
Tom

Take a small group of engineeers, expert in technology, get them to hold
regular meetings focussed on a narrow range of topics and they can make
faster progress, as you cite for 'netmod'.

What is also likely to happen, and I see it with netmod, is that they
will develop their own way of working, their own terminology, their own
technology even, which de facto raises the bar for anyone else who wants
to participate or to understand what happened.  They don't mean to
exclude other people, they just do. (small groups, Psychology 101)

I see 'netmod' as a poster child for this with its issue list, state
machine for issues and so on.  Even though I was tracking the list when
the 'Ynn' issue list was created, I don't know where its state machine
came from.  In recent minutes, I don't know what
"  AB: I am not sure YANG 1.0 specifies C1 explicitly somewhere.
  JS: Does A3 not follow from A2?
  KW: A3 is more a corollary of A2.
  AB: The high-level problem is how to create and maintain the
      information needed to achieve A4. "

is about; a brief search of mailing list and I-Ds gave me no explanation
for A2 to C1.

And if a different group of engineers works on different topics, then
they will likely, in the absence of any guidance, use different
technology, different terminology and end up with a way of working that
is as alien to the first group.

As I said, changing the way we work.

Tom Petch
</tp>

--Tom

>
> Lou
>
>> If you want a real example of how this can actually work, watch Anees
explain how Open Config has done this with just weekly phone calls and a
bunch of people typing on keyboards. They've done this in less than a
year, and have rough consensus and (production) running code.  This is
how the IETF used to operate: people got together, hacked code and got
things working.  The goal was not having meetings, but producing code
with rough consensus.
minutes) will inevitably be incomplete, or possibly inaccurate, and that
relying on the mailing list as an authoritative record is more
effective.
>>>>>>>
>>>>>>> Of course it is disappointing that we can't meaningfully
translate voice discussions into text, in the minutes or in mailing list
threads. If there were some magic tool e.g. that took better minutes
then I'd be happy to use it. But otherwise, I think we just have to
trust chairs to manage WG collaboration in whatever way is most
effective for their WG's collaborators.
>>>>>> Perhaps having meetbot/txt notes that at a min include
actions/decisions like we do in the issue tracker we've used for
NETMOD's Yang 1.1's issues.
>>>>>>
>>>>>> --Tom
>>>>>>
>>>>>>
>>>>>>
>>>>>>> This will inevitably be suboptimal for some part of the
population. (For instance, I've never been able to find an interim
meeting time that fits the schedules of all attendees.) But if they (we)
always revert to the mailing list for decision making then I suspect our
work can remain open and transparent.

---

From: Mary Barnes <[email-masked]>
Date: Fri, 27 Feb 2015 09:45:19 -0600

Personally, I think those of us that can't eat the cookies should be in a
priority queue for the ice cream.

---

From: Nico Williams <[email-masked]>
Date: Fri, 27 Feb 2015 10:08:21 -0600

A low sugar option would be nice.  Cheese, say.  Doesn't need refrigeration.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 27 Feb 2015 11:57:39 -0500

The secretariat has sometimes put out nuts.  Which are great and I love. 
  I suspect however that they tend to be MUCH more expensive than other 
choices.

Yours,
Joel

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 27 Feb 2015 17:02:06 +0000

On 27/02/15 16:57, Joel M. Halpern wrote:
> The secretariat has sometimes put out nuts.

I think the secretariat should evict those participants
much more often:-)

S.

---

From: "John Levine" <[email-masked]>
Date: 27 Feb 2015 22:19:57 -0000

>The secretariat has sometimes put out nuts.

A few times they put out tuna sandwiches, and as I recall those tended
to disappear long before the cookies did.

On the assumption that remote participants can make their own
sandwiches, do we have any estimate of what benefits remote
participants bring to our live WG sessions?  I've certainly been to
plenty of sessions where the chair or document author was on the phone
or typing fast into jabber, and the session would have been vastly
less useful if he or she weren't there.

R's,
John

---

From: <[email-masked]>
Date: Sat, 28 Feb 2015 07:22:33 +0000

I have a nut allergy.

But since he changed countries it's pretty much under control.
________________________________________
From: ietf <[email-masked]> on behalf of Joel M. Halpern <[email-masked]>
Sent: Saturday, 28 February 2015 3:57:39 AM
To: Nico Williams; Mary Barnes
Cc: [email-masked]; [email-masked] Discussion
Subject: Re: 'Paywall, ' IETF self-sufficiency, increasing participation (was Re: Remote participation fees)

Yours,
Joel

---

From: Abdussalam Baryun <[email-masked]>
Date: Sat, 28 Feb 2015 13:12:27 -0800

I don't think IETF is only about individual participation as your message
may mentioned. IETF needs both forces.

> all.  No one is proposing otherwise.

If IETF charges that channel then it will die, that is its way of
progress.  I know no one proposed charging the list, it is all about
meetings.

> or even take much notice, and I don't expect any different, nor should

How does IETF measure quality if not considering average individual
outcome? Usually the receiver determines quality not the server. We need to
know the real average outcome of IETF meeting.

 That example you mentioned was not my meaning of participant-outcome, the
meaning examples of outcome are: 1) one individual in One region wants to
attend one IETF meeting WG session in another region, but he/she must pay
fee for at least one full day. 2) A small company in One region wants to
participate in one IETF meeting day in same region, so it only needs to
send one individual. Both 1 & 2 participations pay the same
meeting-fee, but different outcome of using IETF meeting. 3) an individual
not supported by employer is participating in IETF (full volunteering), and
got WG acceptance of presentation slot. 4) an individual supported by their
company (we know support by naming company name in the IETF, so if it is
named/called then it supports that individual) but was not accepted with
clear reason. Both 3 & 4 are paying same in fees and same outcome (i.
e. good feedback and respond) but different in work support and
different in participation per work.

AB

---

From: Pranesh Prakash <[email-masked]>
Date: Sun, 01 Mar 2015 04:45:07 +0530

> It may seem like a fiction, but all IETF participants are individuals.

It seems like a fiction because it is a fiction.

Check the IETF main list where there is discussion going on about CFOs 
(Chief Financial Officers) footing the bills for participants.

That doesn't sound like a person participating in their personal 
capacity to me.

-- 
Pranesh Prakash
Policy Director, Centre for Internet and Society
T: +91 80 40926283 | W: http://cis-india.org
Twitter: https://twitter.com/pranesh_prakash

---

From: John C Klensin <[email-masked]>
Date: Sat, 28 Feb 2015 18:49:28 -0500

--On Sunday, March 01, 2015 04:45 +0530 Pranesh Prakash
<[email-masked]> wrote:

>>> Thanks. IMHO participants  in IETF are in two categories:
>>> individuals and companies, both needs to have diversity. I
>>> think IETF has shortage in
>> 

Pranesh,

Please be careful about over-generalizing.  Certainly many,
almost certainly most, IETF participants are supported for their
attendance, while they are attending, and for participation by
the companies or clients for whom they work.   But some of us
are not and are participating as individuals by any plausible
way you can figure out to classify "individual".  

    john

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 01 Mar 2015 14:15:05 +1300

> way you can figure out to classify "individual".  

Additionally, the IETF interprets each person's input as individual,
whether it is or it isn't. That in itself is quite effective in
disenfranchising employers.

Also we do, all of us, know how to recognise when someone is parrotting
the BigEvil Corporation's party line, and we all know that if six people
with the same affiliation have identical opinions on a contentious
point, that's roughly equivalent to one individual opinion.

The rough consensus process is actually quite good at resisting gaming
by BigEvil Corporation; see sections 6 and 7 of RFC 7282.

    Brian

---

From: Pranesh Prakash <[email-masked]>
Date: Sun, 01 Mar 2015 07:20:59 +0530

John C Klensin <[email-masked]> [2015-02-28 18:49:28 -0500]:
> Please be careful about over-generalizing.

The negation of "all IETF participants are individuals" is "not all IETF 
participants are individuals".

Where is the generalization?

---

From: Pranesh Prakash <[email-masked]>
Date: Sun, 01 Mar 2015 07:34:41 +0530

Brian E Carpenter <[email-masked]> [2015-03-001 14:15:05 
> disenfranchising employers.

Is there an RFC that states this?

Further, regardless of how the IETF treats inputs in theory, the reality is:
1) A large percentage of IETF participants are paid by their employers 
for participating in IETF proceedings and meetings.
2) Employers are in no way, shape, or form discouraged from putting 
forward their views through their employees.
3) A sizeable number of IETF participants do so using their work e-mail 
addresses.
4) Employers' names are mentioned in each RFC to indicate an 
individual's pedigree.

I would like to know how employers are disenfranchised when this is the 
reality.

I find this entire, "we are all individuals" line to be the foundational 
fiction of

> point, that's roughly equivalent to one individual opinion.

My worry isn't BigEvil corporation.  My concern is that pretending 
individuals who are paid for by their employers and those who 
participate purely in their personal capacity are somehow equal because 
of their equal capacity to hum is fallacious and harmful towards 
achieving diversity.  The first step towards achieving diversity is 
recognizing that a) there is a lack of diversity; b) there are reasons 
preventing diversity.

> by BigEvil Corporation; see sections 6 and 7 of RFC 7282.

Again: my concern isn't gaming of consensus, nor do I believe 
corporations who sponsor their employees' IETF work are evil.  My 
concern is diversity within the IETF and within the larger sphere of 
Internet governance.

---

From: Pranesh Prakash <[email-masked]>
Date: Sun, 01 Mar 2015 07:35:39 +0530

I find this entire "we are all individuals" line to be the foundational 
fiction of the IETF.  Slowly that has gotten extended into the "IETF is 
a the most open, multistakeholder Internet governance body" line.

---

From: Andrew Sullivan <[email-masked]>
Date: Sat, 28 Feb 2015 21:07:56 -0500

On Sun, Mar 01, 2015 at 04:45:07AM +0530, Pranesh Prakash wrote:
> That doesn't sound like a person participating in their personal capacity to
> me.

Quite apart from what John says, I'd like to know what you think
determines when someone is participating in their personal capacity.
You seem to be suggesting that the source of the funding automatically
determines what a participant does or says.

I can tell you that my employer subsidizes participation of a number
of us in the IETF because my employer thinks the Internet is better if
the IETF is strong and full of independent voices.  We do not
co-ordinate our positions.  My employer does not have a view (or at
least not one it has expressed) about what I ought to interest myself
in at the IETF.  The biggest problem I have ever had is juggling these
obligations (which my employer recognizes and supports) with the rest
of my day job.

So, if you prefer to say that I'm still not acting "in my personal
capacity", very well; but I'd like to know what the difference is (in
operationalized terms, please).

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Pranesh Prakash <[email-masked]>
Date: Sun, 01 Mar 2015 07:43:32 +0530

> operationalized terms, please).

As you seem to have noted, I believe funding to take part in IETF 
proceedings (and physical meetings) is the key.

Not because it "influences" the views you put forward at the IETF.  (It 
may or may not do so.)

But because it enables you to participate more steadily in the IETF than 
someone who is not similarly funded.  That ability to participate has 
all manners of implications, including the ability to be chair of WGs, etc.