Subject: Re: I-D Action: draft-jaeggli-interim-observations-00.txt

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 15 Oct 2012 22:41:41 +0100

Joel,

Thanks for this stake in the ground as a starting point for recording
observations on the LIM. As an AD who was not there, I believe it is really
important that we try to capture this experience to decide whether to repeat the
idea, and if so, how to improve it.

Cheers,
Adrian

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of [email-masked]
> Sent: 15 October 2012 22:28
> To: [email-masked]
> Subject: I-D Action: draft-jaeggli-interim-observations-00.txt
> 
> 
> A New Internet-Draft is available from the on-line Internet-Drafts
directories.
> 
> 
> 	Title           : Observations on the experience and nature of Large
Interim
> Meetings
> 	Author(s)       : Joel Jaeggli
> 	Filename        : draft-jaeggli-interim-observations-00.txt
> 	Pages           : 4
> 	Date            : 2012-10-15
> 
> Abstract:
>    Planning, particpipation and conclusions from the experience of
>    participating in the IETF LIM activity on september 29th 2012.
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-jaeggli-interim-observations
> 
> There's also a htmlized version available at:
> http://tools.ietf.org/html/draft-jaeggli-interim-observations-00
> 
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Randy Bush <[email-masked]>
Date: Mon, 15 Oct 2012 11:49:17 -1000

ok, i am lost.  the draft is only an outline and has zero content?  is
it a quiz?

imiho, the sidr meeting was useful and got work done which probably
would have taken a lot more time online and would have had the wonderful
email misunderstanding amplification.

randy

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 15 Oct 2012 22:53:07 +0100

> ok, i am lost.  the draft is only an outline and has zero content?  is

Treat it like that and see if you can give Joel the right answers.

For me: Did it make any difference to you that it was a LIM rather than simply a
SIDR interim? Were logistics and resources worth the fee? Should we hold future
LIMs, or do the scheduling and other issues mean that normal interims are the
way forward?

cheers,
Adrian

---

From: Warren Kumari <[email-masked]>
Date: Mon, 15 Oct 2012 17:53:29 -0400

> ok, i am lost.  the draft is only an outline and has zero content?  is

No, I believe that it is very subtle satire, reflecting Joel's view on the utility of the meeting :-P

> email misunderstanding amplification.

+lots.

Sad as it is, getting folk together in a room helps focus the discussion and provides many fewer opportunities for misunderstanding… 

--
I don't think the execution is relevant when it was obviously a bad idea in the first place.
This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. 
   ---maf

Warren Kumari
[email-masked]

---

From: Randy Bush <[email-masked]>
Date: Mon, 15 Oct 2012 12:04:09 -1000

[ i should know better than to get into this maelstrom, but .. ]

> For me: Did it make any difference to you that it was a LIM rather
> than simply a SIDR interim?

no.  face to face with the ability for remote participation is what
counts.

> Were logistics and resources worth the fee?

not really.  though i heard that meetecho was better than webex had been
at previous interims.

and i note that 'large' is a bit disingenuous, as 42 other wgs held
meetings elsewhere within a few weeks of that 'LIM'.

to me, the real logistics issue is the lack of viable two-way remote
voice, let alone video, participation.  heck, back when dinosaurs roamed
the ietf corridors (instead of circuitzilla), a remote participant used
to be able to speak and be heard.  the whole vmeet bs got so wraped
arount featuritis, control freakism, and the union of everyone's
fantasies that we are stalled having regressed more than a decade.

> Should we hold future LIMs, or do the scheduling and other issues mean
> that normal interims are the way forward?

for the logistics to be worth it, you would have to whack the other wgs
to conform.  if you whack, then how is this different from another
flavor of general meeting?

randy

---

From: joel jaeggli <[email-masked]>
Date: Mon, 15 Oct 2012 15:08:40 -0700

> No, I believe that it is very subtle satire, reflecting Joel's view on the utility of the meeting :-P
or it's unsubtle and I just needed a structure and a 00 before the 
deadline, because the text that exists now isn't cooked enough yet. that 
may be an sube of the process but if there's an 01 before 1700 pacifc 
I'll post it.

---

From: Melinda Shore <[email-masked]>
Date: Mon, 15 Oct 2012 14:37:42 -0800

On 10/15/12 1:53 PM, Warren Kumari wrote:
> Sad as it is, getting folk together in a room helps focus the
> discussion and provides many fewer opportunities for
> misunderstanding…

I don't like interim meetings, but unfortunately the IETF seems
increasingly meeting-deadline driven so more deadlines is probably
the best way to keep work moving forward.  They've also been very
effective in helping to transition the IETF to an organization
that's far friendlier to large companies with big travel budgets
than it is to smaller companies with more constraints.  I also
wonder if some companies think they're IETF contributors by
putting bodies in chairs at meetings instead of focusing on
contributing to the work.  But hey, it's all good.

Melinda

---

From: Jari Arkko <[email-masked]>
Date: Tue, 16 Oct 2012 18:17:32 +0300

Joel,

Thanks for writing this. I have some detailed comments, but perhaps I should first start with my own perception of the meeting.

I traveled to this meeting as a part my trip to attend RIPE a few days, and to catch a few different people in the hallways on separate topics. One datapoint: I would probably not have made the trip just for RIPE this time (although I usually do travel to them), nor would I have attended just for the LIM itself.

I was quite surprised by the small attendance, what I heard from the secretariat was 30+ people registered for the interim. From a pure financial perspective this meeting did not seem to make a lot of sense for the IETF, 30 x 100$ for a lot of secretariat effort/travel, and presumably money for room fees and other costs. Most previous interim meetings have operated on a hosting/sponsoring model, or run in zero cost environments. I'd say that anything under hundred persons is probably easier to deal with in that way than attempt to use an official organization for it.

But money is really a side issue, so lets not focus on the above too much. What matters if we can progress the IETF's work:

The various meeting had a big difference between them. The SIDR meeting was clearly a close group of people with an intent to solve a design problem, going over an issue after another issue, involving people who would not have traveled to the IETF, and at least from my perspective the meeting was making quite a lot of progress in some issues that had been divisive earlier. This reminded me of some earlier SOFTWIRE meetings, for instance, where the designers and implementors met and made a lot of progress. An excellent meeting!

FWIW, in the IETF I would not have had time to go to SIDR, and I would not have traveled anywhere to go to a SIDR meeting. But I'm glad I went to this one because it allowed me to learn more about this technology, and some of the layer nine implications at the IAB force me to try to understand its state.

I did not attend OPSEC, but I attended V6OPS. All discussions were useful, but topics seemed somewhat disconnected, and they did not cover the whole range of usual V6OPS work. It was clear that the room did not have the full set of key contributors in the WG. I had prepared comments on a recent topic in the mailing list, but the relevant people were not there to receive my comments. Nevertheless, the V6OPS meeting did have a fair number of the usual experts in this topic, as well as a couple of extra people from the European routing community. I was able to have a high-bandwidth discussion on several topics with the people present; albeit that was in the hallways and after the meeting. We started one new draft for V6OPS based on this interim, and another (yet to be completed) work item for the HOMENET WG. In the latter case I would not have had the necessary people present in the IETF, so for me this was quite positive.

It is hard to tell why there was no more attendance in V6OPS. Wrong time and place? Or true lack of interest to invest in some of the topics that we are just not seeing it in the IETF because everyone comes there anyway?

Conclusions? Maybe we need more discussion on this, but here are some obvious ones:

o  Interim meetings can be extremely productive in the right situation. Or completely unnecessary in others. It is difficult to find a situation where the planets are properly aligned such that several working groups are in the right phase to have an interim meeting, and that the co-location with some event makes sense for all of those working groups. In other words, the success of a LIM seems somewhat unlikely, though there may be success for individual WG interims.

o  Co-location with RIPE appeared useful. I agree with you Joel that tighter packing would have made a difference. I met some people who noted they will not attend, but probably would have attended if it was during the week. Co-locating individual WG interims with RIPEs and NANOGs seems like a useful concept to consider in the future.

o  LIMs will not create a new big funding source for the IETF. We should also right-size our organization for the task at hand. 30, 50, or even 100 people could probably be handled as part of the RIPE meeting, and might have been something that the RIPE registration system and agenda could have accommodated, or have someone sponsor a room and leave the rest to participants.

o  As usual, hallways matter more than the official meetings. Put the right people in the same building, and I'll attend, even if the topic would be International Charactersets for EBCDIC-based FTP.

o  Saturday is not a good day.

Detailed comments:

>     The LIM was the attempt that I am aware of an interim meeting
>     scheduled by IETF management for the purposes of accumulating interim

Does not parse. Did you mean to say that LIM was the only attempt of its kind? A couple of years ago we also had the Malta proposal which failed (possibly due to the downturn, or possibly due to the same issues that made the Amsterdam event surprisingly small).

>     meetings in a common location rather that scheduled by working-group
>     participants, chairs nad coordinating ADs.

s/and/nad

>     It is, my understanding that discussion of the possbility of a LIM
>     style meeting occured early2011 if not before.

s/possbility/possibility/
s/early2011/early 2011/

There has been discussions of something like this over the years. FWIW I am not aware of the exact details of when and how this LIM discussion started. Maybe Ron or someone else from the current IESG could explain it.

> The stated rational for targeting v6ops
>     involvement in a large interim was the volume of work that we process
>     during and between meetings.

Interestingly, the amount of work in the interim meeting itself did not seem match this as well as I would have expected. We can speculate about the causes.

I came prepared to talk about certain topics, but the draft authors in question were not present. Perhaps an isolated case, but had the impression that you guys dealt with only a subset of the work that v6ops meetings in IETF deal with. Do you agree?

> Superficially only a fraction of the v6ops attendees are
>     represented by the interested segment however when the numbers are
>     mapped against active participants and draft authors, interested
>     participants in the interim represent a bigger purportion of that
>     group

Missing ".".

I think we have a participant list. It would be useful to have an understanding of how many % of the people who presented or made mike line comments in IETF-84 were present in the interim.

>     One osbervation that I would make about the interim submission
>

observation

>     V6ops attendance at an IETF meeting is typically in excess of 200
>     attendees.  An interim meeting that attracts 25 of those and
>     minuscule remote participation is necessarily exclusionary by default
>     if not deliberately.  If useful work that advances drafts, gets done,
>     is that exclusivity a bad thing?  It's not useful for measuring
>     meaningful consensus.

They are different meetings. And for sure the interim meetings do not help in measuring consensus. However, at their best they are meetings where closely knit groups of core developers can progress their work in a high-bandwidth collaboration environment. I have yet to see an interim meeting that would actually represent any meaningful consensus measurement point (to the extent any physical meeting can; the list is what counts). Or perhaps the Montreal IPv6 transition meeting was one, attended by close to a hundred persons. In any case, this does not necesssarily render the interim meetings useless, just that you have to use them as a different tool. Progress work. Move forward in design. Write specs. But far less consensus measurement, far less various experts from different domains, etc. Useful early in the work, not so useful later when finetuning something.

>     Area director's were rather well represent at the LIM, While the
>     attendance of both of our Directors was appreciated I'm not sure that
>     it's a good use of their time.  In particular if the frequency of
>     these events were fixed as some rate in the future, this represents
>     an additional workload for which huge benfits due not appear liekly
>     to ensue

There were also (at least) two IAB members, both of whom were present on site for other reasons as well.

Jari

---

From: joel jaeggli <[email-masked]>
Date: Tue, 16 Oct 2012 08:39:34 -0700

> Treat it like that and see if you can give Joel the right answers.
01 is available. I imagine the SIDR experience was a bit different, 
having been to another SIDR interim, I tried to call that out, but I 
wasn't in the room.

http://tools.ietf.org/html/draft-jaeggli-interim-observations-01

---

From: Randy Bush <[email-masked]>
Date: Tue, 16 Oct 2012 07:19:56 -1000

> o Co-location with RIPE appeared useful. I agree with you Joel that
>   tighter packing would have made a difference. I met some people who
>   noted they will not attend, but probably would have attended if it
>   was during the week. Co-locating individual WG interims with RIPEs
>   and NANOGs seems like a useful concept to consider in the future.

ripe/foonog would not appreciate a meeting in schedule conflict.  would
ietf appreciate a foonog meeting scheduled in conflict with and at the
same venue as an ietf meeting?

fwiw, sidr has met adjacent to a few foonogs and it was quite worthwhile
to have the extra ops that brought in.  i wonder if it would be good to
have a sidr meeting adjacent to a security meeting.  oops, we did that
too i think.

> o LIMs will not create a new big funding source for the IETF. We
>   should also right-size our organization for the task at hand. 30,
>   50, or even 100 people could probably be handled as part of the
>   RIPE meeting, and might have been something that the RIPE
>   registration system and agenda could have accommodated, or have
>   someone sponsor a room and leave the rest to participants.

foonog meeting coordination folk are generally very accommodating and
generous.  but beware that, often due to association with organizations
which have a monopoly on scarce intergers and thus pre-crash budget
contraints, foonogs often meet at expensive venues.

randy

---

From: Jari Arkko <[email-masked]>
Date: Tue, 16 Oct 2012 21:04:24 +0300

Randy,

> same venue as an ietf meeting?

Agreed. But here's at least one idea on how to avoid that. Arrange an interim on a RIPE Friday afternoon, after the meetings have ended. Say, 2pm - 8pm. Long day and somewhat shorter WG time. But probably more doable for participants than a full Saturday. Other meetings may have similar scheduling opportunities.

Jari

---

From: Warren Kumari <[email-masked]>
Date: Tue, 16 Oct 2012 14:40:28 -0400

On Oct 15, 2012, at 5:53 PM, Adrian Farrel <[email-masked]> wrote:

> SIDR interim?

Yes -- because it was LIM multiple WGs met at the same time. This meant that OpSec and V6Ops conflicted with SIDR, and so I missed the SIDR meeting.

> Were logistics and resources worth the fee?

Personally I think so -- I have been involved in multiple interims, and the quality of the remote participation was (IMO) much better at the LIM. Having someone else (Meetecho) handle the audio / video was great…

We (OpSec) were hoping to get more of the RIPE attendees (aka, operators) to show up and participate, but:
A: there was a gap between RIPE and the LIM,
B: the existence of the LIM was not announced far enough in advance for most RIPE attendees to change their plans
C: we (OpSec) did a poor job of announcing the fact that we would be meeting, and asking for operators to come participate.

> Should we hold future

Sorry, but figuring this sort of thing out is what we pay *you* for…

(aka, I don't know, to close to call).
W

--
What our ancestors would really be thinking, if they were alive today, is: "Why is it so dark in here?"

    -- (Terry Pratchett, Pyramids)

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 17 Oct 2012 03:31:08 +0000

On Oct 17, 2012, at 4:19 AM, Randy Bush wrote:

> ripe/foonog would not appreciate a meeting in schedule conflict.  would ietf appreciate a foonog meeting scheduled in conflict with and at the same venue as an ietf meeting?

One hopes he was looking for constructive engagement, such as IETF and IRTF do during an IETF meeting. The interspersed meeting is on the same agenda as all of the other meetings, and not on a separate track. The IETF doesn't promise IRTF meeting slots, but if slots are available, it makes them available.

---

From: Meetecho IETF support <[email-masked]>
Date: Thu, 18 Oct 2012 10:30:53 +0200

Hi Joel,

nice work! Just a quick note about Section 3.2, with my Meetecho 
volunteer hat on :-)

In the mentioned section you state:

>    Remote participation was supported by volunteers from meetecho using
>    their own application.  Hotel okura wireless infrastrucuture was used
>    to support the meeting.  An outage of the hotel network was observed
>    during the opsec meeting with the result that remote participation
>    would have been interupted for about 10 minutes had there been any to
>    speak of.

As a matter of fact, our streaming laptops were not attached to the 
Okura Wifi network, but we were provided with wired connectivity per our 
request, and as such remote participants were not affected by the 
outage. This was done exactly to cope with issues like the one you 
mentioned, and this is the very reason why we always insist on asking 
for a wired connection for the streaming laptops.

Cheers,
Alessandro

Il 16/10/2012 17:39, joel jaeggli ha scritto:
>> than simply a

---

From: Spencer Dawkins <[email-masked]>
Date: Fri, 26 Oct 2012 11:00:02 -0500

Thank you, Joel, for putting pen to paper (pixels to glass?) on this, 
and thank you, Jari, Randy, and Warren for sharing your thoughts.

As was pointed out, we've had conversations about LIMs previously. It 
might be worth asking Ray to provide a paragraph or two on history and 
the motivations when some of the previous LIMs were discussed.

My memory of the proposed 2009 Malta LIM (supplemented by Teh Google) 
was that the working groups planning to meet weren't just from one area 
of interest (something like four RAI groups, plus SOFTWIRES and BEHAVE). 
I don't know if that helps or hurts - at a minimum, RIPE/foonog may not 
be the draw for RAI that it was for the groups that met this time.

FWIW, http://trac.tools.ietf.org/2009/jan-large-interim/ says 60 
potential attendees didn't qualify as a large interim :-)

Spencer

---

From: joel jaeggli <[email-masked]>
Date: Fri, 26 Oct 2012 09:15:57 -0700

On 10/26/12 9:00 AM, Spencer Dawkins wrote:
> was that the working groups planning to meet weren't just from one 
> area of interest (something like four RAI groups, plus SOFTWIRES and 
> BEHAVE). I don't know if that helps or hurts - at a minimum, 
> RIPE/foonog may not be the draw for RAI that it was for the groups 
> that met this time.
yeah I didn't have enough useful data on the proposed meeting in malta 
to really commit it to the initial draft.

for one I had the dates wrong in my notes.

> potential attendees didn't qualify as a large interim :-)
that is useful.