Subject: Re: [87attendees] procedural question with remote participation

From: "Olle E. Johansson" <[email-masked]>
Date: Sat, 3 Aug 2013 08:55:04 +0200

2 aug 2013 kl. 16:12 skrev Dan York <[email-masked]>:

> Olle,
> 
> 
> On 8/2/13 12:24 PM, "Olle E. Johansson" <[email-masked]> wrote:
> 
>> In rtcweb we have remote participants that prefer anonymity for a number
>> of reasons.
>> 
>> The question is how this is handled in regards to note well, when they
>> want jabber scribes to relay opinions or proposals to the meeting.
>> 
>> Just a note for the future. I think we should allow anonymous listeners,
>> but should they really be allowed to participate?
> 
> We don't allow anonymous comments at the microphone in face-to-face
> meetings, requiring all people to clearly state their names and have those
> names recorded in the meeting minutes and in the Jabber log.    I don't
> see why we would change this for remote participants.
> 
> In all the years I've been doing Jabber-scribing, I've actually never run
> into a case where a remote attendee requested anonymity.  I would have
> been puzzled like you.  Obviously, anyone can be "anonymous" when joining
> a Jabber chat room and can even make all the comments they want in the
> chat room, but I think the moment they request a relay to the mic and the
> comment goes into the audio record and thereby also the meeting minutes
> then the comment needs to have a name attached to it.
> 
(moving to ietf mailing list)

Absolutely.

Now, should we add an automatic message when someone joins the chat rooms, or a message when meetings begin that all comments made in the chat room is also participation under the note well?

/O

---

From: John C Klensin <[email-masked]>
Date: Sat, 03 Aug 2013 19:25:30 -0400

--On Saturday, August 03, 2013 08:55 +0200 "Olle E. Johansson"
<[email-masked]> wrote:

>...
>>> Just a note for the future. I think we should allow
>>> anonymous listeners, but should they really be allowed to
>>> participate?
>> 
>> We don't allow anonymous comments at the microphone in
>> face-to-face meetings, requiring all people to clearly state
>> their names and have those names recorded in the meeting
>> minutes and in the Jabber log.    I don't see why we would
>> change this for remote participants.
>...
> Now, should we add an automatic message when someone joins the
> chat rooms, or a message when meetings begin that all comments
> made in the chat room is also participation under the note
> well?

Ole, 

First, probably to the "when meetings begin" part, but noting
that someone who gets onto the audio a few minutes late is in
exactly the same situation as someone who walks into the meeting
room a few minutes late -- announcements at the beginning of the
session are ineffective.

But, more generally...

I've said some of this in other contexts but, as a periodic
remote attendee, including being remote for IETF 87, I'd support
a more radical proposal, for example:

We regularize remote participation [1] a bit by doing the
following.  At some level, if remote participants expect to be
treated as serious members of the community, they (we) can
reasonably be expected to behave that way.

 * A mechanism for remote participants should be set up
	and remote participants should be to register.  The
	registration procedure should include the Note Well and
	any other announcement the IETF Trust, IAOC, or IESG
	consider necessary (just like the registration procedure
	for f2f attendance).
	
 * In the hope of increased equity, lowered overall
	registration fees, and consequently more access to IETF
	participation by a broader and more diverse community,
	the IAOC should establish a target/ recommended
	registration fee for remote participants.  That fee
	should reflect the portion of the registration fee that
	is not specifically associated with meeting expenses
	(i.e., I don't believe that remote participants should
	be supporting anyone's cookies other than their own).  
	
 * In the interest of maximum participation and inclusion
	of people are aren't attending f2f for economic reasons,
	I think we should treat the registration fee as
	voluntary, with people contributing all or part of it as
	they consider possible.  No questions asked and no
	special waiver procedures.  On the other hand,
	participation without registration should be considered
	as being in extremely bad taste or worse, on a par with
	violations of the IPR disclosure rules.
	
 * I don't see a practical and non-obtrusive way to
	enforce registration, i.e., preventing anyone
	unregistered from speaking, modulo the "bad taste"
	comment above.  But we rarely inspect badges before
	letting people stand in a microphone line either.

In return, the IETF generally (and particularly people in the
room) needs to commit to a level of seriousness about remote
participation that has not consistently been in evidence.  In
particular:

	* Remote participants should have as much access to mic
	lines and the ability to participate in discussions as
	those who are present in the room.   That includes
	recognizing that, if there is an audio lag and it takes
	a few moments to type in a question or comment, some
	flexibility about "the comment queue is closed" may have
	to be in order.  For some sessions, it might require
	doing what ICANN has started doing (at least sometimes),
	which is treating the remote participants as a separate
	mic queue rather than expecting the Jabber scribe (or
	remote participant messenger/ channeler) to get at the
	end of whatever line is most convenient.
	
	* It is really, really, important that those speaking,
	even if they happen to be sitting at the chair's table,
	clearly and carefully identify themselves.  Last week,
	there were a few rooms in which the audio was, to put it
	very politely, a little marginal.  That happens.  But,
	when it combines with people mumbling their names or
	saying them very quickly, the result is as little
	speaker identification as would have been the case if
	the name hadn't been used as all.  In addition, some of
	us suffer from the disability of not being able to keep
	track of unfamiliar voices while juggling a few decks of
	slides, a jabber session, audio, and so on.   "I
	identified myself 10 minutes ago" is not generally
	adequate.
	
	* On several occasions this week, slides were uploaded
	on a just-in-time basis (or an hour or so after that).
	The problem was aggravated by the meeting materials and
	tools agenda pages apparently either lacking "Expires"
	headers or having them set rather long as compared to
	in-meeting changes.  It seems to me that, if the IETF is
	serious about remote participation, pages that reflect
	very volatile material should have "expires" headers set
	to appropriately short intervals sometimes Sunday and be
	left that way until the end of the last session on
	Friday _and_ that it be clearly explained to people
	whose comments depend on visual materials that failure
	to get those materials uploaded before the meeting
	(except in the most unusual of situations) are badly
	hurting the IETF's ability to be open and inclusive.
	The only alternative I can think of to getting really
	serious about having material uploaded early is to have
	a high-reliability and decent resolution video feed of
	the slides or the screen on which they are being
	projected.  And that might still be inadequate for some
	remote participants.

Or we can decide that real participation in the IETF requires
that people be in the room, that remote participants are
involved on a "what you get is what you get" basis, and we stop
pretending otherwise.  For many reasons, I'm not enthused about
that idea, but the things that I, and others, are suggesting and
asking for will cost money and require some changes in the
ordinary way of doing things and it is only fair to mention the
alternative and suggest that it be explicitly considered.

best,
   john

[1] I do mean "participants" here.  I really don't care what the
lurkers do, or how anonymous they are as long as they remain
lurkers.  I object to anonymous participation in any standards
process, including the IETF one, for reasons that have been
discussed repeatedly on this list.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 04 Aug 2013 07:27:00 -0400

I attended meetings 36 through 62 in-person, missing about 1 in 4. I've never
attended a meeting in asia-pacific, as about half were paid out of my own
pocket,   That was in the days of multicast, and I never got an mbone tunnel
working, although Joe Abley and I once *saw* them in tcpdump go past us on
the ethernet at ISC, but not get relayed through our tunnels.
Between 63 and 80, I managed to attend 1 in 5, and this one is the first I've
missed since 80.  I missed it because, my WG didn't need to meet, I had no
money, and it abuts an important long weekend. (I got to walk out in 3min)
I have generally good experiences with our remote participation.
Some problems recently:
1) the audio feed started at exactly 9:00 on Monday  A problem if you need
   to check your equipment. I also interrupted at exactly the start time
   of the session, and it took me 20-30s to realize it, and up-arrow-return.

2) Slide decks were late.  PPT(x) files are annoying and inconsiderate.
   Consolidated slide decks are wonderful, even if the agenda order is
   changed.

3) audio delay makes hums via jabber meaningless.

    > and remote participants should be to register.  The

+1. And I would pay a fee.

    > those who are present in the room.   That includes

Yes... but I think it might be worth recognizing that in badly run meetings,
access to the mic is a problem to those in the meeting too!  Multiple roaming
wireless mics, and mic-control from the chair would help here.
I.e. let's use the technology for mic-line-up for everyone, local and
remote.

    > clearly and carefully identify themselves.

+1

    > on a just-in-time basis (or an hour or so after that).

Agreed.  I'd like to have this as a very clear IETF-wide policy.
No slides 1 week before hand, no time allocation.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: John C Klensin <[email-masked]>
Date: Sun, 04 Aug 2013 14:20:02 -0400

--On Sunday, August 04, 2013 07:27 -0400 Michael Richardson
<[email-masked]> wrote:

> Agreed.  I'd like to have this as a very clear IETF-wide
> policy. No slides 1 week before hand, no time allocation.

I had two different WG chairs (from two different WGs) tell me
this week that their WGs really needed the presentations and
discussion to move forward and they therefore couldn't do
anything other than let things progress when they didn't get the
slides and get them posted before the session started.  This is
part of what I mean by the community not [yet] taking remote
participation seriously.  If having the slides in advance is as
important to remote participants as Michael and I believe, then
the community has to decide that late slides are simply
unacceptable behavior except in the most unusual circumstances,
with "unacceptable" being viewed at a level that justifies
finding replacements for document authors and even WG chairs.

I also note that the 1 week cutoff that Michael suggests would,
in most cases, eliminate "had no choice without impeding WG
progress" as an excuse.  A week in advance of the meeting, there
should be time, if necessary to find someone else to organize
the presentation or discussion (and to prepare and post late
slides that are still posted before the meeting if needed).  If
it is necessary to go ahead without the slides, it is time to
get a warning to that effect and maybe an outline of the issues
to be discussed into the agenda.    If the WG's position is that
slides 12 or 24 hours before the WG's session are acceptable,
then the odds are high that one glitch or another will trigger a
"well, there are no slides posted but they are available in the
room and the discussion is important" decision.

Again, I think the real question is whether we, as a community,
are serious about effective remote participation; serious enough
to back a WG chair who calls off a presentation or replaces a
document author, or an AD who replaces a WG chair, for not
getting with the program.

best,
    john

---

From: Ted Lemon <[email-masked]>
Date: Sun, 4 Aug 2013 18:54:18 +0000

While I think getting slides in on time is great for a lot of reasons, reading the slides early isn't that important.   What is important is that remote people see the slides at the same time as local people.   For that, it seems to me that Meetecho support does exactly what is needed.   You just follow the slideshow online, along with the audio.

Of course, in order to have Meetecho support, you need the slides in advance of the meeting, preferably far enough in advance that it doesn't create a massive hassle for the Meetecho team. But by making Meetecho the place where the slides are presented, you ensure that everybody is on an equal footing, without engaging in punitive behavior.

The main reason to want to see the slides in advance is so that the working group chairs can evaluate them to see if they will actually be a good use of time.   But that's completely orthogonal to the remote participation issue.

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 4 Aug 2013 16:10:58 -0400

OK, I'll bite.  Why do you and Michael believe you need to have the slides 1 week in advance?

You have the agenda and drafts 2 weeks in advance.  The slides aren't normative.  Even when they're not about a draft in particular, the slides are not self-standing documents.  They're merely to help with discussion.

Not getting the slides at all is a different matter - but 7 days in advance is counter-productive.  They should be as up-to-date as practical, to take into account mailing list discussions. [or at least that's how I justify my same-day, ultra-fresh slides]

If you need to have them on the website 7 days in advance, you really need to get a faster Internet connection. ;)

-hadriel

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 05 Aug 2013 08:20:35 +1200

> The main reason to want to see the slides in advance is so that the working group chairs can evaluate them to see if they will actually be a good use of time.   But that's completely orthogonal to…

For remote attendees, there is a distinct advantage in having time to
download & store slides in advance. There are still plenty of places
where real-time bandwidth is an issue and audio and jabber may be
all you can get.

There is another equally important reason for having them well in advance,
for both on-site and remote attendees: so that participants can review
them in advance, decide which of several clashing sessions to attend, and
even prepare questions. This applies even if the slides summarise a draft
that was available two or three weeks in advance. Things are often
expressed differently in the slides.

Finally: a deadline one week before the meeting is no harder to meet
than one minute before the meeting. If it's a zero-tolerance deadline,
people will meet it.

   Brian

---

From: Scott Brim <[email-masked]>
Date: Sun, 4 Aug 2013 16:25:42 -0400

I'm less concerned about having slides than having the issues that need
discussion clear. An agenda of documents and issues tells potential
participants what they need. Slides are needed if and only if there is no
document.

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 4 Aug 2013 17:27:37 -0400

> expressed differently in the slides.

By that logic, I should also write out a script of what I'm going to say and post that 1 week in advance, along with a youtube recording of me presenting it.  At least for me, most of the content/meat is verbal, not pictorial.  The slides aren't a script of text I'm reading out loud.

As for conflicts, I agree they truly do suck, but I wouldn't want you to pick/skip mine based on my poor slide-making abilities.  In the worst case, you can review the recordings afterward and email comments/questions/flames to the list.

> than one minute before the meeting. If it's a zero-tolerance deadline,

This would just encourage more presentations to be graphical reproductions of the drafts.  I'm cool with having those types of slides as well, even well in advance - but only separate from the ones I present in a meeting.  We should be encouraging email discussions to take place before the physical meeting hour... and I don't want set-in-stone slides to skip things introduced by, cover ground already covered by, or made moot by, those discussions.  I don't know about other folks, but I really have changed slide content based on mailing list and in-person discussions before the meeting slot.  This even happened to me just this past week at IETF 87.

These aren't presentations of academic papers, corporate position statements, or tutorial classes.  Real-time content and discussion is good for WG meetings. 

What *would* be good to have 7 days or more in advance are the Technical and O&A Plenary slides.  They shouldn't be changing, afaict.  And that way we can figure out if we can have those nights free for other things, or if it's worth going to the Plenaries instead.  But I assume those slides already are made available well in advance. (right?)

-hadriel

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 04 Aug 2013 23:21:05 +0100

On 08/04/2013 09:20 PM, Brian E Carpenter wrote:
> than one minute before the meeting.

Disagree. I often end up updating stuff late in the day and that
should continue to be fine.

Secondarily, its my impression that people are as usual taking all
this too seriously. If we cumulatively do our best and if that works
ok, then overall, we're ok.

Improving on current practice is a fine thing too. But claiming or
implying that the imperfections of current practice are disastrous
for the IETF or for all remote participation seems overblown to me.
And only something potentially disastrous ought imply even considering
a zero-tolerance anything in a volunteer organisation.

So I'd say working on ways to make remote participation better
while not making f2f participation more of a pain would be the way
to go.

S.

---

From: Melinda Shore <[email-masked]>
Date: Sun, 04 Aug 2013 14:37:35 -0800

We're all different, and for my purposes, in all honesty, having
slides unavailable until 45 seconds before a session start hasn't
been an issue as a remote participant.  It's definitely aggravating
as a chair, though, since we need to get those uploaded via the
meeting materials manager.  Overall, though, I'd say my feelings
about this are substantially similar to Stephen Farrel's:

And it's unclear to me that having slides available a week before
a session would improve remote participation much.

Melinda

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 4 Aug 2013 18:41:47 -0400

> room a few minutes late -- announcements at the beginning of the

Jabber appears to have some way of setting a banner/announcement thing that shows up when you first join a jabber session, because I've seen such a thing on occasion.  I don't know if it's defined in some standard way in XMPP or a proprietary extension.  But assuming it's either standard or defacto and popular, we could put the NOTE WELL in it (or a URL to a NOTE WELL).

Likewise for the IETF web pages with the audio links, so that you see the NOTE WELL before clicking the audio link.  Or even have an annoying pop-up if you prefer. (ugh)

> 	consider necessary (just like the registration procedure

Sure - to *participate*, i.e. have a chance at the mic.  Not to listen/watch/read. 

> 	violations of the IPR disclosure rules.

I don't agree - I go to the meetings physically, but I *want* remote people to participate.  It's to everyone's benefit that they do so, including the physical attendees.  I don't want to charge them for it.  Making them register (for free) is fine, but don't make them pay money.  Don't even make them feel guilty.  The people who can afford the time and money to go to the physical meetings still get their money's worth.

> 	letting people stand in a microphone line either.

Sure there is.  Have the current "[wg-name]@jabber.ietf.org" jabber rooms be for open access lurking, from any XMPP domain, and not allow microphone representation in the WG by simply having jabber scribes ignore such requests in those rooms.  And have separate rooms that require registering, like "[wg-name]@members.ietf.org" or whatever, where you have to have a registered account on 'members.ietf.org'.  I assume XMPP servers support such a policy?  It would be a free account, but require filling out the blue-sheet type information, verified email address, etc.  Or maybe even have it all in the same current jabber room but only accounts with "members.ietf.org" as the domain portion are represented at the mic by the jabber scribes.

> 	end of whatever line is most convenient.

Do you find this is an actual problem in WG meetings?  My experience has been the opposite.  The WG chairs always seem to let jabber participants get their 2 cents in, even if the queue line has been capped... especially if there's lag.

> 	identified myself 10 minutes ago" is not generally

Do you find this is an actual problem in WG meetings?  Are the jabber scribes not able to tell you who is at the mic if you ask them?  People have forgotten to state their names in WGs I go to, but the scribes have been fairly vocal about reminding folks to do it.

-hadriel

---

From: Spencer Dawkins <[email-masked]>
Date: Sun, 04 Aug 2013 18:10:24 -0500

> If you need to have them on the website 7 days in advance, you really need to get a faster Internet connection. ;)

I'm a TSV AD, but I'm sending this note wearing no hat (someone last 
week asked me why I wasn't wearing a cowboy hat if I was from Texas - 
no, not even a cowboy hat).

I read through the discussion on this, and I'm only responding to 
Hadriel because his post was the last one I saw before replying. Thank 
you all for sharing your thoughts.

YMMV ("Your Mileage May Vary"), but I have been sponsored for several 
years by a company that sends a sizable number of folks to IETF who are 
not native English speakers. Having slides early helps non-native 
English speakers (I believe I've heard that some slide decks are 
translated into other languages, although I wouldn't know, because I 
read the slides in English).

After his first IETF (Paris/63), Fuyou Maio said to me, "understanding 
spoken English is the short board in the water barrel" (the idea being 
that your effectiveness at the IETF is limited by your ability to 
quickly parse spoken English). The folks I talk to get plenty of chances 
to translate spoken English during Q&A, and don't need additional 
practice translating the presentations in real time. Yes, I know people 
say things that aren't on their slides, but if what's on their slides 
doesn't help other people understand what they are saying, they probably 
shouldn't be using those slides.

In the mid-2000s, I remember an admonition for chairs to write out the 
questions the chairs are taking a hum on, to accommodate non-native 
English speakers (and to write out all the questions before taking the 
first hum, to accommodate anyone who agrees with the second choice but 
prefer the fourth choice when they hear it after humming).

I'm having a hard time making the "a week early or you don't present" 
case for slide cutoffs, because we DO talk during the meeting week - and 
in groups RTCWeb, with a Thursday slot and a Friday slot, we had time to 
talk a lot. If the cutoff was for presentations of new individual 
drafts, that's a different question, so there might be some way to make 
non-Procrustean improvements(*).

I agree with the "chairs looking at slides for sanity" point. I'm 
remembering more than one working group where we chairs got 
presentations that included about a slide per minute for the time 
allocated to the topic - noticing that even one day before saved us from 
the ever-popular "we can't talk about this presentation because we don't 
have time" moment.

During IETF 87, I had reason to consult the proceedings for the 
non-workgroup-forming RUTS BOF ("Requirements for Unicast 
Transport/Sessions" at IETF 43, minutes at 
http://www.ietf.org/proceedings/43/43rd-ietf-98dec-142.html#TopOfPage) 
<http://www.ietf.org/proceedings/43/43rd-ietf-98dec-142.html#TopOfPage>. 
This was the applications-focused wishlist for transport from 1998, when 
COPS, RADIUS, L2TP, HTTP-NG, SIP, NFSv4, SS7, IP Telephony and BGP4 were 
all trying to figure out whether they needed to (continue to, in some 
cases) rely on TCP for transport, or do "something else". I'm 
remembering that there were slides, and I would love to have them to 
refer to, but *none* of the slide decks made it into the proceedings. 
That was pre-Meeting Materials page, but even my experience with the 
Meeting Materials page was that it's easier for slide decks arriving 
late to go missing than for slide decks that arrived early.

As I reminded myself while starting to present v4 of the chair slides in 
TSVAREA and realizing that what Martin was projecting was v1 (only a day 
older), getting slidesets nailed down early limits the number of times 
when you're surprised at what's being projected.

I love consolidated slide decks. I bet anyone does, whose laptop 
blue-screened while hooking up to a projector in the late 1990s. Nothing 
good happens during transitions, whether switching laptops or switching 
presentations :-)

None of this should be taken as disagreement with proposals to 
experiment with room shapes, whiteboards, , etc. that I heard last week.

None of this should be taken as evidence of love for an unbroken stream 
of presentations of drafts that aren't tied to issues discussed on 
mailing lists, or as disagreement with the idea that presentations 
aren't always the best way to communicate at the IETF.

Thanks,

Spencer, who also might disagree with some of this when he wakes up at a 
normal hour ... and that hasn't happened yet ...

(*) from http://www.thefreedictionary.com/Procrustean, "producing or 
designed to produce strict conformity by ruthless or arbitrary means".

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 4 Aug 2013 21:36:22 -0400

Regarding the need for presentations early to get them translated, and the "non-Procrustean"[1] improvement of having cutoffs for presentations of new drafts: new drafts are still submitted 2 weeks in advance, and ISTM that a real non-Procrustean tactic would be to let the WG chairs do their jobs.  If you've got a 40 page slide-deck chock full of text, giving a detailed tutorial on some mechanism, it's a different situation from the norm. (and I'd argue you're probably doing it wrong, but ymmv)  But those appear to be the exceptions, not the rule; and WG chairs can handle push-back for exceptions if they need to.  We don't have to create new draconian rules.

But for the general case, the truth is that Fuyou Maio is right - you really do need to be able to parse English quickly to truly participate effectively in an IETF physical meeting.  And you need to be reasonably swift in either reading it, or following the speaker's words.  It's not nice to say, but it's the truth.  Real-time direct human communication is why we have the physical meetings to begin with, instead of only mailing lists and virtual meetings. (and for cross-wg-pollination, and for cookies)

The good news is the mailing lists and drafts themselves are in plain ascii which should make language translation software easier to use, and the physical meetings are voice/video/jabber recorded so you can get them translated afterwards to listen/watch/read, and once you do you can always raise objections/issues in email afterwards and try to reverse any decisions made in physical meetings.  That's better than any other international standards body I've ever participated in. (IEEE, 3GPP, ETSI, ATIS)  Some have a more strict submissions in-advance policy, but even for them their physical meetings require high-level English abilities to participate effectively, in practice.  I don't see a realistic alternative to that, while still getting things accomplished in a timely manner.

It's easy to say those things since I'm a native English speaker [2], and not a nice concept in general... but if we're honest with ourselves I think we have to recognize the unvarnished truth.  Obviously there are exceptions - the 40 page slide-deck full of text is an exception.  But those appear to be uncommon cases, afaict.

-hadriel
[1] It's ironic you use the word "Procrustean" in an email about non-native English speakers needing translation.  If you'd asked me what the word meant, I'd have guessed it either meant those who enjoyed the edges of bread or pizza, or those who advocate Earth plate tectonics theory.

[2] well, technically English is not my native language, but I learned it at a young enough age to cover that up... mostly.

On Aug 4, 2013, at 7:10 PM, Spencer Dawkins <[email-masked]> wrote:

> I'm a TSV AD, but I'm sending this note wearing no hat (someone last week asked me why I wasn't wearing a cowboy hat if I was from Texas - no, not even a cowboy hat).
> 
> I read through the discussion on this, and I'm only responding to Hadriel because his post was the last one I saw before replying. Thank you all for sharing your thoughts.
> 
> YMMV ("Your Mileage May Vary"), but I have been sponsored for several years by a company that sends a sizable number of folks to IETF who are not native English speakers. Having slides early helps non-native English speakers (I believe I've heard that some slide decks are translated into other languages, although I wouldn't know, because I read the slides in English).
> 
> After his first IETF (Paris/63), Fuyou Maio said to me, "understanding spoken English is the short board in the water barrel" (the idea being that your effectiveness at the IETF is limited by your ability to quickly parse spoken English). The folks I talk to get plenty of chances to translate spoken English during Q&A, and don't need additional practice translating the presentations in real time. Yes, I know people say things that aren't on their slides, but if what's on their slides doesn't help other people understand what they are saying, they probably shouldn't be using those slides.
> 
> In the mid-2000s, I remember an admonition for chairs to write out the questions the chairs are taking a hum on, to accommodate non-native English speakers (and to write out all the questions before taking the first hum, to accommodate anyone who agrees with the second choice but prefer the fourth choice when they hear it after humming).
> 
> I'm having a hard time making the "a week early or you don't present" case for slide cutoffs, because we DO talk during the meeting week - and in groups RTCWeb, with a Thursday slot and a Friday slot, we had time to talk a lot. If the cutoff was for presentations of new individual drafts, that's a different question, so there might be some way to make non-Procrustean improvements(*).
> 
> I agree with the "chairs looking at slides for sanity" point. I'm remembering more than one working group where we chairs got presentations that included about a slide per minute for the time allocated to the topic - noticing that even one day before saved us from the ever-popular "we can't talk about this presentation because we don't have time" moment.
> 
> During IETF 87, I had reason to consult the proceedings for the non-workgroup-forming RUTS BOF ("Requirements for Unicast Transport/Sessions" at IETF 43, minutes at http://www.ietf.org/proceedings/43/43rd-ietf-98dec-142.html#TopOfPage). This was the applications-focused wishlist for transport from 1998, when COPS, RADIUS, L2TP, HTTP-NG, SIP, NFSv4, SS7, IP Telephony and BGP4 were all trying to figure out whether they needed to (continue to, in some cases) rely on TCP for transport, or do "something else". I'm remembering that there were slides, and I would love to have them to refer to, but *none* of the slide decks made it into the proceedings. That was pre-Meeting Materials page, but even my experience with the Meeting Materials page was that it's easier for slide decks arriving late to go missing than for slide decks that arrived early.
> 
> As I reminded myself while starting to present v4 of the chair slides in TSVAREA and realizing that what Martin was projecting was v1 (only a day older), getting slidesets nailed down early limits the number of times when you're surprised at what's being projected.
> 
> I love consolidated slide decks. I bet anyone does, whose laptop blue-screened while hooking up to a projector in the late 1990s. Nothing good happens during transitions, whether switching laptops or switching presentations :-)
> 
> None of this should be taken as disagreement with proposals to experiment with room shapes, whiteboards, , etc. that I heard last week.
> 
> None of this should be taken as evidence of love for an unbroken stream of presentations of drafts that aren't tied to issues discussed on mailing lists, or as disagreement with the idea that presentations aren't always the best way to communicate at the IETF.                                                                             
> 
> Thanks,
> 
> Spencer, who also might disagree with some of this when he wakes up at a normal hour ... and that hasn't happened yet ...
> 
> (*) from http://www.thefreedictionary.com/Procrustean, "producing or designed to produce strict conformity by ruthless or arbitrary means".

---

From: Spencer Dawkins <[email-masked]>
Date: Sun, 04 Aug 2013 21:35:40 -0500

> Regarding the need for presentations early to get them translated, and the "non-Procrustean"[1] improvement of having cutoffs for presentations of new drafts: new drafts are still submitted 2 weeks…

Oh, I wouldn't dream of having rules about that. What I was trying to 
say in my not-enough-sleep-last-week way was that I was imagining so 
many justifiable exceptions (chair slides, on-the-fly hums, reports from 
design teams and hallway conversations the day before) that having rules 
wouldn't help, so, agreed.

> But for the general case, the truth is that Fuyou Maio is right - you really do need to be able to parse English quickly to truly participate effectively in an IETF physical meeting.  And you need…

Right, but Fuyou was talking about *spoken* English being more 
challenging than written English (if you can't *read* English fairly 
quickly, drafts and mailing lists are impenetrable, and you're done in 
the IETF). I'm told that it's easier for non-native English speakers to 
read slides than to parse spoken English for anyone who talks faster 
than I do.

I'm not saying that should override other considerations. I was 
responding to a question several people asked, if anyone would benefit 
from having slides early (for the purposes of this e-mail, 24 hours 
early would be plenty early enough). Other people provided other 
answers, and I hadn't seen that answer go past.

Thanks,

Spencer

p.s. I DID footnote "Procrustean" with a definition, but that's 
perfectly reasonable to point to as "not helpful for non-native English 
speakers" - please feel free to keep me relatively honest.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 05 Aug 2013 02:37:27 -0600

On 8/4/13 4:41 PM, Hadriel Kaplan wrote:
>> First, probably to the "when meetings begin" part, but noting that
>> someone who gets onto the audio a few minutes late is in exactly
>> the same situation as someone who walks into the meeting room a few
>> minutes late -- announcements at the beginning of the session are
>> ineffective.
> 
> Jabber appears to have some way of setting a banner/announcement
> thing that shows up when you first join a jabber session, because
> I've seen such a thing on occasion.  I don't know if it's defined in
> some standard way in XMPP or a proprietary extension.  But assuming
> it's either standard or defacto and popular, we could put the NOTE
> WELL in it (or a URL to a NOTE WELL).
> 
> Likewise for the IETF web pages with the audio links, so that you see
> the NOTE WELL before clicking the audio link.  Or even have an
> annoying pop-up if you prefer. (ugh)

I don't want to promise too much, but in time for Vancouver I'll
probably finish some code that sends you all sorts of helpful
information when you join the jabber room. There is a standardized "room
subject" message but not all IM clients show you that, so I plan to have
it sent as a one-off message when you join the chatroom.

Further discussion, if any, on the [email-masked] list.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Dave Cridland <[email-masked]>
Date: Mon, 5 Aug 2013 10:07:28 +0100

> Further discussion, if any, on the [email-masked] list.

I'd be happy to help, and will meander toward that list.

One thing for this August Body to consider - we can (quite) easily have new
occupants to the room not be "participants" - in the XEP-0045 sense of the
term. This would mean new occupants would lack voice, and be unable to add
to the discussion - until they've jumped through some hoops.

One such hoop might be acknowledging the (privately sent) Note Well message
(thus equating XEP-0045 Participant with IETF Participant to some degree).
Another might be that we tell them to go away if their XEP-0054 vCard
doesn't include sufficient detail (like their full name and email address,
for example), taking us a step toward remote registration.

Dave.

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 05 Aug 2013 10:28:48 +0100

> for example), taking us a step toward remote registration.

I hope folks who invest effort in tooling try to make it all
easier and not harder. Right now we don't have good tools that
allow remote folks to easily provide "live" input (and maybe
that's just because its a hard problem). So I'd say we should
keep trying to make that better and not worry yet about how to
control abuse of what's not currently usable.

On 08/04/2013 11:41 PM, Hadriel Kaplan wrote:
> And have separate rooms that require registering, like
> "[wg-name]@members.ietf.org" or whatever,

We don't have, nor (I believe) do we want, "members." And we do
want good technical input regardless of source. About the only
reason to try control that via registration is due to patent
nonsense. That is (unfortunately) a real reason, and we do have
to take it into account, but please let's all bear in mind that
99% of those patents are total crap (regardless of country afaik)
and let's not be driven by the stupidity but rather let's put
that in its proper place as a regrettable cost of being open.

Sorry to go on about that, but I don't think onerous registration
schemes are really needed to e.g. do floor control. And since the
former ("registration" stuff) is easy, and the latter (esp. with
remote audio input in our environment) is not, we might easily end
up doing the easy thing, and making it all worse.

Cheers,
S.

---

From: SM <[email-masked]>
Date: Mon, 05 Aug 2013 02:26:57 -0700

>slides 1 week in advance?

One generation's bad behavior becomes the next generation's best 
practice.  It would be appreciated if those slides could be made 
available in advance.

>You have the agenda and drafts 2 weeks in advance.  The slides 
>aren't normative.  Even

I do not have the agenda two weeks in advance.

Nowadays, there are "if time permits" slots in addition to A.O.B.

What is the meaning of "normative" in the above?

>If you need to have them on the website 7 days in advance, you 
>really need to get a faster Internet connection. ;)

Ok. :-)

At 14:27 04-08-2013, Hadriel Kaplan wrote:
>What *would* be good to have 7 days or more in advance are the 
>Technical and O&A Plenary slides.  They shouldn't be changing, 
>afaict.  And that way we can figure out if we can have those nights 
>free for other things, or if it's worth going to the Plenaries 
>instead.  But I assume those slides already are made available well 
>in advance. (right?)

The Technical and other Plenary slides are not made available well in 
advance.  Someone asked the following question:

   "Does she have a clue that she just asked for feedback from an audience
    that can't see the link she put on the screen?"

There was a discussion about beer from the tap after that.  Please 
open a new thread if you would like to discuss about that. :-)

>a zero-tolerance anything in a volunteer organisation.

It is after all a volunteer organisation.  I hope that people were 
not surprised that I did not ask for the Spice Girls session to be cancelled.

At 15:41 04-08-2013, Hadriel Kaplan wrote:
>Do you find this is an actual problem in WG meetings?  Are the 
>jabber scribes not able to tell you who is at the mic if you ask 
>them?  People have forgotten to state their names

Some of the Jabber scribes are not able to tell me who are at the 
microphone when I ask them.  If it was my decision to make (and it is 
not), the Jabber scribe would be allowed to comment at the microphone 
even after the microphone line is capped.  A person can always argue 
that it is an arbitrary decision.  :-)

As an off-topic comment, it's not because the Meetecho people are 
nice that one should expect them to act as Jabber scribes.

At 18:36 04-08-2013, Hadriel Kaplan wrote:
>But for the general case, the truth is that Fuyou Maio is right - 
>you really do need to be able to parse English quickly to truly 
>participate effectively in an IETF physical meeting.  And you need 
>to be reasonably swift in either reading it, or following the 
>speaker's words.  It's not nice to
>  say, but it's the truth.  Real-time direct human communication is 
> why we have the physical meetings to begin with, instead of only 
> mailing lists and virtual meetings. (and for cross-wg-pollination, 
> and for cookies)

Yes.

Some sessions are easy to follow.  For example, I read some slides 
posted a few days before and I had an idea of what would be 
discussed.  I looked for the slides for a BoF as it was not clear to 
me what one of these items on the agenda was about.  The slides were 
not available.  I didn't bother asking about them.  The correct 
question would have been about the item on the agenda instead of the slides.

Regards,
-sm

---

From: Scott Brim <[email-masked]>
Date: Mon, 5 Aug 2013 05:38:55 -0400

challenging than written English (if you can't *read* English fairly
quickly, drafts and mailing lists are impenetrable, and you're done in the
IETF). I'm told that it's easier for non-native English speakers to read
slides than to parse spoken English for anyone who talks faster than I do.

Aside: this is yet another reason why a really thorough jabber scribe is
useful.

swb

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 5 Aug 2013 07:31:55 -0400

> control abuse of what's not currently usable.

Yup, afaict we were doing ok until IETF 87... but at least one anonymous jabber participant (named "Guest") did remotely speak multiple times at the mic on one of the RAI working group sessions this past week (at RTCWEB if I recall).  I was personally ok with it, but it was awkward.  If folks feel it's inappropriate, then we need something else.  I'd be ok with just having jabber scribes ignore anonymous participants.

> We don't have, nor (I believe) do we want, "members."

Yeah, "members.ietf.org" was a poor choice of domain name.  I wasn't making a formal proposal - just thinking out loud.  I'd be happier if the tools team figures out something simpler and less onerous anyway.  I was just noting it's not an impossible task to accomplish, some way or other.

> And we do

Amen to that!

-hadriel

---

From: Scott Brim <[email-masked]>
Date: Mon, 05 Aug 2013 07:43:46 -0400

> Yup, afaict we were doing ok until IETF 87... but at least one anonymous jabber participant (named "Guest") did remotely speak multiple times at the mic on one of the RAI working group sessions thi…

It's inappropriate. Tell them they need to provide at least a name.

---

From: Yoav Nir <[email-masked]>
Date: Mon, 5 Aug 2013 11:51:36 +0000

> It's inappropriate. Tell them they need to provide at least a name.

Would it be better if they were called "Emma" instead of "Guest" ?

---

From: Scott Brim <[email-masked]>
Date: Mon, 05 Aug 2013 07:55:41 -0400

> Would it be better if they were called "Emma" instead of "Guest" ?

Yes.  Ask lawyers.

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 5 Aug 2013 08:10:56 -0400

On Aug 5, 2013, at 5:26 AM, SM <[email-masked]> wrote:

>> You have the agenda and drafts 2 weeks in advance.  The slides aren't normative.  Even
> 

Huh.  Sounds like a WG Chair problem.  I believe draft agendas are due 2 weeks in advance, and final agendas due 1 week in advance.  Or at least that was the excuse I was given when I've been denied requests to add stuff to WG agendas on previous occasions - the Chairs told me I was asking too late. (as they should have)

> What is the meaning of "normative" in the above?

I was trying to be cute by using the term from our drafts/RFCs, as in normative references vs. informative, or the document's RFC2119-type text is normative while examples are informative.  I meant the slides are just helpful guides, like pictorial examples, vs. the draft itself which is the real proposal.  I guess the joke flopped. :(

> Some of the Jabber scribes are not able to tell me who are at the microphone when I ask them.  If it was my decision to make (and it is not), the Jabber scribe would be allowed to comment at the microphone even after the microphone line is capped.  A person can always argue that it is an arbitrary decision.  :-)

Again, this sounds like a WG Chair problem.  If you find that happening, send a private email to that working group's Chair(s) reminding them of the jabber folks.  And if they ignore you, then send a private email to the Area Director(s).  WG Chairs and ADs are generally decent human beings, at least in private.  ;)

> As an off-topic comment, it's not because the Meetecho people are nice that one should expect them to act as Jabber scribes.

I don't expect Meetecho people to jabber scribe.  I expect other folks in the room to volunteer.  It's a heck of a lot easier/better than being the minute taker.  That's why I volunteer for being a scribe - to avoid being a minute taker. :)

-hadriel

---

From: Michael Richardson <[email-masked]>
Date: Mon, 05 Aug 2013 10:17:17 -0400

Spencer Dawkins <[email-masked]> quoted Hadiel really poorly,
    > slides 1 week in advance?

1) As a WG chair, I'd like to see the slides from a (new) presenter in
   advance to make sure that the *presentation* is on topic, there aren't
   too many slides, and that ideally, it is a request for discussion rather
   than a presentation.
   That's where the deadline comes from.  I don't suggest that

2) As a remote participate, I'd much rather have consolidate slides.
   That requires a bit of time/effort on the part of the chairs.

3) As an open-standards body, I believe it is hypocritical for us to be
   posting slides in a vendor proprietary format or one from a "standards"
   body that seems to have all of features we dislike (like pay to vote).
   (I'm okay with the secretariat doing conversion, but they are not instant)
   (And,open source tools running on open platforms sometimes do not
   render the slides as intended due to lack of a font or a other thing)

    > Not getting the slides at all is a different matter - but 7 days in
    > advance is counter-productive.  They should be as up-to-date as
    > practical, to take into account mailing list discussions. [or at least
    > that's how I justify my same-day, ultra-fresh slides]

I distinquish between rev-00 of slides and rev-09.  I don't have a problem
with updates to the slides, assuming you can find the "Export as PDF"
button. It would be best if you didn't create new slides due to numbering
changes.
I also understand that ADs running area meeting aren't going to have status
updates 7 days in advance, nor do I expect them to.

I had not considered Spencer's point about translation, and frankly it is a
really really really good point.

    > experiment with
    > room shapes, whiteboards, , etc. that I heard last week.

+1

--
Michael Richardson <[email-masked]>, Sandelman Software Works

---

From: John C Klensin <[email-masked]>
Date: Mon, 05 Aug 2013 13:38:54 -0400

Hi.

I seem to have missed a lot of traffic since getting a few
responses yesterday.  I think the reasons why slides should be
available well in advance of the meeting have been covered well
by others.  And, as others have suggested, I'm willing to see
updates to those slides if things change in the hours leading
up to the meeting, but strongly prefer that those updates come
as new alides with update-type "numbers" or other
identification rather than new decks.  In other words, if a
deck is posted in advance with four slides numbered 1, 2, 3,
and 4, and additional information is needed for 3, I'd prefer
to see the updated deck consist of slides 1, 2, 3, 3a, 3b, 4 or
1, 2, 3a, 3b, 3c, 4, rather than 1, 2, 3, 4, 5, 6.  I also
prefer consolidated decks but, if WG chairs find that too
difficult, I'm happy to do my own consolidating if everyting is
available enough in advance for me to do sol

Almost independent of the above, the idea that one should just
watch the slides on Meetecho implies that Meetecho is
available in every session (it isn't) and that everything
works.  In addition, they either need the slides in advance or
need to be able to broadcast real-time video at a resolution
that makes the slides readable.  The latter was not the case
last week in some of the sessions in which Meetecho was
transmitting the slides sometimes due in part to interesting
speaker-training issues.

The reasons to discourage anonymity aren't just "patent
nonsense" (although that should be sufficient and I rather like
the pun).  Despite all we say and believe about individual
participation, the IETF has a legitimate need to understand the
difference between comments on a specification from an audience
with diverse perspectives and organized campaigns or a loud
minority with a shared perspective.  That requires
understanding whether speakers are largely independent of each
other (versus what have sometimes been referred to as sock
puppets for one individual) or whether they are part of an
organization mounting a systematic campaign to get a particular
position adopted (or not adopted).  The latter can also raise
some rather nasty antitrust / anti-competitiveness issues.
Clear identification of speakers, whether in the room or
remote, can be a big help in those regards, even though it
can't prevent all problems.  And the IETF having a policy that
requires clear identification at least establishes that we,
organizationally and procedurally, are opposed to nefarious,
deceptive, and posslbly illegal behavior.

A rule about having slides well in advance helps in another
way: slides that are bad news for some reasons but posted
several days in advance of the meeting provide opportunities
for comments and adjustments (from WG Chairs and others).  Ones
that are posted five minutes before (or 10 minutes after) a
session lose that potential advantage.  Again, I don't think we
should get rigid about it: if slides are posted in advance
and then supplemented or revised after feedback is received,
everyone benefits.

I want to stress that, while I think registration of remote
people who intend to participate is desirable for many reasons,
I think trying to condition microphone use (either remote on
in-room) with proof of registration and mapping of names would
be looking for a lot of trouble with probably no significant
benefits.

best,
   john

---

From: James Polk <[email-masked]>
Date: Mon, 05 Aug 2013 13:36:04 -0500

At 12:38 PM 8/5/2013, John C Klensin wrote:
>1, 2, 3a, 3b, 3c, 4, rather than 1, 2, 3, 4, 5, 6.

How exactly do you do this in pptx? Numbering slides is a linear 
operation AFAICT, and it's binary (it's either on or off). Please 
educate me if I'm wrong; lord knows I don't know don't know how to do 
everything flag/setting in powerpoint...

And, in my 8 years as TSVWG chair, I've rarely had completely new 
individual slides sprinkled throughout an existing deck. Rather, I've 
received updated slides - each with part of their content altered. 
Does this fall into your desire for a "3a", or is that just "3" 
(because 3a means an entirely new slide from scratch)?

BTW - I'm very much *not* in favor of stipulating to my WG that 
slides must be turned in 7 days in advance of a TSVWG meeting. I 
personally think no more than a 48 hour advanced window should ever 
be considered.

James

>I also

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 06 Aug 2013 02:06:35 +0100

On 08/05/2013 12:31 PM, Hadriel Kaplan wrote:
>  but at least one anonymous jabber participant (named "Guest") did
> remotely speak multiple times at the mic on one of the RAI working
> group sessions this past week (at RTCWEB if I recall).  I was
> personally ok with it, but it was awkward.

Ah. I wasn't aware of that. Not stylish at all IMO on the part of
whoever was "Guest". I'd be confident that the chairs and
participants will deal with it ok though. We do manage to deal
with silliness (e.g. people with bad ideas) all the time, so I
don't see why this'd pose an insurmountable problem to a wg.

On 08/05/2013 06:38 PM, John C Klensin wrote:
> the pun).  

Thanks. The pun was accidental as it happens, but I did leave
it in after I spotted it :-)

Puns aside, its an important point. Most patents are nonsense
(in terms of being really inventive) and we shouldn't base
our processes anywhere near primarily on the existence of that
nonsense.

> Despite all we say and believe about individual
> minority with a shared perspective.

Good point. We have similar issues with folks who do lots of
contract work I guess. But, IMO we should first make sure we can
hear the good points that are to be made, and only then modulate
our reactions to those in terms of who-pays-whom or whatever.

Put another way, regardless of patents or who's paying, if
someone (even anonymously) comes up with a really good technical
point, then we do have to pay attention. But I think we do do that.

In contrast, I think the real challenge remote participants face
is being heard. And when/if we solve that problem, I suspect that
remote participants with bad ideas will be a far worse problem
than those who'd like to submarine a patent or further a subtle
corporate agenda.

So again that leads me back to trying to encourage folks to just
make the tools better for us all and to only then try figure out
how we need to manage that. Perhaps Hadriel's anecdote above
means that how we use jabber is, after about a decade, now mature
enough that we ought think more about how we formalise its use.
I'm ok with waiting another longish time before even thinking
about how to do the same with successful inbound audio for
example.

S.

---

From: John C Klensin <[email-masked]>
Date: Mon, 05 Aug 2013 22:52:25 -0400

--On Tuesday, August 06, 2013 02:06 +0100 Stephen Farrell
<[email-masked]> wrote:

>> nonsense" (although that should be sufficient and I rather
>> like the pun).  
> 

Agreed, modulo observations about how much time we seem to put
into fine-tuning IPR policies and devising threats to make to
those who don't seem inclined to comply.

>> participation, the IETF has a legitimate need to understand
>> the difference between comments on a specification from an
>> audience with diverse perspectives and organized campaigns or
>> a loud minority with a shared perspective.
> 
> contract work I guess. But, IMO we should first make sure we
> can hear the good points that are to be made, and only then
> modulate our reactions to those in terms of who-pays-whom or
> whatever.

Indeed.

> someone (even anonymously) comes up with a really good
> technical point, then we do have to pay attention. But I think
> we do do that.

When, as you indirectly point out, we can "hear" them.

> In contrast, I think the real challenge remote participants
> face is being heard. And when/if we solve that problem, I
> suspect that remote participants with bad ideas will be a far
> worse problem than those who'd like to submarine a patent or
> further a subtle corporate agenda.

Of course, that is also true of participants who show up and
more f2f meetings.

> So again that leads me back to trying to encourage folks to
> just make the tools better for us all and to only then try
> figure out how we need to manage that. Perhaps Hadriel's
> anecdote above means that how we use jabber is, after about a
> decade, now mature enough that we ought think more about how
> we formalise its use. I'm ok with waiting another longish time
> before even thinking about how to do the same with successful
> inbound audio for example.

I'm actually not a big fan of inbound audio, at least not yet.
It is subject to the same technical and operational issues that
make outbound audio fragile, including the difficulties of clear
and "standard" pronunciation plus the same "how to raise your
hand, get in line, or otherwise ask for the floor" issues that
Jabber does.  But, if we are going to rely on Jabber for input,
we need to move toward treating it as a source of input with the
same priority as those in the room (and relatively more real
time), not something that is a nice-to-have when it happens to
work.

best,
    john

---

From: John Curran <[email-masked]>
Date: Mon, 5 Aug 2013 22:58:20 -0400

> room and the discussion is important" decision.

John - 
 
 I've participated in many IETFs in person but have been remote
 for the last two meetings.  I agree that the slides are essential
 in many cases for following the discussion, and furthermore agree 
 that it is a complete pain to have to hunt for the slides when they
 haven't been posted and are instead just in the room, or on the list, 
 or at some URL being passed around, etc.

 Noting all that agreement, I don't support a "one week" slide cutoff;
 the downside of not having a presentation slot as a result would be 
 a disproportionate impact to working group productivity... delaying
 presentation of an important topic for 4 months just because the slides 
 showed up late (but still days before the actual session) would be 
 creating a purely administrative and artificial impediment to getting
 things done.

 Now, something in ietf tools that emailed the WG Chairs (and AD?) noting
 that slides aren't up for presentations occurring _tomorrow_ and/or kept 
 track of presenters with chronic problems missing the deadline getting 
 their slide decks in the night before might be very useful, and help 
 quite a bit with improving remote participants ability to follow along.

My .02,
/John

---

From: Keith Moore <[email-masked]>
Date: Tue, 06 Aug 2013 09:08:10 -0400

> While I think getting slides in on time is great for a lot of reasons, reading the slides early isn't that important.   What is important is that remote people see the slides at the same time as lo…

I agree that remote people should see the slides at the same time as 
local people, except that I think that in both cases this should be well 
before the meeting.  The slides shouldn't be shown at the meeting unless 
needed to illustrate a point of active discussion.

People keep acting as if the purpose of these meetings - the reason 
people spend thousands of euro and travel thousands of km - is to watch 
slides.

Keith

---

From: Andrew Feren <[email-masked]>
Date: Tue, 6 Aug 2013 11:06:24 -0400

On 08/06/2013 09:08 AM, Keith Moore wrote:
>> While I think getting slides in on time is great for a lot of 
>> reasons, reading the slides early isn't that important.   What is 
>> important is that remote people see the slides at the same time as 
>> local people.   For that, it seems to me that Meetecho support does 
>> exactly what is needed.   You just follow the slideshow online, along 
>> with the audio.
>
> local people, except that I think that in both cases this should be 
> well before the meeting.  The slides shouldn't be shown at the meeting 
> unless needed to illustrate a point of active discussion.
>
> people spend thousands of euro and travel thousands of km - is to 
> watch slides.
Hi Keith,

I think this sort of misses the point.  At least for me as a remote 
participant.

I'm not interested in arguing about whether slides are good or bad. I am 
interested in following (and being involved) in the WG meeting.  When 
there are slides I want to be able to see them clearly from my remote 
location.  Having them integrated with Meetecho works fine.  Having 
slides and other materials available to download ahead of time is also 
OK.  I can work with what is available, but having slides brought to the 
meeting on USB (it happens) does me no good.  Also people using pointing 
devices, that can't be seen remotely, to point to areas on each slide 
doesn't help.

As of today when the slides are available (or if there are no slides and 
just talk) I can follow WG meetings quite well.  Being able to actively 
engage in any discussion remotely is, IMO, pretty much limited to the 
mailing lists.  Getting involved in an active discussion at a WG meeting 
remotely is currently difficult at best and impossible at worst.

I'm all in favor of discussions in WG meetings, but from where I sit we 
still have a ways to go to fully integrate remote participants. Making 
slides available soon enough to be viewed by remote attendees during the 
meeting seems like an achievable step towards better integration of 
remote participants.  The usefulness of doing this is also independent 
of whether the slides are for a presentation or to illustrate a point of 
discussion.

As Ted noted, "What is important is that remote people see the slides at 
the same time as local people."  That is the point.

-Andrew

---

From: Keith Moore <[email-masked]>
Date: Tue, 06 Aug 2013 14:03:23 -0400

On 08/06/2013 11:06 AM, Andrew Feren wrote:
>>> exactly what is needed.   You just follow the slideshow online, 
>>> along with the audio.
>>
>> well before the meeting.  The slides shouldn't be shown at the 
>> meeting unless needed to illustrate a point of active discussion.
>>

Actually I think the desire to get slides out early largely misses the 
point.   Or at least, it's an effort optimizing what should be the rare 
case.

I fully agree that slides should be easily available to both local and 
remote participants well prior to any meeting in which a presentation 
will be made.  (Say a plenary session where presentations are normal and 
appropriate.)   While speakers might like to revise their slides at the 
last minute, there's no reason why they shouldn't be expected to upload 
preliminary slides well in advance (because the key to an effective 
presentation is good preparation, after all) and a revised version (if 
necessary) later. This isn't at all rocket science, and there's no 
reason why it should not be done.

But if we really want to make remote participation effective, we need to 
figure out better ways to involve remote participants in _discussions_ - 
not only in plenaries, WG meetings, BOFs, etc., but also in hallway and 
bar conversations.   Having a local speaker read something from a laptop 
that was typed into a Jabber session by a remote participant is better 
than nothing.   But surely we can do better.

> As of today when the slides are available (or if there are no slides 
> and just talk) I can follow WG meetings quite well.  Being able to 
> actively engage in any discussion remotely is, IMO, pretty much 
> limited to the mailing lists.  Getting involved in an active 
> discussion at a WG meeting remotely is currently difficult at best and 
> impossible at worst.

It used to be the case that Internet access at IETF meetings was flaky, 
either because of the wireless network or because of the network 
connection or both.   More recently the performance of the meeting 
Internet access has been stellar.   If we put the same kind of effort 
into facilitating remote participation in discussions, I suspect we 
could move from "difficult at best and impossible at worse" to "works 
well".  Of course, it might take awhile, but it's those very kinds of 
discussions that are so essential to broad consensus that (when it 
works) makes our standards effective.   The fact that it doesn't work 
well now is not a good argument for not making it work well in the future.

(We're supposed to be creating the future, after all.  That's our job.)

It's also the case that the fact that facilities for involving remote 
participants in conversation haven't historically worked well, is used 
as a justification for continuing to have this dysfunctional style of 
conducting working group meetings, thus making very poor use of local 
participants' time and money.

I'm all for making presentation slides available to local and remote 
participants well before the meeting.   But if we're only concerned with 
making presentation slides available, we're selling ourselves very 
short.  That's the point I'm trying to make.

Keith

---

From: Andrew Feren <[email-masked]>
Date: Thu, 8 Aug 2013 08:44:20 -0400

Hi Keith,

Thanks for clarifying.  Put that way I agree 100%.

-Andrew

On 08/06/2013 02:03 PM, Keith Moore wrote:
> point.   Or at least, it's an effort optimizing what should be the 
> rare case.
>
> will be made.  (Say a plenary session where presentations are normal 
> and appropriate.)   While speakers might like to revise their slides 
> at the last minute, there's no reason why they shouldn't be expected 
> to upload preliminary slides well in advance (because the key to an 
> effective presentation is good preparation, after all) and a revised 
> version (if necessary) later. This isn't at all rocket science, and 
> there's no reason why it should not be done.
>
> But if we really want to make remote participation effective, we need 
> to figure out better ways to involve remote participants in 
> _discussions_ - not only in plenaries, WG meetings, BOFs, etc., but 
> also in hallway and bar conversations.   Having a local speaker read 
> something from a laptop that was typed into a Jabber session by a 
> remote participant is better than nothing.   But surely we can do better.
>
>> discussion at a WG meeting remotely is currently difficult at best 
>> and impossible at worst.
>
> It used to be the case that Internet access at IETF meetings was 
> flaky, either because of the wireless network or because of the 
> network connection or both.   More recently the performance of the 
> meeting Internet access has been stellar.   If we put the same kind of 
> effort into facilitating remote participation in discussions, I 
> suspect we could move from "difficult at best and impossible at worse" 
> to "works well".  Of course, it might take awhile, but it's those very 
> kinds of discussions that are so essential to broad consensus that 
> (when it works) makes our standards effective.   The fact that it 
> doesn't work well now is not a good argument for not making it work 
> well in the future.
>
> participants well before the meeting.   But if we're only concerned 
> with making presentation slides available, we're selling ourselves 
> very short.  That's the point I'm trying to make.
>
> Keith
>

---

From: John C Klensin <[email-masked]>
Date: Thu, 08 Aug 2013 10:43:05 -0400

--On Tuesday, August 06, 2013 11:06 -0400 Andrew Feren
<[email-masked]> wrote:

>...
> I think this sort of misses the point.  At least for me as a
> remote participant.
> 
> I'm not interested in arguing about whether slides are good or
> bad. I am interested in following (and being involved) in the
> WG meeting.  When there are slides I want to be able to see
> them clearly from my remote location.  Having them integrated
> with Meetecho works fine.  Having slides and other materials
>...

Let me say part of this differently, with the understanding I
may be more fussy (or older and less tolerant) than Andrew is...

If the IETF is going to claim that remote participation (rather
than remote passive listening/ observation with mailing list
follow up) is feasible, then it has to work.  If, as a remote
participant, I could be guaranteed zero-delay transmission and
receipt of audio and visual materials (including high enough
resolution of slides to be able to read all of them) and that
speakers (in front of the room and at the mic) would identify
themselves clearly and then speak clearly and at reasonable
speed, enunciating every word, I wouldn't care whether slides
were posted in advance or not.  

Realistically, that doesn't happen.  In some cases (e.g.,
lag-free audio) it is beyond the state of the art or a serious
technical challenge (e.g., video that is high enough resolution
that I can slides that have been prepared with 12 point type).
In others, we haven't done nearly enough speaker training or it
hasn't been effective (e.g., people mumbling, speaking very
quickly, swallowing words, or wandering out of microphone or
camera range).   And sometimes there are just problems (e.g.,
intermittent audio or video, servers crashing, noisy audio
cables or other audio or video problems in the room).   

In those cases, as a remote participant, I need all the help I
can get.  I'd rather than no one ever use a slide that has
information on it in a type size that would be smaller than 20
pt on A4 paper.  But 14 pt and even 12 pt happen, especially if
the slides were prepared with a tool that quietly shrinks things
to fit in the image area.  If I'm in the room and such a slide
is projected, I can walk to the front to see if if I'm not
already in front and can't deduce what I need from context.  If
I'm remote and have such a slide in advance, I can zoom in on it
or otherwise get to the information I need (assuming high enough
resolution).  If I'm remote and reading the slide off video,
especially low resolution video, is hopeless.  

More generally, being able to see an outline of what the speaker
is talking about is of huge help when the audio isn't completely
clear.  Others have mentioned this, but, if I couldn't read and
understand slides in English easily in real time, it would be of
even more help if I had the slides far enough in advance to be
able to read through them at my own pace before the WG session
and even make notes abut what they are about in my most-familiar
language ... and that is true whether I'm remote or in the room.

And, yes, for my purposes, 48 hours ahead of the WG meeting
would be plenty.  But I can read and understand English in real
time.  If the IETF cares about diversity as well as about remote
participation and someone whose English is worse than mine is
trying to follow several WGs, 48 hours may not be enough without
requiring a lot of extra effort.

That is not, however, the key reason I said "a week".  The more
important part of the reason is that a one-week cutoff gives the
WG Chair (or IETF or IAB Chairs for the plenaries) the time to
make adjustments.  If there is a nominal one week deadline, then
the WG Chair has lots of warning when things don't show up.  She
can respond by getting on someone's case, by accepting a firm
promise and a closer deadline, by finding someone else to take
charge of the presentation or discussion-leading, or by
rearranging the agenda.  And exceptions can be explained to the
WG on the mailing list.  With a 48 hour deadline, reasonable
ways to compensate are much less likely, the Chair is likely to
have only the choice that was presented this time (accepting
late slides or hurting the WG's ability to consider important
issues) and one needs to start talking about sanctions for bad
behavior.   I would never suggest a firm "one week or no agenda
time" rule.  I am suggesting something much more like a "one
week or the WG Chair needs to make an exception, explain it to
the WG, and be accountable if the late slides cause too much of
a problem".  There is some similarity between this and the
current I-D cutoff rule and its provision for AD-authorized
exceptions.  That similarity is intentional.

--On Monday, August 05, 2013 13:36 -0500 James Polk
<[email-masked]> wrote:

>> available well in advance of the meeting have been covered
>> well by others.  And, as others have suggested, I'm willing
>> to see updates to those slides if things change in the hours
>> leading up to the meeting, but strongly prefer that those
>> updates come as new alides with update-type "numbers" or other
>> to see the updated deck consist of slides 1, 2, 3, 3a, 3b, 4
>> or 1, 2, 3a, 3b, 3c, 4, rather than 1, 2, 3, 4, 5, 6.
> 
> How exactly do you do this in pptx? Numbering slides is a
> linear operation AFAICT, and it's binary (it's either on or
> off). Please educate me if I'm wrong; lord knows I don't know
> don't know how to do everything flag/setting in powerpoint...

Hmm.  First, I missed the IESG Statement requiring the people
use pptx or any other particular piece of
presentation-preparation software, much less the IETF consensus
for that statement.  If the tools you choose to use don't meet
the needs of the IETF, that doesn't seem to me to be an IETF
problem.

Second and more important, while having slide preparation
software number pages, I'm not aware of anything that prevents
you from typing in numbers, especially with something that is as
inherently page (slide)-oriented as presentation-preparation
software.  Yes, it is an annoyance, but having the need to
update slides within a short interval before the meeting/
presentation/ discussion is an annoyance to participants who
depend on the ability to access those slides and who might want
to prepare reference or discussion notes in advance.

> And, in my 8 years as TSVWG chair, I've rarely had completely
> new individual slides sprinkled throughout an existing deck.
> Rather, I've received updated slides - each with part of their
> content altered. Does this fall into your desire for a "3a",
> or is that just "3" (because 3a means an entirely new slide
> from scratch)?

3a.  

> BTW - I'm very much *not* in favor of stipulating to my WG
> that slides must be turned in 7 days in advance of a TSVWG
> meeting. I personally think no more than a 48 hour advanced
> window should ever be considered.

See above.

   best,
   john

---

From: Scott Brim <[email-masked]>
Date: Thu, 08 Aug 2013 10:53:50 -0400

Well, I've worked remotely for 16 years and in most meetings I don't get
to see the slides until the meeting starts.  Usually I can only see them
via some conferencing tool.  Sometimes I get a copy in mail the week
after.  So I think the IETF is already doing pretty well at making
materials available, and insisting on getting slides far in advance of
the meeting is beyond what most people get in reality outside of the
IETF.  I would call this a SHOULD not a MUST.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 08 Aug 2013 08:43:40 -0700

> IETF.  I would call this a SHOULD not a MUST.

  suspect Scott's experiences matches that of many of us, but let's 
consider possible sampling bias here, which might limit the 
applicability of that experience to the IETF...

There is a difference between being a native English speaker who is 
already integrated into an on-going work effort, versus someone who 
might be neither, as is often true for IETF meetings.

Informative slides that are available ahead of time can be extremely 
helpful, for establishing the context of the presentation/discussion and 
for outlining the main points.

For someone new to a topic or with language limitations, that can make 
the difference between understanding the flow of speech, versus not.

So let's be careful about whether slides ahead of time need to be a 
requirement, rather than being considered only a nice-to-have.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Michael Richardson <[email-masked]>
Date: Thu, 08 Aug 2013 15:22:14 -0400

> especially low resolution video, is hopeless.

Also, I can't go back to the previous slide if the system is just remote
projection.

Good quality mumble-free audio + pre-distributed slides locally rendered
beats any amount of lag-free video.

I also can go ahead and find out if the speaker is going to cover an
important point, or if I have to bring it *now*.

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 12 Aug 2013 12:35:26 -0400

[personal disclaimer: I have participated remotely, a few times, and I agree that it's not the same as being there, and I agree that it could be improved.  But I think we need to balance the needs of remote participants, vs. the goals of physical meetings: to get work done that can only get done with direct human interaction, with real-time interactive discussion, and hallway co-mingling, etc.  In my opinion being remote will *never* be the same as being there, and that's why we have physical meetings to begin with, instead of just virtual ones - ultimately I want you to come to the meeting *physically*... but I want to do as much as possible and *practical* to accommodate remote participants, because I know not everyone has the luxury to come, and I want their input regardless.]

I think one challenge with this discussion, at least for me, is that it feels like the solutions being proposed are completely out of whack with the severity and nature of the problems at hand, and the people they apply to.  Adding more rules and processes is about the surest way I know to annoy engineers, let alone volunteer ones.  And if your right hand is sore, you don't cut it off and replace with a mechanical one, or go build a billion-dollar robot to be controlled by brain-waves.

People on this list are saying things like "we don't know who's at the mic", and "we can't see the slides well enough", and "presenters are hard to follow".  ISTM there are practical and low-tech means of fixing those problems.

Here're some ideas of possible solutions - but most of these boil down to either "have WG Chairs do their job", or "get over it":

Problem-1: we don't know who's at the mic, because people forget to say their names.
Solution: (a) train WG Chairs to remind people at mic, and interrupt them if they haven't said their names, (b) have jabber scribes sit next to mics, to do the same as WG chairs, (c) if chairs and scribes forget, send a chat message in jabber to remind them.  Is it annoying?  Sure.  Will it sometimes fail to work?  Sure.  But every other solution will also be annoying and not always work, and this solution is very low-tech and simple.  This is what happens in RAI area WG meetings, and appears to work afaict.  If it doesn't work, then we go to a Plan-B in the future.

Problem-2: we can't see the slides well enough, in video.  Meetecho is ok, but they don't cover all meetings.
Solution: we pay Meetecho to cover all WG meetings.  Last time I checked, we weren't paying them anything; but if we really want that type of technology in all meetings, then it's only fair to pay them. (frankly we should be paying them now already)  We also need to remind people to use reasonable font size.  10pt is bad mojo even in the physical room.  That type of thing is what a WG chair should be telling people, though - we don't need slide police.

Problem-3: presenters are hard to hear/follow/understand.
Solution: that's life.  We're a volunteer organization, not trained professional thespians.  We have drafts written in plain ascii available in advance, and audio, and video, and jabber, and (hopefully) Meetecho-type service, and that's about as good as it can reasonably be.

Problem-4: we want slides 7 days in advance to translate to our native language
Solution: You're doing it wrong - or rather, the presenters are doing it wrong.  The slides aren't a replacement for drafts, or a copy-paste of draft text.  WG Chairs know this, and should remind presenters of it.  As an aside, if you really need a week to translate them, it's highly unlikely you'll be able to truly *participate* interactively during the meeting - and you don't need to be real-time if all you're going to do is listen/watch - we record the whole thing for your later viewing pleasure.

Problem-5: we want slides 48 hours in advance to translate to our native language, download it, etc.
Solution: sounds reasonable.  Have WG chairs use 48 hours as their guideline, but not as a strictly enforced rule a la I-D submission deadlines.  WG chairs can figure out when exceptions need to be made, how to deal with it, etc.  We don't need this in an RFC.

Problem-6: if you change slides, I don't know which slides you changed
Solution: it happens, and that's life.  You've got to follow the speaker's words, not a published script.  The point of these meetings is for real-time discussion on relevant matters that can only be handled live.

Problem-7: 

-hadriel

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 12 Aug 2013 13:55:18 -0400

Ugh.  Ignore that email below - I had sent it a few days ago but somehow it got stuck in the outbox and never got sent, and the discussion is past that point now so it doesn't matter.

-hadriel