Subject: [vmeet] Suggestion

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 8 Apr 2016 14:42:54 +0000

Lee and I chatted this morning, and he wondered aloud what it would take to virtualize IETF 100. Not to remove the f2f component, but to make remote participation work well enough that we can say remote participants are at least second class citizens, as opposed to being an oddball tacked on.

Part of this, and this is why I copied Mirja and Brian, needs to be a meaningful measure of quality of experience and response to it. At minimum, having people at each end of the conversation report something akin to a MOS score (thumbs up or down are obvious signals, or holding up some number of fingers to say how it went, or maybe having a web page that would allow someone to do the equivalent in a timestamped log) could give us some interesting information perhaps. Mirja and Brian can no doubt inform and refine those concepts. If we're having a congestion issue, for example, maybe we need to make it easy to make video content less active (limit it to slide projection as opposed to a human face, or change the codec, or have the human able to turn it off without losing audio, etc). If we have packet sequence numbers, maybe we want to capture realtime statistics on packet lose, delay variation, and so on.

Part of this also has to do with the chair's "cockpit". From Lee and my perspective chairing v6ops on Monday, life was pretty busy. My laptop was projecting slides, and I was interacting with speakers and people in the room. In the room we had a jabber scribe and someone taking notes on what is oddly called an Etherpad. Lee had jabber in a pane, the Etherpad on a pane, his own notes on a pane, and meetecho control on a separate laptop. I didn't actually bring it (intended to but managed to leave it in my room in an overflow hotel), but I usually have an iPad for my own notes. Add to that Lee being able to chase the blue sheets, interact with the A/V team, or otherwise run around the room, and - well, let's just say I'm glad I have a co-chair. It takes two. At this point, I'm wondering how to simplify all that. Not offering suggestions right now, but I think we need to think about that in the context of virtualizing meetings.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 08 Apr 2016 16:16:44 -0400

Fred Baker (fred) <[email-masked]> wrote:
    > video content less active (limit it to slide projection as opposed to a
    > human face, or change the codec, or have the human able to turn it off
    > without losing audio, etc). If we have packet sequence numbers, maybe
    > we want to capture realtime statistics on packet lose, delay variation,
    > and so on.

If we could have all slides uploaded before the meeting, then we wouldn't
have to project the VGA signal, but could project remote controlled PDF
instead.  I think that meetecho did this before.

I like the MOS ideas.

    > Part of this also has to do with the chair's "cockpit". From Lee and my
    > perspective chairing v6ops on Monday, life was pretty busy. My laptop
    > was projecting slides, and I was interacting with speakers and people
    > in the room. In the room we had a jabber scribe and someone taking
    > notes on what is oddly called an Etherpad. Lee had jabber in a pane,
    > the Etherpad on a pane, his own notes on a pane, and meetecho control
    > on a separate laptop. I didn't actually bring it (intended to but
    > managed to leave it in my room in an overflow hotel), but I usually
    > have an iPad for my own notes. Add to that Lee being able to chase the
    > blue sheets, interact with the A/V team, or otherwise run around the
    > room, and - well, let's just say I'm glad I have a co-chair. It takes
    > two. At this point, I'm wondering how to simplify all that. Not
    > offering suggestions right now, but I think we need to think about that
    > in the context of virtualizing meetings.

1) note takers could be remote.
2) why did Lee have both etherpad and his own notes?
3) I like etherpad and use it regularly, but couldn't notes go into jabber?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 8 Apr 2016 16:42:55 -0400

> 3) I like etherpad and use it regularly, but couldn't notes go into jabber?

I tried etherpad once and hated it. I admit that I didn't give it a fair 
test, and maybe I would learn to like it.

The notes I take for myself sometimes have stuff in them that I would 
not want to share. So I have to censor my notes before sharing them with 
the group.

I have thought about remote note takers. But I think often the info 
coming to the remote note taker isn't sufficient to get complete notes. 
That is certainly the case when you have to reconnect.

Has anyone tried just running the audio from a session through a 
speech-to-text processor? (I expect it would be terrible, but one can hope.)

	Thanks,
	Paul

---

From: Lee Howard <[email-masked]>
Date: Sat, 09 Apr 2016 10:15:43 -0300

[sorry, I posted from the wrong address]

On 4/9/16, 9:12 AM, "Howard, Lee" <[email-masked]> wrote:

>
>
>On 4/8/16, 5:16 PM, "Michael Richardson" <[email-masked]> wrote:
>
>>

I agree, and wouldn¹t auto-format slides, but have the MeetEcho dashboard
include the Display Slide function, which would pull them directly from
the materials link.

>>I like the MOS ideas.

I would even put a radio dial or slider into MeetEcho allowing MOS
feedback directly in the app. With sequence numbers and/or flow data,  we
could get useful feedback on what¹s causing problems. A general spec for
this would allow it to be generalized to other apps, too.

>>1) note takers could be remote.

Only if audio is reliably good.

>>2) why did Lee have both etherpad and his own notes?

Sometimes I¹m able to add or correct a speaker¹s name in etherpad, but my
own notes often include comments and followup notes to myself that are not
appropriate for the etherpad notes. Not that they¹re Inappropriate, but
they can include suggestions for future work, or gaps, etc.
I merge my notes with etherpad when preparing minutes.

>>3) I like etherpad and use it regularly, but couldn't notes go into

No. Jabber is also useful for background questions and corrections, which
again are not appropriate for minutes.

Envisioning integrated meeting management tool. . .
Chair View
Agenda
	Chair enters the list of drafts or discussion topics, and time assigned
to each.
	Tool generates agenda, with links to drafts. Assume 5 minute
Administrivia.
Materials
	Agenda automatically prepared from Agenda tool; no upload
	Chair uploads slides, assigns to agenda item. Doesn’t have to be 1:1
slide:agenda item.
Camera view
	As MeetEcho does now
Slides Pane
	Displays slides locally and remotely
	Insert slide # into Etherpad as advanced?
Queue Pane
	Same as participant view, but with a ³close the queue² function

Participant View
Agenda Pane
	Display agenda, with clock. Clock turns yellow with 2 minutes remaining,
red with 0 time remaining.
	Participants can click on draft or materials links.
Etherpad Pane
	Pre-populated with Agenda, including materials links.
	Since it¹s a pane in the Participant View panel, anyone can correct the
notes or spelling of their name.
Slides Pane
	Displays slide view
Jabber Pane
Queue Pane or Control (maybe integrated with Jabber?)
	Current speaker, and people in queue
	Add Myself to Queue
	Insert Myself into Queue to respond to current speaker.

Lee

---

From: "Howard, Lee" <[email-masked]>
Date: Sat, 9 Apr 2016 12:12:18 +0000

>1) note takers could be remote.

Only if audio is good.

Chair uploads slides, assigns to agenda item. Doesn¹t have to be 1:1

Participant View
Agenda Pane

Lee

________________________________

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Sat, 9 Apr 2016 18:46:50 +0000

> On Apr 8, 2016, at 5:16 PM, Michael Richardson <[email-masked]> wrote:
> 

And why do I usually?

Because two (or three) heads are better than one. Ole took good notes, I guarantee it, but Lee might catch something he didn't, and vice versa.

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Sat, 9 Apr 2016 18:46:50 +0000

> On Apr 8, 2016, at 5:42 PM, Paul Kyzivat <[email-masked]> wrote:
> 
> Has anyone tried just running the audio from a session through a speech-to-text processor? (I expect it would be terrible, but one can hope.)

We have actually done something similar. We had a participant who was deaf, and ISOC paid for a speech-to-text PERSON. We took the things she wrote and displayed them on a screen in plenaries in real time, and people said they appreciated it. It helped them understand things that they managed to hear as garbled.

Two comments. First, she did a pretty good job - not perfect, but pretty darn good. Second, just like when you see closed-caption on a TV, what she wrote was largely phonetic, and often picked a homonym or was just plain wrong. I would have to believe that speech-to-text software would not do better than she did.

---

From: John C Klensin <[email-masked]>
Date: Sun, 10 Apr 2016 14:54:18 -0400

--On Saturday, April 09, 2016 18:46 +0000 "Fred Baker (fred)"
<[email-masked]> wrote:

>...
> Two comments. First, she did a pretty good job - not perfect,
> but pretty darn good. Second, just like when you see
> closed-caption on a TV, what she wrote was largely phonetic,
> and often picked a homonym or was just plain wrong. I would
> have to believe that speech-to-text software would not do
> better than she did.

Fred,

As I mentioned in the simultaneous translation thread, I've
experimented extensively with speech-to-text arrangements and
IETF audio feeds, with an without RFC-trained vocabularies.   I
haven't studies that particular human transcriber experiment,
but I've worked with a lot of very good human transcribers,
including but not limited to professional court reporters.   To
the extent to which those experiments and experience are
indicative, "not do better than she did" is vastly optimistic,
with "epic fail" being closer, especially when speakers have
"unusual" accents or odd speech pacing patterns (something of
which I'm often guilty). 

Vocabulary training does help both speech-to-text systems and
human transcribers, especially with homonym-like problems, but
can introduce their own families of "just plain wrong" symptoms.

--On Saturday, April 09, 2016 21:03 -0400 Michael Richardson
<[email-masked]> wrote:

> Brian Rosen <[email-masked]> wrote:
>> I think that means we require presentation materials to
>> be preloaded and in common formats.
> 
> I think that we need this anyway for many other reasons.

And I think we have been saying that for years and that we will
continue to have late arrival of materials or late revisions
until and unless we develop firm rules that "no presentation
materials equals no presentation" as well as its close relative
"no agenda equals WG session cancellation".

However, speaking as a frequent offender, we also need to be
very aware that such a rule involves a tradeoff.  Conversations
about a particular set of issues within a day or so of the
presentation or in-session discussion are common.  They are
actually a natural advantage of gathering people together in one
place for a f2f meeting.  If insights gained from such
discussions cannot be factored into slides and presentations, we
all lose.   I don't know quite how to balance that against
having materials preloaded some significant time in advance, but
we probably need to keep the issues in mind.

    john