Subject: [vmeet] Duties of WG Chairs

From: John Leslie <[email-masked]>
Date: Thu, 7 May 2009 13:53:56 -0400

Russ Housley asked me to make this list aware of an IESG discussion
about difficulties finding WG co-chairs with an assured travel budget.
There seems to be consensus that the absence of both co-chairs creates
problems during the week-long IETF meetings, but that such situations
will arise as travel budgets contract.

   As part of the discussion, one AD mentioned that Virtual Interim
Meetings are being seriously considered. (I have noted three announcement
of Virtual Interim meetings recently.)

   Another AD mentioned co-chair resignations specifically because of
travel limitations.

   Can we do anything to help?

--
John Leslie <[email-masked]>

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 11 May 2009 17:01:00 -0700

> of Virtual Interim meetings recently.)
...
 >    Can we do anything to help?

Excellent timing for the query.

I am starting to suspect that the answer is yes.

I've been thinking about the experiences, so far, in looking at tools, and I'm 
suspecting that the two, immediate, areas of potential benefit are the IETF 
management meetings (IESG, IAB, IAOC) and interim meetings.

A distinction that came up in a private discussion was between 'special' events 
-- less frequent, and amongst a group where participants have less experience 
with each other -- versus regular ones.

The management meetings are quite regular in timing and participation.  That has 
an effect on group dynamics and probably makes it tolerable to have /less/ basic 
functionality, such as sharing slides, but probably makes telephone bridging 
/more/ important, to deal with participants who are in transit.  (The more 
regular the meetings, the more likely someone is in transit, with no useful 
Internet connectivity.)

By contrast, an interim meeting could require good Internet access for 
participants and, therefore, limit voice to VOIP.  But content display and 
meeting management tools probably become more important.

By contrast a plenary or, worse, an entire week-long series of sessions is 
vastly more complicated and demanding.  I think we can see the hints of 
capability emerging but could not reasonably expect to have major impact on main 
IETF meetings quickly.  We can probably improve the current functionality for 
remote participants, but we aren't likely to make a serious dent in the actual 
/need/ for full-week, face-to-face meetings.

Based on this view -- and folks are encouraged to comment -- I think that a good 
near-term goal is to try to find a tool to enable easy, virtual interim meetings.

We should try to converge on the statement of functions and scaling that we 
think are essential for an interim meeting.

 From the ealier discussions, one of the milestones was stated as:

 > MG-C:  Individual virtual session, small scale (30?)

Henning offered these items in response:

 > - Ability to render PowerPoint and PDF MUST, OpenOffice SHOULD.
 >
 > - For audio, allow VoIP and PSTN dial-in (800# is MAY).
 >
 > - Ability to mute remote participants (e.g., to deal with someone's
 >   music-on-hold system)
 >
 > - Audio recording

=============================

 From the earlier postings, here's what I think we are wanting, to support a 
virtual, interim meeting.

I'll note that we need to be careful about an existence proof for a function, 
versus what is comfortable for its regular use.  In particular because we are a 
technical group, we can deal with all sorts of silliness that the rest of the 
world can't, won't, and should not have to.  However I believe that for regular 
use, we need to pretend that we aren't geeks and instead demand features and 
ease of use that makes sense for regular folk.

So...

REQUIRED
--------

0.  Scale

     I think we could get away with a relatively modest initial requirement. 
Say, 30 people?  That will limit applicability for the larger and more active 
working groups.

     I guess the real question is what participation numbers there have been for 
interim meetings over the last couple of years?  We might still settle on a 
small initial number, but we should try to know beforehand.

1.  Meeting management -

     a)  Ability to queue up participants, as if at a microphone

         For meetings of a group that can easily be 20-40 people, with 
relatively little history of group collaboration, the job of meeting management 
is vastly easier when there is some functional help for coordinating who wants 
to speak next.

     b)  Ability to shift who is presenting

         This is the equivalent of having different folk come up to the front of 
the room and plug in their laptop.  I suspect that a failure to support this 
mode of participation would seriously hamper the convenience -- and possibly 
effectiveness -- of meetings.

2.  VOIP for voice

     As noted above, a telephone bridge does not have to be a requirement, for 
meetings that are infrequent.

3.  Shared slide presentation

     Shipping powerpoint files around is tolerable only for special cases, I 
believe.  Since we are talking about broad-based, regular use, we should make 
presenting slides as comfortable as it is in a room.  A major benefit of 
integrated slide presentation and control is that there is then no need to worry 
about people's knowing what slide is showing.

4.  Shared white board

     My gut says that this really is an extremely useful feature for many 
meetings.  And since it's available in a number of products, it seems to make 
sense that we require it.

NICE-TO-HAVE
------------

5.  IM

     Existing IETF jabber can suffice, IMO.  IM integrated into the conferencing 
tool would be nice, but only if it is on a par with the jabber service we are 
used to.

6.  Telephone bridge (in, out)

     mumble.  This adds quite a bit of expense.  I could argue /against/ 
supporting it at all(!)

7.  ???

Comments?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Thomas Narten <[email-masked]>
Date: Tue, 12 May 2009 08:42:49 -0400

Some random thoughts/reaction to your note...

Dave CROCKER <[email-masked]> writes:

> I've been thinking about the experiences, so far, in looking at
> tools, and I'm suspecting that the two, immediate, areas of
> potential benefit are the IETF management meetings (IESG, IAB, IAOC)
> and interim meetings.

We should note that some WGs are already having interim meetings, and
it is not clear that any of them are on this list (not good). I sat in
on an IPSecME meeting a week or two ago, and it worked fine. It was a
smallish meeting (20 people), with a clear agenda, and seemed like an
excellent thing for WGs to be doing (nothing like a meeting as a
forcing function to get people to deliver and focus).

http://trac.tools.ietf.org/wg/ipsecme/trac/wiki/ConferenceCalls

They used TeamSpeak. I thought it worked well overall. Certainly got
the job done.

> By contrast, an interim meeting could require good Internet access
> for participants and, therefore, limit voice to VOIP.  But content
> display and meeting management tools probably become more important.

Actually, I don't agree with this (necessarily). The absolute
requirements to participate in an interim meeting are more
modest. Think about the critical things:

 - audio bridge (could be VoIP, or POTS, that is a detail)
 - access to charts
 - IM capability as a side channel

The above would in practice cover 90% of what WGs really do in
practice during interim meetings. You do not need "good" Internet
access for IM. Just good enough. (You presumably need a lot better
connectivity for whiteboards, etc.)

Sure, there are other nice-to-haves, but I think most of those are not
requirements in the vast majority of cases.

> Based on this view -- and folks are encouraged to comment -- I think
> that a good near-term goal is to try to find a tool to enable easy,
> virtual interim meetings.

Don't underestimate the power of 1) a phone bridge and 2) advance
availability of charts, 3) best practices about how do
teleconferences (e.g., when changing slides, telling everyone which
slide you are on).

I suspect that many of us have lots of experience with the above.

>  > - Ability to mute remote participants (e.g., to deal with someone's

I might add IM for a back-channel. This is good for things like: "I
want to speak now", asking short questions (clearly), telling people
which chart we are on, etc.

>      I think we could get away with a relatively modest initial
> requirement.  Say, 30 people?  That will limit applicability for the
> larger and more active working groups.

I agree that most meetings are small.

>      I guess the real question is what participation numbers there
> have been for interim meetings over the last couple of years?  We
> might still settle on a small initial number, but we should try to
> know beforehand.

Careful about predicting future participation based on the past,
especially if travel issues are now motivating more to stay at home.

>          For meetings of a group that can easily be 20-40 people,
> with relatively little history of group collaboration, the job of
> meeting management is vastly easier when there is some functional
> help for coordinating who wants to speak next.

It may also be necessary for basic fairness. Folk can get mighty
unhappy if it seems to them that they are being treated as second
class citizens in the microphone queue relative to others.

>          This is the equivalent of having different folk come up to
> the front of the room and plug in their laptop.  I suspect that a
> failure to support this mode of participation would seriously hamper
> the convenience -- and possibly effectiveness -- of meetings.

While I agree this is necessary, I am not sure that we require tools
to support this. Charts in advance work pretty well in practice. But
not at the expense of limiting participation if the bandwidth
requirements are too high.

>      Shipping powerpoint files around is tolerable only for special
> cases, I believe.

I disagree. I think shipping charts around in advance has benefits
too. Doing everythig at the very last second (especially for a meeting
that is schedule far out in advance) does not necessarily result in
good meetings and/or good presentations. And it can actually be quite
useful to skim a presentation in advance before a talk begins.

And I believe that in most cases, we are taking about presenting
static charts, not whiteboard sessions.

> Since we are talking about broad-based, regular use, we should make
> presenting slides as comfortable as it is in a room.  A major
> benefit of integrated slide presentation and control is that there
> is then no need to worry about people's knowing what slide is
> showing.

We should strive to make it easier to follow a presentation without
the requirement for such tools, since they don't help when playing
back an audio recording...

>      My gut says that this really is an extremely useful feature for
> many meetings.  And since it's available in a number of products, it
> seems to make sense that we require it.

Useful, but not necessary in the vast majority of WG sessions I've
attended.

Thomas

---

From: John Leslie <[email-masked]>
Date: Tue, 12 May 2009 22:56:53 -0400

Thomas Narten <[email-masked]> wrote:
>> ... an interim meeting could require good Internet access for
>> participants and, therefore, limit voice to VOIP.  But content

   We have been running virtual meetings with that much for some time
now, any many folks are used to it. By human nature, if we provide a
tool that folks aren't used to, they'll gravitate back to _that_ model
they're used to.

   So, while I support Dave's call for more than that, we can't just
make a list -- we need to explain _why_ each feature we recommend is
a significant improvement over bare-bones.

> practice during interim meetings.

   Indeed -- by definition...

> You do not need "good" Internet access for IM. Just good enough.

   Nonetheless, we've all seen cases where "good enough" wasn't
available. (Last week's IESG telechat comes to mind...)

> (You presumably need a lot better connectivity for whiteboards, etc.)

   It depends...

> requirements in the vast majority of cases.

   "Nice-to-have" is really a shorthand for "things that help sometimes".

> Don't underestimate the power of
> 1) a phone bridge and
> 2) advance availability of charts,
> 3) best practices about how do teleconferences (e.g., when changing
> slides, telling everyone which slide you are on).

   1) Right now, a phone bridge is essentially state-of-the-art for
audio. Most everything else flunks Brian's 150 msec test.

   2) Advance availability of charts is egregiously poorly enforced.
Where it is enforced it leads to charts which only exist in the
presenter's imagination. Preparation of slides is the best exercise
for most presenters to organize their thoughts -- and, alas, the
last minute is when it, almost by definition, happens.

   3) Best Practices simply don't get followed, unless somebody is
constantly nagging.

> I suspect that many of us have lots of experience with the above.

   ... but with the reality, not the ideal...

> which chart we are on, etc.

   Absolutely!

> especially if travel issues are now motivating more to stay at home.

   Agreed; but it's a reasonable starting point...

> class citizens in the microphone queue relative to others.

   I regard this as very important; but I'm not happy with any
automated functions I've seen in a telepresence package. A simple
plaintext list of folks who have asked to speak (and not given up
their place or spoken) is what all participants should have.

   I consider this important enough that someone should have the role
of maintaining it if we can't get it automated.

> requirements are too high.

   It is a weakness of many of the available tools that they expect
every participant to have the same available bandwidth. This simply
isn't the world we live in. Anyone without sufficient bandwidth to
support a real-time display should be able to turn that bandwidth
demand off.

   As a general rule, we shouldn't be designing for the lowest
(bandwidth) common denominator.

> too.

   When they actually _get_ shipped, I agree.

> Doing everythig at the very last second (especially for a meeting

   But last-minute changes, even changes _during_ the presentation
are the nature of the beast. I agree with Dave.
   
> static charts, not whiteboard sessions.

   When a presenter has to face and "connect with" an audience, any
whiteboard-style action is too difficult for most of us. In a virtual
meeting, the tradeoffs are very different; and whiteboard actions
are a very good substitute for the waving hands. ;^)

> back an audio recording...

   Playing back an audio recording is not the normal case. We should
design for the normal case.

> Useful, but not necessary in the vast majority of WG sessions I've

   Not _used_ -- while this makes it "not necessary" by definition,
it says little about how useful it could be.

   I don't mean to belittle Thomas (were I even capable of doing so),
but we need to escape the "We've always done it this way" mindset.

====

   My own concerns:

1) Participants need to figure out who's speaking. There are many ways
   to help them, but it deserves to be high among our concerns.

2) Snapshots of slides and/or whiteboards can help keep folks on the
   same page. Things change during meetings, and it's very easy for
   human memories to diverge.

3) There really should be a "record" of a meeting. This is, admittedly,
   an art form; but folks don't want to listen to an audio recording
   or even read a transcript of every word spoken. It very likely would
   be helpful if such a record could be visible in real-time to some
   participants. (I don't claim to be particularly good at doing this,
   but there _are_ folks who are good at it.)

4) Every meeting should create "action items" that someone is responsible
   for following-up on. (Some are group items; some private.) These
   deserve far more visibility than they usually get.

5) Separate applications is very likely better than one integrated
   application. Webex drives me crazy by taking over organization of
   the screen. I don't want anything else interacting with my Jabber
   application.

6) VoIP-as-we-know-it doesn't meld well with POTS conferencing. Until
   VoIP can have consistent delay -- ideally Brian's 150 msec, but
   definitely under 500 msec -- we'll probably have to do without it.

7) Learning curve is way too high for most telepresence packages.
   Folks simply _won't_ learn how to use them effectively. Formal
   training / learning / practicing sessions are essential. With
   enough of them, we can get perhaps ten percent of participants
   fluent enough to _use_ their features. (Much less than that, the
   features will atrophy for lack of use.)

---

From: Dave Cridland <[email-masked]>
Date: Wed, 13 May 2009 17:55:37 +0100

I've been asking around for solutions to these issues which are based  
around open standards. While I fully understand, and indeed support,  
the need for an "out of the box" solution to this, I'd like an "out  
of the box" solution that used a suite of open standards to achieve  
these goals.

I'm speaking here both as an individual fairly heavily involved with  
open standards work, and also as someone who ends up participating  
remotely at the majority of IETFs, and has participated remotely at  
multiple Interim meetings as well. I don't claim exhaustive  
experience, mind. :-)

On Tue May 12 01:01:00 2009, Dave CROCKER wrote:
> failure to support this mode of participation would seriously  
> hamper the convenience -- and possibly effectiveness -- of meetings.
> 
> 
This is, it appears, provided by the XCON working group, and in  
particular by RFC 4582, which it appears is supported by at least  
some people. I apologise for being quite so vague, but SIP isn't my  
field at all (I have some SIP/VOIP contacts via the XSF's work on  
Jingle, and I'm merely relaying).

That said, I'd note that no interim or "design team conference call"  
I've ever participated in has had anything like this, and they've  
worked fine.

> 2.  VOIP for voice
> 
>     As noted above, a telephone bridge does not have to be a  
> requirement, for meetings that are infrequent.
> 
> 
Presumably, we need to be considering SIP or Jingle (XMPP VOIP) here.  
SIP has the conferencing implementations, whereas this isn't really  
defined for Jingle. (A shame, since Jingle would integrate neatly  
with the XEP-0045 chat rooms we're already using with much success).

Of course, some SIP implementations also talk Jingle and/or H.323,  
hence opening the client field a bit.

> cases, I believe.  Since we are talking about broad-based, regular  
> use, we should make presenting slides as comfortable as it is in a  
> room.  A major benefit of integrated slide presentation and control  
> is that there is then no need to worry about people's knowing what  
> slide is showing.
> 
> 
I agree this is nicer than the usual flurry of "what slide are we  
on?" in the MUC room, but in general, the current practise of putting  
slides on a known website, and calling out slide numbers, works well  
enough for me to not care much if this is missing.

> many meetings.  And since it's available in a number of products,  
> it seems to make sense that we require it.

My gut says that since I have seen no use of whiteboards, physical or  
virtual, at any IETF related meeting, then while we might use them if  
we had them, they won't be missed if we don't. So I'd argue this is  
"nice-to-have".

> NICE-TO-HAVE
> ------------
> 
> 5.  IM
> 
>     Existing IETF jabber can suffice, IMO.  IM integrated into the  
> conferencing tool would be nice, but only if it is on a par with  
> the jabber service we are used to.
> 
> 
I'm assuming by this you mean that IM (or rather, multiuser chat) is  
indeed a requirement, but it needn't be integrated, and the existing  
XEP-0045 service is adequate.

>     mumble.  This adds quite a bit of expense.  I could argue  
> /against/ supporting it at all(!)

Actually, one of the real benefits of using an open protocol, perhaps  
SIP, for this kind of things is that not only do many implementations  
support telephone bridges, but it's my [limited] understanding that  
local organizations could even provide their own phone bridges  
relatively simply, which spreads, at least, any additional expense.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 14 May 2009 06:18:35 -0700

> a significant improvement over bare-bones.

and

Dave Cridland wrote:
 > around open standards. While I fully understand, and indeed support, the
 > need for an "out of the box" solution to this, I'd like an "out of the
 > box" solution that used a suite of open standards to achieve these goals.

Folks,

We DO need to be clear about our reasons for choosing anything different from 
what we do today and the reasons for making the choices we recommend.

On the matter of using products based on open standards, I'm frankly viewing 
that as desirable rather than mandatory.  The reason is that I am pretty sure we 
do not want to block our use if the only acceptable solution happens to be 
proprietary. Our need is too strong, I believe.

As for choosing functions that go beyond what is already in use, here is the 
reasoning driving the candidate choices I've listed:

      There is a difference between camping and staying at a 5-star hotel. In 
the range of capabilities for remote participation in an meeting, the current 
set of capabilities that we use are closer to the former than the latter.

      Remote participation is currently quite limited in number and style.  We 
are seeking much greater use.

      I believe that seriously supporting truly inclusive participation for 
long-term, regular virtual meetings, the tools need to attain a level of 
capability, integration and ease of use that is considerably better than we 
currently employ.  We need a lower barrier to use and we need better 
functionality to aid a wider range of meeting content and style.

We might not need a 5-star set of tools, but we need a lot more than a tent.

d/

-- 

   Dave Crocker

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 14 May 2009 10:43:13 -0700

> audio. Most everything else flunks Brian's 150 msec test.

This raises two different questions:

1.  Is passing that test really mandatory?

2.  Is it true that no VOIP capabilities pass it today?

I suspect the issue here is going to be about statistics.  How often is there a 
(serious) problem, not whether there is one.  People can be pretty adaptable, 
and cell phone have taught us all to be particularly forgiving of poor voice 
channel characteristics.  There are limits, but they are not necessarily as 
strict as the discussion on this list seems to suggest.

>    3) Best Practices simply don't get followed, unless somebody is

On both of these point, we should note that the community does, in fact, follow 
certain rigid requirements.  I-D submissions deadlines, meeting registration 
deadlines, etc.  When a requirement is enforced, we conform.  We are sloppy 
about enforcement only when the requirement isn't rigid. When the consequences 
of non-conformance are clear and firm, we tend to conform quite readily.

>> especially if travel issues are now motivating more to stay at home.

This is certainly an important point, but it's the best we have to base our 
estimate on, now.

> their place or spoken) is what all participants should have.

I don't understand what you are specifying.  How would it work and how would 
that be easy to use?

> of maintaining it if we can't get it automated.

Again, in terms of broad-based, on-going use I believe we cannot settle for hack 
solutions.  Let's be careful that we do not do the online equivalent of 
requiring each speaker bring their own public address megaphone.

The more manual the mechanism, the more hassle will be its use. To get work done 
on the scale we are seeking, we cannot have the tools impose hassle, IMO.  And 
just because one or another of us is willing to put up with that hassle does not 
make it reasonable to impose on everyone else.

> (bandwidth) common denominator.

The lowest common denominator is 9.6kbps, or worse.  We aren't designing for that.

But a rule of egalitarian access is that is pushes for a norm that is 
considerably below what is available to some.

> are the nature of the beast. I agree with Dave.

There was some push-back about the suggestion for including a white-board 
function.

I have certainly been in meetings where text was created collaboratively and in 
real-time.  Charter-bashing is perhaps the most common example.

Whether that is through a "shared whiteboard" or simple broadcast powerpoint 
revisions is secondary, IMO.  But we need to be able to have materials 
changeable in realtime through collaborative efforts.

>    to help them, but it deserves to be high among our concerns.

Good point.  Yes.

>    human memories to diverge.

I'm not clear what capability you are suggesting, beyond what's already been listed.

>    but there _are_ folks who are good at it.)

We currently have a jabber transcript, with a scribe, and a separate note taker. 
  Are you suggesting something beyond these?

>    deserve far more visibility than they usually get.

What sort of tool is involved in this, different from what is already in use?

>    the screen. I don't want anything else interacting with my Jabber

Interesting point.  The tradeoff is that tools that aren't integrated each 
require separate registration and management.

>    definitely under 500 msec -- we'll probably have to do without it.

The lengthy discussion about these details was facinating, but too many people 
have had entirely too many, acceptable experiences with group conversations that 
were a mix of telephone and voip.

>    Folks simply _won't_ learn how to use them effectively. Formal

Concern about the learning curve is certainly warranted.  However we need to be 
careful about firm, pre-hoc assertions of what is and is not acceptable in terms 
of what the human factors folk call usability.

-- 

   Dave Crocker

---

From: "Brian Rosen" <[email-masked]>
Date: Thu, 14 May 2009 14:11:34 -0400

Again, we have tested delay.  It matters a lot.  As long as you only want to
observe, it's not much of a problem. If you want to participate, it's a big
problem.

Another thing we know is that engineers think it doesn't matter as much as
it does.  This is a well observed phenomenon.  Its why there are SO many bad
systems around, not being used very much.

You will find in many cases that a videoconference is run with the audio
system shut off, and a regular PSTN conference bridge used instead: the lack
of lip sync is not as bad as the delay of normal videoconference system's
audio subsystem, and its delay is not inherent in the audio processing, it's
deliberately delaying the audio to match the video.

While there is no inherent reason we can't get VoIP to work, the
practicalities are pretty impressive, even Cisco can't get it to work well
in WebEx.  This is more of a network engineering issue than a VoIP mechanism
problem.  On a well engineered IP network, VoIP works better than PSTN.  The
Internet, not being particularly well engineered in the large for the things
that VoIP cares about, is problematic.

My main office phone is VoIP from my home office to our PBX.  I have the
best Internet package my cable company offers for residential use.  It works
well most of the time, but definitely not anywhere near as reliable as the
PSTN.  YMMV.  I do wonder why WebEx doesn't have a SIP AoR available to
connect to: VoIP to them means PC audio system.  It's easier to get a good
VoIP experience on a VoIP phone than on a PC.  The PC has the advantage that
most clients have wideband codecs, which is a good thing.

If all you want is one way, VoIP is okay.  When you need two way, it's hard.
We do have some advantages that may make it more likely to work: we have
lots of bandwidth at our end, as well as very well engineered paths into the
Tier 1 networks.  That's half of the problem.  The other half is not so
easily fixed.  The usual way systems overcome network problems is more
buffering.  Buffering causes delay, and we're back to the big problem.  We
should strive to support VoIP, but we cannot rely on it.

Brian

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Dave CROCKER
> Sent: Thursday, May 14, 2009 1:43 PM
> To: John Leslie
> Cc: [email-masked]; [email-masked]
> Subject: Re: [vmeet] Supporting interim meetings
> 
> 
> 
> John Leslie wrote:
> _______________________________________________
> NOTE WELL: This list operates according to
> http://mipassoc.org/dkim/ietf-list-rules.html.
> https://www.ietf.org/mailman/listinfo/vmeet

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 14 May 2009 11:14:50 -0700

Brian Rosen wrote:
> Again, we have tested delay.  It matters a lot.  

I didn't say it didn't.

d/

-- 

   Dave Crocker

---

From: John Leslie <[email-masked]>
Date: Thu, 14 May 2009 18:12:20 -0400

> 1.  Is passing that test really mandatory?

   For Brian, perhaps... Probably not for the rest of us. But I have
experienced too many cases where VoIP flunks it by an order of magnitude
or more. I argue that if we push it by more than a factor of three, we
will be sorry.

> 2.  Is it true that no VOIP capabilities pass it today?

   I don't know, but I suspect that most VoIP capabilities let the
delay exceed one second while chasing other "quality" issues.

   (I don't believe there's any need to let it get that high, BTW.)

> On both of these point, we should note that the community does, in fact, 
> follow certain rigid requirements.  I-D submissions deadlines, meeting 
> registration deadlines, etc.

   These are enforced by programmed tools.

> When a requirement is enforced, we conform.

   The kind of requirements some are recommending depend upon enforcement
by (already overworked) individuals. And many of these individuals report
that their attempts to enforce, e.g. slide deadlines, fail miserably.

> I don't understand what you are specifying.  How would it work and how 
> would that be easy to use?

   I'm not particular how it works. An example would be a separate
Jabber window (and there's no reason an IRC bot couldn't maintain it).

> Again, in terms of broad-based, on-going use I believe we cannot settle
> for hack solutions. Let's be careful that we do not do the online
> equivalent of requiring each speaker bring their own public address
> megaphone.

   Our timeframe is too short to rule out "hacks" for Stockholm. For
the IETF after that, we should be able to automate things.

> The more manual the mechanism, the more hassle will be its use. To get
> work done on the scale we are seeking, we cannot have the tools impose
> hassle, IMO.

   Alas, _any_ new tool will "hassle" somebody.

> And just because one or another of us is willing to put up with that 
> hassle does not make it reasonable to impose on everyone else.

   Nor, of course, _will_ everyone else put up with it...

>> to help them, but it deserves to be high among our concerns.

   Thanks! (Be sure we don't forget this one.)

> I'm not clear what capability you are suggesting, beyond what's already 

   I'm probably over-sensitive on this one, but the principle is real.
The case which sensitizes me is on IESG telechats, the DISCUSS comments
are changing in real-time, and it's hard to exactly match them to what
is said during the telechat. Issues disappear from the Balloting link
and sometimes get lost -- this leads to unnecessary friction where ADs
could become less willing to let go of a DISCUSS.

   What I would _really_ like to see is an ability to "snapshot" a
working document -- whether a whiteboard, a PowerPoint, or whatever --
to preserve its state at the point an action was agreed.

> We currently have a jabber transcript, with a scribe, and a separate note 
> taker. Are you suggesting something beyond these?

   Yes: almost anything beyond those would help. The Jabber transcript
contains a lot of irrelevant, transitory information; and often misses
the points which deserve to be preserved. The separate Note Taker at
least has a responsibility to record the things which deserve to be
preserved, but these notes seldom appear quickly and too often get lost
entirely.

   If there _were_ a Real-Time-Text "record" -- separate from the rest
of a jabber session, it wouldn't get lost and folks could comment on
perceived errors and omissions in time to fix them.

   (I realize there are a few Jabber Scribes that do this (except for
the separate window), but we never know whether we'll get one of those
until it's too late to fix the absence.)

> What sort of tool is involved in this, different from what is already in 

   Separate Window (or combined with my "record" window. (The amount of
typing is small enough that we could almost make this a Chair duty;
but of course it would be better as a WG-Secretary duty.)

> require separate registration and management.

   I've been running multiple applications on my laptop for years:
what's the problem?

> The lengthy discussion about these details was facinating, but too many 
> people have had entirely too many, acceptable experiences with group 
> conversations that were a mix of telephone and voip.

   Someone who is "only" listening can tolerate 5-second delays. But
to "participate" requires listening _and_ speaking. 150 msec is about
the upper limit on "echo-free" listening. Get it above one second, and
few of us can both listen and speak freely.

   I experienced 5-second delays between POTS and VoIP connections to
webex. The 5-second is way past acceptable for actual participation.

   I do not believe it has to be that bad, but I've had that-bad
experiences with every packaged tool I've tried so far.

   No number of people who haven't experienced multi-second delays can
cancel the confusion this causes. At the very least, we need a warning
label if we try to combine VoIP and POTS in a single group.

   (It might be possible to have folks switch between VoIP for listen
mode and POTS for participate mode, but they'd need to know how and
allow time for switching...)

>> Folks simply _won't_ learn how to use them effectively. 
> 
> Concern about the learning curve is certainly warranted.  However we
> need to be careful about firm, pre-hoc assertions of what is and is
> not acceptable in terms of what the human factors folk call usability.

   I don't expect folks to agree with Brian about 150 msec. (But he's
right.) It's the other end of the spectrum which worries me: folks
who think that everyone can "just adjust" to ten-second delays will
make it nearly impossible for newcomers if we set the bar at ten
seconds.

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 14 May 2009 15:29:37 -0700

> the IETF after that, we should be able to automate things.

The timeframe for vmeet is whatever we set it to be.  My own participation came 
from suggesting that we work towards replacing at least one IETF week with 
virtual meetings.  There is no way that is happening immediately.  It's a goal, 
but we are not trying to meet it instantly.

Since there already are virtual interim meetings, here too we do not have to 
have an "immediate" deadline.

That's why I think we can and should target capabilities beyond what the IETF 
currently use, balancing against easy, reliable use so that folks will not feel 
that it is a burden to participate.

d/

-- 

   Dave Crocker

---

From: John Leslie <[email-masked]>
Date: Thu, 14 May 2009 20:04:26 -0400

Dave CROCKER <[email-masked]> wrote:
> The timeframe for vmeet is whatever we set it to be.

   Not meaning to contradict, but Russ Housley has committed to make
the Stockholm IETF week "remote-participation friendly". I do believe
he is hoping from some assistance from us.

> My own participation came from suggesting that we work towards
> replacing at least one IETF week with virtual meetings.

   A noble goal, according to my bank account! ;^)

   (Do note, however, that the Secretariat is signing contracts for
2013...)

> There is no way that is happening immediately. It's a goal, but we are
> not trying to meet it instantly.

   Absolutely!

> Since there already are virtual interim meetings, here too we do not
> have to have an "immediate" deadline.

   I wasn't thinking of a "deadline" -- more a "milestone".

> That's why I think we can and should target capabilities beyond what
> the IETF currently use, balancing against easy, reliable use so that
> folks will not feel that it is a burden to participate.

   Of course, if we're _really_ thinking "after 2013," we can afford
to concentrate more on open-standard tools than on currently-marketed
proprietary tools...

   Maybe we need to "formalize" some informal goals / milestones?

---

From: Doug Otis <[email-masked]>
Date: Thu, 14 May 2009 19:27:15 -0700

On May 14, 2009, at 5:04 PM, John Leslie wrote:

> the Stockholm IETF week "remote-participation friendly". I do  
> believe he is hoping from some assistance from us.

My hope as well.

>   (Do note, however, that the Secretariat is signing contracts for  

A virtual meeting where all recipients are remote and utilize a phone  
bridge represents an environment where suitable solutions already  
exist and only need to be selected.  Costs for most of these solutions  
supplant a typical hotel room fee.

>> There is no way that is happening immediately. It's a goal, but we  
>> are not trying to meet it instantly.
>
>   Absolutely!

Disagree.  There are solutions today that should suffice for interim  
meeting requirements.  There are weaknesses in some of these schemes.   
The WebEx Event product offers a Question/Answer window that is  
different from that of the WebEx Meeting demonstrated in the trial.   
However, while the client for the Event Center supports integrated  
VoIP, this feature is not ported to OSX.  VoIP should be okay when the  
person speaking is highly regimented (the only participant not muted),  
otherwise audio quality suffers.  For this, the WebEx Event product  
can start participants initially muted.

For smaller groups, WebEx Meeting using a telephone bridge works well,  
but it does not offer a Q & A window.  Raising a hand icon is only  
seen by the presenter.  This will likely frustrate participants who  
would like to make a point, but feel ignored by a likely distracted  
host.  A question queue seems an important requirement to assure  
participants they are being given a fair chance.  Otherwise, the floor  
is likely to ceded to those willing to ask for forgiveness rather than  
permission.  The chat and window control for remote participants does  
not compare favorably to a simpler jabber client.  On the other hand,  
the polling feature is very nice and allows participation in "hums."

>> Since there already are virtual interim meetings, here too we do  
>> not have to have an "immediate" deadline.
>
>> That's why I think we can and should target capabilities beyond  
>> what the IETF currently use, balancing against easy, reliable use  
>> so that folks will not feel that it is a burden to participate.
>
> to concentrate more on open-standard tools than on currently- 
> marketed proprietary tools...

The focus should be on what it takes to standardize on the video  
presentation, as that should be something well suited for low  
bandwidth streaming.  In addition, something like Vonage or Skype feed  
to the room PA can support remote presentations.   Perhaps a voice- 
mail feature at a separate number would also provide a type of  
microphone queue.   A separate line might allow a presenter a means to  
request further elaboration and to have it exchanged verbally.  Two  
VoIP lines supported by each net-box should represent less cost and  
bandwidth than a phone bridge.  A phone bridge intended for non- 
regimented exchanges is likely to be problematic when dealing with  
VoIP or cellular phones.

>   Maybe we need to "formalize" some informal goals / milestones?

Every meeting room uses a digital projector to display sides.  A net- 
box attached to each of these projectors offers a means to make remote/ 
local video sources common.  In this way, presentation material can be  
updated at the last minute, and presenters would not be required to  
read slide numbers (which is rarely done).

To support 15 rooms, hardware costs would be from $4,500 to $10,000  
that range from Asus N202 to Mac minis.  Streaming a sequence of  
changed gif files can capture a presentation window and not require  
much bandwidth for fairly static images.  There are many examples of  
this type of technique demonstrated on various exotic sites for  
example.  :^)  WebEx and Adobe Connect use Flash which works well for  
this purpose, but is not an open standard, albeit widely supported.

The immediate goal should be to support  "remote-participation  
friendly" meetings in Stockholm.  This really seems to imply  
standardizing on video slide and audio streaming.  Remote speakers or  
presenters can be supported by VoIP output by the projector netbox  
selectively enabled into the room PA.  XMPP should fill any gaps.

Interim meetings represent an entirely different environment and  
problem set.  Interim meetings may eventually become a selection of  
existing remote meeting products.  the WebEx Event center looks well  
suited, but the VoIP for OSX should be fixed in order to deal with  
large groups.

-Doug

---

From: Dave Cridland <[email-masked]>
Date: Fri, 15 May 2009 17:06:10 +0100

On Fri May 15 03:27:15 2009, Doug Otis wrote:
> For smaller groups, WebEx Meeting using a telephone bridge works  
> well,  but it does not offer a Q & A window.  Raising a hand icon  
> is only  seen by the presenter.  This will likely frustrate  
> participants who  would like to make a point, but feel ignored by a  
> likely distracted  host.  A question queue seems an important  
> requirement to assure  participants they are being given a fair  
> chance.  Otherwise, the floor  is likely to ceded to those willing  
> to ask for forgiveness rather than  permission.  The chat and  
> window control for remote participants does  not compare favorably  
> to a simpler jabber client.  On the other hand,  the polling  
> feature is very nice and allows participation in "hums."

FWIW, some existing XMPP clients use the MUC presence to signal  
voting, and they could just as easily signal a "I'd like the floor".  
However, the current usage of the MUC rooms as a "back channel", for  
side-discussions, makes actual floor control tricky - XEP-0045 does  
include "voice requests" and control, and the implementation the IETF  
uses supports these, I believe, but I'd consider removal of voice to  
be a problem, unless each working group had both a "presentation" MUC  
*and* a backchannel, which becomes unweidly quite quickly.

Incidentally, my guess would be that a voting/hand-raising  
specification or two could be written, and implemented, by Stockholm,  
in cross-platform clients. If you want this to happen, please say so,  
and I'll be happy to drive this forward.

There's scope for signalling other things through MUC, too, but  
anything that requires server mediation has a longer turn-around  
time, obviously. To get a client-client signalling mechanism going  
only needs, after all, one client developer to bite.

Dave.

---

From: "Brian Rosen" <[email-masked]>
Date: Fri, 15 May 2009 12:19:22 -0400

What I want is a physical light on a pole that lights up when a webex or
other "hand raise" occurs.  I'd also like, but don't expect, a sensor on the
microphones to reflect someone standing at the mic as a hand raise in WebEx.
The former allows the chair, or any other participant in the room, to see at
a glance that a remote participant is queued at the "mic".   The latter
allows a remote participant to know that there is a local participant
queued.  The latter could be integrated into the "who is at the mic"
mechanism (?RFID tag) that has been tried.

I think we could hack the light on a pole fairly easily if WebEx has an API.
Does it?

Brian

> Of Dave Cridland
> Sent: Friday, May 15, 2009 12:06 PM
> To: Doug Otis; Dave CROCKER; IETF remote participation meeting services
> discussion; John Leslie
> Subject: Re: [vmeet] timeframe
>

---

From: Thomas Narten <[email-masked]>
Date: Sun, 17 May 2009 15:54:12 -0400

> tool that folks aren't used to, they'll gravitate back to _that_ model

Actually, I wasn't just reporting what we have today. I was thinking
of my participation in remote meetings and the parts I considered
necessary to participate.

> last minute is when it, almost by definition, happens.

Absolutely. I am assuming that advance preparation would be
enforced. For something scheduled out in advance (like a WG meeting),
the idea that charts are being editted in real time minutes before the
actualy presentation is flawwed. This is poor work habits of the
presentor, not the nature of how one prepares for meetings.

...

> are the nature of the beast. I agree with Dave.

Not for the kinds of meetings we are talking about. We are talking
about WG meetings that are scheduled weeks out in advance. Things
shouldn't be in serious flux moments before the call... There may be
exceptions, but in general charts can be done a few days out in
advance of a meeting.

> design for the normal case.

Actually, I've done this fairly often. It has advantages to following
a meeting in real-time (one can pause, rewind, etc.) in cases where
one is not expecting to actually contribute/speak during the meeting.

Thomas

---

From: Doug Otis <[email-masked]>
Date: Mon, 18 May 2009 15:09:33 -0700

> exceptions, but in general charts can be done a few days out in  

A meeting schedule does not ensure presentation stability and  
technology allows presentations to reflect recent discussions.  Many  
will clarify where discussions have advanced, and some progress occurs  
just prior to meetings.  As with their own names, many tend to forget  
to state which side is being projected as well.  Streaming static  
images that are being projected seems practical, especially when the  
device connected to the projector is standardized.

> Actually, I've done this fairly often. It has advantages to  
> following a meeting in real-time (one can pause, rewind, etc.) in  
> cases where one is not expecting to actually contribute/speak during  
> the meeting.

One might wonder whether a voice-mail feature could serve as a type of  
remote question queue.  Someone might listen to messages privately to  
avoid repeated questions.  A dynamic phone list could allow callbacks  
when further elaboration of a question is desired.  The same callback  
mechanism might support audio for remote presentations as well.

Stay focused on enhancing remote participation with IETF meetings.   
There are many suitable solutions for interim meetings where  
participants are all remote.

-Doug

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 18 May 2009 16:12:10 -0600

On 5/18/09 4:09 PM, Doug Otis wrote:
>>>   But last-minute changes, even changes _during_ the presentation are
>>> the nature of the beast. I agree with Dave.
>>
> A meeting schedule does not ensure presentation stability and technology
> allows presentations to reflect recent discussions.  Many will clarify
> where discussions have advanced, and some progress occurs just prior to
> meetings.  As with their own names, many tend to forget to state which
> side is being projected as well.  Streaming static images that are being
> projected seems practical, especially when the device connected to the
> projector is standardized.

Perhaps we are in part seeking technological solutions to social
problems? I know that I too have been guilty of making slides at the
last minute, but if the WG chairs set the expectation that if you don't
provide slides X hours or days ahead of time then you won't be speaking,
perhaps presenters would provide their slides and folks who are
participating remotely would have what they need.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Doug Otis <[email-masked]>
Date: Mon, 18 May 2009 16:03:48 -0700

On May 18, 2009, at 3:12 PM, Peter Saint-Andre wrote:

> problems?

Whatever the cause, social or human-nature, not knowing what is being  
presented makes for poor remote experiences.  Time-stamps on video  
streams could allow merging audio streams for later combined  
presentations.

> I know that I too have been guilty of making slides at the last  
> minute, but if the WG chairs set the expectation that if you don't  
> provide slides X hours or days ahead of time then you won't be  
> speaking, perhaps presenters would provide their slides and folks  
> who are participating remotely would have what they need.

The WG chairs are people too, who are likely making this exception for  
themselves.  The cost of a device connected to projectors for creating  
and streaming static video images is fairly minor, >$300.  Requiring  
use of these devices ensures presentation materials are retained by  
the IETF.  In addition, placing presentation material on projector  
devices better ensures time is not lost moving projector cables or  
fumbling with screen resolutions.

Projector devices can access file shares on the local network, where  
WG chairs would transfer presentations to the IETF file shares being  
made available both locally and remotely.  Of course, it would be  
safer to have presentations available some period prior to the  
meeting.  The requested advanced period might be based upon durations  
of typical IETF meeting network outages.

-Doug