Subject: Kudos to MeetEcho

From: Chris Inacio <[email-masked]>
Date: Mon, 27 Jul 2020 14:07:53 +0000

I just want to say that I finished chairing a WG this morning and to say kudos to MeetEcho. Since it seems all I do is spend time in (many different) video conferencing apps, I think the updates to MeetEcho for the full virtual meeting are VERY impressive in the time from “Vancouver” till now.

It went smoother than I would have thought. I think MeetEcho did an amazing job. Thanks to them and all the leadership making the tooling work for this meeting.

There is the human problem of people reading the Jabber chat continuously during the meeting, because it really is now multiple stream communications during the meeting. If I knew the human / UI fix to that, I would suggest it, I don’t. I chair a small working group, so we can multitask that way. Large working groups will like have to the use the queue more formally than we did - with the chairs possibly voicing the jabber room.

--
Chris Inacio
[email-masked]

---

From: Richard Barnes <[email-masked]>
Date: Mon, 27 Jul 2020 10:15:12 -0400

While there has been a lot of progress since 107, it is clear from the
first couple of sessions that there are still a many rough edges that are
wasting a lot of WG time.  We should take a serious look at whether it's
worth investing more in refining this tool, or whether a more standard tool
would be appropriate.

On Mon, Jul 27, 2020 at 10:08 AM Chris Inacio <[email-masked]> wrote:

> I just want to say that I finished chairing a WG this morning and to say
> kudos to MeetEcho. Since it seems all I do is spend time in (many
> different) video conferencing apps, I think the updates to MeetEcho for the
> full virtual meeting are VERY impressive in the time from “Vancouver” till
> now.
>
> It went smoother than I would have thought. I think MeetEcho did an
> amazing job. Thanks to them and all the leadership making the tooling work
> for this meeting.
>
> There is the human problem of people reading the Jabber chat continuously
> during the meeting, because it really is now multiple stream communications
> during the meeting. If I knew the human / UI fix to that, I would suggest
> it, I don’t. I chair a small working group, so we can multitask that way.
> Large working groups will like have to the use the queue more formally than
> we did - with the chairs possibly voicing the jabber room.
>
>
>
> --
> Chris Inacio
> [email-masked]
>
>

---

From: Jared Mauch <[email-masked]>
Date: Mon, 27 Jul 2020 10:22:17 -0400

> On Jul 27, 2020, at 10:15 AM, Richard Barnes <[email-masked]> wrote:
> 
> While there has been a lot of progress since 107, it is clear from the first couple of sessions that there are still a many rough edges that are wasting a lot of WG time.  We should take a serious look at whether it's worth investing more in refining this tool, or whether a more standard tool would be appropriate.
> 

I disagree, there are often rough edges around the first time you use any tool, some of this is PBCAK and some is UI, etc.

I have some suggestions to improve it, but the reality is this is a huge improvement and seems to be working well.

- Jared

Ps: suggestions follow 

- remember that I did data tracker login
- show time remaining in meeting (countdown)
- if I’ve scrolled to bottom of chat, and more people type, keep scrolling down so I don’t need to scroll further down

---

From: Daniele Ceccarelli <[email-masked]>
Date: Mon, 27 Jul 2020 14:25:30 +0000

Which ”standard” tool would allow us having totally customized features like e.g. humming or queueing?...or sometimes even the button colored green rather than purple?
Personally I use “standard” tools every day and I don’t feel this any worse than any of them.
This is the first all virtual meeting and like any other product in release 1 there are bugs to fix.

BR
Daniele

From: ietf <[email-masked]> On Behalf Of Richard Barnes
Sent: den 27 juli 2020 16:15
To: Chris Inacio <[email-masked]>
Cc: IETF-Discussion Discussion <[email-masked]>
Subject: Re: Kudos to MeetEcho

On Mon, Jul 27, 2020 at 10:08 AM Chris Inacio <[email-masked]<mailto:[email-masked]>> wrote:

--
Chris Inacio
[email-masked]<mailto:[email-masked]>

---

From: Richard Barnes <[email-masked]>
Date: Mon, 27 Jul 2020 10:32:13 -0400

On Mon, Jul 27, 2020 at 10:25 AM Daniele Ceccarelli <
[email-masked]> wrote:

> Which ”standard” tool would allow us having totally customized features
> like e.g. humming or queueing?...or sometimes even the button colored green
> rather than purple?
>

We should be minimizing our dependence on customized features.

> Personally I use “standard” tools every day and I don’t feel this any

Jared already mentioned two ways in which this tool is worse than standard
(proper SSO, chat scrolling).

--RLB

> *From:* ietf <[email-masked]> *On Behalf Of *Richard Barnes
> *Sent:* den 27 juli 2020 16:15
> *To:* Chris Inacio <[email-masked]>
> *Cc:* IETF-Discussion Discussion <[email-masked]>
> *Subject:* Re: Kudos to MeetEcho
>
>
>

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 27 Jul 2020 15:35:15 +0100

The core-context approach would suggest that we just buy something off the shelf.

That said the ITU are meeting in parallel using a platform with similar properties which is either bespoke or customised COTS. It would be a good idea if the SDOs working on communications technologies worked together to share the costs of these platforms.

Stewart

> On 27 Jul 2020, at 15:15, Richard Barnes <[email-masked]> wrote:
> 
> On Mon, Jul 27, 2020 at 10:08 AM Chris Inacio <[email-masked] <mailto:[email-masked]>> wrote:
> [email-masked] <mailto:[email-masked]>
> 
> 
>

---

From: "Peter Yee" <[email-masked]>
Date: Mon, 27 Jul 2020 07:36:46 -0700

I participated in the session that Chris chaired and my perceptions were similar to his. Everything went quite well and that was my first use of MeetEcho (I missed out on the test session). My second session was also pretty good. There were some minor issues with the third session I’m currently in, but that seems to be down to lack of experience and preparation, not so much the tool.

                                -Peter

Richard Barnes writes:

 

--
Chris Inacio
[email-masked]

---

From: Richard Barnes <[email-masked]>
Date: Mon, 27 Jul 2020 10:39:48 -0400

Not sure what session you were in, but DISPATCH spent literally about 15%
of its time figuring out how to navigate the multi-phase commit to let
someone comment from the floor.

On Mon, Jul 27, 2020 at 10:35 AM Peter Yee <[email-masked]> wrote:

> I participated in the session that Chris chaired and my perceptions were
> similar to his. Everything went quite well and that was my first use of
> MeetEcho (I missed out on the test session). My second session was also
> pretty good. There were some minor issues with the third session I’m
> currently in, but that seems to be down to lack of experience and
> preparation, not so much the tool.
>
>
>
>                                 -Peter
>
>
>
> Richard Barnes writes:
>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 27 Jul 2020 07:45:01 -0700

On Mon, Jul 27, 2020 at 7:17 AM Richard Barnes <[email-masked]> wrote:

> wasting a lot of WG time.
>

I am noticing this as well.

In particular, there are a lot of both glitches and ways in which it is
different from the idioms used by other conferencing tools.

-Ekr

---

From: Simon Pietro Romano <[email-masked]>
Date: Mon, 27 Jul 2020 16:48:21 +0200

> While there has been a lot of progress since 107, it is clear from the first couple of sessions that there are still a many rough edges that are wasting a lot of WG time.  We should take a serious…

It might look weird to request such a clarification on an IETF mailing list, but I am afraid we should first agree on the correct meaning of *standard* here.

Simon

                     				            _\\|//_
                           				   ( O-O )
      ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~
                    				Simon Pietro Romano
             				 Universita' di Napoli Federico II
                		     Computer Engineering Department 
	             Phone: +39 081 7683823 -- Fax: +39 081 7683816
                                           e-mail: [email-masked]

		    <<Molti mi dicono che lo scoraggiamento è l'alibi degli 
		    idioti. Ci rifletto un istante; e mi scoraggio>>. Magritte.
               			                     oooO
       ~~~~~~~~~~~~~~~~~~~~~~~(   )~~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~
					                 \ (            (   )
			                                  \_)          ) /
                                                                       (_/

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 27 Jul 2020 15:52:07 +0100

Chris Inacio wrote on 27/07/2020 15:07:
> amazing job. Thanks to them and all the leadership making the tooling 
> work for this meeting.

sidrops was, sadly, unusable. the mp3 feed was dropping out every 1-2 
seconds.

Nick

---

From: Richard Barnes <[email-masked]>
Date: Mon, 27 Jul 2020 10:52:56 -0400

On Mon, Jul 27, 2020 at 10:48 AM Simon Pietro Romano <[email-masked]>
wrote:

> It might look weird to request such a clarification on an IETF mailing
> list, but I am afraid we should first agree on the correct meaning of
> *standard* here.
>

I would start with something like >1m DAU.  That would give us tools that
are broadly used (thus likely to have fixed the sort of rough edges we see
here), but still leave several options to choose from.

> <[email-masked]>

---

From: Simon Pietro Romano <[email-masked]>
Date: Mon, 27 Jul 2020 17:00:58 +0200

I see. This confirms my doubts, since I had different standards in mind.

Simon

                     				            _\\|//_
                           				   ( O-O )
	             			     Phone: +39 081 7683823 

> Il giorno 27 lug 2020, alle ore 16:52, Richard Barnes <[email-masked]> ha scritto:
> 
> On Mon, Jul 27, 2020 at 10:48 AM Simon Pietro Romano <[email-masked] <mailto:[email-masked]>> wrote:
> I would start with something like >1m DAU.  That would give us tools that are broadly used (thus likely to have fixed the sort of rough edges we see here), but still leave several options to choose from.
> 
> 
>  
> 
> Simon
> 
>                      				            _\\|//_
>                            				   ( O-O )
>                                            e-mail: [email-masked] <mailto:[email-masked]>
>

---

From: Russ Housley <[email-masked]>
Date: Mon, 27 Jul 2020 11:02:28 -0400

Nick:

> sidrops was, sadly, unusable. the mp3 feed was dropping out every 1-2 seconds.

I was in SIDRops, and it works well for me.

Russ

---

From: Richard Barnes <[email-masked]>
Date: Mon, 27 Jul 2020 11:10:00 -0400

Just to be clear, this is not just bigness bias.  There are real concerns
here:

1. Productivity -- The "rough edges" we're talking about here are really
expensive in units of lost meeting time.  More widely-used tools are more
likely to have obvious problems fixed.
2. Inclusiveness -- The higher the bar is to using a tool, the harder it is
for people to participate effectively.  More widely-used tools are more
likely to be things people have experience with, and more likely to adhere
to commonly-used idioms.

---

From: "Maisonneuve, Julien (Nokia - FR/Paris-Saclay)" <[email-masked]>
Date: Mon, 27 Jul 2020 15:23:35 +0000

On  Monday, July 27, 2020 4:35 PM Stewart Bryant wrote
The core-context approach would suggest that we just buy something off the shelf.
That said the ITU are meeting in parallel using a platform with similar properties which is either bespoke or customised COTS. It would be a good idea if the SDOs working on communications technologies worked together to share the costs of these platforms.
Indeed.
However having to follow both meetings, I can testify that Meetecho performs far better than ITU’s MyMeetings, notably for audio, even when video is used. MeetEcho is also more flexible on what it displays.
So not perfect, but better than some of the competition.
JM.

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 27 Jul 2020 15:30:38 +0000

I have used several commercial offerings, and find the differences in usability to be negligible.  They all stink.  But it reminds me of the crude comment about opinions and anatomy.

---

From: Jared Mauch <[email-masked]>
Date: Mon, 27 Jul 2020 11:36:17 -0400

> On Jul 27, 2020, at 10:52 AM, Nick Hilliard <[email-masked]> wrote:
> 

I used the meet echo web/video for this meeting and aside from the people who had issues sending audio, it seemed to work well.

Perhaps we need to turn off/limit choices (eg: mp3) that are not reliable.

I’d rather have one really good tool than several ways to do the same thing (eg: jabber + meet echo chat + relay to slack + relay to satellite).

- Jared

---

From: Meetecho IETF support <[email-masked]>
Date: Mon, 27 Jul 2020 17:45:12 +0200

> Perhaps we need to turn off/limit choices (eg: mp3) that are not reliable.

There has been a room conflict that broke the mp3 stream. Apologies 
about that. We now put a fence that should avoid this.

The Meetecho team

---

From: "Pete Resnick" <[email-masked]>
Date: Mon, 27 Jul 2020 10:54:42 -0500

On 27 Jul 2020, at 9:22, Jared Mauch wrote:

>> While there has been a lot of progress since 107, it is clear from 
>> the first couple of sessions that there are still a many rough edges 
>> that are wasting a lot of WG time.  We should take a serious look at 
>> whether it's worth investing more in refining this tool, or whether a 
>> more standard tool would be appropriate.
>
> I disagree, there are often rough edges around the first time you use 
> any tool, some of this is PBCAK and some is UI, etc.
>
> I have some suggestions to improve it, but the reality is this is a 
> huge improvement and seems to be working well.

FWIW: I agree with Jared here. There are UI issues and problems that 
need to be (and I think can be) addressed (I'm making my own list), but 
I think the overall design and performance of Meetecho are much better 
suited to our uses than any of the other "standard" tools that I have 
used. So I do think that it's worth investing more in refining this 
tool.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 27 Jul 2020 17:39:50 +0100

> about that. We now put a fence that should avoid this.

ok cool - thanks for fixing.

Nick

---

From: Michael Richardson <[email-masked]>
Date: Mon, 27 Jul 2020 14:01:54 -0400

Chris Inacio <[email-masked]> wrote:
    > There is the human problem of people reading the Jabber chat
    > continuously during the meeting, because it really is now multiple
    > stream communications during the meeting. If I knew the human / UI fix
    > to that, I would suggest it, I don’t. I chair a small working group, so
    > we can multitask that way. Large working groups will like have to the
    > use the queue more formally than we did - with the chairs possibly
    > voicing the jabber room.

The problem is that the meetecho jabber is tall, rather than wide.
Don't do that. It cause it to scroll too fast.  I think that it also
auto-scrolls as new stuff arrives.

That's why I use a proper jabber client, and I make it wide, so most lines
never wrap.  And it doesn't auto-scroll unless I tell it to.

{I understand why we integrate jabber into meetecho: it's to be nice to
newbies who need it all in one place.  That doesn't mean others should use it}

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Hugo Salgado <[email-masked]>
Date: Mon, 27 Jul 2020 14:16:07 -0400

> newbies who need it all in one place.  That doesn't mean others should use it}

Yes, it should be an option to not use the built-in
jabber for the oldies. I have to change my name in my
normal jabber client because conflicts with the
meetecho one.

Hugo

---

From: Randy Bush <[email-masked]>
Date: Mon, 27 Jul 2020 11:27:00 -0700

> We should be minimizing our dependence on customized features.

why?  so we can be dependent on webex?  pfui!

randy

---

From: Richard Barnes <[email-masked]>
Date: Mon, 27 Jul 2020 14:28:44 -0400

> why?  so we can be dependent on webex?  pfui!

On the contrary, the more special features we have, the more locked in we
are to the tools with those special features, and the less flexibility we
have to adapt with the times / budgets / needs of the community.

---

From: Randy Bush <[email-masked]>
Date: Mon, 27 Jul 2020 11:31:12 -0700

> I was in SIDRops, and it works well for me.

same here.  same with other wgs.

the gl!tches were presenters/users who had not bothered to go to the
intro session.

i suggest having the analog of a "speaker's room" where folk who plan to
present can pretest.  this is not a change to meetecho, but how we use
it.  if i am to present in a f2f meeting, i test beforehand.

randy

---

From: Ben Campbell <[email-masked]>
Date: Mon, 27 Jul 2020 21:10:16 -0500

In all fairness, DISPATCH was in the first session on Monday morning, at a time when the chairs would normally be asleep. (at least this chair.) We had little opportunity to learn from the mistakes of others :-)

IMO, the biggest time eaters were involved individual participants with media problem. These seemed to be caused by browser choices, or by changing media devices after joining. Hopefully future versions of Meetecho will deal with both more gracefully.

Ben.

> On Jul 27, 2020, at 9:39 AM, Richard Barnes <[email-masked]> wrote:
> 
> Not sure what session you were in, but DISPATCH spent literally about 15% of its time figuring out how to navigate the multi-phase commit to let someone comment from the floor.
> 
> On Mon, Jul 27, 2020 at 10:35 AM Peter Yee <[email-masked] <mailto:[email-masked]>> wrote:

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 28 Jul 2020 12:35:04 +1000

s/browser choice/compatibility of the tool with different browsers/.

Let’s not blame the user for the limitations of the system we’ve chosen. 

Sent from my iPhone

> On 28 Jul 2020, at 12:10 pm, Ben Campbell <[email-masked]> wrote:
>

---

From: Ben Campbell <[email-masked]>
Date: Mon, 27 Jul 2020 21:39:32 -0500

Certainly, that was not my intention. I personally default to Safari, and have more than once had to exit and reenter the room because I forgot to select “Open with…”

It’s not clear to me that Meetecho can be blamed for the inconsistent WebRTC implementations among browser vendors—but it is very clear that users are not to blame.

Ben.

> On Jul 27, 2020, at 9:35 PM, Mark Nottingham <[email-masked]> wrote:
> 
>>> On Jul 27, 2020, at 9:39 AM, Richard Barnes <[email-masked] <mailto:[email-masked]>> wrote:
>>>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 28 Jul 2020 12:50:46 +1000

Ultimately, we share some collective responsibility; they came here to standardize the thing, after all.  

Sent from my iPhone

> On 28 Jul 2020, at 12:39 pm, Ben Campbell <[email-masked]> wrote:
>

---

From: Ben Campbell <[email-masked]>
Date: Mon, 27 Jul 2020 21:52:29 -0500

Fair point. I guess I’m compartmentalizing the blame.

> On Jul 27, 2020, at 9:50 PM, Mark Nottingham <[email-masked]> wrote:
> 
>>> On Jul 27, 2020, at 9:35 PM, Mark Nottingham <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>>> On 28 Jul 2020, at 12:10 pm, Ben Campbell <[email-masked] <mailto:[email-masked]>> wrote:
>>>>

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 27 Jul 2020 20:30:57 -0700

> Fair point. I guess I’m compartmentalizing the blame.

Well, before we apportion the blame it would be useful to know what the
source of the problem is. Do we have bug reports that describe the defects
that are causing problems?

-Ekr

> Certainly, that was not my intention. I personally default to Safari, and
> have more than once had to exit and reenter the room because I forgot to
> select “Open with…”
>
> It’s not clear to me that Meetecho can be blamed for the inconsistent
> WebRTC implementations among browser vendors—but it is very clear that
> users are not to blame.
>
> Ben.
>
> In all fairness, DISPATCH was in the first session on Monday morning, at
> a time when the chairs would normally be asleep. (at least this chair.) We
> had little opportunity to learn from the mistakes of others :-)
>
> IMO, the biggest time eaters were involved individual participants with
> media problem. These seemed to be caused by browser choices, or by changing
> media devices after joining. Hopefully future versions of Meetecho will
> deal with both more gracefully.
>
> Ben.
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 28 Jul 2020 07:30:07 +0200

On 2020-07-27, at 20:28, Richard Barnes <[email-masked]> wrote:
> On the contrary, the more special features we have, the more locked in we are to the tools with those special features, and the less flexibility we have to adapt with the times / budgets / needs of the community.

I would agree with the sentiment on anything that is not essential to our DNA.
We probably should not use specialized accounting software.
But running our internet-drafts repository on Typo3 (*), while it also would get rid of dependencies on customized features, is not what we should do.

Web meetings that are replacing the traditional IETF week are essential to our DNA.
Different from running an insurance company or anything else that has cookie cutter meetings, these really should be based on situated software [1].
Fortunately that can be built on a standard: The Web + WebRTC.
(Unfortunately, implementations of our own dogfood aren’t fully cooked, but that is a problem with any web-meeting software.  Also, A/V fundamentally is hard, most people have little education in this space, and a lot of garbage hardware gets in the way.)

I think we are quite close to where we should be with meetecho.  
We are maybe not doing the development in a sufficiently agile way.
Of course many reasons can be cited why that is hard to do right now, but we should strive for agility.
It is also not clear that we own what we develop; RFC 873 applies.

Grüße, Carsten

(*) I’m sorry, not everyone here has used Typo3, so that sentence may not invoke the horror it should.
[1]: https://en.wikipedia.org/wiki/Situational_application
(The renaming doesn’t get the point.  Well, I hope you do.)

---

From: Richard Barnes <[email-masked]>
Date: Tue, 28 Jul 2020 07:17:14 -0400

This morning I feel quite far from where we should be with MeetEcho, since
I am unable to access any of the sessions ("Unauthorized").  I've heard out
of band that others are having similar problems.  It appears to be a
Datatracker capacity issue compounded by MeetEcho's failure to do SSO
properly (and thus 100x'ing DT load).  Two bespoke tools conspiring to trip
us up.

The point being that these bespoke tools have a cost, not just in units of
dollars, but also in choice, reliability, etc.  We should think hard about
what is so essential in our DNA that it merits all the costs.

On Tue, Jul 28, 2020 at 1:30 AM Carsten Bormann <[email-masked]> wrote:

> > On the contrary, the more special features we have, the more locked in
> we are to the tools with those special features, and the less flexibility
> we have to adapt with the times / budgets / needs of the community.
>
> But running our internet-drafts repository on Typo3 (*), while it also
> would get rid of dependencies on customized features, is not what we should
> do.
>
> Different from running an insurance company or anything else that has
> cookie cutter meetings, these really should be based on situated software
> [1].
> (Unfortunately, implementations of our own dogfood aren’t fully cooked,
> but that is a problem with any web-meeting software.  Also, A/V
> fundamentally is hard, most people have little education in this space, and
> a lot of garbage hardware gets in the way.)
>

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 28 Jul 2020 04:23:15 -0700

> that are causing problems?

The documentation at
https://www.ietf.org/media/documents/Documentation-Meetecho-IETF.pdf
already refers to some limitations about Safari (only allows sharing whole
screen, not individual windows).  I'm not sure whether there were any other
issues people were seeing.

-Ben

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 28 Jul 2020 13:42:57 +0200

> properly (and thus 100x'ing DT load).  Two bespoke tools conspiring to trip

And a lot of FUD not based on reality.

This morning, the maximum number of connections per second to the datatracker
has been 12, and we are currently set up to handle 100 in parallel.  Average
response time including slow pages is ~0.8s.

There's been no 100x of load, and Meetecho is following the OpenID Connect
protocol perfectly.

> what is so essential in our DNA that it merits all the costs.

And base decisions and engineering on real data, instead of guesswork.

	Henrik

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 28 Jul 2020 04:54:09 -0700

> And base decisions and engineering on real data, instead of guesswork.

I have no special insight in what is happening, but I would make two points:

1. A number of people are experiencing authorization failures (as Richard
reports)
2. For some reason, Meetecho seems to be re-contacting the datatracker
every time the user joins a new session rather than remembering that the
user is authenticated. This seems like it potentially exacerbates (1),

-Ekr

---

From: Richard Barnes <[email-masked]>
Date: Tue, 28 Jul 2020 07:54:35 -0400

> response time including slow pages is ~0.8s.

The fact that this had to be manually adjusted at meeting time, apparently
not having been scale-tested in advance, is exactly the problem here.

> There's been no 100x of load, and Meetecho is following the OpenID Connect

To be explicit about what I mean: Normal SSO services cache login results
so that they don't have to hit the OpenID endpoint every time.  MeetEcho
apparently does not, because it requires login for each session.  So every
time a session starts, MeetEcho is slamming the datatracker with login
requests, when it could have sent none at all.  I used "100x" because there
are around 100 people in the MASQUE session this morning, but it's really
much higher because the denominator should be basically zero.

> And base decisions and engineering on real data, instead of guesswork.

I'm not guessing, it's clear from the MeetEcho UI that they're doing SSO
wrong.

--Richard

> >> I would agree with the sentiment on anything that is not essential to
> our
> >> DNA.
> >> Web meetings that are replacing the traditional IETF week are essential
> to
> >> our DNA.
> >> Of course many reasons can be cited why that is hard to do right now,
> but
> >> we should strive for agility.
> >> (*) I’m sorry, not everyone here has used Typo3, so that sentence may
> not
> >> invoke the horror it should.

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 28 Jul 2020 15:45:39 +0200

Hi Ekr,

> 1. A number of people are experiencing authorization failures (as Richard

Yes, and all of them boil down to one issue:  These are people who have
multiple registrations (hackathon, remote) where they have used different
email addresses for the different registrations, and there has been a
difficulty connecting up the registration with the required 'remote' reg_type
for WG/RG session participation with the datatracker login.

> user is authenticated. This seems like it potentially exacerbates (1),

This is as designed.  Meetecho knows nothing about a new connection than
what it gets from the datatracker, and arguably should not.  If you want to
change this, I think you'll need to re-design OpenID Connect.  Anyway,
the load of the OpenID Connect queries is maybe one tenth of the remaining
load at peak login, so why exactly is this an issue?

(FWIW, I thought about caching myself, but realized that there would be
security issues with that, and let it go, since the load simply wasn't an
issue).

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 28 Jul 2020 15:50:08 +0200

> not having been scale-tested in advance, is exactly the problem here.

Again, you're guessing, and guessing wrong.

There was no adjustment of this at meeting time, and no need for it.

> much higher because the denominator should be basically zero.

I'm sorry?  If you have 100 people who need to hit the datatracker 3 times
for 3 sessions, instead of 1 time, the factor is 3x, not 100x.

> I'm not guessing, it's clear from the MeetEcho UI that they're doing SSO

It's perfectly clear that you're guessing at causes, and repeatedly guessing
wrong.

	Henrik

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 28 Jul 2020 06:54:03 -0700

On Tue, Jul 28, 2020 at 6:45 AM Henrik Levkowetz <[email-masked]>
wrote:

> >> > The point being that these bespoke tools have a cost, not just in
> units
> >> of

Well, as I said, I have no special insight into what's going on here, but
why are these issues intermittent? For instance, it was failing for me
yesterday but works today, even though my configuration has not changed.

> change this, I think you'll need to re-design OpenID Connect.

Huh? The conventional approach would be for Meetecho to have a cookie which
it uses for authentication and only reach out to the datatracker when that
cookie expires. This is the case for practically every SSO-based service I
use regularly.

Anyway,
> load at peak login, so why exactly is this an issue?

It's not an issue of load but of how often the datatracker is in the
critical path. Specifically, if Meetecho cached authentication, then people
who had successfully logged in Monday would most likely not be experiencing
failures at the Meetecho/Datatracker interface, whatever the cause of those
failures.

-Ekr

---

From: Richard Barnes <[email-masked]>
Date: Tue, 28 Jul 2020 10:04:00 -0400

On Tue, Jul 28, 2020 at 9:50 AM Henrik Levkowetz <[email-masked]>
wrote:

> >> > I am unable to access any of the sessions ("Unauthorized").  I've
> heard
> >> out

Sorry, that was how I understood you when you said, "has been 12, and we
are currently set up to handle 100 in parallel".

If that's not the case, then what caused the outage?

> for 3 sessions, instead of 1 time, the factor is 3x, not 100x.

This is a silly point to fight over, but I'm talking about the load at
meeting start time.  If MeetEcho were properly caching logins, then there
would be zero or maybe a handful of requests to the datatracker at the
beginning of a session.  Instead there are hundreds.

> It's perfectly clear that you're guessing at causes, and repeatedly

What am I misunderstanding here?  Is MeetEcho actually caching SSO results
and lying to users?  :)

--Richard

> >> >> fundamentally is hard, most people have little education in this
> space,
> >> and

---

From: Simon Pietro Romano <[email-masked]>
Date: Tue, 28 Jul 2020 16:55:32 +0200

> What am I misunderstanding here?  Is MeetEcho actually caching SSO results and lying to users?  :)

I don’t really know where you’re heading to, but I will resist the temptation of getting involved in such a useless discussion. I will rather just politely ask you not to attach any label to the Meetecho team that might even remotely recall anything that is close to a liar.

Thanks,

Simon

                     				            _\\|//_
                           				   ( O-O )

> 
> 
> 
> What am I misunderstanding here?  Is MeetEcho actually caching SSO results and lying to users?  :)
> 
> --Richard
> 
>  
> 
> 
>         Henrik
> 
> 
> 
> 
> > 
> > --Richard
> > 
> > 
> > 
> >>
> >>         Henrik
> >>
> >>
> >> >
> >> >
> >> > On Tue, Jul 28, 2020 at 1:30 AM Carsten Bormann <[email-masked] <mailto:[email-masked]>> wrote:
> >> >
> >> >> [1]: https://en.wikipedia.org/wiki/Situational_application <https://en.wikipedia.org/wiki/Situational_application>

---

From: Richard Barnes <[email-masked]>
Date: Tue, 28 Jul 2020 11:01:25 -0400

On Tue, Jul 28, 2020 at 10:55 AM Simon Pietro Romano <[email-masked]>
wrote:

> I don’t really know where you’re heading to, but I will resist the
> temptation of getting involved in such a useless discussion. I will rather
> just politely ask you not to attach any label to the Meetecho team that
> might even remotely recall anything that is close to a liar.
>

Sorry, didn't actually mean to imply that MeetEcho is displaying something
incorrect.  Just that Henrik seems to be implying that the straightforward
interpretation of what MeetEcho presents is not the case.

--RLB

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 28 Jul 2020 17:54:24 +0200

Hi Ekr,

> yesterday but works today, even though my configuration has not changed.

Without more information and looking into what happened, I can of course
not say why.  However, when there has been issues of the kind I described
above, the meetecho guys have messaged me, and I've sorted out the
registrations.

Improved code to deal with the human inconsistencies involved will be
deployed later today.

> cookie expires. This is the case for practically every SSO-based service I

That sounds reasonable, but there might be reasons at the MeetEcho end
I'm not fully aware of, such as (but not limited to) the different roles
the same person might have in the different sessions (chair vs. not chair,
for instance).

> failures at the Meetecho/Datatracker interface, whatever the cause of those

Agreed, FWIW.  However, the issues we've had reported and investigated till
now have consistently not been intermittent; once the underlying issue with
registration inconsistencies has been resolved, it's stayed resolved.

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 28 Jul 2020 18:48:26 +0200

> are currently set up to handle 100 in parallel".

Partial quote?

I said:

> This morning, the maximum number of connections per second to the
> datatracker has been 12, and we are currently set up to handle 100
> in parallel.

Yes, the maximum number of connections per second observed arriving at
the datatracker this morning was 12.  Yes, the current setup, effective
at that time, as well as before the meeting, provides for 100 parallel
connections.  I don't understand how you can interpret that as saying
there's been an adjustment?

> If that's not the case, then what caused the outage?

There's been no outage.  You in particular could not log in because
you'd used 2 different email addresses on your 2 registrations (hackathon
and regular week remote) and when you logged in, the hackathon registration
was picked up.  I adjusted the discrepancy at the datatracker end as soon
as the Meetecho guys alerted me to the issue, so you could get in.

> beginning of a session.  Instead there are hundreds.

People have to log in at some point, so the denominator can never be zero.

Caching too long causes its own problems; in particular caching more than
one day might be inadvisable if we have one-day passes, and I understand
that Meetecho designed their handling to allow for proper handling of
one-day passes.

So I'll buy 3x, but 100x is hyperbole.

> What am I misunderstanding here?  Is MeetEcho actually caching SSO results

I have no idea what you're misunderstanding.  Meetecho's implementation of
OpenID Connect is fine.

	Henrik

---

From: Michael Richardson <[email-masked]>
Date: Tue, 28 Jul 2020 12:56:45 -0400

> issues people were seeing.

My understanding of the problem on MacOS (which I don't use), for all the
browsers, is that if you share a single application, that you wind up sharing
the File/Edit/etc. menu, so effectively you always transmit a full-screen
sized bit map, but maybe with a lot of blank.
But, with Chrome, you can share a single tab or single Chrome window, so you
avoid this.

But, maybe I do not understand.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: "Pete Resnick" <[email-masked]>
Date: Tue, 28 Jul 2020 14:08:08 -0500

> But, maybe I do not understand.

When I did a bit of testing on this, it turns out that the documentation 
above is not quite accurate:

When using Meetecho with Safari, if you have multiple screens, only one 
screen will be shared (not the entire screen real estate), but it shares 
the currently active screen and does not give you a choice of screens to 
share. That means if you want to share monitor #1 and put your Meetecho 
window on monitor #2, you first move the Meetecho window to monitor #1, 
start up screen sharing, and then move the Meetecho window to monitor 
#2. Not ideal, but not too complicated. As the documentation says, in 
other browsers, you're able to share particular windows, which makes the 
process much easier.

pr

---

From: Meetecho IETF support <[email-masked]>
Date: Tue, 28 Jul 2020 21:15:10 +0200

Hi Pete,

Il 28/07/20 21:08, Pete Resnick ha scritto:
>>     >> On Mon, Jul 27, 2020 at 7:53 PM Ben Campbell <[email-masked]> 
>> wrote:
>>     >>
>>     >> > Fair point. I guess I’m compartmentalizing the blame.
>>     >> >
>>     >>
>>     >> Well, before we apportion the blame it would be useful to know 
>> what the
>>     >> source of the problem is. Do we have bug reports that describe 
>> the defects
>>     >> that are causing problems?
>>
>>     > The documentation at
>>     > 
>>     > already refers to some limitations about Safari (only allows 
>> sharing whole
>>     > screen, not individual windows).  I'm not sure whether there 
>> were any other
>>     > issues people were seeing.
>>

You got this straight. I'd only add that Meetecho has no control 
(unfortunately) on this behavior.

Thanks,
Alessandro

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 29 Jul 2020 04:06:33 -0700

On Tue, Jul 28, 2020 at 8:54 AM Henrik Levkowetz <[email-masked]>
wrote:

> >> >> > dollars, but also in choice, reliability, etc.  We should think
> hard
> >> >> about
> not say why.

FWIW, I just saw a transient failure. I have reported it as:
https://tickets.meeting.ietf.org/ticket/1401

-Ekr

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 29 Jul 2020 13:28:39 +0200

Hi Eric,

> https://tickets.meeting.ietf.org/ticket/1401

Thanks.  We're looking into this from both the Meetecho and Datatracker sides.

	Henrik

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 Jul 2020 00:33:10 -0400

Without singling out any particular comment, I think there are
at least two things have have gotten lost in the discussions and
suggestions.  I assume that, in at least some cases, people
didn't know.  

First, I don't know whether this should have been made explicit
earlier or not, but this is not the first time the IETF has used
Meetecho.  Many of us have been using it for remote
participation for years and a great deal of effort has gone into
making it work smoothly for IETF's way of working [1].  I assume
we are probably a little more critical than many of their
customers but assume "our" changes have become, possibly with
small variations, part of their main product offering.  I
believe their other customers have including many all remote, or
all remote other than a very small number of people in a central
location, setups, so the assumption in some messages that
Meetecho has never been used before in an all-remote situation
(or very close to it) is probably incorrect.  I don't know if
the latter is accurate but, if it is important, I think we
should ask rather than jumping to conclusions.

Second, many changes have occurred, at least to the user
interfaces, between our use for remote participants at IETF 106
and this week.  Personally, I like some of the changes but
believe others show signs of having been done in haste and with
too little thought and/or time for testing and review.  I accept
Jay's assertion that those changes were not micromanaged by the
IETF leadership or staff, but note that Greg Wood indicated
during and after one of the test sessions that at least some of
those changes had been made at the behest of an IETF design
committee and that that the I-D and discussions of the hum
feature are quite explicit that the specifications came from the
IESG.

It is probably helpful to remember something else I learned a
half-century ago about UI design.  An experimental psychologist
colleague I worked with them was fond of staying that, when
people tried to evaluate a system, what they already knew was
almost always better (obviously, just because they are used to
it).  For those who did not actively use Meetecho for remote
participation during IETF 106 and earlier and who have spent a
significant fraction of the last months on Zoom, WebEx,
GoToMeeting, and their competitors, and who did not attend the
test sessions, Meetecho probably feels very strange and is at a
significant disadvantage.  I recommend giving it a chance and
doing so with an open mind.

If, as appears to be the case from the timing of the
announcements, all of this was done on relatively short notice.
We should be impressed that it works and identifying issues and
making suggestions for improvements.  If we want to make
suggestions about replacing all of it with COTS software, we
should consider how much effort has gone into trying to adapt
Meetecho for IETF needs and remember that, before Meetecho came
along, we tried to do remote participation with WebEx and, well,
it didn't work out very well.

best,
  john

[1] In the interest of full disclosure, I participated in a few
design sessions along with Alexa, Ray, and some Meetecho staff
(and maybe others; I don't remember).  I think I got sucked in
because I started being intermittently involved in research in
user interface and usability issues in distributed office
teleconferencing systems in the early 1980s, studies that
involved real experimental psychologists and controlled
comparison of different approaches.  And I may have been
compensated with some travel reimbursements or a registration
fee waiver or two -- not significant enough that I remember.

---

From: George Michaelson <[email-masked]>
Date: Thu, 30 Jul 2020 14:42:33 +1000

To hums: the default *appears* to be "piano" from headcount, if you do
not interact
therefore, graphing hum volume by headcount includes non-interactors
and this "moves the dial"

if not, then why is there no clear 'abstain' button. and how is the
logic behind how the total sum and hum-weighting applied?

APNIC designed a similar (in intent) system called "confer" to show
dynamic state of the room. Its hard. Its hard to explain and get this
stuff right.

I did not find the hum indicated anything I felt I trusted.
Experientally, "hearing" a hum and being told "here is the weighted
sum of things you cannot see" is completely different.

I would prefer to be told how many hummed. how many explicitly chose
to say "no view" and how many did not participate, and I would prefer
to be told how the hum weighting occurs. not in an RFC: on the tool.
all the time.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 31 Jul 2020 15:47:25 +0000

Going back in the thread to this comment...

> to be told how the hum weighting occurs. not in an RFC: on the tool.

I think it's often difficult to define the right or a meaningful metric for a set of measurements.
In a physical room, WG chairs are able to look at how many people are in the room, and whether the hum is loudly (or softly) emanating from a few people or from the room as a whole. The chair bases their consensus metric on all these visible and audible measurements.
What George said in this email resonated with me --
I would prefer to get the raw measurements rather than have the tool define a metric that is inconsistent with my metric.
If we could just be provided (1) number of people in the room (which we already have), and (2) number of people humming at each volume level (loud/medium/soft), and trust the chairs not to consider this a vote (which I think should be a reasonable expectation), it would make it easier for the chair to mentally create a consensus metric, rather than have a metric imposed on them by the tool.
Barbara

---

From: Bob Hinden <[email-masked]>
Date: Fri, 31 Jul 2020 08:54:58 -0700

> On Jul 31, 2020, at 8:47 AM, STARK, BARBARA H <[email-masked]> wrote:
> 

That is my view as well.

Bob