Subject: Re: [OPS-AREA] meeting materials MUST be online well BEFORE the session

From: Ron Bonica <[email-masked]>
Date: Fri, 01 Aug 2008 09:54:52 -0400

Bert,

I apologize for this. If the presenters will send me their slides I will 
post them with the minutes.

                              Ron

Bert Wijnen (IETF) wrote:
> Being a jabber scribe for the OPS area, I clearly see how BAD it
> is if slides are not ONLINE before the meeting starts.
> Also... SLIDES MUST be numbers, and speakers should mention the
> slide numbers.
> 
> I spend considerable amount of time typing slide titles after first 
> having had
> to email slides to remote participants from memory sticks.
> 
> Bert Wijnen
> 
> _______________________________________________
> OPS-AREA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ops-area
> 
_______________________________________________
OPS-AREA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ops-area

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Sat, 2 Aug 2008 00:25:12 +0200

Let me suggest that if a presenter does not provide his/her presentation at 
the latest 24
hours before the session start, then cancel his/her presentation.

Bert
----- Original Message ----- 
From: "Ron Bonica" <[email-masked]>
To: "Bert Wijnen (IETF)" <[email-masked]>
Cc: "ops-area (IETF)" <[email-masked]>
Sent: Friday, August 01, 2008 3:54 PM
Subject: Re: [OPS-AREA] meeting materials MUST be online well BEFORE the 
session

---

From: Pekka Savola <[email-masked]>
Date: Mon, 4 Aug 2008 08:36:41 +0300 (EEST)

> hours before the session start, then cancel his/her presentation.

If we go along this path -- and if the agenda is not uploaded to the 
web a week before the meeting, cancel the meeting :-)

> ----- Original Message ----- From: "Ron Bonica" <[email-masked]>

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
_______________________________________________
OPS-AREA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ops-area

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 4 Aug 2008 12:01:37 +0200

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Pekka Savola
> Sent: Monday, August 04, 2008 8:37 AM
> To: Bert Wijnen (IETF)
> Cc: ops-area (IETF)
> Subject: Re: [OPS-AREA] meeting materials MUST be online well 
> BEFORE the session
> 
> > Let me suggest that if a presenter does not provide his/her 
> > presentation at the latest 24 hours before the session start, then 
> > cancel his/her presentation.
> 
> If we go along this path -- and if the agenda is not uploaded 
> to the web a week before the meeting, cancel the meeting :-)
> 

I would actually drop the smiley and entertain such an idea. I find
practically impossible for someone to prepare for a meeting if the
agenda is not posted less than a week before the meeting. Of course,
changes should be allowed until the last day, but the main items on the
agenda, and the preliminary reading materials list can be made available
one week in advance. 

Please allow to Ron and me to compile these proposals and get back with
a proposal submitted for community consensus. 

Dan

---

From: "Ersue, Mehmet (NSN - DE/Muenich)" <[email-masked]>
Date: Mon, 4 Aug 2008 13:20:31 +0200

+1

It is not helpful for remote attendees and is actually too 
late if the presenter shows up in the last second with a 
USB stick.

OTOH the WG chairs should take a look on the slides 
before the session and request changes if the slides are 
not appropriate (e.g. tutorial instead of issue discussion, 
too many slides for the time available in the session slot).

We need to find a way to force early provision of the 
presentation slides to the meeting material site.

Cheers, 
Mehmet

> [mailto:[email-masked]] On Behalf Of ext Bert Wijnen (IETF)
> Sent: Saturday, August 02, 2008 12:25 AM
> To: Ron Bonica
> Cc: ops-area (IETF)
> BEFORE thesession
> presentation at 
> BEFORE the

---

From: "David Harrington" <[email-masked]>
Date: Mon, 4 Aug 2008 13:19:34 +0100

Hi,

I think we should focus more on carrots than sticks. This is a
volunteer organization.
My name is Dave and I am a sinner.

Sometimes the chairs get tied up with day-job work and have difficulty
finding time to focus on writing up an agenda. Sometimes the chairs
need to wait until they get input from volunteers about presentations
that will will be done before setting up an agenda. Do you recommend
posting empty agendas so that the deadline is met, and then editing it
later when we have content?

Have you thought about the unintended consequences of a rule that
prevents a WG from meeting or discussing a particular topic if the
agenda and materials are not posted in advance? What will be the
consequences if volunteers spend money to travel to an IETF meeting,
only to find that their only session has been cancelled because a
chair was tied down with day-job work, or they could not work on
resolving the issues because the presentation didn't get posted? Will
they continue to travel to IETF meetings? Will they request new
chairs? Where will we get new chairs that are not burdened by day-job
demands? Will the WG accomplish better standards, or get them done
faster, as a result of such rules?

My WG didn't get an early agenda posted because one chair was
travelling internationally for day-job work for the month prior to
Dublin (justifying funding of a team to work on IETF standards), and
the other was on a 3.5 week vacation with his family. Which of those
should we have given up to ensure that the agenda was posted more than
a week in advance? Should we resign as chairs because we cannot
guarantee timely delivery of the agenda? Should the whole WG suffer by
having its sessions cancelled because the chairs faced competing real
world priorities?

We were also guilty of not posting the presentations well in advance.
We had to wait for the presentations to be written, which usually
happened within 24 hours of when the presentation would be delivered.
The content of those presentations was sometimes impacted by the f2f
discussions that occurred during the IETF week, including during
evening bar time when compromises were worked out between major
contributors on open issues.

Some chairs had problems with connectivity. I know my co-chair could
not access the web during our first session, so he could not even
access the meeting materials himself. I had spotty connectivity; I
frequently did not get emails from my co-chair or the contributors for
hours after they were sent to me, so I did not have access to the
presentations or to the meeting materials page on a timely basis.
Should the WG session be cancelled or the topic be banned because the
network connectivity was not always perfect enough to ensure postings
at least 24 hours in advance?

I would be very careful about trying to legislate too much and to
penalize whole WGs because the chairs failed to post meeting materials
on time. I think the cure would be worse than the problem.

I know I failed to post on a timely basis this time, and I will try to
do better in the future, although some are real-world obstacles beyond
my control. If you start punishing the whole WG for late postings by
the chairs, I will probably just ask you to find a new chair.

David Harrington
[email-masked]
[email-masked]
[email-masked]

> [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
> Sent: Monday, August 04, 2008 11:02 AM
> To: Pekka Savola; Bert Wijnen (IETF)
> Cc: ops-area (IETF)

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 4 Aug 2008 15:33:53 +0200

Hi David,

Wow, I think that I pushed a red-hot button :-) 

Let me try to address the issues that you are raising. We all want to
make the standards process work better and faster, get more people
involved and motivated and issue better standards at the end of the
process. The intent of such rules would not be to punish and to exclude
but to make agendas and materials available in advance for people to be
more efficient at the meeting. I believe that at the meeting last week
some people could not prepare well because agendas (not only from your
WG) came too late, and the remote participation lacked efficiency
because many presentations were not available even by the time the
meetings started.  

My experience is that if rules are in place and known in advance
responsible people do their best in order to respect them. In the case
of your WG if the one week agenda rule was in advance I am sure that you
and David would have done the best to submit a preliminary agenda by the
deadline, or would have announced the WG and the AD before traveling
'look, we are traveling this period for such and such reasons,
preliminary agenda will not be available on the Monday deadline but only
Wednesday before the meeting'. Being predictive is what we seek here,
being able for all participants in the process to plan and allow the
best possible fairness and preparation time is what we seek. 

There are always exceptions and life situations that make some deadlines
impossible to match. For these purposes all the IETF systems or the
majority of them accept human override with proper approval level. In
the case such rules will be adopted in the future, ADs will have the
capability of accepting agendas after the deadline and WG chairs will be
able to accept presentations or updates even in the last 24 hours before
the meeting. However, this would be an exception, the rule would be that
agendas and presentations need to be submitted in time, and I am sure
that the majority of the volunteers who are all on both provider and
receiver roles will make the rules work in most cases. 

Regards,

Dan

BTW - you criticized my use of some kind of stick menace, but you did
not suggest any carrot. I would love a carrot-only based system if
somebody can come with one. 

> From: David Harrington [mailto:[email-masked]] 
> Sent: Monday, August 04, 2008 3:20 PM
> To: Romascanu, Dan (Dan); 'Pekka Savola'; 'Bert Wijnen (IETF)'
> Cc: 'ops-area (IETF)'
> Subject: RE: [OPS-AREA] meeting materials MUST be online well 
> BEFORE thesession
> 
> Hi,
> 
> I think we should focus more on carrots than sticks. This is 
> a volunteer organization.
> Sometimes the chairs get tied up with day-job work and have 
> difficulty finding time to focus on writing up an agenda. 
> Sometimes the chairs need to wait until they get input from 
> volunteers about presentations that will will be done before 
> setting up an agenda. Do you recommend posting empty agendas 
> so that the deadline is met, and then editing it later when 
> we have content?
> 
> Have you thought about the unintended consequences of a rule 
> that prevents a WG from meeting or discussing a particular 
> topic if the agenda and materials are not posted in advance? 
> What will be the consequences if volunteers spend money to 
> travel to an IETF meeting, only to find that their only 
> session has been cancelled because a chair was tied down with 
> day-job work, or they could not work on resolving the issues 
> because the presentation didn't get posted? Will they 
> continue to travel to IETF meetings? Will they request new 
> chairs? Where will we get new chairs that are not burdened by 
> day-job demands? Will the WG accomplish better standards, or 
> get them done faster, as a result of such rules?
> 
> travelling internationally for day-job work for the month 
> prior to Dublin (justifying funding of a team to work on IETF 
> standards), and the other was on a 3.5 week vacation with his 
> family. Which of those should we have given up to ensure that 
> the agenda was posted more than a week in advance? Should we 
> resign as chairs because we cannot guarantee timely delivery 
> of the agenda? Should the whole WG suffer by having its 
> sessions cancelled because the chairs faced competing real 
> world priorities?
> 
> We had to wait for the presentations to be written, which 
> usually happened within 24 hours of when the presentation 
> would be delivered.
> The content of those presentations was sometimes impacted by 
> the f2f discussions that occurred during the IETF week, 
> including during evening bar time when compromises were 
> worked out between major contributors on open issues.
> 
> Some chairs had problems with connectivity. I know my 
> co-chair could not access the web during our first session, 
> so he could not even access the meeting materials himself. I 
> had spotty connectivity; I frequently did not get emails from 
> my co-chair or the contributors for hours after they were 
> sent to me, so I did not have access to the presentations or 
> to the meeting materials page on a timely basis.
> Should the WG session be cancelled or the topic be banned 
> because the network connectivity was not always perfect 
> enough to ensure postings at least 24 hours in advance?
> 
> I would be very careful about trying to legislate too much 
> and to penalize whole WGs because the chairs failed to post 
> meeting materials on time. I think the cure would be worse 
> than the problem.
> 
> I know I failed to post on a timely basis this time, and I 
> will try to do better in the future, although some are 
> real-world obstacles beyond my control. If you start 
> punishing the whole WG for late postings by the chairs, I 
> will probably just ask you to find a new chair.
> 
> David Harrington
> [email-masked]
> [email-masked]
> [email-masked]
> 
> > Subject: Re: [OPS-AREA] meeting materials MUST be online 
> well BEFORE 
> > thesession
> > 
> >  
> > 
> > > Subject: Re: [OPS-AREA] meeting materials MUST be online 
> well BEFORE 
> > > the session
> > > 
> > changes should be allowed until the last day, but the main items on 
> > the agenda, and the preliminary reading materials list can be made 
> > available one week in advance.
> > 
> > Please allow to Ron and me to compile these proposals and get back 
> > with a proposal submitted for community consensus.
> > 
> > Dan

---

From: "David Harrington" <[email-masked]>
Date: Mon, 4 Aug 2008 09:50:16 -0400

Hi Dan,

IETF volunteers are here to produce standards. That is not always a
fun thing to do, and the sloeness of IETF work is extrememly
frustrating, but moist of our main volunteer contributors do this work
because they believe in the benefits of an open and fair volunteer
process of standards development. Some even get paid to do it; many do
not. Helping to develop a better Internet by working on standards
**is** the carrot. 

The discussion "hey, folks, it is hard for remote contributors to
participate effectively if the agendas and/or the presentations are
posted too late" focuses on the carrot; when agendas and presentations
are not posted early enough, if skews the process and makes the WG
less effective - it affects the carrot.

When the discussion removed the smiley and started talking about
punative actions - cancelling sessions and removing presentations from
the sessions, then you turned the discussion into a "stick" discussion
rather than a carrot discussion.

dbh

> From: Romascanu, Dan (Dan) [mailto:[email-masked]] 
> Sent: Monday, August 04, 2008 9:34 AM
> To: David Harrington; Pekka Savola; Bert Wijnen (IETF)
> Cc: ops-area (IETF)

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 4 Aug 2008 16:03:15 +0200

Hi David,

It is not only the remote participants who are affected by not having
the agendas in place before the meeting. It is also those participants
who want to prepare for the meeting, to download print (or not) and read
the Internet-Drafts in advance (many of them use the long
inter-continental flights for this). Carrots are not only for the
editors. Mehmet rightly pointed to the fact that a WG chair needs to see
the documents a day in advance in order to fine tune them in the context
of the meeting that he/she needs to run. 

I see that you took the punitive side too seriously, and I apologize
because maybe it is me who made the proposal sound too harsh (or was it
Bert who started? It is always good to blame the previous AD :-)).
Cancelling a meeting or a presentation should indeed be an extreme
action. What we want to change is to make of posting agendas and
presentations in advance to allow proper preparation of the meetings and
remote participation efficient the rule to be followed by all. Let us
find the ways to do it with as much carrot and as little stick as
possible. 

Dan

> Sent: Monday, August 04, 2008 4:50 PM
> IETF volunteers are here to produce standards. That is not 
> always a fun thing to do, and the sloeness of IETF work is 
> extrememly frustrating, but moist of our main volunteer 
> contributors do this work because they believe in the 
> benefits of an open and fair volunteer process of standards 
> development. Some even get paid to do it; many do not. 
> Helping to develop a better Internet by working on standards
> **is** the carrot. 
> 
> The discussion "hey, folks, it is hard for remote 
> contributors to participate effectively if the agendas and/or 
> the presentations are posted too late" focuses on the carrot; 
> when agendas and presentations are not posted early enough, 
> if skews the process and makes the WG less effective - it 
> affects the carrot.
> 
> When the discussion removed the smiley and started talking 
> about punative actions - cancelling sessions and removing 
> presentations from the sessions, then you turned the 
> discussion into a "stick" discussion rather than a carrot discussion.
> 
> dbh
> 
> > Subject: RE: [OPS-AREA] meeting materials MUST be online 
> well BEFORE 
> > thesession
> > 
> > Hi David,
> > 
> > process. The intent of such rules would not be to punish and to 
> > exclude but to make agendas and materials available in advance for 
> > people to be more efficient at the meeting. I believe that at the 
> > meeting last
> week
> > of your WG if the one week agenda rule was in advance I am 
> sure that 
> > you and David would have done the best to submit a 
> preliminary agenda 
> > by the deadline, or would have announced the WG and the AD before 
> > traveling 'look, we are traveling this period for such and such 
> > reasons, preliminary agenda will not be available on the Monday 
> > deadline but only Wednesday before the meeting'. Being 
> predictive is 
> > what we seek
> here,
> > There are always exceptions and life situations that make some 
> > deadlines impossible to match. For these purposes all the 
> IETF systems 
> > or the majority of them accept human override with proper approval 
> > level.
> In
> > capability of accepting agendas after the deadline and WG 
> chairs will 
> > be able to accept presentations or updates even in the last 
> 24 hours 
> > before the meeting. However, this would be an exception, the rule 
> > would be that agendas and presentations need to be 
> submitted in time, 
> > and I am
> sure
> > > Subject: RE: [OPS-AREA] meeting materials MUST be online 
> well BEFORE 
> > > thesession
> > > 
> > > Hi,
> > > 
> > > volunteers about presentations that will will be done 
> before setting 
> > > up an agenda. Do you recommend posting empty agendas so that the 
> > > deadline is met, and then editing it later when we have content?
> > > 
> > > agenda and materials are not posted in advance?
> > > What will be the consequences if volunteers spend money 
> to travel to 
> > > an IETF meeting, only to find that their only session has been 
> > > cancelled because a chair was tied down with day-job 
> work, or they 
> > > could not work on resolving the issues because the presentation 
> > > didn't get posted? Will they continue to travel to IETF meetings? 
> > > Will they request new chairs? Where will we get new 
> chairs that are 
> > > not burdened by day-job demands? Will the WG accomplish better 
> > > standards, or get them done faster, as a result of such rules?
> > > 
> prior to 
> > > should we have given up to ensure that the agenda was posted more 
> > > than a week in advance? Should we resign as chairs 
> because we cannot 
> > > guarantee timely delivery of the agenda? Should the whole 
> WG suffer 
> > > by having its sessions cancelled because the chairs faced 
> competing 
> > > real world priorities?
> > > 
> co-chair could 
> > > frequently did not get emails from my co-chair or the 
> contributors 
> > > for hours after they were sent to me, so I did not have access to 
> > > the presentations or to the meeting materials page on a timely 
> > > basis.
> > > Should the WG session be cancelled or the topic be banned because 
> > > the network connectivity was not always perfect enough to ensure 
> > > postings at least 24 hours in advance?
> > > 
> > > penalize whole WGs because the chairs failed to post meeting 
> > > materials on time. I think the cure would be worse than 
> the problem.
> > > 
> > > I know I failed to post on a timely basis this time, and 
> I will try 
> > > to do better in the future, although some are real-world 
> obstacles 
> > > beyond my control. If you start punishing the whole WG for late 
> > > postings by the chairs, I will probably just ask you to 
> find a new 
> > > chair.
> > > 
> > > David Harrington
> > > [email-masked]
> > > [email-masked]
> > > [email-masked]
> > >

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 4 Aug 2008 10:13:32 -0700

Hi -

> From: "Ersue, Mehmet (NSN - DE/Muenich)" <[email-masked]>
> To: "ext Bert Wijnen (IETF)" <[email-masked]>; "ops-area (IETF)" <[email-masked]>
> Sent: Monday, August 04, 2008 4:20 AM
> Subject: Re: [OPS-AREA] meeting materials MUST be online well BEFOREthesession
...
> presentation slides to the meeting material site.
...

YES, especially for BOFs.  I've found that BOF presenters often stray
far from the agreed BOF focus in counter-productive ways, despite
advance discussion.  Though carrots are nice in theory, this is
one of those cases where having a stick would really help.  But
I agree with David Harrington that completely dropping the meeting
or presentation in question may in some cases be a bit too
draconian, even though I think such a policy might be better in
the aggregate than what we have now.  But perhaps we could
try another path: construct some kind of public pillory (with company
affiliation for identification purposes) for the offendors - folks with
excessive fluff, excessive marketing, or who hand chairs memory
sticks loaded with beside-the-point material.

Randy

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 11 Aug 2008 09:08:15 -0700

>>>>> On Mon, 4 Aug 2008 13:20:31 +0200, "Ersue, Mehmet (NSN - DE/Muenich)" <[email-masked]> said:

ME> It is not helpful for remote attendees and is actually too 
ME> late if the presenter shows up in the last second with a 
ME> USB stick.

Speaking as someone who is one of the sources of a recent USB stick: I
agree things should be encouraged as much as possible in advance.  The
problem comes in that we're a real-world seat-of-the-pants type of
standardization body.  We work on things in the hallway sometimes
minutes before meetings that affect presentations and may require slide
changes.  Yes, we could force presentations/meetings to be canceled if
slides/agendas aren't available.  But the question is: how does this
removal of work items actually benefit us?  It actually will inhibit our
already too-slow work process, not help it.  It would certainly help
some, as probably 50% of the people would get things submitted in time.
But I'm sure the other 50% probably wouldn't.

In my particular case, I simultaneously sent slides in for two
presentations I was giving (more than 24 hours in advance).  One of the
two bounced.  Real life servers got in the way.  Had I been allowed to
submit my own slides instead via the presentation submission tool (I
checked; I couldn't) this wouldn't have been an issue as I could get to
that page (though even that could have gone down).

(Another option is to force them to give a talk at a microphone without
their slides; it'd still allow discussion without the presentation
material ;-)

-- 
Wes Hardaker
Sparta, Inc.

---

From: "David Harrington" <[email-masked]>
Date: Mon, 11 Aug 2008 13:53:39 -0400

> Yes, we could force presentations/meetings to be canceled if
> already too-slow work process, not help it.  

> their slides; it'd still allow discussion without the presentation

I think the "another option" suffers from the same problem as the
cancellation option. I do not see how forcing people to discuss things
at the microphone without the benefit of the slides they submitted
late, benefits the IETF. I think this restriction would inhibit
progress.

Working for a Chinese company whose personnel often have difficulty
attending meetings due to travel budget or visa concerns, and who
frequently utilize our tools for remote participation, I am very
sensitive to the importance of having the slides available for remote
participants. But at this point in time, remote particiants are still
a small minority in the IETF community, and penalizing the IETF
community in attendance at the meeting because the remote participants
might be inconvenienced does not make much sense to me.

If we want to be serious about supporting remote participants (and I
think we should), I think there are a lot of things besides having
slides available that are important. In Dublin, as I watched chairs
sometimes talk into their mikes, but not consistently, and chairs
counting hands but not verbally reporting the results, and people at
the mikes rushing through what they had to say rather than thinking
about how well remote participants would be able to understand them, I
thought to myself maybe every chair should be required to actually
listen to and replay their recorded sessions to see just how difficult
it is to be a remote participant in their WG. That might have a much
greater effect toward supporting remote participants than cancelling
presentations.

dbh

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 11 Aug 2008 21:55:19 +0300

A few notes from my side:

* I doubt that the reason for the slow progress in the IETF is related 
to the face-to-face meeting. The problem is the time in between meetings.

* Those who do not know what drafts have to be read for the meeting 
(because of an absent agenda) haven't followed the work in the working 
group and are therefore not very likely to read the drafts anyway. These 
folks need a tutorial or a monthly status update (we distribute these 
status update in DIME).

* BOFs are a completely separate subject. They typically need far more 
preparation for a variety of reasons. A different story.

* Reading through everyone's slides: Well. If you are really serious 
about that issue and you think you know everything better than the folks 
in the group then the chairs should write the slides for the group. 
(This makes the time management much easier. Additionally, you have to 
consider that some folks speak slow, are hard to understand, never get 
to the point, talk about things not written on the slides, etc. etc. ) 
The question is how much "show" do you want to provide during the meeting.

* Demanding that folks have their slides ready a week before the meeting 
is not realistic. We would have to cancel almost all our meetings. It 
would be nice to have everything available, great meeting minutes, good 
jabber scribes, briliant audio recordings, etc. Just a bit hard to make 
this happen. I tried to create podcasts myself during meetings but it 
was really tough to get a good output.

* If you believe that you have to wait for the IETF meeting to make 
"progress" then you are wrong. Do it in the time between meetings on the 
mailing list (or potentially via phone conf. calls). This is obviously 
best for "remote participants".

* The most important time during an IETF meeting are the hallway 
conversations and the socializing in the evening, before the IETF 
meeting starts and after the IETF meeting is finished. You can hardly 
deliver that experience to remote participants.

Ciao
Hannes