Subject: [edu-team] IETF 90 - Working Group Chairs Training Lunch - REMINDER

From: IETF Agenda <[email-masked]>
Date: Thu, 03 Jul 2014 13:52:24 -0700

Hello Everyone,

This is a reminder to please register for the WG Chairs training lunch at IETF 90 in Toronto.  All current WG chairs, BOF chairs, IRTF chairs, and IESG members are welcome to attend. The lunch training will take place on Wednesday, July 23rd, 2014 at 11:30am, location TBD, and a complimentary lunch will be provided.

Topic: English as a Second Language
1. Introduction and general problem statement
2. RFC Editor approach and experience with the workshop (Alice Russo,
Heather Flanagan)
3. Experiences of a non-native English speaker (Ning Zong)
4. Open discussion - collect ideas, what are the best next steps?

If you would like to attend this session, please RSVP using this registration page: 
https://www.ietf.org/registration/ietf90/rsvp.py?eventid=1&eventcode=348906 

The location will be sent to those who RSVP.

Please register prior to the cutoff on Friday, July 4.

If you have RSVP'd for the training lunch but will not be able to attend, or have any issues registering for the event, please send a message to [email-masked].  

Thank You!

Stephanie McCammon
(on behalf of Mirjam Kuehne, Radia Perlman, & the Edu Team)

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 3 Jul 2014 23:13:43 +0000

Can following topic be added to the Chairs training?

Topic: WG drafts organization:
      - Major content to be included in WG's "framework draft" and "architecture draft". Show good examples of WGs that have produced good quality "framework" and/or "architecture" draft. Maybe show some bad examples for chairs to steer away.

      - Use case drafts: show some good examples (and some bad examples). Some use case drafts also list requirement. What are the criteria for the content?

      - Many WGs requirement drafts are authored by vendors, how to verify the completeness and enforce the solution drafts comply with the requirement drafts?

      - when there are too many drafts submitted, what are the effective method to consolidate? Is it necessary to consolidate all those drafts?

My two cents.

Linda
-----Original Message-----
From: WGChairs [mailto:[email-masked]] On Behalf Of IETF Agenda
Sent: Thursday, July 03, 2014 3:52 PM
To: Working Group Chairs
Cc: [email-masked]; [email-masked]
Subject: IETF 90 - Working Group Chairs Training Lunch - REMINDER

Hello Everyone,

Topic: English as a Second Language
1. Introduction and general problem statement 2. RFC Editor approach and experience with the workshop (Alice Russo, Heather Flanagan) 3. Experiences of a non-native English speaker (Ning Zong) 4. Open discussion - collect ideas, what are the best next steps?

If you have RSVP'd for the training lunch but will not be able to attend, or have any issues registering for the event, please send a message to [email-masked]<mailto:[email-masked]>.

Thank You!

Stephanie McCammon

---

From: Paul Wouters <[email-masked]>
Date: Thu, 3 Jul 2014 20:58:23 -0400 (EDT)

> https://www.ietf.org/registration/ietf90/rsvp.py?eventid=1&eventcode=348906

That requires a confirmation number that was not specified in this
email?

Paul

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 07 Jul 2014 15:10:55 +1200

Linda,

It's really too late for IETF 90 as the agenda is already full.
But I have some comments about your interesting question:

>       - Major content to be included in WG's "framework draft" and "architecture draft". Show good examples of WGs that have produced good quality "framework" and/or "architecture" draft. Maybe sho…

I think this is worth some debate, and not just among WG/BOF chairs.
My personal experience is that "framework" drafts often fail, simply
because no two people agree about their purpose or contents.
"Architecture" is a bit less troublesome, once people accept that
it is a question of protocol architecture, not systems or product
architecture.

So yes, two good topics but I think they need a lot of preparation.

>       - Use case drafts: show some good examples (and some bad examples). Some use case drafts also list requirement. What are the criteria for the content?

It's become a modern fashion to produce use cases, as a tool for
developing protocol requirements. Personally, I'm not sure we need
guidelines, because the use cases are really throw-away documents
and they probably will be very different for different cases.
But again, a good topic.

>       - Many WGs requirement drafts are authored by vendors, how to verify the completeness

First of all, WG drafts are controlled by the WG, so it's the WG's
job to verify the completeness and correctness. If the nominal
authors don't follow the WG consensus, the chairs need to sort it out.

> and enforce the solution drafts comply with the requirement drafts?

Again, that's a job for the WG as a whole.

>       - when there are too many drafts submitted, what are the effective method to consolidate? Is it necessary to consolidate all those drafts?

No. This can be very hard, but the WG has to take collective
responsibility for choosing a direction, not adopting the drafts
that don't support that direction, and then consolidating what
remains. And some authors will be unhappy at the end.

As for specific methods for managing this, that's another
really good topic. The problem is not exactly new - after all,
IPv6 is the result of exactly this problem, except that multiple
WGs were involved and it was the IESG that had to resolve it,
at the Toronto IETF twenty years ago!

I think you have given us enough ideas for the next 3 WG Chair lunches.

Regards
    Brian

---

From: <[email-masked]>
Date: Mon, 7 Jul 2014 14:06:09 +0100

Interesting topics

Framework /architecture - I think the key roles for this document are
- agree on the baseline assumptions (constraints on solutions)
- agree on the scope (what trying to solve, and what not)
- agree a high-level protocol model [RFC4101]
- agree on terminology

Use cases - I think this document is to motivate people and help with the framework doc

Requirements - personally, and perhaps controversially, I don’t find these helpful because the IETF doesn’t do top-down  design (from requirements to protocol) (unlike some standards bodies). Sometimes I've found them unhelpful, due to the time taken &/or too big a list of requirements (simply addition being an easy way of reaching consensus).  The framework doc gets the WG to agree what problem is being solved and broadly how. 

Best wishes
Phil

> From: WGChairs [mailto:[email-masked]] On Behalf Of Brian E
> Carpenter
> Sent: 07 July 2014 04:11
> To: Linda Dunbar
> Cc: Working Group Chairs; [email-masked]; [email-masked]
> Subject: Re: [edu-team] IETF 90 - Working Group Chairs Training Lunch -
> REMINDER
> 
> Linda,
> 
> >       - Major content to be included in WG's "framework draft" and
> "architecture draft". Show good examples of WGs that have produced good
> quality "framework" and/or "architecture" draft. Maybe show some bad
> examples for chairs to steer away.
> 
> "Architecture" is a bit less troublesome, once people accept that it is
> a question of protocol architecture, not systems or product
> architecture.
> 
> >       - Use case drafts: show some good examples (and some bad
> examples). Some use case drafts also list requirement. What are the
> criteria for the content?
> 
> guidelines, because the use cases are really throw-away documents and
> they probably will be very different for different cases.
> But again, a good topic.
> 
> First of all, WG drafts are controlled by the WG, so it's the WG's job
> to verify the completeness and correctness. If the nominal authors
> don't follow the WG consensus, the chairs need to sort it out.
> 
> >       - when there are too many drafts submitted, what are the
> effective method to consolidate? Is it necessary to consolidate all
> those drafts?
> 
> responsibility for choosing a direction, not adopting the drafts that
> don't support that direction, and then consolidating what remains. And
> some authors will be unhappy at the end.
> 
> As for specific methods for managing this, that's another really good
> topic. The problem is not exactly new - after all,
> IPv6 is the result of exactly this problem, except that multiple WGs
> were involved and it was the IESG that had to resolve it, at the
> Toronto IETF twenty years ago!
>

---

From: "Andrew G. Malis" <[email-masked]>
Date: Mon, 7 Jul 2014 09:14:01 -0400

Phil,

Actually, there have been cases where the IETF has produced a real
requirements document prior to protocol work to ensure that the
requirements have been satisfied. RFC 5654 is a good example.

Cheers,
Andy

On Mon, Jul 7, 2014 at 9:06 AM, <[email-masked]> wrote:

> Requirements - personally, and perhaps controversially, I don’t find these
> helpful because the IETF doesn’t do top-down  design (from requirements to
> protocol) (unlike some standards bodies). Sometimes I've found them
> unhelpful, due to the time taken &/or too big a list of requirements
> (simply addition being an easy way of reaching consensus).  The framework
> doc gets the WG to agree what problem is being solved and broadly how.
>
> Best wishes
> Phil
>

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 8 Jul 2014 08:00:43 +0000

My impression is that the statement ‘the IETF doesn’t do top-down  design (from requirements to protocol)’ is inaccurate, at least in part. An RFC search with the keyword ‘requirements’ in the title leads to a list of more than 250 RFCs. Since the end of 2010 (we were around RFC 60xx then) about 40 RFCs were published, most of them as Informational, many of them that I know were considered as input and referred to by protocol design work that followed.

I believe that a proper discussion on this topic should start from looking at this type of work, asking and answering the questions – did the ‘requirements’ RFCs help? Was the input taken into consideration? did it delay the process – if yes, how much? Did it improve the quality of the resulting protocols? Could we have done the work without them?

Regards,

Dan

From: WGChairs [mailto:[email-masked]] On Behalf Of Andrew G. Malis
Sent: Monday, July 07, 2014 4:14 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]; Brian E Carpenter
Subject: Re: [edu-team] IETF 90 - Working Group Chairs Training Lunch - REMINDER

Phil,

Actually, there have been cases where the IETF has produced a real requirements document prior to protocol work to ensure that the requirements have been satisfied. RFC 5654 is a good example.

Cheers,
Andy

On Mon, Jul 7, 2014 at 9:06 AM, <[email-masked]<mailto:[email-masked]>> wrote:
Interesting topics

Best wishes
Phil

> From: WGChairs [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Brian E
> Carpenter
> Sent: 07 July 2014 04:11
> To: Linda Dunbar
> Cc: Working Group Chairs; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

---

From: <[email-masked]>
Date: Tue, 8 Jul 2014 10:02:29 +0100

I agree it would be very interesting to look at some different examples in the IETF where doing a requirements RFC has been more or less useful – was this just the culture of the WG, the type of requirements, the nature of the protocol…  What lessons are there?

To clarify – what I meant is that there is no formal IETF process for moving from requirements to protocol, which there is in some standards bodies.

Best wishes
phil

From: Romascanu, Dan (Dan) [mailto:[email-masked]]
Sent: 08 July 2014 09:01
To: Andrew G. Malis; Eardley,PL,Philip,TUB8 R
Cc: [email-masked]; [email-masked]; [email-masked]; Brian E Carpenter
Subject: RE: [edu-team] IETF 90 - Working Group Chairs Training Lunch - REMINDER

Regards,

Dan

From: WGChairs [mailto:[email-masked]] On Behalf Of Andrew G. Malis
Sent: Monday, July 07, 2014 4:14 PM
To: [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Brian E Carpenter
Subject: Re: [edu-team] IETF 90 - Working Group Chairs Training Lunch - REMINDER

Phil,

Cheers,
Andy

Best wishes
Phil

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 08 Jul 2014 11:01:11 +0100

On 08/07/14 10:02, [email-masked] wrote:
> I agree it would be very interesting to look at some different
> examples in the IETF where doing a requirements RFC has been more or
> less useful – was this just the culture of the WG, the type of
> requirements, the nature of the protocol…  What lessons are there?

If someone had the time and energy, that could be a
really good bit of work. I think we've seen a number
of wgs where the whole use-cases, framework, architecture
and requirements RFCs have taken ages, by which time
there wasn't much energy for developing an actual
protocol spec. I'm sure there are more nuances as well,
and we all probably remember the cases that irked us
and forget others that trigger our biases less well,
so if someone had the cycles, this could be really
interesting and useful.

What I'd love to see (but won't get to see probably:-)
would be an analysis of what happened when protocol
specs are preceded by RFCs that document use-cases,
a framework, architecture or requirements. I'd like to
see both the relative timings and some measure (probably
subjective, maybe from protocol spec authors) of how
much better/worse/different the eventual RFC was compared
to what would have been done say 6 months after the
wg was chartered, and also some evaluation (also subjective,
perhaps not involving any RFC authors:-) of how useful
the protocol has been for the Internet compared to
expectations at say the BoF stage. I'd also be interested
in the same subjective information for wg protocol spec
RFCs that were not preceded by any of those kinds of
RFC. If someone did have cycles I'd be happy to comment
on plans like the above before they spent much effort.

S.

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Tue, 8 Jul 2014 10:32:32 +0000

I'd contend that every WG starts with requirements, even if they can only be implicitly derived from the charter.

Many WG find it useful to explicitly document those requirements.

Those requirements are then used to identify that the main product of the working group (call it the solution, the protocol, or whatever) has actually achieved what was set out to do, and therefore the resultant RFC can be published, and / or the WG closed.

When those requirements are explicitly documented, then it is a separate question as to whether those requirements might be published and whether a published document is useful.

What is sometimes useful to be published, in terms of providing a basis for why the protocol is like it is for external readers, is an analysis of how the protocol addressed those requirements.

Coming to the other aspect of the discussion, so called "framework" documents are a rather more nebulous concept. Sometimes they seem to contain an underlying architecture, sometimes they contain the above mentioned requirements, sometimes they are any overview of how a group of other published RFCs may fit together and so on.

At the end of the day I would leave it up to the WG and IETF consensus to publish what is considered useful, and not publish what is not.

regards

Keith

________________________________
From: WGChairs [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: 08 July 2014 09:01
To: Andrew G. Malis; [email-masked]
Cc: [email-masked]; [email-masked]; Brian E Carpenter; [email-masked]
Subject: RE: [edu-team] IETF 90 - Working Group Chairs Training Lunch - REMINDER

My impression is that the statement 'the IETF doesn't do top-down  design (from requirements to protocol)' is inaccurate, at least in part. An RFC search with the keyword 'requirements' in the title leads to a list of more than 250 RFCs. Since the end of 2010 (we were around RFC 60xx then) about 40 RFCs were published, most of them as Informational, many of them that I know were considered as input and referred to by protocol design work that followed.

I believe that a proper discussion on this topic should start from looking at this type of work, asking and answering the questions - did the 'requirements' RFCs help? Was the input taken into consideration? did it delay the process - if yes, how much? Did it improve the quality of the resulting protocols? Could we have done the work without them?

Regards,

Dan

Phil,

Cheers,
Andy

Requirements - personally, and perhaps controversially, I don't find these helpful because the IETF doesn't do top-down  design (from requirements to protocol) (unlike some standards bodies). Sometimes I've found them unhelpful, due to the time taken &/or too big a list of requirements (simply addition being an easy way of reaching consensus).  The framework doc gets the WG to agree what problem is being solved and broadly how.

Best wishes
Phil

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 8 Jul 2014 10:01:22 -0400

> and they probably will be very different for different cases.

The throw-away nature of some of these documents and their long-term value
as artifacts worth preserving is where I think we run into a lot of
problems.  IMO, architectures and requirements are great places to start but
seldom survive intact through the development process.  It should be no
surprise that's a common problem in companies as well.

Use cases are even more disposable.

Much of our process is tied up with producing I-Ds and eventually
shepherding them to RFC status.  One thing that's never clear from any given
RFC without a lot of WG or IETF context is the value of such a thing if it's
not an outright protocol document.  At best it's a "here's what we were
thinking at the time".

What long term process requirements do we have as an organization to
enshrine these transient artifacts?  If we don't have such requirements, can
we find some way to burn less time on them?

The other end of this type of process discussion, updating development
artifacts is probably worth ignoring for the time being.  Even meticulous
companies are pretty bad about that part of the process, IMO.

-- Jeff