Handling IESG ballot positions

So, you’ve got a protocol or an idea that you care deeply about and think the IETF should work on. You wrote an initial Internet-Draft, you made a number of presentations, and your document was finally adopted as a Working Group (WG) document. You then spent many months in discussions in the WG, discussing many topics at length; you laughed, you cried, you fought over complex technical details. Hundreds of emails have been exchanged, many revisions have been made, compromises have been agreed to. A few of the fights were contentious, and many seemed like bike shedding.

Whatever the case, the document was finally done and went through a Working Group Last Call (WGLC). A few issues were discovered; almost definitely there were some annoying comments and a pile of nits. You addressed them, consensus was declared, and then much of this got repeated during IETF Last Call.

By this point, you just want the document published; your primary source of solace is that the document is basically done… and then it hits Internet Engineering Steering Group (IESG) Evaluation. Some set of people (whom you may or may not know) review your document and dump a whole bunch of comments on you. Some of these are DISCUSS ballots, many of them are things you’ve already discussed and settled, many of them are nits, and some of them seem to be from people who have no real knowledge about the topic

So, what do you need to know about addressing these? 

To better understand how to deal with comments, it’s worth knowing some background and viewing these from the Area Director’s (AD) viewpoint. 

The IESG is composed of Area Directors, representing different Areas: Apps and Real-Time (ART), General (GEN), Internet (INT), Ops & Management (OPS), Routing (RTG), Security (SEC), and Transport (TSV). Just like any other IETF participants, ADs care more deeply about, and have more interest in and knowledge of some protocols than others; there are many many WGs in the IETF, and, as you’ve already noticed, many many emails.

The IESG operates on a 2-week cycle, reviewing around 7-14 documents (roughly 200-400 pages) each cycle. ADs ballot on the documents and then discuss the document and their ballot positions on biweekly telechats. More information on balloting is in the IESG Ballot Procedures.

ADs ballot by choosing a position and (often) providing comments to explain their position and point out places where they think the document could be improved. Discuss ballots (usually capitalized as DISCUSS, either to draw attention to them, or to strike terror into the recipient’s hearts) are special, in that they a: require an explanation of the position and b: are blocking. 

In almost all cases, reviewing a document for the telechat is the first time an AD sees your document. This means that they are not familiar with the contentious discussions that led to the contrived wording in section 4.3.5.3, nor the history behind why the document uses the term ‘widget’ instead of ‘device’ or ‘router’.  

This is a feature, not a bug - you are writing an RFC, an archival document, that will (hopefully!) be read by many people who were not involved in its creation. If it isn’t clear to a random AD, it’s not likely to be clear to a random implementer. 

While it is possible that the AD is an expert in your particular work, there are currently more than 120 WGs, and so the AD being an expert in your particular protocol is not a likelihood.

ADs tend to be generalists. They have usually been participating in a bunch of WGs in the IETF, and they have a good idea of how the various protocols fit together.

In addition to having their pet protocols, ADs are mainly looking for a few things:

1. Has the process been followed correctly?
2. Is it clear enough that someone on a desert island could implement it using just the document and the normative references?
3. Does it conflict with other RFCs or break existing implementations? 
4. How does it impact my area?

Unsurprisingly, SEC ADs pay lots of attention to the security implications, OPS ADs pay attention to the operations and manageability, TSV ADs focus on the transport implications, etc.

Great, now you have a general understanding about where these ballots come from, but getting a bunch of additional work when you thought you were done can be a bit disheartening. Some of the ballots are marked DISCUSS, some seem to just have comments, some of them make no sense, and some of them (annoyingly) are pointing out typos. What does all this mean? What do you do?

DISCUSS

Authors often view DISCUSS as a nuclear issue. While they do need to be addressed, the name is important - they are DISCUSSes, not “YOUR IDEA IS BAD, AND YOU SHOULD FEEL BAD”s. ADs are careful and considered when balloting DISCUSS. As generalists, focusing on their area, they are raising a (blocking) concern that they think really really needs to be addressed to make the document publishable. 

From IESG Ballot Procedures: “Discuss” may mean “I cannot in good conscience send this document forward, but if it were fixed in these ways, I would change my ballot position to either Yes or No Objection”, or it may literally mean “I think we need to talk about this.”. There is also the “Discuss Criteria” document, which explains which classes of concerns are DISCUSS worthy.

Sometimes the DISCUSS is pointing out technical errors (“You say the NEIGHBOURS field is 8 bits, but you are trying to stuff the value 257 into it in section 3. 257 > 255.”), and internal inconsistencies (ranging from "table 1 says this thing is MAY but table 7 says it is SHOULD" to "your example seems to not include this field that the protocol says is mandatory"). Often, however, the DISCUSS is raising issues with how the protocol interacts with other protocols or systems. Most often, though, the DISCUSS comes from the fact that the AD is coming to the document “fresh”. Remember, I have no idea on the background of why it is a MUST, not a SHOULD in step 42 of section 17… and neither will an implementer who wasn’t involved in the WG discussion. 

There are also so-called “discuss discusses” - don’t worry, this isn’t “DISCUSS*2” or “DISCUSS^2” - this is more along the lines of “I think that there is something really important here, but I’m not sure/suspect there is a good reason for why you did what you did. Let’s have a discussion.” An example of this is https://datatracker.ietf.org/doc/rfc6272/ballot/

The bottom line is that a DISCUSS ballot is a request to have a discussion. The responsibility for initiation and driving this discussion lives mainly with the authors, but the DISCUSSing AD has a responsibility to review and respond in a timely manner. The Responsible AD (one who brought the document to the IESG in the first place) is also expected to help with this discussion.

While the outcome is often a change to the document, the changes are usually ones that serve to make the document clearer, and that’s an improvement we should all be happy with. It’s also the case that sometimes the outcome is education of the AD, and “Nothing to see here; let’s move on.”

The important things to remember when handling DISCUSSes are:

1. Take a deep breath; we are all here to try to write the best RFCs we can. ADs are not trying to be jerks, nor on a power trip (and, if they are, there is always the recall procedure!).
2. These are blocking, and you do need to address them to move the document forward. It’s your responsibility to follow up, etc.
3. ADs do not ballot DISCUSS lightly (apart from just not wanting to be jerks, a DISCUSS adds to the ADs’ workload as they need to discuss and clear it). 
4. The title is important - they are DISCUSS positions; DISCUSS with the balloting AD.
5. Sometimes ADs are just wrong; we too have bad days, we sometimes rush through balloting, sometimes we accidentally put on a pot of decaf instead of regular coffee and misread something. If a DISCUSS makes no sense, see bullet 4…
6. Even if you immediately agree with the DISCUSS position and know exactly what you need to do to fix it, it’s important to explicitly reply to the email message and let the AD know what the status is and what you’re doing about it.  ADs are dealing with many documents, and without a direct response it’s very hard for us to keep track of everything.

There is no “magic bullet” to dealing with a DISCUSS. Some of them will be trivial to address (often the AD will include text which will address their concerns), but some will need a number of rounds of discussion and back and forth with the AD (and possibly the WG). While doing this, it's (still!) important to remember that this is a discussion between people, and that we are all here to try and make the best standards possible. 

Work with the DISCUSSing AD, and feel free to include the responsible AD and WG chairs, and whomever else you need to talk to. Once the AD (or ADs!) holding the DISCUSS position(s) are satisfied, they will clear their DISCUSS position. The responsible AD will probably notice that this has occurred, but it’s also possible that this will fall through the cracks, so feel free to send them email letting them know that all DISCUSSes are now cleared.

Comments

Comments do not have to be addressed; they are not blocking. However, ADs really do want to make your document better, and they took some time to consider and add these comments. You are writing an RFC to be part of a venerable, archival, and important document series, not a throwaway shopping list. 

Please consider the comments, and take some time to think about why the AD added them. 

Comments should be reviewed and digested carefully. The ADs might spot something they think is minor but actually turns out to be a problem in the protocol. If they had the specialist knowledge, they might have raised it as a DISCUSS but that didn't happen. ADs are fairly representative of the eventual audience of the document; if something wasn’t clear to them or seems confusing, it’s going to be unclear or confusing to others; make your best effort to address these to make the document better. In any case, it’s just courtesy to consider reviews and comments from people who took the time to make them.

Nits

Nits are included in the comments section of the ballot, and are almost always editorial in nature. Once again, ADs are reading your document with fresh eyes, and so are likely to stumble across things like typos and similar items that you missed. You already know what the sentence meant to say, and so your eyes are performing autocorrect, but when reviewing a document if I come across “the router send hte packet out the interfce” I basically have to point out the typos. Some authors believe that the IESG isn’t responsible for editorial matters - while that’s technically true, we are all here to write the best documents possible. While the RFC Editor is ultimately responsible for catching and fixing these things, they, too, are people, and it’s somewhere between impolite and outright rude to have the “Meh, I can’t be bothered to fix these, that’s the RFC Editor’s job…” view.

In summary, we are all trying to write good quality, understandable, and implementable documents. IESG review provides the benefit of review by people with different knowledge bases and skill sets, and a focus on particular areas. DISCUSS is a blocking position, but it’s specifically “discuss”, not “die, die, die!!!” [“the document is broken”]. Comments are not blocking but will almost definitely make the document better (and you are encouraged to follow up to ask for clarification, etc.).

Once all of the DISCUSS ballots have been cleared, be sure to remind the Responsible AD. After they’ve checked that everything is good, they can approve and send it to the RFC Editor to be published as an RFC. As part of their process, the RFC Editor may have further questions, suggestions or edits to ensure the document is as clear as possible and conforms to the RFC format and style. You likely will be engaged again to confirm the document is exactly as it should be. But, that’s a topic for another blog post. 

Helpful links

• The Tao of IETF - A Novice's Guide to the Internet Engineering Task Force
  This is basically “required reading” if you don’t want to spend your IETF time being confused about what’s going on.
• Scott Bradner’s “IETF Publishing an RFC” video
  This is part of the (excellent) IETF Newcomers presentations videos
• The IETF process: an informal guide
• IESG Ballot Procedures
• Mark Nottingham’s excellent blog post on "How to read an RFC”