Subject: Re: [Rfced-future] Replacement text for Section 4.2

From: Michael StJohns <[email-masked]>
Date: Sun, 1 Aug 2021 16:48:33 -0400

Let me break this down into two section:  1) What is the scope of 
confidentiality?  2) What does the RSEA get to see and comment on prior 
to it going to the LLC - if anything?

For confidentiality, I have these questions:

1) Are truly anonymous comments permitted?
2) Does the IED get to see all of the comments with the source unredacted?
3) Does the RSAB get to see all of the comments?
    3a) paraphrased?
    3b) source unredacted?
    3c) source redacted?
       3c.1) With redactors comment on the source?
       3c.2) Without comment
4) Does the LLC get to see all of the comments?
    4a) paraphrased?
    4b) source unredacted?
    4c) source redacted?
        4c.1) With redactors comment on the source
        4c.2) Without comment on the source

I assume that if the IED gets to see all comments, they will act as 
redactor for any further distribution if redaction is required.

In the draft I assumed that 1, 2), 3c.1), and 4c.1) were the answers to 
the above questions because of the comments that were made by the call 
about confidentiality.    Obviously, if the LLC and the RSAB get to see 
everything, there won't be a redaction and comment on the source 
necessary.   If the LLC doesn't get to see everything, then it at least 
needs the context of possible biases from the submitters of the comments 
to help it understand the reality on the ground.

For the RSEA, I believe that it needs to be a bit more that Joel 
suggests below - and instead we should be making sure that any comment 
that will be considered by the LLC has to be provided to the RSEA for 
feedback and rebuttal in at least a paraphrased form.  I don't think the 
RSEA gets to know about any of the sources, but it seems both unfair and 
legally fraught not to provide them with an opportunity to make an 
opposing case for any negative comments.

And I'm glad that EKR brought up "legal".   Unlike the Nomcom process 
there is/will be a financial relationship between the LLC and the RSEA 
in the form of either an employment agreement or contract.   In the 
event that the LLC moves to terminate that relationship, if the RSEA 
feels it is being done for "unfair" reasons and can make a case to a 
legal standard in the appropriate legal venue, _*each and every comment, 
and all emails and notes discussing those comments will be subject to 
subpoena and the related depositions*_.

And lastly - no company I know of solicits company wide comments (unless 
the company is really tiny) on a particular employee's performance, nor 
does the manager not know who made the comment and hence that manager 
can accurately weigh said comments.  It would be a truly bad manager 
that did not ask an employee about their take on any  negative comments 
the manager received and the manager should help them redress any actual 
problems.

Mike

ps - Martin - "derogatory" is a term of art, mostly used related to 
credit worthiness,   it differs from negative, in that the derogatory 
info may affect the credit score, but there might be other negative 
information that can't be taken into account due to legal issues (e.g. 
for this "the RSEA is too old for the job" would be a negative comment, 
but not a derogatory one as age discrimination is generally prohibited 
and the comment couldn't be considered and would need to be discarded 
when received).   Happy though to change to "negative and positive" vice 
derogatory.

On 7/29/2021 11:49 PM, Joel M. Halpern wrote:
> I mostly agree with Ekr here.
>
> The only edge case I see is one that I know has come up with nomcoms. 
> If nomcoms see a pattern of reports of an issue, they have been known 
> to say to a candidate "there have been issues raised about X.  Do you 
> have any comment on that?"   This is not the same as showing the 
> candidate the feedback, even sanitized. It does reveal that 
> information has been provided.  (And, even more importantly, ti does 
> not give even a large portion of the feedback.)  The nomcoms where I 
> know this came up concluded that the combination of seeing the 
> candidates reaction and giving then a chance to respond was worth 
> pushing that envelop.   Do we consider that the selection committee or 
> LLC or someone has that right under this process already?  Do we want 
> to say anything about it?
>
> Yours,
> Joel
>
> On 7/29/2021 11:34 PM, Eric Rescorla wrote:
>>
>>
>> On Thu, Jul 29, 2021 at 5:00 PM Michael StJohns 
>> <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>     Replacement for section 4.2:
>>
>>               <<< Set a schedule - could be omitted, but this is a
>>     promise to
>>     the community
>>
>>                      as well as a schedule for the IED/RSEA >>>
>>          No more often than semi-annually nor less often than 
>> bi-yearly, the
>>          IETF Executive Director will send out to the community a 
>> call for
>>          input on the performance of the RSEA.
>>
>>           <<< Separate the IED's recommendation from the feedback 
>> process -
>>
>>            there may not actually be an action that needs an immediate
>>     recommendation >>>
>>
>>          <<<< the IED acts as the confidentiality mediator, anonymous 
>> and
>>     confidential
>>
>>                   comments get tagged in the document that goes to the
>>     LLC -
>>     this provides
>>
>>                   needed context. >>>>
>>          The IETF Executive Director and the stream managers will 
>> review the
>>          received feedback and produce a draft summary (a paraphrase)
>>          including both positive and derogatory information. Any 
>> feedback
>>          provided anonymously shall be tagged as such in the summary. 
>> Any
>>          feedback provided with a request for confidentiality shall 
>> likewise
>>          be tagged in the summary and the IETF Executive Director MAY
>>          comment on the weight that may be appropriate to apply to the
>>          comment based on their knowledge of the submitter.  The stream
>>          managers should ensure that any of their concerns are added 
>> to the
>>          document at this point and MAY also provide any commentary 
>> on the
>>          externally provided feedback that they feel appropriate.
>>
>>          <<< Make sure that the LLC hears both sides of the coin >>>
>>          That draft feedback, redacted with respect to any names 
>> associated
>>          with confidential comments and with respect to the IED's weight
>>          evaluations, shall be provided to the RSEA with for comment or
>>          rebuttal, and such comments shall be incorporated as an 
>> appendix to
>>          the final feedback document provided to the IETF LLC Board.
>>
>>
>> This doesn't seem appropriate to me.
>>
>> If people want to provide confidential comments, then they shouldn't 
>> be supplied to the RSEA,
>> nor should the IED's weights. This is not a legal proceeding, it's a 
>> performance review, and
>> there's no generic reason that the RSEA should get to hear all the 
>> feedback that the LLC is
>> receiving.
>>
>> -Ekr
>>
>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 1 Aug 2021 17:11:12 -0400

I am not following the bottom part of your note.

We are not a government agency, subject to government contracting 
regulations.
As far as I know, the restrictions on a company for hiring and firing 
contractors is always more generous than for employees.  The closest I 
have gotten in a review to knowing what the comments were from other 
people was when my boss at my first job out of school told me that he 
had received complaints (no idea of number, I did not get to see the 
complaints) that I typed too loudly.   (I took this as a sign that I 
should work somewhere else.)

I would expect that the LLC would want to get concerns discussed (which 
is why I raised the question to see if others agreed that an aggregating 
question was acceptable.

Forwarding the actual responses (even somewhat redacted) seems likely to 
cause problems.   Among other things, it seems likely to discourage 
negative feedback.

Sure, a law suit can of course be filed in the US under almost any 
circumstance.  And can attempt to subpoena almost anything.  But that 
does not justify over-sharing information during the normal process.

Yours,
Joel

---

From: Michael StJohns <[email-masked]>
Date: Sun, 1 Aug 2021 17:36:50 -0400

> We are not a government agency, subject to government contracting 

I know that - but we are a non-profit corporation incorporated within US 
and mostly subject to US labor laws and practices.  All I'm saying is 
that telling people that their comments will always remain confidential 
may run up against the law which says that's not the case.

> should work somewhere else.)

It a) depends on the contract - board members and C-suite staff 
contracts tend to have some pretty interesting poison pills as do 
by-name contracts for people with rare skills, but the typical employee 
is at-will and can be fired for any reason*, and b) it depends on the 
country or state.     *As long as those reasons don't implicate a few 
fair labor laws and regulations such as age, gender, race, national 
origin, political and religious beliefs, etc.

Re your comment - I took this that your boss had a sense of humor and 
provided you with information that had no effect on your actual review 
and compensation.  The key thing here is that your boss knew who said 
what, and you were given an opportunity to hear the issue.

>
> I would expect that the LLC would want to get concerns discussed 
> (which is why I raised the question to see if others agreed that an 
> aggregating question was acceptable.

>
> Forwarding the actual responses (even somewhat redacted) seems likely 
> to cause problems.   Among other things, it seems likely to discourage 
> negative feedback.

Contrariwise, not providing negative feedback to the RSEA is unlikely to 
result in any improvements if they are warranted.  And given this 
community, it's possible that the RSEA can provide missing context on a 
prospectively negative comment to assist the LLC in deciding what 
actions need to be taken.

And sorry - I'm not sure who gets to see the actual results in your 
model.  Can you tell me what each of RSAB, IED, LLC and the RSEA see please?

> does not justify over-sharing information during the normal process.

I still hesitate to see how providing the RSEA a summary/redaction of 
the comments received and allowing them to provide a response is 
"over-sharing".

Later, Mike

>> confidentiality?  2) What does the RSEA get to see and comment on 
>> prior to it going to the LLC - if anything?
>>
>> In the draft I assumed that 1, 2), 3c.1), and 4c.1) were the answers 
>> to the above questions because of the comments that were made by the 
>> call about confidentiality.    Obviously, if the LLC and the RSAB get 
>> to see everything, there won't be a redaction and comment on the 
>> source necessary.   If the LLC doesn't get to see everything, then it 
>> at least needs the context of possible biases from the submitters of 
>> the comments to help it understand the reality on the ground.
>>
>> suggests below - and instead we should be making sure that any 
>> comment that will be considered by the LLC has to be provided to the 
>> RSEA for feedback and rebuttal in at least a paraphrased form.  I 
>> don't think the RSEA gets to know about any of the sources, but it 
>> seems both unfair and legally fraught not to provide them with an 
>> opportunity to make an opposing case for any negative comments.
>>
>> there is/will be a financial relationship between the LLC and the 
>> RSEA in the form of either an employment agreement or contract.   In 
>> the event that the LLC moves to terminate that relationship, if the 
>> RSEA feels it is being done for "unfair" reasons and can make a case 
>> to a legal standard in the appropriate legal venue, _*each and every 
>> comment, and all emails and notes discussing those comments will be 
>> subject to subpoena and the related depositions*_.
>>
>> And lastly - no company I know of solicits company wide comments 
>> (unless the company is really tiny) on a particular employee's 
>> performance, nor does the manager not know who made the comment and 
>> hence that manager can accurately weigh said comments.  It would be a 
>> truly bad manager that did not ask an employee about their take on 
>> any  negative comments the manager received and the manager should 
>> help them redress any actual problems.
>>
>> Mike
>>
>>
>> information that can't be taken into account due to legal issues 
>> (e.g. for this "the RSEA is too old for the job" would be a negative 
>> comment, but not a derogatory one as age discrimination is generally 
>> prohibited and the comment couldn't be considered and would need to 
>> be discarded when received). Happy though to change to "negative and 
>> positive" vice derogatory.
>>
>>
>>
>>> The only edge case I see is one that I know has come up with 
>>> nomcoms. If nomcoms see a pattern of reports of an issue, they have 
>>> been known to say to a candidate "there have been issues raised 
>>> about X.  Do you have any comment on that?"   This is not the same 
>>> as showing the candidate the feedback, even sanitized. It does 
>>> reveal that information has been provided. (And, even more 
>>> importantly, ti does not give even a large portion of the 
>>> feedback.)  The nomcoms where I know this came up concluded that the 
>>> combination of seeing the candidates reaction and giving then a 
>>> chance to respond was worth pushing that envelop.   Do we consider 
>>> that the selection committee or LLC or someone has that right under 
>>> this process already?  Do we want to say anything about it?
>>>
>>> Yours,
>>> Joel
>>>
>>>>          <<<< the IED acts as the confidentiality mediator, 
>>>> anonymous and
>>>>     confidential
>>>>
>>>>          provided anonymously shall be tagged as such in the 
>>>> summary. Any
>>>>          managers should ensure that any of their concerns are 
>>>> added to the
>>>> If people want to provide confidential comments, then they 
>>>> shouldn't be supplied to the RSEA,
>>>> nor should the IED's weights. This is not a legal proceeding, it's 
>>>> a performance review, and

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 1 Aug 2021 14:53:17 -0700

It might be useful to distinguish here between what we think is good
practice and what this RFC is going to require.

In general, the executive management of a company has a fair amount of
latitude in what kind of HR practices they want to employ (subject to local
law, of course) They can do 360 reviews or not, solicit peer feedback or
not, share it with the employee or not, etc. I would expect the same here,
with the LLC and the IED being responsible for making those decisions. For
us to put specific requirements about how that feedback is collected and
shared is a bit like writing that into the company bylaws. Perhaps there is
some point which is truly necessary to be encoded in such a way that it
constrains the LLC but none of this text seems to me to fit that bill.

What I would prefer, therefore, is to omit essentially all of this
material. To the extent to which the community wishes to have input on
those processes, well, that's why the LLC board is answerable to the
community. And of course the RSWG can always write an RFC to add new
requirements if it deems it necessary in the future.

-Ekr

On Sun, Aug 1, 2021 at 2:37 PM Michael StJohns <[email-masked]>
wrote:

> --
> Rfced-future mailing list
> [email-masked]
> https://www.iab.org/mailman/listinfo/rfced-future
>

---

From: Michael StJohns <[email-masked]>
Date: Sun, 1 Aug 2021 18:10:29 -0400

Strangely enough I'd be happy with omitting any requirement for the LLC 
to accept comments from the community and that includes the RSAB.  But I 
grok, that is not what some people want, and if there is a requirement 
for community feed back, then there needs to be a way of ensuring that 
feedback goes through some form of fairness filter and a way for the 
target of that feedback to rebut or provide context for negative feedback.

Jay has indicated that the LLC desires to defer mostly to what the 
community wants.  ISTM that makes sense in the macro processes, but less 
sense when the target is one specific person employed or contracted by 
the LLC where the law and contracts have to outweigh the community 
desire of the moment.

Later, Mike

On 8/1/2021 5:53 PM, Eric Rescorla wrote:
> latitude in what kind of HR practices they want to employ (subject to 
> local law, of course) They can do 360 reviews or not, solicit peer 
> feedback or not, share it with the employee or not, etc. I would 
> expect the same here, with the LLC and the IED being responsible for 
> making those decisions. For us to put specific requirements about how 
> that feedback is collected and shared is a bit like writing that into 
> the company bylaws. Perhaps there is some point which is truly 
> necessary to be encoded in such a way that it constrains the LLC but 
> none of this text seems to me to fit that bill.
>
> On Sun, Aug 1, 2021 at 2:37 PM Michael StJohns <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     >>>> <[email-masked] <mailto:[email-masked]>
>     <mailto:[email-masked] <mailto:[email-masked]>>>
>     wrote:
>     >>>>
>     >>>>          with confidential comments and with respect to the
>     IED's
>     >>>> weight
>     [email-masked] <mailto:[email-masked]>
>     <https://www.iab.org/mailman/listinfo/rfced-future>
>

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 1 Aug 2021 15:14:48 -0700

On Sun, Aug 1, 2021 at 3:10 PM Michael StJohns <[email-masked]>
wrote:

> Strangely enough I'd be happy with omitting any requirement for the LLC to
> accept comments from the community and that includes the RSAB.  But I grok,
> that is not what some people want, and if there is a requirement for
> community feed back, then there needs to be a way of ensuring that feedback
> goes through some form of fairness filter
>

The details of this seem like something that we could leave to the LLC to
decide.

and a way for the target of that feedback to rebut or provide context for
> negative feedback.

This seems like something that might be good practice in many cases and bad
in some others, but in any case doesn't need to be encoded in an RFC.

-Ekr

-Ekr

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 2 Aug 2021 11:05:40 +1200

I have to say that the notion of our colleague Joel as a young man
being dinged for typing too loudly has made my day.

(I certainly typed too loudly as a grad student, since all the PDP-8
code that I wrote had to be keyed into an ASR-33 teletype, and typing
quietly on that device was physically impossible.)

Regards
   Brian Carpenter

---

From: Michael StJohns <[email-masked]>
Date: Sun, 1 Aug 2021 19:09:10 -0400

So delete section 4.2 in its entirety? I'm good with that.

Mike

On 8/1/2021 6:14 PM, Eric Rescorla wrote:
>
>
> On Sun, Aug 1, 2021 at 3:10 PM Michael StJohns <[email-masked] 
>     Strangely enough I'd be happy with omitting any requirement for
>     the LLC to accept comments from the community and that includes
>     the RSAB.  But I grok, that is not what some people want, and if
>     there is a requirement for community feed back, then there needs
>     to be a way of ensuring that feedback goes through some form of
>     fairness filter
>
>
> The details of this seem like something that we could leave to the LLC 
> to decide.
>
>
>     and a way for the target of that feedback to rebut or provide
>     context for negative feedback.
>
>
> This seems like something that might be good practice in many cases 
> and bad in some others, but in any case doesn't need to be encoded in 
> an RFC.
>
> -Ekr
>
>
>
>
>> In general, the executive management of a company has a fair amount 
>> of latitude in what kind of HR practices they want to employ (subject 
>> to local law, of course) They can do 360 reviews or not, solicit peer 
>> material. To the extent to which the community wishes to have input 
>> on those processes, well, that's why the LLC board is answerable to 
>> the community. And of course the RSWG can always write an RFC to add 
>> new requirements if it deems it necessary in the future.
>>
>> -Ekr
>>
>>
>> On Sun, Aug 1, 2021 at 2:37 PM Michael StJohns

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 2 Aug 2021 11:38:24 +1200

> So delete section 4.2 in its entirety?  I'm good with that.

Really? I agree that it is not a good idea to specify too many details; the LLC clearly owns the process, but stating that community feedback is part of the process seems desirable. Something like:

   Periodically, the IETF LLC will evaluate the performance of
   the RSEA, including a call for confidential input from the
   community.

Also the last sentence of the current text needs to be said somewhere:

   Whether the RSEA role is structured as a contractual or employee
   relationship is a matter for the IETF LLC and the IETF Executive
   Director to determine.

 Brian

<mailto:[email-masked]>> wrote:
>>
>>     Strangely enough I'd be happy with omitting any requirement for the LLC to accept comments from the community and that includes the RSAB.  But I grok, that is not what some people want, and if there is a requirement for community feed back, then there needs to be a way of ensuring 
that feedback goes through some form of fairness filter
to decide.
>>
>>
>>     and a way for the target of that feedback to rebut or provide context for negative feedback.
>>
>>
>> This seems like something that might be good practice in many cases and bad in some others, but in any case doesn't need to be encoded in an RFC.
>>
>> -Ekr
>>
>>
>>
>>
>>> It might be useful to distinguish here between what we think is good practice and what this RFC is going to require.
>>>
>>> In general, the executive management of a company has a fair amount of latitude in what kind of HR practices they want to employ (subject to local law, of course) They can do 360 reviews or not, solicit peer feedback or not, share it with the employee or not, etc. I would expect the same 
here, with the LLC and the IED being responsible for making those decisions. For us to put specific requirements about how that feedback is collected and shared is a bit like writing that into the company bylaws. Perhaps there is some point which is truly necessary to be encoded in such a way that it constrains the LLC but none of this text seems to me to fit that bill.
>>>
>>> What I would prefer, therefore, is to omit essentially all of this material. To the extent to which the community wishes to have input on those processes, well, that's why the LLC board is answerable to the community. And of course the RSWG can always write an RFC to add new requirements 
if it deems it necessary in the future.
>>>
>>> -Ekr
>>>
>>>
>>> On Sun, Aug 1, 2021 at 2:37 PM Michael StJohns <[email-masked] <mailto:[email-masked]>> wrote:
>>>
>>>     >>>> <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>>>     >>>>
>>>     >>>>          be tagged in the 
summary and the IETF Executive Director MAY
>>>     >>>>          comment based on 
their knowledge of the submitter. The stream
>>>     >>>>          document at this 
point and MAY also provide any commentary
>>>     https://www.iab.org/mailman/listinfo/rfced-future <https://www.iab.org/mailman/listinfo/rfced-future>
>>>
>>
> 
>

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 2 Aug 2021 10:24:18 +1000

I thought the current 4.2 has about the right amount of detail; it's vague enough (e.g., 'periodically', 'criteria could include') to convey the general lay of the land without constraining things too exactly.

Cheers,

> On 2 Aug 2021, at 9:38 am, Brian E Carpenter <[email-masked]> wrote:
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Michael StJohns <[email-masked]>
Date: Sun, 1 Aug 2021 20:24:39 -0400

>     the RSEA, including a call for confidential input from the

Really.  We're going to inject a requirement that the LLC make a call 
for community input, that implies that the LLC will do that whether or 
not it makes contractual or HR sense to do and that the LLC is required 
to pay an undefined level of attention to any input from that process.  
I'd rather leave the whole evaluation thing in the LLC's hands to decide 
whether and when it needs to make a community call, and for the 
candidates being hired or contracted to be able to negotiate the impact 
of such input.    So if you want to specify the requirement for 
"confidential input from the community", then it's only fair to specify 
how that input will impact the LLC's decisions and provide the 
appropriate safety valves to allow the RSEA to push back against and the 
LLC to ignore inappropriate or biased input.

I'd rather leave this to the contracting process and for the LLC to do 
the right thing in conjunction with any HR and Legal help that they 
engage.  I expect they probably will ask for input, but it would be 
their choice to do so.

Was there any language in Jay's contract or employment agreement 
requiring the LLC to ask for public comment on his performance? If not, 
I don't see any reason to do that here.

>     relationship is a matter for the IETF LLC and the IETF Executive

Was there something similar in one of the documents when Heather was 
hired/contracted?  I forget.  I don't actually think its necessary as 
the LLC holds the money and hence the decision process on the engagement 
mode.

Mike

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 2 Aug 2021 13:23:28 +0900

Hello Mike, others,

> when received).   Happy though to change to "negative and positive" vice 

The problem here is that the dictionary meaning of "derogatory", which 
isn't just factually negative, but the equivalent of demeaning.
(see https://www.merriam-webster.com/dictionary/derogatory)

While that page also mentions the specific meaning in a credit report 
(which I didn't know about), I think that credit reports and personal 
evaluations are two different things, and we should not expect the 
average (engineer) reader to know the credit report meaning, nor to have 
to guess by themselves that it's the credit report meaning that applies 
in this (different) context if they happen to know that meaning.

Regards,   Martin.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 2 Aug 2021 09:33:20 +0200

Most of us having been in the position of having had anonymous feedback 
submitted in our own NOMCOM evaluations, I think we can all sympathize 
with Mike's concern about fairness (it's one reason that as a general 
rule I do not provide anonymous feedback).  I'm especially concerned 
when people who have little to lose throw such darts.

Notwithstanding that sympathy, I would be very concerned that REQUIRING 
feedback to someone whose contract or employment is not going to be 
continued might actually cause legal difficulties for the organization.  
What if the feedback turned out to be factually wrong?  What if it's 
debatable? What if the feedback was not the basis for not continuing the 
relationship?  Worse, that requirement might tie the IED's hands in 
terms of what employment mechanisms and jurisdictions are used.

If we're going to pursue this line, I would want us to hear from the LLC 
and the IED about any such risks.

Eliot

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 2 Aug 2021 13:46:04 -0600

On 8/1/21 5:38 PM, Brian E Carpenter wrote:
> Really? I agree that it is not a good idea to specify too many details; the LLC clearly owns the process, but stating that community feedback is part of the process seems desirable. 

Well, and also stating that the RSEA will be evaluated.

> Something like:

Short and sweet. :-)

>    relationship is a matter for the IETF LLC and the IETF Executive

No matter what we do with §4.2, I suggest moving this text to §4.

Peter