IETF 108 Non-Participation Survey

Methodology

The survey was sent directly to the ~5250 unique email addresses that were used to register for IETF 101 to IETF 106 (2018/2019) but not used to register for IETF 108. There were a number of questions on the front page to screen out anyone who did participate in IETF 108 but just used a different email address, and to screen out anyone who did not know about IETF 108 and so would not be able to properly answer questions about the agenda and other meeting elements. Finally, the front page also asked people if they still participate in the IETF and if they don’t then they were asked an additional question about why they no longer participate.

In the body of the survey, if anyone gave the registration fee as their primary reason for non-participation or answered “extremely” or “very” for how much the fee affected their decision, then they were asked an additional question about the fee waiver.

504 people responded but 65 of those actually participated in IETF 108 leaving 439 who did not. The results presented here and in the download have been filtered to only include those 439 and when this post talks about “respondents” that is shorthand for “respondents who did not participate in IETF 108”.

Whether we take 504 from a population of ~5250 or 439 from a population of ~4573 (scaled down using the 439/65 split) the margin of error stays around +/- 4%, however this is a complex survey to analyse using a margin of error. 

Results

The results are available to download as a PDF IETF 108 Non-Participation Survey Results or live web page (third party site).

General insights

The first point of note is in the regional demographics (Q1), which has a quite different spread from meeting attendance for IETF 106 and IETF 108, with a far higher percentage of respondents from Europe and Africa than meeting participants and correspondingly less from North America and Asia. There are no questions that specifically drill down into why this might be the case.

The next point is that 22% of respondents did not know about IETF 108 (Q5), which we will address in future by sending details to previous participants.

Participation decision making

Questions Q7 to Q11 were only asked of people who did know about IETF 108 (78%), which means that the reason for non-participation for 22% was because they did not know about the meeting and so crudely, reduce the answer percentages by 22% if you want to apply the margin of error. Where that has been done below, this is shown as (adjusted). 

Looking at the factors (Q7) and primary reasons (Q8) for non-participation provides some useful insights:

• The most important factor and the biggest single reason for non-participation at 37% (29% adjusted) was because people were too busy to participate.  That’s to be expected and does not indicate any issue to be considered.
• The next factor by a big gap was the registration fee, which 19% (15% adjusted) gave as their primary reason. This is discussed in detail below.
• After that, again with a bit of a gap, comes “Because it was a fully online meeting” with 15% (12% adjusted) giving that as their primary reason.
• Then there are four factors all with pretty much the same relevance - the meeting timezone, schedule, agenda and productivity.  While all of these are important they were not overall strong factors affecting non-participation, which in the case of meeting timezone is probably not what people would have guessed.
• Finally, the issue of technology and working environment was the least important factor and only 3% (2% adjusted) listed that as their primary reason for non-participation.
• The free text responses (10%) largely fell into the categories above, albeit expressed differently.

Q9 asked people if they wanted to participate in the IETF 108 (Q10) and therefore missed an opportunity, with 69% (54% adjusted) saying yes. If we exclude those whose primary reason was outside of our control (too busy, averse to an online meeting or technology/workspace issues) then this figure stays at 69%.  This tells us that there is possible scope for us to improve the situation.

Impact of the registration fee

93 of the original 439 were asked why they did not apply for a fee waiver (Q11) and they were allowed to pick multiple answers to this question.  These people were chosen because they either gave the registration fee as their primary reason for non-participation, or they answered “extremely” or “very” for how much the fee affected their decision. The highest answers were:

• 48% were concerned that they were not eligible. 
• 45% did not know about the fee waivers.
• 33% did not want to stop someone from getting a waiver.
• 14% did not want someone to decide if they were eligible.
• The free text responses (15%) largely fell into one of the categories above.

If we filter down to just those who gave the registration fee as their primary reason for non-participation (50 of those 93 people) then the answers are roughly the same (40%, 52%, 32%, 20%, 10%).

All of this points to a need to revise the information around fee waivers so that it is much more accessible and much clearer that there are no checks on eligibility and an unlimited number on offer.

The same question also produced one very useful low result:

• Only 5% (5 people) did not want anyone to know that they needed a waiver

This suggests that concerns that the fear of fee waivers being public prevents the uptake of fee waivers are unfounded.

No longer participating in the IETF

We asked on the first page if the respondent was still participating in the IETF and 114 said no of which 113 explained why (Q12) with multiple answers allowed.  The top answers were:

• 35% because their employer no longer supports them in participating.
• 32% because they cannot spare the time needed.
• 12% because their work in the IETF is complete.
• 10% because the IETF was not the right place for their work.
• 30% of people gave a free text response and these are hard to categorise.

One answer, while much lower than the others, was nonetheless concerning:

• 4% said they stopped participating because they had a bad experience with the IETF.

Conclusion

This has been an exceptionally valuable survey and we want to thank all those people who took time to respond.  There are some obvious actions to take around fee waivers and telling people about our upcoming meetings a lot of useful data to be considered further.

(The section on "Impact of the registration fee" has be updated since this article was first published to provide more details on the numbers surveyed and to correct the final insight.)