Subject: The challenges of survey research (was Re: vacation locations)

From: Dave Crocker <[email-masked]>
Date: Sat, 23 Aug 2014 10:25:28 -0700

The IETF periodically conducts a poll, such as after every meeting.

An NPR article this week reminded me of the concerns we should have
about doing surveys:

     A Tale Of Two Polls

     http://www.npr.org/blogs/ed/2014/08/20/341668003/a-tale-of-two-polls

The URL includes the tape and a transcript.  Listen first, and then look
at the last line of the article.

Surveys are known to be quite sensitive to sampling, wording, and
organization.  Ask the same question in different ways and you are
likely to get different answers.  Ask the wrong people, and of course
you'll get wrong answers.

For example, our recent survey on meeting venue preferences queried the
entire community.  Most IETF meeting attendees are well-funded and have
plenty of travel time.  Not surprisingly, most pointed at popular
tourism sites, some of which are not all that convenient to very many
IETF participants.

If we want to improve IETF diversity at meetings, we need to hold the
meetings at places that are convenient, with good, inexpensive hotel and
food options near the main venue.

If we do a survey about venues, we need to talk to folk with limitations
in travel funds, travel time or constraints on food and possibly other
'resource' requirements.

Cater to the diverse, not the well-funded majority.

Cater in terms of meeting fundamentals, not pleasant and appealing
tack-ons that have no immediate benefit for doing IETF work.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Abdussalam Baryun <[email-masked]>
Date: Sat, 23 Aug 2014 21:17:55 +0200

I support your input and to continue updates and follow up with our
IETF surveys and their formats. Formatting the community survey needs to
match our challenges/objectives and to simplify valid findings.

AB