Annex X: When to Use KIIs, FGs and Surveys

Interviews: These interviews are structured conversations with specifically selected individuals (key informants, giving them the name key informant interview, KII) with the knowledge or lived experience to explain a particular topic, event or viewpoint. Interviews offer a flexible format, where you can prepare certain questions in advance but, in the moment, can adapt and follow up on unexpected or unanticipated answers or lines of inquiry. However, they are time-intensive, and even a group of interviews do not represent public opinion. 

Pros

• Low to no cost (except time)
• Flexible: Can supplement a process with expert testimony and can add questions on the fly
• Informative: Offer a moment to get rich, detailed information on a particular topic

Cons

• Will not result in a representative sample
• May require time spent scheduling or (if face-to-face) traveling to meet interviewees
• Transcripts may result in a large quantity of data that is difficult to draw concrete conclusions from

When to use interviews

• When you need particular expertise (on a topic or a particular identity group)
• Early in your design process, when you are determining what questions to ask and how best to reach people
• During data analysis, to understand confusing or difficult-to-parse results or to determine particular solutions in response to data collection

Focus groups: A focus group is a structured, facilitated discussion with a particular group. Participants of a focus group are often selected (as randomly as possible) because they share a particular, important characteristic or characteristics (women under age 30, for example). These discussions allow researchers to go more in-depth on a particular issue but generate large amounts of data in the form of a transcript or written record of the spoken testimony. In addition, facilitators must be trained to avoid biasing the respondents and ensure everyone is comfortable and able to speak. Because a focus group is a particular research methodology, it is likely to incur costs in training and preparation for your team or costs from hiring an external research firm to conduct them. In addition, the data is not representative of the population, but the in-depth conversation can reveal hidden or surprising sentiments or reasons for policy preferences that a normal survey would not.

Pros

• Provides a wealth of in-depth, quality data on the sentiments of a particular group
• Can reveal the reasoning behind public sentiment, including potentially false or misleading narratives
• Can probe respondents for their ideas on solutions and the root cause of problems

Cons

• Will not result in a representative sample, and participant selection process may introduce bias
• Is a technical methodology that requires a trained research team to implement and may be costly in time, money and human resources
• Transcripts may result in a large quantity of data that is difficult to draw concrete conclusions from

When to use focus groups

• When you are trying to understand the viewpoints of a particular demographic or group of people (and you are confident you can effectively recruit a randomized sample of participants)
• If you already have some data on sentiment (for example, most residents think crime is a problem) but want to understand more about how people view the problem and potential solutions
• If your survey response rate is too low to be representative, and you want to validate your next steps with residents who might be directly affected by the investment

Surveys: A survey aims to gather sufficient responses from a representative sample of the local community. Survey questionnaires should be short and structured simply to ensure that respondents understand the questions and can respond accurately. The key challenge is obtaining responses from a representative sample, which is linked to how surveys are distributed to residents. Your team will likely issue surveys that residents can answer and fill out themselves, referred to as “self-administered” surveys. For example, surveys can be mailed or distributed online, but these tend to have low response rates. Surveys that are handed out personally in a door-to-door canvassing effort have a higher response rate, but canvassing is time-intensive and works best in a small community. You can also use convenience sampling (handing out surveys in a public place), but this can introduce bias. In addition, consider how respondents will return the surveys and how long they have to do so. Finally, consider your abilities to analyze data – will you aim for only a quantitative ranking of responses (the simplest), or will you include open-ended questions? Open-ended questions can be very time-consuming to analyze, especially if surveys are handwritten (and the data needs to be transferred into digital software to analyze).

As a final note, local research firms may also offer services to conduct surveys, so consider if this is within your budget (and if you view the data as worth the cost). Be sure to ask about their sampling and interviewing methodology, which can involve phone or online interviews. 

Pros

• Data is representative of the local community and may be more inclusive as you can reach community members who do not often engage with government
• Quantitative analysis is relatively straightforward and can be visualized easily in charts and graphs
• Various sampling and distribution strategies can be deployed (and mixed) to reach a representative sample

Cons

• Is less personal than face-to-face methods (unless combined with door-to-door canvassing), which can lead to lower response rates
• Qualitative analysis (of open-ended answers, for example) can be time-consuming and complex or even require special software
• Risks of bias from sampling error are difficult to manage

• When your issues or problem areas lend themselves to a simple ranking of priorities – i.e., they are relatively coherent and easy to distinguish from each other ("water and sanitation" vs. "roads," for example).
• When you are confident you can receive surveys from a sufficient and representative sample of residents within a reasonable time frame