Survey and Interview Design

Bad questions produce confident-looking nonsense. You design instruments where every question earns its place by serving a research objective, nothing leads the respondent, and the sample can support the conclusions the owner wants to draw. You choose the method to fit the question: surveys measure how many, interviews explain why.

When to use

• "Let's survey our customers about the new pricing."
• "Write an interview guide for talking to churned users."
• "I want to validate this product idea with ten people."
• "How many responses do we need?"
• "Interview some experts about the market."

What you need

• The decision the research will inform and what the owner most needs to learn.
• Who the participants are and how to reach them: a customer list from a connected CRM (HubSpot, Salesforce), an email list, a panel the owner pays for, or personal introductions. You never contact participants without the owner's go-ahead.
• The survey tool the owner already uses (Google Forms, Typeform, Microsoft Forms, a CRM's survey feature) or a free one; for interviews, a calendar (connected Google Calendar) and a call tool.
• Any constraints: incentives, deadline, languages, legal or ethical limits.

Method

1. Write research objectives first. Three to six objectives, each a thing to learn ("understand why trial users do not convert", "measure how many customers would pay for X"). Every question later must map to one; questions that do not are cut.
2. Choose the method:
   • Interviews to explore why, how and in what context; to find problems you did not know about; early in a topic. Typically 5 to 15 per distinct segment. In one well-known study of a homogeneous group, most themes appeared within the first 12 interviews (Guest, Bunce and Johnson, 2006); stop when new interviews stop adding new themes.
   • Surveys to measure how common something is, compare groups, or track change, once you know the right questions and answer options.
   • Often both: interviews first to learn the language and options, then a survey to size them.
3. Define who, and screen. Write inclusion and exclusion criteria. For surveys, add two or three screener questions at the start. For interviews, recruit a mix within each segment (heavy and light users, recent and long-term customers, churned users), and avoid only talking to fans.

Interview guides

4. Structure a 30 to 45 minute guide: introduction and consent (2 minutes), warm-up about their role and context (5), core topics (25 to 30), wrap-up and "anything I should have asked?" (5).
5. Ask about past behaviour, not hypothetical futures. "Tell me about the last time you ..." gets facts; "Would you use ...?" gets polite guesses. Ask what they did, what it cost them, what they tried instead.
6. Use open, neutral questions and probes: "how", "what happened next", "why was that", "can you show me". Let the participant lead, tolerate silence (it often brings the best answers), and do not pitch, defend or explain the product.
7. Run one-to-one or two-to-one with a separate note-taker if possible. Record only with explicit consent, and note timestamps of key quotes.
8. Think about harm. Avoid questions that could distress people without good reason; offer to skip any question.

Surveys

9. Keep it short: aim for under 10 minutes, usually 15 to 25 questions. Completion falls as length grows.
10. Write each question to the rules in references/question-writing-checklist.md: one idea per question, neutral wording, no assumptions, balanced and exhaustive answer options, "don't know" or "not applicable" where honest, consistent scales (5 or 7 points, labelled at every point or at least both ends).
11. Order matters: screeners, then easy general questions, then specific ones, sensitive and demographic questions last. Randomise the order of option lists where order could bias answers (not for ordered scales).
12. Size the sample if you want to generalise to a population. For a proportion, with a simple random sample:
    • n = z^2 x p(1 - p) / e^2, with z = 1.96 for 95% confidence, p = 0.5 if unknown, e = margin of error
    • for e = 5 percentage points: n = 1.96^2 x 0.25 / 0.0025, about 385 completed responses
    • for small populations, adjust: n_adj = n / (1 + (n - 1) / N), where N is the population size (for N = 1,000, about 278)
    • divide by the expected response rate to get how many to invite Compute with execute_code. If the sample is a convenience sample (social followers, an open link), say plainly that margins of error do not apply and results describe respondents only.
13. Pilot before launch: test with 3 to 5 people from the target group, watching or asking them to think aloud. Fix confusing questions, then freeze the instrument.

Consent, privacy and launch

14. Consent and privacy: say who is running it, why, how long it takes, what data is collected, how it is stored and for how long, whether answers are anonymous or confidential, and how to withdraw. Collect only the personal data you need. Where data protection law applies (for example GDPR), check the lawful basis and privacy notice with the owner; consent must be freely given.
15. Get the go-ahead. Show the owner the final instrument, the invitation text and the participant list. Only after explicit approval do you send invitations or book interviews.

Choosing quickly

The owner wants to know	Use
Why customers do something, or what problems exist	Interviews (5 to 15 per segment)
How many customers have a problem or preference	Survey, after interviews have shown the options
Whether a new idea has demand	Interviews about current behaviour, then a real commitment test (pre-order, waitlist, pilot), not a "would you buy" survey
How satisfaction changes over time	Short repeated survey with identical questions
What experts think about a market	Expert interviews, with each view treated as one source, not a measurement

Incentives

Small, equal incentives for everyone who takes part (a voucher, a donation) improve response without buying answers. Agree the amount, the budget and how it will be delivered with the owner before inviting anyone; never promise one you cannot approve.

Output

• Interview guide or survey instrument, with each question tagged to its objective.
• A one-page plan: objectives, method, participants and screening, sample size reasoning, timeline, incentive, consent text, analysis plan.
• Invitation and reminder drafts for the owner to approve. Put the plan on a show_card for sign-off.

Checks before you finish

• Every question maps to an objective; orphans were cut.
• No leading, loaded, double-barrelled or hypothetical-future questions remain.
• Answer options are exhaustive, non-overlapping and include "don't know" or "not applicable" where honest.
• Sample size reasoning is shown, or the convenience-sample limit is stated.
• The instrument was piloted and revised.
• Consent and privacy text is included and the owner approved before any contact.

Pitfalls

• Asking people to predict their own behaviour. "Would you buy this?" overstates demand. Ask about past behaviour and real commitments.
• Leading questions. "How much do you love our new feature?" measures politeness.
• Surveying too early. If you do not know the answer options yet, you need interviews first.
• Over-long surveys. Every extra question costs completions and quality.
• Claiming precision from convenience samples. No margin of error without a random sample.
• Sign-off. The owner approves all contact with participants and any incentives. Research involving children, health, employees' performance or other sensitive topics needs the owner's legal or ethics review before it starts.

Question checklist: references/question-writing-checklist.md. Credits: references/CREDITS.md.