NPS and CSAT Analysis

Survey scores are easy to calculate and easy to misread. You compute them correctly, show how much of any change is just noise, and spend most of the effort on what customers wrote, because that is where the reasons are. The standard: every score is reported with its sample size and margin of error, no change inside the margin is called a trend, every theme is counted and quoted, and the report ends in actions with owners, including closing the loop with the people who answered.

When to use

• "Analyse the NPS survey that closed on Friday."
• "Why did CSAT drop this month?"
• "What are detractors complaining about?"
• "Build a monthly survey report."
• "Which customers should we follow up with after the survey?"

What you need

• Raw responses, not just the dashboard number: respondent or account ID, score, comment, date and channel. From the survey tool or helpdesk as a connected app, or a CSV export.
• Account attributes to segment by: plan, revenue band, tenure, region, product used, account owner (CRM export).
• How the survey was sent: to whom, when, how many invitations, the question wording and the scale. This decides what the numbers can mean.
• Previous results for comparison, collected with the same method.

Method

1. Clean the data.
   • Remove test responses and exact duplicates (same person, same score, same minute).
   • Keep one response per person per survey wave unless the design says otherwise.
   • Record what you removed.
   • Calculate the response rate (responses / invitations) and note who did not respond. Low response rates and skews (only power users answering) limit what you can claim.
2. Calculate the scores with execute_code (formulas and code in references/survey-math.md):
   • NPS (0 to 10, "how likely to recommend"): promoters 9 to 10, passives 7 to 8, detractors 0 to 6. NPS = percent promoters minus percent detractors, from -100 to +100.
   • CSAT: state the scale and the definition. A common one is the percent of responses that are 4 or 5 on a 1 to 5 scale. Report the mean only alongside it.
   • Customer effort (CES): state the scale and its direction; report the distribution and the share reporting low effort.
3. Add margins of error.
   • NPS, 95 percent: about 1.96 x the square root of ((p + d - (p - d) squared) / n), times 100, where p and d are the promoter and detractor shares.
   • CSAT proportion: 1.96 x the square root of (s x (1 - s) / n).
   • With 100 responses, NPS margins are often around plus or minus 15 to 20 points; small movements mean nothing. Say so.
4. Compare carefully.
   • Compare with the previous wave only if the method, audience and timing match.
   • Treat a change as real only if it is larger than the combined margin, and say how confident you are.
   • Look at the distribution, not just the net score: a shift from passives to detractors matters even when NPS barely moves.
5. Segment. Scores by plan, revenue band, tenure, region, product and account owner.
   • Report a segment only when it has at least about 30 responses; below that, show the count and mark it indicative.
   • Add a revenue-weighted view: a few large unhappy accounts matter more than many small happy ones.
6. Code the comments.
   • Read a sample of 50 to 100 comments and build a codebook of 8 to 15 themes (for example ease of use, reliability, support quality, price, a specific missing feature, onboarding, performance), each with a definition and an example.
   • Code every comment with up to two themes and a sentiment (positive, negative, mixed).
   • Re-read a random 10 percent to check your coding is consistent.
   • Count themes by score group.
7. Find the drivers. Which themes appear far more among detractors than promoters, and the reverse? Rank negative themes by frequency among detractors times the revenue they represent. Pull 2 or 3 short, representative quotes per theme, anonymised in the report unless the owner says otherwise.
8. Check against behaviour. Where you can, compare scores with what accounts did next (renewed, expanded, churned, kept using). This tells the owner how far to trust the survey as an early warning.
9. Build the close-the-loop list.
   • Detractors with comments, especially high-revenue accounts: a suggested owner, the issue, a suggested response.
   • Promoters: candidates for a review, referral or case study request. Each message is a draft for the owner's approval and is sent by a person. Never contact respondents who asked not to be contacted, and respect any promise of anonymity made in the survey.
10. Write the report (one page plus appendix): headline score with sample size and margin; change versus the last wave and whether it is meaningful; distribution; segment highlights; the top 3 negative themes with counts and quotes; the top 2 positive themes; recommended actions with owners and dates; the close-the-loop list. Put the headline on a show_card.
11. Improve the survey if needed: timing (CSAT shortly after the interaction; relationship NPS at a steady point in the customer's life, not during an outage), a follow-up question ("what is the main reason for your score?"), and consistent wording between waves.

Worked example: is it a real change?

• This wave: 120 responses; 45 percent promoters, 25 percent detractors. NPS = 20, margin about plus or minus 15.
• Last wave: NPS 12 from 110 responses, a similar margin.
• The change is 8 points; the combined margin for the difference is about 20 points.
• Report: "NPS 20 (plus or minus 15, n = 120), up from 12; within the margin of error, so not a meaningful change." Then move to the comments, where detractors cite export reliability 3 times as often as promoters do.

Output

• The one-page report, and an appendix with method, codebook, segment tables and all coded comments.
• A chart of the score distribution over time (stacked promoters, passives, detractors) with sample sizes labelled.
• The close-the-loop list with drafts for approval.

Checks before you finish

• Sample size, response rate and margin of error are shown beside every headline score.
• No change inside the margin is called an improvement or a decline.
• Segments under about 30 responses are marked indicative.
• Every theme has a count and at least one quote; the codebook is included.
• No respondent is identified where anonymity was promised; no message is sent without approval.
• Any outside benchmark is cited with source and date, or left out.

Pitfalls

• Reporting the number, not the reasons. The score tells you something moved; the comments tell you why.
• Chasing noise. A 4-point NPS change on 80 responses is not news.
• Averaging NPS. Averaging account-level NPS or the raw 0 to 10 scores is not NPS. Use the formula.
• Survey timing bias. Surveying right after an outage or a price rise measures that event.
• Benchmark worship. Industry NPS benchmarks vary widely by source and method. Your own trend is more useful.
• Asking and never answering. Customers who give feedback and hear nothing stop giving it.

See also: churn-risk-signals, product-feedback-loop.