Hermes Wiki
Developer/Analytics/ProductAnalytics/Fundamentals/product-analytics-funnels-cohorts-and-retention

Product Analytics: Funnels, Cohorts, and Retention

Concept

Product analytics is the practice of using in-product behavioral event data (not surveys or opinion) to understand what users actually do, in order to inform product decisions. Three analysis types do most of the work and answer three genuinely different questions:

  • Funnel analysis measures conversion through a defined, ordered sequence of steps — signup → email verified → first search → first booking — and answers "where do users drop off, and at what rate does each step convert to the next." It's a snapshot of a single flow, usually within a bounded time window per user (e.g. "completed within 7 days of starting").
  • Cohort analysis groups users by a shared starting point — everyone who signed up in the same week, everyone who first used a given feature — and tracks that fixed group's behavior over time. A cohort answers "do the people who came in through X actually stick around or behave differently from the people who came in through Y."
  • Retention analysis (usually run per-cohort) tracks what fraction of a cohort is still active N days/weeks after their starting point — "Day 1 retention," "Day 7 retention," "Day 30 retention." Retention curves that flatten out at some non-zero level indicate a durable core user base; curves that decay toward zero indicate the product isn't creating lasting habit for that cohort.

The three compose rather than compete: a funnel finds the leak in a specific flow (e.g. 40% of signups never complete step 2); a cohort tells you whether the users who did make it through the funnel actually stick around three months later. Activation is the metric that sits between the two — the percentage of new users who reach a defined "aha moment" (the first core action that correlates with long-term retention), which is usually the last step of the onboarding funnel and the entry point into the retention curve.

Tradeoffs

Approach Benefit Cost
Funnel analysis alone Fast to build, pinpoints exact drop-off step in a known flow Says nothing about what happens after the flow completes — a "successful" funnel completion can still churn immediately after
Cohort + retention analysis alone Reveals long-term stickiness and whether growth is durable Doesn't localize where in the product experience users are failing — a low Day-7 retention number doesn't say why
Funnel + cohort/retention together Diagnoses both the immediate leak and the downstream durability, and lets you segment retention by acquisition cohort to spot which channels/flows produce durable users More instrumentation and more dashboards to maintain; requires the tracking plan (see Event Tracking Plans and Naming Conventions) to already be consistent, or the funnel/cohort numbers themselves become untrustworthy
"Any activity" as the retention definition Simplest to compute — any event counts as active Inflates retention with users who logged in but got no value, masking real churn behind vanity activity
"Completed core action" as the retention definition Reflects users who are getting the product's actual value, a stronger signal of health Requires agreeing on and instrumenting what the core action even is — itself a product decision, not just an analytics one

When to use / when not to

  • Build a funnel first, once there's a specific multi-step flow worth measuring (signup, checkout, onboarding) and enough traffic per step for the drop-off numbers to be statistically meaningful rather than noise from a handful of users.
  • Add cohort/retention analysis once the funnel's endpoint (activation) is well defined — retention only means something once you know what "active" or "core action" means for this product; measuring retention against a vague or overly broad definition of activity produces numbers nobody trusts.
  • Don't instrument a full product-analytics stack (funnels, cohorts, retention dashboards) before there's a specific decision the numbers would change — e.g. "should we redesign onboarding step 3" or "is channel A producing better long-term users than channel B." Analytics built without a driving question tends to accumulate dashboards nobody looks at.
  • Segmenting a retention curve by acquisition cohort (signup week, referral source, plan tier) is usually more actionable than a single blended retention number — a blended curve can hide a channel that's actively bad even while the overall average looks acceptable.

Common pitfall

Defining retention as "any event logged" rather than "completed the core action," which produces a retention curve that looks healthier than the product actually is. A user who opens the app, sees nothing interesting, and leaves within thirty seconds still counts as "retained" under an any-activity definition — so a product with a real engagement problem can show a flat, reassuring-looking retention chart right up until a metric tied to actual revenue or usage (bookings, messages sent, purchases) reveals the gap. Agreeing on a specific, instrumented "core action" before building the retention dashboard avoids this — and that decision belongs to product and analytics together, not to whichever event happened to be easiest to query.

Engineering Lens

The event schema underneath all three analysis types is the actual constraint on what's answerable — a funnel step, a cohort's starting point, and a retention definition are each just a query over consistently named events with the right properties (user ID, timestamp, and enough context to segment by). Building product-analytics dashboards on top of an inconsistent or sparse tracking plan (see EventTracking) produces funnel and retention numbers that look precise but are quietly wrong — garbage in, confident-looking dashboard out. The engineering discipline here is less about the analysis math (funnels and cohorts are conceptually simple) and more about making sure the event data underneath is trustworthy before anyone builds a decision on top of it.

Sources

Hermes Wiki