Chapter 1 — Anatomy of Metrics

Retention: did they come back

18 min
What you'll learn
  • what a is, and how calendar dates turn into relative days D0, D1, D7
  • compute : the denominator is always the original cohort
  • pin down what "came back" means — the number depends on it
  • tell exact-day retention from rolling retention
  • understand immature cohorts: an empty cell is not a zero

Five cats and one calendar

Retention answers a question neither DAU nor the can ask: did the same people come back.

To ask it you need a — a group of users bound together by one starting event. Most often that is the day of the first visit: "everyone who arrived for the first time on March 1". A cohort is fixed forever: a person joins it once and stays there for the rest of the observation.

Here are five cats from the black box, their first-visit dates and every later visit:

catfirst visitalso visited
BarsikMarch 1March 2, March 8
SnezhokMarch 1
PukhMarch 2March 3, March 9
MusyaMarch 5March 6
RyzhikMarch 5March 12

The calendar dates show nothing: different numbers, different weeks. Meaning appears when we move from the absolute calendar to the user's age inside the product.

A cadet and robot-cat observe a hologram in which groups of capsules follow matching timelines and some capsules return at checkpoints.
Retention is measured against the size of the cohort, with calendar dates turned into days of life: D0, D1, D7.

Calendar → relative days

D0 is the day of the first visit. Not "day zero of the month" but "day zero of THIS user". Barsik's D0 is March 1, Musya's D0 is March 5. Everything else is counted from each cat's own D0:

catD0Mar 2Mar 3Mar 6Mar 8Mar 9Mar 12
BarsikMar 1D1D7
SnezhokMar 1
PukhMar 2D1D7
MusyaMar 5D1
RyzhikMar 5D7

The same "D1" sits on different calendar dates for different cats: March 2 for Barsik, March 6 for Musya. And the other way round: March 8 is D7 for Barsik and D3 for Musya. That translation is the central trick of : we align everyone on their own start, after which people who arrived on different days can be summed into one table.

Why bother instead of simply asking "how many yesterday versus today"? Because total DAU mixes old and new users. A product can lose every new user within a week and still grow its DAU — if ads pour new ones in faster than the old ones leak out. A isolates the question: did these particular cats come back.

CalendarAsignup Mar 1Bsignup Mar 5Csignup Mar 512345678MarchCohort ageAMar 1Mar 2Mar 8BMar 5Mar 6Mar 12CMar 5Mar 6Mar 12D0D1D7retention is measured in cohort age, not on the calendar
Each user is aligned on their own D0: calendar dates become age, and Barsik's D1 lines up with Musya's D1.

The denominator is the original cohort

Now let us compute for our five cats.

The March 1 — Barsik and Snezhok, size 2. On D1 only Barsik returned → D1 retention = 1 / 2 = 50%. On D7 — Barsik again → D7 = 50%.

The March 5 cohort — Musya and Ryzhik, size 2. On D1 Musya returned → 50%. On D7 Ryzhik returned → 50%. Note: those are different cats. Retention does not require the same person to return at every mark — at each mark we take a share of the same cohort.

At realistic volumes it looks like this:

Cohort 'arrived March 1': 1000 cats ← THE DENOMINATOR for every mark
  active March 2 (D1):    380  → D1 = 380 / 1000 = 38%
  active March 8 (D7):    190  → D7 = 190 / 1000 = 19%
  active March 31 (D30):   90  → D30 = 90 / 1000 = 9%

The rule that breaks half of all homemade calculations:

The denominator is always the original cohort — those very 1000 people, not however many are left.

It is tempting to compute D7 as "190 out of the 380 who returned on D1" — that gives 50%, a pretty number, but it is a different metric: the survival share from D1 to D7. It is useful too and it means something ("half of those who came back on day two lasted a week"), but it must not be called D7 retention: comparing it across cohorts is meaningless, because each cohort would have its own denominator.

One more consequence of a fixed denominator: retention marks need not decrease monotonically. A person may skip D1 and show up on D7 — like Ryzhik. So D7 is sometimes slightly above D3, especially when day seven lands on a weekend or a mailing. A always decreases; retention does not.

Two decisions to make before computing

Decision 1: what "came back" means

Exactly as with "active" in the DAU lesson, the word "returned" defines nothing on its own. On the same March 1 :

definition of "returned"returned on D1D1
any event in the log45245.2%
opened the app (app_open)38038.0%
placed an order (purchase)616.1%

The number swings sevenfold, and all three variants are legitimate. For a game "opened the app" is apt; a marketplace often takes "placed an order", because an open without a purchase pays no bills. Only one thing matters: the definition is chosen deliberately, written down, and never changed silently.

Decision 2: "exactly on day N" or "on day N and later"

These are two different kinds of retention, and they give different numbers.

Exact-day (classic) retention. D7 is the share of the cohort active precisely on day seven. A strict, sensitive metric: someone who visited on D6 and D8 but skipped D7 does not count in D7 retention. It is convenient for comparing cohorts and standard in mobile analytics. Throughout this course DN means exactly this kind.

Rolling (unbounded) retention. D7 is the share of the cohort with activity on day seven OR later. That definition answers "is this person still alive?" rather than "did they show up on a given day". The numbers are always higher than exact-day and much smoother, but late marks become known with a delay.

March 1 cohort, a cat visited on March 2 and March 9
  exact-day D7 (March 8): NOT counted — he was absent that day
  rolling  D7:            counted — March 9 is later than day seven

Neither kind is "more correct". What is correct is knowing which one you are looking at and never comparing one with the other. If D7 differs twofold between two reports, check the kind of retention before you check the product.

Immature cohorts: an empty cell is not a zero

Today is March 14. Look at two .

The February 1 cohort. Its D30 falls on March 3 — long past, the data is collected, the number is known and final.

The March 10 cohort. Its D1 (March 11) is known. Its D7 falls on March 17 — that day has not happened yet. Its D30 falls on April 9.

           D1     D7     D30
Feb 1      41%    22%    11%
Mar 10     39%     ?      ?     ← not lived through yet

The question mark is not a zero. Zero would mean "nobody came back" — a catastrophe. What it actually means is "we do not know yet": the cohort is immature, it simply has not had enough calendar time.

In tables and code such an "unknown" is usually marked by a special value — in pandas it is NaN (not a number), which we will study in the messy-data chapter. For now the key point is enough:

NaN ≠ 0. An unknown value must not be summed or averaged with known ones, nor drawn on a chart as a dip.

Why this is dangerous in practice. Average D7 across all cohorts with zeros substituted for the immature ones, and the average collapses — producing a " drop" that does not exist. Plot a D30 line by day and it will honestly slide down towards the recent dates purely because there is nothing to show there yet.

The investigator's rule: when comparing cohorts, take only those that have lived the required age. Comparing D30 means considering cohorts at least 30 days old and leaving the rest blank. The one correct way to get an earlier read is to look at early marks (D1, D3, D7) that have already matured for everyone.

The March 1 cohort on a calculator. Note that all three formulas share one denominator — 1000. The last line computes a different metric — survival from D1 to D7. Above, we established that it cannot be calculated from the final figures: we do not know how many cats were active on both D1 and D7. Therefore, here we make this assumption explicit — we assume that all 190 cats active on D7 were also active on D1.
python · pandas
Practice: write the code
The March 3 cohort: 800 new cats. Active exactly on March 4 (D1) — 296; exactly on March 10 (D7) — 120. Compute d1 and d7 as percentages. Watch the denominator: it is the same for both marks.
python · pandas
Practice: write the code
The same March 3 cohort (800 cats, 296 on D1, 120 on D7; as in the calculator, assume that all 120 active on D7 were also active on D1). Compute two things easily confused with retention: survival_d1_to_d7 — the share of D1 returners still there on D7 — and churned_by_d7 — how many cats from the ORIGINAL cohort were not active on D7.
python · pandas
Check yourself
The March 1 cohort has 1000 people. 380 returned on D1, 190 are active on D7. What is D7 retention?
Check yourself
Today is March 14. The D30 cell for the March 10 cohort is empty. A colleague suggests putting a 0 there "so the average computes". What is wrong with that?
Key takeaways
  • a is a group sharing a first-visit day; a person joins once and forever
  • D0 is THIS user's first-visit day; calendar dates become age, and different cats align on their own start
  • the denominator is always the original cohort; 190 / 380 without data on the overlap between active D1 and D7 users is simply the ratio of two , not D7 retention
  • "came back" must be defined: any event, an app open, or an order — the gap is several-fold
  • exact-day DN ("precisely on day N") and rolling DN ("on day N or later") are different metrics and must not be compared
  • immature cohorts stay blank: NaN ≠ 0

Next — the cohort table: how all these rows stack into one grid and what to read in it first.