Retention: did they come back

Five cats and one calendar

Retention answers a question neither DAU nor the funnel can ask: did the same people come back.

To ask it you need a cohort — a group of users bound together by one starting event. Most often that is the day of the first visit: "everyone who arrived for the first time on March 1". A cohort is fixed forever: a person joins it once and stays there for the rest of the observation.

Here are five cats from the black box, their first-visit dates and every later visit:

The calendar dates show nothing: different numbers, different weeks. Meaning appears when we move from the absolute calendar to the user's age inside the product.

Calendar → relative days

D0 is the day of the first visit. Not "day zero of the month" but "day zero of THIS user". Barsik's D0 is March 1, Musya's D0 is March 5. Everything else is counted from each cat's own D0:

The same "D1" sits on different calendar dates for different cats: March 2 for Barsik, March 6 for Musya. And the other way round: March 8 is D7 for Barsik and D3 for Musya. That translation is the central trick of retention: we align everyone on their own start, after which people who arrived on different days can be summed into one table.

Why bother instead of simply asking "how many yesterday versus today"? Because total DAU mixes old and new users. A product can lose every new user within a week and still grow its DAU — if ads pour new ones in faster than the old ones leak out. A cohort isolates the question: did these particular cats come back.

The denominator is the original cohort

Now let us compute retention for our five cats.

The March 1 cohort — Barsik and Snezhok, size 2. On D1 only Barsik returned → D1 retention = 1 / 2 = 50%. On D7 — Barsik again → D7 = 50%.

The March 5 cohort — Musya and Ryzhik, size 2. On D1 Musya returned → 50%. On D7 Ryzhik returned → 50%. Note: those are different cats. Retention does not require the same person to return at every mark — at each mark we take a share of the same cohort.

At realistic volumes it looks like this:

Cohort 'arrived March 1': 1000 cats ← THE DENOMINATOR for every mark
  active March 2 (D1):    380  → D1 = 380 / 1000 = 38%
  active March 8 (D7):    190  → D7 = 190 / 1000 = 19%
  active March 31 (D30):   90  → D30 = 90 / 1000 = 9%

The rule that breaks half of all homemade calculations:

The denominator is always the original cohort — those very 1000 people, not however many are left.

It is tempting to compute D7 as "190 out of the 380 who returned on D1" — that gives 50%, a pretty number, but it is a different metric: the survival share from D1 to D7. It is useful too and it means something ("half of those who came back on day two lasted a week"), but it must not be called D7 retention: comparing it across cohorts is meaningless, because each cohort would have its own denominator.

One more consequence of a fixed denominator: retention marks need not decrease monotonically. A person may skip D1 and show up on D7 — like Ryzhik. So D7 is sometimes slightly above D3, especially when day seven lands on a weekend or a mailing. A funnel always decreases; retention does not.

Two decisions to make before computing

Decision 1: what "came back" means

Exactly as with "active" in the DAU lesson, the word "returned" defines nothing on its own. On the same March 1 cohort:

The number swings sevenfold, and all three variants are legitimate. For a game "opened the app" is apt; a marketplace often takes "placed an order", because an open without a purchase pays no bills. Only one thing matters: the definition is chosen deliberately, written down, and never changed silently.

Decision 2: "exactly on day N" or "on day N and later"

These are two different kinds of retention, and they give different numbers.

Exact-day (classic) retention. D7 is the share of the cohort active precisely on day seven. A strict, sensitive metric: someone who visited on D6 and D8 but skipped D7 does not count in D7 retention. It is convenient for comparing cohorts and standard in mobile analytics. Throughout this course DN means exactly this kind.

Rolling (unbounded) retention. D7 is the share of the cohort with activity on day seven OR later. That definition answers "is this person still alive?" rather than "did they show up on a given day". The numbers are always higher than exact-day and much smoother, but late marks become known with a delay.

March 1 cohort, a cat visited on March 2 and March 9
  exact-day D7 (March 8): NOT counted — he was absent that day
  rolling  D7:            counted — March 9 is later than day seven

Neither kind is "more correct". What is correct is knowing which one you are looking at and never comparing one with the other. If D7 differs twofold between two reports, check the kind of retention before you check the product.

Immature cohorts: an empty cell is not a zero

Today is March 14. Look at two cohorts.

The February 1 cohort. Its D30 falls on March 3 — long past, the data is collected, the number is known and final.

The March 10 cohort. Its D1 (March 11) is known. Its D7 falls on March 17 — that day has not happened yet. Its D30 falls on April 9.

           D1     D7     D30
Feb 1      41%    22%    11%
Mar 10     39%     ?      ?     ← not lived through yet

The question mark is not a zero. Zero would mean "nobody came back" — a catastrophe. What it actually means is "we do not know yet": the cohort is immature, it simply has not had enough calendar time.

In tables and code such an "unknown" is usually marked by a special value — in pandas it is NaN (not a number), which we will study in the messy-data chapter. For now the key point is enough:

NaN ≠ 0. An unknown value must not be summed or averaged with known ones, nor drawn on a chart as a dip.

Why this is dangerous in practice. Average D7 across all cohorts with zeros substituted for the immature ones, and the average collapses — producing a "retention drop" that does not exist. Plot a D30 line by day and it will honestly slide down towards the recent dates purely because there is nothing to show there yet.

The investigator's rule: when comparing cohorts, take only those that have lived the required age. Comparing D30 means considering cohorts at least 30 days old and leaving the rest blank. The one correct way to get an earlier read is to look at early marks (D1, D3, D7) that have already matured for everyone.