A seat in last autumn’s Studio Bench presented a handsome drop in week-two retention. The product team had shipped a calmer onboarding. The curve agreed, until someone asked who was missing. iOS 18.1 had landed in the same window. A share sheet they treated as a load-bearing step was gone for a slice of the cohort. The onboarding was not innocent, but it was not the whole plot.
Store updates do three unkind things to App Analytics. They change the surface without your release notes. They concentrate installs among people willing to update, which is not a random sample. They create a week in which old builds and new builds share a calendar and not a product.
Write the version into the inclusion rule
Our marking rubric for cohorts requires a sentence that a sceptical reader can falsify: who is in, which app versions, which platforms, and what you do with people who never received the screen. “All new users in March” is not a rule. It is a mood.
If the store can remove a step without asking you, the funnel is a collaboration you did not consent to.
Forced upgrades are a special case. They look like a gift to instrumentation — everyone is on a known build — and they wreck comparability with the weeks before. We teach seats to freeze a baseline cohort on the last voluntary build and to present the forced week as a break, not a continuation. Boards dislike breaks. They dislike silent incomparability more, once they understand it.
Review freezes belong in the same family. If you cannot ship a fix for a leaking event, the dictionary should say the event is untrusted until a dated resume. Continuing to chart it is a form of theatre we do not mark kindly.