Retention: what winning apps actually measure

Duolingo scores return every day

What does Duolingo know about tomorrow?

Inside Duolingo's code, one folder is called churn. Every day, for every account, it recalculates latest_tomorrow_prob: the probability that you will not come back tomorrow. The number is individual and refreshed daily, where most teams look at a monthly average.

The score is only as good as its inputs. Duolingo records:

• The surface that brought you back: widget, launcher, shortcut.
• Lessons completed.
• Crashes.
• session_end and app_open events.
• The Wi-Fi network you connect to.

Each signal answers its own question. Did you return on your own, or did a notification push you? Did you make progress, or only open the app?

The answers shape the messages. A friend who just passed you, a leaderboard that moved, a streak freeze to reclaim: the tone stays the same while the reason to return changes.

A smaller app can build a rough version of the same score. Writing the formula forces you to name the signals that matter in your product.

Games adjust the board

How does a game retain without new content?

• Block Blast. An XGBoost model runs locally to estimate a player's ad value from phone model, country, past ad views and settings such as vibration turned off. That score drives difficulty and reward pricing. The rule MultiClearBoostThenkillTrait eases the board just before triggering death, so the run feels like the best game of your life. The revive reward adds 10 percent to the score, and teams test the exact minute the offer appears.
• Candy Crush. After several failed attempts on a level, the probability of drawing the color you need goes up. The teardown counts 231 configurable feature records, different rule sets per account, and collection that reaches screen brightness and system font size.
• Crossy Road. The first three gifts are worth 100 to 120 coins, enough for a full spin. Then the average drops to around 31 coins, the wait between spins grows from 3 minutes to 180 minutes as you own more characters, and the chance of landing a missing character falls from 70 percent to 10 percent, unless you watch a second ad, which can double the odds.

These systems retain, and they also read like a manual for compulsion. Before copying one, ask what the user keeps when they close the app.

Retention or compulsion?

Would the user come back if the pressure stopped?

Each game above manufactures a loss, then sells the way to undo it. Block Blast lets you clear lines right before it ends the run, then offers the revive. Crossy Road shrinks the gift as your collection grows, then sells the ad that doubles the odds.

Three tests separate retention from pressure:

• The Monday test. Cut notifications for a week and count who comes back: that is your real retention.
• The value test. Can a user explain in one sentence what they accomplished in the app this week? If the answer is "I kept a streak," the streak has become the product.
• The hidden-price test. Does the mechanic still work once the user knows how it works? If it collapses when they understand it, you are selling ignorance rather than a service.

A streak that stands for a skill survives the close, and so does a social graph that exists outside the app. A gauge that only moves when you pay leaves nothing behind.

What to instrument

Which numbers tell you the first week worked?

Signal	What it tells you
First-session completion	How many installs reach the event that defines value
D1, D7 and D30 by source	Whether one channel is buying users who do not stay
Spontaneous return	Sessions that do not follow a notification, divided by active users
Open rate by message type	Whether a message beats another message or does nothing
Time to second use	The best early predictor of whether the first session worked

Ayo Omolewa's iteration method adds OpenTelemetry traces on the critical path and session recordings on a sample. An AI agent wired to that data can find the screen where people drop off in hours, while a dashboard takes weeks.

The seven-day checklist

• Name one number: activation, D7 retention or upgrade.
• Instrument the first session screen by screen, including drop-offs.
• Separate spontaneous return from pushed return.
• Cut notifications for 72 hours on a sample and measure the difference.
• List the mechanics that manufacture a loss, and decide consciously whether to keep them.
• Run one 50/50 experiment on the most suspicious variable, keep the winner, repeat.

Sources

• Ayo Omolewa, Duolingo teardown: the churn folder, daily return probability, entry signals and personalized notifications — instagram.com/reel/DeUlQTQxv5O
• Ayo Omolewa, Block Blast teardown: local XGBoost model, adjusted difficulty, revive and experiments — instagram.com/reel/DecfvksvHiW
• Charles Liu, Candy Crush teardown: color probabilities, 231 configs, segmentation — instagram.com/reel/DerlD7DxnCW
• Charles Liu, Crossy Road teardown: shrinking gifts, wait times, reroll odds — instagram.com/reel/DeqQs8ztMKE
• Ayo Omolewa, iteration method: one number, OpenTelemetry traces, session recordings, 50/50 test — instagram.com/reel/DeNKjDeyYIl