Skip to content
product-skill
product-skill/ Field Guide/ Growth, PLG & experimentation Growth

Growth, PLG & Experimentation

Growth is compounding loops and trustworthy experiments — and most of the discipline is refusing to fool yourself before you read the result.

When product-led growth applies, why it's hybrid at scale, and the pre-flight gates that make an A/B result worth believing.

Experiment pre-flight · clear every gate before you infer

OEC + guardrails set — before launch

One Overall Evaluation Criterion + guardrails that must not regress. Choose metrics first, never after peeking.

SRM check — split matches design?

Twyman's Law: a deviating split or a too-good result is probably a bug. Debug, don't interpret.

Validity gate — balance, no confounds, decision rule locked

A/A or pre-period balance · no overlapping confounds · pre-commit win→X / flat→Y / loss→Z before unblinding.

Invariant check — the metric that should NOT move

If it shifts, you have an instrumentation / randomization bug, not a real effect.

Only now — infer

Segment on static attributes only, and weigh practical against statistical significance.

Kohavi / Tang / Xu · Trustworthy Online Controlled Experiments

Above the fold

The three growth moves that matter most.

01

Know when PLG applies — and that it's hybrid.

Use it when the product is the acquisition / activation / expansion engine. Pure PLG is rare at scale; winners layer PLG + sales-led + marketing-led and sequence them.

02

Score accounts, not lone users.

B2B buys at the account level — score Product-Qualified Accounts (usage per logo), not lone-user PQLs, or sales chases noise. And never leave a fake door live.

03

Check SRM before you read a result.

A split that deviates from design is a broken test — debug, don't interpret. Any too-good result is probably an instrumentation bug (Twyman's Law).

Product-led growth

Use it when the product is the engine.

PLG needs low time-to-value, self-serve onboarding, freemium / trial, and a collaborative or viral use case. Not for high-touch, low-frequency, or compliance-gated buying.

The metrics that matter — NRR is the PLG North Star

  • PQL
  • Time-to-value (TTV)
  • Activation rate
  • Free → paid conversion
  • NRR · PLG North Star
  • Expansion revenue

Hybrid-PLG is the reality

Elena Verna

Pure PLG is rare at scale — winners layer PLG + sales-led + marketing-led motions; sequence them, don't pick one. Build growth loops (output re-feeds input — Reforge) over one-shot funnels.

PQA > PQL

Verna

B2B buys at the account level — score Product-Qualified Accounts (usage aggregated per logo), not lone-user PQLs. Ring-fence product-led sales: reps only chase PQAs past a usage threshold, and don't cannibalize accounts that would self-convert.

PLG design principles

design

Progressive disclosure, in-app education, usage-based upgrade triggers, and collaborative features that drive invites.

Painted-door test

fake-door

Gauge demand before building — ship the entry point, count intent, then show “coming soon.” Cheap signal; never leave it live (trust cost).

Benchmarks

Three numbers that anchor PLG realism.

Directional and point-in-time — cite them to calibrate expectations, then verify the current figure before you lean on it.

~2–5%

freemium free → paid conversion → a small top-of-funnel slice pays.

Freemium Directional · verify current
>100% / ~120%+

NRR >100% is healthy; ~120%+ is best-in-class expansion.

Net revenue retention Directional · verify current
~80%

of shipped features are rarely used → optimize adoption over features-shipped.

Pendo Directional · verify current

Experimentation rigor · Kohavi / Tang / Xu (MS EXP)

Ten checks that make a result trustworthy.

The hero's pre-flight in full. Choose metrics first, gate before you infer, and know when a bet shouldn't be an A/B test at all.

OEC + guardrails

before launch

One Overall Evaluation Criterion (a short-term proxy that predicts long-term value) plus guardrail metrics that must not regress (latency, crashes, unsubs). Choose metrics first — never after peeking.

SRM-first + Twyman's Law

triage

Before reading any result, check Sample Ratio Mismatch — a split that deviates from design (e.g. 50/50) means a broken test; debug, don't interpret. Any too-good / surprising result is probably an instrumentation bug.

Novelty vs primacy

steady state

Week-1 lift can be curiosity (fades) or learning (grows) — date-segment and trust the steady state, not the launch spike.

Validity gate before inference

pre-commit

A/A or pre-period balance check + no overlapping confounds (outage, holiday, price change, PR spike, campaign) + pre-commit the decision rule before unblinding (win→X / flat→Y / loss→Z) so you don't rationalize post-hoc.

Static segments only

no fake wins

Segment only on static pre-treatment attributes (country, device, signup cohort); slicing on post-treatment behavior re-introduces selection bias and invents fake wins.

Invariant-metric check

should not move

Watch a metric the change should NOT move (an upstream count, a control-surface event) — if it shifts, you have an instrumentation / randomization bug, not a real effect.

Practical vs statistical

effect size

A statistically significant tiny lift may not be worth shipping once you price the opportunity cost (eng time, complexity, surface risk) — judge the effect size against the next-best use of the slot, not just the p-value.

Multiple comparisons + SUTVA

networked

Apply a multiple-comparison correction (Bonferroni / Benjamini-Hochberg) across many metrics, variants, or segments; in networked or marketplace products account for interference / SUTVA via cluster or switchback randomization.

Regulatory guardrails gate the test

mandatory

In fintech / healthcare experiments, compliance guardrails (consent, adverse-action, clinical-safety) are not optional guardrail metrics — they gate the test.

Don't test everything

Tal Raviv

Skip low-traffic, strategic / irreversible, or un-instrumentable bets — use judgment plus qualitative. State a hypothesis (the causal why), not just a prediction (the what). STEDI names the org's experimentation maturity — a program capability, not a per-test step.

Calibration trap · significance isn't a green light

A too-good result is probably a bug, and a statistically significant tiny lift may still lose to its opportunity cost. And on the PLG side: never leave a painted / fake door live — the trust cost outlasts the signal you gathered.

Try it

Model your own loop.

The growth-loop amplifier, live: a viral loop's gain K and its 1 ÷ (1 − K) steady state, plus the engagement loop's sustained active base. A loop that leaks is a funnel wearing a costume.

The growth-loop amplifier needs JavaScript to run. Run it on the Tools page.

All 13 tools →