Hybrid-PLG is the reality
Elena VernaPure PLG is rare at scale — winners layer PLG + sales-led + marketing-led motions; sequence them, don't pick one. Build growth loops (output re-feeds input — Reforge) over one-shot funnels.
Growth, PLG & Experimentation
When product-led growth applies, why it's hybrid at scale, and the pre-flight gates that make an A/B result worth believing.
Experiment pre-flight · clear every gate before you infer
OEC + guardrails set — before launch
One Overall Evaluation Criterion + guardrails that must not regress. Choose metrics first, never after peeking.
SRM check — split matches design?
Twyman's Law: a deviating split or a too-good result is probably a bug. Debug, don't interpret.
Validity gate — balance, no confounds, decision rule locked
A/A or pre-period balance · no overlapping confounds · pre-commit win→X / flat→Y / loss→Z before unblinding.
Invariant check — the metric that should NOT move
If it shifts, you have an instrumentation / randomization bug, not a real effect.
Only now — infer
Segment on static attributes only, and weigh practical against statistical significance.
Kohavi / Tang / Xu · Trustworthy Online Controlled Experiments
Above the fold
Know when PLG applies — and that it's hybrid.
Use it when the product is the acquisition / activation / expansion engine. Pure PLG is rare at scale; winners layer PLG + sales-led + marketing-led and sequence them.
Score accounts, not lone users.
B2B buys at the account level — score Product-Qualified Accounts (usage per logo), not lone-user PQLs, or sales chases noise. And never leave a fake door live.
Check SRM before you read a result.
A split that deviates from design is a broken test — debug, don't interpret. Any too-good result is probably an instrumentation bug (Twyman's Law).
Product-led growth
PLG needs low time-to-value, self-serve onboarding, freemium / trial, and a collaborative or viral use case. Not for high-touch, low-frequency, or compliance-gated buying.
The metrics that matter — NRR is the PLG North Star
Pure PLG is rare at scale — winners layer PLG + sales-led + marketing-led motions; sequence them, don't pick one. Build growth loops (output re-feeds input — Reforge) over one-shot funnels.
B2B buys at the account level — score Product-Qualified Accounts (usage aggregated per logo), not lone-user PQLs. Ring-fence product-led sales: reps only chase PQAs past a usage threshold, and don't cannibalize accounts that would self-convert.
Progressive disclosure, in-app education, usage-based upgrade triggers, and collaborative features that drive invites.
Gauge demand before building — ship the entry point, count intent, then show “coming soon.” Cheap signal; never leave it live (trust cost).
Benchmarks
Directional and point-in-time — cite them to calibrate expectations, then verify the current figure before you lean on it.
freemium free → paid conversion → a small top-of-funnel slice pays.
NRR >100% is healthy; ~120%+ is best-in-class expansion.
of shipped features are rarely used → optimize adoption over features-shipped.
Experimentation rigor · Kohavi / Tang / Xu (MS EXP)
The hero's pre-flight in full. Choose metrics first, gate before you infer, and know when a bet shouldn't be an A/B test at all.
One Overall Evaluation Criterion (a short-term proxy that predicts long-term value) plus guardrail metrics that must not regress (latency, crashes, unsubs). Choose metrics first — never after peeking.
Before reading any result, check Sample Ratio Mismatch — a split that deviates from design (e.g. 50/50) means a broken test; debug, don't interpret. Any too-good / surprising result is probably an instrumentation bug.
Week-1 lift can be curiosity (fades) or learning (grows) — date-segment and trust the steady state, not the launch spike.
A/A or pre-period balance check + no overlapping confounds (outage, holiday, price change, PR spike, campaign) + pre-commit the decision rule before unblinding (win→X / flat→Y / loss→Z) so you don't rationalize post-hoc.
Segment only on static pre-treatment attributes (country, device, signup cohort); slicing on post-treatment behavior re-introduces selection bias and invents fake wins.
Watch a metric the change should NOT move (an upstream count, a control-surface event) — if it shifts, you have an instrumentation / randomization bug, not a real effect.
A statistically significant tiny lift may not be worth shipping once you price the opportunity cost (eng time, complexity, surface risk) — judge the effect size against the next-best use of the slot, not just the p-value.
Apply a multiple-comparison correction (Bonferroni / Benjamini-Hochberg) across many metrics, variants, or segments; in networked or marketplace products account for interference / SUTVA via cluster or switchback randomization.
In fintech / healthcare experiments, compliance guardrails (consent, adverse-action, clinical-safety) are not optional guardrail metrics — they gate the test.
Skip low-traffic, strategic / irreversible, or un-instrumentable bets — use judgment plus qualitative. State a hypothesis (the causal why), not just a prediction (the what). STEDI names the org's experimentation maturity — a program capability, not a per-test step.
Calibration trap · significance isn't a green light
A too-good result is probably a bug, and a statistically significant tiny lift may still lose to its opportunity cost. And on the PLG side: never leave a painted / fake door live — the trust cost outlasts the signal you gathered.
Try it
The growth-loop amplifier, live: a viral loop's gain K and its 1 ÷ (1 − K) steady state, plus the engagement loop's sustained active base. A loop that leaks is a funnel wearing a costume.
The growth-loop amplifier needs JavaScript to run. Run it on the Tools page.