Shipping the Winning Variant and Cleaning Up Feature Flags After You PostHog Run Experiments
This skill teaches you how to graduate a winning A/B test variant from experiment to production default, remove the losing variant's code paths, and archive the associated feature flag so your codebase and PostHog workspace stay clean.
After your PostHog experiment reaches statistical significance, stop the experiment in the Experiments tab, then update your feature flag to serve the winning variant to 100% of users. Deploy a code change that hardcodes the winning experience and removes all branching logic tied to the flag. Finally, archive the feature flag in PostHog so it no longer clutters your active flag list. This keeps your codebase lean and prevents stale flags from causing unexpected behavior in future experiments.
Outcome: Every user receives the winning experience as the hardcoded default, all experiment branching logic is removed from the codebase, and the feature flag is archived in PostHog, leaving zero technical debt from the experiment.
Prerequisites
- A completed PostHog experiment with a statistically significant winning variant
- Familiarity with PostHog feature flags and the Experiments tab
- Access to the codebase where feature flag evaluation logic lives
- Understanding of your deployment pipeline (CI/CD, staging, production)
Overview
Running an experiment is only half the job. The other half, the part that most teams neglect, is shipping the winner cleanly. When you posthog run experiments through the PostHog Experiments Onboarding A/B Test Method, every experiment creates a feature flag, branching code in your application, and a mental model in your team's head about which variant is "live." If you stop the experiment but leave the flag and branching logic in place, you accumulate what practitioners call "flag debt." Flag debt slows down future experiments because developers have to reason about stale conditional paths. It also introduces real production risk: a misconfigured flag rollout can accidentally re-enable a losing variant months after the experiment ended.
This skill covers the full post-experiment lifecycle. You will learn how to transition a feature flag from experiment mode to a 100%-rollout state, verify that the winning experience is reaching all users, remove the losing variant's code from your application, and archive the flag in PostHog. The concrete artifact you produce is a clean deployment where the winning variant is the new baseline, the codebase contains no references to the old experiment, and the PostHog workspace shows the flag as archived with a clear audit trail linking it to the original experiment and its results.
The skill matters most for teams running multiple onboarding experiments in sequence. If your first experiment's flag is still live when you launch your second experiment, you risk interaction effects between flags and confusing analytics. Cleaning up promptly is what separates teams that run one experiment a quarter from teams that run one a week. The discipline of shipping and cleaning is what makes the entire experimentation loop in the PostHog Experiments Onboarding A/B Test Method sustainable over time.
How It Works
The mental model behind this skill is a three-phase state transition: Experiment Active, Winner Rolling Out, and Cleanup Complete. Understanding why each phase exists, and why you cannot skip any of them, prevents the most common post-experiment failures.
During the Experiment Active phase, your feature flag is managed by PostHog's experiment allocation engine. Users are bucketed into variants based on a hash of their distinct ID, and the flag's rollout percentage and variant weights are controlled by the experiment. When you stop the experiment, PostHog freezes the statistical results but does not automatically change the flag's behavior. This is by design. PostHog separates the experiment lifecycle from the flag lifecycle because teams need time to review results, discuss with stakeholders, and plan the rollout. The flag continues serving whatever distribution it had when the experiment was running. If your experiment was 50/50, half your users are still seeing the control after you click "stop."
The Winner Rolling Out phase is where you manually update the feature flag to serve the winning variant to 100% of traffic. You do this inside PostHog's Feature Flags interface by setting the rollout percentage to 100% and the variant payload to the winner. This is a deliberate, reviewable action. You are not merging code yet. You are telling PostHog, "From now on, every user who evaluates this flag should get variant B." This phase exists as a safety net. You can monitor dashboards and error rates to confirm the winning variant behaves correctly at 100% traffic before you commit to a code change. If something goes wrong, you can roll the flag back to the control variant in seconds, which is much faster than reverting a deployment.
The Cleanup Complete phase is the code-level change. Once you are confident the winning variant works at full traffic, you deploy a code change that removes the feature flag evaluation entirely. The winning variant's code becomes the only code path. The losing variant's code is deleted. The conditional logic that checked the flag is deleted. After this deployment, the feature flag in PostHog is no longer being evaluated by any client. You then archive the flag in PostHog, which removes it from the active flags list but preserves the audit trail. Archiving does not delete the flag's history, so you can always look back at what the experiment tested and when the flag was retired.
The reason you cannot skip the rollout phase and jump straight to code cleanup is risk management. If you deploy the hardcoded winner and discover a problem, your only option is a code revert, which takes minutes to hours depending on your pipeline. The flag rollout gives you a sub-second rollback mechanism. The reason you cannot skip code cleanup after the rollout is sustainability. Every unarchived flag is a branching point that developers must understand, QA must test, and future experiments must account for. Teams that skip cleanup routinely find themselves debugging "ghost" experiments six months later.
Step-by-Step
Step 1: Confirm the experiment has reached a valid conclusion
Open the Experiments tab in PostHog and navigate to your completed experiment. 05). Check all of your success metrics, not just the primary one. If a variant wins on the primary metric but loses badly on a guardrail metric like error rate or page load time, it may not be safe to ship.
Record the winning variant name, the lift percentage, and the confidence level in a brief summary. This summary becomes your audit trail for why you chose to ship this variant.
Tip: If the experiment is inconclusive, do not default to shipping the control. Decide whether to extend the experiment, redesign the variant, or accept the null result. Shipping the control is a valid decision, but it should be deliberate and documented, not an accident of inaction.
Step 2: Stop the experiment in PostHog
Click the "Stop" button on your experiment in the PostHog Experiments tab. This freezes the experiment results and stops new users from being allocated into experiment cohorts. Note that stopping the experiment does not change the feature flag's behavior. Users who were already bucketed into variants will continue seeing their assigned variant because the feature flag is still active and still distributing traffic according to the experiment's allocation logic.
Stopping the experiment simply tells PostHog to stop collecting experiment-specific data and to preserve the final results.
Tip: After stopping, wait at least one full analytics cycle (typically 24 hours) before proceeding. This ensures any delayed events from the final day are captured and your results snapshot is complete.
Step 3: Update the feature flag to serve the winning variant at 100%
Navigate to the Feature Flags section in PostHog. Find the flag associated with your experiment (it will have the same name or a linked reference). Edit the flag's rollout conditions. Set the rollout percentage to 100% and configure the flag to return only the winning variant's value.
If your flag was a boolean (true/false) and the winning variant was the treatment, set it to return true for all users. If your flag used multivariate payloads (like "control" and "test"), set the variant distribution to 100% for the winner and 0% for everything else. Save the flag. At this point, every user evaluating this flag will receive the winning variant.
Tip: If your experiment had more than two variants (A/B/N testing), make sure you zero out all losing variants, not just one. A common mistake is setting the winner to 100% but leaving the distribution controls showing stale percentages for other variants.
Step 4: Monitor production metrics at 100% rollout
With the flag now serving the winning variant to all users, watch your production dashboards for 24-48 hours. Pay attention to the same metrics your experiment tracked: conversion rate, error rate, page load time, and any guardrail metrics you defined. Compare the all-traffic performance to the experiment's treatment group performance. A small discrepancy is normal because experiment cohorts are sometimes not perfectly representative of your full user base.
A large discrepancy, say more than 20% deviation from the experiment's measured lift, suggests something is wrong. Also monitor for technical issues like increased error logs, support tickets, or performance degradation that might not show up in your experiment metrics.
Tip: Set up a PostHog dashboard or a simple alert in your monitoring tool that compares the current day's key metrics to the 7-day average. This gives you an early warning system without requiring you to stare at charts.
Step 5: Deploy the code change that hardcodes the winning variant
Create a pull request or code change that removes all feature flag evaluation logic for this experiment. isFeatureEnabled('your-experiment-flag')`. Replace the conditional branching with the winning variant's code only. Delete the losing variant's code entirely.
Do not comment it out or wrap it in a dead code block. Commented-out code creates confusion for future developers who will not know whether it is intentionally dead or accidentally disabled. Run your full test suite to make sure the removal did not break anything. Deploy to staging, verify the winning experience loads correctly, and then deploy to production.
Tip: Search your entire codebase for the flag's key string (e.g., `your-experiment-flag`) to make sure you catch every reference. Flags are sometimes evaluated in unexpected places like analytics wrappers, server-side middleware, or configuration files.
Step 6: Verify the flag is no longer being evaluated
After your code deployment reaches production, go back to the PostHog Feature Flags interface and check the flag's evaluation activity. PostHog shows recent evaluations for each flag. Over the next few hours, the evaluation count for this flag should drop to zero. If evaluations are still happening, it means some part of your codebase, or an older cached version of your application, is still checking the flag.
Investigate and remove any remaining references. This verification step is critical because archiving a flag that is still being evaluated will cause those evaluation calls to return a default value, which might not match the winning variant and could break the user experience.
Tip: Mobile apps and SPAs with aggressive caching may continue evaluating stale flags for hours or even days after deployment. If you have a mobile app, you may need to wait for a forced app update cycle before evaluations fully stop.
Step 7: Archive the feature flag in PostHog
Once evaluations have dropped to zero, return to the Feature Flags section in PostHog. Find your experiment's flag and archive it. Archiving removes the flag from the default active flags list but preserves its full history, including creation date, variant configuration, rollout changes, and linked experiment results. This is not the same as deleting.
You should almost never delete a flag because deletion erases the audit trail. Archiving keeps the workspace clean while maintaining the historical record. Add a note to the flag's description before archiving, something like "Experiment concluded [date]. Winner: variant B (+12% onboarding completion).
" This note makes it trivial for anyone reviewing flag history to understand what happened.
Tip: Some teams create a quarterly review calendar event to audit for flags that were stopped but never archived. This catch-all prevents flags from slipping through the cracks when engineers get pulled onto other work.
Step 8: Update your experiment log and notify the team
Add a final entry to your team's experiment log, spreadsheet, or documentation system. Record the experiment name, the winning variant, the measured lift on each metric, the date the winner was shipped, the PR or deployment reference for the code change, and the date the flag was archived. Notify stakeholders (product managers, designers, other engineers) that the experiment is fully closed. This communication prevents someone from accidentally referencing a flag that no longer exists or asking about an experiment whose results are now production defaults. If you use a tool like Notion, Confluence, or a shared spreadsheet to track experiments, mark this experiment's status as "Shipped and Cleaned Up" rather than just "Completed."
Tip: Include a screenshot of the final experiment results in your log entry. PostHog's experiment results page can change format over time, and a screenshot preserves the exact numbers you used to make your decision.
Examples
Example: Small startup shipping a winning onboarding tooltip sequence
A 5-person B2B SaaS startup ran a PostHog experiment testing two onboarding tooltip sequences for new users. The experiment ran for 3 weeks with 800 users per variant. Variant B (interactive tooltips with progress indicators) showed a 15% lift in onboarding completion with 97% Bayesian probability. The team has a single Next.js repository and deploys via Vercel.
The product engineer stops the experiment in PostHog's Experiments tab and records the results: variant B, +15% onboarding completion, 97% probability. She navigates to Feature Flags, finds experiment-onboarding-tooltips, and sets the rollout to 100% variant B. Over the next 24 hours she checks the PostHog onboarding completion dashboard and confirms the rate matches the experiment's treatment group performance. She then opens the codebase and searches for experiment-onboarding-tooltips.
She finds three references: the tooltip component that checks the flag, a server-side page prop that evaluates the flag, and an analytics wrapper that tags events with the variant name. She creates a PR that removes all three conditional paths, hardcodes the interactive tooltip component, removes the old static tooltip component file entirely, and updates the analytics wrapper to stop tagging the variant. After the PR passes tests and deploys to Vercel, she waits 4 hours and checks PostHog's flag evaluation activity, which shows zero evaluations. She archives the flag with the note "Winner: interactive tooltips (variant B).
+15% onboarding completion, 97% probability. " She updates the team's Notion experiment log and posts in Slack that the experiment is fully closed.
Example: Growth team at a mid-size B2C app rolling out a new signup flow
A consumer app with 50,000 daily active users ran a multivariate experiment (A/B/C) on their signup flow. Variant C (social proof + simplified form) won with a 22% lift in signup-to-activation rate. The app has a React Native mobile client, a React web client, and a Node.js backend, all of which evaluate the feature flag. The mobile app has a 48-hour cache cycle for updated bundles.
The growth PM stops the experiment and documents the results across all three variants. In PostHog Feature Flags, she sets experiment-signup-flow to 100% variant C, zeroing out both variant A (control) and variant B. The team monitors production metrics for 48 hours, paying special attention to signup error rates and backend validation failures. Metrics look clean.
js backend. Each PR removes the flag evaluation, deletes the control and variant B code paths, and hardcodes variant C. The web and backend PRs deploy the same day. The mobile PR goes through the app store review process.
The team waits 72 hours (longer than usual to account for the mobile cache cycle) and checks PostHog's flag evaluation panel. Web and backend evaluations dropped to zero within hours, but mobile evaluations tapered off over 48 hours as cached bundles expired. Once all evaluations hit zero, the engineer archives the flag with a detailed note linking to all three PRs and the experiment results. The PM updates the experiment tracker in Notion with a "Shipped and Cleaned Up" status.
Example: Enterprise B2B team shipping a winning pricing page variant
A B2B SaaS company with a Webflow marketing site and a React app ran an experiment on their pricing page layout. The experiment used PostHog's JavaScript snippet on the Webflow site to evaluate the flag and swap content blocks. Variant B (comparison table with competitor pricing) showed a 9% lift in pricing-page-to-signup conversion with 96% frequentist confidence. The marketing team manages Webflow while the engineering team manages the React app.
The growth engineer stops the experiment and shares the results with both the marketing and engineering teams. In PostHog, he sets the flag experiment-pricing-layout to 100% variant B. The marketing team monitors pricing page analytics for 48 hours and confirms the conversion rate matches expectations. Since the flag is evaluated by a PostHog JavaScript snippet embedded in Webflow's custom code section, the cleanup requires changes in two places.
The marketing team updates the Webflow pricing page to permanently display the comparison table layout (variant B's design), removing the conditional custom code that swapped layouts based on the flag. The engineering team searches the React app codebase and finds one analytics event that tagged the pricing variant, which they remove. After deployment, the growth engineer checks PostHog and sees zero flag evaluations within 6 hours. He archives the flag with the note "Winner: comparison table layout (variant B).
- " He updates the cross-team experiment log in Confluence and closes the experiment's Jira ticket.
Example: Shipping a control variant win (null result that favors the original)
A product team ran an experiment on their dashboard layout, hypothesizing that a card-based layout would increase feature discovery. After 4 weeks and 3,000 users per variant, the card-based layout showed no statistically significant improvement on any metric and actually showed a small (non-significant) decrease in task completion rate. The team decides to keep the original layout.
The PM stops the experiment and documents the null result. Even though the control won, the cleanup process is identical. The engineer navigates to PostHog Feature Flags and sets experiment-dashboard-cards to 100% control variant. She monitors for 24 hours to confirm nothing changed (expected, since this is the same experience users were already having).
She then creates a PR that removes all flag evaluation calls, deletes the card-based layout component and its associated CSS, and removes the conditional rendering logic. After deployment, she verifies zero flag evaluations in PostHog and archives the flag with the note "Null result. Card layout showed no significant improvement. Control (original list layout) retained.
" She records the null result in the experiment log with the same rigor as a winning result, because null results inform future experiment design and prevent the team from re-testing the same idea without new evidence.
Best Practices
Always separate the flag rollout from the code deployment. Rolling the flag to 100% first gives you a sub-second rollback mechanism if the winning variant behaves differently at full traffic than it did during the experiment. Skipping this step and going straight to a code deployment means your only rollback is a full code revert, which is slower and riskier.
Search your entire codebase for the flag key string, not just the files you remember editing. Feature flag checks often get added to logging wrappers, analytics calls, server-side rendering logic, and configuration files that are easy to forget. A global search for the flag name catches these orphaned references before they cause problems.
Archive flags rather than deleting them. Archived flags in PostHog preserve the full history of variant configurations, rollout changes, and linked experiment results. Deleting a flag erases this audit trail, which makes it impossible to reconstruct what happened when someone asks about an old experiment six months later.
Add a descriptive note to the flag before archiving, including the experiment name, the winning variant, the measured lift, and the PR reference for the code change. This note transforms the archived flag from a cryptic identifier into a self-documenting record of what was tested, what won, and where the code change landed.
Set a hard deadline for flag cleanup after shipping the winner. Many teams use a rule like "flag must be archived within one sprint of the experiment ending." Without a deadline, cleanup gets deprioritized indefinitely, and you accumulate flag debt that makes future experiments harder to reason about.
Monitor for stale flag evaluations after code deployment, especially if you have mobile apps or aggressively cached single-page applications. A flag that is being evaluated after its code is supposedly removed indicates an incomplete cleanup. PostHog's flag evaluation activity panel shows you whether evaluations are still happening.
When removing the losing variant's code, delete it entirely rather than commenting it out. Commented-out code creates ambiguity for future developers who cannot tell whether the code is intentionally dead or accidentally disabled. If you need to reference the losing variant later, the code is preserved in your version control history.
Coordinate flag cleanup across frontend and backend if your experiment spanned both layers. A common failure mode is removing the frontend flag check but leaving the backend still branching on the flag, or vice versa. Both layers must be cleaned up in the same deployment cycle.
Common Mistakes
Stopping the experiment but never updating the feature flag, leaving 50% of users on the control variant indefinitely
Correction
Stopping an experiment in PostHog freezes the results but does not change the feature flag's behavior. Users continue receiving whatever variant they were allocated during the experiment. You must manually navigate to the Feature Flags section and update the flag to serve the winning variant at 100%. The telltale sign of this mistake is that your post-experiment conversion rate is suspiciously close to the blended rate during the experiment rather than matching the winning variant's rate.
Check your flag's rollout configuration immediately after stopping any experiment.
Deploying the hardcoded winner without first rolling the flag to 100% and monitoring
Correction
This skips your fastest rollback mechanism. If the winning variant performs differently at 100% traffic than it did at 50% (due to interaction effects, infrastructure scaling, or edge cases in user segments that were underrepresented in the experiment), your only option is a full code revert. Rolling the flag to 100% first gives you a one-click rollback to the control variant while you investigate. The monitoring period should be at least 24-48 hours and should cover the same metrics your experiment tracked plus any infrastructure health metrics.
Archiving a feature flag while it is still being actively evaluated by production code
Correction
When you archive a flag that is still being called by your application, the flag evaluation returns a default value (typically false or the first variant). If the default does not match the winning variant, users will see the wrong experience. Before archiving, always check the flag's evaluation activity in PostHog. Evaluations should be at zero for at least several hours after your code deployment.
If evaluations persist, search your codebase again for the flag key. The most common culprit is a cached bundle on a CDN that has not been invalidated yet.
Leaving losing variant code in the codebase as commented-out blocks "just in case"
Correction
Commented-out code creates maintenance burden and confusion. Future developers will not know whether the code is deliberately dead or was accidentally commented out during a merge conflict. ), which is the correct place to retrieve old code if you ever need it. Delete the code, delete the conditional branching logic, and let your version control system serve as the archive.
If your team has anxiety about this, add a commit message that explains the experiment and links to the results.
Cleaning up the flag in the frontend but forgetting about server-side or API-level flag checks
Correction
Experiments that span multiple layers of the stack require cleanup in every layer. A frontend-only cleanup leaves the backend still branching on a flag that will eventually be archived, causing undefined behavior. Before starting cleanup, grep the flag key across all repositories, not just the one you are most familiar with. Many teams maintain a simple checklist of repositories to search when cleaning up any flag: frontend, backend API, mobile apps, background workers, and infrastructure configuration.
Not documenting which variant won and why before archiving the flag
Correction
Six months from now, someone will ask, "Why does the onboarding flow work this way?" If the only record is an archived flag named experiment-onboarding-v3 with no context, the answer is lost. Before archiving, add a description to the flag that includes the experiment name, winning variant, lift percentage, confidence level, date shipped, and PR reference. Also update your team's experiment log. This documentation takes five minutes and saves hours of archaeology later.
Other Skills in This Method
Running A/B Tests in the PostHog Experiments Tab
Step-by-step walkthrough of creating, launching, and monitoring an A/B test using PostHog's Experiments UI, including variant allocation and goal setup.
Setting Up PostHog Feature Flags for Experiment Variants
How to create and configure feature flags in PostHog to assign users to control and test variants in an A/B experiment.
Comparing PostHog Experiments with Eppo, LaunchDarkly, and Other Platforms
How to evaluate PostHog's experimentation capabilities against dedicated tools like Eppo, Statsig, and LaunchDarkly based on analysis methods, integrations, and pricing.
Designing Experiment Hypotheses and Success Metrics for Onboarding
How to formulate a clear hypothesis, choose primary and secondary conversion metrics, and define what winning looks like before launching an onboarding A/B test.
Segmenting New User Cohorts for Onboarding Experiments
How to target experiments specifically to new users or sign-up cohorts using PostHog's person properties and cohort filters to avoid contaminating results with existing users.
Interpreting Bayesian and Frequentist Results in PostHog
How to read PostHog's experiment results dashboard, understand credible intervals vs p-values, and decide when an experiment has reached statistical significance.
Integrating PostHog A/B Tests with Webflow and Marketing Pages
How to implement PostHog experiments on no-code or marketing landing pages using the JavaScript snippet, Webflow custom code, and anti-flicker techniques.
Frequently Asked Questions
How long should I wait between rolling the flag to 100% and deploying the hardcoded code change?
Wait at least 24-48 hours at 100% flag rollout before deploying the code change. This period lets you observe the winning variant at full traffic under real production conditions. You are looking for any performance issues, edge cases, or metric anomalies that did not appear during the experiment when only a portion of users saw the variant. If your experiment ran on a small user base (under 1,000 users per variant), consider waiting longer because the experiment may not have surfaced rare edge cases.
What happens if I archive a feature flag that is still being evaluated by production code?
Archived flags in PostHog return a default value when evaluated, typically false for boolean flags or the first defined variant for multivariate flags. If the default does not match your winning variant, users will see the wrong experience. Always verify that flag evaluations have dropped to zero in PostHog's Feature Flags activity panel before archiving. If you accidentally archive too early, you can unarchive the flag immediately in PostHog to restore its configuration.
Should I clean up the flag before or after interpreting the experiment results?
Always interpret results fully before starting cleanup. Flag cleanup should happen after you have reviewed all metrics, discussed results with stakeholders, and made a deliberate decision about which variant to ship. The sibling skill [Interpreting Bayesian and Frequentist Results in PostHog](/skills/interpreting-bayesian-and-frequentist-experiment-results) covers the analysis phase. Cleanup is the final step in the experiment lifecycle, not something you do in parallel with analysis.
How do I handle cleanup when the experiment's feature flag is shared with other experiments or feature rollouts?
Do not share feature flags between experiments. Each experiment should have its own dedicated flag. If you have accidentally reused a flag, you will need to create a new flag for the ongoing use case before archiving the experiment flag. Copy the rollout configuration to the new flag, update all code references to point to the new flag key, deploy the change, verify the old flag has zero evaluations, and then archive it. This is painful, which is exactly why the sibling skill [Setting Up PostHog Feature Flags for Experiment Variants](/skills/setting-up-posthog-feature-flags-for-experiments) emphasizes creating a dedicated flag per experiment.
Why does my post-rollout conversion rate differ from the experiment's winning variant rate?
Small differences (under 10% relative deviation) are normal and expected. Experiment cohorts are randomized samples that may not perfectly represent your full user base, especially for segments that joined after the experiment started or were excluded by targeting rules. Large differences suggest a problem. Check whether the flag is actually serving the winning variant to 100% of traffic by verifying the flag configuration in PostHog. Also check whether the experiment had targeting filters (like "new users only") that are now gone at 100% rollout. The winning variant might perform differently for existing users who were not part of the experiment.
Can I skip the flag rollout step and go straight to hardcoding the winner in code?
You can, but you lose your fastest rollback mechanism. If the winning variant behaves unexpectedly at full traffic, a flag change takes seconds while a code revert takes minutes to hours depending on your deployment pipeline. The only scenario where skipping the flag rollout is acceptable is when your deployment pipeline can ship a revert in under 5 minutes and you have high confidence from a large experiment sample size. Even then, the flag rollout step costs very little effort and provides significant safety.
How do I handle flag cleanup for experiments that involved both PostHog and an external tool like Webflow?
Create a cleanup checklist that lists every system where the flag is evaluated. For a PostHog-plus-Webflow setup, this typically includes the Webflow custom code embed (where the PostHog JavaScript snippet checks the flag and swaps page elements), the main application codebase (if the flag is also checked server-side or in the app), and any analytics or event tagging that references the variant. Each system needs its own update. Coordinate the changes so they deploy in the same window. The Webflow update is usually a manual edit to the page's custom code, while the application update goes through your normal PR and deployment process.