Sign InGet Started

Segmenting New User Cohorts for PostHog Experiments with New Users

Teaches you how to use PostHog's person properties and cohort filters to restrict onboarding experiments exclusively to new sign-ups, preventing existing users from contaminating your A/B test results.

Define a cohort in PostHog using person properties like initial signup date, first-seen timestamp, or a custom "is_new_user" property set during registration. Apply this cohort as a filter on your experiment's feature flag so only users matching the criteria receive variant assignments. This prevents existing users from entering the experiment and skewing onboarding metrics with their established behavior patterns.

Outcome: Your onboarding experiments will only include genuinely new users in their variant assignments, producing clean data that reflects actual first-time user behavior rather than a mixture of new and returning users.

Synthesized from public framework references and reviewed for accuracy.

ExperienceIntermediate45-90 minutes

Prerequisites

  • Basic familiarity with PostHog's event tracking and person properties
  • Understanding of feature flags and how PostHog assigns users to experiment variants
  • A working PostHog installation that captures sign-up and identification events
  • Clarity on what 'new user' means for your product (first 24 hours, first session, pre-activation, etc.)

Overview

Every onboarding experiment has a contamination problem hiding in plain sight. If you launch an A/B test on your sign-up flow or welcome sequence without restricting who enters the experiment, existing users who revisit the onboarding path, users who clear cookies and re-trigger identification, and internal team members testing the product will all receive variant assignments. Their behavior pollutes your metrics because they already know your product. A returning user completing onboarding in two minutes tells you nothing about whether your new flow is clearer for a genuine first-timer. This skill teaches you how to build cohort definitions in PostHog that precisely target new users and exclude everyone else from your experiment.

Within the PostHog Experiments Onboarding A/B Test Method, segmenting new user cohorts sits between designing your experiment hypothesis and launching the feature flag. The hypothesis tells you what you are testing. The cohort definition tells you who should be in the test. The feature flag then uses that cohort as a release condition, so variant assignment only happens for users who match. If you skip this step or get the definition wrong, your experiment's statistical results may look decisive but point in the wrong direction because the underlying sample was impure.

The concrete artifact you produce is a saved PostHog cohort, configured with person property filters, that you attach to your experiment's feature flag. You will also produce a brief specification document listing the exact property names, operators, and values that define "new user" for this experiment, along with the rationale for each filter. This specification becomes a reference for your team so that future onboarding experiments use a consistent definition. Without this consistency, two experiments targeting "new users" might mean different things, making cross-experiment comparisons unreliable.

When done correctly, your experiment will include only users who signed up after the experiment started, who have not previously completed onboarding, and who are not internal team members. The result is a clean sample that lets you trust your conversion rate differences and make a confident shipping decision.

How It Works

PostHog identifies users through two mechanisms: anonymous distinct IDs assigned on first page load or app open, and identified person profiles created when your code calls posthog.identify(). Person properties are key-value pairs attached to these profiles. They persist across sessions and can be set from your backend, your frontend, or PostHog's API. Cohorts are saved groups of persons defined by filters on these properties, on events those persons have performed, or on combinations of both.

The core idea behind new-user segmentation is straightforward: you need at least one person property that distinguishes a genuinely new user from everyone else. The most reliable approach is a property set at the moment of account creation, because it cannot be retroactively altered by user behavior. Common choices include a created_at timestamp set during registration, a signup_source property indicating the acquisition channel, or a boolean has_completed_onboarding property that starts as false and flips to true when the user finishes the flow. The timestamp approach is the most flexible because you can filter for users who signed up after your experiment's start date, which automatically excludes anyone who existed before the test began.

PostHog's feature flags accept cohort membership as a release condition. When a user triggers a flag evaluation, PostHog checks whether that user belongs to the specified cohort. If yes, the user receives a variant assignment (control or test). If no, the flag returns the default value, and the user never enters the experiment. This filtering happens server-side when you use the API, or client-side when you use the JavaScript SDK with posthog.onFeatureFlags(). The critical detail is that the cohort check must happen before the user sees any experiment-specific UI. If you render the onboarding flow first and then check cohort membership, the user has already experienced the variant even if you later exclude them from analysis.

Cohort definitions in PostHog can be static or dynamic. Static cohorts are fixed lists of person IDs uploaded or calculated once. Dynamic cohorts recalculate on the fly based on current property values. For onboarding experiments, you almost always want a dynamic cohort because new users appear continuously. A static cohort would require constant manual updates.

The reason this segmentation matters statistically is tied to variance. Existing users have lower variance in onboarding-related metrics because they have already formed habits and expectations. Mixing them with new users compresses the observed effect size and can either mask a real improvement or create a false positive when returning users disproportionately land in one variant. By restricting the sample to genuinely new users, you increase the signal-to-noise ratio and reduce the sample size needed to reach significance. This is why the PostHog Experiments Onboarding A/B Test Method emphasizes cohort segmentation as a prerequisite for trustworthy onboarding results, not an optional refinement.

Step-by-Step

  1. Define what 'new user' means for this experiment

    Before touching PostHog, write down the precise definition of a new user for this specific experiment. Consider three dimensions: temporal (signed up after experiment start date), behavioral (has not completed onboarding before), and identity (is not an internal team member or test account). Decide which dimensions matter. For most onboarding experiments, you need all three.

    Write this definition in a shared document or your experiment brief so that everyone on the team agrees on who qualifies. This written definition is your source of truth when you translate it into PostHog filters in the next steps.

    Tip: The definition should be falsifiable. Instead of 'recently signed up,' write 'created_at is after 2024-06-01 AND has_completed_onboarding is false AND email does not contain @yourcompany.com.' Ambiguity in the definition becomes ambiguity in the results.

  2. Audit your existing person properties in PostHog

    Open PostHog and navigate to Persons. Click on a few recently created user profiles and examine their person properties. Look for properties that map to your definition from Step 1. Do you have a created_at or signed_up_at timestamp?

    Is there a has_completed_onboarding boolean? Is there an email property you can use to filter out internal accounts? Make a list of properties you have, properties you need to add, and properties with inconsistent formats. If a property exists but only on some profiles, note when it started being set so you know whether it covers your experiment window.

    Tip: PostHog's property definitions page (Data Management > Properties) shows every property, its type, and how many events or persons have it. Check there first before clicking through individual profiles.

  3. Instrument any missing person properties

    For each property in your 'need to add' list, implement the tracking code. The most common addition is a created_at timestamp set during registration. toISOString(), has_completed_onboarding: false } } }). identify().

    For filtering out internal users, set a property like is_internal: true on team member accounts. Deploy these changes before launching the experiment and verify they appear on test profiles.

    Tip: Set properties from the backend whenever possible. Frontend-set properties can be lost if the user closes the page before the API call completes, creating holes in your cohort definition.

  4. Create the cohort in PostHog

    Go to People > Cohorts > New Cohort. ' Select 'Dynamic' as the cohort type. Add your first filter: person property created_at is after your experiment start date. Add a second filter: person property has_completed_onboarding is false (or is not set, depending on your implementation).

    com(oris_internal` is not true). Set the match condition to 'all' so every filter must pass. Save the cohort and wait a moment for PostHog to calculate the initial membership count. Verify the count looks reasonable given your sign-up volume.

    Tip: If the cohort count is zero or suspiciously low, click into a few recent sign-ups and confirm they have the expected properties. A common issue is a property name mismatch, like `createdAt` in your code versus `created_at` in your filter.

  5. Attach the cohort to your experiment's feature flag

    Navigate to your experiment's feature flag (or create it if you have not yet). Under Release Conditions, remove any existing conditions that target all users. ' Set the rollout percentage to 100% within this cohort, since the cohort itself handles the filtering. PostHog will only assign a variant to users who pass the cohort check.

    Save the flag. At this point, the flag is not yet active unless you have toggled it on, so you can safely configure without affecting production.

    Tip: If you need to target a subset of new users (for example, only new users from a specific acquisition channel), add that as an additional person property filter on the flag release condition rather than complicating the cohort definition. This keeps the cohort reusable.

  6. Validate the segmentation with test users

    Create two test accounts: one that meets all cohort criteria (new sign-up, has not completed onboarding, not internal) and one that fails at least one criterion (for example, an account with is_internal: true or a created_at before your cutoff). Log in as the qualifying user and trigger the feature flag evaluation. Confirm they receive a variant assignment by checking the feature flag value in your frontend or by inspecting the $feature_flag_called event in PostHog's Activity tab. Log in as the disqualifying user and confirm they receive the default value (no variant).

    If either check fails, revisit your cohort filters and flag conditions.

    Tip: In PostHog, you can use the Feature Flag detail page's 'Test' panel to evaluate the flag for a specific user by entering their distinct ID. This is faster than logging in as each test user.

  7. Set up an exclusion list for edge cases

    Even with a well-built cohort, some edge cases will slip through. Users who sign up, leave, and return weeks later might technically be 'new' by timestamp but behaviorally stale. Users who sign up with multiple emails create duplicate entries. Handle these by creating a static cohort called 'Experiment Exclusions' where you can manually add problematic users.

    ' This gives you a manual escape valve without changing the dynamic cohort definition.

    Tip: Keep a running log of why each person was added to the exclusion list. This log helps you refine your dynamic cohort definition for the next experiment so fewer manual exclusions are needed.

  8. Document the cohort specification and share with your team

    Write a brief specification that includes the cohort name, every filter and its operator, the rationale for each filter, the experiment it is attached to, and the date the cohort was created. Store this alongside your experiment hypothesis document. Share it with engineering so they know which person properties are critical and must not be renamed or deprecated without updating the cohort. Share it with product and data teams so anyone analyzing the experiment results understands exactly who was included.

    This documentation takes ten minutes and prevents hours of confusion during analysis.

    Tip: If you run multiple onboarding experiments over time, maintain a table of all cohort definitions with their date ranges. This makes it easy to see whether experiments had overlapping or conflicting definitions.

Examples

Example: SaaS product testing a new welcome wizard for self-serve sign-ups

A B2B SaaS tool with 200 sign-ups per day wants to test whether a three-step welcome wizard (variant) converts more users to first project creation than the current single-page setup (control). The team needs to exclude 15,000 existing users, internal team accounts using @company.com emails, and partner demo accounts.

com. js backend during the registration handler. They create a dynamic cohort in PostHog called 'New Users - Welcome Wizard Test - Jun 2024' with all four filters combined using AND logic. The initial cohort count shows 0 users, which is correct since the experiment has not started.

They attach the cohort to their welcome-wizard-experiment feature flag as the sole release condition with 100% rollout within the cohort. After launching, they verify by creating a test account and confirming variant assignment, then checking that a pre-existing account receives the default flag value. After three days, the cohort count is 587, which aligns with their 200/day sign-up rate minus a small percentage filtered by the internal and demo account exclusions. The experiment runs for two weeks and captures 2,400 clean new-user profiles, with no contamination from existing users.

Example: Mobile app testing simplified sign-up for organic installs

A consumer fitness app receives 5,000 installs per week from organic App Store and Play Store traffic. The product team wants to test a simplified three-field sign-up screen against the existing five-field version. They need to exclude users who previously installed the app and deleted it, as well as users on the team's internal TestFlight or Firebase distribution.

The team defines 'new user' as a user whose first_app_open_at timestamp is after July 1, 2024, whose install_source is 'organic' (not 'internal_test'), and who has not previously had a registration_completed event. identify(). For reinstall detection, they check whether a user ID already exists in their backend database and set is_reinstall: trueif so. The PostHog cohort 'New Organic Users - Signup Field Test - Jul 2024' uses four filters:first_app_open_atafter July 1,install_sourceequals 'organic',is_reinstallis not true, andis_internal_tester` is not true.

After one week, the cohort contains 4,120 users. The team notices 880 fewer than expected and investigates. They discover that 600 were reinstalls and 280 were internal testers, confirming the filters are working correctly. The experiment produces clean data showing the three-field sign-up form improves registration completion by 12%.

Example: E-commerce platform testing onboarding email sequence for new sellers

A two-sided marketplace has 50 new seller sign-ups per day. The growth team wants to test whether a five-email onboarding drip (variant) produces more first listings within 14 days than the current two-email sequence (control). The challenge is that some sellers also have buyer accounts, and the team needs to target only users who are new to the seller side.

The team defines 'new seller' as a user with seller_account_created_at after August 1, 2024, user_type containing 'seller', and has_listed_first_product equal to false. They choose seller_account_created_at rather than general created_at because many sellers had existing buyer accounts. The person properties are set in their Django backend when a user completes seller registration. The PostHog cohort 'New Sellers - Email Drip Test - Aug 2024' filters on all three properties using AND logic.

They attach this cohort to a feature flag called seller-onboarding-email-variant. io) checks this feature flag via PostHog's API when deciding which email sequence to trigger. Testing reveals a subtle issue: some users convert from buyer to seller and their user_type changes, but seller_account_created_at correctly captures when the seller account was created. After four weeks, 1,400 new sellers enter the experiment, split evenly between control and variant.

The team excludes 12 users manually via the exclusion cohort because they were test accounts created by the partnerships team. The final analysis shows the five-email sequence increases first-listing rate by 18% with 95% confidence.

Example: Developer tool testing interactive tutorial for new API users

An API-first developer tool gets 80 new API key registrations per day. The developer experience team wants to test whether an interactive tutorial (variant) increases the percentage of users who make their first successful API call within 48 hours compared to the current documentation link (control). The complication is that many developers create multiple API keys across different projects.

' They deduplicate by user account rather than API key, setting person properties on the account-level distinct ID. The PostHog cohort 'New API Users - Tutorial Test - Sep 2024' uses three person property filters with AND logic. They attach the cohort to the api-onboarding-tutorial feature flag. The flag is evaluated server-side when the dashboard renders the post-registration page, ensuring anonymous pre-registration visits never receive a variant.

After two weeks, the cohort contains 1,050 users. The team spots that 30 users who entered the experiment had prior API call events from a beta period before the property was instrumented. They add these users to the static exclusion cohort and note in their specification document that api_call_success event history should be checked for users with accounts predating the property instrumentation date. The experiment ultimately shows the interactive tutorial increases first-successful-call rate from 34% to 51%.

Best Practices

  • Set the cohort's temporal filter to match the experiment's start date exactly, not a rough approximation. If your experiment launches on June 15 at 2pm UTC, use that datetime as the created_at cutoff. A one-day rounding error can include hundreds of pre-existing users on a high-traffic product, diluting your signal.

  • Always use a dynamic cohort for onboarding experiments rather than a static one. New users arrive continuously, and a static cohort would need manual updates every day. Dynamic cohorts recalculate automatically, which means every qualifying sign-up enters the experiment without any intervention from your team.

  • Combine timestamp-based filtering with behavioral filtering. A created_at filter alone catches most cases, but adding has_completed_onboarding is false handles the edge case of users who signed up recently but somehow bypassed or completed onboarding through a non-standard path, like a direct invite link or a demo account conversion.

  • Exclude internal team members by property, not by email domain alone. Contractors, advisors, and agency partners may use personal email addresses. Set an is_internal boolean property on all non-customer accounts and filter on that property. If you rely only on email domain matching, you will miss these accounts and their atypical behavior will affect your results.

  • Test your cohort definition against the Persons list before attaching it to a flag. Go to People > Persons, apply the same filters you used in the cohort, and manually review 10-15 profiles. Confirm they look like genuine new users. If you spot returning users, internal accounts, or bot-generated profiles, tighten your filters before proceeding.

  • Keep one cohort per experiment rather than reusing cohorts across experiments. Even if two experiments target 'new users,' their start dates differ, which means the temporal filter must differ. Sharing a cohort creates a dependency where changing the date for Experiment B breaks the analysis for Experiment A.

  • Name cohorts with the experiment name and date range included, for example 'New Users - Onboarding CTA Test - Jun 2024.' Generic names like 'New Users' become ambiguous within weeks as experiments multiply. A descriptive name makes it possible to audit cohort usage months later without clicking into each one.

Common Mistakes

Using event-based filters instead of person property filters to define the cohort

Correction

Filtering by 'performed signup event after date X' seems equivalent to filtering by 'created_at after date X,' but it is not. Event-based filters check whether the event exists in PostHog's event log, which can include duplicate or replayed events. If a user's sign-up event is replayed during a data migration, they re-enter the cohort even though they are not new. Person properties are set once and persist, making them a more stable foundation.

If you notice your cohort count jumping unexpectedly, check whether an event replay or a tracking code change is creating phantom matches.

Forgetting to filter out users who signed up before the experiment but have never completed onboarding

Correction

Some products have a long tail of users who signed up weeks or months ago and never returned. If you use only a created_at filter with a generous date range, these dormant users will enter the experiment if they return during the test window. Their behavior is fundamentally different from a same-day sign-up. Add a behavioral filter like has_completed_onboarding is false AND first_seen is after [experiment start] or tighten your created_at window to within hours of the experiment launch.

Watch for unusually high time-to-conversion in your experiment data, as this often signals dormant users re-engaging.

Applying the cohort filter at the analysis stage instead of at the assignment stage

Correction

Some teams let all users enter the experiment via the feature flag and then filter down to new users when analyzing results. This introduces survivorship bias because existing users who performed well inflate one variant's metrics even after filtering, if the filtering is imperfect. It also wastes your sample size budget on users whose data you will discard. The cohort filter must be a release condition on the feature flag itself, so variant assignment never happens for disqualified users.

If you see a large gap between your flag's total evaluations and your experiment's analyzed user count, the filter is in the wrong place.

Not accounting for anonymous-to-identified user merging

Correction

identify() is called. If a user visits your site anonymously, gets assigned a feature flag variant as an anonymous user (before sign-up), and then signs up and identifies, the flag assignment persists even though the user might not match your cohort's person property filters at initial page load. This means anonymous visitors can receive an experiment variant before you even know whether they are new. identify() completes.

Check for $feature_flag_called events where the user's distinct ID is an anonymous hash rather than an identified ID.

Setting the cohort date filter to a date in the past and never updating it for new experiments

Correction

If you copy a cohort from a previous experiment and forget to update the created_at cutoff date, users from the old experiment's time window will be included in the new one. These users may have already experienced a previous onboarding variant, which contaminates your new test. Always set the temporal filter to the launch date of the current experiment. Before launching, double-check the date by viewing the cohort's filter summary in PostHog.

A quick sanity check is to compare the cohort count to your expected daily sign-up rate multiplied by the number of days since the cutoff, as the count should be in the same order of magnitude.

Using 'OR' logic instead of 'AND' logic when combining cohort filters

Correction

PostHog cohorts let you combine filter groups with AND or OR logic. If you accidentally set filters to OR, a user qualifies by matching any single filter. This means an internal team member who signed up recently would pass the created_at filter even though they fail the is_internal filter. Always set the match condition to 'Match ALL filter groups.' When you save the cohort, re-read the filter summary to confirm it says 'all of the following' rather than 'any of the following.' A cohort count that is much larger than expected is the clearest signal that OR logic is active.

Other Skills in This Method

Running A/B Tests in the PostHog Experiments Tab

Step-by-step walkthrough of creating, launching, and monitoring an A/B test using PostHog's Experiments UI, including variant allocation and goal setup.

Setting Up PostHog Feature Flags for Experiment Variants

How to create and configure feature flags in PostHog to assign users to control and test variants in an A/B experiment.

Comparing PostHog Experiments with Eppo, LaunchDarkly, and Other Platforms

How to evaluate PostHog's experimentation capabilities against dedicated tools like Eppo, Statsig, and LaunchDarkly based on analysis methods, integrations, and pricing.

Shipping the Winning Variant and Cleaning Up Feature Flags

How to roll out the winning experiment variant to 100% of users, remove the losing variant's code, and archive feature flags to keep your codebase clean after an experiment concludes.

Designing Experiment Hypotheses and Success Metrics for Onboarding

How to formulate a clear hypothesis, choose primary and secondary conversion metrics, and define what winning looks like before launching an onboarding A/B test.

Interpreting Bayesian and Frequentist Results in PostHog

How to read PostHog's experiment results dashboard, understand credible intervals vs p-values, and decide when an experiment has reached statistical significance.

Integrating PostHog A/B Tests with Webflow and Marketing Pages

How to implement PostHog experiments on no-code or marketing landing pages using the JavaScript snippet, Webflow custom code, and anti-flicker techniques.

Frequently Asked Questions

How do I define 'new user' if my product does not have a clear registration event?

Use PostHog's `$initial_referrer`, `$initial_current_url`, or the automatically captured `$created_at` person property as proxies. If users can access your product without registering (for example, a freemium tool with anonymous usage), set a custom `first_meaningful_action_at` property when the user performs their first significant action, such as creating a project or saving a file. Use this timestamp as your cohort filter instead of a registration date. The key is choosing a moment that reliably distinguishes first-time engagement from return usage.

Should I create the cohort before or after setting up the feature flag?

Create the cohort first. The feature flag configuration screen lets you select an existing cohort as a release condition, but you cannot create a new cohort inline during flag setup. Building the cohort first also lets you verify its membership count and spot-check profiles before anything is connected to the experiment. If the cohort looks wrong, you can fix it without risk of accidentally exposing a broken experiment to users.

How long should I wait after instrumenting new person properties before launching the experiment?

Wait at least 48 to 72 hours after deploying the tracking code that sets new person properties. This buffer lets you verify that properties are being set consistently, catch any instrumentation bugs in production, and accumulate enough profiles to validate your cohort filters against real data. Check that the property appears on 95%+ of new sign-ups during this window. If the hit rate is lower, investigate missing code paths (such as social login flows or mobile app sign-ups that bypass your main registration handler).

Can I use PostHog's built-in 'first seen' or 'initial properties' instead of custom person properties?

PostHog automatically sets `$initial_referrer`, `$initial_current_url`, `$initial_browser`, and similar properties on first identification. However, PostHog does not automatically set a `$created_at` property in all configurations. If your setup does capture `$created_at` automatically, you can use it for temporal filtering. Check a few recent profiles to confirm. The advantage of custom properties like `created_at` or `has_completed_onboarding` is that you control exactly when they are set and what they mean. Built-in properties may be set at unexpected times, such as when a user is first seen as anonymous rather than when they register.

What happens if a user qualifies for the cohort, gets assigned a variant, and then later stops qualifying?

PostHog's feature flag assignment is sticky by default. Once a user receives a variant, they keep that variant for the duration of the experiment even if their person properties change. For example, if a user completes onboarding and `has_completed_onboarding` flips to true, they remain in the experiment with their original variant. This is correct behavior. You want to track their full journey from first exposure through conversion, not drop them mid-experiment. However, they will not be included in the cohort's current member count, which can cause confusion during monitoring. Track experiment participation through PostHog's experiment results page rather than the cohort member count.

How do I handle PostHog experiments targeting new users across both web and mobile platforms?

Use a server-side person property set during registration that is platform-agnostic. If you set `created_at` from your backend API when the account is created, the property exists regardless of whether the user signed up on web, iOS, or Android. Your cohort filters on this property will work across all platforms. The feature flag evaluation then happens per-platform using the same underlying person profile. Avoid setting the cohort-defining property from the client SDK because different platforms may have different initialization timing, leading to inconsistent property availability.

Why does my PostHog experiments cohort for new users include users I did not expect?

The most common causes are: person property values with unexpected formats (for example, a `created_at` stored as a Unix timestamp integer rather than an ISO 8601 string, which breaks date comparison operators), OR logic on the cohort filters instead of AND logic, or anonymous user profiles that received properties during pre-identification tracking. Open the cohort, click on a few unexpected members, and examine their person properties. Compare the actual property values to your filter conditions character by character. Also check whether PostHog has merged multiple anonymous profiles into one identified profile, which can carry over properties from unexpected sessions.