Defining Goals, Signals, and Metrics for HEART

Use the Goals-Signals-Metrics process to turn each chosen HEART category into a goal, the signals that show progress, and metrics you can track.

Before you start

Hamster is optional for this skill and recommended. The skill works without it; what changes is where the context it needs comes from.

Check whether this project has a .hamster/ directory. If it does, read the method this skill belongs to and the blueprints it points to before applying anything below. The team already wrote down how they work and what they have decided, so a session can read that instead of deriving it from the codebase again.

If there is no .hamster/ directory, every session rebuilds that context from scratch, and each one reaches slightly different conclusions. Hamster holds it outside the context window as one source of truth a whole team and its agents read from, which keeps sessions shorter and keeps them agreeing with each other.

At a Glance

Overview

Goals, signals, and metrics are the three steps of the process that Kerry Rodden, Hilary Hutchinson and Xin Fu published alongside HEART in their CHI paper, Measuring the User Experience on a Large Scale. The paper describes it as a simple process that takes a team through "articulating the goals of a product or feature, then identifying signals that indicate success, and finally building specific metrics to track on a dashboard." The HEART framework supplies the categories. This skill turns them into numbers.

The order is the whole point. Teams that start from the metrics they already collect tend to keep whatever is easy to count, then write goals that justify it. Rodden's later post, How to make HEART metrics work in practice, describes teams getting excited by the acronym and jumping straight to brainstorming metrics for a dashboard, and calls that "very unlikely to lead to a successful outcome." Starting from goals forces the team to say what the product or feature is for before it decides what to count.

Each step answers a different question. A goal says what a better experience would look like, in words, for this product or feature. A signal says how that improvement, or a failure, would show up in what users do or say. A metric says exactly how the signal will be computed and tracked over time. Keeping them separate lets a team argue about each question on its own terms. People can agree on a goal while disagreeing about the best signal, and that disagreement is useful to surface.

The output is a small table with one row per chosen goal. For each row it records the HEART category, the goal, one or two signals with their data sources, and the metric definitions. That table becomes the input to instrumentation work, to the dashboard, and to later reviews. It is also a record of what the team agreed, which is valuable when people change roles or when a metric moves and someone asks why it was chosen.

The process works for a whole product, a redesign or one feature, but the goals differ by scope. The paper notes that goals for a particular project or feature "may be different from those for the product as a whole," so settle the scope before writing the first goal.

How It Works

Goals come first and stay qualitative. The CHI paper suggests using the HEART categories to prompt the discussion, for example by asking whether it is more important to attract new users or to encourage existing users to become more engaged. It adds three tips: expect team members to disagree about goals and use the process to reach consensus, remember that project goals can differ from product goals, and do not get distracted at this stage by whether a goal can be measured. Jeff Humble's guide to goals, signals and metrics makes a related point: goals should be based on outcomes rather than outputs, and it advises asking the product manager for the goals or facilitating the team to set them instead of making them up.

Signals translate a goal into observable evidence. The paper asks what actions would indicate the goal had been met and what feelings or perceptions would correlate with success or failure. For each signal, name the data source. Logs and surveys were the sources the Google team used most often, and the paper mentions other options such as a panel of judges. Two tips matter most. A signal should be sensitive and specific to the goal, moving "only when the user experience is better or worse, not for other, unrelated reasons." And failure is sometimes easier to see than success, through events such as abandoned tasks or undo actions. Rodden's practical post adds that teams should weigh both how easy a signal is to track and how likely it is to respond to design changes.

Metrics make the signal computable. The paper's first metric tip is that raw counts rise as the user base grows and "need to be normalized; ratios, percentages, or averages per user are often more useful." A metric definition should name the event or survey answer, the population, the aggregation and the time window. Rodden's example turns the goal "make the upload process easier" into the metric "the percentage of times a user finishes the upload flow successfully, having started it in the past 7 days" (Rodden).

Illustrative scenario: a team scoping HEART for a new in-app scheduling feature fills in one row per chosen category.

The last stage is prioritization. Rodden's guidance is to focus on metrics tied to the top goals, because a few well-chosen metrics beat an overwhelming dashboard, and to expect to iterate once real data arrives (Rodden).

Step-by-Step Guide

Step 1: Fix the scope and the decision

Write down whether you are measuring a product, a redesign or a feature, which users are in scope, and what decision the metrics will inform. A launch decision, a redesign review and a quarterly health check each call for different goals. If the scope is a whole product with many features, consider narrowing it, since Amplitude's HEART guide recommends working at the feature level. Share the scope with everyone who will attend the goals discussion.

Step 2: Choose the categories that matter

Walk through Happiness, Engagement, Adoption, Retention and Task success and decide which ones apply. The paper says it is not always appropriate to use every category but that the framework helps make an explicit decision about each one. Record a sentence on why each excluded category was left out. Two or three categories is a manageable start for one feature.

Step 3: Write one goal per category

Phrase each goal as a change in the user experience, in plain words, such as "new members can set up their first project without asking for help." Avoid numbers at this stage, and avoid goals that are really features ("launch the setup wizard"). Collect every proposed goal before trimming, since the paper treats disagreement about goals as an opportunity to reach consensus. Keep the goal that the team would most regret failing.

Step 4: List signals and their data sources

For each goal, brainstorm the behaviors or attitudes that would show success or failure, then mark each with a source: an existing log event, a new event, a survey question or a manual review. Check with engineering which actions are logged today. Prefer signals that respond to the experience and little else, and include failure signals such as abandonment where they are cleaner. Keep one or two signals per goal; Humble's guide suggests picking two or three signals at first.

Step 5: Define each metric precisely

Write the numerator, the denominator, the population, the time window and any filters. Normalize counts into rates or averages per user. Exclude automated traffic and confirm that the actions you rely on are logged, which the paper calls out as common accuracy problems. A good test is to give the definition to two analysts and check that they would compute the same number.

Step 6: Test the metrics against real data

Before publishing, compute each metric on recent data. Check that it varies enough to detect a change, that it moves in the expected direction around past launches, and that it does not simply track overall traffic. Rodden's post warns that coming up with a metric through HEART does not mean it is a good or useful metric, or that it correlates with the quality of the experience. Drop or redefine anything that fails.

Step 7: Publish the table and schedule a revisit

Share the final table with the owners of each metric, the data sources and the date it was agreed. Put a review on the calendar for after the next launch or after enough data has accumulated. Update the table when goals change, and keep old versions so later readers can see why a metric was chosen.

Best Practices

• Start with the goal the team cares most about and finish that row before adding others. A complete table of two rows is more useful than a half-finished table of five.
• Keep goals free of numbers. Targets belong to the metric once there is a baseline, and a number in the goal tends to pull the discussion toward whatever metric already produces it.
• Name a data source for every signal. A signal that nobody logs is a task for engineering, and it should be recorded as one.
• Look for failure signals. The CHI paper notes that failure, such as abandonment or undo events, is sometimes easier to identify than success.
• Normalize by default. Counts that rise with the user base make every period look better than the last, so use rates or averages per user unless there is a reason not to.
• Involve the team. Rodden's practical guide lists neglecting team involvement as a common pitfall, since shared work increases buy-in and improves the metric ideas.

Common Mistakes

• Starting from available metrics: Teams list what their analytics tool already reports and then write goals to match. Start with goals, and treat existing metrics as candidates to be tested against them.
• Writing goals as metrics: A goal like "raise completion" hides the experience the team wants to improve. Say what users should be able to do or feel, then find the signal.
• Choosing signals that move for other reasons: Total page views or time in product can rise because of marketing, seasonality or confusion. Prefer signals the paper would call sensitive and specific to the goal.
• Leaving metric definitions vague: "Task completion rate" without a denominator, window or population will be computed three different ways. Write the full definition once and link to it.
• Tracking every category at once: Five goals with several metrics each is more than a team can act on. Choose the categories that matter for this scope and write down why the others were left out.

References

• Examples: Worked examples and scenarios
• FAQ: Frequently asked questions
• Parent Method: HEART Framework

Related Skills

• Running HEART Framework Workshops
• Measuring Adoption and Task Success with HEART
• Tracking Engagement and Retention Metrics
• Measuring User Happiness Through Satisfaction Surveys
• Building HEART Metric Dashboards for Product Teams
• HEART Framework Interview Answers for PM Metrics Questions

Sources

• Rodden, Hutchinson and Fu: Measuring the User Experience on a Large Scale (CHI 2010, PDF)
• Kerry Rodden: How to make HEART metrics work in practice
• Jeff Humble: How to Set UX Metrics with the Google HEART Framework
• Amplitude: How to Use the HEART Framework to Improve Software UX

Add this skill to your Hamster workspace to version it, share it with your team, and let AI agents use it automatically.