Custom success metrics in Agent Analytics let you define what a successful conversation looks like for an AI agent and track how often your agent achieves it. They go beyond usage volume to measure whether your agent is meeting the outcome it was built for.
Use cases
Use custom success metrics to:
- Measure whether your agent completes the job it was designed for, such as resolving a billing question or finishing onboarding.
- Compare success rates over time to see whether changes to your agent improve outcomes.
- Track progress toward a target by setting a goal on a metric.
How custom success metrics work
Each custom success metric has a type, which you choose when you create it:
- Intent: Measures conversations that meet your success rules.
- Event: Measures how often conversations with this agent lead to product events you choose.
- Hybrid: Measures how often conversations that meet your success rules lead to product events you choose.
When you create an intent or hybrid metric, you describe what a successful conversation looks like in plain language. Pendo generates a set of yes-or-no rules from your description, which you review and edit before saving. Pendo then evaluates each conversation against those rules and counts it as successful when it meets at least one rule.
Event and hybrid metrics also use a funnel. The first step is your agent: every conversation for an event metric, or only the conversations that meet your rules for a hybrid metric. The next steps are the pages, features, track events, guides, or AI agents you expect visitors to reach afterward, and the metric reports how many conversations complete the funnel.
Because a conversation passes when it meets any one rule, each rule is a separate way for a conversation to succeed. Adding a rule makes a metric broader, so expect the pass rate to stay the same or go up.
After a conversation ends, Pendo evaluates it against your rules. To check that a metric is working as intended, review the conversations it counted as successful. For details, see Read your results.
Each custom success metric belongs to a single agent and appears alongside the built-in metrics in the agent's Adoption section.
Note: Evaluation runs after a conversation ends rather than in real time, so results can take some time to appear.
Before you begin
Any user who can access Agent Analytics can create custom success metrics and edit or delete metrics they created. Users with admin access can also edit or delete metrics created by other users; for other users, those actions aren't available.
Custom success metrics are available for both prompt agents and conversation agents. You can add up to 10 custom success metrics to each agent.
Create a custom success metric
Create a custom success metric from your agent's Adoption section. After you define the metric, the remaining steps depend on its type: an intent metric has a Review rules step, an event metric has a Set up funnel step, and a hybrid metric has both.
- Go to Product > Agent Analytics, then select an AI agent or open a saved report.
- In the Adoption section of the Overview tab, select Create metric.
- Enter a Metric name. The name must be different from any default or existing custom success metric.
- Choose a metric type: Intent, Event, or Hybrid. You can't change the type after you create the metric.
-
Under What to track, answer How can you tell a conversation succeeded? Describe an outcome you could confirm by reading the conversation. A detailed description generates more accurate detection rules. After you save, this description appears in a tooltip next to the metric name.
- Select Next.
- For an intent or hybrid metric, on the Review rules step, review the yes-or-no rules Pendo generated from your description. Edit, remove, or add rules so they match what you want to measure. Write each rule so it can be answered with yes or no. A hybrid metric needs at least one rule.
- For an event or hybrid metric, on the Set up funnel step, build the funnel. The first step is your agent and can't be changed. Add up to nine more steps, choosing a page, feature, track event, guide, or AI agent for each, and add filters to a step if you need them. Set the Funnel timeout (minutes), which defaults to 60, and the Date range, which defaults to the last 30 days. To measure only some visitors, add a segment or account filter.
- Select Create metric to save.
The metric is added to your agent and appears in the Adoption section.
Tip: Be specific about what counts as success. A description like "The visitor confirmed their issue was resolved" generates better rules than "Improves the experience." Because a conversation passes when it meets any one rule, write each rule so it confirms success on its own. A rule that's only sometimes true in successful conversations, such as "The agent asked a follow-up question," can inflate your pass rate.
Read your results
Custom success metric results appear at two levels in the Adoption section:
-
In the metric tile, you see the number of conversations that passed, the pass rate for your selected filters and date range, and the change compared to the previous period of the same length. Read the pass rate as "conversations that met at least one of my success criteria."
- In the trending chart, select View details at the top of the chart or select a data point to see the conversations counted as successful, as well as the related visitors and accounts. Each conversation contains a summary of why it was considered successful. If you review conversations that pass or fail in a way you didn't expect, consider refining your rules.
- For an event or hybrid metric, the metric tile shows the conversion rate: the share of conversations that went on to complete the funnel. Select the funnel widget to see the conversations, accounts, and visitors behind it.
Results for custom success metrics update once daily. Conversations that begin today won't be included in your results until the following day.
You can also use a custom success metric as a step in a funnel or the starting point of a path, where it's listed as Tracked intent metric.
Track a metric against a goal
Set a goal to track a metric's performance against a target over time.
- After the metric finishes processing, select it to open the trending chart.
- In the top-right corner of the chart, select Create goal.
- Choose a cadence, such as Weekly.
- Enter a Goal value, such as the number of conversations to reach in that period. For an event or hybrid metric, the goal is a conversion rate.
- To check your goal matches the metric's intent, review the name and success description at the bottom of the window.
- Select Save.
After you set a goal, the metric tile indicates that there's a Goal set at the bottom of the tile. The metric's trend chart shows the metric value over time, the goal, a baseline reference, and a summary of how many recent periods met or exceeded the goal. You can edit or remove a goal from the metric's trend view at any time.
Manage your metrics
Manage which metrics appear, keep them up to date, and remove the ones you no longer need. You can't edit or delete the default metrics, but you can hide them from the view.
You manage all of these from your AI agent's details in the Adoption section of the Overview tab.
Change which metrics appear
Use the column manager icon in the top-right corner of the Adoption section to choose which metrics to show and in what order.
Your visibility and order settings are saved for your account and persist across sessions.
- Reorder metrics by dragging them into the order you want.
- Remove a metric from the view without deleting it, and add it back with + Add metric at the bottom of the window. You can also remove it from view by selecting the options menu in the metric tile, then choosing Hide.
Edit a metric
Update a custom success metric's name, success description, rules, or funnel as your agent evolves. You can't change a metric's type after you create it.
Important: Editing a metric's rules changes how past and future prompts and conversations are scored against it.
- Select the options menu in the top-right corner of the metric tile, then choose Edit.
- Update the name, success description, rules, or funnel.
- Select Save.
Delete a metric
- Select the options menu in the top-right corner of the metric tile, then choose Delete.
- Confirm the deletion when prompted.
Deleting a custom success metric is permanent and also removes any goal data for that metric. You can add it as a new metric at any time.