Product Engagement Score (PES) Overview

Overview

Product Engagement Score (PES) is a single number that represents the combined impact of feature adoption, return user stickiness, and new user growth. PES is a quantitative value that summarizes your existing usage data without requiring any additional data collection or user participation. Similar to NPS, which is a subjective measurement of user sentiment collected with a survey, PES is an objective measurement of user behavior. Track PES over time and compare PES for different Segments to discover who has the most engagement with your product and why. PES provides a benchmark of your product engagement which can be used to measure the impact product decisions have on your users' behavior.

PES uses Core Events, Visitor, and Account data to calculate the percentile score for each component and combine them into a single score from 0 - 100. Your end users don't need to do anything other than use your product to contribute to the score. Product usage data in Pendo is used to calculate each component of the PES.

Product Engagement Score Overview - Runtime: 1:28

 

Requirements

• Pendo Admin users can configure PES
• Any Pendo user can view PES
• Core Events are required for the Adoption component of PES

 

Using PES

Product Engagement Score (PES) is a new metric that Pendo introduced to the SaaS business landscape. PES provides a singular, quantitative, business metric to measure the overall engagement of a product with the goal of quickly diagnosing how a product is performing. Historically, product leaders have had to rely on qualitative metrics like NPS to get a feel for customer sentiment. NPS uses a simple recommendation score to determine the virality of a product or service. It has always depended on polling customers and has never provided an accurate look at actual user behavior. While important, NPS cannot be the only business metric for a product. PES is intended to fill that quantitative gap and partner with NPS, by truly examining user behavior and quantifying how users are engaging with a product.

PES is a high-level way to assess the health and success of a product through quantitative engagement data, as a counterpart to the sentiment-driven and highly subjective NPS. It is applicable as a business-level KPI for a product and, when analyzed further with Segments and date filters, can be used to gain a more granular understanding of the engagement of user groups over the life of your product. PES partners with other Pendo tools, like Core Events, Goals, and NPS, to provide a comprehensive understanding of user behavior. These insights can then be used to drive product development, in-app messaging, and outreach campaigns with the ultimate goal of increasing product engagement and user satisfaction indicated by improved metrics, or as we like to say, delivering great products that users love.

PES is visible to all Pendo users on the PES page in the Behavior section. Filters for Segment, time period, and app are available for all users to analyze PES.

Tip: Use a Custom Date Range filter to measure PES quarterly and report your app's PES with your NPS.

 

Understanding the Data

The final PES is the average of each component score. Possible scores for all components range from 0 - 100. Any percentages above 100 are adjusted to a score of 100 and percentages below 0 are adjusted to a score of 0.

 

Adoption

Adoption looks at the percentage of users who are using your product effectively as indicated by your Core Events. Adoption can look at Visitor or Account-level usage for any or all Core Events versus all active Visitors or Accounts in the selected time period. A Visitor or Account with any event recorded by Pendo is considered active during the time period.

Core Event selection may significantly impact the Adoption score. If all Core Events are weighted towards specific groups of users or focus on features with limited use, like first-time setup, it may lower adoption. If Core Events include site navigation or features that a user must engage with, like login, it may increase adoption. A distributed selection of Core Events that includes high-value features for all user groups will result in a balanced Adoption score. There is nothing wrong with Core Event selection deliberately focusing on certain user groups and feature sets for the core of your user base. Just understand how that will shift the Adoption score and account for it in your analysis. Consider calculating adoption at the Account level if Core Events aren't applicable to all users.

 

Stickiness

Stickiness looks at the average percentage of users who are high-frequency return users in Daily/Weekly/Monthly active user metrics. The Stickiness component finds the ratio of average daily or weekly active Visitors to average weekly or monthly active Visitors in the selected time period. A Visitor or Account with any event recorded by Pendo is considered active during the time period. The DAU/WAU/MAU values use the same calculation and data as the Stickiness Dashboard widget.

Active user metrics measure unique users who used your product during a time period. A user who uses the product today is a daily active user and also a weekly active user. A user who used the product this week but not today is a weekly active user. In this example, if we look at DAU/WAU for today, it's 50%. There is one unique active user today and two unique active users this week. This calculation can be processed for every day and averaged to produce the average DAU/WAU for a larger time period. If users are engaging with a product and finding value, they will return frequently and the ratio of short term active users to long term active users shortens since weekly active users are frequently also daily active users producing a DAU/WAU closer to 100%. The time periods used are up to you based on how frequently you expect Visitors to use your product.

Note: The Stickiness Dashboard widget calculates DAU/WAU/MAU for all apps by default. PES calculates Stickiness for a single app. Make sure the app configured for the Stickiness widget matches PES when comparing stickiness metrics.

 

Growth

Growth compares the number of active Visitors or Accounts in the current time period with the preceding time period and compounds that growth rate for a year to determine if the product has strong growth and virality. Growth does not look at new Visitors or Accounts, only activity. A Visitor or Account with any event recorded by Pendo is considered active during the time period.

The Growth score and measured Growth rate are both displayed in the PES visualization.

The Growth score calculation consists of two parts, the measured growth rate, and the projected annual growth rate. First, the percent change in total active users between the current and preceding time periods is calculated, the growth rate. Then the projected annual growth rate is calculated to determine the Growth component score. Possible scores range from 0, if there is no growth or negative growth, to 100, if the calculated annual growth rate is 100% or greater.

Projected annual growth above 100%, or more than doubling of the current number of Visitors or Accounts, will not increase the Growth score further. This was done with consideration for most companies experiencing typical annual user growth from 0 - 100%. A product’s user base can grow rapidly, in excess of 100%, when it’s introduced to the marketplace or as companies release internal products to their teams. This aggressive short term growth rate varies by industry and addressable market and can’t be accurately described by a single growth metric. Capping Growth at 100 prevents a large growth rate from diluting the other PES components for rapidly growing products. The raw projected annual growth percentage is displayed next to the Growth score and will display values below 0% and above 100%.

Time Period Power is the multiplier that compounds the current growth rate into the projected annual growth rate. Growth rates continue to increase exponentially, like interest in a bank account, resulting in large changes annually.

Time Period Power values are

• 7 days - 52

• 30 days - 12

• 90 days - 4

• 180 days - 2

• Custom Date Range - (365/(Number of Days))

 

Manage PES

PES is managed from the PES page in the Behavior section. PES can be configured individually for each app and filtered by Segment, time period, and app. The configured PES for an app is visible to everyone in the subscription. Filters are applied to a user's unique PES view. Only Pendo Admin users can configure PES. Anyone can apply filters on the PES page.

Core Events are required to calculate the Adoption score. Access to Core Events setup is available on the PES page for Pendo Admin users.

 

Configure PES

1. Navigate to PES in the Behavior section.

 

2. Click Manage PES in the top right to open the Configure Product Engagement Score overlay.

 

3. Select App on a multi-app subscription.

 

4. Change dropdown selections for Adoption, Stickiness, and Growth. A preview of the 30 day PES updates as changes are made.

 

5. Select a different app and make additional changes if necessary.

6. Save & Exit to apply all changes to PES for the subscription. Save & Exit is not available if no changes are made to PES configuration.

 

Adoption

Adoption measures the proportion of users engaging with Core Events during the time period. Core Events are required to calculate an Adoption score for an app.

Adoption can be configured to measure usage for

• Individual Visitors or anyone in an Account
• Any Core Event or every Core Event

Optimal selections depend on your product, usage patterns, and Core Events selections. A SaaS app may expect individual Visitors to access the app and interact with one or two Core Events while a CRM platform may expect every Core Event to be used across an Account as individual contributors perform their tasks.

Note: Selecting Visitor and All Core Events will likely result in an Adoption score of 0. A single Visitor typically does not interact with every Core Event in an app. You know if you expect your Visitors to use all Core Events during their normal usage but if you unexpectedly have a very low Adoption score this may be the cause.

 

Stickiness

Stickiness measures the how frequently users return to the app as a ratio of short term active users to long term active users. Select the pairing that most closely represents your ideal usage. 

Possible Combinations

• Daily Active Users/Weekly Active Users - Active user base. When users use the app they use it frequently but maybe not habitually. Typically produces a moderate Stickiness score as unique daily Visitors are compared with a smaller sample of unique weekly Visitors.
• Daily Active Users/Monthly Active Users - Highly active users. Users are expected to use the app habitually every day. Typically produces a lower stickiness score as unique daily Visitors are compared with all unique Visitors for the month.
• Weekly Active Users/Monthly Active Users - Involved user base, when users use the app they do infrequent tasks or maintenance but aren't expected to use it daily. Typically produces a high stickiness score due to relatively small number of weeks in a month and the longer time window available for a user to qualify as a WAU.

Exclude Weekends removes the data collected on weekends from the overall average calculation. Business apps with most usage between Monday and Friday will likely see improved metrics after removing days with no expected usage from calculations. This generally improves stickiness for DAU ratios as weekend days with no daily usage are removed and no longer pull down average usage on week days. Ratios with WAU may decrease slightly as users who only use the app on weekends are removed.

Note: Weekly/weekly is not an available combination. It's the same number and would produce a Stickiness score of 100 every time. That's cheating. Make your users want to come back for more! Earn that stickiness.

 

Growth

Growth compares the difference in the total number of active Visitors or Accounts between time periods normalized as a one year growth rate.

• Active Visitor growth is likely the preferred choice if individual Visitors using the app regularly make up the majority of the user base.
• Active Account growth may be preferred if some Visitors use the app infrequently or are only involved briefly, for example for demos or implementation, but Account health and retention is not impacted by the churn of these personas. Measuring Growth by Account will prevent expected user inactivity from negatively impacting Growth.

PES Dashboard Widget

The PES Dashboard widget lets any Pendo user view PES on the Dashboard page. The widget can be added multiple times with unique names, date ranges, Segments, and apps set for each version of the Widget. With these filters you can create a dashboard showing the PES for different segments like account tiers, custom date ranges to assess PES quarter over quarter, or monitor PES changes before and after major releases or events, each broken down by app. This gives you a powerful view of product engagement across your apps and user groups which you can use to monitor and investigate your best and worst performing areas.

Pendo Admin users can access Manage PES from the widget. The PES configuration set in Manage PES is applied at the app-level. If it's changed from the widget, it will change the PES for that app for everyone in the subscription everywhere PES is visible.

Add and Edit the PES Widget

On the Dashboard Page, click + Add Widget to add a new widget to the current Dashboard.

1. Hover and select the Product Engagement Score widget.

 

2. Edit widget filters. Each widget can have unique settings. These can be changed later by clicking Edit Widget in the ellipse menu on the widget.

• Name - Display name shown on the Dashboard
• Date Range - Date ranges in previous 7/30/90/180/Custom day increments from today used to filter activity data and calculate component scores
• Segment - Segment used to filter Pendo usage data
• App - App used for usage data and PES configuration

3. Save to add the widget to your Dashboard.

 

Frequently Asked Questions

Do Track Events count towards any of my PES metrics?

Track Events can count towards the Adoption score when Track Events are setup as Core Events. By default, Track Events are not counted as activity for the Stickiness and Growth scores. Some customers want Track Events to count as activity in the application. The Count Track Events Received for a Visitor as Time in the Application setting in Subscription settings can be enabled by a Pendo Admin user and Track Events will be counted as activity when calculating the Stickiness and Growth scores.