Product Engagement Score (PES) Overview

Overview

Product Engagement Score (PES) is a single number that represents the combined impact of feature adoption, return user stickiness, and new user growth. PES is a quantitative value that summarizes your existing usage data without requiring any additional data collection or user participation. Similar to NPS, which is a subjective measurement of user sentiment collected with a survey, PES is an objective measurement of user behavior. Track PES over time and compare PES for different Segments to discover who has the most engagement with your product and why. PES provides a benchmark of your product engagement which can be used to measure the impact product decisions have on your users' behavior.

PES uses Core Events, Visitor, and Account data to calculate the percentile score for each component and combines them into a single score from 0 - 100. Your end users don't need to do anything other than use your product to contribute to the score. Product usage data in Pendo is used to calculate each component of the PES.

 

Requirements

• Pendo Admin users can configure PES
• Any Pendo user can view PES
• Core Events are required for the Adoption component of PES

Using PES

Product Engagement Score (PES) is a new metric that Pendo introduced to the SaaS business landscape. PES provides a singular, quantitative, business metric to measure the overall engagement of a product with the goal of quickly diagnosing how a product is performing. Historically, product leaders have had to rely on qualitative metrics like NPS to get a feel for customer sentiment. NPS uses a simple recommendation score to determine the virality of a product or service. It has always depended on polling customers and has never provided an accurate look at actual user behavior. While important, NPS cannot be the only business metric for a product. PES is intended to fill that quantitative gap and partner with NPS, by truly examining user behavior and quantifying how users are engaging with a product.

PES is a high-level way to assess the health and success of a product through quantitative engagement data, as a counterpart to the sentiment-driven and highly subjective NPS. It is applicable as a business-level KPI for a product and, when analyzed further with Segments and date filters, can be used to gain a more granular understanding of the engagement of user groups over the life of your product. PES partners with other Pendo tools, like Core Events, Goals, and NPS, to provide a comprehensive understanding of user behavior. These insights can then be used to drive product development, in-app messaging, and outreach campaigns with the ultimate goal of increasing product engagement and user satisfaction indicated by improved metrics, or as we like to say, delivering great products that users love.

PES is visible to all Pendo users on the PES page in the Behavior section. Filters for segment, date range, account, and app are available for all users to analyze PES.

Tip: Use a Custom Date Range filter to measure PES quarterly and report your app's PES with your NPS.

Understanding the Data

The final PES is the average of each component score. Possible scores for all components range from 0 - 100. Any percentages above 100 are adjusted to a score of 100 and percentages below 0 are adjusted to a score of 0.

The PES chart shows changes in the overall PES score and each component compared to the previous time period. Trending PES shows the score change with trend direction indicated by the arrow and the score for the previous time period in parenthesis.

 

Adoption

Adoption looks at the percentage of users who are using your product effectively as indicated by your Core Events. Adoption can look at Visitor or Account-level usage for any or all Core Events versus all active Visitors or Accounts in the selected time period. A Visitor or Account with any event recorded by Pendo is considered active during the time period.

Core Event selection may significantly impact the Adoption score. If all Core Events are weighted towards specific groups of users or focus on features with limited use, like first-time setup, it may lower adoption. If Core Events include site navigation or features that a user must engage with, like login, it may increase adoption. A distributed selection of Core Events that includes high-value features for all user groups will result in a balanced Adoption score. There is nothing wrong with Core Event selection deliberately focusing on certain user groups and feature sets for the core of your user base. Just understand how that will shift the Adoption score and account for it in your analysis. Consider calculating adoption at the Account level if Core Events aren't applicable to all users.

 

Adoption Drilldown

Click on the Adoption component to open the Adoption drilldown. Adoption drilldown shows the filters for your current PES score and the definition for the Adoption component as configured for the application. Drilldown analytics include trending Adoption for the date range and a breakdown of adoption for each core event.

 

The Trending Adoption chart shows the score as it would have been reported on the date at the bottom of the chart for the number of days in the selected date range. Hover over a point on the chart to see the score and date range used to plot that point. The score is plotted in increments of one day, one week, or one month depending on the size of the date range.

 

The Core Event Adoption chart shows how many active visitors or accounts have used each core event. The total number is displayed with the percentage of active visitors or accounts who have used the core event. This breakdown allows you to see the distribution of usage across your core events and find if a single core event is carrying your Adoption score. Click on a core event to open the Details page for that tagged feature or track event and see additional usage data.

 

• If the Adoption score is based on using all core events, this chart shows which core events aren't being used and are lowering the Adoption score.
• If the Adoption score is based on using any core events, the percent of active visitors or accounts may not match any single core events percentage. The combined adoption score could be higher if active visitors or accounts use different core events.

Stickiness

Stickiness looks at the average percentage of users or accounts who are high-frequency return users in Daily/Weekly/Monthly active user metrics. The Stickiness component finds the ratio of average daily or weekly active visitors or accounts to average weekly or monthly active visitors or accounts in the selected time period. A Visitor or Account with any event recorded by Pendo is considered active during the time period. The DAU/WAU/MAU values use the same calculation and data as the Stickiness Dashboard widget.

Active user metrics measure unique users who used your product during a time period. A user who uses the product today is a daily active user and also a weekly active user. A user who used the product this week but not today is a weekly active user. In this example, if we look at DAU/WAU for today, it's 50%. There is one unique active user today and two unique active users this week. This calculation can be processed for every day and averaged to produce the average DAU/WAU for a larger time period. If users are engaging with a product and finding value, they will return frequently and the ratio of short term active users to long term active users shortens since weekly active users are frequently also daily active users producing a DAU/WAU closer to 100%. The time periods used are up to you based on how frequently you expect Visitors to use your product.

Note: The Stickiness Dashboard widget calculates DAU/WAU/MAU for all apps by default. PES calculates Stickiness for a single app. Make sure the app configured for the Stickiness widget matches PES when comparing stickiness metrics.

Stickiness Drilldown

Click on the Stickiness component to open the Stickiness drilldown. Stickiness drilldown shows the filters for your current PES score and the definition for the Stickiness component as configured for the application. Drilldown analytics include trending Stickiness and a breakdown of DAU, WAU, and MAU for the date range.

 

The Trending Stickiness chart shows stickiness over time with the average for the entire date range. Hover over a point on the chart to see the raw data used to calculate the percentage for every day or week on the chart depending on the PES configuration. This data matches the visualization in the Stickiness Dashboard widget.

 

The MAU, WAU, DAU breakdown chart shows the number of daily, weekly, monthly users or accounts at each interval during the date range. This shows you the proportion of each user group to the others and how those proportions influence the Stickiness score. If two groups have a similar trend line, that combination would result in a higher Stickiness score and provides insight into the frequency patterns of your users. Hover over a point on the chart to see the raw data for each user group on that day.

 

Growth

Growth compares the number of active Visitors or Accounts in the current time period with the preceding time period and compounds that growth rate for a year to determine if the product has strong growth and virality. Growth does not look at new Visitors or Accounts, only activity. A Visitor or Account with any event recorded by Pendo is considered active during the time period.

The Growth score and measured Growth rate are both displayed in the PES visualization.

The Growth score calculation consists of two parts, the measured growth rate, and the projected annual growth rate. First, the percent change in total active users between the current and preceding time periods is calculated, the growth rate. Then the projected annual growth rate is calculated to determine the Growth component score. Possible scores range from 0, if there is no growth or negative growth, to 100, if the calculated annual growth rate is 100% or greater.

Projected annual growth above 100%, or more than doubling of the current number of Visitors or Accounts, will not increase the Growth score further. This was done with consideration for most companies experiencing typical annual user growth from 0 - 100%. A product’s user base can grow rapidly, in excess of 100%, when it’s introduced to the marketplace or as companies release internal products to their teams. This aggressive short term growth rate varies by industry and addressable market and can’t be accurately described by a single growth metric. Capping Growth at 100 prevents a large growth rate from diluting the other PES components for rapidly growing products. The raw projected annual growth percentage is displayed next to the Growth score and will display values below 0% and above 100%.

Time Period Power is the multiplier that compounds the current growth rate into the projected annual growth rate. Growth rates continue to increase exponentially, like interest in a bank account, resulting in large changes annually.

Time Period Power values are

• 7 days - 52

• 30 days - 12

• 90 days - 4

• 180 days - 2

• Custom Date Range - (365/(Number of Days))

Growth Drilldown

Click on the Growth component to open the Growth drilldown. Growth drilldown shows the filters for your current PES score and the definition for the Growth component as configured for the application. Drilldown analytics include active visitor growth compared to the previous date range and a breakdown of accounts by active visitor growth.

 

The Active Visitors chart shows the number of active visitors for the current date range compared to the previous date range. The chart includes the growth rate between the two time periods.

 

The Accounts by Active Visitor Growth chart shows the number of active visitors and percent change for the top 10 or bottom 10 accounts. This data is a breakdown of the Active Visitors chart and shows which accounts contributed to positive and negative changes in your growth rate. The chart can be sorted by the Visitors or % Change columns. The % Change column indicates the percent change from the previous time period and the number of active visitors in the previous time period. Click on an account ID to go to the Account Details page and see additional data .

 

Trending PES Chart

The Trending PES chart shows your PES scores over the four preceding time periods. The time period is set in the filters at the top of the PES page. The PES score is shown at each interval and a line graph shows the trend of each score over time.

 

Hover over any of the time periods to see a summary of the values for PES and each component score.

 

Click on the score at the bottom of the chart to toggle visibility for that line. This can help focus the visualization to certain scores if you're sharing the chart in a presentation. The hover score summary only shows the visible scores.

 

Manage PES

PES is managed from the PES page in the Behavior section. PES can be configured individually for each app and filtered by Segment, time period, and app. The configured PES for an app is visible to everyone in the subscription. Filters are applied to a user's unique PES view. Only Pendo Admin users can configure PES. Anyone can apply filters on the PES page.

Core Events are required to calculate the Adoption score. Access to Core Events setup is available on the PES page for Pendo Admin users.

 

Configure PES

1. Navigate to PES in the Behavior section.

 

2. Click Manage PES in the top right to open the Configure Product Engagement Score overlay.

 

3. Select App on a multi-app subscription.

 

4. Change dropdown selections for Adoption, Stickiness, and Growth. A preview of the 30 day PES updates as changes are made.

 

5. Select a different app and make additional changes if necessary.

6. Save & Exit to apply all changes to PES for the subscription. Save & Exit is not available if no changes are made to PES configuration.

 

Adoption

Adoption measures the proportion of users engaging with Core Events during the time period. Core Events are required to calculate an Adoption score for an app.

Adoption can be configured to measure usage for

• Individual Visitors or anyone in an Account
• Any Core Event or every Core Event

Optimal selections depend on your product, usage patterns, and Core Events selections. A SaaS app may expect individual Visitors to access the app and interact with one or two Core Events while a CRM platform may expect every Core Event to be used across an Account as individual contributors perform their tasks.

Stickiness

Stickiness measures how frequently users return to the app as a ratio of short term active users to long term active users. Select the pairing that most closely represents your ideal usage. Stickiness can be calculated using active visitors or accounts.

Possible Combinations

Measuring by active users is used when each individual user's activity is important to the success of the business. Your app could be targeted to single users with personal subscriptions or to businesses using a per-seat pricing model where each user should be finding value to justify the cost. Maybe cost isn't a concern and you just want your app to be a valuable tool that all users return to frequently. In any of these scenarios, measuring stickiness by user could be the right choice.

• Daily Active Users/Weekly Active Users - Active user base. When users use the app they use it frequently but maybe not habitually. Typically produces a moderate stickiness percentage as unique daily visitors are compared with a smaller sample of unique weekly visitors.
• Daily Active Users/Monthly Active Users - Highly active users. Users are expected to use the app habitually every day. Typically produces a lower stickiness percentage as unique daily visitors are compared with all unique visitors for the month.
• Weekly Active Users/Monthly Active Users - Involved user base, when users use the app they do infrequent tasks or maintenance but aren't expected to use it daily. Typically produces a high stickiness percentage due to the relatively small number of weeks in a month and the longer time window available for a user to qualify as a WAU.

Measuring by active accounts is used when individual users aren't expected to return to the app frequently but someone in the account should. This may be expected for system monitoring tools, financial apps, or platforms that use automatic alerts to prompt user activity. Users don't have daily tasks in the app but return regularly for maintenance, completing workflows on a schedule, or responding to issues.

• Daily Active Accounts/Weekly Active Accounts - Active customers. The app is part of a frequent workflow, likely belonging to a designated team, that's performed more than once a week. Typically produces a moderate Stickiness percentage as unique daily accounts are compared with a smaller sample of unique weekly accounts.
• Daily Active Accounts/Monthly Active Accounts - Highly active customers. The app is used daily by someone at the account, maybe to complete an important daily workflow or access information. Typically produces a lower stickiness percentage as unique daily accounts are compared with all unique accounts for the month.
• Weekly Active Accounts/Monthly Active Accounts - Habitual customers. Someone at the account is expected to access your app on a weekly basis, possibly to complete a transaction or run a report. Typically produces a high stickiness score due to the relatively small number of weeks in a month and the longer time window available for an account to qualify as a weekly active account.

Exclude Weekends removes the data collected on weekends from the overall average calculation. Business apps with most usage between Monday and Friday will likely see improved metrics after removing days with no expected usage from calculations. This generally improves stickiness for DAU ratios as weekend days with no daily usage are removed and no longer pull down average usage on week days. Ratios with WAU may decrease slightly as users who only use the app on weekends are removed.

Note: Weekly/weekly is not an available combination. It's the same number and would produce a Stickiness score of 100 every time. That's cheating. Make your users want to come back for more! Earn that stickiness.

Growth

Growth compares the difference in the total number of active Visitors or Accounts between time periods normalized as a one year growth rate.

• Active Visitor growth is likely the preferred choice if individual Visitors using the app regularly make up the majority of the user base.
• Active Account growth may be preferred if some Visitors use the app infrequently or are only involved briefly, for example for demos or implementation, but Account health and retention is not impacted by the churn of these personas. Measuring Growth by Account will prevent expected user inactivity from negatively impacting Growth.

 

Frequently Asked Questions

Do Track Events count towards any of my PES metrics?

Track Events can count towards the Adoption score when Track Events are setup as Core Events. By default, Track Events are not counted as activity for the Stickiness and Growth scores. Some customers want Track Events to count as activity in the application. The Count Track Events Received for a Visitor as Time in the Application setting in Subscription settings can be enabled by a Pendo Admin user and Track Events will be counted as activity when calculating the Stickiness and Growth scores.