What Sample Group number mean?

I understand that Pendo is assigning random group numbers from 0 to 99 to visitors, but what these numbers mean? e.g if an user has sample group 35 and one has sample group 98, how I can interpret this data?

0

Comments

13 comments
  • Sample Group is an automatic metadata flag assigned to each unique Visitor at random between 0 and 99.  The effect of this is that each Sample Group contains, roughly, 1% of your total Visitor population.  You can use this flag to establish Test and Train groups for experimentation and analysis within your product.

    1
  • Thank you Greg! 

    To get a better sense of what this means: for example I can create a segment of sample group 98, and this will guarantee that whatever I plan to test I will get to 1% of the total Visitor population. the more sample groups I add the more I control the increase of the % reach of the visitors?

    1
  • You've got it!  I will reiterate there is a roughness to that percentage due to how Visitors are randomly assigned and growing nature of Visitors to your product.  But the percentages will be ~1% per Sample Group.

    1
  • Hello! Is the Sample Group numbering applied across a Pendo Subscription or by the applications you can set up in your subscription?

    0
  • Is the Sample Group number assigned to each user static or dynamic? I'd like to know which (very product-specific) actions users took based on which guide they saw, and the "action" in question can't be tracked via Track Event/Feature Click.

    I'm assuming we can export the data of who's seen which guide and match it with our internal product analytics data on user/account ID, and want to verify this is possible.

    0
  • Sample Group is assigned during a Visitor's first visit and is not touched again.  Since Visitor IDs are shared across all apps within a Subscription (a big reason to keep Visitor IDs consistent across applications), that Sample Group is shared across applications.  However, if you have multiple Subscriptions, then a given Visitor may have different Sample Group values.

    1
  • Greg Nutt (or someone else), could you give more explanation about the Sample Group values? I read through the discussion here and the Sample Groups section and I'm still confused. Does it basically just mean that you can have up to 99 Sample Groups? 

    Maybe it'll help if I explain what I'm trying to do. I would like to A/B test a guide. Should I create these two Segments?

    • Segment A: Sample Group is equal to 1
    • Segment B: Sample Group is equal to 2

    Thanks in advance!

    0
  • Sample Group is a value that is automatically assigned by Pendo when a Visitor is first seen by the system.  The value can be any number between 0 and 99 and is randomly assigned.  So, in theory, if you were were create one segment where Sample Group < 50 and one where Sample Group >= 50, then these two would split your population in half.  Similarly, if you were to create a segment where Sample Group < 20, this would provide you with ~20% of your population.

    The caveat to this is that Sample Groups are assigned randomly by Visitor, not by any other piece of metadata.  So if you were to try and further segment down by, say, a metadata value called "Role", the two segments of equal numbers of Sample Groups would no longer hold statistically even populations.

    There is another consideration based on Population size.  If your Visitor Count is 100k (on the exaggerated high-end), then each Sample Group will hold ~1k Visitors.  However, if your Visitor Count is only 100, then each Sample Group, in theory, would hold only 1 Visitor.  But due to how randomness is applied, this isn't always strictly true, e.g. if you flip a coin 10 times, you may not end up with a true 50/50 split.  But if you did so 10k times, you'll find the odds even out.  So the assumption of even distribution works best the larger the population.

    I hope this helps!

    0
  • Greg Nutt Got it, that makes sense! Thanks for the detailed explanation! 

    0
  • Greg Nutt, I'm so glad to see there is a Sample Group feature. I have a question about leveraging this field:

    How can I have an ongoing guide across the product that will sample, say, 10% of my Visitor population on a rolling basis. I know bucketing Sample Groups (i.e., Group 1 = Sample Groups 1-10, Group 2 = Sample Groups 11-20, etc.) would accomplish the 10%, but how do I calibrate a Guide + Visitor Group so that the Guide would be shown to a different 10% bucket every 2 weeks (for example)? 

    So the implementation would look like this:

    January 1-14: Group 1 receives the Guide

    January 15-29: Group 2 receives the Guide

    January 30-February 12: Group 3 receives the Guide

    ...and so on.

    So in summary, two steps I need answered:

    1. How to bucket Sample Groups?

    2. How to deploy the same Guide iteratively to those buckets on a rolling basis?

    0
  • Hello Abe Barranca - great questions.  Your understanding of bucketing Sample Groups is correct.  Sample Groups 1-10 would constitute ~10% of your population, and so on.  I think the ask, however, is primarily around shifting the Guide's Segment over time.  There is, unfortunately, no way to manage this automatically within Pendo today.  You would need to go into Pendo and manually change your target segment for your guide every 2 weeks to accomplish the scenario you described.

    That said, could you elaborate on your scenario and what you are trying to accomplish?

    0
  • Thanks for the response, Greg Nutt. We want to be able to ask our users if they're interested in joining an ongoing initiative, but don't want to hit everyone at once, ideally. So the idea would have been to roll out the same guide iteratively as I described above. It looks like we'll have to pursue another timing strategy to do so.

    0
  • Ah. That’s an interesting use case. It reminds me of the ‘Even Distribution’ functionality we have for our NPS surveys. But I can’t think of a way to do this with current Segment functionality.

    I would recommend submitting a Feature Request for this functionality.

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post