Set up Data Sync with Amazon S3 using IAM roles

Last updated: December 02, 2024 16:03

Pendo Data Sync provides the ability to export Pendo data (excluding Pendo Feedback data) to your Amazon S3 account. This article describes how to set up Data Sync with Amazon S3 using IAM roles. Alternatively, you can set up Data Sync with Amazon S3 using user access keys.

Prerequisites

Data Sync is a paid feature. Contact your Pendo representative for access.
Data sync IAM role setup must be manually enabled by your Pendo representative.
You must be a subscription admin in Pendo to set up a cloud storage destination as part of this process.
You must have sufficient permissions to create an IAM role in your AWS account.

Setup overview

Data Sync with Amazon S3 is set up in Pendo and configured in your S3 account. Setting up Data sync with Amazon S3 using IAM roles involves the following steps:

Choose a provider in Pendo.
Create an IAM role.
Create an S3 bucket.
Create a folder in your S3 bucket.
Configure the bucket policy to grant write permissions on the bucket to the newly created IAM role.
Enter your role ARN and bucket URL in Pendo.

Step 1. Choose a provider in Pendo

In Pendo, go to Settings > Data Sync.
Select Connect to destination.
Hover over the Amazon Simple Storage Service (Amazon S3) tile in the panel that slides out on the right.
Select Choose destination. The panel then summarizes the next steps for you.
Take note of the Pendo User ARN and External ID values. These are needed to create an IAM role in Step 2.

Step 2. Create an IAM role

This setup involves creating an IAM role for Pendo to adopt, using the ARN and external ID generated for your subscription and provided in Step 1. The role must be given write permissions to the bucket you create in Step 3.

Go to the AWS Management Console.
Select Services > IAM in the top-left corner.
Select Roles in the left-side menu under Access Management.
Select Create role at the top right of the page.
Under Trusted entity type, select Custom trust policy.

Under Custom trust policy, copy the following JSON snippet and paste it into the text editor. You can overwrite the default JSON snippet provided.

Replace PENDO_USER_ARN with the Pendo user ARN value provided in Step 1.

Replace EXTERNAL_ID with the external ID value provided in Step 1.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "PENDO_USER_ARN"
                ]
            },
            "Action": [
                "sts:AssumeRole"
            ],
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "EXTERNAL_ID"
                }
            }
        }
    ]
}

Select Next in the bottom-right corner of the page.
You don't need to add any permissions to the Add permissions page, so select Next in the bottom-right corner of the page.
Name and review your role.
- Under Role details, give the role the following name: pendo_datasync. The role name must be an exact match to pendo_datasync.
- Review the trust policy and check that it matches the JSON snippet you pasted earlier in this procedure.
- Ensure the list of permissions policies is empty.
- Optionally, add a tag. We don't require a tag but it might be meaningful for your organization.
Select Create role in the bottom-right corner of the page.
Take note of the role's ARN, which can be found in the IAM dashboard by going to Roles > Role Name > Summary.

Step 3. Create an S3 bucket

Pendo data is copied to an S3 bucket. To create an S3 bucket:

Go to the AWS Management Console.
Open the Amazon S3 Console and choose Buckets.
Choose Create bucket at the top right of the page.
Create a meaningful name for your S3 bucket.
Follow the workflow to set the appropriate settings for your S3 bucket.
Select Create bucket at the bottom of the form.

Step 4. Create a folder in your S3 bucket

While in the bucket you just created for Step 3:

Select Create folder.
Create a meaningful name for your folder.
Follow the workflow to set the appropriate settings for your S3 folder.
Select Create folder at the bottom of the form.

Step 5. Configure the bucket policy

Configure your bucket to allow PutObject permissions to anyone in the role you created in Step 2. This grants write permissions on the bucket to the newly created IAM role.

Go to the bucket you created in Step 3. You can find this in S3 > Buckets > Bucket Name.
Select the Permissions tab, scroll down to Bucket policy, and select Edit.

Copy and paste the following JSON snippet into the editor under Policy.

Replace ROLE_ARN with the ARN of the role you created in Step 2. You can find this in the Summary section of your role created in IAM Dashboard > Roles > Role Name.

Replace BUCKET_ARN with the bucket ARN shown above the editor on this page.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "ROLE_ARN"
            },
            "Action": "s3:PutObject",
            "Resource": [
                "BUCKET_ARN/*",
                "BUCKET_ARN"
            ]
        }
    ]
}

Select Save changes in the bottom-right corner of the page.

Step 6. Enter your role ARN and bucket URL in Pendo

Provide Pendo with the name of the bucket you created in Step 3 and the ARN for the role you created in Step 2. This allows Pendo to adopt the role you provide, which allows us to write to your bucket.

In Pendo, enter your Bucket URL. You can find the bucket URL in the Properties tab of your Data Sync bucket created in S3 > Buckets > Bucket Name > Folder Name. Copy the URL in the S3 URI column of the Folder overview table.
Under Role ARN, enter the ARN of the role you created in Step 2. You can find this on the Summary section of your role in IAM Dashboard > Roles > Role Name.
Select Validate destination to test the credentials you entered.
When successfully validated, select Create destination.

Set up destination.png

Updating your cloud storage destination

You have the option to move your cloud storage destination from one S3 bucket to another, or to change your cloud storage service to another provider, such as Google Cloud Storage (GCS) or Microsoft Azure.

Update your S3 bucket

If you want to change from one S3 bucket to another existing S3 bucket, update the S3 URI field in Pendo:

In Pendo, go to Settings > Data Sync.
Select Manage destination in the top-right corner of the page.
Replace the entry in the S3 URI field with the name of the new S3 bucket that you'd like to use for Data Sync.
Select Update destination.

Change your cloud storage provider

If you want to change your cloud storage destination from Amazon S3 to an alternative service, such as Google Cloud Storage (GCS) or Microsoft Azure, you can delete the existing destination and create a new one.

To delete a destination:

In Pendo, go to Settings > Data Sync.
Select Manage destination in the top-right corner of the page.
Select Delete destination.
Confirm that you want to delete the destination.

After you delete a destination, any recurring reports and ongoing metadata updates are automatically turned off.

You can then set up a different cloud storage destination by following the instructions outlined in the following article:

Alternatively, you can set up Data Sync with a Pendo-hosted GCS destination. For more information, contact your Pendo account team.