Putting primary audiences into practice, a beginner’s guide

The time has come to pull everything you’ve learned from earlier articles in the hub and put your primary audience-based personalization framework into action, allowing you to regularly collect meaningful learnings and use them to inform your efforts over the long-term.

This guide will act as a recap of the two core types of audience-based tests and then outline a step-by-step process for beginners to run a few initial personalization campaigns. These start with A/B testing variations against a control to see which performs best for each of your primary audiences – the findings of which serve as a jumping-off point for further optimization of the digital experience for each segment.

The two core types of audience-based personalization tests

Understanding your primary audiences and yielding valuable insights for ongoing personalization success can be achieved by running two different types of experiments, which should be used in tandem for the best results.

Those are:

Product recommendations tests

One of the most straightforward types of personalization, and a campaign type that should be seen as an evergreen element on the site, product recommendations leverage attributes in the data feed, mine user interactions, and aggregate trends in behavior to present custom-tailored content or products.

A range of global, contextual, and personalized product recommendation strategies can be used throughout the funnel to expedite a visitor’s overall time-to-purchase. For example, on the homepage to accelerate product discovery and improve the site’s navigability. Or, further down on product detail pages (PDPs) and within the shopping cart to help cross-sell and drive conversions without users having to leave the page in high-intent moments (which are typically areas responsible for the most revenue generated).

Everything from the recommendation strategies deployed to the look and feel of the widget (including any messaging) all the way down to site or page location should not only be tested but regularly optimized for the best results per primary audience.

Content tests

A website is chock full of content elements like images, copy, calls-to-action (CTAs), promotional modules, and more that are typically static in nature but primed for personalization based on the makeup of a particular primary audience.

Examples of content campaigns include tailoring the homepage hero banners, personalizing the site navigation (on desktop and mobile), surfacing time-sensitive promotions, incorporating social proof messaging on PDPs or in other locations, and even making copy changes to inactive browser tabs.

The sky is the limit in terms of the various web components available for testing. However, teams should always keep impact, effort, and scale in mind when prioritizing ideas.

Step-by-step to launching your first two primary-audience-based campaigns

Below is a streamlined process to follow that will help you establish your primary audiences and use them to launch and analyze your first few personalization campaigns, one product recommendations test and one content test:

1. Create a set of primary audiences
2. Conduct a product recommendations A/B test
3. Conduct a content A/B test
4. Analyze test results for each of your primary audiences
5. Launch new tests based on your hypotheses

Let’s begin.

1. Create a set of primary audiences

Pre-requisite reading: Choosing a set of primary audiences.

Your primary audiences are the fundamental segments that inform your rooted personalization strategy. They simplify your personalization program without sacrificing its potency, and adopting them should make scaling your efforts easier.

Intended to be evergreen and enduring segments, even as your business evolves, it is, therefore, crucial to be deliberate and take the necessary time to carefully identify these core groups.

Their selection should start with choosing a primary website KPI to optimize for based on your industry and business need. Then, look into your data to identify a consistent behavior for those who have completed the KPI event, using that behavior as a segmentation principle to break down a set of audiences into 2-4 distinct groups. These groups should encompass 100% of your site traffic, with each one covering at least 10%.

The goal of this process is to establish a standardized universal lens to examine your user base and the results of your tests. With a fundamental understanding of your audience, you can begin to leverage experimentation to grow your knowledge, applying newfound insights to future tests that hone in on your primary KPI for each audience, further enhancing the customer experience.

2. Conduct a product recommendations A/B test

Pre-requisite reading: Getting Started with Product Recommendations Testing and Personalization.

A great place to get comfortable implementing and analyzing a foundational personalization tactic is to add a recommendation widget to your PDPs and test two different algorithms against a control to compare results.

It’s important to keep in mind that no single strategy will suit all visitors, as some may visit your site directly and browse several times a week, while others may be first-time visitors who have arrived via a search engine query. A strong product recommendation engine assesses the available data for a specific visitor and deploys the most appropriate strategy based on contextual clues (current page, referring URL, etc.), user behavior, and more to drive action.

On to running our test, which includes:

1. Choosing two algorithms to test against each other (as well as a control)
• While winners can vary greatly according to the brand, user base, and audience, “Similarity” and “Viewed Together” are some basic initial algorithms to try on the PDP
2. Letting the campaign run for at least two weeks to reach statistical significance, with equal traffic being distributed to each variation and the control before using an 80% threshold for gauging Probability to Be Best (P2BB)
3. Analyzing the A/B test results for each segment to see which audience prefers which algorithm for use in future recommendations tests

When running future recommendation tests for each segment within your primary audiences, keep the page type and stage of the customer journey in mind. For example, if your segmentation principle is purchase intent (measured by pageviews), it makes sense for visitors with low data availability (low intent) to receive generic ranking recommendations such as “Most Popular.” Whereas returning users with more page view data should be served contextually-driven product recommendations.

You can learn more about techniques, strategies, and best practices in this product recommendations guide.

3. Conduct a content A/B test

Social proof messaging requires little effort from an implementation perspective but can have a massive impact on KPIs, which is why we recommend it as a first content test.

The concept of social proof itself has to do with the influence of group thinking or activity on the behavior of others. For example, positive customer reviews and ratings instill a sense of trust in purchase decisions, highlighting product demand validates the visitor’s own thinking to buy it as well, and showcasing the remaining stock of a particular product can create urgency.

On to running our test, which includes:

1. Choosing a quantitative factor to trigger the social proof content test
   • This helps to tell the story of a given product’s perceived value at a particular moment in time, like the number of recent purchases or page views, or even remaining inventory
   • The number of page views or purchases for a given product are easily understood by site visitors and are easy to incorporate, so let’s start there
2. Determining a threshold for your quantitative factor that triggers the social proof campaign
   • This should mean something to a consumer, as a single product view or purchase in a given day is likely not going to leave an impression.
   • The threshold should qualify enough products and data to draw a conclusion – a single product with 10K product views will produce extremely limited results
3. Selecting a period of time for your threshold
   • What makes the most sense for your business, products, and potential buyers? A day, week, or month?
   • We recommend starting with round numbers like 200 views in a day or 1000 views in a week
4. Letting the campaign run for at least two weeks to reach statistical significance, following the same rules as the product recommendations test to gauge the best variation
5. Analyzing the results for each segment to note the impact of the social proof content on conversions (we’ll use purchases per user)

Similar to the product recommendations campaign, your future social proof messaging should be geared toward the motivations and intentions of your primary audiences. The content test’s messaging and quantitative information you show (like products sold or pages viewed) will differ for each audience. For instance, new users with little-known intent should see product reviews from happy customers to build interest, while a returning user with high intent may need only a little push to convert and should see low-stock notifications that drive urgency and purchase actions.

You’ll want to analyze the results for all of your primary audience groups to see how the content performed for each segment. This will make it easier for you to optimize quickly after seeing winning data.

4. Analyze test results for each of your primary audiences

Additional reading: How to Analyze Personalization Campaigns by Primary Audiences

After launching your initial campaigns, analyze the results through the lens of your primary audiences to determine the winning variation for each segment.

Ask yourself:

• Which variation performed better based on the primary metric?
• If the result was negative or produced a downturn, could the placement, content, CTA/lack of CTA or any other element have been a factor?
• Why might a variation have resonated (or not) with the primary audience based on what you already know about them?
• What messaging might resonate better with the primary audiences?
• What can you test next to gain more clarity about each audience group?

Use these considerations to solidify an audience insight and inform the hypothesis that you’ll put to the test in your next experiments.

5. Launch new personalization tests based on your hypotheses

Once you have an insight from the initial campaigns for each audience segment, you have an elevated baseline for launching future tests.

For example, let’s say you find that low-intent users (one of your primary audiences) preferred similar products in the PDP recommendations. This result may lead you to believe they’re more interested in learning about a particular style of products over what’s been viewed together with it. A natural next step would be to refine your use of the similarity algorithm for the low-intent audience segment within the PDP product recommendations widget.

And so the campaign ideation process begins again, with new insights applied as second iterations of your recommendations and content tests – this time for each of your primary audiences.

A repeatable process for ongoing success

There are multiple tactics and strategies within both core types of audience-based personalization tests beyond PDP recommendations and social proof. When approaching a new one, simply apply the same step-by-step process:

1. Choose the personalization campaign
2. Formulate a hypothesis to test for your entire audience
3. Run an A/B test with a control and two variations
4. Let the test run for two weeks (at least) to reach statistical significance, with equal traffic distributed to each variation and the control before using an 80% threshold for gauging Probability to Be Best (P2BB)
5. Analyze the results through the lens of each of your primary audiences
6. Institutionalize and compile the insights from each test

Personalization is never truly done, as you’re always capable of learning more about your audience. Implementing the playbook above will allow you to begin this rewarding process, allowing you to not only improve their customer experience but also continually optimize your results.