Client-side testing and personalization explained

This post is one part of a multi-part series on the different technical approaches to A/B testing and personalization. In this article, I will provide an in-depth analysis of the most important considerations when implementing experiments and campaigns on the client-side.

How client-side testing and personalization works

The client-side approach is by far the most common solution employed in the market today. With this approach, all tests and experiences are rendered from the client-side browser using Javascript. 

So, what is client-side rendering?

When a vendor asks you to place their Javascript tag(s) into the HTML of all your site pages, Javascript-based testing and personalization SDKs can then be injected onto your site. Along with their core SDKs, vendors embed code representations of the campaigns that you define from their UIs. Once the tags are on your site, the vendor can execute all necessary logic to test and personalize the content on your pages, using client-side Javascript.

The page modification technique is the same across all vendors – a piece of code in the SDK performs an action based on the desired use case. 

A few client-side rendering examples:

• Identifies one or more elements on the page via DOM selectors, then executes Javascript code to modify such element(s). Examples: hide, remove, move, duplicate, change attributes, change CSS styling, and so on

• Identifies one or more elements on the page via DOM selectors, then executes Javascript code to replace the element(s) with new content

• Identifies one or more elements on the page via DOM selectors, then executes Javascript code to insert new content before or after the element(s)

• Injects global CSS styles onto the page, changing the styling of elements on the page

• Injects other Javascript code onto the page, which can be used for a variety of open-ended use cases (e.g. inserting a 3rd-party library, making an AJAX request, calling an API, etc.)

Configuration options for client-side testing and personalization

In order to provide more exact control over your site loading and rendering flow, most vendors also offer a variety of configurations to more granularly optimize the loading and execution behaviors of their Javascript SDKs. 

Some of those include:

1) Optimizing the network delivery of the Javascript SDK

CDNs: Virtually all client-side testing and personalization vendors serve their Javascript files via content-delivery networks (CDNs). CDNs optimize the network delivery of the Javascript files by mirroring the files in different server locations and then optimizing the delivery routes taken when a user makes the request. Some vendors also allow you to host their files on your own selected CDN, providing your IT teams with finer control over delivery and loading.

DNS prefetching: Because most vendors serve their Javascript files from a different hostname than your website, when the browser initiates the request for the Javascript SDK files, it can take a small amount of extra time to resolve the Domain Name Server (DNS). To expedite the DNS resolution process, some vendors recommend placing prefetching tags before their Javascript tags.

2) Optimizing the load and render of the Javascript SDK

Tag placement: To provide the best experience for your users, it is almost always recommended that you place a vendor’s Javascript tags as high in the HTML markup as possible, ideally, in the head section. This allows the browser to quickly load the tags, which minimizes the delay between when a page’s HTML content loads and the vendor’s SDK makes its modifications to the page. Failure to minimize this delay will result in a so-called “flicker” effect – when a user sees the server-rendered content, typically in the milliseconds (or, in bad cases, seconds), before the vendor’s SDK changes the content. This flicker effect is universally considered detrimental to the user experience, as it often confuses and frustrates visitors.

Synchronous load: By default, most vendors recommend you set their Javascript tags to load synchronously to avoid flicker effects. Synchronously-loaded Javascript blocks the browser rendering of HTML content at the location of the tag in the HTML markup. At first glance, this may seem undesirable from a page load perspective, but page load concerns are not a serious concern for major vendors. Typically a variety of techniques including CDN usage, small file footprints, code minification, safely-designed code to prevent breaking of the page, etc. are used to ensure very fast, reliable file-loading. Furthermore, as mentioned above, most UX designers see the trivial load time differences of optimized synchronously-loaded Javascript as a worthy tradeoff for minimizing flicker effects.

Asynchronous load: You can also set a vendor’s script tags to load asynchronously. Asynchronous loading stops the browser from render-blocking when it loads the vendor’s scripts. In other words, the browser will render HTML content in parallel with its load of the vendor’s Javascript files. The benefit of asynchronous loading is that there will be zero effect on page load times, but this comes with the tradeoff of increased risk of flicker. To mitigate this, vendors can utilize certain styling and rendering tricks to prevent the appearance of an element until after the vendor’s SDK modifies it. For example, you can use CSS to hide the element initially, then unhide the element once the vendor’s SDK is done loading. As another example, you can typically avoid flicker entirely if your testing and personalization use cases are not above the fold, since, on most connections, the vendor’s Javascript SDK is loaded by the time a user gets below the fold.

Which one: synchronous vs. asynchronous? At the end of the day, there is no universally right or wrong answer when it comes to synchronous versus asynchronous loading – the appropriate choice depends on your business priorities. If avoiding flicker is the highest priority, then synchronous loading should be used. If minimizing page load times is the highest priority, then asynchronous loading should be used. That said, the impacts on load time and other risks associated with synchronous loading are trivial enough when using a major vendor that synchronous loading is an appropriate approach for most websites. 

As a cheat sheet for this discussion, refer to the following table:

Caching: Most vendors utilize smart caching techniques on their Javascript SDK files, both at the server level and the browser level. Caching allows the vendor’s SDK to effectively be downloaded only on the first page view since the SDK files will be cached locally on the browser thereafter. However, keep in mind that cache expiration times are typically set to a low number of minutes, allowing you to make changes to your testing and personalization campaigns and push those changes to users in near real-time. Once the cache expiration time expires, the user’s browser will have to re-download the vendor SDK on the next page request. That said, for the vast majority of websites, session times aren’t long enough for most users for this to be a noticeable issue.

3) Reducing the file size of the Javascript SDK

Vendors also utilize a variety of techniques to reduce the file sizes of their scripts, through minification, removal of non-essential embedded libraries, dynamically removing old or unused experiments and personalization campaigns, and more.

4) Utilizing lazy loading techniques 

In addition to purely reducing file sizes, some vendors implement lazy loading techniques on your testing and personalization campaigns so that only the absolutely necessary campaigns are loaded with the SDK. In practice, the necessary campaigns are typically the ones that apply above-the-fold on the most popular pages of your site. Other campaigns, such as those that execute below-the-fold or are triggered by specific events, are marked for lazy load. This usage of lazy loading reduces file sizes, delaying downloads to later network requests with smaller payloads.

5) Configuring the Javascript SDK to integrate with sites utilizing client-side rendering

Client-side rendering has become increasingly popular over the last few years, led by libraries and frameworks such as React and Angular. Since most A/B testing and omnichannel personalization platforms were developed before the rise of client-side rendering, most vendors still do not have great support for client-side rendered sites (as of early 2020). 

For the few vendors that do provide “support,” their approaches would be more accurately described as workarounds rather than robust solutions, and typically come with many feature caveats. 

Common approaches include:

• Extended implementations that promote the firing of special tracking functions on the vendor’s SDK as the context of the page changes, allowing the vendor to keep track of the site lifecycle. This approach allows developers to have full flexibility over when the vendor’s SDK takes actions, at the cost of increased implementation effort and ongoing maintenance code.

• Usage of client-side DOM monitoring functions such as the Mutation Observer, allowing the vendor to keep track of page changes automatically without much extra implementation effort required from your team. However, usage of such functions is typically not robust; DOM monitoring may be unreliable, especially if your site has a more complicated rendering flow. 

Client-side rendering SEO impact

Since SEO traffic is one of the most important sources of visitors for most websites, we would be remiss to not discuss the potential risks involved when utilizing dynamic Java-Script-generated content in A/B testing and optimization. In fact, it’s such an important topic that we wrote an entire blog post about the client-side rendering SEO impact, outlining a series of experiments we conducted which ultimately found Google does crawl and index dynamic, JavaScript-based content.

Is there a difference in client-side performance between vendors?

When it comes to comparing the technical performance of client-side testing and personalization platforms, there is very little difference between them. All client-side approaches use Javascript to perform their functions, using tags inserted into the page as described in the previous sections. 

Furthermore, the industry has matured to the point that all major vendors in the space offer similar optimization and configuration options when it comes to the Javascript tags. The only main differences in technical implementation across vendors tend to be either in data tracking and collection methods or in how integration is handled with client-side rendering frameworks. Therefore, when comparing vendors, it is much more important to compare the core features of the platforms themselves, as well as their service models, above performance.

Next up: server-side testing and personalization

Based on our experience working with clients over the past 8 years, these are the most important technical aspects to consider for client-side testing and personalization. In the next part of this blog series, we will conduct a similar analysis on server-side testing and personalization.