When conducting a scientific experiment, defining a control group is required in order to properly benchmark the impact of a certain treatment against another. In such experiments, the control group represents the group of subjects that are set aside and do not receive the tested treatment, thus allowing researchers to minimizing the effect of all variables except the impact of the independent variable in the treatment.
In website A/B/n tests, several variations of a certain experience can be tested among a website’s traffic, dividing visitors and allocating them, usually evenly, among each variation. In such cases, researchers can quickly discern the impact of each variation relative to the others and determine which drove the best results. In the setup described, each variation represents a change or shift from a certain status-quo, leaving it unclear whether any change performs better than no change at all. In order to measure whether any change at all is better than the status-quo, it is required to measure the performance of ‘no-change’ in the experience, and hence setting up a control group is required. The control group will represent the portion of visitors designated not to receive any new variation, representing the ‘no-change’ baseline and effectively allowing researchers to gauge whether any change is required and, if so, which change performs best.
In Dynamic Yield, establishing a control group is essential for measuring uplift generated by any Dynamic Yield Experience treatment, as the control group represents visitors who are randomly chosen and either receive no treatment or receive a random treatment – not driven by any optimization or personalization algorithm – depending on the behavior chosen in the control group settings.
Read more:
