When running any type of test to help you determine what’s working and what’s not for your Internet marketing campaign, it would be ideal if – after all the data is gathered – you could say “A clearly beat B, let’s stick with A.” However, we don’t all live in an ideal world. Sometimes, the data isn’t as helpful and you’re left with non-conclusive results.
This doesn’t mean you have bad data or that your data is wrong, it just means that the date you’ve collected doesn’t provide the information necessary to pick one clear, winning option over another. In the world of website optimization, this might happen after an A/B test or a multivariate test. If you receive a non-conclusive result at the end – it means that nothing outperformed anything – all configurations are performing equally without significant favor going in any direction.
These results can be frustrating as we all hope for clear statistics to support decisions. If you’ve had this feeling that changing “hey there” to “hi there” on your landing page would triple conversions – and it simply didn’t – you’re back at square one to come up with a new solution.
However, you can flip that logic and decide that non-conclusive results release you from obsessing on factors that are apparently not as important as you thought they were. “Hi” versus “hey” may have cost your team 47 working hours. Now you have clear-cut proof that you shouldn’t spend one more hour on that discussion. Move on. Find something else to change and test. You’re welcome.
