The 2 Breakthrough Technologies Driving Smarter Product Recommendations

For the past decade, personalization has delivered targeted experiences, found the best variations through testing, and recommended similar products. But it’s not without its limitations.

For example, I recently bought a treadmill from a certain eCommerce behemoth. I don’t need two treadmills (obviously), and yet, my personalized product recommendations are reliably all treadmills—which will probably be the case until I realize I also need new running shoes.

It’s 2024. Can’t we account for situations like this?

To get some answers, I talked to Udi Zisquit, a Senior Product Manager at Dynamic Yield with 10+ years of experience in building, managing, and deploying some of the world’s leading software solutions in customer experience and personalization. Here’s what he had to say:

Ernie: Why is understanding the relationship between products and user behavior so important right now?

Udi: The way I see it, there are three major contributing factors:

1. Rapid technological advancement

To state the obvious, we’re living through unprecedented advancements and explosive growth in AI. New capabilities that seemed like science fiction just a couple of years ago are being dropped on us almost daily. From generative AI and LLMs to advancements in robotics, autonomous vehicles, computer vision, mixed reality headsets, and more.

2. Empathy as the new currency

A byproduct of the rise of AI is the blurring lines between the natural and artificial, which is cultivating an expectation for human-like interactions from any other form of “sentient” beings, artificial or not. In our context, consumers expect all digital interfaces they interact with— websites, apps, digital assistants, outdoor kiosks—to see them, understand them, grasp their life journey, and account for their feelings and needs.

This is especially apparent in commerce, where the new generation of shoppers is unlike anything retailers have ever experienced. They expect digital channels to match offline physical channels, where real people can see and engage with your individuality. Research shows that shoppers aged 18-34 are the most indecisive online and exhibit the highest dropout rates. They want to feel seen.

Whether one is the chicken and the other an egg, the concurrence of both phenomena mandates today’s marketers to embrace technology that can transform their digital offerings into virtual worlds that cater to the human beings behind the online user.

3. Data overload

For the last decade, businesses have been encouraged to aggregate as much data as possible. They assume that if you invest enough in collecting and organizing your data, you’ll solve your problems and grow your business. However, this isn’t the case. Because businesses can’t easily sort, analyze, and act on such vast data, they’re losing billions in potential revenue. Collecting data isn’t enough. They need the technology to transform it into real engagement with their customers.

So, why is it so important right now? Consumers are savvy, they know what’s possible, they’re well aware of how much of their data is out there, and poor experiences tell them that businesses don’t see them or their needs. If you recommend beef jerky after a customer adds vegan products to their cart, it signals to them that you don’t care.

Fortunately, deep learning-powered affinity profiling is optimizing predictive personalization.

How is it helping marketers better understand their audiences?

Well, predictive personalization used to look like this:

• Collect the data
• Analyze it
• Set up conversion funnels
• Identify patterns and characteristics of cohorts
• Segment using primarily contextual or transaction-based data points
• Set up programmatic targeted experiences
• Track and analyze results
• Repeat

The problem is that sequencing is very difficult, long-term dependencies are impossible to track, and all optimization efforts are short-term and myopic.

But Recurrent Neural Networks (RNNs) now let us understand intricate, nuanced user journeys for the first time in history. It’s like a powerful, outsourced “brain” that can zoom out and analyze the hundreds of millions of user behaviors, predict their next step, and let us take action for all of them simultaneously.

Unlike regular feedforward neural networks where data points are given at the same time (like an engine that recommends a movie to watch based on the viewer’s age, location, and watch history), a Recurrent Neural Network specializes in sequential data on a timeline. Here, the values are given to the RNN starting with the first value, and then the second value is given to it along with the output of the first, and this continues recurrently with every next input consisting of itself and the previous outputs.

For this reason, RNNs are often used for speech recognition or language processing, as the next word in every sentence can be predicted based on all of the words preceding it that have already been processed.

An RNN can train on every action, every turn, every hesitation, and every decision, then accurately place them in order and understand which patterns and sequences lead to certain outcomes.

There are also LSTM (Long Short-Term Memory) layers that carry important information through recurrent learning over long periods, maintaining context/memory over extended and often fragmented data sequences.

So, even when online user journeys are segmented, fractioned, and divided into multiple sessions across devices, at different places and times – nothing gets lost. This is similar to how ChatGPT maintains the context of your conversation focused on a certain topic as opposed to just a single exchange.

Today’s computing power allows these models to be lightning-fast, so they can update and make predictions upon every new pageview, based on real-time behavior.

So how do you set these models up for the best results? What are the best practices for marketers to follow?

Well first off, they’re only as good as the data they train on, so I’d recommend every merchant to make sure their products are tagged correctly and their eventing logic is sound. Marketers should make sure their product feed design follows best practices (e.g., is rich with meaningful tagging and that every SKU is tagged correctly with all relevant categories and attributes).

Beyond that, I would recommend focusing on 4-6 attributes that are the most meaningful and reflective of your customer’s preferences and behavior, and make sure that the attributes offer a value for every one of your products.

Looking ahead, where does all this go from here?

The options are limitless. While Dynamic Yield’s new AffinityML focuses on predicting user affinity based on product attributes, these models can predict any aspect of the user journey. I believe down the road, interactive commerce experiences will transform into fully human-like conversational experiences that mimic offline engagement with real sentient beings in the store. They’ll combine speech, vocal interactions, empathy, emotional understanding, humor, and profound product knowledge that only computers can hold.

Outside of our field, things like crosswalk light automation based on real-time needs, healthcare monitoring systems, individual and predictive healthcare, automated therapy, automated content moderation on social media platforms, smart-home automation based on daily routines (lighting, temperature, etc.), generative music… it’s all on the table.

RNN and LTSM Are a Step Toward a More Empathetic Digital Experience

As new machine learning models enable us to keep track of highly complex user journeys across devices, sessions and places—regardless of how fragmented they are—marketers can better see, understand, and empathize with their customers.