Remember me

Register  |   Lost password?


The Practical Quant Blog Header

How AI and machine learning are improving customer experience

Tue, 14 May 2019 11:01:26 GMT

[A version of this post appears on the O'Reilly Radar.]

From data quality to personalization, to customer acquisition and retention, and beyond, AI and ML will shape the customer experience of the future.

By Ben Lorica and Mike Loukides.

What can artificial intelligence (AI) and machine learning (ML) do to improve customer experience? AI and ML already have been intimately involved in online shopping since, well, the beginning of online shopping. You can’t use Amazon or any other shopping service without getting recommendations, which are often personalized based on the vendor’s understanding of your traits: your purchase history, your browsing history, and possibly much more. Amazon and other online businesses would love to invent a digital version of the (possibly mythical) sales person who knows you and your tastes, and can unerringly guide you to products you will enjoy.

Everything begins with better data

To make that vision a reality, we need to start with some heavy lifting on the back end. Who are your customers? Do you really know who they are? All customers leave behind a data trail, but that data trail is a series of fragments, and it’s hard to relate those fragments to each other. If one customer has multiple accounts, can you tell? If a customer has separate accounts for business and personal use, can you link them? And if an organization uses many different names (we remember a presentation in which someone talked of the hundreds of names—literally—that resolved to IBM), can you discover the single organization responsible for them? Customer experience starts with knowing exactly who your customers are and how they’re related. Scrubbing your customer lists to eliminate duplicates is called entity resolution; it used to be the domain of large companies that could afford substantial data teams. We’re now seeing the democratization of entity resolution: there are now startups that provide entity resolution software and services that are appropriate for small to mid-sized organizations.

Once you’ve found out who your customers are, you have to ask how well you know them. Getting a holistic view of a customer’s activities is central to understanding their needs. What data do you have about them, and how do you use it? ML and AI are now being used as tools in data gathering: in processing the data streams that come from sensors, apps, and other sources. Gathering customer data can be intrusive and ethically questionable; as you build your understanding of your customers, make sure you have their consent and that you aren’t compromising their privacy.

ML isn’t fundamentally different from any other kind of computing: the rule “garbage in, garbage out” still applies. If your training data is low quality, your results will be poor. As the number of data sources grows, the number of potential data fields and variables increases, along with the potential for error: transcription errors, typographic errors, and so on. In the past it might have been possible to manually correct and repair data, but correcting data manually is an error-prone and tedious task that continues to occupy most data scientists. As with entity resolution, data quality and data repair have been the subject of recent research, and a new set of machine learning tools for automating data cleaning are beginning to appear.


One common application of machine learning and AI to customer experience is in personalization and recommendation systems. In recent years, hybrid recommender systems — applications that combine multiple recommender strategies—have become much more common. Many hybrid recommenders rely on many different sources and large amounts of data, and deep learning models are frequently part of such systems. While it’s common for recommendations to be based on models that are only retrained periodically, advanced recommendation and personalization systems will need to be real time. Using reinforcement learning, online learning, and bandit algorithms, companies are beginning to build recommendation systems that constantly train models against live data.

Machine learning and AI are automating many different enterprise tasks and workflows, including customer interactions. We’ve all experienced chatbots that automate various aspects of customer service. So far, chatbots are more annoying than helpful—though, well-designed, simple “frequently asked question” bots can lead to good customer acquisition rates. But we’re only in the early stages of natural language processing and understanding—and in the past year, we’ve seen many breakthroughs. As our ability to build sophisticated language models improves, we will see chatbots progress through a number of stages: from providing notifications, to managing simple question and answer scenarios, to understanding context and participating in simple dialogs, and finally to personal assistants that are “aware” of their users’ needs. As chatbots improve, we expect them to become an integral part of customer service, not merely an annoyance that you have to work through to get to a human. And for chatbots to reach this level of performance, they will need to incorporate real-time recommendation and personalization. They will need to understand customers as well as a human.

Fraud detection is another technology that is now digesting machine learning. Fraud detection is engaged in a constant battle between the good guys and the criminals, and the stakes are constantly increasing. Fraud artists are inventing more sophisticated techniques for online crime. Fraud is no longer person-to-person: it is automated, as in a bot that buys up all the tickets to events so they can be resold by scalpers. As we’ve seen in many recent elections, it is easy for criminals to penetrate social media by creating a bot that floods conversations with automated responses. It is much harder to discover those bots and block them in real time. That’s only possible with machine learning, and even then, it’s a difficult problem that’s only partially solved. But solving it is a critical part of re-building an online world in which people feel safe and respected.

Advances in speech technologies and emotion detection will reduce friction in automated customer interactions even further. Multi-modal models that combine different kinds of inputs (audio, text, vision) will make it easier to respond to customers appropriately; customers might be able to show you what they want or send a live video of a problem they’re facing. While interactions between humans and robots frequently place users in the creepy “uncanny valley,” it’s a safe bet that customers of the future will be more comfortable with robots than we are now.

But if we’re going to get customers through to the other side of the uncanny valley, we also have to respect what they value. AI and ML applications that affect customers will have to respect privacy; they will have to be secure; and they will have to be fair and unbiased. None of these challenges are simple, but technology won’t improve customer experience if customers end up feeling abused. The result may be more efficient, but that’s a bad tradeoff.

What will machine learning and artificial intelligence do for customer experience? It has already done much. But there’s much more that it can do—and that it has to do—in building the frictionless customer experience of the future.

Related Content: