As part of Qubit's continued deep-dive into how machine learning, and artificial intelligence, can change the way that marketers engage with and communcate with their customers, we hosted a panel discussion with Gerry Brown, Ana Sanandres and Bud Goswami.
Watch the video, or read the transcript of the discussion below.
To find out more about how Qubit can help you understand how to leverage machine learning for market segmentation, click here.
To download the co-authored research between IDC and Qubit then click here.
George Barker, Customer Advocacy at Qubit:
Hello, and welcome to this panel discussion with IDC and Qubit. We're here to discuss more about how machine learning and artificial intelligence can help with customer centricity right now. Firstly, I've got Gerry, who's our research director at IDC. Ana's here as one of the lead strategists for Qubit, who works with some of our biggest customers on a day-to-day. Bud is our lead data scientist who works with the research team on understanding how machine learning can apply to businesses right now, and into the future. Gerry, I'm wondering whether you can set the scene of how marketing departments are set up today, and how they're thinking about new, innovative techniques in getting close to the customer.
Gerry Brown, Research Director - European Software Group, IDC:
Marketing is an amazingly interesting place to be at the moment because it's a time of massive change, driven by technology.
Digital technology is revolutionising the way that marketing is being done. Today, frankly, it's not done particularly well.
In most organisations, the way they capture their data using digital marketing, CRM, core centre technologies and customer service technologies, it's very siloed. There's no complete view of a single version of the truth about the customer. Within a marketing department itself there's often silos of digital competence, so silos of social, silos of email, and various other pieces of technology, website analysis, et cetera. This has been fine, and it's moved the needle. A lot of focus of marketing departments today has been on moving the performance needle half a percent this way, half a percent that way. Getting more click throughs, et cetera.
What marketeers are finding, when we talk to CMOs, is that it's not working the way it used to. It's reached the end of its life cycle and a new kind of digital marketing needs to emerge. When we look at the objectives, we've recently done research with 450 marketeers across the globe, asking them about the future of marketing, and they say well today, we tend to be very performance driven and numbers driven, but tomorrow, we're going to be much more customer focused, looking for greater customer satisfaction, customer loyalty, and having more complete customer journeys, and having an end-to-end conversation experience for our customers, which is going to be highly personalised. So, it's going to delight the customer and make sure that the customer comes back. Which most of us, as consumers today, we probably don't feel in that kind of way. I heard a quote saying loyalty is for Labradors. For many people, it is a little bit like that today.
What we see in the marketing department is a joining up of these digital technologies. So, much more prevalence of a platform approach, bringing a unified technology platform within the marketing department. Also, we see connectiveness with sales and service, the customer-facing departments. Sharing of data, sharing of systems, and trying to get a holistic view of the customer and map towards customer experience. That's what the CMO wants today.
CEOs are putting out big statements of intent around customer experience, how it's going to differentiate them, how are they going to delight the customer. So, the CMOs are having to react. They find that the technology tools they have today are not delivering against that fit for purpose in terms of delivering customer experience.
What we're finding is that, one, is a view of unifying technologies within marketing and sales and service, and looking at data from a complete customer view. Two, machine learning has gone from 0 to 100 in two years and there's massive opportunities around this. It's very interesting that Qubit has decided to integrate artificial intelligence and machine learning into its products because it's definitely the future of digital technology for marketeers.
That whole ability to continuously learn and take away some of the grunt work of marketeers, and help them to move onto those high level activities of strategic thinking, activating marketing campaigns, and designing unique customer experiences is really what machine learning is all about.
Bud, the CEOs are coming out with these big statements around customer centricity and being customer focused. I've heard you mention machine learning, Gerry, in that answer. What are the techniques and ideas around machine learning and artificial intelligence which seem to be the buzzword or the thing of today? Lift the lid for us on what that actually means, and how that applies to those departments right now.
Bud Goswami, Lead Data Scientist, Qubit:
One thing to note, actually, is I think a lot of technology companies today are doing a particularly bad job of differentiating between these terms and also conveying the message of exactly how these techniques are actually more useful than cool to marketing departments. I really like that you mentioned that the focus of machine learning is actually to take the grunt work away so that the marketers can actually focus on brand differentiation.
Machine learning is actually a subset of artificial intelligence. Frankly, I don't really think we have artificial intelligence. We have something called artificial narrow intelligence, which is where we have some bits of technology that learn to efficiently encode history, and then show super human performance in very, very narrow tasks, like face recognition. I think I recently heard that the tasks that AI, as we call it today, does well are things that take a human less than three seconds to compete, or perhaps things that humans don't do very well, which is predicting the future within a limited time span.
Machine learning is slightly different. The kinds of problems that machine learning solves frankly within the Qubit context are finding groups of customers, so we call this unsupervised learning, or learning from historically annotated data to learn about customer preferences, for instance, or to serve bespoke customer experience like product recommendations, as an example.
Moreover, the approach at Qubit is one of putting the customer business goals first and then using our experience in machine learning to deploy the optimal methodologies to achieve this purpose: this phrase that I like called empirical empathy. Marketers want a very quick turnaround from their consolidation of data, consolidation of insights to understand what users, at an increasingly granular level, from every user that visits their websites to our idea of segment, customer segments, to individual customers which are context driven, and then driving those insights to create a bespoke personalised experience. Then, serving those experiences across a variety of channels, some of which were actually also in the process of creating for our marketers.
The question may be to both of you. What is this sort of tech enabling? Is it the only thing that we should be thinking about as marketers, or people in e-commerce? Or how should we be thinking about using this technology straightaway?
I would say progressive profiling is an obvious example. You today, maybe I know some of the channels that you use. You've got a mobile phone. You use a laptop computer. I know that you've been looking for a car. I have some really basic information. What I need to do is to enrich that information over time, and to track your activities to give me signals in terms of your intent.
I don't really want to activate a marketing message to you until I know that you're in a sweet spot of potential interest buying kind of thing. A lot of marketeers go too early, and salespeople too, in trying to close deals too quickly. This frustrates customers because we're not ready, in a state of readiness, to appreciate that kind of activity.
Using progressive profiling, I can get to understand you really much better. I think Amazon does this brilliantly, by the way. The more you use Amazon, the more you feel that they understand you, they have empathy with your goals, with the kind of things your interest is, your preference is. So, we feel much more comfortable about using them as a first source for virtually any product or service today.
I think machine learning can really help us to get better personalization and granular understanding of the customer. Not to be creepy in the way that we approach him, but to be appropriate and relevant and by giving the right message at the right time at the right place in order to provide marketing as a service rather than marketing as a selling tactic.
Ana, when you're going in and talking about this to our customers at Qubit, what are the sorts of questions that they're coming back to you? What are the challenges that they're grappling with right now? How are they taking this challenge on?
Ana Sanandres, Manager - Professional Services, Qubit.
I guess it's no news to anyone that the terms of personalization are a bit of a buzz theme at the moment. What we often get asked by customers is where do we start with personalization? How do we improve customer experience for certain segments of users? What machine learning can help us do is uncover underperforming segments and segments that require a certain amount of attention. Also, the monetary value attached to that opportunity.
I think obviously the quantitative data only offers part of that story. What we've seen lots of successes with at Qubit is combining the opportunity serviced through machine learning with more qualitative data. One example that I can think of from one of our retail clients was they saw an underperformance in users from France browsing shoes. So, we launched a survey just to ask users abandoning the shoe page from France, "Why have you abandoned today?"
What we found is they had a specific problem with the UK sizing on that page. French customers were not used to UK shoe sizes. Once we converted those shoe sizes into European shoe sizes, we saw a really good increase in conversion for that specific segment of users. So yeah, the kind of questions we often get asked is where to start, what should our priority segments be, and what kind of experiences should we be targeting those segments with.
How do you go about starting on this journey with our customers? What are the first steps, and how do they get to grips with machine learning because it is quite an ethereal, mythical term. I'm wondering how you apply that straightaway.
It's not always an easy journey. I think sometimes there are certain foundations that must be in place before machine learning can be valuable. One of the key points is having the right data in place so that these opportunities can be surfaced. Without a proper data layer in place, without collecting that granular information about users, you can't necessarily surface interesting opportunities.
I think the key places to start are with the bigger segments and the bigger monetary opportunities. Which segments are under-performing, how big a segment is that, and how must lost revenue does that represent for our business? Because obviously, there's a big problem with content creation and also resourcing to power personalization. Obviously, if you want to personalise for certain segments, there has to be certain strategies put in place, content created for that segment.
You can't get too granular from Day One; that would be my key recommendation. Focus on the biggest segments, the biggest opportunities. Prove that segmentation and personalization can work for the bigger segments, and then get more granular.
Going back to your point at the start, Gerry, around how marketing departments are changing. How do you see the roles and responsibilities of the makeup of the marketing department moving forward? If we were to look two or three years in the future, how would a marketing department be different to how it is today and maybe yesterday?
Yes. Great question. I think it's going to become much more formalised, the use of digital technology. We've recently been asking marketeers, we did a survey of 300 marketeers just last month. When we asked them about how you're measured and are you being measured on digital performance, we found between 50% and 60% of people now are having some element in their annual appraisals around digital performance. Management are evaluating people in terms of how they work with agencies, their ability to produce good digital work, the creative work, and segmentation, as we've just been talking about.
What we're going to find, there's been a lot of shadow IT, as we call it in the IT Department. Sneaking in through the back door really, under the radar from the IT Department because of the need to have decent technology to drive campaigns and other marketing activities.
It's been like an informal, unstructured approach. There's going to be much more rigour to the way that marketing deploys technology. Data scientists are going to play a major role in that. We see actively today that marketing departments are employing data scientists who are obviously highly qualified and skilled at working with data. Most marketeers- and I can say this as an ex-marketing lecturer- are great at the creative stuff and the fluffy qualitative stuff. They're not so good with the numbers.
You really need to get people who've got specialty skills. It's actually easier to get a skilled mathematician and teach them the significance of the numbers in a marketing context, rather than the other way around. All marketeers need to be brought all around us in the future, and they need to be able to use an harness digital technologies.
For that specialist data analysis piece, I would suggest that you really need someone with a statistical background and bent towards mathematics, which isn't common in marketing departments. I think that we're seeing that today, the hybrid marketing department of creative and content driven, and on the other side the data side. Pulling those two pieces together to get a complete view of the customer and to drive really intelligent conversational marketing campaigns is where we're going.
Talking about data scientists, Bud. You sit on both sides of the table, in marketing with Qubit and in data science team. How do you think marketers of today can learn more about, or should be getting to grips with, some of the things that you're working on in your department around machine learning and artificial intelligence. What should we be understanding from this?
I think something the Qubit platform does well, which is something you alluded to before, is that if a marketer is worried about conversational personalization, then the natural way to progress with this is to create some sort of experience, a conversational experience, with that customer to launch that using their digital touch-points, to measure its impact through AB testing, as an example. And then subsequently, to take away the insight from that experience, I guess, the learning experience, and then just metre it. I refer to this as quantitative systematic marketing.
Rolls off the tongue.
Indeed. Well, it doesn't, but it's really important because it's almost like a combination of the right brain approach, which is the content creation that appeals to your marketer versus the left brain approach, which is measure its impact in a scientific way, rather than just an opinion driven way. That leads to the incremental gains in the marketing performance and frankly, that's a win-win both for brands as well as for customers. That's why I'm excited about the integration of data science within marketing.
It's a fascinating time because there's a huge amount of money going into this sort of thing. What are the projections that you're seeing, from an AI and machine learning point of view, in two or three years time, the increase in spend around this sort of technologies? Talk to me about the it's not just you plug in machine learning and you're doing machine learning; it's incremental, isn't it? How do marketing departments get to grips with this sort of thing straightaway?
We've done some forecast and some market sizing around artificial intelligence. Not a term that we particularly like, actually, because it's so broad in terms of its use and applications, and it's not clear. When we look at machine learning within marketing orientated kind of departments, we see globally about 360 million spend in 2016 on this. That we see growing around about 59% on a cumulative annual growth rate through to over two billion by 2020. It's one of a number of applications.
The way we look at it, we do these spending guides. We look at it from an enterprise perspective. A lot of the applications are only just bubbling up now for machine learning, and it's rally almost your own creativity. I would suggest using Qubit's tools. It's partly the creativity of the marketer to get the best out of those tools.
It's important to have marketeers who really can think outside the box and are prepared to experiment, to fail fast, and to look for new, innovative ways of doing things. We see a massive growth within the marketing department and that whole alignment of AI with traditional marketing technology, which, to be honest, hasn't been very effective. It's helped, at the decision point, like in a call centre, having your profile in front of me, I can have a little bit more of a personalised experience. But it's not that intelligent and often, those call centre operators are switching between screens and scrabbling around, trying to get pieces of data from different databases.
It's important to get this unified view of the customer and using machine learning to continue to learn about the customer. If you're talking, you're not learning, a wise man once said. So, it's important for marketeers to listen to the customer. Machine learning is a brilliant way of doing that in an automated fashion to increase our understanding of customer needs and to segment them, as Ana so rightly said.
Ana, on the fail fast approach and the culture change that you're seeing within businesses, how are people using that approach to test things, to be creative, but also using data to back up that creativity, in a way?
Yeah, I think Gerry raises a good point about the more data science-y, science-y marketeer and the more creative one. I think there's definitely a space for both in delivering personlizations in customer experience. I think something like machine learning surfaces opportunities. The way that you tackle those opportunities and customer segments that may be underperforming, you can do it in millions of different ways.
What we tend to recommend is that you AB test any sort of segmentational personalization that you launch because it's all on hypothesis. Everything starts with a hypothesis. We think that making this change will have a positive impact on this metric. A lot of the time, it doesn't. Sometimes it has no impact, sometimes it does have positive impact, and sometimes even a negative one.
It's really important for businesses to test the changes that they're making, and then iterate. So, be it a new creative, a new way of messaging that certain piece of content. I think businesses are definitely adopting this culture of testing, then iterating and improving on what they originally launched.
Bud, we hear a lot about AB testing. What are its limitations, and what role is it going to play in the next 12-20 months and into the future, do you think?
One of the things that I hear sometimes is that traditional AB testing is dead. There's actually a subtle clarification that's perhaps worth teasing up there. AB testing is just a way of using a scientific method in order to measure causality. You've done something; is it actually having an impact, positive or negative? What's changed is just that the hypotheses that people were using to test within an AB test, you've got to the area where most site experiences are pretty well consolidated, things are reasonably optimised as far as you have a website; people can do things on it.
Where machine learning has come in, and I really think it's a useful way to improve your site experience, it's actually lending focus to the kinds of hypotheses that you should use in order to AB test. As you just mentioned, the French shoes instance. For instance, one of the things that we've recently done. We're building a model to measure the customer propensity to purchase.
When you build a statistical model, a machine learning model, of this, the model itself, not only does it tell you the answer, how likely they are to purchase, but also which metrics affect that likelihood. That means that you now have a very focused way of addressing okay, I need to focus my next creative experiences on metric A, perhaps time on site, metric B, what kinds of content are they engaged with, or what kinds of journeys are they experiencing?
I don't think AB testing is dead; in fact, frankly, I think that it's now not something that you should shout about because the inherent assumption should be you're doing things in a scientific way. You can't really iterate without the science. But where machine learning comes in is that you have a very specific question that you want to ask of the data, perhaps behavioural changes in your customer segments or something. Then, you measure that, you automate that using machine learning, and you get two things for free: some indications of where you should focus your next set of hypotheses on and the model that you might then use to automate an aspect of your website.
Gerry, finally. I was wondering how conversations in the boardroom go around this sort of thing. You mentioned CEOs making huge sweeping statements about customer centricity and being customer focused, but when you talk to the CMOs and understand the big challenges that they're faced with, what are the two or three things that they've really got to come to terms with right now for 2017 and into the future?
I think from an enterprise perspective, first of all, thinking about what are we trying to achieve through customer experience. It's a term that is bandied around, and what is it that we're actually trying to decide on. How do we want our customers to feel about us? I think it's a change of insight. Okay, what products are we going to sell them? How much profit are we going to make out of selling those products? What's our cost of sale?
That kind of traditional approach to business is completely changing and people are looking at the outside in. How can I get George into my stable and how can I keep him? How can I continue to deliver value to him? How can I connect him into a whole ecosystem of potential services which will be of value to him so that I can have you at the centre?
There's a complete change of thinking. There's a real struggle, from a top management perspective, in getting this thinking because they think inside out. To think outside in, as us as marketeers traditionally think, is a very important aspect.
What we're seeing is in the early digital transformation projects that came in, they were typically migrated to the chief digital officer and to the CIO. We see that balance changing. When we talk to marketeers around who is the sole authorization point and influence point for customer experience, 60% said it's the CMO. That was followed by 12% for the CIO, 11% for the CDO, 10% for the CEO. Marketeers have a tactic acceptance that customer experience is the way you need to go.
What is that customer experience that we're trying to gain? A lot of marketing is still push communications. Not very intelligent, pushing out stuff. You just have to open your PC this morning and there's all kinds of adverts being thrown at you, or things you've already bought. There's a lot of frustration out there from the customer in the way that we're treated. We're not treated with a feeling of a great deal of respect. It's really people throwing technology and hoping some of it will stick. We need to be much more intelligent.
The CMOs do get this; they are very frustrated with the way digital technologies have been applied, and that we're upsetting our customers and losing customers. We're hounding them because they go into a database and then emails go out to them automatically. Eventually, they go mad and they hate you, rather than love you. That whole managed customer experience is really an important port.
Such is the complexity of delivering that. You need to have the necessary technologies. Traditional technologies don't do the whole job. Machine learning as a layer on top of the traditional digital technologies, and a wider view of the customer experience, is what you really need.
I just want to say to your point. I think one of the changes that needs to be made for CMOs or any C-suite is to make sure we are measuring the right KPIs to measure that customer centricity. Traditionally in reporting, we look at things like conversion rate, revenue, returns, et cetera. But I think, not to my knowledge, nobody has really nailed what those customer centric KPIs should be and incorporated those into reporting.
Now, in order to put that customer focus on reporting on the business, we are going to have to start looking at measures for good customer experience.
When we were talking to these marketeers, when we asked them, "What are the key measures that you use to measure customer experience?" they said conversion, first of all. Then, they said customer satisfaction. Then, they said customer loyalty. That's how they do it today, and that needs to switch. The whole measurement game around customer experience is very early days.
A lot of people talk about NPS. NPS has a lot of down sides, too. It's not a perfect measure. I think a very interesting metric coming through is customer effort scores, understanding how much pain and effort you need to go through to buy from us. When you ask people that question, they tend to be much more honest. Often, there's emotive things about net promoter. Would you recommend this to one of your friends? You say sure, sure, just because you want to get off the phone, for example.
It's very early days for the measurement game around customer experience, and there will be new measures coming out. Hopefully Qubit, with your technology, you'll be helping us to understand our customers better by helping us to create new measures and metrics that we can harness in order to get better focus on our customer needs.
Do you have anything to say on how we're thinking about measurement and that sort of thing?
Yes. I think there has to be a separation between the business focus of metrics and the engagement focus of the metrics. The engagement focus metrics that we all measure today all focus on the quantity of engagement that a user has with a website, which is actually a remnant from the page views structure of the Internet.
The Internet itself has changed, and websites themselves have changed. Now, certainly at Qubit, we now have this more sophisticated event-based way of capturing first pass user behavioural data. We're really focused on ensuring that you can differentiate between the quantity of user engagements. A simple example is a lot of people say oh, you want to decrease the time onsite so that someone comes onto your site, they find the perfect product, they purchase and they leave. By that count, the more time people spend, maybe that's actually a measure of loyalty.
I would also argue that that's actually a measure of somebody being lost. That's exactly my point. Something like time on site is actually insufficient to discern between the lost customer and the engaged customer. We have to be a bit more sophisticated. Honestly, because the Internet itself has changed, now that you have the proliferation of more devices, the proliferation of different sorts of technologies like single page apps, and so on and so forth, you have to be a lot more sophisticated in how you actually measure the engagement, the quality of the engagement, and then link that back to business metrics of do you have now a high conversion rate because you have more engaged customers?
One thing I'd say from my experience at Qubit is not everyone is at the same stage in terms of accepting that failure is almost as good as success, or the learnings from failure are almost as good as a success. I think what machine learning helps you do is succeed more often because it's surfacing opportunities within your data that you didn't necessarily know existed. Then, it's up to the more creative marketeers to come up with a solution to address that issue, or that underperforming segment.
There's definitely, amongst our customers, an increasing move towards a culture of test and learn. If something doesn't work the first time, how can we iterate and improve on that, if we have a very strong hypothesis and a very strong business goal that we're working towards. It's much easier to identify problem areas, identify solutions, and then hopefully execute successful personalizations and segmented experiences.
There's an interesting point, just to add to that one, was we did this research around machine learning in marketing departments from 450 marketeers worldwide. When we asked them what the challenges were around the adoption of machine learning, they said three things, actually. They said one is clear monetization; two, applications; three, brand control.
We talked about first party data earlier. It's getting increasingly sensitive and there's a lot of regulation. And, of course, we've got GDPR, the European standard around data privacy is coming in. Large companies are very cognizant of the need to control their own data. Actually, they're starting to bring a lot of these technologies in house so that they can control the data.
It's interesting what Qubit's done because you've actually chosen a specific application for machine learning. Generic machine learning applications are not really gaining traction in the marketplace. They have to show clear business benefit and monetization, and a clear application, in order to be successful and also ensure that there is control of your data sources, and there's no leakage into the marketplace of your precious customer data.
I think that's a good point because people might think that machine learning is just oh, you plug it in and now you're going to be amazing. What are the limitations of machine learning, to the example that we spoke about a couple of weeks ago?
An obvious one is that, again, machine learning is not intelligent. It doesn't work if you don't give it an objective function to learn. Furthermore, it's not robust, so you can't just say ah yes, well, I would like to measure how a customer is likely to convert, and then expect that same machine learning model to answer a different question. It's not really how it works.
What it will do is potentially generalise. If you haven't seen some customer behaviour before, it might predict how well that person is going to do, but it can't answer any questions beyond the very narrow, very specific question that it has been asked. By that count, that's why I think it's not a replacement for marketers; it's a technology that amplify a marketer's skill.
It's up to the market to ensure that they ask the right business questions off the machine learning to ensure that focus. Some of the limitations, for instance, often the limitations are actually imposed by the quality of the data.
Let me give you an example. Product category information is something that's bandied about. Everyone has it. But what people forget is that product categories, you can categorise products by the type of thing: this is a pair of trousers. You can also categorise it by its merchandise thing: this is new in versus this is last season. And, you can categorise this by its pricing structure. This sale, is it an expensive item? It is a less expensive item?
Currently, the product category information that we see on a lot of websites mix and match between all of these, so then it becomes really difficult for any machine learning to recognise that there's this notion of dynamic ontology that's going on. This is all of those three things. This is a pair of trousers, this is a new in, and this is whatever price it is. Encoding that now becomes really important. The more granular and accurate your data quality is, the better the machine learning performs.
A limitation of machine learning actually is the quality of the question that you ask of it and the quality of the data that you give to it. These are challenges. This is why I think it's really good that marketers are now embracing the scientific requirements that are imposed on them for machine learning.
I think further to that, what machine learning gives you is a scientific what: this specific segment of users is underperforming on this metric. What it doesn't give you is the why: why are they not converting at the rate that we expect them to convert at? Why are they not spending as much as we expected them to convert? That's where things like talking to customers, be it through a survey, be it face to face, provides that last piece of the puzzle. You are not converting as we expected because you're not getting the information you need, or your size is not available, whatever that may be.
I think the whole customer listening piece is really important because now, everybody's asking questions. You just go anywhere and they say, "What was your experience like?" I personally find it kind of irritating because I just want a transaction. I want to buy something. I don't want to have an experience. I want to get in and get out with the greatest speed, and then they want me to answer a questionnaire at call centres and on the web. You're getting a dramatically low level of response. One percent or less of people actually respond to those customer experience questionnaires after they've had an interaction.
You have to be really smart about the way that you gather information, and building communities of customers. That's where digital can help you to create stickiness and build customer communities who are advocates, and are sharing of their data, and who are open because you have proved to them that you're more than just a supplier; you're a friend, a partner, and someone with whom they have a relationship.
A lot of the big brands have done this amazingly well, and social people like Coca-Cola and Unilever and Procter & Gable. They haven't skimped in their spend. Incredible amounts of money doing beautifully crafted advertisements, but when the end result is 50 million downloads, the impact and influence is enormous on the market.
It's not like machines are going to take over; there still requires that human interaction element, which is great.
Thank you all for participating in this panel. It's been fascinating, hearing more about all the things machine learning can do for customers. Thank you for listening and hearing from these three. If you need to find more information, or you want to understand more, there's a report that we've produced with IDC, and that gives you more, I guess, colour to some of the things we've discussed today. Thank you.
To get a full download of the report to understand more then click here.
George BarkerRead More