Smart Data & Analytics

Applying AI to create a market research solution that works

In 2014 – which seems a lifetime ago now! – I was a consultant, helping the world’s largest companies launch new products in markets across the world. But one issue kept cropping up, and I couldn’t solve it – finding data that companies would trust. 

Existing market research solutions, like surveys based on claimed behaviour, clearly didn’t give accurate answers that companies believed in. Unfortunately, big corporates don’t make billion-dollar decisions based on assumptions and guesswork – and without strong data, they simply wouldn’t go ahead with new product development in markets that they didn’t know, leaving opportunities on the table.

To me, the situation didn’t make sense. There are millions of people who already interact with goods and services, who would share information with us if we asked them to. Maybe they would even share photos and videos too, if we engaged with them and incentivised them. Then, we’d have a huge amount of rich data that would really allow us to understand how people lived and behaved, and we could make accurate decisions about what they’d buy. And you could apply this approach across the world, not just the West. 

However, every research company said the idea was either impossible, or would cost a ridiculous amount. Instead, they suggested we rely on out-dated, ineffective methods like surveys and panels – which my clients had already tried, and were sick to death of.

A new approach

That’s how Streetbees came about. I started the company with £5,000 of my own money, looking to create a solution that would collect real stories from people, including photo and video content, and let brands access the detail within people’s lives around the world. The idea gained traction quickly with some of the biggest FMCG companies, and they began using us to understand their customers better. 

Fast-forward four years, and we now have some two million global users – we call them our bees! – sharing their stories with us. That’s an enormous amount of data to process and understand; if you think how much information is contained in a photo or video (not just what the user is doing, but what they are saying, where they are and who they are with), the number of data-points about human behaviour we now have numbers in the billions. 

So how do we make sense of all of that information? Well, we’ve raised $20m from investors including Atomico, Europe’s leading VC firm, which we’re using to develop machine learning, neural networks and natural language processing to extract the meaning behind the unstructured data we collect, enabling us to understand the clusters that lurk within the chaos. In this way, we can find groups of people or certain behaviours that big companies had no idea existed – helping them uncover not just what people do, but why they do it, and what they may do next. 

So in practice, how does it work? Here’s three examples of how our solution has helped some major companies shift their operations, based on insights provided by our AI and machine learning. Our clients are private about exactly what we’ve done for them as they don’t want all the secrets to escape, but I hope this gives a taste of how we’re disrupting the market research sector forever! 

Flexing the diet message

Our client, a global food manufacturer, wanted to understand why their category was not growing. What they produce is traditionally viewed as a healthy choice and – as consumers were putting health more top of mind in their food choices – they should have been growing; but they weren’t. 

We collected the diet preferences of our bees up front, in their own words, and then logged several thousand in-the-moment occasions of food and beverage consumption to understand what people were eating and why. We then applied our machine learning and natural language processing to the results to look at the relationship between the language used to describe their diet, versus how they spoke about their food and beverage choices. 

In this way, we found that many of our bees were replacing traditional ‘low’ diets – such as low sugar, low calories or low fat – by ‘high’ diets, where people try to eat lots of elements like protein, plant-based foods or natural foods. And when people were making that switch, the consumption of our client’s category declined significantly, especially in urban areas. 

Why? Well, it turned out that the category’s traditional messaging was directed towards those on a ‘low’ diet, and it wasn’t resonating with this new type of consumer. 

Armed with this information, our client was able to prioritise products in their innovation pipeline that delivered against the needs of ‘high’ dieters, review its messaging and claims on its current product lines to reflect changing consumer priorities, and develop an urban strategy that accelerated these changes in cities and towns. 

Uncovering the true drivers of brand experience

A global personal care brand was facing a decline in market share within a declining category – even though their current research was telling them they had the best products in the market, and the strongest brand.

To help understand the disconnect, we collected thousands of brand experiences – from usage to purchase to brand messaging – building a complete picture of what was really happening both with when consumers used this company’s products, as well as those of its competitors – a challenge that our AI was up to! 

We found that traditional models of brand performance that are largely built on functional attributes didn’t really drive the brand experience. It was how people felt about themselves after a personal care experience that really mattered, not how well the products worked. We used natural language processing to build a comprehensive list of attributes from open text in-the-moment responses, then used Explainable AI to identify which attributes drove brand choices.

Based on this new model, we were able to show that rather than being a market leader, our client’s brand was failing on several critical attributes that consumers really cared about. Through iterative workshops, we were able to reveal gaps in our client’s packaging development, messaging and innovation that they could address to deliver against these new drivers.

Assessing the value in promoting

A global snack manufacturer was investing significantly in promotional activity, these investments were returning less and less value over time, despite price and promotions being the key drivers of consumer choice identified in their brand tracking data.

We sought to find what really drives snacking choices in the moment of purchase. To do so, we asked hundreds of consumers to record their purchases of snacks when they were in store, allowing us to compare in-the-moment purchase drivers to claimed data (again, using AI to do the legwork of checking thousands of purchasing decisions). 

What we discovered was truly stunning; while lots of shoppers said that price and promotion was important in their choice, when it actually came to the moment of purchase barely any of them mentioned it. At that moment, what really mattered to them was taste, cravings, and wanting their favourite brand.

We also found that in the instances they did buy on promotion, this usually happened at the end of aisles (gondola ends), rather than in the main supermarket shelf. Whereas in their recent promotional deals with retailers, our client hadn’t been getting gondola end presence.

Our client used the data to persuade teams to reduce spend on promotions, or to insist on gondola end representation if they did promote – saving significant amounts of money without losing market share.

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.
Please note that your e-mail address will not be publicly displayed.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles