How Pandora is Using AI and Machine Learning to Enable One to One Personalization at Scale
The quest to deliver one to one personalization at scale is a huge business need and fast becoming a fundamental component of a 21st Century Commercial model. Businesses that are able to exploit the power of Artificial Intelligence (AI) and Machine Learning (ML) to better understand customer behavior and personalize their customer experience will outperform their competition.
Advanced personalization and recommendation systems are the brains behind every customer facing technology in your business - including voice enabled devices, configurable offerings, sales enablement systems, dynamic pricing, response management systems, and personalized ads.
90% of organizations are using AI to improve their customer journeys, revolutionize how they interact with customers and deliver more compelling experiences. 75% of Netflix users select films recommended to them by the company’s machine learning algorithms. 83% of IT leaders say AI & ML is transforming customer engagement.
To realize this potential, businesses in every industry are pouring money into the talent, tools, and data to exploit the potential of AI to grow revenues, profits, and firm value. Spending on AI software will top $125B by 2025. Marketers are allocating over 10% of overall marketing budgets for advanced analytics. AI hiring grew 32% in the last year.
Despite all the hype and expectations around the ways AI will transform business, the reality is that mainstream adoption of AI in customer engagement is still in its early days. The runway for innovation is long. And the number of ways AI-enabled personalization can improve margins, conversion rates, and firm value are myriad.
One of the biggest obstacles preventing businesses from realizing the full potential of AI in commercial applications is the inability of executives to prioritize, direct, and allocate resources to the most profitable AI applications. “Most of the success and failures to harness the power of AI to transform business lies in management’s understanding of how to apply, deploy and direct these powerful tools”, according to Kartik Hosanagar, Professor of Operations, Information and Decisions and Marketing at The Wharton School and author of the influential book A Humans Guide to Machine Intelligence. Another factor delaying the business impact of AI is that it takes an enormous amount of learning data to develop robust Machine Learning models.
Few businesses have the level of experience, learning data, and technical acumen that Pandora does when it comes to developing AI driven personalization and recommendation systems. Pandora literally “jump started” the development of their recommendation systems with the Music Genome project twenty years ago, and since has had access to one of the largest learning data sets and best data scientists in the world.
“Pandora was born 20 years ago from what was called the Music Genome Project which became the first music recommender system for consumers,” according to Scott Wong, VP of Machine Learning for Foundation, Search, and Voice Science at SiriusXM and Pandora. “The initiative started with a team of professional musicologists sitting down and listening to music tracks day in and day out annotating over 450 different attributes about those tracks. They looked at things like the time signature, harmonics, vocals, genre, time periods, instrumentation language, and tons of different dimensions about what is really under the hood in any given song.”
That foundation of data gave Pandora what Wong calls a great solution to the “cold start” problem for building a recommendation engine before they even had access to customer interactions and other learning data.
When Pandora did start to access listener data to refine their algorithm, their progress and the quality of their prediction models improved dramatically. “The music genome project really enabled us with the first round of content-based strategies for the recommender system,” says Wong. “But after people started using the product and actually listening, it provided us the implicit and explicit feedback that's really the source of digital gold when it comes to machine learning. Listener feedback data really enables the development of advanced and predictive recommender systems.”
Content-based recommender systems like this are valuable because they are very good at identifying and recommending highly relevant content, songs, or products regardless of preconceived notions, popularity, or other influences, according to Professor Hosanagar. That’s hugely important in a world where every business is trying to differentiate its products, services, and brands through personalized and highly contextual experiences.
From that foundation, Pandora evolved their approach to combine other data signals and modeling techniques, like collaborative filtering, which was popularized by Amazon, to take their predictive models to the next level. This lets them take advantage of both explicit signals from their millions of users (such as thumbs up ratings) and implicit signals (like how long they listen, and what actions they take).
“What’s happened over time is we started to combine techniques all at the same time,” recalls Scott. “That flywheel, once you get it going, yields extremely powerful models for predicting and recommending. We started to incorporate other categories of strategies into our recommender systems in addition to the content based strategy we started with. We employed user-based strategies which look at whether people like you also like this other thing and collaborative filtering strategies that simultaneously look at which listeners are similar, and which content is similar. All of these can help you explain and learn based on what people do and discover the hidden similarities between content and preferences.”
“Our access to millions of listeners is helping us build the next generation of recommender strategies,” according to Wong. “As an example, we’ve evolved our models to the point where we can take the best of what humans have - the best eye for analyzing the genes underlying music - and scale it up to a much, much larger catalog of songs. We call the combination using humans for quality and machines for scale. One way we’ve been able to accomplish that is to develop a machine listening system where a model listens directly to the audio of untagged songs in our back catalog and new songs when they come in against the label training data we have from the Music Genome Project. Using this approach, we can predict alot of those attributes about new music that none of the humans internally have ever listened to. And so collectively when you put all these different techniques together we can power algorithmic radio stations that draw from 30 million different sources instead of two million that are human analyzed. In our business, it lets individual people personalize their own stations to their own tastes, and in aggregate it helps us track what is most successful on which stations.”
Wong’s team builds reusable machine learning systems that power recommendations and discovery across Sirius XM, Pandora, and other products. Over the last two years, he worked to integrate the best data science and technology from Pandora and Sirius XM into a powerful long-term platform. Today Pandora uses these insights in all aspects of their business. Applications include helping individual people personalize their own stations to their own tastes, tracking what is most successful on which stations, and understanding where, when and how to upsell a customer.
Most of the recommender engine fundamentals Pandora has developed are universally applicable to the personalization problems every business faces. So, the learning is transferrable to industries without such a great head start or access to data science.
For example, one of the big applications of AI and Machine learning at Pandora is to figure out how to upsell a customer on a subscription. In this use case, Scott’s team is using their advanced models to answer difficult questions that are common to every business – Who should we target? When is the right time to reach them? And in what context? Pandora’s models are able to dig into very granular contextual variables from the promotion type, content, product fit, channel of engagement and presentation of an offer.
“We’ve built machine learning models to maximize the effectiveness of how we engage with users in the upsell process,” says Wong. “We call that smart conversations. Our models look at all aspects of user behavior – what they are doing in the app at any point in time? Who are the right people to engage? When is the right time to engage them? Should we be including any kind of promotions at the same time? Our models maximize the effectiveness of those interventions by determining the right subjects, the right topics, and the best artists for that person. We use recommendations inside the content itself.”
Pandora’s approach is unique and in some ways more personalized than those used by peers like Spotify. This is because Pandora’s models are built on a strongly contextual user experience with inputs that are very explicit in terms of the specific albums, artists, and content users prefer. That generates really powerful feedback data for developing models.
Wong paints a long runway and wide array of applications for AI enabled recommender systems going forward. Pandora continues to push the envelope on the next generation of recommender strategies by incorporating search and voice signals. Search provides early signals on trends and event driven actions that make recommendations richer. And advances in voice science allow Pandora to explore thematic queries for even deeper insights into user preferences. And the combination of Sirius XMs audio and broadcast capabilities and Pandora’s digital streaming data sources gives Scott’s team a real edge in evolving their models by cross pollinating listeners experiences across millions of listeners, stations, and inputs. In the end state, Wong envisions a one-to-one relationship between the user and the algorithm that adapts to exactly who they are and what they like.
To help business leaders who don’t have Pandora’s experience and acumen to catch up and apply AI to enable 1:1 personalization at scale, Kartik Hosanager and the team at Wharton’s AI for Business is working with leaders like Scott Wong to create an executive curriculum to help leaders understand how to apply AI and ML to grow their businesses. This curriculum will incorporate the decades of experience of leaders like Pandora to give executives the benefit of millions of hours of modeling, analysis and learning to give them a leg up in the market.
“This specialization will help improve managerial understanding of AI, explore the many ways in which AI is being used across industries, and provide a strategic framework for how to bring AI to the center of digital transformation efforts,” says Professor Hosanagar.
Coursera has just launched the new Wharton course for business leaders: AI Applications in Marketing and Finance online. Managers that want to learn practical ways to apply the power of AI to personalize their customer experiences can register for the complete four-course specialization AI For Business Specialization.
“Just like digital technologies have fundamentally transformed business over the past twenty years, AI is set to do the same over the next twenty years,” advises Professor Hosanagar. “AI is no longer just for engineers and data scientists. It’s for everyone. Professionals can no longer afford to have a poor understanding of something so fundamental to business and society today.”
—————-
Scott Wong Is Vice President Of Machine Learning For Recommendation Search And Voice Science at Sirius XM. His team builds reusable machine learning systems that power recommendations and discovery across Sirius XM, Pandora, and other products. Over the last two years, he worked to integrate the best science data and technology from Pandora and Sirius XM and a powerful long-term platform.
