The Sun, The Times and The Sunday Times, owned by News UK, reach millions of readers every week. To help engage their growing readership, News UK requires a cutting-edge solution that matches articles to readers - all in real-time and at scale. Alongside the News UK Data Science team, Datatonic have been turning this need into a reality by developing and productionizing state-of-the-art Recommender Systems on Google Cloud Platform. These systems incorporate the latest advances in AI, including content recommenders, natural language processing and image processing. As a key requirement throughout the project, Explainable AI features have also been included in the design to understand why recommendations are served to each reader at any given time.
News Corp declared in their 2019 Annual Report that their mission was to “deliver content in a more engaging, timely and personalized manner”. As a key step in this direction, News UK requires greater control in personalizing article recommendations for The Sun, The Times and The Sunday Times. These recommendations will be used in key parts of the user experience – alongside editorial curation – to help readers discover relevant stories.
Their key requirements are:
“We have experimented with personalization in the past, however we were always left frustrated with the opaque and inflexible nature of the solutions. We need to cater for a broad range of user experiences across our different titles, where different levels and types of personalization are appropriate.” – Dan Gilbert, Director of Data, NewsUK.
“At NewsUK, we wanted to develop a set of capabilities that we could control, experiment with, and which provide the transparency we need to explain and monitor how the recommender system is working and why recommendations are being made.” – Dan Gilbert, Director of Data, NewsUK.
Datatonic has helped with the first step in transforming this need into a reality by designing, developing and productionizing content recommender systems with AI explainability on GCP.
Example of Article Recommenders – The Sun and The Times
The team kickstarted the engagement by conducting exploratory data analysis on the user, context and content trends for The Sun and The Times and The Sunday Times. As a result, a suite of features were engineered to use in the Machine Learning models. Advanced feature engineering was additionally tested, such as image processing with the Google Cloud Vision API and text processing with natural language processing techniques.
In the design phase, the models were constructed to maximize business objectives such as readership, user engagement and satisfaction catering to advertisement and subscription-based revenue models for The Sun and The Times and The Sunday Times respectively. For this purpose, a list of the latest Machine Learning methods were surveyed and tested, such as deep learning, the attention mechanism and generalized matrix factorisation.
To speed up model evaluation, the most significant features were prioritized – for example, reader interests and reading behaviors were more relevant than demographics. For the final recommender, adjustable levers were included to prioritize factors such as popularity and diversity.
In order to automate the recommender system (including periodic re-training of the model), Datatonic used an open-source, cloud-agnostic software tool known as Terraform. User and content features were processed with an architecture comprising tools such as Dataflow, Pub/Sub, cloud functions, Compute Engine and Memorystore.
News UK is now experimenting with the recommendation capabilities Datatonic helped to develop on The Sun and will shortly be rolling out the first experiment on The Times and The Sunday Times. Once the initial positive results have been assessed, the capability will be rolled out more widely across Sun and Times products. The News UK team is also looking at how they can bring personalization into their Wireless radio station products such as Virgin Radio, Times Radio and talkSPORT.
“Datatonic brought a wealth of expertise and experience to our team, and collaborated with our engineers and data scientists, working to their standards, so that we can continue to evolve the system they developed.”- Dan Gilbert, Director of Data -