Following an Enterprise Engineering project with Datatonic, Sky put big data at the heart of their product strategy, leveraging a system to collect, store and visualise behavioural events from millions of set-top boxes across the UK. With access to a virtually limitless database of customer insights, they are now able to draw insights that will improve the quality of their services faster than ever before.
Before partnering with Datatonic, the company was losing up to 50% of its incoming customer data each day. Their old on-premise system was designed to gather set-top box data, process it hourly, and format it for the Product Research Team daily. However, when their next-generation set-top box hit the market, this system was pushed to breaking point. Their Product Research Team —who rely on this data to suggest product improvements— often lost entire datasets, and what remained was difficult to visualise. The data overload also meant their DevOps teams were constantly reacting to issues and manually operating the system.
We intend to capture all diagnostic and viewing data in GCP and use it to inform decision-making. Management sees this as the beginning of a new era in data management, analytics and data science.
They needed to empower their people and support the company’s future growth plans, which included international expansion and an evolution toward on-demand TV.
Sky’s Director of Customer Insights envisioned their analysts being able to access all kinds of customer data on-demand, and their Director of Data Engineering needed a solution that would operate cost-effectively at scale and eliminate manual DevOps work.
To put big data at the heart of their strategy, they needed a partner capable of designing an enterprise-grade solution, and a track-record for operating at the cutting-edge.
In partnership with Google, Datatonic rose to the challenge as data engineering and analytics experts on Google Cloud Platform. Datatonic has designed and implemented numerous cutting-edge systems in the media space, and are known for delivering highly-stable and reliable solutions.
Datatonic brought Sky’s big data vision to life in just 6 weeks, enabling the Product Research Team to make rapid service improvements and visualise large amounts of data through highly-customised Tableau dashboards. Now, their Data Engineering teams are able to focus on the future, rather than maintaining old infrastructure.
Their new cloud-based solution gathers and stores data from the company’s entire portfolio of products and services. It’s globally distributed, immune to data loss, and regularly processes up to 500 million of events daily.
Datatonic leveraged the full power of GCP to make this possible. PubSub collects set-top box data and sends it to DataFlow, where it’s standardised and enriched, before being saved in BigQuery — their performance-optimised data warehouse. The system automatically scales up-and-down to keep costs in check, and if anything does go wrong Stackdriver instantly alerts the right people. Plus —thanks to DataFlow and Apache Beam— they can easily upgrade or replace individual components in the future.