Client
Sky
Tech stack
Google Cloud
Solution
Streaming analytics
Service
Data + Analytics
Creating a Powerful Streaming Analytics Platform for Sky
Sky, one of Europe’s largest media and telecommunications companies wanted to improve the quality of their services, fast. Following an Enterprise Engineering project with Datatonic, Sky put big data at the heart of their product strategy, leveraging a system to collect, store and visualise behavioural events from millions of set-top boxes across the UK. With access to a virtually limitless database of customer insights, they are now able to draw insights that will improve the quality of their services faster than ever before.
Our impact
- Enabled over 1 billion+ events to be tracked daily
- Achieved 100% data retention (vs. 50% previously)
- Implemented one globally scalable solution
The challenge
Before partnering with Datatonic, Sky was losing up to 50% of its incoming customer data each day. Their old on-premise system was designed to gather set-top box data, process it hourly, and format it for the Product Research Team daily.
However, when their next-generation set-top box hit the market, this system was pushed to breaking point. Their Product Research Team —who rely on this data to suggest product improvements— often lost entire datasets, and what remained was difficult to visualise. The data overload also meant their DevOps teams were constantly reacting to issues and manually operating the system.
They needed to empower their people and support the company’s future growth plans, which included international expansion and evolution toward on-demand TV. Sky’s Director of Customer Insights envisioned their analysts being able to access all kinds of customer data on-demand, and their Director of Data Engineering needed a solution that would operate cost-effectively at scale and eliminate manual DevOps work.
“We intend to capture all diagnostic and viewing data in Google Cloud and use it to inform decision-making. Management sees this as the beginning of a new era in data management, analytics and data science.” – Oliver Tweedie, Director of Data Engineering, Sky
Our solution
To put big data at the heart of their strategy, Sky needed a partner capable of designing an enterprise-grade solution, and a track record for operating at the cutting-edge. In partnership with Google, Datatonic rose to the challenge as data engineering and analytics experts on Google Cloud.
Datatonic has designed and implemented numerous cutting-edge systems in the media space, and are known for delivering highly-stable and reliable solutions. Datatonic brought Sky’s big data vision to life in just 6 weeks, enabling the Product Research Team to make rapid service improvements and visualise large amounts of data through highly-customised Tableau dashboards.
Now, their Data Engineering teams are able to focus on the future, rather than maintaining old infrastructure. Their new cloud-based solution gathers and stores data from the company’s entire portfolio of products and services.
This new platform:
- Is globally distributed
- Is Immune to data loss
- Regularly processes up to 500 million events daily
Datatonic leveraged the full power of Google Cloud to make this possible. PubSub collects set-top box data and sends it to DataFlow, where it’s standardised and enriched, before being saved in BigQuery – their performance-optimised data warehouse. The system automatically scales up-and-down to keep costs in check, and if anything does go wrong Stackdriver instantly alerts the right people. Plus – thanks to DataFlow and Apache Beam – they can easily upgrade or replace individual components in the future.
