Redefining Data Management with Dataform for Usay Compare

Our impact

• Enhanced Usay Compare’s analytics and insight capabilities by building and implementing a reliable, scalable and easy-to-use data warehouse in BigQuery
• Used Google Cloud’s latest offering, Dataform, to implement a fully-managed, deeply integrated and serverless data management solution to reduce manual overhead
• Upskilled and enabled Usay Compare to scale the solution in-house to continue to deliver growth and value

 

The challenge

Usay Compare’s goal was to improve how its data was organised to make it reliable and easily accessible. To achieve this, it wanted to develop a future-proof data stack on Google Cloud and data management processes to enable digital efficiency at scale. This would reduce the effort that goes into manual data reporting, eliminate siloed data, and enable better data visualisation. 

Usay Compare needed a data transformation solution that is straightforward to implement and use, without the complications or costs of third-party software. The Usay Compare data landscape required a data warehouse that is capable, but also simple and cost-effective to deploy and use.

After careful consideration, Dataform was highlighted as the best solution for Usay Compare data management needs. However, additional challenges remained:

 

• Dataform was still in Preview*
• Dataform as a managed service is very new and somewhat untested
• Community-driven knowledge base on Dataform isn’t as mature as on other tools

* Dataform has since gone GA, which organically alleviates any further concerns

“Our previous database environment restricted the data team to read-only access, limiting our ability to create bespoke tables within our database, and we were unable to create a data gateway to our reporting platform. This meant we had a reliance on virtual machines running various Excel reports to create our bespoke tables, which were prone to crashing” – Mike Squires, Head of Business Intelligence at Usay Compare

 

Our solution

Datatonic worked with Usay Compare to develop a best-practice Enterprise Data Warehouse on BigQuery, and used Dataform to deliver a deeply-integrated and easy-to-use data model and management solution. This included:

• Developing a deeply integrated solution for building and maintaining data models in BigQuery
• Building data transformation best practices in Dataform to create a high degree of efficiency and reliability in data modelling, with principles such as DRY at the heart of the solution
• Implementing a data testing and assertion framework using Dataform’s testing capabilities
• Proving the capability and value of Dataform through a comprehensive UAT phase, and reassuring Usay Compare around any product maturity concerns
• Implementing a best-practice deployment framework for data product development life cycle in Dataform + BigQuery 
• Upskilling and enabling Usay Compare data teams to continue to build and maintain data models and tests in Dataform

 

“The project team at Datatonic delivered exactly what was promised, providing a data model which we continue to build on since they completed the integration. The training they provided was comprehensive and accessible.” – Mike Squires, Head of Business Intelligence at Usay Compare

With this new solution, Usay Compare has high-quality, accessible and reliable data in its BigQuery data warehouse. Dataform implemented with best practices ensures that data is tested, to make sure it is reliable. This enables the Usay Compare team to increase their efficiency when working with their data to drive valuable business insights. 

About Dataform

Dataform enables users to develop and operationalise scalable data transformation pipelines in BigQuery using SQL. It allows businesses to simplify their data processing architecture, collaborate using software development practices, and build production-grade SQL pipelines. Dataform is simple to set up and often requires fewer steps and less manual effort than with other tooling. Read more here, and contact us to find out more.