Establishing a Single Source of Truth on Google Cloud for a Global Leader in Cargo Handling

Our impact

• Established a single source of truth, democratised data and embedded best practices in data governance on Google Cloud
• Seamless ingestion and visualisation of a hundred million rows of financial data, from on-prem Hyperion Financial Management to BigQuery, with rigorous access management controls
• Enabled business users to create their own data-driven insights and unlock innovative future use cases, such as demand forecasting and inventory optimisation

 

The challenge

Our client is a global leader in cargo and load handling solutions, and currently operates in over 100 countries with annual revenues of over 3 billion euros.

The business was relying exclusively on Oracle’s on-premise system, Hyperion Financial Management, to store and manage its vast amounts of financial data. This system has been in place for a number of years across the client’s regions and business units. However, as the business evolved, and fostered more advanced business users aiming to work more closely with the data, the on-premise system was limited in accessibility and support to easily share results.

Our client had taken initial steps to rectify this common on-prem challenge. The business had created a trial on Google Cloud with a single HFM dataset, however, this trial was highly manual, and struggled with speed, performance and scalability to cover business-wide datasets. They were therefore looking for an end-to-end solution on Google Cloud that would transform its vast amounts of financial (and highly sensitive) data, and make this data accessible to relevant business users in a scalable, cost-effective and governed way.

 

Our solution

As the 3X Google Cloud Partner of the Year with proven experience in enterprise data warehouse modernisation and cloud data engineering solutions, Datatonic was chosen as tech consulting partner to audit, design and deploy the solution.

Understanding our client’s challenges in data silos, scalability and security, the team identified the following key objectives:

1. Design and deploy an end-to-end data ingestion pipeline from Hyperion to BigQuery 
2. Develop rigorous and fine-grained access management controls, given the sensitivity of the data sets
3. Establish Infrastructure as Code (IaC) to scale and support future ingestion projects

Given the sensitive nature of the financial data being ingested, Datatonic have designed and implemented a robust solution that allows for highly customisable yet rigorous data access controls that are applied on row-level basis in BigQuery, effectively ensuring that the business users have access to just the data they need.

Over the course of 10 weeks, Datatonic answered these requirements by:

1. Reviewing the current data architecture in Oracle Hyperion platform
2. Creating Infrastructure As Code templates using Terraform
3. Creating flexible data pipelines for ingesting large volumes of data from Hyperion
4. Creating an automated workflow for continuously updating access rights in BigQuery
5. Orchestrating the data ingestion and enable delta loads for improved performance
6. Creating a semantic layer in BigQuery enabling users to more easily query the data while enforcing data access on a row-level basis

Since the solution has gone into production on Google Cloud, our client has been enabled with a single source of truth, from executive level to business analyst, across all business areas within the company. The business has the confidence their highly sensitive data is handled correctly from ingestion and storage to consumption and analysis.