Let's talk

LinkedIn

Twitter

Medium

Youtube

Didn&#8217;t get a chance to attend <a href="https://coalesce.getdbt.com/" target="_blank" rel="noopener">dbt Coalesce</a>?
Hear from Datatonic’s Lead Data Architect, Ash Sultan, DevOps Lead, Alex Peissel, and Senior Cloud Architect, Stan Hill, as they explore using modern data analytics at scale with dbt!
Learn how you can reduce your time to insights, increase the productivity of your data teams, and optimise your costs.
<iframe loading="lazy" title="Enterprise MDS deployment at scale: dbt &amp; DevOps - Coalesce 2023" width="500" height="281" src="https://www.youtube.com/embed/6NbQlbp_JDM?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
<div class="content single-post__content">
&nbsp;
Datatonic, 5x Google Cloud Partner of the year, is the leading cloud data + AI consultancy for the world’s most ambitious brands, challenging traditions to deliver tangible innovation at the leading edge of Google Cloud. Learn about our experience with <a href="https://datatonic.com/partners/dbt/" target="_blank" rel="noopener">dbt</a>, and <a href="https://datatonic.com/contact/" target="_blank" rel="noreferrer noopener">contact us</a> to find out more.
</div>

<h1>Download 
the guide</h1>
A guide by data leaders, for data leaders, looking to turn Data Mesh principles into practice on Google Cloud.

Enterprise MDS Deployment at Scale: dbt + DevOps

Datatonic has won the Google Cloud Services Partner of the Year award for both the UK/IE and Alps regions. This is strong recognition of our ongoing partnerships with leading businesses, such as <a href="https://datatonic.com/case-studies/ai-booster-vodafone-supercharging-ai-ml-at-scale/" target="_blank" rel="noopener">Vodafone</a>, <a href="https://datatonic.com/case-studies/bt-ai-accelerator-artificial-intelligence/" target="_blank" rel="noopener">BT</a>, and <a href="https://datatonic.com/case-studies/marketing-analytics-asos/" target="_blank" rel="noopener">ASOS</a>, and our expansion throughout the Alps working with leading luxury retailers and travel companies. 
Datatonic was also named Google Cloud Industry Solution &#8211; Services Partner of the Year for Gaming, recognizing the impact of collaborations with leading game developers, including our <a href="https://datatonic.com/case-studies/hiber-gaming-generative-ai-world-creation/" target="_blank" rel="noopener">Generative AI text-to-3D-world tool</a> developed in partnership with Hiber. 
Datatonic’s Google Cloud Specialization Partner of the Year win in the Marketing Analytics category demonstrates its commitment to unlocking valuable insights with data + AI to help businesses, such as <a href="https://datatonic.com/case-studies/marketing-analytics-asos/" target="_blank" rel="noopener">ASOS</a>, increase personalization and strengthen the relationships with their customers. 
Solidifying the long-standing partnership and proven expertise in creating impactful solutions for joint clients, Datatonic received two services, one industry and one specialization award:
&nbsp;
<ul>
<li style="font-weight: 400;" aria-level="1">Google Cloud Services Partner of the Year – UK/IE</li>
<li style="font-weight: 400;" aria-level="1">Google Cloud Services Partner of the Year – Alps</li>
<li style="font-weight: 400;" aria-level="1">Google Cloud Industry Solution &#8211; Services Partner of the Year – Gaming</li>
<li style="font-weight: 400;" aria-level="1">Google Cloud Specialization Partner of the Year – Marketing Analytics</li>
</ul>
&nbsp;
<img loading="lazy" decoding="async" class="alignnone wp-image-23622" src="https://cms.datatonic.com/app/uploads/2024/04/Google-Cloud-Partner-of-the-Year-300x102.png" alt="Google Cloud Partner of the Year" width="750" height="255" srcset="https://cms.datatonic.com/app/uploads/2024/04/Google-Cloud-Partner-of-the-Year-300x102.png 300w, https://cms.datatonic.com/app/uploads/2024/04/Google-Cloud-Partner-of-the-Year-1024x347.png 1024w, https://cms.datatonic.com/app/uploads/2024/04/Google-Cloud-Partner-of-the-Year-768x260.png 768w, https://cms.datatonic.com/app/uploads/2024/04/Google-Cloud-Partner-of-the-Year-1536x520.png 1536w, https://cms.datatonic.com/app/uploads/2024/04/Google-Cloud-Partner-of-the-Year-2048x693.png 2048w" sizes="(max-width: 750px) 100vw, 750px" />
&nbsp;
&#8220;We are thrilled to be taking home four awards this year to celebrate the success of our Google Cloud partnership in some of our key markets, but also for the industry and solution impact we&#8217;re making. This recognition is a great boost for our team to keep pushing for maximum customer impact with the rapidly evolving Google Cloud data and AI technology that we have specialized in since 2015.” &#8211; Louis Decuypere, CEO, Datatonic
“Google Cloud&#8217;s Partner Awards celebrate the transformative impact and value that partners have delivered for customers,&#8221; said Kevin Ichhpurani, Corporate Vice President, Global Ecosystem and Channels at Google Cloud. &#8220;We&#8217;re proud to announce Datatonic as a 2024 Google Cloud Partner Award winner and recognize their achievements enabling customer success from the past year.”
Datatonic is leveraging Google Cloud’s latest technologies to enable clients to become data-driven organizations and unlock new business models, services, products and processes, as well as innovating with AI, Machine Learning, and Generative AI.
&nbsp;
About Datatonic
Datatonic is the leading cloud data + AI consultancy for the world’s most ambitious businesses, challenging traditions to deliver tangible innovation at the leading edge of Google Cloud. Datatonic works with industry-leading organizations to unlock the value of data and solve their most complex business challenges fast; delivering more accurate predictions, automating decisions, optimizing processes and improving customer experiences, and helping clients do the same by upskilling tech teams to stay ahead of what’s next.
For more information, visit <a href="https://datatonic.com/" target="_blank" rel="noopener">https://datatonic.com/</a>.
Take a look at our <a href="https://datatonic.com/events/" target="_blank" rel="noopener">upcoming events</a> to learn about Generative AI, Marketing Analytics, and upskill on all things data + AI. 
For press enquiries, contact <a href="mailto:oliver.webb@datatonic.com" target="_blank" rel="noopener">oliver.webb@datatonic.com</a>.

London, April 8, 2024 – Datatonic today announced that it has won four 2024 Google Cloud Partner of the Year Awards, demonstrating its impact and commitment to being the leading data + AI partner for customers to generate business value with Google Cloud, globally.

Datatonic Wins Four 2024 Google Cloud Partner of the Year Awards

Earlier this week, Datatonic and Cognify Search’s Stacked Pathways hosted our Inspiring Women in Data and Analytics panel. Our panelists were five women data leaders: Naomi Johnson, Director of Data Platform at Lyst, Priya O’Shea, Partner Sales Manager at Google Cloud, Jessica Franks, Senior Engineering Manager at Not On The High Street and Rachel Christian, Senior Manager at GoCardless, with Amber Cella, Co-Founder at Datatonic Academy, hosting the panel.
They discussed their career journeys as women, and how they overcame common challenges in their industries. Today, we share their key messages and responses to frequently asked questions from the audience. 
The tech industry, often hailed for its innovation and advancement, has long grappled with gender disparities. Despite progress, women in tech still face numerous challenges, ranging from assumptions about their expertise to the burden of imposter syndrome. Hear from our panelists about challenges they faced, and how they overcame them.
<img loading="lazy" decoding="async" class="alignnone wp-image-23582" src="https://cms.datatonic.com/app/uploads/2024/03/IMG_2070-300x200.jpg" alt="Women in Data and Analytics Panel" width="600" height="400" srcset="https://cms.datatonic.com/app/uploads/2024/03/IMG_2070-300x200.jpg 300w, https://cms.datatonic.com/app/uploads/2024/03/IMG_2070-1024x683.jpg 1024w, https://cms.datatonic.com/app/uploads/2024/03/IMG_2070-768x512.jpg 768w, https://cms.datatonic.com/app/uploads/2024/03/IMG_2070-1536x1024.jpg 1536w, https://cms.datatonic.com/app/uploads/2024/03/IMG_2070-2048x1365.jpg 2048w" sizes="(max-width: 600px) 100vw, 600px" />
<h4>Assumptions and Bias in Expertise</h4>
One recurring theme of the panel highlighted a prevalent issue: the assumption that women are less senior or knowledgeable. 
“With a consulting background, I’ve had experiences in the past where myself and male colleagues have been presenting and even as the most senior person there, questions were directed to male colleagues.” &#8211; Rachel Christian, Senior Manager, GoCardless.
This biased behavior not only undermines women&#8217;s expertise but also starts a cycle where they must make a significant effort to simply prove themselves worthy of respect and recognition.
Jess Franks, Senior Engineering Manager at Not On The High Street, echoed this sentiment, emphasizing the need for women to go above and beyond to prove their capabilities. The transition between managers poses a particular challenge, as preconceived notions can overshadow qualifications that would have been evident in job interviews and formal hiring processes. 
Our speakers stressed the importance of overcoming these biases, urging women to seize opportunities presented to them and to not be afraid to assert their worth, by volunteering to answer questions and proving their knowledge.
<h4>Navigating Male-Dominated Spaces</h4>
Priya&#8217;s narrative shed light on the isolation experienced by women in tech, especially those from underrepresented backgrounds. 
“I’ve often found myself in a room as the only female, only brown person and only person under the age of fifty. It can feel uncomfortable not being around people with similar backgrounds and having to mention or subtly show how long you’ve been there instead of people thinking you’re the new intern.” &#8211; Priya O’Shea, Partner Sales Manager, Google Cloud
As the only female or minority in the room, it can be easy to feel additional scrutiny and the need to validate your presence. This emphasizes the importance of intentional networking and seeking support from communities and people in similar positions. By actively engaging with peers and mentors, women can create meaningful connections vital for professional growth, and become more confident in these situations. 
<h4>Confronting Imposter Syndrome</h4>
Imposter syndrome, a pervasive challenge for women in tech, undermines confidence and stifles progress. Naomi Johnson, Director of Data Platform at Lyst, reflected on her career journey, acknowledging the need to combat self-doubt and embrace your achievements. This can be done by maintaining a strong internal dialogue and keeping a record of positive outcomes at work to remind yourself of your success and abilities. 
A Hewlett Packard internal report found that men are significantly less likely to self-reject when applying to jobs; men apply for a job when they meet only 60% of the qualifications, but women are only likely to apply only if they meet 100% of them.
“I spent so many years being terrified and feeling like I got lucky, thinking that if people look hard enough they’d see I’m not qualified. I didn’t get lucky, I’m smart, I earned my way here. Enough of the imposter syndrome.” &#8211; Rachel Christian, Senior Manager, GoCardless.
Rachel&#8217;s advice to her younger self resonates with many women battling imposter syndrome. She urges them to recognize their worth and reject the notion of luck by reminding yourself that even if you are the only woman in the room, you earnt your way there. By embracing their intelligence and placing emphasis on accomplishments, women can shatter the idea of self-doubt and thrive in their careers with confidence.
<h4>Embracing Opportunities and Growth</h4>
One of the key themes that emerged was the need to seize opportunities and embrace growth, despite fears or setbacks. Whether it&#8217;s confronting public speaking or pursuing new endeavors, there is transformative power in stepping outside of your comfort zone. 
“I hate public speaking but I’ve been putting myself forward for things ever since I was first asked to speak. I still feel nervous when I do it, but a little bit less, and I’ve met so many amazing people through doing it. When it scares you it’s probably what you should be trying to do.” &#8211; Naomi Johnson, Director of Data Platform, Lyst
Priya&#8217;s advice highlights the importance of perseverance and self-assurance, reminding women that career paths are not linear and comparing ourselves to our colleagues or friends is not always useful. 
“It&#8217;s not always helpful to compare yourself to other people. If it sounds good to you, go and do it. If you’re passionate about it or have an interest in it, your ability to succeed will be so much greater. Stop worrying about what you’re leaving behind and focus on what will make you happy. You have to enjoy what you do.” &#8211; Priya O’Shea, Partner Sales Manager, Google.
<h4>Conclusion</h4>
Our speakers provided many perspectives on the challenges and triumphs of women in the tech industry, but the same recurring themes shone through. Despite facing biases, isolation, and self-doubt, these women navigate their careers with resilience. Through mentorship, community support, and self-advocacy, they pave the way for future generations of women in tech, challenging norms and fostering a more inclusive and equitable industry. 
Find out more about <a href="https://cognifysearch.com/stackedpathways/" target="_blank" rel="noopener">Stacked Pathways</a> mentoring, and take a look at our <a href="https://www.eventbrite.co.uk/o/datatonic-academy-ltd-62058669953" target="_blank" rel="noopener">upcoming free training sessions</a> to see how you can learn more to take the next step in your data + AI career.
Looking to take your next step? Take a look at our <a href="https://datatonic.com/careers/vacancies/" target="_blank" rel="noopener">current vacancies</a>.
Datatonic is Google Cloud’s 5x Partner of the Year with a wealth of experience developing data + AI solutions for leading businesses. <a href="https://datatonic.com/contact/" target="_blank" rel="noopener">Get in touch</a> to kickstart your data + AI journey.

Coding Confidence: Inspiring Women in Data and Analytics

Every day, over 1 billion people globally use streaming services to find their next favorite movies, TV shows, and music albums. Cineverse wanted to take the searching experience to the next level. 
The growing abundance of content and content sources is becoming overwhelming for audiences; users now spend an average of <a href="https://www.nielsen.com/insights/2023/data-driven-personalization-2023-state-of-play-report/" target="_blank" rel="noopener">10-and-a-half minutes</a> searching for something to watch, with 20% abandoning their search and doing something else instead.
With this in mind, Cineverse, Datatonic, and Google Cloud set out to develop an AI-powered search and discovery tool designed to make deciding what to watch as entertaining as the movie itself. 
Its new &#8220;artificial intelligence-based video advisor&#8221; will be launched initially with the persona known simply as &#8220;Ava.&#8221; The primary persona, Ava, was developed to be the ultimate movie nerd: Part critic. Part drinking buddy. All cinephile. Future updates will involve building out a collection of personas to suit different users&#8217; tastes and moods. 
The Ava persona, including its personality and language, was meticulously created by Cineverse, and brought to life through a combination of prompt engineering and LLM fine tuning. 
<img loading="lazy" decoding="async" class="alignnone wp-image-23516" src="https://cms.datatonic.com/app/uploads/2024/03/Cineverse-Icons-300x169.png" alt="Cineverse Generative AI Icons" width="625" height="352" srcset="https://cms.datatonic.com/app/uploads/2024/03/Cineverse-Icons-300x169.png 300w, https://cms.datatonic.com/app/uploads/2024/03/Cineverse-Icons-1024x576.png 1024w, https://cms.datatonic.com/app/uploads/2024/03/Cineverse-Icons-768x432.png 768w, https://cms.datatonic.com/app/uploads/2024/03/Cineverse-Icons-1536x864.png 1536w, https://cms.datatonic.com/app/uploads/2024/03/Cineverse-Icons-2048x1152.png 2048w" sizes="(max-width: 625px) 100vw, 625px" />
“Built entirely on Google Cloud, cineSearch leverages a sophisticated system of multiple Large Language Models that guides users through a highly personalized user journey to deliver content recommendations tailored to individual preferences. Under the hood, Ava’s Large Language Models and semantic retrieval methods recognize both highly specific, such as genre, decade and actors, and nuanced user intents, such as cinematic mood and atmosphere, as well as combinations of these. 
A collection of guardrails and Google Cloud security best practices ensure the solution is safe, robust and ethical. A monitoring and alerting system scans both user inputs for malicious attacks, inappropriate and off-topic messages, and LLM outputs for unintended behaviors.
The heart of the technology lies in creating a natural and engaging interaction that is designed to ensure the users feel comfortable and empowered throughout their discovery process. We are confident that cineSearch can satisfy even the most eclectic cinematic tastes.” &#8211; Valentin Cojocaru, Head of Data Science, Datatonic
To ensure cineSearch fulfills its purpose, it was developed with crucial features, such as guardrails and LLM best practices to reduce risk for Cineverse by avoiding unintended replies from the chatbot and keeping the conversation on the topic of movies. cineSearch will also provide direct links to where its recommendations can be purchased or streamed.
&#8220;Effective search and discovery is currently the most pressing problem for users of streaming services today,&#8221; says Tony Huidor, Chief Technology Officer &amp; Chief Operating Officer of Cineverse. &#8220;We first developed cineSearch as an answer to our own problem. Using Google Cloud&#8217;s Vertex AI platform, we now have the ability to expand this feature well beyond our initial expectations. We feel this is a great example of leveraging the power of AI to not only create better user experiences, but to also help aid in the discovery of great films that fans may not have otherwise found.&#8221;
&#8220;Generative AI has the potential to fundamentally transform the media industry, fueling a new era of cloud-enabled capabilities that can significantly improve the consumer experience,&#8221; said Anil Jain, Global Managing Director of Strategic Consumer Industries at Google Cloud. &#8220;By leveraging Google Cloud&#8217;s leading gen AI capabilities, cineSearch can further personalize and streamline the user experience, providing viewers a vastly improved way to discover new content, with relevant recommendations for movies and television shows at their fingertips.&#8221;
The partnership will enable Cineverse to rapidly embark on new data + AI initiatives, leading to improved customer experience, and operational efficiencies.
“Cineverse is thrilled to partner with Google and Datatonic for the cineSearch project,” Huidor says. “We needed all hands on-deck to do something that’s never been done before. Internally, we’ve joked that it was our ‘Avengers, assemble’ moment – bringing together the best in both streaming and GenerativeAI, and I’m excited for everyone to see what we’ve been working on.” 
To keep up to date with cineSearch and try it out for yourself, join the <a href="https://cinesearch.com/waitlist/" target="_blank" rel="noopener">public beta waitlist</a>. And if you’re in New York, come along to our Generative AI panel with Cineverse’s Senior Vice President, Marketing, Lauren McCarthy, to learn more. <a href="https://genai-in-the-cineverse-dt.rsvpify.com/" target="_blank" rel="noopener">Register here</a>.

Datatonic is thrilled to announce its partnership with leading global streaming technology and entertainment company, Cineverse, to develop its Generative AI-powered movie search platform, cineSearch, using Google Cloud’s Vertex AI Search.

Datatonic Partners with Cineverse to Deliver Generative AI Movie Recommendations

Author: Thomas Udale, Senior ML Trainer
Prompt engineering has quickly become a hot topic in the world of Large Language Models (LLMs) for a good reason; without it, the models are much less capable of generating relevant outputs. Prompt engineering gives users and enterprises the tools to tune both the knowledge base of the model and the output it produces. In this blog, we explore the relevance of prompt engineering and look at how consumers and enterprise users can improve LLM output to create more value with Generative AI. 
<h3>What is prompt engineering?</h3>
<a href="https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-prompt-engineering" target="_blank" rel="noopener">Prompt engineering</a> is “the practice of designing inputs for generative AI tools that will produce optimal outputs”. This allows businesses to provide the LLM with additional information and context, that improves the quality and relevance of its responses to questions or tasks.
<img loading="lazy" decoding="async" class="alignnone wp-image-23474" src="https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-300x143.png" alt="Prompt Engineering Example" width="773" height="368" srcset="https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-300x143.png 300w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-1024x486.png 1024w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-768x365.png 768w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-1536x730.png 1536w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering.png 1600w" sizes="(max-width: 773px) 100vw, 773px" />
Traditional (if two years is long enough for traditions) prompt engineering focused on changing the user prompt to change the content, style, and format of model responses. Adding in context, asking for bullet points, or even adding a few examples of prompt-response pairs are all ways in which we can change the model output.
These are time-efficient methods for improving the quality of response by bringing output closer to our implicit expectations. We can even use prompts to guide the model into better logical reasoning by explaining the required thinking process, through <a href="https://www.promptingguide.ai/techniques/cot" target="_blank" rel="noopener">Chain-of-Thought (CoT) prompting</a>. 
<h3>Why do we need prompt engineering?</h3>
The key idea in traditional prompt engineering is to better leverage the internal knowledge in the models, gleaned from billions of training sentences. The models have internalized a great number of patterns from this training data, including any biases, false information, and inappropriate language! Guiding the model to replicate patterns closer to our desired output is the end goal of traditional prompt engineering. 
<img loading="lazy" decoding="async" class="alignnone wp-image-23473" style="font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif;" src="https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-2-300x187.png" alt="Prompt Engineering Example" width="483" height="301" srcset="https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-2-300x187.png 300w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-2-1024x640.png 1024w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-2-768x480.png 768w, https://cms.datatonic.com/app/uploads/2024/02/Prompt-Engineering-2.png 1231w" sizes="(max-width: 483px) 100vw, 483px" />
Training LLMs is difficult. It required vast quantities of data, data centers full of GPUs and TPUs whirring away for days on end, as well as strong technical knowledge requirements around networking to move data and coordinate hardware for training. 
This is why larger companies, such as Google have produced many of the top-performing models of the last few years. This also means that they have control over the data that is fed into the models.
<h3>The history of LLMs and prompt engineering</h3>
Looking at the context window of LLM models helps us to understand why prompt engineering is now so relevant. In 2017, an early LLM, <a href="https://en.wikipedia.org/wiki/BERT_(language_model)" target="_blank" rel="noopener">BERT</a>, had a context window of 512 tokens. That meant you could feed roughly 500 words into your model at once. This is a big limitation that would be overcome as models became ever larger. 
The largest models now have context windows of over 1 million tokens, which means you can feed novels into them. This has also opened the door to bringing in external knowledge to an LLM via the prompt. 
<h3>The impact of prompt engineering</h3>
Techniques such as retrieval augmented generation (RAG) allow models to bring in relevant information from external sources into the prompt, where the model can be asked to make sense of and use the new text. 
Without having to retrain the model (a time-consuming, expensive process), enterprises can leverage their own unstructured data to improve LLM output. This is the real step forward in prompt engineering. Expanding the set of patterns the model has access to, with information particularly relevant to your task is a vital way to bring enterprise use cases, such as chatbots, knowledge workers, and analytics assistants, to life. 
<h3>Conclusion</h3>
To summarise, prompt engineering broadly falls into two categories. Firstly, to make better use of the internalized patterns from the training data. Second, to help bring in external knowledge from company documentation, enhancing the patterns that the model can use to generate responses. With prompt engineering, users can receive more relevant and useful outputs from LLMs, increasing efficiency and giving them the information they need faster. 
&nbsp;
Want to get started? Secure your spot on our free Prompt Like a Pro taster session and embark on a transformative journey into the world of Generative AI. Visit our <a href="https://datatonic.com/events/" target="_blank" rel="noopener">event page</a> for more information and to register.
Turn Generative AI hype into business value with our workshops and packages. <a href="https://datatonic.com/contact" target="_blank" rel="noopener">Get in touch</a> to discuss your Generative AI, ML, or MLOps requirements!

Prompt Engineering 101: Using GenAI Effectively

To remain competitive, businesses strive to stay ahead by embracing cloud technologies and optimizing their data infrastructure. A successful data migration to Google Cloud can lead to many benefits, such as increased efficiency, higher team satisfaction, and unlock new streams of revenue with advanced analytics and AI. 
In this blog, we’ll look at data migration, shedding light on the common challenges that organizations face. From handling skills gaps when <a href="https://datatonic.com/events/driving-analytics-adoption-with-admiral-insurance/" target="_blank" rel="noopener">adopting the new platform</a> to addressing security concerns and mitigating downtime, each data migration stage presents unique hurdles that impact its overall success. We’ll look at how to overcome or avoid these challenges and how Datatonic’s Turbo Data Migration Framework ensures a seamless data migration.  

&nbsp;
<img loading="lazy" decoding="async" class="alignnone wp-image-23450" src="https://cms.datatonic.com/app/uploads/2024/02/Turbo-Data-Migration-Framework-300x226.png" alt="" width="700" height="527" srcset="https://cms.datatonic.com/app/uploads/2024/02/Turbo-Data-Migration-Framework-300x226.png 300w, https://cms.datatonic.com/app/uploads/2024/02/Turbo-Data-Migration-Framework-1024x772.png 1024w, https://cms.datatonic.com/app/uploads/2024/02/Turbo-Data-Migration-Framework-768x579.png 768w" sizes="(max-width: 700px) 100vw, 700px" />
&nbsp;
<h3>1. Skill Gaps</h3>
As data migration involves transferring data from one system to another, it often requires <a href="https://datatonic.com/insights/building-data-fluency-throughout-your-business/" target="_blank" rel="noopener">expertise in both the source and target systems</a>, as well as a comprehensive understanding of the data structures and formats involved. 
In this complex process, the absence of teams skilled in these systems can lead to various complications. Without individuals proficient in data mapping, extraction, transformation, and loading (ETL) processes, businesses may encounter difficulties in preserving data integrity, ensuring accuracy, and mitigating potential data quality issues. 
Furthermore, a lack of proficiency in data security and compliance may expose companies to risks, such as data breaches or regulatory non-compliance. 
Bridging these skill gaps requires a multidisciplinary approach, focusing on upskilling, such as through co-development or <a href="https://datatonic.academy/" target="_blank" rel="noopener">ongoing training</a> during the migration to ensure everyone has the skills required to leverage the new platform. 
“Datatonic embraces a collaborative approach, taking our clients&#8217; teams on the migration journey with us. By the project’s conclusion, we aim to ensure we leave behind a high-performing team, well-equipped to handle the maintenance of a state-of-the-art cloud platform.” &#8211; Tom Pindar, Business Intelligence Lead 
&nbsp;
<h3>2. Setting unrealistic expectations</h3>
Setting unrealistic expectations and timelines can pose a significant challenge during data migration. Eager to get onto a new cloud platform, businesses may underestimate the complexity involved in transferring large volumes of data between systems, leading to overly optimistic expectations regarding the duration and seamless execution of the migration. 
Unrealistic timelines can result in rushed decision-making, and overlooking crucial steps such as data validation, testing, and troubleshooting. Down the line, this may cause data corruption or data loss, or more downtime, creating frustration among end users.
“We have witnessed first-hand the repercussions of setting unrealistic expectations reverberate across projects, manifesting in budget overruns, compromised quality, and employee burnout. These three consequences are deeply intertwined: a hurried attempt to adhere to budget constraints often leads to compromises in quality, while the strain of unrealistic deadlines diminishes team morale. Embracing a realistic outlook on project complexity and timelines will not only ensure the delivery of a high-quality product but also motivate and engage a workforce, yielding long-term benefits for the company.”  &#8211; Tom Pindar, Business Intelligence Lead 
To overcome this challenge, it is crucial to establish transparent communication channels, educate stakeholders about the intricacies of data migration, and conduct thorough assessments to accurately estimate the time and resources required for a successful and smooth transition.
With Datatonic’s Turbo Data Migration Framework and years of experience, we have developed a repeatable approach. This includes a comprehensive planning phase and templates that make it easy to understand the timeframe and budget requirements for a successful migration. 
&nbsp;
<h3>3. Not having a clear understanding of business value</h3>
Without a well-defined understanding of specific business objectives and the value that data migration is expected to bring, businesses risk investing resources into a process that may not align with strategic goals. 
This lack of clarity can result in misguided priorities, leading to the migration of irrelevant or obsolete data, and may overlook opportunities for strategic data projects or transformations that could enhance business insights, such as advanced analytics with <a href="https://datatonic.com/partners/looker/" target="_blank" rel="noopener">Looker</a>, or advanced AI + ML. 
Key stakeholders must be engaged early in the process to conduct a comprehensive analysis of business requirements and establish clear metrics for success, to ensure that the data migration aligns closely with the overall business objectives and delivers tangible value to the organization.
&nbsp;
<h3>4. Minimizing security risks</h3>
Addressing security concerns is paramount in ensuring a seamless migration without compromising sensitive information. To mitigate these challenges, businesses must implement robust security measures, including encryption protocols, an <a href="https://datatonic.com/insights/secure-efficient-access-control-plan-bigquery/">access control plan</a>, and comprehensive auditing mechanisms. 
Additionally, thorough planning and risk assessments should be conducted before starting any data migration project, identifying potential areas of vulnerability and devising strategies to address them. By prioritizing security in the data migration process, organizations can safeguard valuable information and maintain the trust of stakeholders and end users of the platform. 
“A single breach of security can have far-reaching consequences, impacting not only the organization directly but also its partners and customers. Datatonic has vast experience in implementing Google Cloud&#8217;s wide range of security tools, including but not limited to encryption, identity and access management, threat detection, and compliance controls. These features are designed to address the most pressing security concerns, while providing scalability and flexibility to adapt to changing needs.”  &#8211; Tom Pindar, Business Intelligence Lead 
&nbsp;
<h3>5. Ensuring high data quality</h3>
Inaccurate, incomplete, or inconsistent data can lead to a multitude of problems, impacting decision-making, customer experience, and overall business efficiency. These issues are key drivers for migration in the first place. 
During migration, data may encounter compatibility issues between old and new systems, resulting in data quality degradation. As well as this, duplicate records, missing information, or formatting discrepancies can arise, requiring meticulous attention to detail and comprehensive data-cleansing strategies.
Organizations must invest time and resources in profiling, cleaning, and validating data to ensure that the migrated information meets the highest standards of accuracy and reliability. By addressing data quality issues head-on, businesses can optimize the benefits of migration, enhance operational performance, and make informed decisions based on new trustworthy data. This can then be enhanced with analytics and visualization, using tools such as <a href="https://datatonic.com/partners/looker/" target="_blank" rel="noopener">Looker</a>. One example of this is ASOS, which uses BigQuery and Looker as its <a href="https://datatonic.com/case-studies/marketing-analytics-asos/" target="_blank" rel="noopener">marketing analytics platform</a>.
&nbsp;
<h3>Conclusion</h3>
Migrating to Google Cloud can unlock rapid value for your business, providing better access to crucial data to inform decision-making. It also helps to prepare businesses for advanced analytics and AI to further increase efficiency and enhance customer experiences, opening the door for Generative AI. By understanding the challenges of data migration, companies can plan ahead to prevent any issues.
Datatonic’s Turbo Data Migration ensures a seamless migration to Google Cloud, taking necessary planning measures to minimize risk, and quickly addressing challenges as they arise. Read about our Enterprise Data Warehouse experience, with <a href="https://datatonic.com/case-studies/old-data-new-tricks/" target="_blank" rel="noopener">Pets at Home</a>. 
Datatonic, 5x Google Cloud Partner of the Year, is the leading cloud data + AI consultancy for the world’s largest businesses, delivering tangible innovation at the leading edge of Google Cloud. Learn about our experience with <a href="https://datatonic.com/partners/google-cloud/" target="_blank" rel="noopener">Google Cloud</a>, and <a href="https://datatonic.com/contact/" target="_blank" rel="noopener">contact us</a> to find out more.

Future-Proof Your Data: Overcoming Migration Challenges

Designing a Secure + Efficient Access Control Plan for BigQuery

Proven Turbo Data Migration Framework propels global giants in Telecoms, Financial Services, Manufacturing, Healthcare and Life Sciences, and Retail towards rapid AI success
&nbsp;
January 24th, 2024 &#8211; 5X Google Cloud Partner of the Year, Datatonic, today announced its Turbo Data Migration Framework, leveraging its experience in delivering complex data migrations for major enterprises. Together with Google Cloud, they are helping enterprises drive measurable business value through rapid, cost-effective migrations to fuel AI success.
<a href="http://datatonic.com/contact" target="_blank" rel="noopener">Find out</a> how the Turbo Data Migration Framework can be applied to your business too.
<h4>What is the Turbo Data Migration Framework?</h4>
Datatonic’s proprietary Turbo Data Migration Framework and pre-built accelerators take you from evaluation to activation, fast. Based on several successful data migrations, across multiple industries (Telecoms, Financial Services, Manufacturing, Healthcare and Life Sciences, and Retail), the framework ensures reliability, and scales to your unique needs and team structures.
<h4>Why Turbo?</h4>
Seamlessly integrating Datatonic’s experts with your internal resources or chosen partners, this approach leverages accelerators, pre-built templates, and best practices, finding the optimal balance between guidance and existing capabilities to deliver fast, cost-effective migrations.
Turbo Data Migrations take a co-development approach, which, coupled with hands-on instructor-led training by <a href="https://datatonic.academy/" target="_blank" rel="noopener">Datatonic Academy</a> allows businesses to minimize disruption and fully upskill to maximize value from the migration. Trained teams are 80% faster to adopt cloud, and 2 X more likely to go beyond limited deployment, identifying further use cases for value creation.
<h4>Which businesses has Datatonic turbocharged?</h4>
Datatonic has guided global giants through their toughest cloud journeys, laying the foundation and unlocking the value of data for enterprises, including:
<ul>
<li style="font-weight: 400;" aria-level="1">AWS to Google Cloud: Accelerating the migration of a leading telecom provider’s Big Data Platform from AWS to Google Cloud, enabling 20+ consumer-facing use cases to improve customer experience.</li>
<li style="font-weight: 400;" aria-level="1">Hadoop to Google Cloud: Migrating 150+ highly scientific and unique data pipelines for a multinational healthcare company to empower researchers to operate a world-class R&amp;D function, streamlining costs and leveraging scalability.</li>
<li style="font-weight: 400;" aria-level="1">Snowflake to Google Cloud: Migrating a leading commerce platform provider to Google Cloud in just two months, reducing data warehouse costs by 25%.</li>
<li style="font-weight: 400;" aria-level="1">Customer Data Platform Build: Aggregating 20+ marketing data sources for leading UK retailer, <a href="https://datatonic.com/case-studies/old-data-new-tricks/" target="_blank" rel="noopener">Pets at Home</a>, to enhance customer experience.</li>
</ul>
&nbsp;
Prepare for what’s next: Start with a <a href="https://datatonic.com/services/cloud-data-migration/data-warehouse-modernisation/" target="_blank" rel="noopener">Data Warehouse Modernization Workshop</a> or <a href="http://datatonic.com/contact" target="_blank" rel="noopener">contact our experts</a> to discuss your unique migration journey. 
Want to upskill before you get started? Register for our upcoming free training sessions: 
26th January: <a href="https://www.eventbrite.co.uk/e/google-cloud-fundamentals-core-infrastructure-tickets-780020902517?aff=oddtdtcreator" target="_blank" rel="noopener">Google Cloud Fundamentals: Core Infrastructure</a>
31st January: <a href="https://www.eventbrite.co.uk/e/introduction-to-generative-ai-tickets-780130299727?aff=oddtdtcreator" target="_blank" rel="noopener">Introduction to Generative AI</a>
9th February: <a href="https://www.eventbrite.co.uk/e/prompt-like-a-pro-tickets-780171543087?aff=oddtdtcreator" target="_blank" rel="noopener">Prompt Engineering: Prompt Like a Pro</a>

Datatonic Signs Up Major Enterprises with New Turbo Data Migration Framework

Datatonic is excited to announce the launch of our highly anticipated training course, &#8220;Prompt Like a Pro”. In today&#8217;s dynamic landscape, where innovation is key to staying ahead, mastering <a href="https://datatonic.com/services/generative-ai/" target="_blank" rel="noopener">Generative AI</a> is crucial for professionals seeking to revolutionize their use of data and artificial intelligence. <a href="https://www.eventbrite.com/e/780171543087?aff=oddtdtcreator" target="_blank" rel="noopener">Register today</a> for our free taster session on 9th February.
<h3>Course Overview</h3>
Prompt Like a Pro is Datatonic’s comprehensive training program designed to equip users with the skills and knowledge needed to harness the full capabilities of Generative AI and prompt engineering. 
Whether you&#8217;re a Data Scientist, Machine Learning Engineer, or business professional, this course will help you to unlock the power of Large Language Models (LLMs) through advanced prompting techniques, including best practices for Prompt Engineering and advanced Chain-of-Thought Prompting.
<h3>Highlights</h3>
<ul>
<li style="font-weight: 400;" aria-level="1">Expert-Led Sessions: Learn directly from seasoned professionals in the field who bring a wealth of practical experience, enabling you to gain industry-relevant knowledge.</li>
<li style="font-weight: 400;" aria-level="1">Cutting-Edge Curriculum: Our course is crafted by industry experts and covers the latest advancements in Prompt Engineering and LLMs, ensuring you stay at the forefront of this technology.</li>
<li style="font-weight: 400;" aria-level="1">Industry-Relevant Case Studies: Explore real-world examples and case studies from various industries, providing valuable insights into how LLMs are transforming businesses globally.</li>
<li style="font-weight: 400;" aria-level="1">Hands-On Exercises: Put theory into practice through hands-on labs that simulate real-world scenarios, allowing you to build a robust skill set that can be immediately applied to your professional projects.</li>
<li style="font-weight: 400;" aria-level="1">Networking Opportunities: Connect with like-minded professionals and experts in the field during interactive sessions, fostering collaboration and expanding your professional network.</li>
</ul>
&nbsp;
<h3>Who Should Attend?</h3>
This session is especially relevant for:
<ul>
<li style="font-weight: 400;" aria-level="1">GenAI beginners who want to gain a better understanding of Prompt Engineering techniques applied to Large Language Models (LLMs), including but not limited to Developers, Data Scientists, Machine Learning Engineers, Technical Managers, and Business Managers.</li>
<li style="font-weight: 400;" aria-level="1">Professionals looking to unveil the full potential of LLMs through the most advanced techniques of Prompt Engineering and Prompt Design. </li>
<li style="font-weight: 400;" aria-level="1">Individuals who are familiar with Machine Learning or deep learning concepts and want to expand their knowledge to LLMs and Prompt Engineering.</li>
</ul>
&nbsp;
<h3>Register today!</h3>
Secure your spot on our free Prompt Like a Pro taster session and embark on a transformative journey into the world of Generative AI. Visit our <a href="https://www.eventbrite.com/e/780171543087?aff=oddtdtcreator" target="_blank" rel="noopener">event page</a> for more information and to register. We look forward to welcoming you to our Prompt Engineering community! 
Don&#8217;t miss this opportunity to stay ahead in this rapidly evolving area of AI. <a href="https://datatonic.academy/courses/#contact-section" target="_blank" rel="noopener">Get in touch</a> to book private instructor-led training classes for your team.

Maximize the Value of GenAI: Register to Prompt Like a Pro

Data is one of the most important assets in a business. Data fluency is about being able to describe and interact with your data to bring value to the business, accurately and efficiently. Currently, only <a href="https://www.gov.uk/government/publications/quantifying-the-uk-data-skills-gap/quantifying-the-uk-data-skills-gap-full-report" target="_blank" rel="noopener">58% of businesses</a> think that their organisation has sufficient data skills to meet their current and future needs. Without fluency, data remains an untapped resource, and decision-making across the business is limited. To find out more, we spoke to Data Science + ML Trainer, Tom Udale.
<h4>Part 1: What is data fluency?</h4>
What do we mean when we talk about data fluency and why is it important?
“The first main part is describing data; employees should be able to identify the data within their area of the organisation and also be able to understand fundamental statistics about their data. And the second main area is interacting with that data. Employees should be able to identify issues with that data and know how to resolve them (or who to ask!)”
“This also means employees should be able to manipulate data to answer business questions and suggest ways to use their data to technical teams.”
While technical teams need to understand and be able to explore their data, for truly data-driven businesses, this also applies to business users.
&nbsp;
Does data fluency need to be business-wide?
“Data science teams don’t have all the business knowledge of users. Each employee will have specific knowledge about their area of expertise, which allows them to identify opportunities within data.
The amount of data in an organisation is massive! A small team of analysts and data scientists won’t be able to keep track of all the data generated in a company, so having active data citizens is vital.”
This means that business users will be able to identify innovative and realistic opportunities. If business users aren’t aware of what can be done with data, they won’t raise it with the technical teams. Equally, users need to be aware of what makes a good data opportunity, either for analytics, advanced machine learning or AI.
<h4>Part 2: How to build data fluency</h4>
What are the challenges to building data fluency?
&nbsp;
Building data fluency is a challenge that needs to be approached strategically and with best practices in mind. Here are four key attributes of successful data fluency development: 
1. Large-scale learning
Data fluency is a programme which needs to upskill whole organisations, which often have different schedules and restrictions on time. With proper planning, entire organisations can be prepared to use data effectively.
2. Specialized
The programme should be specific to different departments (Business, HR, Finance), with tailored use cases and exercises to show real-life examples and prepare employees with the most required skills for their role. 
3. At the right level
The programme should not focus too much, or spend too much time on technical implementation. Rather, it should focus on identifying business value, and understanding how data fluency can be used to make a difference.
4. Continuous training + upskilling
Instead of a one-off session, the programme should enable new employees on an ongoing basis, with a rolling programme to facilitate continuous improvement and skill retention.
&nbsp;
How can training solve your fluency shortfall?
“Datatonic Academy can align training to business objectives around data specific to your business. Training can be further tailored to each business unit, with relevant examples and exercises, and can be delivered in both an on-demand or instructor-lead format allowing for flexibility around users&#8217; other commitments”.
Furthermore, we can provide materials for ongoing in-house training, as well as create “super-users” within the organisation who can upskill colleagues, including new joiners.
<h4>Part 3: Leveraging data fluency to create AI + ML innovation in your business</h4>
It’s important to consider, once your team has been trained and upskilled: how can businesses measure a successful data fluency programme? And what is the impact we’ll see in the long run? 
There are three main areas that businesses who immerse themselves in a successful data fluency programme can expect to see:
&nbsp;
1. Data Opportunities
With a strong data fluency programme in place, you would see more opportunities for analytics and ML to come to the technical team. This will lead to higher productivity and efficiency
2. Higher quality data
As business users who interact with datasets across the organisation grow in fluency, they will be able to identify issues with data and help technical teams resolve problems.
3. Data-first culture
If the entire organisation is aware of the value of data, and how that can be realised, then business challenges of all types will be addressed with the relevant data at hand. This means faster and more accurate decision-making throughout your company.
&nbsp;
How can training accelerate your ML workflow and generate value for your business?
“Training doesn’t just have an impact at the foundational and cultural level within an organisation, realising the potential of AI, machine learning and generative AI all require a base level of knowledge, along with continuous upskilling and training.”
Strong data fluency and data foundations make it much easier for people to first identify new ways of using their data, and how this can be done with AI + ML, and then much easier to get started as the data is ready to use.
&nbsp;
Conclusion
Datatonic Academy can provide training at all levels, from fluency to state-of-the-art ML techniques, from experimentation to <a href="https://datatonic.com/events/mlops-101-ml-at-scale/">MLOps</a> and from on-premises to the cloud. As tools and technologies improve, help your workforce improve and stay motivated in AI and ML. See our upcoming public training events <a href="https://datatonic.com/events/">here</a>. 
&nbsp;
Find out more about <a href="https://datatonic.academy/">Datatonic Academy</a> and <a href="https://datatonic.academy/courses/#contact-section">get in touch</a> to book a free scoping session to kick-start your journey to building data fluency and secure 20% off any training courses booked before the end of January 2024.

Building Data Fluency Throughout Your Business

MLOps encompasses the set of practices, technologies, and culture that enable Machine Learning to deliver real business value in an organisation. Borrowing concepts from DevOps and applying them to Machine Learning, MLOps enables teams to move rapidly from offline experimentation to a deployed production system and beyond, unlocking business potential from Data Science.
In this webinar, Datatonic’s Machine Learning Engineering Lead, Felix Schaumann and Google Cloud&#8217;s Customer Engineer, Ivan Nardini, show you how to use <a href="https://datatonic.com/insights/datatonic-updated-turbo-templates-mlops-adoption-2024/" target="_blank" rel="noopener">Datatonic&#8217;s new Turbo Templates</a> to kickstart your Machine Learning development. 
&nbsp;
<iframe loading="lazy" title="MLOps in action: Operationalize your ML workflow using pipeline templates" width="500" height="281" src="https://www.youtube.com/embed/SdR5P2BDGow?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
&nbsp;
Looking to get started with MLOps but need help finding where to start? Datatonic is Google Cloud&#8217;s 5x Partner of the Year and 2x Machine Learning Partner of the Year. <a href="https://datatonic.com/contact/" target="_blank" rel="noopener">Get in touch</a> to start planning your next data + AI project.

MLOps in Action: Operationalize Your ML Workflow Using Pipeline Templates

London, 21st November 2023 &#8211; Datatonic announced today that it has released a new version of its open-source MLOps Turbo Templates, developed in partnership with Google Cloud’s Vertex Pipelines product team, to help data teams kickstart their Vertex AI MLOps initiatives. Customers like <a href="https://datatonic.com/case-studies/reliable-predictive-models-mlops-delivery-hero/" target="_blank" rel="noopener">Delivery Hero</a>, <a href="https://datatonic.com/case-studies/ai-booster-vodafone-supercharging-ai-ml-at-scale/" target="_blank" rel="noopener">Vodafone</a>, <a href="https://datatonic.com/case-studies/bt-ai-accelerator-artificial-intelligence/" target="_blank" rel="noopener">BT</a>, and many more have already adopted MLOps and so can you. 
Join today’s <a href="https://goo.gle/mlops-event" target="_blank" rel="noopener">Google x Datatonic webinar</a> to experience the MLOps Turbo Templates in action and continue reading to get started with MLOps.
<h4>What is MLOps?</h4>
MLOps encompasses the set of practices, technologies, and culture that enable Machine Learning to deliver real business value in an organisation. Borrowing concepts from DevOps and applying them to Machine Learning, MLOps enables teams to move rapidly from offline experimentation to a deployed production system and beyond, unlocking business potential from Data Science. 
Even in the era of Generative AI, MLOps is a valuable investment for companies. Core MLOps capabilities for traditional Machine Learning can be used for <a href="https://datatonic.com/services/generative-ai/" target="_blank" rel="noopener">Generative AI</a> use cases. Watch the Vertex AI product team talking at Google Cloud Next 2023 about <a href="https://www.youtube.com/watch?v=UfUoG_Ijgq4" target="_blank" rel="noopener">Building your MLOps strategy for Generative AI</a> to understand common MLOps components and unique requirements for Generative AI.
<h4>What is Vertex AI?</h4>
<a href="https://cloud.google.com/vertex-ai" target="_blank" rel="noopener">Vertex AI</a> is Google Cloud’s unified platform for AI + ML. It helps teams build, deploy, and scale ML systems faster, with tooling to support the entire ML lifecycle, from experimentation to production. Since its release in 2021, Vertex AI has continued to evolve, introducing powerful new features such as model monitoring, model versioning, and experiment tracking. Most recently, a new set of tooling has been released to support generative AI including <a href="https://cloud.google.com/generative-ai-studio" target="_blank" rel="noopener">Generative AI Studio</a>, <a href="https://cloud.google.com/model-garden" target="_blank" rel="noopener">Model Garden</a>, and <a href="https://cloud.google.com/blog/products/ai-machine-learning/create-generative-apps-in-minutes-with-gen-app-builder" target="_blank" rel="noopener">Gen App Builder</a>.
<h4>What are the “MLOps Turbo Templates” and why were they created? </h4>
The MLOps Turbo Templates are an open-source codebase that provides a reference implementation of the end-to-end ML lifecycle. Co-developed by Datatonic and Google Cloud’s Vertex Pipelines product team to help businesses kickstart their MLOps initiatives, they enable both Data Scientists and ML Engineers to use Google Cloud better and faster. Today, Datatonic is announcing a new version, bringing new features and capabilities to help organisations adopt MLOps practices on Google Cloud.
With more clients deploying ML models to production than ever before, our teams realised the power of MLOps solutions for maximising the value of clients’ ML models early on. Our MLOps solution for <a href="https://datatonic.com/case-studies/ai-booster-vodafone-supercharging-ai-ml-at-scale/" target="_blank" rel="noopener">Vodafone</a>, for example, has helped them significantly reduce time to production, and <a href="https://datatonic.com/case-studies/reliable-predictive-models-mlops-delivery-hero/" target="_blank" rel="noopener">Delivery Hero</a>’s MLOps Platform allows their teams to seamlessly leverage Machine Learning best practices. 
The MLOps Turbo Templates combine Datatonic’s experience of implementing MLOps solutions with Google Cloud’s technical excellence to help reduce technical debt and adoption challenges many businesses face.
The templates can be used to create a production-ready MLOps solution on Google Cloud, covering the key components of a production ML system, including:
<ul>
<li style="font-weight: 400;" aria-level="1">Infrastructure-as-Code</li>
<li style="font-weight: 400;" aria-level="1">ML code versioning and deployment with CI/CD</li>
<li style="font-weight: 400;" aria-level="1">Data preprocessing</li>
<li style="font-weight: 400;" aria-level="1">Training a model</li>
<li style="font-weight: 400;" aria-level="1">Scheduled ML batch predictions</li>
<li style="font-weight: 400;" aria-level="1">Skew/drift detection</li>
</ul>
&nbsp;
Although the Turbo Templates can be used in isolation for individual ML use cases, they can also form one of the key ingredients needed for adopting MLOps at scale across an organisation. The diagram below shows how a strong MLOps blueprint can fit into the wider picture of enterprise MLOps adoption.
&nbsp;
<img loading="lazy" decoding="async" class="alignnone wp-image-23282" src="https://cms.datatonic.com/app/uploads/2023/11/MLOps-Turob-Templates-Diagram-300x140.png" alt="MLOps-Turob-Templates-Diagram" width="750" height="350" srcset="https://cms.datatonic.com/app/uploads/2023/11/MLOps-Turob-Templates-Diagram-300x140.png 300w, https://cms.datatonic.com/app/uploads/2023/11/MLOps-Turob-Templates-Diagram-1024x479.png 1024w, https://cms.datatonic.com/app/uploads/2023/11/MLOps-Turob-Templates-Diagram-768x359.png 768w, https://cms.datatonic.com/app/uploads/2023/11/MLOps-Turob-Templates-Diagram-1536x718.png 1536w, https://cms.datatonic.com/app/uploads/2023/11/MLOps-Turob-Templates-Diagram.png 1600w" sizes="(max-width: 750px) 100vw, 750px" />
<h4>What’s new in the MLOps Turbo Templates?</h4>
Since the initial release last year, the MLOps Turbo Templates have continued to evolve, building on Datatonic’s experience implementing MLOps with leading enterprises, as well as Google’s continued investment and innovation in state-of-the-art ML tooling. The new enhancements to the template cover two key themes:
<ol>
<li style="font-weight: 400;" aria-level="1">Adopting the latest and greatest in Google Cloud</li>
<li style="font-weight: 400;" aria-level="1">Improving the developer experience, making the template more lightweight and easy to use.</li>
</ol>
In terms of new functionality, the MLOps Turbo Template now uses Vertex AI’s model versioning to make managing your trained models much easier and neater, with model evaluations now viewable natively in the Vertex Model Registry. 
Vertex AI Model Monitoring has replaced Tensorflow Data Validation to provide a built-in solution for monitoring data skew/drift, with added support for alerting. The Kubeflow Pipelines (KFP) SDK has been updated to the new KFP V2, which adds support for custom container components, pipelines-as-components, and the ability to store and version pipelines in Artifact Registry.
A lot of work has been done to improve the developer experience. By simplifying the pipelines and using more native Vertex AI functionality, the run time of the pipelines has been reduced by over 50%, allowing for faster iteration and less time spent waiting for pipelines to finish. The codebase has been significantly refactored, reducing the number of custom pipeline components and simplifying the approach to testing, both of which contribute to a lower barrier to entry and a reduced maintenance burden for teams using the template. 
Lastly, the Infrastructure-as-Code has been rewritten from the ground up to provide full cloud infrastructure for dev, test and production environments using Terraform, making it even easier for Data Science teams to get started on Vertex AI.
<h4>Try it out for yourself!</h4>
The new release of the MLOps Turbo Template is now available in Datatonic’s open-source repository <a href="https://github.com/teamdatatonic/vertex-pipelines-end-to-end-samples" target="_blank" rel="noopener">here</a> (and <a href="https://github.com/GoogleCloudPlatform/vertex-pipelines-end-to-end-samples" target="_blank" rel="noopener">Google Cloud’s repository</a> in future), including all the documentation needed to help teams start with their use cases. 
Get started today with the new <a href="https://github.com/teamdatatonic/vertex-pipelines-end-to-end-samples/blob/develop/docs/notebooks/01_infrastructure_setup.ipynb" target="_blank" rel="noopener">three-part notebook series</a> which will guide you through deploying your own MLOps project on Google Cloud and the productionisation of an ML model on Vertex AI in minutes.
Datatonic enables leading companies, like Vodafone, Sky, and Inmarsat, to make better business decisions with a powerful Modern Data Stack and accelerate impactful solutions with MLOps. For enquiries about new projects, <a href="http://datatonic.com/contact" target="_blank" rel="noopener">get in touch</a>.

Datatonic Launches Updated “Turbo Templates” to Supercharge MLOps Adoption in 2024

<img loading="lazy" decoding="async" class="wp-image-23160 alignleft" src="https://cms.datatonic.com/app/uploads/2023/11/Marketing-Analytics-Specialization-300x300.png" alt="" width="178" height="178" srcset="https://cms.datatonic.com/app/uploads/2023/11/Marketing-Analytics-Specialization-300x300.png 300w, https://cms.datatonic.com/app/uploads/2023/11/Marketing-Analytics-Specialization-150x150.png 150w, https://cms.datatonic.com/app/uploads/2023/11/Marketing-Analytics-Specialization.png 726w" sizes="(max-width: 178px) 100vw, 178px" />
London, 15th November 2023 — Datatonic today announced that it has achieved the Marketing Analytics Partner Specialization in Google Cloud Partner Advantage. By earning the Partner Specialization, Datatonic has proven its expertise and success in building powerful Marketing Analytics solutions for customers using Google Cloud technology.
Datatonic has received this Specialization following a range of advanced marketing analytics projects for industry-leading businesses, including: 
<ul>
<li style="font-weight: 400;" aria-level="1">Developing a Customer Marketing Dataset to process over one billion events daily for a <a href="https://datatonic.com/industries/retail/?utm_source=linkedin&amp;utm_medium=organic&amp;utm_campaign=Announcement_MASpecialization_202311&amp;utm_term=" target="_blank" rel="noopener">leading UK retailer</a></li>
<li style="font-weight: 400;" aria-level="1">Enabling a leading global <a href="https://datatonic.com/industries/telecommunications/?utm_source=linkedin&amp;utm_medium=organic&amp;utm_campaign=Announcement_MASpecialization_202311&amp;utm_term=" target="_blank" rel="noopener">telecommunications</a> company to monitor + reduce customer churn on its broadband products</li>
<li style="font-weight: 400;" aria-level="1">Delivering real-time analytics and innovative new <a href="https://datatonic.com/partners/looker/?utm_source=linkedin&amp;utm_medium=organic&amp;utm_campaign=Announcement_MASpecialization_202311&amp;utm_term=" target="_blank" rel="noopener">Looker</a> dashboards and Explores to improve marketing metrics for a <a href="https://datatonic.com/industries/retail/?utm_source=linkedin&amp;utm_medium=organic&amp;utm_campaign=Announcement_MASpecialization_202311&amp;utm_term=" target="_blank" rel="noopener">global luxury retailer</a></li>
</ul>
Datatonic’s solutions have enabled clients to make data-driven decisions to generate value at scale. We’ve given our customers the ability to transition from disparate datasets to data-driven marketing.
“At Datatonic, we&#8217;re passionate about impact and feel privileged that our customers choose us to accelerate and support their data and AI journeys &#8211; allowing us to realise new value for their businesses, delivering real-time analytics and powerful marketing dashboards using Looker. We are proud of our deep collaboration with Google and are delighted to have been awarded the Marketing Analytics Specialization.” &#8211; Andy Harding, CTO at Datatonic
We look forward to our continued collaboration with Google Cloud, helping clients drive value from their data from cloud migrations to advanced analytics, <a href="https://datatonic.com/services/generative-ai/?utm_source=linkedin&amp;utm_medium=organic&amp;utm_campaign=Announcement_MASpecialization_202311&amp;utm_term=" target="_blank" rel="noopener">Generative AI</a> and beyond.
Discover <a href="https://datatonic.com/services/?utm_source=linkedin&amp;utm_medium=organic&amp;utm_campaign=Announcement_MASpecialization_202311&amp;utm_term=" target="_blank" rel="noopener">our services</a> to start your next data + AI project.

Generative AI

Cloud Data Migration

Marketing Analytics

Looker

Retail

Telecommunications

Gaming

Media

Financial Services

Events

Whitepapers

Insights

Enterprise MDS Deployment at Scale: dbt + DevOps

Related

Datatonic Wins Four 2024 Google Cloud Partner of the Year Awards

Coding Confidence: Inspiring Women in Data and Analytics

Prompt Engineering 101: Using GenAI Effectively