CloverDX Blog on Data Integration

Why data quality is crucial for data integration projects

Written by By CloverDX | October 17, 2024

‘Garbage in, garbage out’ is a saying often associated with the foods we eat and their consequences, but it applies equally to data.

Key Takeaways

  • Poor data quality costs organizations an average of $12.9 million a year, according to Gartner.
  • The three biggest obstacles to data quality in integration projects are emerging AI/ML technology, privacy regulations like GDPR, and weak data governance.
  • 95% of IT leaders say integration issues are holding back their AI initiatives, according to Salesforce's Connectivity Benchmark Report.
  • Over a quarter of organizations report losing more than $5 million a year to poor data quality, according to IBM's 2025 CDO Study.
  • Strict naming conventions, clear configuration documentation, and thoughtful front-end UI design are three practical ways to improve data quality during integration work.
  • Data quality initiatives only stick long-term when leadership can see the tangible business benefits.

Data quality is a vital attribute of any data project, as the accuracy, completeness, validity, consistency, uniqueness, timeliness, and fitness for purpose of any dataset ultimately influence the decisions we make.

David Howell, Director of IT and Digital Transformation at Chugai Europe, shared his insights on this topic during a recent episode of our Behind the Data podcast.

With over 24 years of experience at Chugai, Howell spoke at length about the importance of accurate, timely, and reliable data in driving business decisions and achieving digital transformation goals. Read on for his hard-won wisdom on the subject.

 


The top three obstacles to data quality

According to Gartner, poor data quality costs organizations an average of USD 12.9 million each year. There are three major roadblocks to data quality in most businesses:

Emerging technologies

Over 65% of businesses are now using artificial intelligence (AI) in some capacity, according to McKinsey’s State of AI report. And the integration of AI and machine learning (ML) with business intelligence will make it increasingly difficult for human practitioners to keep up with big data. As real-time and predictive analytics become more mainstream, there will be far more opportunities for consequential mistakes, and (hopefully) far more emphasis on data quality at the outset. As David says: "Just because you can automate and bring information into a warehouse doesn't mean it's good data.”

This is exactly why integration projects can't treat data quality as an afterthought. According to Salesforce's Connectivity Benchmark Report, 95% of IT leaders say integration issues are directly holding back their AI initiatives, since AI is only as good as the data feeding it.

Privacy concerns

Privacy and protection laws like GDPR have intensified the need for businesses to store accurate data records. These regulations grant individuals the right to access their personal data, which means organizations require the ability to access and retrieve accurate data quickly. Any discrepancy due to inaccurate or inconsistent data can lead to legal complications and damage to the organization's reputation, which puts added pressure on those accountable for data quality.

This matters just as much at each point of an integration pipeline as it does in the original source system. Inconsistent or duplicate records can just as easily get introduced while data moves between systems as they can sit unnoticed within a single one.

Data governance (or lack of)

Data governance practices involve setting internal standards and policies for data collection, storage, and sharing. This ensures consistency, trustworthiness, and compliance with regulations. When data governance is poor, there can be misalignment between systems, resulting in inaccurate, incomplete data that leads to faulty (and potentially harmful) decision-making. A data governance framework should be set firmly in place if data quality is a priority.

This kind of misalignment carries a real price tag. IBM's 2025 CDO Study found that over a quarter of organizations lose more than $5 million a year to poor data quality, and weak governance is very often the reason two systems disagree about what a customer record, or a product code, is actually supposed to look like.

Strategies to improve data quality

Improving data quality doesn’t need to be an odious task. A few simple strategies baked into how your organization does things can make a world of difference over the long term. In our recent podcast, David shared some of his favourites:

Establish strict naming conventions

Following a simple yet strict naming convention for data across disparate systems is a powerful way to ensure consistency and traceability. It also reduces the risk of misinterpretations and mistakes between stakeholders.

We've done it relatively simply," David told us, describing how Chugai Europe uses clear and consistent naming conventions to manage data across multiple systems. “We generally use the object name, like CRM.dot.prod.dot.accounts or CRM.dot.prod.dot.contacts, to distinguish where data is coming from.”

Building data pipelines to handle bad data: How to ensure data quality

Ensure data configuration management

Business goals change. New technology can be added to your existing stack. All of this can have an impact on the quality and integrity of your data. David emphasises that the proper documentation and tracking of data sources, transformations, and dependencies is absolutely essential to long-term data quality.

"Configuration documents are important," he says. "I work in IT. I hate documenting stuff. But it is important. Like when you code, you've got to put comments in, because otherwise, you go back to it six months later, and you can't remember what that function does.”

This matters most in integration work specifically, where a single pipeline might touch a dozen source and target systems, each with its own formats, update schedules, and quirks. Without clear documentation, nobody six months from now will know why a particular transformation rule exists, or whether it's still needed.

Consider user interface design on customer-facing assets

Doing everything in your power to standardize and structure data on the front end is a powerful way to maintain data quality. “There’s things you can do to make this happen, like make mandatory fields and drop-down lists,” David explained.

However, as AI becomes more integrated into data projects, it might be better to have free-text boxes rather than limit the variation of responses. “ if you've got a free text box, you can glean quite a lot of information from having that text and running it through natural language processing,” he advises.

Getting this right at the point of capture means fewer surprises to catch further down the integration pipeline, and less rework once that data is being combined with data from other systems.

The support of leadership is the key to data quality success

It’s easy to harp on about data quality, but if no one can see the tangible benefits, not least of all your leadership team, no one will be incentivised to improve things or maintain the momentum required for lasting change.

"Management buy-in is really important because it takes effort,” David says. "It's important to show the benefits of good quality data in driving business decisions.” Only then will data quality initiatives receive the status and resources they deserve.

Upgrade your data integration projects

For more insights on data quality and a look into Chugai Europe’s digital transformation journey, we recommend listening to the full Behind the Data podcast with David Howell.

If you're building this into a wider data quality strategy rather than a single project, our guide on why data-driven companies invest in a data quality strategy is a good next read.

If you want to know more about how CloverDX can help you improve your data quality, or to schedule a demo, get in touch with us here.