Customer story

Automated Data Migration

CloverDX Case Study - Data Migration

About:

When a software company realized that their team of developers writing custom scripts wasn’t going to be able to meet their deadline, they called us.

Legacy Data Challenges:

  • Complex, frequently changing data
  • Custom script approach couldn't scale fast enough
  • Not enough in-house resource to hit the deadline

Industry
Software

Schedule
2 months

Size
200,000 XML files

Results
  • Migrated 200,000 files across 3,000 mapped fields.
  • Adapted to multiple structure changes with zero manual intervention.
  • Delivered on time with a smaller team than planned.
  • Reusable framework built for ongoing migrations post go-live.
Timeline
2 months
From start to finish, the entire migration had to be completed within 2 months — regardless of any complications along the way.

The manufacturing company had 200,000 XML files with constantly changing schemas, 3000 fields that needed to be mapped into 150 database tables.

...plus, the team had only 2 months until the migration had to go live.

 

The main challenge

Complex data migration, little time

The data was complex, was changing frequently throughout the project and there was a lot of it. While the client had had a team of developers creating separate scripts to migrate individual data structures, it quickly became apparent that the approach couldn’t scale quickly enough to handle the 200,000 files that needed to be migrated.

Time was critical; the migration automations needed to be fully operational within weeks of the project starting.

 

The result

A more scalable data migration approach

Rather than trying to migrate so many different data structures manually, we took a different, more scalable approach and created a system that could automate the migration. The tool recognizes XML elements and attributes and automatically creates mapping documents, all without needing consultants’ input.

Despite the XML structures changing multiple times over the course of the project, the automated migration tool detected and adapted to structure changes and could successfully process all the files generically, automatically analyzing and mapping 3,000 fields to the new database. The project was delivered on time and accurately, and with a smaller team than originally planned.

 

A reusable framework for the future

The benefits of the automated data migration didn’t end there. A data migration is always an iterative process that continues beyond the go-live, so the fact that the framework is completely reusable and configurable means that the client can continue tackling further changes in data or structures for as long as they need.

data-migration

Thinking about migrating your data platform soon?

Explore what a migration to CloverDX would look like, including how to save time and reduce errors during the migration process.

Learn more

Read more customer stories

image-for-listing-NDP
Data Helps Canada's NDP Know Potential Voters

Political party NDP needed to match and merge data from hundreds of databases. CloverDX helped them integrate data sources, clean data and migrate.

Read full story
sisk-hero-img
From late nights to hands-off automation for Sisk Fulfillment Service

Sisk Fulfillment Service handles millions of shipments a year, but the process for dealing with customer orders had been labor-intensive and had led to late-night working. But automating those data processes with CloverDX has meant faster order handling, fewer errors, and no more late nights.

Read full story
vanmossel-hero-img
Enabling data integration across Van Mossel Automotive Group

The largest automotive company in the Benelux is rapidly expanding, and CloverDX is enabling centralized data integration across the business, with automated standardization, transformation and ingestion of data from multiple sources.

Read full story