280 suppliers. 12 different data formats. One small team.
If you're a B2B product distributor, chances are that you need to ingest product data from multiple suppliers. But one has sent you a CSV via email, one has uploaded a file to an FTP site, one is sending you JSON files. All of it needs to get into your ERP system, and your e-commerce site.
This is the supplier data integration challenge for a lot of B2B industrial distributors, and it can often be a bottleneck to growth.
CloverDX has helped companies solve data integration problems just like these.
We often have companies come to us having tried to overcome the problem by hiring more data entry staff and build more manual processes. But that only works up to a point.
Airline Hydraulics - a value-add distributor of pneumatic, hydraulic, and automation products, serving factory and mobile automation customers across North America - came to us because they needed to list three million SKUs on an e-commerce site. Their products came from almost 300 vendors, and they needed to normalize all that supplier data - which arrived to them in different shapes and sizes - to get it into their ERP in near-real-time.
And they needed to do it all without creating more manual work, and without needing to hand-scrub Excel files.
"We were looking at just mountains of dissimilar data and saying, okay, how do we take this and turn it into something that now becomes useful to us, ourselves and our customers?"
- Todd Schnirel, Director of Business Development and Agility, Airline Hydraulics
A million-line Excel file that took 4 days to load
The incoming data that Airline Hydraulics had to work with included text files, raw data dumps, JSON feeds, FTP transfers, emailed CSVs and PDFs.
Pricing, item numbers, descriptions, weights, lead times: all arriving in incompatible formats, all needing to be normalized and loaded into Epicor Prophet 21 (their ERP of over a decade), and then surfaced accurately on the e-commerce site.
Manual processing was already their reality - and it was starting to strain under the load. One particularly vivid illustration of how things were: a single price update from a major supplier came as a million-line Excel file. Loading it into the ERP took three to four days of batched processing, often causing memory issues - sometimes kicking users off the system, sometimes simply freezing it entirely.
Bridging a legacy ERP and the modern data world with CloverDX
Airline Hydraulics needed to make their Epicor Prophet 21 ERP system, which they describe as a ‘steady, traditional distributor ERP system’, work with modern web tools and API interfaces.
“CloverDX gives us the bridge to get between a legacy ERP system and many of the modern applications we have to deal with.”
- Todd Schnirel, Director of Business Development and Agility
By using CloverDX, Airline Hydraulics can now ingest supplier data in whatever format the supplier provides — from a RESTful API delivering JSON in near-real time to an emailed CSV or even a scanned PDF catalog. CloverDX transforms, normalizes, and routes it into both the ERP and the e-commerce platform consistently.
Rather than forcing suppliers to conform to Airline's systems, they built flexibility into their own pipelines. They decided to meet partners where they were on their data journey — and accept any format, from any supplier, without penalizing smaller partners who didn’t always have modern data systems.
CloverDX also helps them process large amounts of data that was a challenge before. An example was that million-row Excel file that previously took 3-4 days to load into the ERP system.
Now with CloverDX, it takes about 3 hours to process those million records. And as Jennifer McCalicher, Data Quality and Integrity Team Manager at Airline Hydraulics, says: “It’s a single file, a single graph within CloverDX that we utilize. We don’t have to develop 25, 30 different sub-processes just to get the information in”.
Onboard data in any format, from any source
See how CloverDX can help you onboard supplier data faster - automating data ingestion and transformation to reduce manual work and engineering effort.
From 4 days to 3 hours: the before and after of data ingestion with CloverDX
Automating their supplier data ingestion not only enabled Airline Hydraulics to bridge their legacy ERP and their modern data stack, and to power their e-commerce site, but it also freed up people's time.
People who had previously been doing repetitive, manual data entry are now freed up to contribute more right across the organization.
"It's been amazing to see what they can do for us when they get to use their mind as opposed to just their fingers on a keyboard....all of these little talents and hidden abilities that our employee owners had that we just didn't realize because we had them head down at a computer typing away - a needed job, but not a job that was adding value in the way that they're adding value now."
- Todd Schnirel
Data ingestion before CloverDX
- Million-line supplier price files took 3–4 days to batch-process into the ERP
- PDF catalogs required someone to read them manually and type data in by hand
- Pricing and availability data was stale by the time it reached customers
- Data entry was a significant, permanent headcount category
- E-commerce at scale was functionally impossible
Data ingestion with CloverDX
- 1 million-line price file processes in ~3 hours, fully automated
- PDF data extraction is handled programmatically, not manually
- Any supplier format — CSV, JSON, XML, PDF, API — can be onboarded via a pipeline
- Pricing and availability is updated in near-real time, directly from manufacturers' ERP systems
- Former data entry staff are now in higher-value-add roles
- A 3-million-SKU e-commerce site runs reliably with a lean team
The challenge of data from hundreds of suppliers
The problem for most distributors is not their core systems such as an ERP - it's the gap between that system and the data reality of working with dozens or hundreds of suppliers in a modern, API-driven, e-commerce world. The right tool bridges that gap without forcing a costly, disruptive rip-and-replace.
CloverDX is the data integration platform that can help you connect different data sources and targets, handle files in multiple formats, and transform incoming supplier or client data files in an automated, end-to-end pipeline.
Hear how Airline Hydraulics automated data ingestion with CloverDX
Todd Schnirel and Jennifer McCalicher of Airline Hydraulics tell this story in their own words in the CloverDX Behind the Data podcast.
Listen to the episode here: The Human Impact of Data Transformation
If you're a B2B distributor managing product data from multiple suppliers and want to understand how CloverDX could help your data ingestion, request a demo.
Share
