Customer story

Reducing Salesforce Data Loading Time From 12 Hours to 41 Minutes

CloverDX Case Study - Speeding up Salesforce Data Loading

Summary:

CloverDX enabled fundamental improvements in how the company runs their business, by shaking the status quo of data processing. This helped the business transition away from an in-house developed solution which was time consuming to run, expensive to maintain and highly error prone.

Challenges:

  • Tens of millions of records processed daily
  • Salesforce API limits were causing timeouts and errors
  • Script approach was hard to maintain

Industry
Financial Services

Region
USA

Solution
  • Data ingest solution pulls data multiple times a day
  • Sequencing data overcomes Salesforce limitations
  • CloverDX expertise helped a fast implementation
Results
  • Significantly reduced data load time
  • Can provide daily refreshed data to their clients
  • Can onboard new customers much more quickly

10 years ago, a script based approach to integration seemed like a good idea for a leading US wealth management firm.

9 years later, after very substantial growth in clients and data sources, the maintainability, error rates and lag times had become a serious threat to transparency and overall quality of service.

 

The Challenge & Solution

The company wanted to have complete control of their systems and steer clear of expensive ETL tools and inflexible platforms. This strategy served them well initially but as the business grew so did the complexity and the data volumes. The difficulties peaked as they developed a highly configurable Salesforce portal for their financial advisors.

With tens of millions of records of data being processed daily, things were out of hand. Maintaining ever-growing libraries of legacy scripts was now a serious problem and error rates had become very significant, requiring a full day of corrections. In the end, the published data  was often 2 days out of date.

At this point the company realized that they had to squarely address the problem or the ability for the business to maintain a dominant position would be jeopardized. Choosing a platform partner was not just a question of product, it was equally about the team behind the product. CloverDX stood out from the competition in this respect with our experience in the integration field and the ability to co-engineer a solution.

Reducing Salesforce data loading time and freeing up resource

With this new solution, the company has been able to vastly reduce the Salesforce data loading time and free up significant resource for the team, so they can focus on adding value - not just fixing errors. The solution now sequences data appropriately to overcome the Salesforce data loading challenges, and data validation is now robust and reliable. It’s also enabled better information to be provided to their customers with data being pulled multiple times a day from different sources.

Square icons - Green Avatar
SVP of Technology
Technology
"The single best thing this company has done for data integration."

The Results

With improved data ingest, their platform can now also integrate with legacy systems; new integrations are significantly faster; customers can be onboarded more quickly; and the business can focus on their growth.

  • Needed to feed a complex Salesforce portal providing financial advisors a 360 degree view of the client
  • Tens of millions of records processed daily
  • Success of the new business model hinged on loading data, quickly and error free, into Salesforce
  • Needed to overcome the timeouts and batch restrictions imposed by the Salesforce API
  • Company needed not just product but expertise and a team to help accelerate the solution
  • Move away from hard to maintain scripts
  • Load times reduced from 12 hours down to 41 minutes
  • Error rate reduced to virtually nothing
  • Data sources due to double does not raise concern for the business
  • Dev team now focuses on adding value, not fighting fires
  • Can now integrate with any legacy accounting systems
  • Bolt on integrations that used to take a month now take less than a week

Want the blueprint for a faster, more scalable data ingestion pipeline?

Watch our quick  webinar where our experts walk through actionable steps to optimize your data ingestion processes.

Learn more

See everything CloverDX can do for your data

Book a demo and see how you can reduce time-consuming data tasks with the latest features available in CloverDX.

Book a demo

Read more customer stories

Flight Centre Travel Group accelerated customer solutions with the help of CloverDX

Find out how CloverDX helped Flight Centre replace legacy code, streamline onboarding, and scale data delivery.

 

Read full story
sisk-hero-img
From late nights to hands-off automation for Sisk Fulfillment Service

Sisk Fulfillment Service handles millions of shipments a year, but the process for dealing with customer orders had been labor-intensive and had led to late-night working. But automating those data processes with CloverDX has meant faster order handling, fewer errors, and no more late nights.

Read full story
Saving hours of manual work with a control center for all of PeopleInsight's data processes

HR SaaS software PeopleInsight, part of HireRoad’s cloud-based talent management and insights platform, has created a centralized, automated control center for all their data processes with CloverDX.

 

Read full story