For Darren Rooney, Senior Data Engineering Manager at Benchmark Analytics, the strength of a data pipeline isn’t measured by how it runs on a good day. It’s about how well it holds up when the data is messy, the stakes are high, and accuracy is critical.
His team supports more than 800 law enforcement agencies, including the New York Police Department (NYPD), helping them make informed, timely decisions about officer performance, wellness, and early intervention.
“We’re making a real impact with that,” Darren says. “The data scientists are making interpretations and feeding the models.”
The data Darren’s team works with includes sensitive inputs, such as internal affairs records, behavioral reports, and personally identifiable information (PII). The variation between sources is wide, which makes cleanup and validation essential to remove inconsistencies.
“You would expect it to be not very accurate and all over the place,” Darren says. “So it's very important that we wrangle that data.”
Fixing data issues isn’t a nice-to-have. It can directly affect Benchmark’s performance metrics. “If we didn't fix the errors, our efficacy rate would drop. We want it to go up. The cleaner data that we can get through, the better job our data scientists can do.”
Darren and his team are exploring generative AI, but with specific boundaries. They treat it as a supporting tool, not a replacement for expertise.
“I use it for strategy discussions,” he explains. “Not necessarily to write code, but to interact with it on a conversational level. Is this cost-effective? Is this the direction we need to go?”
His engineers occasionally lean on AI for lightweight coding tasks. “We’re trying to see what other companies are doing well, what they did poorly, and take lessons learned,” Darren says. “We’re not trying to reinvent the wheel.”
Strict rules are in place for how and where AI is used. “We don’t want our public sensitive PII data to be in these open models. That’s an absolute no-no.”
Before CloverDX, the data team handled each agency’s integration as a one-off. Custom scripts were written by hand, edge cases were resolved individually, and workloads varied greatly depending on the engineer’s experience.
“We were doing agency-by-agency extraction scripts from source to target,” Darren says. “We had a team of engineers with varying levels of skill sets.”
Now the team runs a standardized, modular pipeline architecture. “The time on data for a human is very short. The time on data for processing is very large. That was the goal.”
Shifting to this new model not only improved scalability but also eased a heavy burden on the team. “There was some stress and some strain all the way down to the team morale because there was a lot of manual work,” he says, reflecting on the challenges before standardization.
The Benchmark Analytics team built its pipelines with robustness in mind. When something breaks, the goal is to handle the error automatically whenever possible.
“We call it recovery,” Darren says. “Right now we get an error, and a human needs to intervene and make a change. What we would like to see is more of a recovery process. So here are the errors for today and let’s try to fix them automatically.”
They separate issues into two categories. “Type 1 is source data errors. Type 2 is internal system errors. Type 2 errors need to go away,” Darren says. “We take those errors and then we rebuild. The idea is to not have them.”
To avoid recurring issues with data inputs, Benchmark establishes schema contracts with its partner agencies. These outline what data will be delivered, how often, and in what format.
“We enter into a contract with an agency,” Darren explains. “They promise to send us data in this format and this cadence, and then we build pipelines around that.”
For teams building or modernizing their pipelines, Darren’s guidance is rooted in iteration and focus.
“Proof of concept (POC) a lot,” he advises. “Get your use cases set up. Get the functionality that you need for the business. Try to keep it tight.”
And don’t worry if everything doesn’t work the first time. “We did several POCs that didn't work,” Darren says. “But we learned a lot. We learned what not to do is most important.”
Listen to the full Behind the Data episode with Darren Rooney: Building resilient data pipelines for sensitive, high-impact use cases