Under the Hood of CloverDX

Intuitive for developers

Build on Java/Eclipse with the familiarity of traditional programming IDEs and methodologies. It’s easy to version source files, interactive debugging and logging.

Simple architecture

CloverDX Designer
Desktop application for Windows/Mac/Linux, if you need a Designer to develop jobs, you can run them locally or deploy to Server.

CloverDX Server
Enterprise runtime environment for cloud or on premise. Server adds collaboration, job orchestration, APIs, parallelism and much more.

Automate processes inside and outside of CloverDX

Data transformations are at CloverDX’s core, but that’s not all our platform is for. CloverDX can help bring your important, but disparate, systems, processes and scripts together.

Jobflow orchestration

CloverDX is a workflow orchestrator as well as a data transformation tool. You can visually define job flows with arbitrary branching, error handling and looping and control them with code.

Engine-based, no compilation, pure Java

Jobs you design in CloverDX are stored as easy-to-read-or-version XML files. They’re interpreted by a CloverDX Runtime when a job is executed. CloverDX allows you to code, but it is not a code generator itself.

Built-in scripting language (CTL)

CTL stands for CloverDX Transformation Language. It’s a simple, yet powerful language, resembling Java, that makes writing code to manipulate data a breeze. As well as CTL, you can also use Java, Javascript or Python (through 3^rd party libraries).

Extensibility (custom functions, Java components, Designer plugins)

You can extend, customize and tailor anything to your needs in CloverDX. Missing a connector to a data source? No problem. You can implement it visually and package it into a single component, or delve into code and write a Java component. Missing functions in CTL? Simply write your own extension library and share it among your teams and projects.

Complex data mapping

Despite being primarily a visual tool, most of CloverDX’s “good stuff” uses code in the background, which is accessible to developers. For example, mapping rules can be edited visually by “connecting wires”, but in reality these connections generate lines of code you can extend.

Global parameters & Shared resources

Who doesn’t hate duplicating stuff? Code, data, functionalities or any other piece of information can be controlled and customized from a single point . You can then share it among multiple jobs or even projects.

Parallel processing & Phases

Running a transformation is like watching a liquid (data) flow through a system of pipes with functional components attached to them. Everything happens in parallel, as fast as components are ready to pass the data. CloverDX benefits from high-core systems for optimal performance. You can control the sequence of operations by grouping functional parts of a transformation into “phases” – each waiting for all operations to finish before the next one can start.

Parallel streaming = Constant memory requirements

Data flows through CloverDX transformations as a stream of data records. CloverDX jobs are not susceptible to memory problems when experiencing a growing number of records.

Plugins to Designer and Server

You can add useful functions to the desktop Designer by using the standard Eclipse plugin framework. The server can be augmented too, with simple data apps (no coding needed) or more complex functional or API extensions.

Generate jobs’ XML code or use parameters to control flows

In some cases, it’s easier to generate jobs than to design them manually. This applies when building customizable frameworks, driven by metadata (even something as simple as an Excel definition document), and then generating appropriate jobs from them.

Nested “subgraphs” with parameters

A “graph” is what we call a data transformation job. A “subgraph” is when you take a portion of a graph and package it into a new functional component. Subgraphs can be shared with others. You can pull out parameters in order to have universal components that can be reused in similar situations.

Data partitioning

Do you have lots of CPU cores or multiple nodes available and need more speed? You can distribute a single data flow onto multiple workers, even on a single machine, to increase parallelism and performance.

Scalability & High Availability

Multiple CloverDX Server instances can be connected into a cluster that can: act as a load balancer, distribute a single job onto multiple nodes, or simply provide robustness and reliability.