There are cases where running a transformation component can create a bottleneck because it runs significantly slower than the rest of the transformation. A typical example might be an API based component that has to deal with network latency issues.
CloverDX allows you to deal with this by partioning your data using a dedicated component. This splits the data up and invokes the subsequent component in parallel and allowing you to control the number of partitions.
The above transformation example shows
CloverDX allows you to cluster in different ways.
CloverDX Designer and Server are both designed to be as open as possible, allowing many sorts of customization wherever needed.
The source format for CloverDX transformations is XML. If you encounter edge cases, being able to modify a transformation by altering the transformation source allows you to deal with almost any scenario you are likely to encounter.
There are many locations where configuration of items such as database parameteres, lookups, data structure definitions etc. are stored.
All of these can be modified without having to use a CloverDX front end.
The text based nature of CloverDX allows developers to collaborate using tools such as Git and Mercurial and integrate transformations into devops type workflows.
It is important that your transformation tool allows developers to build complete pipelines consisting of several transformations and processes with inter-process dependencies.
CloverDX handles this using Jobflows. A Jobflow executes and monitors internal processes (data transformations or other Jobflows) as well as external processes (typically scripts). Together with the powerful error handling and visual nature of the jobflow design process, CloverDX allows developers to very quickly build entire pipelines that properly react to success and failure.
Thanks to its ability to automate entire processes, monitor and send appropriate notifications, CloverDX can quickly become a central orchestration point, notifying responsible and interested parties on the progress of migrations and other data integration processes in real time.
CloverDX exposes multiple different APIs that are useful for integration with 3rd party tools. For example it is possible to trigger jobs via REST or SOAP API calls or to monitor various external systems for changes such as arriving files or JMS messages.