Latest production version of the new CloverETL 4.0 is out and our developers are working hard on the next version of CloverETL. You can read release notes for CloverETL 4.0 here. We will try to explain the biggest changes and new features you can look forward to. Today, we will give you an overview of the most important new features. In the following weeks, we will publish future articles and videos, where we will explain those features in detail and present you with real life examples of these features in action.
The main star of this release is subgraphs. The addition of subgraphs in CloverETL 4.0 represents one of the biggest conceptual changes we have ever made to CloverETL. But that’s not all. There are many more improvements in CloverETL 4.0 that will make your life easier and give you even more possibilities to work with your data. We hope that you find them as wonderful as we do.
Lets take a look now at the four major improvements in CloverETL 4.0
1. Subgraphs
To explain subgraphs, let’s use an analogy with Legos. Imagine that you bought a Lego set with a selection of standard pieces, but with the ability to create your own blocks. And not only basic shapes – you can create blocks with any kind of functionality. Need an engine for your car model? You can go ahead and create it yourself. And when you are done, it will be the same size as a basic block.
This is very similar to what you can do with subgraphs in CloverETL 4.0. Rather then building complex logic from basic components every time, you only need to create it once, and then you can use it or share with others.
Here are a few of the benefits that using subgraphs will give you:
- Increase speed of development of new graphs by reusing already-created subgraphs
- A change in a subgraph will be applied to all graphs that use the subgraph, simplifying change management
- Improve collaboration - sharing logic or splitting work within the team
- Visually simplify your graphs by hiding advanced logic into subgraph components, making graphs easier to understand
- Extend functionality beyond the standard components and connectors supplied, making it a more versatile data tool
A subgraph can contain a sophisticated function while looking like a simple component.
2. Automated Metadata Propagation
In the past, working with metadata was not easy as it required a lot of attention to manually manage metadata. You had to keep your eye on your data flows to make sure that you were using the correct metadata. The automated metadata propagation feature in CloverETL 4.0 now significantly reduces this burden. It makes the process much easier: metadata are assigned to an edge and then automatically propagated throughout the graph, depending on the inputs and outputs expected by different components.
Automatic metadata propagation will make work with graphs significantly faster and smoother, shortening the time it takes to create a graph.
3. Execution view
We’ve implemented a new and improved runtime for CloverETL 4.0. This allowed us to create a new debugging tool called the Execution View. The Execution View allows you to inspect the whole hierarchy of your jobs at the same time. You can do this both during and after the execution of your graphs. If something goes wrong, you can inspect the job by simply clicking on the error in the Execution View. This will open the job graph editor and highlight the component that caused the error.
The Execution View will be appreciated by developers working on complex graphs, as well as by support teams inspecting jobs in production, as it substantially simplifies finding and debugging errors.
4. Improvements in the user interface
As we are always striving to give users better control over CloverETL, we also introduced several changes in the user interface that will make your life easier. One of them is the detection of unused graph elements. Unused elements are visualized in the Outline view, so you are able to quickly identify and remove them. Another improvement is an updated edge algorithm that produces less intersections between edges, which will make your graphs easier to navigate and to understand. Also, CloverETL now monitors manual changes in your metadata and notifies you with a small tip that hovers over an edge where metadata was recently changed.
CloverETL 4.0 represents a major technological leap forward in the development of CloverETL. With the use of subgraphs, automatic metadata propagation, and all of the other new features for cooperation and teamwork, this version will change the way you work with your data, making it faster, easier, and more efficient than ever before. With the new version of CloverETL, you can work with data within minutes of deploying CloverETL.
