In this article, I will walk you through the process of extending CTL (CloverDX Transformation Language) with your own custom functions implemented in Java. Think of this as adding your own Java functions to the standard CTL function library so that you can transparently use them wherever there’s a piece of CTL code—in Reformat transformations, Filter expressions, etc.
Imagine you have some custom complex logic already implemented in Java packages, maybe some 3rd party library with potentially dozens of outside dependencies. Let’s say you want to share that functionality across applications, CloverDX included. There might be thousands of lines of code that you don’t want to be re-implementing in CTL.
Other reasons for using Java over pure CTL are performance and the expressive power of Java. While performance begins to matter only with large volumes of data, CTL has never been designed to fully replace general purpose languages. CTL makes working with data records easy, however for complex algorithms it lacks the expressiveness and eco-system of libraries that Java can give you.
In this article, we’re talking about native Java code working as CTL functions. That’s quite different from simply writing a shared function library in plain CTL. To do that, save a piece CTL code as a .ctl
file somewhere in your project and then import it into other transformations using import
construct like this:
//#CTL2
import "trans/filterFunctions.ctl";
function integer transform() {
$out.0.field1 = filterChars($in.0.field1); // filterChars() is declared in filterFunctions.ctl
return ALL;
}
From then, the code and functions declared in the .ctl
file will be available in the scope the library was imported in.
As an example, we’ll be implementing two functions:
string getWords()
string getWords(integer count)
These functions will produce “Lorem Ipsum”—Latin-like bogus text for testing. For this purpose, we’ll use an external loremipsum Java library by Sven Jacobs (see, we already have external dependencies!).
CloverDX is an open platform that can be extended by users in many ways. The core of CloverDX lies in its modular “engine”. This is the execution layer that takes care of running your data transformations and that sits at the center of CloverDX Designer (Runtime) and Server. CTL and all user extensions are treated as “engine plugins”. For our purpose we’ll need to extend the engine with our own plugin (Java package) containing our Lorem Ipsum functions.
CloverDX Designer presents another layer, working rather independently from the engine/Runtime. Designer is built on Eclipse and it comes with its own set of plugins that form the GUI. For Designer to recognize our new functions we’ll need to make it aware of our engine plugin by wrapping it with a separate “Designer plugin”. This will make the new functions available for context assist, correct syntax highlighting in editors, etc.
Here’s what we’ll do:
org.jetel.ctl.extensions.TLFunctionLibrary
.plugin.xml
descriptor with ctlfunction extension definition.org.jetel.ctl.extensions.TLFunctionAnnotation
annotation to mark “entry point” methods that will serve as the implementation of our CTL functions (as you may have some additional supporting private methods that you don’t want to turn into CTL functions).plugin.xml
descriptor—this time with engineplugin extension definition.Once finished, the new functions will be available in Designer dialogs (e.g. Source tab in Transform Editor, Reformat code, etc.).
So, let’s get to it!
com.cloveretl.custom.loremipsum
. Hit Next when done.
lib
in the project root directory (right-click the project in Package Explorer or in Navigator to create that folder)lib/loremipsum-1.0.jar
)
lib
directory ⇒ Choose loremipsum-1.0.jar
) and click OK.
LoremIpsumCTLLibrary
and it will extend org.jetel.ctl.extensions.TLFunctionLibrary
.
Let’s put the class into a package called com.cloveretl.customctl
.
TLFunctionLibrary comes from the CloverDX Library we added to the project earlier. However, you don’t need to worry about this for now, it is already a part of the code below).
package com.cloveretl.customctl;
import org.jetel.ctl.Stack;
import org.jetel.ctl.extensions.TLFunctionAnnotation;
import org.jetel.ctl.extensions.TLFunctionCallContext;
import org.jetel.ctl.extensions.TLFunctionLibrary;
import org.jetel.ctl.extensions.TLFunctionPrototype;
import de.svenjacobs.loremipsum.LoremIpsum;
public class LoremIpsumCTLLibrary extends TLFunctionLibrary {
LoremIpsum loremIpsum = new LoremIpsum();
@Override
public TLFunctionPrototype getExecutable(String functionName)
throws IllegalArgumentException {
TLFunctionPrototype ret =
"getWords".equals(functionName) ? new GetWordsFunction() : null;
return ret;
}
@Override
public String getName() {
return "LoremIpsumCTLLibrary";
}
@TLFunctionAnnotation("Generates Lorem Ipsum text")
public String getWords(TLFunctionCallContext context) {
return this.loremIpsum.getWords();
}
@TLFunctionAnnotation("Generates Lorem Ipsum text")
public String getWords(TLFunctionCallContext context, Integer count) {
if(count==null) {
return this.getWords(context);
}
return this.loremIpsum.getWords(count);
}
class GetWordsFunction implements TLFunctionPrototype {
@Override
public void execute(Stack stack, TLFunctionCallContext context) {
if (context.getParams().length > 0) {
stack.push(getWords(context, stack.popInt()));
}else{
stack.push(getWords(context));
}
}
@Override
public void init(TLFunctionCallContext context) { }
}
}
plugin.xml
in the root of the engine plugin project:
<plugin
id="com.cloveretl.custom.loremipsum"
version="0.0.0.devel"
provider-name="CloverETL">
<runtime>
<library path="bin/"/>
<library path="lib/loremipsum-1.0.jar"/>
</runtime>
<extension point-id="ctlfunction">
<parameter id="libraryName" value="LoremIpsumCTLLibrary"/>
<parameter id="className"
value="com.cloveretl.customctl.LoremIpsumCTLLibrary"/>
</extension>
</plugin>
Alright, you’re half way through! Now let’s get on to setting up the Designer plugin.
com.cloveretl.designer.loremipsum
.CLProvider
com.cloveretl.gui.plugin.engine.AbstractClassLoaderProvider
com.cloveretl.gui
plugin to the list of Required Plug-ins
com.cloveretl.gui.engine
to Plugin extension
plugins
com.cloveretl.designer.loremipsum.CLProvider
<exportedPlugin
pluginId="com.cloveretl.custom.loremipsum">
</exportedPlugin>
plugins
under the com.cloveretl.designer.loremipsum
project folder. As best practice, put your engine plugins into folders named after the plugin IDs.
com.cloveretl.custom.loremipsum
(see image below):
plugins/com.cloveretl.custom.loremipsum
folder and select Import... ⇒ General ⇒ File system.
com.cloveretl.custom.loremipsum
project location (the engine plugin project).plugin.xml
(in the right panel).bin
.lib
.com.cloveretl.designer.loremipsum
project—the one in the main directory, not the one that we have just imported to plugins folder!.
META-INF
plugin.xml
plugins
.
(the Designer plugin folder)plugins/com.cloveretl.custom.loremipsum/bin/
plugins/com.cloveretl.custom.loremipsum/lib/loremipsum-1.0.jar
com.cloveretl.designer.loremipsum
(the Designer plugin) project and select “Export…” from the context menu.
Install into host. Repository:
(no need to change anything else)Now you can try your new functions anywhere in CloverDX, let's see it in Reformat's Source tab: