Watch now

What's new in CloverDX 7.5?

Explore the latest features from version 7.5 of CloverDX in this live demo and Q&A session led by CloverDX's VP of Product, Branislav Repcek.

Explore how teams are building flexible data pipelines with CloverDX

Discover real-world examples and success stories from CloverDX users to see how teams are building adaptable pipelines that meet evolving business needs.

data ingestion pipeline
Get in touch

Have questions?
Our team is here to help

Request an in-depth feature demo or enquire about upgrading to the latest version of CloverDX.

Get in touch and we'll help you out.

illustration-cdx7--data-analyze@2x
Jump to the section you need

Timestamps

  • 2:22 - Overview of AI Assistant in Designer
  • 10:07 - Data Manager Improvements
  • 17:44 - Wrangler and Platform Improvements
  • 26:48 - Live Q&A: Licensing and Providers
  • 29:34 - Demo: AI Assistant in Designer
  • 35:45 - Demo: MCP and Data Manager Customization Skills
  • 40:34 - Demo: Wrangler


Transcript

Introduction [0:03]

Jonathan Burnett: Hello everyone, and thanks for joining us as we take a quick look at what's new in CloverDX 7.5. I'm Jonathan from CloverDX, and in just a moment I'll hand over to our VP of Product, Branislav, who will walk us through the latest features, enhancements, and everything else you should know about the latest CloverDX release.

As always, if you have any questions during the session, just drop them into the Q&A box and we'll get round to answering them as soon as possible. Also, rest assured that we'll send you the full webinar recording plus slides after the webinar. With that said, let's get started. So, over to you, Branislav.

Branislav Repček: All right — thank you, and thank you everyone for joining us on the call today. As Jonathan mentioned, let's talk about CloverDX 7.5.

As always, a little reminder of the release cadence we have every year: we're at CloverDX 7.5, which is our middle release this year — we'll have two more before the end of the year.

Overview [0:44]

Branislav Repček: If you look at CloverDX 7.5 at a high level (and I'll go through all of this in more detail throughout the webinar), we've introduced quite a lot of AI features — that's the first thing I'll talk about. The biggest one is the AI Assistant in Designer. It's a technology preview of a new feature that allows you to use agents to develop and build your solutions within CloverDX Designer.

This is seamlessly integrated into Designer, so you don't have to use other tools — but you can, because the way we've built it means you can use the same back end to work with other tools. For example, if you're using Claude Code, Visual Studio Code, or similar, you can connect via our MCP server and work that way.

Of course, there are many other changes that don't relate to AI as much. We're introducing a new SDK for Data Manager that lets you configure your own customizations — similar to what we already allow with Data Apps. And there are many other improvements across the board in Business Tools and across the whole product. For example:

  • New data set categories in Data Manager
  • Better, more configurable permission management in Wrangler and Data Manager
  • Many performance and usability improvements, and more

AI Assistant in Designer [2:22]

Let's start with the AI Assistant — probably the biggest thing, and I guess what you've been waiting for for a while with CloverDX. We're releasing the Assistant as a technology preview, meaning we're still heavily working on it and it will likely look different the next time you use it. Please take that into consideration, and don't run this in production — use it for experimenting and figuring out how it works.

The goal for the Assistant is simple: we wanted to build something that improves your productivity within Designer and across the whole platform, so you can offload development work and have more time for other tasks — designing, working with your data, delivering to clients, and so on.

It has multiple modes covering the whole lifecycle of what you can build in CloverDX — from smaller tasks like building jobs, all the way to designing whole solutions based on a specification.

Importantly, this is built so that you use your own model provider. We don't provide the AI service ourselves — instead, you configure, for example, OpenAI or Anthropic large language models within Designer, and the Assistant uses those.

What it can do [3:47]

You can think of the Assistant as three roles rolled into one:

  1. Companion for building jobs — it can help with basic tasks like understanding files or data, building graphs or job flows, and modifying existing jobs (it doesn't have to be a new solution — it understands things built in previous versions of CloverDX too). It's great for offloading mechanical work: build one connector, and ask it to build 20 more just like it.
  2. Architect mode — for much bigger tasks. Often you'll have a specification written in a Word document, PDF, markdown, or text file. Drop it into the project, have the Assistant read it, and it will plan the whole development for you — splitting it into multiple sprints so it produces consumable tasks rather than one massive deliverable at the end. It helps you navigate the whole process from build through to completion.
  3. Troubleshooter — you can ask it to figure out what's wrong with a solution, whether built by the Assistant or not. It can look at logs, investigate what happened, and suggest a solution — whether that means fixing the data or modifying the job to handle exceptions (it can even make the modification for you).

It automatically picks the right role depending on what you ask.

How it works under the hood [5:47]

We don't provide any assistant service ourselves, so there are no credits to worry about. You use Designer to talk to the Assistant — that's the main orchestration point. Designer talks to the server where your projects live (so the Assistant only works with server projects, not local-only ones), and the server provides multiple tools for the Assistant to use:

  • An MCP server that the Assistant uses to work with the server itself — providing dozens of tools to work with files, view logs, build jobs, debug them, and so on.
  • A knowledge base — everything the Assistant needs to understand how to build things in CloverDX: how components work, how graphs and job flows fit together, and larger patterns (e.g. how to build a warehouse in CloverDX).

The server doesn't need connectivity to LLM providers — that's handled by Designer. This is good for security, especially if your environment needs to stay tightly locked down: you don't need to open any ports on the server, since all LLM communication happens from Designer.

This is a bring-your-own-key (or even bring-your-own-model) setup — you configure Designer to talk to whichever provider you like; all the popular ones are supported. You pick models based on your budget and the performance/quality you need — no complicated configuration required. You can also use local models (e.g. via Ollama), though you'll need very large models (tens of billions of parameters) to get reasonable results — not something you'll run on a laptop.

MCP server improvements [8:05]

We've also had an MCP server in CloverDX for a while (since CloverDX 7.3), but we've made significant improvements to it. There are now more than 50 tools, compared to six or seven before — ranging from simple ones (navigating the file system on the server) to more advanced ones (working with the knowledge base, debugging jobs, spawning sub-agents, and more).

You can access the MCP server through any compatible external tool — for example, Claude, ChatGPT, and so on — which lets you use CloverDX Server from third-party tools: building jobs, diagnosing issues, asking about recent failures, and even having fixes applied automatically.

Broader LLM connectivity [9:26]

As we worked on this, we realized there are many different ways people connect to LLMs, so we've improved connectivity across the board — for example, adding the ability to connect to Microsoft Foundry models via Azure AI Gateway/API Management. This gives you more options for securing your communication with LLMs, and it applies to the Assistant in Wrangler, the Designer Assistant, and the AI Client component if you're using LLMs as part of your jobs.

Data Manager Improvements [10:07]

Categories [10:10]

The first thing to mention is the introduction of categories in Data Manager. This helps when you have a lot of data sets you want to group for easier navigation. You can now assign user-defined categories to each data set — create new categories or pick from existing ones — and data sets will be organized accordingly. This works for both reference and transactional data sets. It's purely organizational: nothing changes in how your data sets are used, so you don't need to change any jobs that work with them. This is something we plan to extend to other parts of the product in future releases.

Usability improvements [11:10]

  • Larger values: Data Manager has never had strict size limits on data, so you could already write several paragraphs of text into a string value. Now there's a more comfortable string editor that automatically expands into a resizable, scrollable multi-line editor as you type more. You can also use Shift+Enter to insert new lines.
  • Multi-select across pages: Previously you could only select items within a single page (up to 1,000 items — the default max page size), so working with more required multiple passes. Now you can select all records in the entire data set, including with filters applied, with no limit — and any subsequent operation (approving, deleting, etc.) applies across the whole selection. This makes bulk operations on large volumes of data much faster.

Reference data set improvements [12:53]

  • Custom label columns for lookups: Previously, when you defined a reference data set for use as a lookup, the label column had to have the technical name "label" (though it could have a friendly display name). Now you can use any column as the label — and this can differ per place the same lookup is used. For example, one lookup for country codes could show the country name as the label in one column and the country code in another, all from the same data set.
  • Auto-generated keys as lookup keys: Data Manager creates an internal ID for every row. You can now use this ID directly as the lookup key. This is especially useful for small data sets that exist purely to provide a simple enumeration of values — previously you had to create two columns (a display value and an artificial key); now you don't need the extra key column at all. Together, these two changes reduce the number of columns you need and make navigating lookup-heavy data sets easier.

Customization framework [14:39]

A bigger piece of work sits between business users and the developers supporting them: we've improved the customization framework for Data Manager. Using an approach similar to Data Apps, you can create a zip file of additional resources, configure it on your server (via two configuration properties), and use it to customize Data Manager's UI and inject new behaviors — for example, data profiling within columns, or the categories feature shown earlier (which was originally built and tested this way).

This lets you fine-tune Data Manager to your users' needs — adding custom buttons, new behaviors that operate on your data, and so on — all with a bit of injected JavaScript. We've split this into two parts:

  1. The framework itself — the technical scaffolding that makes this possible.
  2. A set of AI skills for Data Manager, shared via GitHub (link provided in the webinar materials). Installing these skills into your development environment of choice (e.g. Visual Studio Code) loads four skills that help you build these customizations.

In practice: write a specification (often just markdown or text files), combine it with the skills and an LLM in your IDE of choice, and it produces a zip file you deploy to your server. After the first deployment (which requires a server restart), further updates can be applied without restarting the server — so you can iterate on user requests within minutes.

Wrangler and Platform Improvements [17:44]

Dynamic previews [17:47]

One of the more visible changes in Wrangler is to dynamic previews, introduced in a previous release. These let you preview the output of a step as you build it. We've now extended this to work better for steps that remove data (filtering rows by formula, removing rows with errors, etc.) — rows that will be removed are now highlighted in yellow, so you can immediately see whether your filter condition matches what you expect, not just the resulting output.

Configurable sample size [18:47]

Previously, the initial data sample in Wrangler was capped at 1,000 records. We've improved rendering and back-end performance, so you can now select far larger sample sizes and, in many cases, see the whole data set in preview — while the job itself still processes everything at run time. (Very large samples may be demanding on your browser, given the volume of values being rendered, but it will work.)

Group-based permissions [19:45]

For both Data Manager and Wrangler workspaces, you can now assign permissions to groups, not just individual users. Groups can be tied to something like Active Directory on the server side, so in large organizations you can manage access for many users at once — add a group to a workspace or data set, and membership changes (e.g. someone new joining the team) are reflected automatically, with no extra work in Wrangler or Data Manager.

Data Apps [20:45]

  • Immediate execution: Data Apps with no parameters now run immediately when clicked, skipping the empty configuration screen that previously appeared before the "Run" button — less friction, faster to use.
  • Change notifications: Applications can now notify Data Manager when they change a data set — even one other than the one they were launched from. Data Manager shows a notification with a refresh option once the app finishes. Refreshing is manual (not automatic) so it doesn't interrupt whatever you're currently doing — the same notification and refresh behavior appears in the expanded "details" view as well.

Other platform changes [22:43]

  • Salesforce OAuth authentication: Salesforce will remove the ability to authenticate with username and password to call their APIs roughly a year from now. Well ahead of that, CloverDX now supports OAuth 2.0 authentication for Salesforce components. This is implemented by layering a Salesforce-specific OAuth provider on top of the familiar OAuth connection type — you configure and authorize it once, then reference it from your Salesforce connections. Your job logic (queries, business logic) doesn't need to change at all; only the connection configuration does.
  • S3 protocol: Now uses a newer AWS SDK, improving security and compatibility with newer AWS services. There are no new features yet, but this lays the groundwork for better AWS ecosystem integration going forward. Most jobs won't need any changes; this happens automatically. It's worth being aware of if anything in your setup specifically depended on AWS SDK v1 behavior.
  • Copy Files / Move Files — timestamp handling: Previously, how the "last modified" timestamp behaved during copy/move operations depended on many factors (file system, system call used, etc.) — consistent per file system, but hard to predict without testing. A new "preserve file timestamps" attribute now lets you choose the behavior explicitly. The old default behavior is now available as "no special handling." Other options let you force timestamp preservation, though support depends on the target file system — for example, S3 lets you read timestamps but not set them (so forcing this will fail), while FTP support varies by server. Check the linked documentation for a full table of which file systems support what, and review your configuration if you rely on specific timestamp behavior.

Live Q&A (during slides) [26:48]

Jonathan Burnett: Branislav, we have two questions about the CloverDX AI Assistant. First: is the CloverDX Assistant part of the existing license, or would someone need to buy a new one?

Branislav Repček: Good question. For the technology preview, you can get a license simply by writing to our sales team and they'll provide one. In the future you'll still need a separate license, with commercial terms to be announced later — but for now the technology preview is free of charge, and you can use it as much as you want.

Jonathan Burnett: Nice. And a second question — could you give a bit more context on the licensing model for the AI Assistant overall?

Branislav Repček: Sure. The way it works, you need two licenses in Designer. [Shows the license configuration screen.] One activates Designer itself, and the other activates just the Assistant — you can use Designer without the Assistant, but you need the second license to use it. That's just for what's inside Designer.

You'll also need an account with an LLM provider — for example OpenAI, Anthropic, Google, or Microsoft — so you can use their models. That configuration sits within Designer itself. Since we're not providing the AI service, there are no CloverDX tokens or credits involved; you pay the provider directly for consumption. In my setup here, I'm using OpenAI, providing my own API key, and I've selected a set of models to use — this is all quite configurable. You'll need an account with one of these providers, and there are a good number of provider options to choose from.

Jonathan Burnett: Perfect, thank you. One quick follow-up — have we tested, or do we plan to test, Langdock as an AI provider too?

Branislav Repček: We haven't tested it, but if it's OpenAI-compatible, it should work — if a provider exposes an OpenAI-compatible API, then it should work with the Assistant.

Jonathan Burnett: Thank you.

Demo — AI Assistant in Designer [29:34]

Branislav Repček: Coming back — I wanted to show you how the Assistant works, because there's actually quite a lot it can do.

Example 1 — understanding an unfamiliar file: As mentioned, it can start with something small, like looking into a file for you. [Opens the Assistant sidebar, enlarges it.] Say you have a file you don't recognize — in this case, a file in Czech, which is a little garbled. This might be something a client sends you, saying "I need this processed and I don't know what it is." You can ask the Assistant, and it will investigate — calling what we call the Data Inspector and any number of other MCP tools — and explain what the file is: it worked out the encoding and the overall structure.

As a follow-up, we asked it to build a graph that works with that data — which turned out to be tax-return data involving dividends — and it built a simple graph accordingly, which you can then run (it completed in about 350 milliseconds) and continue iterating on through conversation.

Example 2 — certification exercise (larger task): This is one of the exercises used at the end of CloverDX training, proving you've been paying attention — building a set of jobs that load data into a database, etc. Here we simply told the Assistant to read the certification exercise PDF and work out what to build. It loaded a number of tools to read and understand the specification, then invoked what we call the Architect, which plans the implementation. In this case it split the work into eight sprints and executed them one after another (with a fairly long conversation, including a few failures along the way) until the job was finished. I won't run the job itself, since it depends on a database I managed to crash before the webinar — but you can see the result: it performs aggregations, filtering, and even created its own subgraphs. Everything in this solution was built by the AI system, starting from an empty project.

Example 3 — a much bigger project: Here the specification was provided across six files rather than a few sentences on a slide. The Assistant handles this the same way, but for bigger problems it will also produce supporting documentation — for example, writing an architecture document describing the data flow it built. Again, this started as an empty solution with just some example files, and the Assistant built all of this itself.

So you can start small — asking the Assistant to tweak jobs you already have — or go much bigger, even without a full specification, prompting it step by step. Everything from small jobs to large, end-to-end solutions (including running and verifying results) can be done through the Assistant.

This all runs through Designer, backed by the same MCP server you could also use from other tools. Using it from Designer specifically gives you real benefits — you can see the resulting jobs and graphs immediately, whereas with an external tool like Claude Code, jobs would be created in the project but you'd still need to switch to Designer to view them. Using it directly in Designer is a much more integrated experience for your users.

Demo — MCP + Data Manager Customization Skills [35:45]

Next, I wanted to show the MCP server and the Data Manager customization skills in action. I used Visual Studio Code to build a data profiler extension for Data Manager, starting from a short specification file (not shown). As expected, it worked with the server, built everything, and produced a simple deployable artifact — a zip file that I loaded into Data Manager.

[Switches to Data Manager, opens and refreshes a data set.] With the plugin now activated, you can see some changes — a new button (left visible on purpose so it stands out) and some new icons. None of this is part of standard Data Manager; it was injected via this AI-built extension. The whole process — from a specification about half a page long to this working result — took about an hour and a half, roughly half of which was just me thinking through what I wanted. This kind of customization can be anything: we've also built category/tab views, and a custom search-and-replace implementation based on customer requirements.

Other Data Manager features shown:

  • Immediate-run Data Apps: apps with no parameters show a small icon and run instantly on click — no confirmation screen — both in the Data Apps catalog and the Data Manager toolbar.
  • Change notifications: running an app that loads more records into a data set (in this demo, bringing it to 1,500 records) triggers a notification rather than an automatic full-page refresh — clicking refresh updates just that data set without disrupting your current view or scroll position.
  • Expandable string editor: starts expanded for already-long text, and automatically expands, resizes, and can be repositioned as you type more.
  • Performance improvements (not easily demoable): faster data set loading via more parallel processing and lazy loading of large lookups (previously slow with lookups containing hundreds of thousands of values, now much faster even with several such lookups open at once) — and much faster search/replace on large data sets, in some cases reduced from minutes to seconds.

Demo — Wrangler [40:34]

Branislav Repček: Let's switch to Wrangler for a couple more things. [Opens a job.]

Q&A interruption — Jonathan Burnett: Quickly, a question has come in: is an additional license required just for using the MCP server?

Branislav Repček: Yes, the MCP server will require an additional license as well. For now it's activated for everyone as part of the technology preview, but in the future it will require a separate license too.

Jonathan Burnett: Thank you.

Branislav Repček: Coming back to Wrangler — first, the ability to change the sample size. This controls how many records are read initially — for example, when reading from a file. That number can shrink as you filter data further down the pipeline. I can increase it well beyond the previous default — say to 10,000 — and thanks to performance improvements, this stays responsive even on a data set that's full of errors on nearly every page. Scrolling to the end shows close to 10,000 records with only a handful filtered out.

Second, the dynamic preview for filtering steps: it recomputes the result of the step you're editing and highlights the rows that will be removed in yellow. Changing the filter condition — say, to a much larger threshold that removes most rows — refreshes the preview immediately, so you can easily see the effect of your filter before running the job. The same rendering approach applies to other transformations shown in the dynamic preview.

Closing [42:58]

Branislav Repček: All right, that's pretty much it. I'm not sure if we have any more questions.

Jonathan Burnett: No more questions in the Q&A box for now.

Branislav Repček: All right — thank you, Jonathan, for the great presentation, and thank you everyone for joining us today. We hope you found it useful. Keep an eye on your inbox for the full recording and slides with links, and hopefully we'll see you on one of our future webinars. Thanks again, everybody — have a great day.

Jonathan Burnett: Thank you everyone, and have a great day. Bye-bye.