• Blog
  • Contact
  • Sign in
CloverDX
Product
  • Overview
  • CloverDX Data Integration Platform
  • What's new in CloverDX 6
  • Pricing
  • CloverDX plans
  • Deployment
  • CloverDX on AWS
  • CloverDX on Azure
  • CloverDX on Google Cloud
  • CloverDX on-premise
  • Resources
  • Customer Portal
  • Documentation
  • Downloads & Licenses
  • Webinars
  • Academy & Training
  • Release Notes
  • CloverDX Forum
  • CloverDX Blog
  • Tech Blog
  • Other resources
isometric-illustration--product@2x 1

Get under the hood of CloverDX

See how CloverDX can benefit your business with a live demo. Simply get in touch with our team and we’ll handle the rest.

Book a demo
Solutions
  • By Industry
  • Banking
  • Capital Markets
  • Consultancy & Advisory
  • FinTech
  • Government Agencies
  • Healthcare
  • By Use Case
  • Data Quality
  • Data Ingest
  • Data Warehousing
  • Data Migration
  • Digital Transformation
  • Enterprise Data Management
  • Risk & Compliance
  • Anonymization
How F3 Group use CloverDX to ingest more client data - webinar
Customer interview

Formula 3: Staying Small And Agile While Working With Large Enterprise Ecosystems

Browse webinars
Services
  • Services
  • Onboarding & Training
  • Professional Services
  • Customer Support

More efficient, streamlined data feeds

Discover how Gain Theory automated their data ingestion and improved collaboration, productivity and time-to-delivery thanks to CloverDX.

 

Read case study
Customers
  • By Use Case
  • Analytics and BI
  • Data Ingest
  • Data Integration
  • Data Migration
  • Data Quality
  • Data Warehousing
  • Digital Transformation
  • By Industry
  • App & Platform Providers
  • Banking
  • Capital Markets
  • Consultancy & Advisory
  • E-Commerce
  • FinTech
  • Government
  • Healthcare
  • Logistics
  • Manufacturing
  • Retail
Migrating data to Workday - case study
Case study

Effectively Migrating Legacy Data Into Workday

Read customer story
Company
  • About CloverDX
  • Our story & leadership
  • Contact us
  • Partners
  • CloverDX Partners
  • Become a partner
Pricing
Demo
Trial

Improving subgraphs in CloverDX 4.1

Posted June 30, 2015
3 min read
Improving subgraphs in CloverDX 4.1

Last year, we introduced the subgraph as the most important innovation in CloverDX 4.0 (formerly CloverETL). Since the introduction, we've received positive feedback from our customers and we've been improving subgraphs in the following releases to refine the feature.

We strive for CloverDX to become a more versatile tool, capable of tackling new data challenges. In release 4.1, we added many new features.

In this blog, we will focus on new features improving subgraphs such as optional ports and the capability to dynamically disable components based on the connected edges.

With the help of optional ports, you are creating an elegant subgraph that can work in many different scenarios, rather than run several specific subgraphs to achieve the same goal. In addition, it will allow an experienced developer to change the behaviour of a subgraph based on different conditions.

We are very fond of these features so we re-engineered the example called “Read and Analyze Tweets” from CloverDX 4.0 to incorporate optional ports to demonstrate the enhanced capability.

Let's take a look at the example.

Example of improved subgraphs in latest CloverETL

As you can see, it is pretty straight forward graph. It reads the Twitter feeds and then it uses the subgraph “Sentiment classify” to sort tweets as positive, negative or neutral based on the keywords.

Here is the “Sentiment classify” subgraph processing the Twitter feed.

Inside of improved subgraph in CloverETL

We are using two keyword lists in this example. The lists are simple text files, however, you can use any data source. „Default sentiment keywords“ reader is located in the subgraph and „My sentiment keywords“ is located before the subgraph. If users choose to use custom keywords, they just simply connect „My sentiment keywords“ to the subgraph.

Improved subgraphs in CloverETL with possibility to set port as optional.

It is really easy to set ports as optional. You simply select a port directly in the subgraph or in the outline and set it as a required or an optional port.

We are also illustrating another new feature in this example as well. Dynamically disabled components. Right-click on the „Default keyword list“ reader and choose enable.

Improved subgraphs with conditional ports in CloverETL

This reader is conditionally disabled as we do not want to mix the two keywords lists. As you connect the edge with custom keywords, the default keywords reader is automatically disabled.

Optional Ports

Whereas previously all ports in subgraphs had to have edges connected, regardless of whether data is coming through the edge, it can now be defined as optional.

In our example you can see, there are several behavioral options for ports. If you declare a port as optional, you need to choose its behavior. The behavioral options are a) removal of edge if port is not connected or b) keep the edge, but discard all records. Every behavior type has a specific use. You can learn more about all behavior in our other blog dedicated to practical use of optional ports.

Dynamically enabled/disabled components

With the implementation of optional ports, we've encountered a few issues. What if the removal of edge in the subgraph prevents some component within the subgraph to work properly. How to avoid such trouble?

The solution we came up with was to expand the already existing disable component feature. In addition to the existing options to disable component either manually or based on graph parameter, we've added the capability to dynamically disable components based on edge connection to the port of the component.

We utilized this feature in the previous Twitter example. „Default sentiment keywords“ reader was conditionally disabled based on the connection of the other edge. Once you connect the edge with “My sentiment keywords”, the subgraph automatically disable “Default sentiment keywords”.

Although we introduced this as a subgraph feature to complement optional ports, it can be very handy in common graphs as well.

If you are veteran user of CloverDX, you will probably notice missing “pass through” option from previous versions of CloverDX. We decided, that disabling components is the same as “pass through” and we combined those two options. So if you disable component now the edge will pass through the components as it is not there.

We believe these new additions will be very beneficial to making your data processes easier to prepare and faster to run.

These new features and improvements are available in our latest release of CloverDX 4.1. Download your evaluation copy today.

Share

Facebook icon Twitter icon LinkedIn icon Email icon
Try CloverDX for 45 days  Full access to Tech Support as if you were a customer

Newsletter

Subscribe

Join 54,000+ data-minded IT professionals. Get regular updates from the CloverDX blog. No spam. Unsubscribe anytime.

CloverDX logo
Book a demo
Get the free trial
  • Company
  • Our story
  • Contact
  • Partners
  • Our partners
  • Become a partner
  • Product
  • Platform overview
  • Plans & Pricing
  • Customers
  • By Use Case
  • By Industry
  • Deployment
  • On-premise
  • AWS
  • Azure
  • Google Cloud
  • Services
  • Onboarding & Training
  • Professional Services
  • CloverCARE Support
  • Resources
  • Customer Portal
  • Documentation
  • Downloads & Licenses
  • Webinars
  • Academy & Training
  • Release Notes
  • CloverDX Forum
  • CloverDX Blog
  • Tech Blog
  • Other resources
Blog
Why data accessibility is essential for your digital transformation
Data Integration
4 barriers to making data-driven decisions
Data Strategy
4 steps to providing a data-driven customer experience
Data Integration
Implementing data democratization: 3 ways to make your data more accessible
Data Innovation
© 2023 CloverDX. All rights reserved.
  • info@cloverdx.com
  • sales@cloverdx.com
  • ●
  • Legal
  • Privacy Policy
  • Cookie Policy
  • EULA
  • Support Policy