• Blog
  • Contact
  • Sign in
CloverDX
Product
  • Overview
  • CloverDX Data Integration Platform
  • What's new in CloverDX 6
  • Pricing
  • CloverDX plans
  • Deployment
  • CloverDX on AWS
  • CloverDX on Azure
  • CloverDX on Google Cloud
  • CloverDX on-premise
  • Resources
  • Customer Portal
  • Documentation
  • Downloads & Licenses
  • Webinars
  • Academy & Training
  • Release Notes
  • CloverDX Forum
  • CloverDX Blog
  • Tech Blog
  • Other resources
isometric-illustration--product@2x 1

Get under the hood of CloverDX

See how CloverDX can benefit your business with a live demo. Simply get in touch with our team and we’ll handle the rest.

Book a demo
Solutions
  • By Industry
  • Banking
  • Capital Markets
  • Consultancy & Advisory
  • FinTech
  • Government Agencies
  • Healthcare
  • By Use Case
  • Data Quality
  • Data Ingest
  • Data Warehousing
  • Data Migration
  • Digital Transformation
  • Enterprise Data Management
  • Risk & Compliance
  • Anonymization
How F3 Group use CloverDX to ingest more client data - webinar
Customer interview

Formula 3: Staying Small And Agile While Working With Large Enterprise Ecosystems

Browse webinars
Services
  • Services
  • Onboarding & Training
  • Professional Services
  • Customer Support

More efficient, streamlined data feeds

Discover how Gain Theory automated their data ingestion and improved collaboration, productivity and time-to-delivery thanks to CloverDX.

 

Read case study
Customers
  • By Use Case
  • Analytics and BI
  • Data Ingest
  • Data Integration
  • Data Migration
  • Data Quality
  • Data Warehousing
  • Digital Transformation
  • By Industry
  • App & Platform Providers
  • Banking
  • Capital Markets
  • Consultancy & Advisory
  • E-Commerce
  • FinTech
  • Government
  • Healthcare
  • Logistics
  • Manufacturing
  • Retail
Migrating data to Workday - case study
Case study

Effectively Migrating Legacy Data Into Workday

Read customer story
Company
  • About CloverDX
  • Our story & leadership
  • Contact us
  • Partners
  • CloverDX Partners
  • Become a partner
Pricing
Demo
Trial

Querying Twitter in CloverDX: What You Need to Know

API
Posted January 23, 2014
2 min read
Querying Twitter in CloverDX: What You Need to Know

These days, social networks are pervasive. It’s virtually impossible to avoid some kind of interaction with at least a few of them. Not only that, but the volume of traffic using them means there’s a ton of interesting data available for analytics and BI within.

A typical example of such a popular network is Twitter, with more than 500 million tweets sent each day. Wouldn’t it be useful if you were able to start querying Twitter to find tweets you want and then process them in bulk? Being able to dig through heaps of social interactions in an effective manner is one of the core promises of Big Data. Simplifying data preparation in this way is very valuable. In this blog, I will show you how to do it with CloverDX.

First of all, you need to grant yourself access to Twitter so that you can use it to access the API later. Log in to your Twitter developer account and select “Create new application” to set up your application. Fill in the name, description, and website here if you want, leave the Callback URL field empty, and submit the form.

After submitting, you’ll get to a page with application details. There is an OAuth settings section on this page where you can find “Consumer key” and “Consumer secret.” You’ll need these to connect from CloverDX.

querying twitter

Further down on the page, there is a “Your access token” section. Use “Create my access token” button. This might take some time, so wait a few seconds and then reload the page. You should see your “Access token” and “Access token secret” there. These two values will be used in CloverDX too.

querying twitter

With that, you’re done working on the Twitter side. Let’s now proceed to CloverDX.

We’re going to be using REST API, so we’ll basically be performing HTTP requests. The best component to achieve this target is the HTTPConnector.

querying twitter

To configure the HTTPConnector component, you need to specify these five attributes:

  • URL
  • OAuth Consumer key
  • OAuth Consumer secret
  • OAuth Access token
  • OAuth Access token secret

All OAuth attributes are taken from the registered Twitter application (see above). The URL depends on REST API method you want to use. For example, using https://api.twitter.com/1.1/search/tweets.json?q=%40CloverDX will search for tweets related to @CloverDX.

The result is returned in JSON format. You can either store it into file (Output file URL attribute of the component) or map the response content to an output port and process with other downstream components (e.g. JSONReader).

The attached example graph queries for the current Twitter trends and tweets related to them. Parses returned JSON for tweets attributes and stores them into XML file. 

querying twitter

And with that, you've now waded through the noise to find exactly what you're looking for.

Share

Facebook icon Twitter icon LinkedIn icon Email icon
Try CloverDX for 45 days  Full access to Tech Support as if you were a customer

Newsletter

Subscribe

Join 54,000+ data-minded IT professionals. Get regular updates from the CloverDX blog. No spam. Unsubscribe anytime.

Related articles

Back to all articles
The importance of API services in today's integrations
API
4 min read

The importance of API services in today's integrations

Continue reading
Building APIs and user interfaces for your data workloads
API CloverDX
5 min read

Building APIs and user interfaces for your data workloads

Continue reading
Publish Data and Transformations over API 3 Core Design Patterns blog
API Data Integration
4 min read

Publish Data and Transformations over API: 3 Core Design Patterns

Continue reading
CloverDX logo
Book a demo
Get the free trial
  • Company
  • Our story
  • Contact
  • Partners
  • Our partners
  • Become a partner
  • Product
  • Platform overview
  • Plans & Pricing
  • Customers
  • By Use Case
  • By Industry
  • Deployment
  • On-premise
  • AWS
  • Azure
  • Google Cloud
  • Services
  • Onboarding & Training
  • Professional Services
  • CloverCARE Support
  • Resources
  • Customer Portal
  • Documentation
  • Downloads & Licenses
  • Webinars
  • Academy & Training
  • Release Notes
  • CloverDX Forum
  • CloverDX Blog
  • Tech Blog
  • Other resources
Blog
Why data accessibility is essential for your digital transformation
Data Integration
4 barriers to making data-driven decisions
Data Strategy
4 steps to providing a data-driven customer experience
Data Integration
Implementing data democratization: 3 ways to make your data more accessible
Data Innovation
© 2023 CloverDX. All rights reserved.
  • info@cloverdx.com
  • sales@cloverdx.com
  • ●
  • Legal
  • Privacy Policy
  • Cookie Policy
  • EULA
  • Support Policy