Snowflake is a cloud-based data storage and analytics engine – or a “data warehouse as a service”. It allows you to store massive amounts of data in your warehouse and make it accessible to all your users while providing maximum query performance.
CloverDX acts as an ETL tool for Snowflake, enabling you to easily connect to your Snowflake data warehouse, load or unload data and run various queries to manage your data.
CloverDX offers two ways of connecting to Snowflake.
The first way is via a JDBC driver that allows the largest flexibility to run various queries on the target Snowflake instance. The only requirement in this case is a driver that can be downloaded directly from Snowflake’s website and then added to Designer or Server.
The second way – using a custom subgraph-based connector – allows you to connect to your Snowflake instance and perform high-performance bulk data loads that use all the power Snowflake offers.
You can download code below to show you how to create this subgraph.
As an ETL tool for Snowflake, CloverDX offers fast loading of data.
The bulk writer component is implemented as a subgraph (a user-defined component) that performs several operations to load the data into Snowflake instance – it loads the data into staging area in Snowflake, then loads data from the staging area into a table and finally deletes any local temporary files.
This is the fastest way of getting your data to Snowflake, as the data is uploaded to the cloud in parallel in multiple threads. This means Snowflake can run its data loads in parallel as well. As such, this is the best way to load large volumes of data into your warehouse.
Watch the webinar
See how CloverDX helps automate data feeds into a Snowflake or Redshift cloud data warehouse.
The sample project shows how to create a CloverDX subgraph (reusable component) that allows you to write data to your Snowflake instance via bulk-write operation (i.e. using PUT and COPY commands). You can use the subgraph to write data to Snowflake instance (included in the example). Additionally, you can compare the performance of writes between the bulk method and regular JDBC connection.
We'd be happy to walk you through how CloverDX's Snowflake connector simplifies bulk-loading.
