Estuary Flow VS Matillion

Estuary was founded in 2019. But the core technology, the Gazette open source project, has been evolving for a decade within the Ad Tech space, which is where many other real-time data technologies have started.

Estuary Flow is the only real-time and ETL data pipeline vendor in this comparison. There are some other ETL and real-time vendors in the honorable mention section, but those are not as viable a replacement for Fivetran.

While Estuary Flow is also a great option for batch sources and targets, where it really shines is any combination change data capture (CDC), real-time and batch ETL or ELT, and loading multiple destinations with the same pipeline. Estuary Flow currently is the only vendor to offer a private cloud deployment, which is the combination of a dedicated data plane deployed in a private customer account that is managed as SaaS by a shared control plane. It combines the security and dedicated compute of on prem with the simplicity of SaaS.

CDC works by reading record changes written to the write-ahead log (WAL) that records each record change exactly once as part of each database transaction. It is the easiest, lowest latency, and lowest-load for extracting all changes, including deletes, which otherwise are not captured by default from sources. Unfortunately ELT vendors like Airbyte, Fivetran, Meltano, and Hevo all rely on batch mode for CDC. This puts a load on a CDC source by requiring the write-ahead log to hold onto older data. This is not the intended use of CDC and can put a source in distress, or lead to failures.

Estuary Flow has a unique architecture where it streams and stores streaming or batch data as collections of data, which are transactionally guaranteed to deliver exactly once from each source to the target. With CDC it means any (record) change is immediately captured once for multiple targets or later use. Estuary Flow uses collections for transactional guarantees and for later backfilling, restreaming, transforms, or other compute. The result is the lowest load and latency for any source, and the ability to reuse the same data for multiple real-time or batch targets across analytics, apps, and AI, or for other workloads such as stream processing, or monitoring and alerting.

Estuary Flow also has broad packaged and custom connectivity, making it one of the top ETL tools. It has 150+ native connectors that are built for low latency and/or scale. While may seem low, these are connectors built for low latency and scale. In addition, Estuary is the only vendor to support Airbyte, Meltano, and Stitch connectors as well, which easily adds 500+ more connectors. Getting official support for the connector is a quick “request-and-test” with Estuary to make sure it supports the use case in production. Most of these connectors are not as scalable as Estuary-native, Fivetran, or some ETL connectors, so it’s important to confirm they will work for you. Flow’s support for TypeScript and SQL also enables ETL.

• Modern data pipeline: Estuary Flow has the best support for schema drift, evolution, and automation, as well as modern DataOps.
• Modern transforms: Flow is also both low-code and code-friendly with support for SQL, TypeScript (and Python coming) for ETL, and dbt for ELT.
• Lowest latency: Several ETL vendors support low latency. But of these Estuary can achieve the lowest, with sub-100ms latency. ELT vendors generally are batch only. 
• High scale: Unlike most ELT vendors, leading ETL vendors do scale. Estuary is proven to scale with one production pipeline moving 7GB+/sec at sub-second latency.
• Most efficient: Estuary alone has the fastest and most efficient CDC connectors. It is also the only vendor to enable exactly-and-only-once capture, which puts the least load on a system, especially when you’re supporting multiple destinations including a data warehouse, high performance analytics database, and AI engine or vector database.
• Deployment options: Of the ETL and ELT vendors, Estuary is currently the only vendor to offer open source, private cloud, and public multi-tenant SaaS.
• Reliability: Estuary’s exactly-once transactional delivery and durable stream storage makes it very reliable.
• Ease of use: Estuary is one of the easiest to use tools. Most customers are able to get their first pipelines running in hours and generally improve productivity 4x over time. 
• Lowest cost: for data at any volume, Estuary is the clear low-cost winner in this evaluation. Rivery is second.
• Great support: Customers consistently cite great support as one of the reasons for adopting Estuary.

• On premises connectors: Estuary has 150+ native connectors and supports 500+ Airbyte, Meltano, and Stitch open source connectors. But if you need on premises app or data warehouse connectivity make sure you have all the connectivity you need.
• Graphical ETL: Estuary has been more focused on SQL and dbt than graphical transformations. While it does infer data types and convert between sources and targets, there is currently no graphical transformation UI.

Of the various ELT and ETL vendors, Estuary is the lowest total cost option. Estuary only charges $0.50 per GB of data moved from each source or to each target, and $100 per connector per month. So you can expect to pay a minimum of a few thousand per year. But it quickly becomes the lowest cost pricing. Rivery, the next lowest cost option, is the only other vendor that publishes pricing of 1 RPU per 100MB, which is $7.50 to $12.50 per GB depending on the plan you choose. Estuary becomes the lowest cost option by the time you reach the 10s of GB/month. By the time you reach 1TB a month Estuary is 10x lower cost than the rest.

Matillion ETL is an on-premises ETL platform that was founded before the advent of cloud data warehouses, and is still primarily on premises. But its main destinations today are cloud data warehouses such as Snowflake, Amazon Redshift, and Google BigQuery.

Matillion combines many features to extract, transform, and load (ETL) data. More recently Matillion has been adding cloud options as part of the Matillion Data Productivity Cloud. It consists of a Hub for administration and billing, a choice of working with the on-premises Matillion ETL deployed as “private cloud” or Matillion Data Loader, a free cloud batch and CDC replication tool built on Matillion ETL but lacking many of its capabilities including transforms.

As with most of the mature ETL tools, Matillion has a strong set of features, but is harder to learn and use and is more expensive.

Perhaps one of the biggest advantages of Matillion is its ETL and orchestration, especially when compared to various ELT tools.

• Advanced transforms: Matillion ETL supports a variety of transform options, from drag-and-drop to code editors for complex transformations.
• Orchestration: Matillion offers advanced graphical workflow design and orchestration.
• Pushdown optimization: Matillion ETL can push down transformations to the target data warehouse.
• Reverse ETL: Matillion provides the ability to extract data from a source, cleanse it, and insert data back into the source.

• SaaS: Matillion ETL, its flagship product, is on-premises only. It does offer Data Loader, which is built on ETL, as a free cloud service for replication. There is also integration between Matillion ETL and the Matillion Cloud Hub for billing. While you can migrate work in Data Loader to ETL if you choose, it is a migration from the cloud to your own managed environment. 
• Free tier: Matillion Data Loader is free, but it’s limited and doesn’t support transforms. This can make it challenging to fully evaluate the tool before committing to a paid plan.
• Connectors: Matillion has fewer connectors than most (120+ in total.) You can invoke external APIs to access other systems, but access to all your sources and destinations can become an issue. Matillion is only used for loading data warehouses. 
• No CDC: Matillion ETL CDC, which was based on Amazon DMS (in turn based on Attunity) has been deprecated. So right now there is no CDC option with Matillion. 
• Schema evolution: Matillion does support adding columns to existing destination tables, deleting a column, and handling data type changes as sources change. But adding a table requires creating a new pipeline and there is no automation for schema evolution.
• dbt integration for SaaS: While Matillion ETL has a connector for dbt, there is no integration between Data Loader and dbt.
• Pricing: Compared to more modern ELT vendors, Matillion is expensive. It starts at $1000/month for 500 credits where each credit is a virtual core-hour similar to an AWS, Azure, or Google virtual core. This is really in the $1000s per month minimum. Data productivity Cloud consumes a credit per running task every 15 minutes, and only consumes when tasks are running. The smallest ETL unit is two cores, which means you consume 2 cores an hour, or nearly 3x the 500 credits every month.

Matillion doesn’t have a pay-as-you-go model. It starts at $1000/month for 500 credits where each credit is a virtual core-hour similar to an AWS, Azure, or Google virtual core. Pricing increases 25% per credit for advanced and 35% for enterprise with higher base commitments.

This is really in the $1000s per month minimum. Data productivity Cloud consumes a credit per running task every 15 minutes, and only consumes when tasks are running. The smallest ETL unit is two cores, which means you consume 2 cores an hour, or nearly 3x the 500 credits every month.