Skip to main content

CDC Sink: Overview

  • CDC Sink is a RavenDB ongoing task that reads Change Data Capture (CDC) streams from a relational database
    and writes the resulting documents into RavenDB.

  • CDC Sink is the reverse of SQL ETL:
    instead of pushing data from RavenDB to a relational database,
    CDC Sink pulls data from the relational database into RavenDB.

  • The relational database remains the source of truth,
    while the CDC Sink task continuously maps normalized relational tables into RavenDB documents.

  • Supported source databases:

    • PostgreSQL - via logical replication
    • SQL Server - via native CDC (polling change tables)
    • MySQL / MariaDB - via binlog replication

  • In this article:

Why use CDC Sink​

CDC Sink solves the problem of moving data from a relational database into RavenDB without changing the source application.

  • Migrate from SQL to RavenDB
    Automatically and continuously transform normalized relational tables into RavenDB documents,
    such as an Order document that embeds LineItems and references a Customer document,
    without changing your application.

  • Build a read-optimized view
    Your transactional system uses a relational database, but your API layer needs denormalized documents.
    CDC Sink creates and maintains those documents without touching your existing application.

  • Gradual migration
    Keep your existing application running while RavenDB documents are built in the background.
    Applications can start reading from RavenDB while writes still go to the relational database.

  • Event-driven side effects
    Using JavaScript patches, every INSERT, UPDATE, or DELETE in the source database can trigger custom logic in RavenDB - computing derived fields, maintaining running totals, or writing custom transformations.

How it works​

A CDC Sink task continuously reads changes from the source relational database and applies them to RavenDB documents.

Initial load​

By default, when a CDC Sink task starts for the first time, it performs a full scan of every configured table using keyset pagination. This populates RavenDB with the current state of the data before streaming begins.

The task can also be configured to skip the full scan and start from new CDC changes only, without scanning existing rows - useful when you only need to capture changes going forward.

Initial load progress is persisted per-table.
If the task is restarted, it resumes from where it left off rather than re-scanning.

Streaming changes​

After the initial load, CDC Sink switches to streaming changes continuously.
Changes are grouped into transactions, preserving the exact order of operations from the source database.
Partial transactions are never written to RavenDB - all changes within a source database transaction are applied together.

Document model​

The relational model is mapped to RavenDB documents through configuration:

  • Root tables map to RavenDB collections (one document per row)
  • Embedded tables become nested arrays or objects within parent documents
  • Linked tables become document ID references to related documents

See Schema design for details.

Task lifecycle​

  1. Create
    Define the CDC Sink task in Studio or via the Client API.
    Specify the connection string, table mappings, and transformation options.
    See Create a CDC Sink task. You can later Update or Enable/Disable the task.

  2. Verify
    CDC Sink verifies that the source database is properly configured,
    including permissions, replication prerequisites, and table configuration.

  3. Initial load
    Full table scan populates RavenDB with current data.
    Progress is tracked per table and persists across restarts.

  4. Stream
    Continuous change streaming begins.
    All INSERTs, UPDATEs, and DELETEs are applied to RavenDB documents as they occur.

  5. Monitor
    Monitor the task state, or Get task info via Studio or via the Client API.

  6. Retire
    Delete the task in RavenDB when no longer needed.
    Retiring a task removes it on the RavenDB side only.

    Change-tracking artifacts created in the source database (such as a PostgreSQL replication slot and publication)
    are not removed automatically and must be cleaned up by the source-database administrator.
    Learn more in Delete a CDC Sink task.

Licensing​

CDC Sink is available on an Enterprise license.

Learn more about licensing in Licensing overview.

In this article