Skip to main content
Version: Next

Cassandra

Cassandra source connector

Description​

Read data from Apache Cassandra in batch mode.

The Cassandra source supports two read modes:

  • Single-table read with cql.
  • Multi-table read with tables_configs, where each entry contains one cql.

The source gets column names and data types from the result set returned by the configured CQL, so the CQL should return the columns that downstream steps need.

Key features​

Data Type Mapping​

Cassandra Data TypeSeaTunnel Data Type
asciiSTRING
varchar/textSTRING
varintSTRING
uuid/timeuuidSTRING
inetSTRING
tinyintBYTE
smallintSHORT
intINT
bigint/counterLONG
floatFLOAT
double/decimalDOUBLE
booleanBOOLEAN
timeTIME
dateDATE
timestampTIMESTAMP
blobARRAY\<BYTE>
listARRAY
setARRAY
mapMAP

Options​

NameTypeRequiredDefaultDescription
hostStringYes-Cassandra cluster address. Use host:port, and separate multiple hosts with commas.
keyspaceStringYes-Cassandra keyspace used by the session.
cqlStringNo *-CQL used to read one table.
tables_configsList\<Map>No *-Multi-table read configuration. Each item must contain one cql.
usernameStringNo-Cassandra username. Configure it together with password.
passwordStringNo-Cassandra password. Configure it together with username.
datacenterStringNodatacenter1Local datacenter name used by the Cassandra Java driver.
consistency_levelStringNoLOCAL_ONERead consistency level, such as LOCAL_ONE, ONE, QUORUM, or LOCAL_QUORUM.
common-optionsNo-Source plugin common parameters, such as plugin_output.

* Exactly one of cql or tables_configs must be provided.

host [string]​

Cassandra cluster address, the format is host:port , allowing multiple hosts to be specified. Such as "cassandra1:9042,cassandra2:9042".

keyspace [string]​

The Cassandra keyspace.

cql [String]​

The query CQL used to read data from Cassandra. Use this for single-table reads. It is mutually exclusive with tables_configs.

The connector uses the CQL result metadata to build the output schema. In normal cases, use a query that returns real table columns, for example select * from source_table or select id, name from source_table.

tables_configs [List\<Map>]​

Multi-table read configuration. Each entry must contain a cql field with the query for that table. It is mutually exclusive with root-level cql.

Do not configure the same source table more than once in tables_configs; the connector checks for duplicate table names during startup.

Example entry:

{
  cql = "SELECT id, name FROM keyspace.table1"
}

username [string]​

Cassandra user username.

password [string]​

Cassandra user password.

datacenter [String]​

The Cassandra datacenter, default is datacenter1.

consistency_level [String]​

The Cassandra read consistency level, default is LOCAL_ONE.

common-options​

Source plugin common parameters. For details, see Source Common Options.

Notes​

  • username and password are a pair. Configure both when authentication is enabled; omit both when the cluster does not require authentication.
  • datacenter must match the Cassandra cluster local datacenter name. The default is datacenter1, which is also the common Testcontainers default.
  • cql and tables_configs are mutually exclusive. Use cql for one result table and tables_configs when one source should read multiple Cassandra tables.
  • The source is a batch source. It reads the current query result and then finishes.
  • A single CQL query is read as one source split. Increasing job parallelism does not split one Cassandra table scan automatically.

Examples​

Single-table read​

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  Cassandra {
    host = "localhost:9042"
    username = "cassandra"
    password = "cassandra"
    datacenter = "datacenter1"
    keyspace = "test"
    cql = "SELECT * FROM test.source_table"
    plugin_output = "source_table"
  }
}

sink {
  Console {}
}

Multi-table read​

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  Cassandra {
    host = "localhost:9042"
    username = "cassandra"
    password = "cassandra"
    datacenter = "datacenter1"
    keyspace = "test"
    tables_configs = [
      {
        cql = "select id, c_int from mt_source_a"
      },
      {
        cql = "select id, c_int from mt_source_b"
      }
    ]
  }
}

sink {
  Cassandra {
    host = "localhost:9042"
    username = "cassandra"
    password = "cassandra"
    datacenter = "datacenter1"
    keyspace = "test"
    table = "mt_sink_table"
  }
}

Changelog​

Change Log
ChangeCommitVersion
[improve] cassandra connector options (#8608)https://github.com/apache/seatunnel/commit/d9201108cf2.3.10
[Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786)https://github.com/apache/seatunnel/commit/6b7c53d03c2.3.9
[Improve] Improve some connectors prepare check error message (#7465)https://github.com/apache/seatunnel/commit/6930a25edd2.3.8
[Improve][Common] Introduce new error define rule (#5793)https://github.com/apache/seatunnel/commit/9d1b2582b22.3.4
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755)https://github.com/apache/seatunnel/commit/8de74081002.3.4
[Feature][Connector V2] expose configurable options in Cassandra (#3681)https://github.com/apache/seatunnel/commit/73f63a50442.3.2
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd6010512.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab1665612.3.1
[Feature][API &amp; Connector &amp; Doc] add parallelism and column projection interface (#3829)https://github.com/apache/seatunnel/commit/b9164b8ba12.3.1
[Improve][Connector-V2] The log outputs detailed exception stack information (#3805)https://github.com/apache/seatunnel/commit/d0c6217f272.3.1
[Improve][Connector-V2][Cassandra] Unified exception for cassandra source & sink connector (#3435)https://github.com/apache/seatunnel/commit/28868797b72.3.0
[Feature][Connector-V2][Cassandra] Add Cassandra Source And Sink Connector (#3229)https://github.com/apache/seatunnel/commit/12268a6f4b2.3.0