Skip to main content
Version: Next

Typesense

Typesense Source Connector

Description​

Reads documents from a Typesense collection. The source supports bounded batch reads and can pass Typesense search parameters through query.

Key Features​

Options​

NameTypeRequiredDefault
hostsarrayyes-
collectionstringyes-
schemaconfigyes-
api_keystringyes-
protocolstringnohttp
querystringno-
batch_sizeintno100
common-optionsno-

hosts [array]​

The access address of Typesense. Use the host:port format, for example: ["typesense-01:8108"]. Multiple hosts are supported.

collection [string]​

The name of the Typesense collection to read from, for example: "companies".

schema [config]​

The columns to be read from Typesense. For more information, please refer to the guide.

api_key [string]​

The api_key for Typesense security authentication.

protocol [string]​

The protocol used to connect to Typesense. The default value is http. Use https for Typesense Cloud or other TLS-enabled endpoints.

query [string]​

Typesense search parameters, for example q=*&filter_by=num_employees:>9000. If it is not set, the source reads all documents returned by the default search.

batch_size [int]​

The number of records to query per batch when reading data.

Common Options​

For common parameters of Source plugins, please refer to Source Common Options.

Task Example​

Read Documents With A Filter​

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  Typesense {
    hosts = ["localhost:8108"]
    collection = "companies"
    api_key = "xyz"
    query = "q=*&filter_by=num_employees:>9000"
    batch_size = 100
    schema = {
      fields {
        company_name_list = array<string>
        company_name = string
        num_employees = long
        country = string
        id = string
        c_row = {
          c_int = int
          c_string = string
          c_array_int = array<int>
        }
      }
    }
  }
}

sink {
  Console {}
}

Changelog​

Change Log
ChangeCommitVersion
[Improve][API] Optimize the enumerator API semantics and reduce lock calls at the connector level (#9671)https://github.com/apache/seatunnel/commit/9212a771402.3.12
[improve] typesense options (#9398)https://github.com/apache/seatunnel/commit/bf20a3e6a82.3.12
[Feature][Checkpoint] Add check script for source/sink state class serialVersionUID missing (#9118)https://github.com/apache/seatunnel/commit/4f5adeb1c72.3.11
[Fix] Fix error log name for SourceSplitEnumerator implements class (#8817)https://github.com/apache/seatunnel/commit/55ed90ecaf2.3.10
[Improve] restruct connector common options (#8634)https://github.com/apache/seatunnel/commit/f3499a6eeb2.3.10
[Feature]Check Chinese comments in the code (#8319)https://github.com/apache/seatunnel/commit/d58fce1caf2.3.9
[Improve][dist]add shade check rule (#8136)https://github.com/apache/seatunnel/commit/51ef8000162.3.9
[Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786)https://github.com/apache/seatunnel/commit/6b7c53d03c2.3.9
[Fix][Connector-V2] Fix known directory create and delete ignore issues (#7700)https://github.com/apache/seatunnel/commit/e2fb6795772.3.8
[Feature][Connector-V2] Support typesense connector (#7450)https://github.com/apache/seatunnel/commit/138d2a4eb22.3.8