Sls

Sls sink connector

Support Those Engines

Spark
Flink
SeaTunnel Zeta

Key Features

• exactly-once
• cdc
• timer flush

Description

The Sls sink connector writes SeaTunnel rows to Alibaba Cloud Simple Log Service (SLS). Each SeaTunnel row is serialized as JSON and written to SLS as a log item whose content key is content.

Supported DataSource Info

To use the Sls connector, download the following dependency by using install-plugin.sh or from the Maven central repository.

Datasource	Supported Versions	Maven
Sls	Universal	Download

Sink Options

Name	Type	Required	Default	Description
endpoint	String	Yes	-	Alibaba Cloud SLS endpoint, for example cn-hangzhou.log.aliyuncs.com or an intranet endpoint.
project	String	Yes	-	Alibaba Cloud SLS project.
logstore	String	Yes	-	Alibaba Cloud SLS logstore.
access_key_id	String	Yes	-	Alibaba Cloud AccessKey ID.
access_key_secret	String	Yes	-	Alibaba Cloud AccessKey secret.
source	String	No	SeaTunnel-Source	Source tag written to SLS log groups.
topic	String	No	SeaTunnel-Topic	Topic tag written to SLS log groups.

Notes

• The configured RAM user must have permission to write logs to the target project and logstore.
• The sink writes data as soon as write is called. It does not provide exactly-once commit semantics.
• Do not print access_key_secret in logs or job descriptions.

Task Example

Write Rows to SLS

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  FakeSource {
    row.num = 10
    map.size = 10
    array.size = 10
    bytes.length = 10
    string.length = 10
    schema = {
      fields = {
        id = "int"
        name = "string"
        description = "string"
        weight = "string"
      }
    }
  }
}

sink {
  Sls {
    endpoint = "cn-hangzhou-intranet.log.aliyuncs.com"
    project = "project1"
    logstore = "logstore1"
    access_key_id = "xxxxxxxxxxxxxxxxxxxxxxxx"
    access_key_secret = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    source = "seatunnel-demo"
    topic = "fake-source"
  }
}

Changelog

Change Log

Change	Commit	Version
[Chore] fix typos filed -> field (#9757)	https://github.com/apache/seatunnel/commit/e3e1c67d29	2.3.12
[Improve][API] Optimize the enumerator API semantics and reduce lock calls at the connector level (#9671)	https://github.com/apache/seatunnel/commit/9212a77140	2.3.12
[improve] sls options (#9260)	https://github.com/apache/seatunnel/commit/126164508b	2.3.11
[Feature][Checkpoint] Add check script for source/sink state class serialVersionUID missing (#9118)	https://github.com/apache/seatunnel/commit/4f5adeb1c7	2.3.11
[Improve] restruct connector common options (#8634)	https://github.com/apache/seatunnel/commit/f3499a6eeb	2.3.10
[Feature]Check Chinese comments in the code (#8319)	https://github.com/apache/seatunnel/commit/d58fce1caf	2.3.9
[Improve][dist]add shade check rule (#8136)	https://github.com/apache/seatunnel/commit/51ef800016	2.3.9
[Improve][Sls] Add sls sink connector、e2e、doc (#7830)	https://github.com/apache/seatunnel/commit/048c47d966	2.3.9
[Fix][Connector-V2] Fix some throwable error not be caught (#7657)	https://github.com/apache/seatunnel/commit/e19d73282e	2.3.8
[Feature][Connector-V2] add Aliyun SLS connector #3733 (#7348)	https://github.com/apache/seatunnel/commit/527c7c7b5f	2.3.7