SelectDB Cloud
SelectDB Cloud sink connector
Descriptionโ
Used to send data to SelectDB Cloud. Both support streaming and batch mode. The internal implementation of SelectDB Cloud sink connector upload after batch caching and commit the CopyInto sql to load data into the table.
Version Supported
- supported
SelectDB Cloud version is >= 2.2.x
Key featuresโ
Optionsโ
name | type | required | default value |
---|---|---|---|
load-url | string | yes | - |
jdbc-url | string | yes | - |
cluster-name | string | yes | - |
username | string | yes | - |
password | string | yes | - |
table.identifier | string | yes | - |
sink.enable-delete | bool | no | false |
selectdb.config | map | yes | - |
sink.buffer-size | int | no | 10 1024 1024 (1MB) |
sink.buffer-count | int | no | 10000 |
sink.max-retries | int | no | 3 |
load-url [string]โ
SelectDB Cloud
warehouse http address, the format is warehouse_ip:http_port
jdbc-url [string]โ
SelectDB Cloud
warehouse jdbc address, the format is warehouse_ip:mysql_port
cluster-name [string]โ
SelectDB Cloud
cluster name
username [string]โ
SelectDB Cloud
user username
password [string]โ
SelectDB Cloud
user password
table.identifier [string]โ
The name of SelectDB Cloud
table, the format is database.table
sink.enable-delete [string]โ
Whether to enable deletion. This option requires SelectDB Cloud table to enable batch delete function, and only supports Unique model.
ALTER TABLE example_db.my_table ENABLE FEATURE "BATCH_DELETE";
selectdb.config [map]โ
Write property configuration
CSV Write๏ผ
selectdb.config {
file.type="csv"
file.column_separator=","
file.line_delimiter="\n"
}
JSON Write:
selectdb.config {
file.type="json"
}
sink.buffer-size [string]โ
The maximum capacity of the cache, in bytes, that is flushed to the object storage. The default is 10MB. it is not recommended to modify it.
sink.buffer-count [string]โ
Maximum number of entries flushed to the object store. The default value is 10000. it is not recommended to modify.
sink.max-retries [string]โ
The maximum number of retries in the Commit phase, the default is 3.
Exampleโ
Use JSON format to import data
sink {
SelectDBCloud {
load-url="warehouse_ip:http_port"
jdbc-url="warehouse_ip:mysql_port"
cluster-name="Cluster"
table.identifier="test.test"
username="admin"
password="******"
selectdb.config {
file.type="json"
}
}
}
Use CSV format to import data
sink {
SelectDBCloud {
load-url="warehouse_ip:http_port"
jdbc-url="warehouse_ip:mysql_port"
cluster-name="Cluster"
table.identifier="test.test"
username="admin"
password="******"
selectdb.config {
file.type="csv"
file.column_separator=","
file.line_delimiter="\n"
}
}
}