Apache Iceberg
Apache Iceberg sink连接器
Iceberg 版本支持
- 1.10.2,支持写入表格式版本(format version)1、2 和 3 的表
引擎支持
Spark
Flink
SeaTunnel Zeta
描述
Apache Iceberg 目标连接器支持 CDC 写入、自动建表、表结构变更以及多表写入。
主要特性
支持的数据源信息
| 数据源 | 依赖项 | Maven依赖 |
|---|---|---|
| Iceberg | hive-exec | 下载 |
| Iceberg | libfb303 | 下载 |
数据库依赖
为了确保与不同版本的 Hadoop 和 Hive 兼容,项目 pom 文件中的 hive-exec 依赖范围被设置为 provided。因此,如果您使用 Flink 引擎,可能需要将以下 Jar 包添加到 <FLINK_HOME>/lib 目录中;如果您使用的是 Spark 引擎并且已经集成了 Hadoop,则无需添加以下 Jar 包。
hive-exec-xxx.jar
libfb303-xxx.jar
某些版本的 hive-exec 包中不包含 libfb303-xxx.jar,因此您还需要手动导入该 Jar 包。
数据类型映射
| SeaTunnel 数据类型 | Iceberg 数据类型 |
|---|---|
| BOOLEAN | BOOLEAN |
| TINYINT | INTEGER |
| SMALLINT | INTEGER |
| INT | INTEGER |
| BIGINT | LONG |
| FLOAT | FLOAT |
| DOUBLE | DOUBLE |
| DATE | DATE |
| TIME | TIME |
| TIMESTAMP | TIMESTAMP |
| TIMESTAMP_TZ | TIMESTAMPTZ |
| STRING | STRING |
| BYTES | FIXED BINARY |
| DECIMAL | DECIMAL |
| ROW | STRUCT |
| ARRAY | LIST |
| MAP | MAP |
当 Sink 以 format-version = 3 创建表时,纳秒精度(列精度 scale 为 9,例如 Iceberg Source 读取的 timestamp_ns 列)的 TIMESTAMP 和 TIMESTAMP_TZ 列会被创建为 TIMESTAMP_NS 和 TIMESTAMPTZ_NS。这两种类型只能表示 1677 年到 2262 年之间的值,超出范围的值会导致写入失败。其他精度、嵌套字段、通过 schema evolution 新增的列以及格式版本 1 和 2 的表仍使用微秒精度的时间戳。在 Spark 引擎上,行数据中的 TIMESTAMP 为微秒精度、TIMESTAMP_TZ 为毫秒精度,多余的精度在到达 Sink 之前就已被截断。
写入已存在的格式版本 3 表时,同样支持 TIMESTAMP_NS 和 TIMESTAMPTZ_NS 列。UNKNOWN 列只接受 null 值。暂不支持 VARIANT 列:写入的数据中不能包含该列,且仅适用于 Parquet 数据文件。暂不支持写入包含 GEOMETRY 或 GEOGRAPHY 列的表。
Sink 选项
| 名称 | 类型 | 是否必须 | 默认 | 描述 |
|---|---|---|---|---|
| catalog_name | string | 否 | default | 用户指定的目录名称,默认为default |
| namespace | string | 否 | default | backend catalog(元数据存储的后端目录)中 Iceberg 数据库的名称,默认为 default |
| table | string | 否 | - | backend catalog(元数据存储的后端目录)中 Iceberg 表的名称。不配置时使用上游表的表名 |
| iceberg.catalog.config | map | 是 | - | 用于指定初始化 Iceberg Catalog 的属性,这些属性可以参考此文件:CatalogProperties.java |
| hadoop.config | map | 否 | - | 传递给 Hadoop 配置的属性 |
| iceberg.hadoop-conf-path | string | 否 | - | 指定core-site.xml、hdfs-site.xml、hive-site.xml 文件的加载路径 |
| case_sensitive | boolean | 否 | false | 列名匹配时是否区分大小写 |
| iceberg.table.write-props | map | 否 | - | 传递给 Iceberg 写入器初始化的属性,这些属性具有最高优先级,例如 write.format.default、write.target-file-size-bytes 等设置。具体参数可以参考:TableProperties.java。 |
| iceberg.table.auto-create-props | map | 否 | - | Iceberg 自动建表时指定的配置 |
| iceberg.table.schema-evolution-enabled | boolean | 否 | false | 设置为 true 时,Iceberg 表可以在同步过程中支持 schema 变更 |
| iceberg.table.primary-keys | string | 否 | - | 用于标识表中一行数据的主键列列表,默认情况下以逗号分隔 |
| iceberg.table.partition-keys | string | 否 | - | 创建表时使用的分区字段列表,默认情况下以逗号分隔。多表场景可使用占位符 ${partition_keys} |
| iceberg.table.upsert-mode-enabled | boolean | 否 | false | 设置为 true 以启用 upsert 模式,默认值为 false |
| schema_save_mode | Enum | 否 | CREATE_SCHEMA_WHEN_NOT_EXIST | schema 变更方式, 请参考下面的 schema_save_mode |
| data_save_mode | Enum | 否 | APPEND_DATA | 数据写入方式, 请参考下面的 data_save_mode |
| custom_sql | string | 当 data_save_mode 为 CUSTOM_PROCESSING 时是 | - | CUSTOM_PROCESSING 数据写入方式使用的自定义 delete SQL,例如 delete from ... where ...。 |
| iceberg.table.commit-branch | string | 否 | - | 提交的默认分支 |
| multi_table_sink_replica | int | 否 | 1 | 多表写入模式下每张表对应的 Sink 写入并发数。一个作业写入多张 Iceberg 表,并且每张表都需要多个写入器时使用。 |
| flush_row_size | long | 否 | - | Sink 写入器处理这么多行后就轮换,不再一直保持到下一次 checkpoint,用来限制大批量 upsert 和 CDC 写入时的内存。必须大于 0。请参考下面的 flush_row_size。 |
| krb5_path | string | 否 | /etc/krb5.conf | krb5.conf 文件的路径,用于 Kerberos 认证。 |
| kerberos_principal | string | 否 | - | Kerberos 认证的 principal。 |
| kerberos_keytab_path | string | 否 | - | Kerberos 认证的 keytab 文件路径。 |
Sink 选项说明
iceberg.table.auto-create-props [map]
Sink 自动建表时使用的表属性。通过 format-version 选择 Iceberg 表格式版本,支持 1、2 和 3;未设置时 Iceberg 创建格式版本 2 的表。其他取值会在建表前使作业失败。该属性只作用于 Sink 新建的表:已存在的表保持原有格式版本,如需升级请使用 Iceberg 引擎执行,例如在 Spark 中执行 ALTER TABLE ... SET TBLPROPERTIES ('format-version' = '3')。
iceberg.table.auto-create-props = {
"format-version" = "3"
}
格式版本 3 的表支持追加写入。Upsert 和 CDC 写入会在写入任何数据前以错误 ICEBERG-04 拒绝,因为 Sink 写出的是 position delete 文件,而格式版本 3 要求使用 deletion vector。这包括配置了 iceberg.table.primary-keys 或 iceberg.table.upsert-mode-enabled、写入带有 identifier fields 的已有表,以及为带主键的源表创建格式版本 3 的表(源表主键会成为 identifier fields)。Upsert 和 CDC 场景请使用格式版本 2 的表。
iceberg.table.upsert-mode-enabled [boolean]
当该选项为 true 时,必须显式配置 iceberg.table.primary-keys。Sink 不会再自动继承 Source 表的主键。
iceberg.table.partition-keys [string]
使用英文逗号分隔多个分区字段,例如 dt,region,也可以使用 Iceberg transform,例如 days(ts)。多表写入场景可以使用 ${partition_keys} 占位符,从上游表元信息中取分区字段。
iceberg.table.commit-branch [string]
将提交写入指定的 Iceberg 分支。不配置时写入表的默认分支。
flush_row_size [long]
默认情况下,Sink 写入器会把当前的 task writer 一直保持到下一次 checkpoint。表有主键时(配置了 iceberg.table.primary-keys、开启 upsert 模式或上游是 CDC 源),task writer 会记住写过的每个主键,以便同一行后续的变更可以写成 position delete,这些主键要等 task writer 完成时才释放。因此 checkpoint 间隔很长的大批量导入,或者只在结束时做 checkpoint 的批作业,可能会内存溢出。
设置 flush_row_size 后,task writer 每处理这么多行就完成一次并新建一个。这样会提前关闭当前文件并释放记住的主键,写入器内存不再随每个 checkpoint 的行数增长。不设置时只在 checkpoint 时轮换,与之前的行为一致。
轮换之后写出的 equality delete 需要作用到轮换之前写入的行,而 Iceberg 只对更早快照中提交的文件应用 equality delete。所以当一个 checkpoint 中包含轮换后写出的 equality delete 时,它会按 task writer 完成的顺序拆成多个快照依次提交,而不是一个。读端可能逐个看到这些快照,每个快照都包含该 checkpoint 变更的一个前缀。部分提交的拆分 checkpoint 只支持在 Zeta 上续提交:从该 checkpoint 恢复的作业只会补提交缺失的快照。Flink 翻译层不会调用这条恢复路径,未拆分的 checkpoint 也受影响。Flink 1.13 和 1.15 会丢弃恢复出来的全局 committable,其中缺失的快照不会被补提交,checkpoint 可能停留在部分已发布的状态。Flink 1.20 会把恢复出来的提交交给普通提交路径,该路径不会跳过表中已有的快照,可能再次提交它们的文件。可能拆分 checkpoint 的 upsert 和 CDC 作业(设置了 flush_row_size,或开启 iceberg.table.schema-evolution-enabled 后发生 schema 变更)请在 Zeta 上运行。作业启动时不会检查这一点。纯追加写入仍然是每个 checkpoint 一个快照。同一个 checkpoint 的多个快照会在 checkpoint 完成前依次提交,所以取值过小也会让 checkpoint 变慢。
取值越小内存占用越低,但文件更多更小。分区表轮换时会完成所有已打开分区的文件,而不只是当前分区。upsert 模式下每一行都会写 equality delete,所以每次轮换都会单独成为一个快照:一个 checkpoint 大约提交(该 checkpoint 内单个写入器的行数 / flush_row_size)个快照。CDC 写入在每次轮换后出现更新或删除时拆分 checkpoint。因此取值过小会让表元数据膨胀、查询规划变慢;请取较大的值,例如几十万到几百万行,并定期清理过期快照。
sink {
Iceberg {
...
iceberg.table.primary-keys = "id"
iceberg.table.upsert-mode-enabled = true
flush_row_size = 1000000
}
}
custom_sql [string]
当 data_save_mode = CUSTOM_PROCESSING 时,配置在 Sink 写入前删除目标数据的 delete SQL。该模式下必须配置此选项。
对于 Iceberg timestamptz 列,比较以及 IN/NOT IN 条件支持带偏移量的时间戳字面量,例如 event_time >= '2026-09-12 10:00:00.123456+05:30'。偏移量会转换为对应的 UTC 时间点,精度为微秒。不带偏移量的字面量保持现有的 UTC 解释方式。不带时区的 Iceberg timestamp 列仍要求使用不带偏移量的字面量。
现有的文件级删除限制仍然适用:如果数据文件中只有部分行匹配条件,Iceberg 会拒绝删除。
krb5_path [string]
krb5.conf 文件的路径,用于 Kerberos 认证。
kerberos_principal [string]
Kerberos 认证的 principal。
kerberos_keytab_path [string]
Kerberos 认证的 keytab 文件路径。
任务示例
简单示例
env {
parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 5000
}
source {
MySQL-CDC {
plugin_output = "customers_mysql_cdc_iceberg"
server-id = 5652
username = "st_user"
password = "seatunnel"
table-names = ["mysql_cdc.mysql_cdc_e2e_source_table"]
url = "jdbc:mysql://mysql_cdc_e2e:3306/mysql_cdc"
}
}
transform {
}
sink {
Iceberg {
catalog_name="seatunnel_test"
iceberg.catalog.config={
"type"="hadoop"
"warehouse"="file:///tmp/seatunnel/iceberg/hadoop-sink/"
}
namespace="seatunnel_namespace"
table="iceberg_sink_table"
iceberg.table.write-props={
write.format.default="parquet"
write.target-file-size-bytes=536870912
}
iceberg.table.primary-keys="id"
iceberg.table.partition-keys="f_datetime"
iceberg.table.upsert-mode-enabled=true
iceberg.table.schema-evolution-enabled=true
case_sensitive=true
}
}
Hive Catalog
sink {
Iceberg {
catalog_name="seatunnel_test"
iceberg.catalog.config={
type = "hive"
uri = "thrift://localhost:9083"
warehouse = "hdfs://your_cluster//tmp/seatunnel/iceberg/"
}
namespace="seatunnel_namespace"
table="iceberg_sink_table"
iceberg.table.write-props={
write.format.default="parquet"
write.target-file-size-bytes=536870912
}
iceberg.table.primary-keys="id"
iceberg.table.partition-keys="f_datetime"
iceberg.table.upsert-mode-enabled=true
iceberg.table.schema-evolution-enabled=true
case_sensitive=true
}
}
Hadoop catalog
sink {
Iceberg {
catalog_name="seatunnel_test"
iceberg.catalog.config={
type = "hadoop"
warehouse = "hdfs://your_cluster/tmp/seatunnel/iceberg/"
}
namespace="seatunnel_namespace"
table="iceberg_sink_table"
iceberg.table.write-props={
write.format.default="parquet"
write.target-file-size-bytes=536870912
}
iceberg.table.primary-keys="id"
iceberg.table.partition-keys="f_datetime"
iceberg.table.upsert-mode-enabled=true
iceberg.table.schema-evolution-enabled=true
case_sensitive=true
}
}
提交到 Iceberg 分支
sink {
Iceberg {
catalog_name = "seatunnel_test"
iceberg.catalog.config = {
type = "hadoop"
warehouse = "file:///tmp/seatunnel/iceberg/hadoop-sink/"
}
namespace = "seatunnel_namespace"
table = "iceberg_sink_table"
iceberg.table.commit-branch = "audit_branch"
}
}
Glue Catalog
sink {
Iceberg {
catalog_name = "seatunnel_test"
iceberg.catalog.config = {
warehouse = "s3://your-bucket/warehouse/"
catalog-impl = "org.apache.iceberg.aws.glue.GlueCatalog"
io-impl = "org.apache.iceberg.aws.s3.S3FileIO"
client.region = "your-region"
}
namespace = "seatunnel_namespace"
table = "iceberg_sink_table"
iceberg.table.write-props = {
write.format.default = "parquet"
write.target-file-size-bytes = 536870912
}
iceberg.table.primary-keys = "id"
iceberg.table.partition-keys = "f_datetime"
iceberg.table.upsert-mode-enabled = true
iceberg.table.schema-evolution-enabled = true
case_sensitive = true
}
}
AWS S3 Tables REST Catalog
Amazon S3 表类数据存储服务提供针对分析工作负载进行优化的 S3 存储,其功能旨在持续提高查询性能并降低表的存储成本。S3 表类数据存储服务专为存储表数据而设计,例如每日购买交易、流传感器数据或广告展示次数。表数据以列和行表示数据,就像在数据库表中一样。
您可以将 Iceberg REST 客户端连接到 Amazon S3 表类数据存储服务 Iceberg REST 端点,然后进行 REST API 调用来创建、更新或查询 S3 表存储桶中的表。该端点实现了 Apache Iceberg REST Catalog Open API specification 中指定的一组标准化 Iceberg REST API。该端点的工作原理是将 Iceberg REST API 操作转换为相应的 S3 表类数据存储服务操作。
S3 表类数据存储服务中的数据存储在新的存储桶类型中:表存储桶,它将表存储为子资源。表存储桶支持以 Apache Iceberg 格式存储表。使用标准 SQL 语句,您可以通过支持 Iceberg 的查询引擎来查询表,例如 Amazon Athena、Amazon Redshift 和 Apache Spark。
sink {
Iceberg {
catalog_name = "s3_tables_catalog"
namespace = "s3_tables_catalog"
table = "user_data"
iceberg.catalog.config = {
type: "rest"
warehouse: "arn:aws:s3tables:<Region>:<accountID>:bucket/<bucketname>"
uri: "https://s3tables.<Region>.amazonaws.com/iceberg"
rest.sigv4-enabled: "true"
rest.signing-name: "s3tables"
rest.signing-region: "<Region>"
}
}
}
Kerberos 认证
以下示例演示了在使用 Hadoop Catalog 和 HDFS 时如何配置 Iceberg Sink 的 Kerberos 认证:
sink {
Iceberg {
catalog_name = "seatunnel_test"
iceberg.catalog.config = {
type = "hadoop"
warehouse = "hdfs://your_cluster/tmp/seatunnel/iceberg/"
}
namespace = "seatunnel_namespace"
table = "iceberg_sink_table"
iceberg.table.write-props = {
write.format.default = "parquet"
write.target-file-size-bytes = 536870912
}
krb5_path = "/etc/krb5.conf"
kerberos_principal = "hive/your_host@EXAMPLE.COM"
kerberos_keytab_path = "/path/to/your.keytab"
iceberg.table.primary-keys = "id"
iceberg.table.partition-keys = "f_datetime"
iceberg.table.upsert-mode-enabled = true
iceberg.table.schema-evolution-enabled = true
case_sensitive = true
}
}
说明:
krb5_path:用于 Kerberos 认证的krb5.conf文件路径。kerberos_principal:Kerberos 认证的 principal,格式为primary/instance@REALM。kerberos_keytab_path:Kerberos 认证的 keytab 文件路径。
Multiple table(多表写入)
示例1
env {
parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 5000
}
source {
Mysql-CDC {
url = "jdbc:mysql://127.0.0.1:3306/seatunnel"
username = "root"
password = "******"
table-names = ["seatunnel.role","seatunnel.user","galileo.Bucket"]
}
}
transform {
}
sink {
Iceberg {
...
namespace = "${database_name}_test"
table = "${table_name}_test"
}
}
示例2
env {
parallelism = 1
job.mode = "BATCH"
}
source {
Jdbc {
driver = oracle.jdbc.driver.OracleDriver
url = "jdbc:oracle:thin:@localhost:1521/XE"
user = testUser
password = testPassword
table_list = [
{
table_path = "TESTSCHEMA.TABLE_1"
},
{
table_path = "TESTSCHEMA.TABLE_2"
}
]
}
}
transform {
}
sink {
Iceberg {
...
namespace = "${schema_name}_test"
table = "${table_name}_test"
}
}
变更日志
Change Log
| Change | Commit | Version |
|---|---|---|
| [Chore] fix typos filed -> field (#9757) | https://github.com/apache/seatunnel/commit/e3e1c67d29 | 2.3.12 |
| [Improve][Core] Unify the aws-sdk-v2 version to 2.31.30 (#9698) | https://github.com/apache/seatunnel/commit/41c251cc8a | 2.3.12 |
| [Improve][API] Optimize the enumerator API semantics and reduce lock calls at the connector level (#9671) | https://github.com/apache/seatunnel/commit/9212a77140 | 2.3.12 |
| [Bug][Connector-V2] Fix the issue of writing the ORC format Iceberg report "Illegal provider-class name" (#6754) (#9588) | https://github.com/apache/seatunnel/commit/74b193dd5a | 2.3.12 |
| [Bug][Connector-V2] Updates Iceberg version to 1.6.1 (#9387) (#9451) | https://github.com/apache/seatunnel/commit/7b92a6c5c1 | 2.3.12 |
| [Fix][Connector-Iceberg] Fix Time Zone Issue for Iceberg Timestamp Type (#9460) | https://github.com/apache/seatunnel/commit/60cd497610 | 2.3.12 |
| [Feature][Connector-V2] Iceberg add glue catalog support (#9247) | https://github.com/apache/seatunnel/commit/ecff2e8618 | 2.3.11 |
[Improve] Remove useless iceberg sink config iceberg.table.config (#9307) | https://github.com/apache/seatunnel/commit/fbdf39ebf2 | 2.3.11 |
| [Improve][connector-iceberg] fix schema change event (#9217) | https://github.com/apache/seatunnel/commit/56669095b7 | 2.3.11 |
| [Feature][Transform] Support define sink column type (#9114) | https://github.com/apache/seatunnel/commit/ab7119e507 | 2.3.11 |
| [Feat][Connector-v2][Iceberg]support filter conditions in iceberg source (#9095) | https://github.com/apache/seatunnel/commit/0eb72780ee | 2.3.11 |
| [Feature][Checkpoint] Add check script for source/sink state class serialVersionUID missing (#9118) | https://github.com/apache/seatunnel/commit/4f5adeb1c7 | 2.3.11 |
[Fix][API] Fixed not invoke the SinkAggregatedCommitter's init method (#9070) | https://github.com/apache/seatunnel/commit/df0d11d632 | 2.3.11 |
| [Improve] iceberg options (#8967) | https://github.com/apache/seatunnel/commit/82a374ec87 | 2.3.10 |
| [Improve] restruct connector common options (#8634) | https://github.com/apache/seatunnel/commit/f3499a6eeb | 2.3.10 |
| [Feature][Iceberg] Support read multi-table (#8524) | https://github.com/apache/seatunnel/commit/2bfb97e502 | 2.3.10 |
| [Improve][Iceberg] Filter catalog table primaryKey is empty (#8413) | https://github.com/apache/seatunnel/commit/857aab5e83 | 2.3.9 |
| [Improve][Connector-V2] Reduce the create times of iceberg sink writer (#8155) | https://github.com/apache/seatunnel/commit/45a7a715a2 | 2.3.9 |
| [Improve][dist]add shade check rule (#8136) | https://github.com/apache/seatunnel/commit/51ef800016 | 2.3.9 |
| [Feature][Iceberg] Support custom delete sql for sink savemode (#8094) | https://github.com/apache/seatunnel/commit/29ca928c36 | 2.3.9 |
| [Improve][Connector-V2] Reduce the request times of iceberg load table (#8149) | https://github.com/apache/seatunnel/commit/555f5eb404 | 2.3.9 |
| [Feature][Core] Support cdc task ddl restore for zeta (#7463) | https://github.com/apache/seatunnel/commit/8e322281ed | 2.3.9 |
| [Improve][Iceberg] Support table comment for catalog (#7936) | https://github.com/apache/seatunnel/commit/72ab38f317 | 2.3.9 |
| [Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786) | https://github.com/apache/seatunnel/commit/6b7c53d03c | 2.3.9 |
| [Fix][Connector-V2] Fix iceberg throw java: package sun.security.krb5 does not exist when use jdk 11 (#7734) | https://github.com/apache/seatunnel/commit/116af4febc | 2.3.8 |
| [Hotfix][Connector-V2] Release resources when task is closed for iceberg sinkwriter (#7729) | https://github.com/apache/seatunnel/commit/ff281183bd | 2.3.8 |
| [Fix][Connector-V2] Fixed iceberg sink can not handle uppercase fields (#7660) | https://github.com/apache/seatunnel/commit/b7be0cb4a1 | 2.3.8 |
| [Hotfix][CDC] Fix ddl duplicate execution error when config multi_table_sink_replica (#7634) | https://github.com/apache/seatunnel/commit/23ab3edbbb | 2.3.8 |
| [Improve][Iceberg] Add savemode create table primaryKey testcase (#7641) | https://github.com/apache/seatunnel/commit/6b36f90f4d | 2.3.8 |
| [Hotfix] Fix iceberg missing column comment when savemode create table (#7608) | https://github.com/apache/seatunnel/commit/b35bd94bfb | 2.3.8 |
| [Improve][Connector-V2] Remove hard code iceberg table format version (#7500) | https://github.com/apache/seatunnel/commit/f49b263e65 | 2.3.8 |
| [Improve][API] Move catalog open to SaveModeHandler (#7439) | https://github.com/apache/seatunnel/commit/8c2c5c79a1 | 2.3.8 |
| [Feature][Connector-V2][Iceberg] Support Iceberg Kerberos (#7246) | https://github.com/apache/seatunnel/commit/e3001207c8 | 2.3.8 |
| [Improve][Connector] Add multi-table sink option check (#7360) | https://github.com/apache/seatunnel/commit/2489f6446b | 2.3.7 |
| [Feature][Core] Support using upstream table placeholders in sink options and auto replacement (#7131) | https://github.com/apache/seatunnel/commit/c4ca74122c | 2.3.6 |
| [Bug][Connector-Iceberg]fix create iceberg v2 table with pks (#6895) | https://github.com/apache/seatunnel/commit/40d2c1b213 | 2.3.6 |
| [Feature][Connector-V2] Iceberg-sink supports writing data to branches (#6697) | https://github.com/apache/seatunnel/commit/e3103535cc | 2.3.6 |
| [Fix][Connector-V2] Fix connector support SPI but without no args constructor (#6551) | https://github.com/apache/seatunnel/commit/5f3c9c36a5 | 2.3.5 |
| [Improve] Add SaveMode log of process detail (#6375) | https://github.com/apache/seatunnel/commit/b0d70ce224 | 2.3.5 |
| [Improve][Zeta] Add classloader cache mode to fix metaspace leak (#6355) | https://github.com/apache/seatunnel/commit/9c3c2f183d | 2.3.5 |
| [Improve][API] Unify type system api(data & type) (#5872) | https://github.com/apache/seatunnel/commit/b38c7edcc9 | 2.3.5 |
| [Feature] Supports iceberg sink #6198 (#6265) | https://github.com/apache/seatunnel/commit/18d3e86194 | 2.3.5 |
| [Test][E2E] Add thread leak check for connector (#5773) | https://github.com/apache/seatunnel/commit/1f2f3fc5f0 | 2.3.4 |
| [Improve][Common] Introduce new error define rule (#5793) | https://github.com/apache/seatunnel/commit/9d1b2582b2 | 2.3.4 |
| [BUG][Connector-V2] Iceberg source lost data with parallelism option (#5732) | https://github.com/apache/seatunnel/commit/7f3b4be075 | 2.3.4 |
| [Dependency]Bump org.apache.avro:avro in /seatunnel-connectors-v2/connector-iceberg (#5582) | https://github.com/apache/seatunnel/commit/13753a927b | 2.3.4 |
| [Improve][Pom] Add junit4 to the root pom (#5611) | https://github.com/apache/seatunnel/commit/7b4f7db2a2 | 2.3.4 |
| Support config column/primaryKey/constraintKey in schema (#5564) | https://github.com/apache/seatunnel/commit/eac76b4e50 | 2.3.4 |
| [Doc][Iceberg] Improved iceberg documentation (#5335) | https://github.com/apache/seatunnel/commit/659a68a0be | 2.3.4 |
| [Hotfix] Fix com.google.common.base.Preconditions to seatunnel shade one (#5284) | https://github.com/apache/seatunnel/commit/ed5eadcf73 | 2.3.3 |
| [Hotfix][Connector][Iceberg] Fix iceberg source stream mode init error (#4638) | https://github.com/apache/seatunnel/commit/64760eed4d | 2.3.2 |
| Merge branch 'dev' into merge/cdc | https://github.com/apache/seatunnel/commit/4324ee1912 | 2.3.1 |
| [Improve][Project] Code format with spotless plugin. | https://github.com/apache/seatunnel/commit/423b583038 | 2.3.1 |
| [Improve][SourceConnector] Unifie Iceberg source fields to schema (#3959) | https://github.com/apache/seatunnel/commit/20e1255fab | 2.3.1 |
| [improve][api] Refactoring schema parse (#4157) | https://github.com/apache/seatunnel/commit/b2f573a13e | 2.3.1 |
| [Improve][build] Give the maven module a human readable name (#4114) | https://github.com/apache/seatunnel/commit/d7cd601051 | 2.3.1 |
| [Improve][Project] Code format with spotless plugin. (#4101) | https://github.com/apache/seatunnel/commit/a2ab166561 | 2.3.1 |
| [Improve][Connector-V2][Iceberg] Unified exception for iceberg source connector (#3677) | https://github.com/apache/seatunnel/commit/e24843515f | 2.3.1 |
| [Feature][Connector] add get source method to all source connector (#3846) | https://github.com/apache/seatunnel/commit/417178fb84 | 2.3.1 |
| [Feature][API & Connector & Doc] add parallelism and column projection interface (#3829) | https://github.com/apache/seatunnel/commit/b9164b8ba1 | 2.3.1 |
| [Hotfix][OptionRule] Fix option rule about all connectors (#3592) | https://github.com/apache/seatunnel/commit/226dc6a119 | 2.3.0 |
| [Feature][Connector-V2][Iceberg] Modify the scope of flink-shaded-hadoop-2 to provided to be compatible with hadoop3.x (#3046) | https://github.com/apache/seatunnel/commit/b38c50789f | 2.3.0 |
| [Feature][Connector V2] expose configurable options in Iceberg (#3394) | https://github.com/apache/seatunnel/commit/bd9a313ded | 2.3.0 |
| [Improve][Connector][Iceberg] Improve code. (#3065) | https://github.com/apache/seatunnel/commit/9f38e3da74 | 2.3.0-beta |
| [Code-Improve][Iceberg] Use automatic resource management to replace 'try - finally' code block. (#2909) | https://github.com/apache/seatunnel/commit/b7f640724b | 2.3.0-beta |
| [Feature][Connector-V2] Add iceberg source connector (#2615) | https://github.com/apache/seatunnel/commit/ffc6088a79 | 2.2.0-beta |