Iceberg

Sink plugin: Iceberg [Spark]

Description

Write data to Iceberg.

Options

name	type	required	default value
path	string	yes	-
saveMode	string	no	append
target-file-size-bytes	long	no	-
check-nullability	bool	no	-
snapshot-property.custom-key	string	no	-
fanout-enabled	bool	no	-
check-ordering	bool	no	-

Refer to iceberg write options for more configurations.

path

Iceberg table location.

saveMode

append or overwrite. Only these two modes are supported by iceberg. The default value is append.

target-file-size-bytes

Overrides this table’s write.target-file-size-bytes

check-nullability

Sets the nullable check on fields

snapshot-property.custom-key

Adds an entry with custom-key and corresponding value in the snapshot summary eg: snapshot-property.aaaa="bbbb"

fanout-enabled

Overrides this table’s write.spark.fanout.enabled

check-ordering

Checks if input schema and table schema are same

Example

iceberg {
    path = "hdfs://localhost:9000/iceberg/warehouse/db/table"
  }