Incompatible Changes
This document records the incompatible updates between each version. You need to check this document before you upgrade to related version.
dev
Http sink: array_mode = true sends each row as a JSON object
- Behavior change: the elements of the request body are JSON objects instead of strings
- Description: With
array_mode = true, the Http sink added each serialized row to the request array as a string, so the body was["{\"id\":1}","{\"id\":2}"]instead of the documented JSON array of rows. It now sends[{"id":1},{"id":2}]. Every version since 2.3.11 sent the array of strings.array_mode = falseis not affected. - Impact: An endpoint that rejected the old body with an error now accepts it. A receiver written for the old body, which decodes each element as a JSON string, now receives objects and has to be changed.
- Migration Guide: No configuration change is needed. If a receiver decodes each array element from a string, change it to use the element directly.
- Description: With
Lance sink: BIGINT columns are created as int64
- Behavior change: a
BIGINTcolumn in a dataset the Lance sink creates is Arrowint64- Description: When the Lance sink created a dataset, it declared
BIGINTcolumns as Arrowint32and wrote each value withNumber.intValue(), so a value outside the signed 32-bit range was silently wrapped. For example,3000000000was stored as-1294967296, and the epoch millisecond1739586083123as124328243. The sink, and the catalog'screateTable, now declareBIGINTasint64and store the value unchanged. - Impact: A dataset created by this version has an
int64column where an earlier version createdint32. A dataset that already exists keeps its column types, because the sink writes to the schema it finds, so itsint32column still wraps large values. - Migration Guide: No configuration change is needed. If a consumer expects
int32for these columns, update it to readint64. To store large values in a dataset created by an earlier version, recreate the dataset and write the rows again from the source.
- Description: When the Lance sink created a dataset, it declared
Databend source: a DECIMAL column is read as DECIMAL instead of STRING
- Behavior change: a Databend
DECIMAL(p, s)column reaches the job asDECIMAL(p, s)- Description: The Databend source takes every column type from
DatabendCatalog. The catalog compared the type name the driver reports, such asdecimal(10, 2), with the bare nameDECIMAL, so every decimal column fell back toSTRINGand its values were read as text. It now maps the column toDECIMALwith the column's own precision and scale. On a worker whose default locale is Turkish or Azerbaijani,TINYINT,SMALLINT,INT,BIGINT,TIMESTAMPandBINARYcolumns also fell back toSTRING; they now get their mapped types there too. - Impact: A job that reads a Databend
DECIMALcolumn now emits decimal values. A sink table that SeaTunnel creates throughschema_save_modegets a decimal column instead of a string column, and a transform or sink that expected a string now receives a decimal. - Migration Guide: No configuration change is needed. If a downstream step needs text, convert the column explicitly, for example
CAST(price AS STRING)in the SQL transform. A sink table that an earlier run already created with a string column keeps that type.
- Description: The Databend source takes every column type from
JDBC source: a nullable key column is no longer chosen as the split key
- Behavior change: the automatic split key selection skips nullable columns
- Description: Without
partition_column, the JDBC source takes the split key from the primary key and then the unique indexes. Range and hash split conditions never match NULL (CockroachDB hash splits excepted), and a unique index can hold NULL values, so when a nullable key column was chosen and the table was split by it, the rows with NULL in it were read by no split and the job finished without an error. The automatic selection now skips nullable primary key and unique key columns, as the CDC sources do, and logs a WARN naming each skipped column. - Impact: A table that was split on a nullable primary key or unique key column is now split on the next NOT NULL key column, or read as one split if it has none, so all its rows are read, with less parallelism if it falls back to one split. Tables split on a NOT NULL primary key or unique key column,
querysources withoutpartition_column, and jobs that setpartition_columnare not affected. A job restored from a checkpoint keeps the splits it had already planned; the change applies to the tables it splits after the restore. - Migration Guide: No configuration change is needed. To read such a table in parallel again, declare the key column
NOT NULL, or setpartition_columnto a column without NULL values. If a downstream table was loaded by an affected job, reload it to get the rows that were skipped.
- Description: Without
JDBC Sink: fail safely after losing uncommitted work
- Affected jobs: non-XA JDBC sinks with checkpointing enabled and a manual-commit connection
(
auto_commit = false, or Oracle, which always uses manual commit). - Behavior change: if flushed but uncommitted work is lost through connection replacement or
transaction rollback, the sink refuses the commit so the job can recover from the last successful
checkpoint. A failed flush is not retried while earlier flushed work is pending. In multi-table
row-error handling, a rollback that discards another table's work also prevents further commits.
A commit exception also prevents further commits by the failed writer and, when flushed work is
pending, by all writers sharing its transaction. Commit exceptions remain system errors even with
SQLState class
22or23; retrying only the commit cannot replay a rolled-back transaction. Previously, these paths could complete a checkpoint with records missing. - Migration: no configuration or checkpoint-format migration is required. Affected jobs may restart instead of retrying a batch locally or continuing after a row error. Auto-commit connections, jobs without checkpointing, and XA writers retain their existing behavior.
GraphQL connector: a response with GraphQL errors fails the job
- Behavior change: the GraphQL sink and the HTTP query mode of the GraphQL source fail when the server reports errors
- Description: GraphQL servers usually answer a rejected query or mutation with HTTP 200 and an
errorsarray. The sink only checked the status code, and for a status other than 200 it only logged the response, so a rejected mutation was dropped and the job finished. The source read such a response as if it were data, so a rejected query produced no rows. Both now fail with the server's errors in the message, and the sink also fails on a status other than 200. Date and time fields are now sent as ISO 8601 strings; before, they were sent as nested JSON objects on Java 11 and the write failed on Java 17. Whenvariablesis not set, an empty object is now sent instead of the string"{}", which servers reject; before, a sink withoutvariablesfailed on every row. - Impact: A job whose requests the server rejects now fails instead of finishing with missing rows. This includes a source response with both
dataand a non-emptyerrorsarray, which was read as rows withnullin the fields that failed: it now fails the job, and there is no option to keep the old behavior. In the streaming HTTP query mode, one poll answered with errors now fails the task, where before that poll produced no rows and polling went on. Subscription mode is unchanged. - Migration Guide: No configuration change is needed. If a job starts failing, the error message shows what the server rejected. Fix the query, the mutation, the variable types or the permissions on the server; for example, Hasura expects
float8for a PostgreSQLdouble precisioncolumn, notFloat.
- Description: GraphQL servers usually answer a rejected query or mutation with HTTP 200 and an
Lance sink: TIMESTAMP values keep their microseconds
- Behavior change: a
TIMESTAMPwritten by the Lance sink keeps the digits below the millisecond- Description: The Lance sink declares a
TIMESTAMPcolumn with microsecond precision, as documented, butBaseTypeWriter.convertToEpochMicroconvertedLocalDateTimeandjava.sql.Timestampvalues through milliseconds, so every stored value ended in000microseconds. Values are now converted from the instant directly. - Impact: For the same input, a newly written value can differ from one written by an earlier version in its last three microsecond digits. Rows already in a dataset are not rewritten, so one dataset can hold both. Millisecond columns, and values with no digits below the millisecond, are unchanged.
- Migration Guide: No configuration change is needed. If a consumer compares or deduplicates new rows against rows written by an earlier version, compare at millisecond precision, or rewrite the older rows from the source.
- Description: The Lance sink declares a
JDBC MySQL: DATETIME writes preserve local date/time fields
The MySQL JDBC sink now binds SeaTunnel TIMESTAMP (LocalDateTime) directly instead of converting it through java.sql.Timestamp. With different worker JVM and JDBC connection time zones, affected versions could shift DATETIME values or reject valid boundary values. Writes now preserve the local fields and the fractional seconds supported by the target column. The DATETIME source mapping and TIMESTAMP_TZ handling are unchanged.
Remove any manual offset used to compensate for this defect. Existing stored values are not changed automatically; verify them against the original source before correcting or reloading them. No new configuration is required.
Doris and StarRocks sources: a null MAP or ROW column no longer reads as an empty map or fails the job
- Behavior change: a null complex cell from an Arrow stream now reads as
null- Description:
ArrowToSeatunnelRowReader, which the Doris and StarRocks sources read with, sends a MAP column toMapConverterand a ROW column toStructConverter. Neither checked the Arrow validity bit in the path that converts element types, so a null MAP cell was returned as an empty map and a null ROW cell ended the read withNullPointerException: Cannot invoke "java.util.Map.entrySet()" because "valueMap" is null. Both now returnnull, the same asListConverter,LargeListConverterandFixedSizeListConverter, which have guarded this since the Arrow reader was added. - Impact: A nullable MAP column now yields
nullfor a row that holds no map, where it previously yielded an empty map. A row that holds a genuinely empty map still yields an empty map, so absent and empty are no longer the same value. A nullable ROW column is now read instead of failing the whole split. - Migration Guide: Consumers that treated the previous empty map as "no value" need no change. A consumer that assumed every row carries a non-null map must handle
nullfor that column.
- Description:
ORC sink: a decimal that does not fit the target column now fails instead of being written as NULL
- Behavior change: an out-of-range
DECIMALfails the write rather than silently becomingNULL- Affected component:
seatunnel-connectors-v2,connector-file-base,OrcWriteStrategy - Description: The ORC sink converted a
BigDecimalwithdecimalColumnVector.set(row, HiveDecimal.create(value)). Both halves of that returnnullrather than reporting an error:HiveDecimal.createreturnsnullabove ORC's maximum precision of 38, andDecimalColumnVector.setinternally callsenforcePrecisionScale, which returnsnullwhen the value does not fit the precision and scale the column actually declares. Neither result was checked, so the row was written with the value replaced byNULL, with no exception, no warning and no log line. The conversion now reports a failure naming the value and the target column. - Impact: A job that silently dropped values now fails at the offending row. The reachable case is a value merely wider than its declared column, which arises from a narrow or inferred JDBC precision, a sampled schema, a schema-less source, or a SQL transform that multiplies decimals. In-range values are unaffected and are written exactly as before.
- Consistency with the Parquet sink: the Parquet sink in the same module already refused the
same overflow loudly, with
AvroTypeException: Cannot encode decimal with precision 39 as max precision 38. The two sinks no longer disagree about whether losing a value is acceptable. - Unchanged: a declared precision above 38 was already rejected when the schema was built, so this entry is about VALUES rather than DECLARATIONS. No option, schema or file layout changes.
- Still rounded, not failed:
enforcePrecisionScalerounds fractional digits beyond the declared scale rather than rejecting them, so1.239written into adecimal(10,2)column is still stored as1.24(HALF_UP), exactly as before this change. Only an integer part that does not fit the declared precision, or a value above ORC's maximum precision of 38, now fails. - Migration Guide: Widen the declared precision or scale so the value fits, or round the
value before it reaches the sink. To find affected rows before upgrading, compare the written
ORC data against the source for unexpected
NULLdecimals.
- Affected component:
Doris sink: TIMESTAMP_TZ converts to the Doris session timezone
- Behavior change: Doris sink stores
TIMESTAMP_TZas timezone-adjusted wall-clockDATETIME- Affected component:
seatunnel-connectors-v2/connector-doris - Description:
TIMESTAMP_TZvalues are converted to the effective Doris target timezone before being stored asDATETIME. Setsink.datetime-timezoneto choose that target timezone explicitly; if it is unset, SeaTunnel uses the JVM default timezone. For example, withsink.datetime-timezone = "Asia/Shanghai",2024-01-01T03:00:00+09:00is stored as2024-01-01 02:00:00. - Impact: Before this change, the Doris sink followed the v3.0.0 downgrade contract and dropped the timezone offset while keeping the wall-clock digits unchanged. Stored
DATETIMEvalues for the sameTIMESTAMP_TZinput can differ when the JVM default zone differs from the Doris session zone, or whensink.datetime-timezoneis set explicitly. When the option is unset and the table hasTIMESTAMP_TZcolumns, the writer logs a warning. - Migration Guide: Set
sink.datetime-timezoneto the Doris session zone. Verify existing stored data if the JVM default zone and Doris session zone differed. (#11524)
- Affected component:
Socket connector: connections now have a 10-second connect timeout by default
- Behavior change:
connect_timeout_msdefaults to 10000 instead of leaving the wait to the operating system- Description: The Socket source and sink used to establish a TCP connection with no application-level timeout (
connect(address, 0)for the source,new Socket(host, port)for the sink). A peer that was unreachable, stopped responding, or silently dropped the SYN blocked the connector thread until the OS TCP timeout expired. Both now default to a 10-second connect bound. - Impact: Jobs whose target took longer than 10 seconds to accept a connection now fail at startup or on reconnect where they previously succeeded after a longer OS-managed wait. This applies to existing configurations that do not set the option. The source's blocking read behaviour is unchanged:
read_timeout_msdefaults to0, so an idle stream is not treated as a failure. - Migration Guide: Set
connect_timeout_ms = 0on the source or sink to restore the previous OS-governed behaviour, or raise the value above 10000 for a legitimately slow peer.read_timeout_msis source-only; it has no effect on the sink, which never reads from its socket.
- Description: The Socket source and sink used to establish a TCP connection with no application-level timeout (
Zeta: runtime metric counters restart from zero after a pipeline failover restore
- Behavior change: runtime counters no longer accumulate values from a failed pipeline attempt
- Description: When the Zeta engine restores a pipeline after a worker failover (or retries it after a failure), the new attempt's runtime counters (
SourceReceivedCount,SinkWriteCount, queue metrics) now start from zero. Previously the replacement attempt continued from the failed attempt's values, because task locations are stable across retries and the new attempt's counters were seeded from the shared metrics store, so finished-job metrics could include work already done and canceled by earlier attempts. - Impact: For batch jobs this removes over-counted finished-job metrics. For streaming jobs resumed after a failover, reported totals now restart at the new attempt's values instead of continuing the pre-failure totals; external dashboards or alerts that aggregate these counters must not read the reset as data loss. The reset is best effort: a worker report already in flight during the restore can still carry a stale value for at most a millisecond-scale window.
- Migration Guide: No action is required for correctness. If you need lifetime totals that survive failovers, accumulate them outside the running job, for example from the stored metrics of finished jobs in the job history.
- Description: When the Zeta engine restores a pipeline after a worker failover (or retries it after a failure), the new attempt's runtime counters (
JSON format: ARRAY and MAP columns reject values of the wrong container type
When a field is declared as ARRAY, a JSON value that is not an array (a string, a number, a boolean or an object) previously produced an empty array [], or an array holding null for every member of the object, without any warning. When a field is declared as MAP, a JSON value that is not an object silently produced an empty map {}. Both happened with ignore-parse-errors disabled as well as enabled.
Such a value now causes a parse error that identifies the field and the expected container type; with ignore-parse-errors = true only that container value is set to null, and every other value of the row is kept. The same holds for a wrong container type nested inside an array or a map: only the offending nested value becomes null while its siblings and the enclosing container are retained, and empty arrays and objects of the declared container type, null and missing values, and the conversion of elements and keys are unchanged. To migrate, provide an actual JSON array for an ARRAY field and an actual JSON object for a MAP field, or set ignore-parse-errors = true so that malformed container values become null instead of failing the job.
The same strictness applies wherever the shared JSON-to-row conversion is used without ignoring parse errors: the JsonPath transform (column dest_type), the Python transform (declared output columns and their declared types) and the Assert sink's equals_to check for ARRAY and MAP values. These callers do not consult the JSON format's ignore-parse-errors option, so that switch does not apply to them: a value that does not match the declared ARRAY or MAP type still fails there, while the existing plugin-specific error-handling behavior of those transforms is unchanged.
JDBC Dameng: NVARCHAR and NVARCHAR2 report their declared length
- Schema change: the Dameng catalog reports
NVARCHAR(n)andNVARCHAR2(n)with length n instead of the byte length- Description:
NVARCHAR(n)andNVARCHAR2(n)hold n characters, but the catalog took the length fromALL_TAB_COLUMNS.DATA_LENGTH, which counts bytes: 2n on a GB18030 database and 4n on a UTF-8 database, whateverLENGTH_IN_CHARis set to.NVARCHAR(10)was therefore reported asNVARCHAR(20)orNVARCHAR(40). The length now comes fromSYS.SYSCOLUMNS.LENGTH$, the declared length. - Impact: The source type of these columns changes, for example from
NVARCHAR(20)toNVARCHAR(10), and the column length used for sink auto-create is halved (GB18030) or quartered (UTF-8), for example a MySQL sink getsVARCHAR(40)instead ofVARCHAR(80)forNVARCHAR(10). Each new column still holds every value of the source column, because the length is counted in characters. Tables that already exist downstream are not altered.CHAR,VARCHAR,VARCHAR2andNCHARcolumns are not affected. - Migration Guide: No change is needed. Jobs that rely on the wider auto-created columns, for example to write longer values from another source into the same table, should create the sink table themselves.
- Description:
Iceberg sink: a checkpoint with equality deletes after a writer rotation is committed as several snapshots
- Behavior change: the Iceberg sink can commit one checkpoint as several ordered snapshots
- Affected component:
seatunnel-connectors-v2/connector-iceberg - Description: The sink committed all files of a checkpoint in one snapshot. When the task writer was completed in the middle of a checkpoint, which happens on a schema change with
iceberg.table.schema-evolution-enabledand now also with the newflush_row_sizeoption, an equality delete written after that point landed in the same snapshot as the row it targets. Iceberg does not apply an equality delete to data files with the same sequence number, so an upsert, update or delete of a row written earlier in the same checkpoint left the old row in the table. Such a checkpoint is now committed as several snapshots, in the order the task writers were completed. Each snapshot carriesseatunnel.checkpoint-id, plusseatunnel.commit-groupandseatunnel.commit-group-count. - Impact: Only upsert and CDC writes that rotate the task writer inside a checkpoint are affected; append-only writes and checkpoints without a rotation still produce one snapshot. Readers can see the snapshots of such a checkpoint one by one, and the snapshot count per checkpoint grows with the number of rotations.
- Migration Guide: No configuration change is needed. Jobs restored from a checkpoint committed by an older version are handled as before: a snapshot with the checkpoint id and without
seatunnel.commit-groupis treated as the whole checkpoint. If a job stops while a checkpoint that was split into several snapshots is being committed, restore it on Zeta with this version or a later one. An older version skips the whole checkpoint once it finds any of its snapshots, and the Flink translation layers do not call the restore path of the sink, so in either case the remaining snapshots of that checkpoint are not resumed correctly; seeflush_row_sizein the Iceberg sink documentation.
- Affected component:
File sink: XML applies date_format, datetime_format and time_format
- Behavior change: the XML writer of the file sinks formats DATE, TIME and TIMESTAMP values with the configured formats
- Description: With
file_format_type = "xml", DATE, TIME and TIMESTAMP values were written with JavatoString(), ignoringdate_format,datetime_formatandtime_format. Timestamps came out as2026-01-02T10:13:45.123, and as2026-01-02T10:13when the seconds were zero, so a file could mix several layouts and could not be read back with a singledatetime_format. They are now formatted with the configured options, like the CSV, Text and Excel writers. - Impact: With the defaults, timestamps are written as
yyyy-MM-dd HH:mm:ss(a space instead ofT, fractional seconds dropped) and times asHH:mm:ss. Readers that expect the previous ISO layout must be adjusted. - Migration Guide: To keep an ISO layout, set
datetime_formaton the sink, for exampleYYYY_MM_DD_HH_MM_SS_ISO8601(yyyy-MM-dd'T'HH:mm:ss) orYYYY_MM_DD_HH_MM_SS_SSS_ISO8601(yyyy-MM-dd'T'HH:mm:ss.SSS) to keep milliseconds, and read the files with the matching sourcedatetime_format.
- Description: With
Native CLI help exit code
seatunnel.sh -h now exits with 0 instead of 234.
Update scripts that recognize native help by exit code 234. The Spark/Flink Java launcher
protocol still uses 234; its wrapper scripts continue to translate help to exit 0.
CDC: nullable columns are no longer used as the snapshot split column
- Behavior change: JDBC-based CDC sources (MySQL, PostgreSQL, Oracle, SQL Server, DB2) no longer split the snapshot on a nullable column
- Description: For a table without a primary key, the snapshot split column was taken from a unique key even when that column was nullable. Rows with NULL in it matched no snapshot chunk and were silently skipped. A nullable column is now skipped, whether it comes from a unique key, from
snapshotSplitColumnor from a primary key configured intable-names-config.primaryKeys; if no non-nullable key column remains, the table is read as a single split. Nullability is read from the database metadata. - Impact: Such tables are now read completely, but with one snapshot split instead of several. With
exactly_once = true, a table whose only key is a nullable unique key, or a nullable column configured intable-names-config.primaryKeys, now fails at startup withExactly once is enabled, but not found non-nullable primary key or unique key for table ...instead of running and losing rows. This also applies to an existing job whose configuredprimaryKeyscolumn is nullable in the database but holds no NULLs. - Migration Guide: Declare a NOT NULL unique column, or configure
table-names-config.primaryKeys/snapshotSplitColumnwith a non-nullable column, to keep parallel snapshot reading. Forexactly_once = true, add a primary key or a non-nullable unique key to the table, or make the configuredprimaryKeyscolumn NOT NULL in the database.
- Description: For a table without a primary key, the snapshot split column was taken from a unique key even when that column was nullable. Rows with NULL in it matched no snapshot chunk and were silently skipped. A nullable column is now skipped, whether it comes from a unique key, from
PostgreSQL CDC pgoutput Schema
- Behavior change: For streaming jobs using
decoding.plugin.name = "pgoutput", columns not published by the PostgreSQL publication are excluded from the source schema. This includes generated columns before PostgreSQL 18, virtual generated columns, and columns omitted by a publication column list. Snapshot-only jobs and other decoding plugins are unchanged. - Impact: On the next job start or restore, snapshots and auto-created sink tables no longer include unpublished non-key columns. Existing sinks and downstream consumers that expect them may need schema or mapping changes; previously, streamed updates could write NULL into those columns or fail on a schema mismatch. If a required primary key column is unpublished, including in an INSERT-only publication, startup now fails instead of retaining the column and streaming NULL keys.
- Migration Guide: Check the publication and downstream schema before upgrading. On PostgreSQL 18, a publication with
publish_generated_columns = storedcan retain stored generated columns; virtual generated columns remain unpublished. For a column-list publication, include every required primary key column (including configuredtable-names-config.primaryKeys) and all other required columns. PostgreSQL also requires replica identity columns when publishing UPDATE/DELETE. Update sink schemas and downstream mappings to omit other unpublished columns; publications without an explicit column list still follow the generated-column rules.
Markdown file source child_ids array
The Markdown file source now declares child_ids as ARRAY<STRING> and emits a String[] for every row. Previously, it declared STRING, emitted comma-separated IDs for parents, and used null for leaves. Update downstream schemas and consumers to read an array; leaves now contain an empty array. The element IDs, types, page numbers, and position indexes retain their existing formats.
FTP connector: control connection replies are now bounded by a 60 second timeout
- Behavior change: the FTP source and sink stop waiting for a control-connection reply after 60 seconds
- Description: The FTP file system created its client and called
connect(host, port)without setting a read timeout. The underlying client's read timeout defaults to0, which means wait indefinitely, so a server that accepted the TCP connection but never sent a reply — a stalled server, or a connection accepted by a firewall or load balancer that is silently dropping traffic — left the calling thread blocked forever. The newcontrol_timeoutoption bounds every reply read on the control connection, including the greeting, the login exchange and the per-command replies that precede a transfer, and defaults to 60000 ms. It applies to both the source and the sink. - Impact: Jobs against a server that is reachable but does not answer the control channel now fail with a timeout instead of hanging. A job that previously stayed in
RUNNINGwith no progress and no error now fails with theSocketTimeoutExceptionraised by the underlying client, which reports the read that timed out but not the FTP host and port. Connections that answer within 60 seconds are unaffected. This option bounds control-channel replies only: the TCP connect is not covered, and neither is the data connection, so a host that silently drops the initial SYN, or a transfer that stalls after it has started, is not bounded by this option. - Migration Guide: No action is required. If a server legitimately needs more than 60 seconds to answer a control command, for example on a slow link, raise
control_timeouton the connector. Set it to0to restore the previous wait-indefinitely behavior. Note that the upper bound applies per reply, so a sink that uploads a large file and whose server post-processes it can wait longer than 60 seconds for the final226 Transfer complete; raisecontrol_timeoutfor such servers.
- Description: The FTP file system created its client and called
Runtime Requirements
- Breaking Change: Minimum Java runtime raised from Java 8 to Java 11
- Affected component: every module — the whole distribution, the Zeta engine, all connectors, and the published Docker image
- Description: The build now targets Java 11 (
maven.compiler.sourceandmaven.compiler.targetare11), so every published jar contains class file version 55. GitHub CI compiles and tests on JDK 17 against this Java 11 baseline, and the published Docker image moved fromseatunnelhub/openjdk:8u342toeclipse-temurin:11-jdk, keeping a full JDK rather than a JRE sojps/jstack/jmapstay available for diagnosing a running node. - Impact:
- A Java 8 JVM can no longer load SeaTunnel classes. Startup fails with
java.lang.UnsupportedClassVersionError: ... has been compiled by a more recent version of the Java Runtime (class file version 55.0). This applies to the client, to the Zeta master and worker nodes, and to any process that loads connector jars. - Flink: the JobManager and TaskManager JVMs load SeaTunnel connector classes, so the whole Flink cluster must run Java 11 or later, not just the submitting client. Flink supports Java 11 from 1.13 onward, and the official Flink images publish
-java11tags. - Spark: the driver and executor JVMs load SeaTunnel connector classes, so the whole Spark cluster must run Java 11 or later. Spark only gained official Java 11 support in Spark 3.0 (SPARK-24417). Spark 2.4 does still start under Java 11, but it logs illegal reflective access warnings and puts an old commons-lang3 on the classpath that some connectors trip over, so Spark 3.x is strongly recommended.
- Third-party connectors that are themselves compiled for Java 8 keep working. A Java 11 JVM loads older class files without changes, so only the JVM version matters, not the bytecode level of your own jars.
- A Java 8 JVM can no longer load SeaTunnel classes. Startup fails with
- Migration Guide:
- Upgrade the JVM to Java 11 or Java 17 on every node that runs SeaTunnel code: the client, the Zeta master and workers, and the Flink or Spark cluster you submit to.
- If you submit to Flink, move the cluster onto an image or deployment that runs Java 11 or later.
- If you submit to Spark 2.4, upgrade to Spark 3.x running on Java 11 or later. There is no Spark 2.x release that supports Java 11.
- If you customized
${SEATUNNEL_HOME}/config/jvm_options(or the client, master and worker variants), check your additions for flags that Java 11 removed, such as-XX:+UseConcMarkSweepGCor-XX:MaxPermSize, because the JVM refuses to start on an unrecognized flag. The options shipped by default are already Java 11 compatible. - You do not have to copy the new JDK module flags into a preserved config directory.
seatunnel.shandseatunnel-cluster.shappend the mandatory--add-opens/--add-exportsflags (java.base/java.lang,java.net,java.nio,java.util,sun.nio.ch, andjava.security.jgss/sun.security.krb5) themselves and skip any yourjvm_*_optionsalready carries, so an in-place upgrade that keeps an oldconfig/directory (a mounted Docker volume or a Kubernetes ConfigMap) still starts with them. The same scripts stop with an explicitSeaTunnel requires Java 11 or newermessage when the detected JVM is older, instead of the rawUnrecognized optionerror a Java 8 launcher would print.
Application Mode internal API changes
ResourceManagerFactoryis now a single concrete class. Its no-argument constructor selects the standalone manager, while the four-argument constructor creates anApplicationResourceManagerfor application mode. Platform-specific resource-manager subclasses and factory SPI are removed.ResourceManagerDriverinitialization receives aResourceEventHandler, main-thread executor, IO executor, and master-address supplier instead of aResourceManagerContext. Drivers now publish the terminal outcome throughfinish(...)and release their platform clients inclose().AbstractResourceManager.close()is now thefinal, idempotent lifecycle entry point, so it can no longer be overridden. Move deployment-specific cleanup intocloseResourceManager(), whichclose()invokes exactly once; subclasses that overrodeclose()no longer compile and must migrate.
User Variables(-i variables)
-ivalues are no longer visible viaSystem.getProperties()- Previously,
-iparameters were injected intoSystem.getProperties()and accessible viaSystem.getProperty(). Starting from this version,-iparameters are resolved throughConfig.resolveWith()and are no longer visible viaSystem.getProperties().
- Previously,
Duplicate keys via
-iare no longer supported- Duplicate keys passed via
-iare now rejected with an error. Previously, duplicate keys were silently overridden, which could cause unexpected data sync errors due to mistakes such as copy-paste typo. - If your code generation or CLI produces duplicated keys, please remove the unused ones.
- Duplicate keys passed via
Double-quoted
-ivalues are now unwrapped- For example,
-i k="v"is now equivalent to-i k=v(quotes are stripped). Double quotes are mainly intended for string values that contain structural delimiters such as commas (,), curly braces ({}), and square brackets ([]). Plain values without such characters do not need double quotes.
- For example,
Unbalanced
{}or[]are partially parsed into maps and lists- In
-i k=[a,b],c],[a,b]is parsed as an array, andc]is ignored because it has no = (it is not kept as a string). - Balanced values that start with
{or[but are not valid HOCON/JSON now fail with aBadValueerror at parse time. This includes previously working unquoted values such as-i pattern={a,b}. To keep them as plain strings, wrap the value in escaped double quotes:-i pattern=\"{a,b}\"
- In
-ikeys and values are now trimmed, and empty keys are rejected-i 'k1= , k2=v2'now explicitly removes whitespace:k1becomes an empty string, and the key of the second pair becomesk2instead ofk2.-i ' =v1 , k2=v2'now reports an error, because the key forv1is empty after trimming.-i: Meaningful leading and trailing whitespace is silently removed.
SQL TINYINT array schema
The Zeta SQL ARRAY function now declares TINYINT elements as ARRAY<TINYINT>, matching the Byte values it emits. The previous ARRAY<STRING> declaration could fail in schema-dependent row consumers. Update downstream declarations that assumed STRING elements; cast the SQL values to STRING explicitly when that schema is required. Restart affected jobs with the corrected schema rather than restoring state that relies on the old declaration.
Zeta SQL Transform: CAST to INT rejects out-of-range numeric values
- Behavior change:
CASTtoINTfrom a numeric source now fails instead of silently wrapping- Affected component:
seatunnel-transforms-v2,SystemFunction.castAs(Zeta SQL transform) - Description: Converting a numeric value to
INT|INTEGERwent throughNumber.intValue(), which keeps only the low-order 32 bits, soCAST(bigint_col AS INT)on3000000000returned-1294967296and a value just belowInteger.MIN_VALUEcame back as2147483647, flipping sign. A string source already went throughInteger.parseIntand reported the overflow, so the same expression either failed or corrupted the value depending only on the source column type. The numeric path now reports it too.TINYINT,SMALLINTandBYTEare unchanged, because they convert withByte.parseByteandShort.parseShort, which already rejected an out-of-range value. Planner acceptance is unchanged: the set of source types allowed for each target is exactly what it was. - Impact: A job that relied on the wraparound, deliberately or not, now fails at the row that
overflows rather than writing a wrong number. In-range values, widening casts and identity casts
are unaffected.
FLOATandDOUBLEare out of scope for this change and still returnInfinityfor values they cannot represent. - Also affects
COALESCEandIFNULL: these reach the same conversion boundary, and their result type is inferred from the first non-null argument rather than the widest one. SoCOALESCE(int_col, bigint_col)targetsINT, and an out-of-range value taken from theBIGINTargument was silently truncated in the same way. It now fails too.CASEexpressions are not affected, because their type is inferred as the widest branch, so no narrowing occurs. - Covers wide and non-finite numeric sources: the range check runs per numeric family rather
than after a single widening step, because
Number.longValue()is itself lossy for some of them. ADECIMALbeyond 64 bits (reachable asCOALESCE(int_col, decimal_col)), aBigInteger, andNaNor an infinity from aDOUBLEare each rejected now instead of arriving as a truncated value. - Fractional sources are unchanged: reached through
COALESCEorIFNULL, a value such as5.7is still truncated towards zero rather than rejected, which is what this conversion has always done. Only the range behaviour changes. A floating-point source written as an explicitCASTto an integral target is rejected while the statement is prepared, both before and after this change. Note the truncating path still differs from a string source, whereInteger.parseInt("5.7")fails; aligning those is a separate decision. BIGINTis covered by a separate change: it kept its existing conversion when this entry was written; the range check for it is described below and was tracked in #12612.- Migration Guide: Use
TRY_CASTto getNULLinstead of an error for values the target cannot hold, or widen the target type so the value fits. To keep a truncating conversion, compute it explicitly rather than relying onCAST.
- Affected component:
Zeta SQL Transform: CAST to BIGINT rejects out-of-range numeric values
- Behavior change: converting a numeric value to
BIGINTnow fails instead of silently wrapping- Affected component:
seatunnel-transforms-v2,SystemFunction.castAs(Zeta SQL transform) - Description: The
BIGINT|LONGnumeric branch converted withNumber.longValue(), which keeps only the low-order 64 bits of aDECIMALor aBigInteger, mapsNaNto0, and saturates an out-of-rangeDOUBLEatLong.MAX_VALUE. Measured ondev:18446744073709551621(2^64+5) arrived as5,18446744073709551616(2^64) as0,NaNas0, and1e30as9223372036854775807. None of it was reported, so the row was written with a wrong number. The conversion is now range-checked againstLong.MIN_VALUE..Long.MAX_VALUE, reusing the same per-family check added forINT. - Impact: A job that relied on the wraparound, deliberately or not, now fails at the row that overflows rather than writing a wrong number. In-range values, the exact integral wrappers, widening casts and identity casts are unaffected.
- The reachable route is
COALESCEandIFNULL, notCAST: an explicitCAST(... AS BIGINT)could never produce this, because the planner admits onlyTINYINT,SMALLINT,INT,BIGINTandSTRINGas sources for aBIGINTtarget; the four integral families widen to a long exactly, and aSTRINGalready reported its own overflow throughLong.parseLong. ADECIMALorDOUBLEreaches the target throughCOALESCEandIFNULL, whose result type is inferred from the first non-null argument rather than the widest, soCOALESCE(bigint_col, decimal_col)targetsBIGINTand narrowed the decimal silently. TRY_CASTcannot turn these intoNULL: for the same reason.TRY_CAST(decimal_col AS BIGINT)andTRY_CAST(double_col AS BIGINT)are rejected while the statement is prepared, withUnsupported CAST FROM ... AS type: BIGINT, so the per-row path whereTRY_CASTcatches a failure is never reached.TRY_CASTdoes returnNULLfor an overflowingSTRINGsource, which is the one route that reaches the conversion as a cast.- Fractional sources are unchanged: a value such as
5.7reachingBIGINTthroughCOALESCEorIFNULLis still truncated towards zero rather than rejected. Only the range behaviour changes. - Migration Guide: Use
TRY_CASTto getNULLinstead of an error where the source is a string, widen the target type so the value fits, or compute a truncating conversion explicitly rather than relying on the inferred narrowing inCOALESCEandIFNULL. To keep the widest type rather than the type of the first argument, write the cast yourself:COALESCE(CAST(int_col AS BIGINT), bigint_col). That works for an integral source only: aDECIMALorDOUBLEhas no cast toBIGINT, so put it first instead, as inCOALESCE(decimal_col, bigint_col). Reordering is not a drop-in change though: it also changes which argument wins when both are non-null.
- Affected component:
DuckDB BIT and ENUM automatic DDL
- Scalar
BITandENUMcolumns with no catalog length now retain an unspecified STRING length instead of the previous 1/255 fallback. Positive lengths are unchanged. Automatically generated columns use MySQLLONGTEXTor PostgreSQLtextinstead of the old bounded string types. - Existing target tables are not resized. Review their column definitions and widen them manually before transferring values that exceed the existing limits.
- With
create_index = true(the default), MySQL automatic table creation fails when one of these columns is a primary key:LONGTEXTcannot be used as a full-column primary key. Pre-create the target table with an explicitly bounded key type that fits the source data and MySQL index limits, and useschema_save_mode = "ERROR_WHEN_SCHEMA_NOT_EXIST"to preserve that schema. Do not useRECREATE_SCHEMAfor a manually defined target. An arbitrary key-prefix length can reject distinct source keys that share that prefix, so it is not a semantics-preserving substitute.
Helm Chart: Zeta REST API v1 disabled by default
- Behavior change: the Kubernetes Helm chart no longer enables the unauthenticated Zeta REST API v1
- Affected component: Helm chart
deploy/kubernetes/seatunnel(conf/hazelcast-master.yaml,conf/hazelcast-worker.yaml,values.yaml) - Description: The chart previously set
hazelcast.network.rest-api.enabled: true, exposing the deprecated Zeta REST API v1 (includingsubmit-job,stop-job,encrypt-config, logs and thread dump) on the Hazelcast member port (5801) without authentication. It is nowfalse, matching the standaloneconfig/hazelcast.yamldefault and the v1 documentation. The default Prometheus pod annotations are repointed from5801(/hazelcast/rest/instance/metrics) to the REST API v2 / Jetty listener on8080(/metrics), which returns the same samples. - Impact: Deployments that called REST API v1 on port 5801 must switch to REST API v2 on port
8080. Prometheus setups that scraped
5801/hazelcast/rest/instance/metricsdirectly (rather than through the pod annotations) must update the target to8080/metrics. Job submission through the Hazelcast client protocol and REST API v2 on 8080 are unaffected. Because the ConfigMap is mounted withsubPathand the Deployments carry no config checksum annotation, running pods keep the old setting until restarted, so restart the master/worker pods afterhelm upgrade. - Migration Guide: Use REST API v2 on port 8080 (the chart's documented interface). If Zeta REST
API v1 is genuinely required, set
rest-api.enabled: truein a custom ConfigMap (existingConfigMap) and restrict the member port (5801) with aNetworkPolicy. Restart the pods after upgrading so the new configuration is applied.
- Affected component: Helm chart
JsonPath Transform
- Recognized destination-type conversion failures now honor the existing column
and row error policies. Column
SKIPyields a null field; columnSKIP_ROWor rowSKIP(without a column override) drops the row. Explicit columnFAILstill takes precedence; the default remainsFAIL. - Under
FAIL, recognized conversion errors now surface asErrorDataTransformExceptionwithJSONPATH_ERROR_CODE-07, rather than the raw converter exception or itsCOMMON-*code. Source records, extracted values and raw causes are intentionally omitted from these diagnostics and skip logs. Update alerts/runbooks keyed on the old exception text or code. Unsupported conversions and unexpected failures retain their existing behavior; this does not addROUTE_TO_TABLEsupport or change checkpoint formats. - Path-reading error policies are unchanged.
ErrorDataTransformExceptionnow attaches a sanitizedJsonPathExceptioncause, rather than the original exception, because its message or stack trace may contain source data or a private path literal. Previously there was no cause. Diagnostics retain only safe field identifiers and the original exception type; they omit source JSON, configured paths and raw exception text. The cause chain remains available, but the original cause subtype and stack detail are intentionally lost. Update stack-trace parsers and alerts that assumed no nested cause or relied on the previous message text.
Replace Transform Schema Refresh
- Behavior change: Replace resolves
replace_fieldsagainst the refreshed input schema after schema changes and checkpoint restoration. Previously cached positions could replace an unrelated column or leave the configured field unchanged after column positions shifted. - Impact and migration: If a configured field was dropped or renamed, schema refresh now fails with the existing missing-field error instead of processing subsequent rows with stale positions. Update
replace_fieldsto match the source schema before restarting. No option, default, API, or checkpoint format changes are introduced.
SQL and Calcite Vector Arithmetic
- Behavior correction:
VECTOR_NORM,INNER_PRODUCT,COSINE_DISTANCE,L1_DISTANCE,L2_DISTANCEandVECTOR_NORMALIZEnow widen float elements before intermediate multiplication or subtraction. Finite inputs no longer produce incorrect infinity, NaN or zero merely because an intermediate float operation overflows or underflows. Results can also differ in their low-order digits. - Migration: Review thresholds and previously computed vector metrics or normalized embeddings before replaying data. No options, types, schemas or state formats change. Null inputs and actual zero vectors retain their handling. No non-finite input validation or sanitization is added; corrected finite intermediates can also affect results when mixed with non-finite elements. Normalized output elements remain float, and random-projection arithmetic is unaffected.
SQL ARRAY_MAX and ARRAY_MIN Precision
- Behavior correction: exact integral and DECIMAL array extrema
- Affected component:
seatunnel-transforms-v2,Sqltransform withengine = ZETA(default) orINTERNAL, on Zeta, Flink and Spark execution engines. - Description: Arrays containing only Byte, Short, Integer, Long and BigDecimal
values are compared exactly, including mixtures of integral and DECIMAL values.
For example,
ARRAY_MAXover BIGINT values[9007199254740992, 9007199254740993]now selects9007199254740993. A Long1and BigDecimal1.00000000000000000001no longer compare equal. If any Float, Double or other Number subtype (including BigInteger) is present, the whole array retains legacy double-based ordering and its possible precision loss. - Impact: Jobs relying on rounded comparisons may select different values. The selected original element retains its type and DECIMAL scale, so a different result may also have a different original scale. Equal values retain the first element. Declared output types, input coercion, null handling and floating-point ordering are unchanged.
- Migration Guide: No configuration or state migration is required. Review downstream expectations that depended on the previous rounded results.
- Affected component:
Redis Authentication
- Redis sources and sinks now authenticate as the configured nonblank
userin bothSINGLEandCLUSTERmode. Previously,SINGLEused password-only authentication followed byACL SETUSER, andCLUSTERignoreduser. Connection setup no longer creates or modifies ACL users. - Before upgrading, create the intended ACL user and grant its required command and key permissions,
including
INFOfor connector initialization,SELECTinSINGLEmode, andCLUSTER SLOTSfor topology discovery inCLUSTERmode. Setauthto that user's password. An omitted or empty password is sent as an empty string whenuseris nonblank. - To keep using the default user, remove
userand retainauthwhen a password is required. Named users require Redis 6 or later. Legacy configurations without a username remain unchanged.
Zeta SQL Transform: built-in AES_ENCRYPT / AES_DECRYPT
- Behavior change: AES_ENCRYPT / AES_DECRYPT are now built-in functions
- Affected component:
seatunnel-transforms-v2(Zeta SQL transform). - Description:
AES_ENCRYPT(value, key[, iv])andAES_DECRYPT(value, key[, iv])are now built-in Zeta SQL functions and are dispatched before user-registeredZetaUDFs. They useAES/CBC/PKCS5Paddingwith Base64 output; without an explicit IV a random IV is generated and prepended to the ciphertext soAES_DECRYPTcan recover it without an explicit IV. - Impact: A job that registered a custom
ZetaUDFnamedAES_ENCRYPTorAES_DECRYPT(the previous workaround for the missing built-in) will, after upgrading, silently start using this built-in implementation instead of the UDF. If the UDF used a different key derivation, IV handling or output encoding, ciphertext already written by the UDF may fail to decrypt (or, roughly once in 256 for CBC padding, decrypt to garbage). - Migration Guide: Rename the existing UDF, or switch to the built-in functions. To stay
wire-compatible with the
FieldEncryptAesCbcEncryptor, supply the key with thebase64:prefix (a bare key is derived as a passphrase via SHA-256 and is not interchangeable withFieldEncrypt). See SQL Functions for the full contract.
- Affected component:
Zeta SQL Transform: built-in UNIX_TIMESTAMP
- Behavior change: UNIX_TIMESTAMP is now a built-in function
- Affected component:
seatunnel-transforms-v2(Zeta SQL transform). - Description:
UNIX_TIMESTAMP()/UNIX_TIMESTAMP(string[, pattern])/UNIX_TIMESTAMP(dateTime)is now a built-in Zeta SQL function, dispatched before user-registeredZetaUDFs, and its output type is resolved asBIGINT(BasicType.LONG_TYPE). - Impact: A job that registered a custom
ZetaUDFnamedUNIX_TIMESTAMP(the previous workaround for the missing built-in) will, after upgrading, silently start using this built-in implementation instead of the UDF. The declared output type is now forced toBIGINTregardless of what the UDF'sresultTypereturned, which may break the downstream sink schema if the UDF returned a different type. - Migration Guide: Rename the existing UDF, or switch to the built-in function. See SQL Functions for the full contract.
- Affected component:
Zeta SQL Transform: FROM_UNIXTIME and UNIX_TIMESTAMP text patterns use English locale
- Behavior change: text pattern letters are now interpreted in English regardless of the worker JVM locale
- Affected component:
seatunnel-transforms-v2(Zeta SQL transform). - Description:
getFormatter, the sharedDateTimeFormatterfactory used byFROM_UNIXTIME(format path) andUNIX_TIMESTAMP(parse path), now passesLocale.ENGLISHtoDateTimeFormatter.ofPattern. Previously the JVM default FORMAT locale was captured, so text pattern letters such asMMM(month name),EEE(weekday), ora(AM/PM) rendered or parsed in the worker's language. - Impact: On a non-English worker (for example
de-DE),FROM_UNIXTIME(epoch, 'dd MMM yyyy')previously produced23 Okt 2026and now produces23 Oct 2026;UNIX_TIMESTAMP('23 Oct 2026', 'dd MMM yyyy')previously returnedNULL(parse failure) and now returns the correct epoch. English-locale workers are unaffected. This is consistent withDAYNAMEandMONTHNAME, which already pin English viaDateFormatSymbols.getInstance(Locale.ENGLISH). - Migration Guide: No action is needed for correctness — English was always the intended locale for text patterns. If a downstream consumer relied on localized month or weekday names in
FROM_UNIXTIMEoutput, update it to expect English names. See SQL Functions for the full pattern reference.
- Affected component:
RabbitMQ Connector
- Breaking Change:
amqps://connections now verify broker certificates- Affected component:
seatunnel-connectors-v2/connector-rabbitmq - Description: Previously, connecting with an
amqps://url/uriimplicitly installed a trust-all trust manager without hostname verification. Certificate verification is now enforced foramqps://connections, consistent with thessl = truehost/port path. - Impact: Jobs that connect with
amqps://URLs to brokers using self-signed or private-CA certificates will fail to connect after upgrading. - Migration Guide: Import the broker certificate (or your private CA chain) into the JVM
trust store of the SeaTunnel runtime, or switch to the
host/port+ssl = trueconfiguration with a properly configured trust store.
- Affected component:
FakeSource (connector-fake)
- Declarative option constraints are now enforced at factory validation time instead of
silently passing and failing only at runtime. Affected options:
split.num,vector.dimensionandbinary.vector.dimensionmust be > 0;row.num,split.read-interval,map.size,array.size,bytes.lengthandstring.lengthmust be= 0;
tinyint.min/max,smallint.min/max,int.min/max,bigint.min/max,float.min/max,double.min/maxandvector.float.min/maxmust satisfy min <= max. Note thatrow.num = 0(empty source) is still valid. Existing jobs that set invalid values and previously ran successfully will now fail fast at startup with a validation error.
SQL Numeric Predicate Precision
- Behavior change: exact numeric operands no longer compare through
double- Affected component:
seatunnel-transforms-v2,Sqltransform (ZETAandINTERNAL). - Description: Comparisons and
IN/NOT INchecks preserve precision when both evaluated operands are integral orDECIMAL, including comparisons in searched and simpleCASEexpressions. For example,BIGINT9007199254740993no longer equals the integer literal9007199254740992. - Impact: Filters can select different rows, and
CASEexpressions can choose different branches. Comparisons involvingFLOATorDOUBLEretain their existing behavior. Literals containing a decimal point still evaluate asDOUBLE. Unary minus preservesDECIMALprecision for both-decimal_coland-CAST('...' AS DECIMAL(p, s)). EachINelement independently selects the exact or floating path, so mixing exact and floating operands can still match distinct large values and does not guarantee transitive equality. - Migration Guide: Review affected predicates and
CASEexpressions, and reconcile previously written data that relied on rounded comparison results. For exact decimal constants, use quoted strings with sufficient scale, such asCAST('-123456789012345678.99' AS DECIMAL(38, 2))or-CAST('123456789012345678.99' AS DECIMAL(38, 2)). Existing CAST rounding is unchanged: string/DECIMALinputs useCEILINGwhen reducing scale, so negation before or after CAST can produce different results.FLOAT/DOUBLEinputs useHALF_UPand cannot recover precision already lost to floating-point evaluation. See Numeric comparisons for examples and limits. No configuration, output schema or state format changes are required.
- Affected component:
DataValidator Array Length
The LENGTH rule now counts array elements rather than characters in the
array's Java identity string. This can change which array-valued rows pass
validation. Review array length limits that were adjusted to the previous
behavior, and check previously skipped or routed data before replaying it.
String, collection, binary and null handling remain unchanged. No configuration
or state format migration is required.
Zeta REST Pagination Parameter Validation
- Behavior change:
pageandrowsare validated on paginated endpoints- Affected component:
seatunnel-engine-server, REST endpointsGET /finished-jobs/:state,GET /running-jobsandGET /running-jobs/summary. The latter two are served by the sameRunningJobsServletinstance, so both receive the validation. - Description: These endpoints now reject a
pageorrowsvalue that is not an integer or is not greater than 0, and reject a page whose start offset would overflow a 32-bit integer. Previouslyrows=0was accepted and returned an empty page, a negativerowsproduced an internal error, and a sufficiently largepagecombined withrowscould wrap to a small positive offset and silently return the wrong page. - Impact: Requests that relied on
rows=0returning an empty page now receive400with a message naming the offending parameter. Callers passing valid positive values are unaffected. The response shape, the{"data": [...], "total": n}envelope, and the behaviour of a page starting exactly attotal, which still returns an empty page, are all unchanged.
- Affected component:
MySQL-CDC: NULL in a nullable unique key column is emitted as NULL
- Behavior change: MySQL-CDC emits NULL instead of the type default for nullable unique key columns of tables without a primary key
- Affected component:
seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql - Description: For a table without a primary key, the MySQL DDL parser (Debezium 1.9.8) promoted the first unique key to the primary key and marked its columns NOT NULL, although the database allows NULL in them. NULL values in these columns were emitted as the type default (for example
0,''or the epoch) in the snapshot and binlog phases, including after a binlog DDL such asALTER TABLE ... ADD UNIQUE KEYorCREATE UNIQUE INDEX. These columns now keep their declared nullability and NULL is emitted as NULL. Real primary key columns are unchanged. This relies on nullable columns no longer being used as the snapshot split column (see "CDC: nullable columns are no longer used as the snapshot split column"): withexactly_once = true, a NULL in the split column would otherwise fail the split range check. - Impact: Sinks now receive NULL where they previously received the type default. A sink column declared
NOT NULL, or a sinkprimary_keysthat includes such a column, can reject these rows. - Migration Guide: Make the affected sink columns nullable, and do not use a nullable column as the sink primary key. A job restored from a checkpoint that already holds the schema of such a table (saved after a binlog DDL on it) keeps the old behavior for that table until it is started without that checkpoint.
- Affected component:
Connector Changes
Breaking Change: Doris Source option key
doris.request.retriesdoris.deserialize.queue.sizerenamed todoris.deserialize.queue.size- Affected component:
seatunnel-connectors-v2/connector-doris(DorisSourceOptions.DORIS_DESERIALIZE_QUEUE_SIZE) - Description: The option key for the asynchronous Arrow deserialization queue size has been a typo since it was introduced in #7895: the key was accidentally concatenated as
doris.request.retriesdoris.deserialize.queue.size, gluing the preceding option's name (doris.request.retries) onto the intended key (doris.deserialize.queue.size). The option key is now the intendeddoris.deserialize.queue.size. The default value (64) and the option behavior are unchanged. - Impact: Configurations that explicitly set the old malformed key
doris.request.retriesdoris.deserialize.queue.sizewill no longer be picked up; the connector will fall back to the default queue size of64. The old key was a concatenation artifact and could only be discovered by copying it from the docs, so most users are unaffected. - Migration Guide: If you explicitly tuned this option, rename the key to
doris.deserialize.queue.sizein your source configuration.
- Affected component:
Behavior change: HTTP sink write failures now fail the task instead of being silently dropped
- Affected component:
seatunnel-connectors-v2/connector-http/connector-http-base - Description: Previously,
HttpSinkWriter.doHttpRequesthandled both a non-200 HTTP response and any request exception (network error, timeout, serialization error) by logging aterrorlevel and returning normally, so the failed row/batch was silently dropped while the job kept running and checkpoints completed. The writer now throwsHttpConnectorException(REQUEST_FAILED) for both cases, so the failure propagates to the engine and fails the task/job. - Impact: Jobs whose downstream HTTP endpoint occasionally returns non-200 or is occasionally unreachable used to keep running with silent data loss; after this change they fail loudly at the first failed write. The connector still has no built-in retry or dead-letter mechanism, so re-submitting a failed job may deliver rows that succeeded before the failure again — make sure the receiver tolerates duplicate delivery on retry/restart.
- Migration Guide: No configuration change is required. If your endpoint is expected to return non-200 responses as part of normal operation, handle them upstream of the sink or add an external retry mechanism before upgrading.
- Affected component:
Transform Changes
- Behavior change: every
table_transformentry is validated at submission time- Affected component: all transforms that support
table_transform:Copy,Calcite,DataValidator,DefineSinkType,DynamicCompile,FieldEncrypt,FieldMapper,Filter,FilterRowKind,JsonPath,Metadata,Python,RegexExtract,Replace,Sql,Split,TableMergeandTextChunk. - Description: An entry without a non-blank
table_pathused to be filtered out silently, so the entry was ignored and the job ran anyway; entries were not checked against the per-table rule until a row was processed. Entries are now validated before the job starts: each one has to select its table with a non-blanktable_pathand to be a complete single-table rule of that transform, and the transform-level rule has to be either absent or complete. - Impact: A config that still carries a stale entry - for example
table_transform = [{ field_rules = [...] }]withouttable_path- was accepted before and now fails at submission withEach 'table_transform' entry must select its table with a non-blank 'table_path', or withInvalid 'table_transform' entry for table [...]when a mandatory option of the entry is missing. Single-table jobs (withouttable_transform) are not affected. - Migration Guide: Add
table_pathto every entry and complete it like a single-table config of the same transform. Entries that are no longer needed can also be removed, because atable_transform-only config is valid.
- Affected component: all transforms that support
- [BREAKING] Assert sink:
sink_typeincolumn_ruleis now checked against the actual column type. It was ignored before, so a mismatching declaration fails the job while an omitted one stays unchecked; remove the key or set the actual type when upgrading.
Format Changes
CSV source defaults for missing and configured null fields: When a CSV source declares
schema.columns[].defaultValue, a column absent from the file header or a field matchingnull_formatnow receives that default instead of null. Jobs without configured defaults are unchanged, and an empty field remains null unless it matchesnull_format. If downstream logic intentionally relied on null despite a configured default, remove that default from the source schema before upgrading. Configured defaults are now converted to the column type when the CSV reader is created. A default that cannot be converted, such asCURRENT_TIMESTAMPfor a TIMESTAMP column, now fails the job at reader creation instead of being ignored, even if no field needs the default. Defaults must be literal values, not SQL expressions. Fix or remove invalid defaults before upgrading.Breaking Change: JSON serialization of numeric fields now follows the runtime value type
- Affected component:
seatunnel-formats/seatunnel-format-json(RowToJsonConverters) - affects every connector that serializes rows with the JSON format (for example Kafka, RabbitMQ, Pulsar, and file JSON sinks) - Description: Previously, a field declared as a numeric type in the catalog (
TINYINT,SMALLINT,INT,BIGINT,FLOAT,DOUBLE,DECIMAL) was serialized by blindly casting the runtime value to the Java type implied by the declared type (for example(long) valueforBIGINT). In multi-table jobs (for example CDC jobs writing JSON to RabbitMQ/Kafka) where several tables share one catalog schema but carry different physical column types, aStringorBigDecimalruntime value in such a field threw a rawClassCastExceptionand killed the job. Now numeric fields are serialized according to their runtime type: any numeric wrapper (Byte,Short,Integer,Long,Float,Double,BigInteger,BigDecimal) becomes the corresponding JSON number; numeric character sequences are parsed into JSON numbers, while non-numeric text is emitted as a JSON string;Float/Doublevalues in a field declared asDECIMALare serialized viaBigDecimal.valueOfto avoid floating-point representation artifacts. - Impact: Heterogeneous numeric values that previously crashed the job with
ClassCastExceptionnow serialize successfully, and the emitted JSON numeric shape follows the runtime value rather than the declared column type (aStringorBigDecimalvalue in aBIGINTcolumn keeps its exact numeric value). Runtime values that can neither be represented as a number nor parsed from text (for examplebyte[],Map,LocalDateTime) now fail fast with a typedSeaTunnelJsonFormatException(UNSUPPORTED_DATA_TYPE) instead of a rawClassCastException. Downstream consumers that assume the JSON numeric shape always matches the declared column type should be reviewed. (#11415)
- Affected component:
Behavior change: JSON deserialization keeps the precision of
DECIMALvalues nested in arrays- Affected component:
seatunnel-formats/seatunnel-format-json(JsonDeserializationSchema) - affects every connector that reads rows with the JSON format (for example Kafka, RabbitMQ, Pulsar, and file JSON sources) - Description: A
DECIMALdeclared inside anARRAY(for exampleARRAY<DECIMAL>,ARRAY<ARRAY<DECIMAL>>orARRAY<ROW<DECIMAL>>) was not found when deciding whether the numbers of a document must be parsed asBigDecimal, because an array is not a composite type. Its elements were parsed asdoublefirst and converted toDECIMALafterwards, so they lost every digit beyonddoubleprecision, while a scalarDECIMALcolumn of the same schema kept them. ADECIMALnested in an array now retains its numeric precision exactly like a scalarDECIMAL. No option, default value or row shape changed. - Impact: Elements of a declared
DECIMALarray are no longer rounded to about 17 significant digits. Since the underlying mapper setting applies to the whole document, in a schema that declares aDECIMALinside an array a bare number assigned to aSTRINGfield now keeps all of its digits (3.14159265358979323846instead of3.141592653589793). The same happens to floating-point numbers nested inside a JSON object or array that aSTRINGfield captures as text: the object or array is re-rendered from the decimal values, so those numbers can keep more digits or switch to exponent notation (1e-10becomes1E-10), while numbers quoted as JSON strings inside the same object or array keep their original text. Quoted JSON strings, and schemas without anyDECIMAL, are unaffected. (#12871) - Migration Guide: If a downstream step compares the text of such numeric values, review those comparisons against the exact digits, or make the source provide quoted JSON strings for
STRINGcolumns that carry bare numbers. JSON objects or arrays stored in aSTRINGcolumn belong to the same category: compare their serialized text (digits and exponent notation) before relying on it, or have the source send those values as quoted JSON strings.DECIMALcolumns require no configuration change to retain their declared numeric precision.
- Affected component:
Engine Behavior Changes
File-based IMap storage: legacy job model class names in WAL files are remapped on startup
- Affected component:
seatunnel-engine/seatunnel-engine-storage/imap-storage-plugins/imap-storage-file - Description: SeaTunnel 3.0 relocated the job status model classes (
JobStatus,JobResult,JobStatusData) fromorg.apache.seatunnel.engine.core.job.*toorg.apache.seatunnel.engine.common.job.*(#9689). WAL files written by releases before that relocation still record the old fully-qualified class names, so replaying them on a current distribution fails withClassNotFoundException. The file-based IMap storage now transparently rewrites these three legacy names to their current locations while reading WAL entries. No configuration change is needed. - Impact: Clusters using file-based IMap persistence can upgrade and restart across the #9689 relocation without manual WAL surgery. Writing is unaffected: new WAL entries always record the current class names. Note that the rewrite is one-directional: WAL files written by a current distribution contain class names that older releases cannot resolve, so downgrading a cluster after it has written new WAL entries requires deleting or manually rewriting the persisted WAL data. (#11781)
- Affected component:
Behavior change: the REST log-content endpoints return at most 64 MB by default
- Affected component:
seatunnel-engine-server, REST v2 endpointsGET /logs/:fileandGET /log/:fileand their REST v1 equivalentsGET /hazelcast/rest/maps/logs/:fileandGET /hazelcast/rest/maps/log/:file. - Description: These endpoints read the requested log file whole, which materialises it on the
heap twice, so a single request for the log of a long-running streaming job could exhaust a
node's memory. The new
seatunnel.engine.http.log-response-max-size-mboption caps how much is read and defaults to64. A larger file is represented by its lastlog-response-max-size-mbof UTF-8 content, aligned to a complete line when possible (or a partial tail of an oversized line). The response notice names the actual retained bytes and the file-size snapshot. - Impact: A cluster upgraded without editing
seatunnel.yamlstarts receiving the tail rather than the whole of any log file above 64 MB, with status200as before. Anything that archives logs through these endpoints -curl .../logs/<job-id> > job.log, or the log-analysis flow indocs/en/engines/zeta/log-analysis-with-ai.md- keeps a partial file unless the limit is raised. The truncation notice on the first line makes a partial response recognisable. - Migration Guide: Set
log-response-max-size-mb: 0underseatunnel.engine.httpto restore the previous unlimited reads, or raise it to a value that covers the log sizes you collect. Leaving it at the default is recommended, since an unlimited read of a multi-gigabyte log has to fit in the node's heap.
- Affected component:
Behavior change: part of the engine startup log output changes
- Affected component:
seatunnel-engine-server(SeaTunnelServer,AbstractResourceManager,StandaloneResourceManager); only log-based monitoring is affected. - Description: The
SeaTunnel server start...,Init ResourceManagerandinitWorker...lines are no longer written. Worker synchronisation now logsSynchronizing existing worker profiles, and the standalone resource manager logsInit standalone ResourceManager. - Impact: Alerting or log parsing that matches the old text stops matching; runtime behaviour and the cluster lifecycle are unchanged.
- Migration Guide: Match the new messages, or key on the surrounding lifecycle events instead of the exact text.
- Affected component:
Dependency Upgrades
3.0.0
MySQL CDC Schema-Change Parsing
- Behavior change: DDL parser listener errors are propagated
- Affected component:
connector-cdc-mysql - Description: Errors raised while processing a parsed DDL are no longer swallowed and treated as a no-op. They are now propagated as parsing failures so that a CDC job cannot silently skip a schema change.
- Impact: A job may fail on a DDL statement that was previously ignored after an internal parser/listener error. Review the source DDL and update it to syntax supported by the connector before restarting the job. This change does not alter checkpoint or savepoint formats.
- Affected component:
JDBC Connector
Breaking Change: JDBC XA restore now uses recovery-order evidence and fail-closed gaps
- Affected component:
seatunnel-connectors-v2/connector-jdbcsink exactly-once XA path - Description: SeaTunnel now consumes
max_commit_attemptswithin a single aggregated-commit or restore invocation, and restore replays only the still-prepared suffix starting from the first checkpoint XID that remains in the XA recovery scan. Missing XIDs before that boundary are treated as already resolved only after the suffix commits successfully. If none of the checkpoint XIDs remain in the recovery scan, SeaTunnel treats the whole batch as already resolved and skips replay. If a missing XID appears after the first recovered checkpoint XID, restore still fails closed instead of inferring a successful commit fromXAER_NOTA-like absence alone. - Impact: Jobs that previously relied on restore inferring success from a missing XA branch may now fail during recovery when the XA recovery scan still contains later checkpoint XIDs but shows a gap after them. Operators may also observe that
max_commit_attemptsis exhausted within one restore/commit invocation rather than across repeated task restarts. - Migration Guide: Before upgrading, inspect the resource manager for dangling prepared XA transactions (for example
XA RECOVERon MySQL orpg_prepared_xactson PostgreSQL). If recovery fails closed because a later checkpoint XID still exists but a following one is missing, investigate whether the missing XID was rolled back, expired, or cleaned up externally before retrying the job. XA recovery cannot distinguish a SeaTunnel-committed XID from one rolled back or removed by an external cleanup actor. Therefore, a missing prefix or all-absent batch is inferred to be resolved; do not externally clean up SeaTunnel-owned prepared branches while their jobs may be restored, and coordinate any cleanup with job recovery.
- Affected component:
Breaking Change: Mapping of timezone-aware timestamp columns to
TIMESTAMP_TZtype- Affected component:
seatunnel-connectors-v2/connector-jdbc,seatunnel-connectors-v2/connector-iceberg,seatunnel-connectors-v2/connector-cdc-base,seatunnel-connectors-v2/connector-cdc-tidb,seatunnel-connectors-v2/connector-starrocks,seatunnel-connectors-v2/connector-hudi,seatunnel-connectors-v2/connector-snowflake(via JDBC dialect) - Description: Previously, JDBC sources mapped both timezone-naive (e.g., MySQL
DATETIME) and timezone-aware (e.g., MySQLTIMESTAMP) timestamp columns to SeaTunnel's internalTIMESTAMPtype. Now, timezone-aware columns like MySQLTIMESTAMP, PostgreSQLtimestamptz, OracleTIMESTAMP WITH LOCAL TIME ZONE, SQL Serverdatetimeoffset, SnowflakeTIMESTAMP_LTZ/TZ, and others are explicitly mapped toTIMESTAMP_TZ. This ensures that timezone semantics are accurately preserved when writing to formats like Iceberg, whereTIMESTAMPis saved astimestamp(without timezone) andTIMESTAMP_TZis saved astimestamptz(with timezone). - Impact: If your downstream Sink relies on receiving
TIMESTAMPtypes and does not supportTIMESTAMP_TZnatively, you may encounter type mismatch errors. For Iceberg users, this means columns previously written astimestamp(without timezone) may now be written astimestamptz(with timezone) and change the table schema. You may need to cast the column in sql transform or update your sink configurations. (#10685) - Connector-specific behavior changes:
- Snowflake:
TIMESTAMP_LTZandTIMESTAMP_TZcolumns are now mapped toOFFSET_DATE_TIME_TYPE(TIMESTAMP_TZ) instead ofLOCAL_DATE_TIME_TYPE. This affects both Source and Sink paths for Snowflake. - StarRocks:
TIMESTAMP_TZvalues written to StarRocks Sink are stored asDATETIME(wall-clock only, timezone offset is dropped) due to StarRocks not having a native timezone-aware datetime type. - Hudi:
TIMESTAMP_TZis now mapped to AvrotimestampMillis(UTC epoch). Existing Hudi tables written with the old schema may need to be re-created if schema evolution is not supported. - CDC (Debezium-based, TiDB): CDC connectors now correctly handle
TIMESTAMP_TZtype in the Debezium deserialization layer. Previously,TIMESTAMP_TZwas unsupported and would throwUnsupportedOperationException. Users who were previously unable to use timezone-aware columns in CDC pipelines can now do so. - Iceberg (existing tables): Before this PR, SeaTunnel's
TIMESTAMPtype was incorrectly written to Iceberg astimestampwith timezone (withZone()). After this PR,TIMESTAMPis written astimestampwithout timezone (withoutZone()), and IcebergwithZone()columns are read back asTIMESTAMP_TZ. Upgrade impact: If you have existing Iceberg tables where timestamp columns were created by an older SeaTunnel version, those columns are stored aswithZone(). After upgrading, SeaTunnel will read them asTIMESTAMP_TZinstead ofTIMESTAMP. Downstream sinks or transforms that expectedTIMESTAMPmay encounter type mismatch errors. Migration: Re-create the affected Iceberg table with the new schema, or use a SQL Transform to castTIMESTAMP_TZback toTIMESTAMPin your pipeline configuration. - TIMESTAMP_TZ downgrade contract: SeaTunnel applies a two-tier serialization contract for
TIMESTAMP_TZdepending on what the sink format can represent:- DB column-typed sinks without native timezone support (Doris, StarRocks, Xugu): The timezone offset is dropped and the wall-clock value (local datetime) is stored. For example,
2024-01-01T03:00:00+09:00is stored as2024-01-01 03:00:00. This is a lossy operation — the original UTC instant cannot be recovered from the stored value alone. - String/text-based sinks (Text file, Kafka, Pulsar, RocketMQ, RabbitMQ, Redis, etc.): The full ISO 8601 offset is preserved (e.g.,
"2024-01-01T03:00:00+09:00"). These formats can represent timezone offsets as strings, so no information is lost. If you need wall-clock behavior for a string sink, use a SQL Transform to castTIMESTAMP_TZtoTIMESTAMPbefore writing.
- DB column-typed sinks without native timezone support (Doris, StarRocks, Xugu): The timezone offset is dropped and the wall-clock value (local datetime) is stored. For example,
- Xugu TIMESTAMP_TZ (lossy): Xugu
TIMESTAMP WITH TIME ZONEcolumns are exposed asTIMESTAMP_TZat the type layer, but the actual write path drops the timezone offset and stores only the wall-clock value due to a Xugu JDBC driver batch limitation (bug [E19138]). A warning is logged on the first write.
- Snowflake:
- Affected component:
API Changes
Breaking Change: An unknown log level is rejected by the runtime log level endpoint
- Affected component: SeaTunnel Engine REST API —
POST /hazelcast/rest/maps/log-level - Description: The endpoint answered
200with{"status":"SUCCESS"}for every request, including a level name it could not resolve (DEBUGG,verbose, a lowercase name of a level that does not exist, an empty value). Nothing was applied in that case, and the unresolved level was handed to log4j2 asnull, which removes the explicit level of the logger instead of leaving it alone — so the logger silently fell back to its parent, or toERRORfor the root logger. An unknown level, a blank level and a missinglevelparameter are now rejected with400and a message listing the valid levels; a level name is still accepted in any letter case. - Impact: Scripts and automation that only check the HTTP status now see
400where they used to see200, for requests that never took effect in the first place. Requests with a resolvable level are unchanged. - Migration Guide: Send a level log4j2 knows (
OFF,FATAL,ERROR,WARN,INFO,DEBUG,TRACE,ALL, or a level registered by the configuration). The response body of a rejected request names the levels the node accepts.
- Affected component: SeaTunnel Engine REST API —
Breaking Change:
Condition.of(option, null)no longer allowed- Affected component:
seatunnel-api—org.apache.seatunnel.api.configuration.util.Condition - Description: The
Conditionconstructor now validates that binary literal operators (such asEQUAL,NOT_EQUAL,GREATER_THAN, etc.) must have a non-nullexpectValue. Previously,Condition.of(option, null)was silently accepted; it now throwsIllegalArgumentExceptionat construction time. - Impact: No production code in the main repository uses
Condition.of(option, null), so the practical impact is zero. However, any custom or third-party connector code that relied on this pattern will need to be updated. - Migration Guide: If you need to check whether an option is absent or unset, use
Conditions.notBlank(option)(for strings) or handle the absence at theOptionRule.Builderlevel withoptional(...)instead of passingnullas the expected value.
- Affected component:
Breaking Change:
OptionValidationExceptionmessage format changed to structured aggregation- Affected component:
seatunnel-api—org.apache.seatunnel.api.configuration.util.ConfigValidator - Description:
ConfigValidator.validate(OptionRule)now collects all structural and value constraint errors and throws a singleOptionValidationExceptionwith a structured multi-line message instead of failing on the first error.
Before (fail-fast, single error)
ErrorCode:[API-02], ErrorDescription:[Option item validate failed] - There are unconfigured options, the options('host') are required.After (aggregated, structured)
ErrorCode:[API-02], ErrorDescription:[Option item validate failed] - Option validation failed (2 errors):
[1] option: 'host'
type: required
constraint: required option is not configured
[2] option: 'port'
type: value
constraint: 'port' >= 1- Impact: Code that parses the exception message by matching substrings like
"are required"or assumes a single-error format will need to be updated. The error code (API-02) and the" - "separator between the code prefix and the body remain unchanged. - Migration Guide: Update any string-matching logic on
OptionValidationException.getMessage()to handle the new multi-line numbered format. UsegetRawMessage()to get the body without theErrorCodeprefix if needed.
- Affected component:
Configuration Changes
Breaking Change: Released connector installation defaults to direct HTTPS downloads
- Affected component:
bin/install-plugin.shon Linux and macOS - Description: Fixed release versions are now downloaded directly from Maven Central over HTTPS and verified with a published SHA-512 or SHA-1 checksum. Previously, every connector was resolved through the bundled Maven Wrapper.
- Impact: Existing environments that depend on Maven
settings.xmlfor mirrors, authenticated repositories, proxies, or custom TLS policies may no longer install released connectors with the default command. - Migration Guide: Set
SEATUNNEL_PLUGIN_DOWNLOAD_METHOD=mavenwhen runninginstall-plugin.shto preserve the previous Maven resolution behavior. Alternatively, setSEATUNNEL_MAVEN_REPOSITORYto an HTTPS Maven-compatible mirror that publishes connector checksum files.
- Affected component:
Breaking Change: CatalogFactory creation path now validates
optionRule()- Affected component:
seatunnel-api—FactoryUtil.createOptionalCatalog() - Description: The
FactoryUtil.createOptionalCatalog()method now callsConfigValidator.validate(catalogFactory.optionRule())before creating a catalog instance. Previously, no validation was performed on the catalog factory's option rules during catalog creation. - Impact: Catalog factories whose
optionRule()declares options asrequiredthat are not always present in the config passed tocreateOptionalCatalog()will now throwOptionValidationException. This primarily affects the JDBC connector path viaJdbcCatalogUtils.findCatalog(). - Migration Guide: If you have a custom
CatalogFactoryimplementation, ensure that itsoptionRule()accurately reflects which options are truly mandatory vs optional in the config that reaches it at runtime.
- Affected component:
Connector Changes
Breaking Change: BigQuery Sink Connector — default schema save mode introduces automatic table creation
- Affected component:
seatunnel-connectors-v2/connector-bigquery - Description: The BigQuery sink connector (
connector-bigquery) now implementsSupportSaveModewith support forschema_save_modeanddata_save_mode. The defaultschema_save_modeis set toCREATE_SCHEMA_WHEN_NOT_EXIST. - Impact: Upgrading existing pipelines targeting a non-existent table will now automatically create the table in BigQuery with the source schema instead of failing fast at the BigQuery API layer.
- Migration Guide: To preserve the legacy fail-fast behavior, explicitly configure
schema_save_mode = "ERROR_WHEN_SCHEMA_NOT_EXIST"in your BigQuery sink configuration.
- Affected component:
Breaking Change: ORC file sink preserves case of nested struct field names
- Affected component:
seatunnel-connectors-v2/connector-file/connector-file-base(used by all File/HDFS/S3/OSS ORC sinks that shareOrcWriteStrategy) - Description: Previously,
OrcWriteStrategy.buildFieldWithRowType(...)forced every nestedROW(struct) field name to lowercase when building the ORC schema, so a nested field declared asMD5was persisted asmd5in the file footer. Downstream consumers that read the column by its declared original-case name received null/missing values. The.toLowerCase()call has been removed from the recursive nested-field branch, so nested struct field names are now written verbatim in the file schema. - Impact: ORC files written by SeaTunnel after this change embed the original-case nested field names in their schema footer. Users that adapted to the old behavior (for example, case-sensitive ORC readers with
orc.schema.evolution.case.sensitive=true, Spark withspark.sql.caseSensitive=true, or pipelines that expectedmd5rather thanMD5) will see the inverse problem: null values or schema mismatches when reading new files. Directories that mix pre-upgrade files (lowercase nested names) with post-upgrade files (original case) will contain inconsistent nested-schema shapes for the same logical column, which case-sensitive schema merging cannot reconcile. - Migration Guide:
- Mixed-version directories: Re-materialize the directory so every file is produced by the new version, or write pre- and post-upgrade files into separate directories and read them independently.
- Case-sensitive consumers: Configure the reader for case-insensitive schema evolution where supported, or remap the column at read time.
- Case-only sibling fields (for example
MD5andmd5in the same struct): now representable; case-insensitive downstream consumers (such as Hive) may treat them as ambiguous — disambiguate at the source if needed.
- Affected component:
Breaking Change: Google Bigtable Source
scan_row_limitis now a per-split cap- Affected component:
seatunnel-connectors-v2/connector-google-bigtable - Description: The enumerator now partitions a table (or the configured
start_rowkey/end_rowkeyrange) into tablet-sized splits viasampleRowKeys.scan_row_limitis still applied withquery.limit(...)once per split in the reader. Before this change the source always produced exactly one split, soscan_row_limitacted as a table-wide row cap. After this change a table with multiple tablets yields multiple splits even whenparallelism = 1(the single reader is assigned every split), and the job-level upper bound is aboutscan_row_limit × split count. See Google Bigtable Source. - Impact: Existing jobs that set
scan_row_limitto bound total output (sampling, testing, cost control, or downstream capacity) can read far more rows after upgrade with no config change. - Migration Guide: If you need a table-wide cap, narrow the scan with
start_rowkey/end_rowkey, or lowerscan_row_limitso thatscan_row_limit × expected split countstays within the previous budget. To keep the previous single-split behavior, the connector still falls back to one split when sampling fails, returns no keys, or the intersection is empty — that is not a supported way to pin the old cap. (#11876)
- Affected component:
CDC Connector: restored state for tables removed from the capture set is no longer reused
- Affected component:
seatunnel-connectors-v2/connector-cdc/connector-cdc-baseand CDC connectors built on it. - Description: When a CDC job restores from a checkpoint or savepoint, SeaTunnel now filters per-table incremental state against the currently captured table set before assigning the restored split. State for tables that have been removed from the job's capture configuration is not reused. If table discovery is unavailable or returns no tables, SeaTunnel keeps the restored state unchanged to avoid discarding checkpoint metadata during a transient source-database problem.
- Impact: A job that removes captured tables and then restores from an older checkpoint no longer attempts to resume incremental state for those removed tables. This avoids restore failures caused by stale table metadata. The behavior applies only during checkpoint/savepoint restore; newly started jobs are unchanged.
- Migration Guide: No configuration change is required. Before restoring an existing CDC job after changing its capture table set, verify that the removed tables are intentionally no longer part of the job.
- Affected component:
Breaking Change: Iceberg Connector — source table primary key is no longer silently inherited
- Affected component:
seatunnel-connectors-v2/connector-iceberg - Description:
SchemaUtils.toIcebergSchema()previously fell back to the CDC source table's primary key wheniceberg.table.primary-keyswas not explicitly configured. This silently setidentifier-field-idson auto-created Iceberg tables, activating equality-delete semantics and causing silent INSERT data loss in append-only CDC pipelines (see #10747). The fallback has been removed. - Impact: Jobs that set
iceberg.table.upsert-mode-enabled=truewithout an expliciticeberg.table.primary-keyswill now fail at startup with a clearIllegalArgumentException. Jobs that relied on implicit PK inheritance to drive upsert semantics must now seticeberg.table.primary-keysexplicitly. - Migration Guide:
- Upsert mode jobs: Add
iceberg.table.primary-keys = "<your key columns>"to the Iceberg sink config. - Append-only CDC jobs: No action needed — omitting
iceberg.table.primary-keysnow correctly routes writes through the pure append writer with no equality deletes. - Existing Iceberg tables that already have
identifier-field-idsstored in their Glue/Hive metastore schema are not affected at runtime; only newly auto-created tables change behavior.
- Upsert mode jobs: Add
- Affected component:
Breaking Change: File source connectors reject POI-engine Excel files larger than
poi_excel_max_file_size(default 50 MB)- Affected component:
seatunnel-connectors-v2/connector-file(LocalFile, HdfsFile, S3File, FtpFile, SftpFile, OssFile, OssJindoFile, ObsFile, CosFile) - Description: Apache POI fully materializes an Excel workbook into memory before any row can be read, which can drive a Zeta worker into heavy GC pressure or OOM on large
.xls/.xlsxfiles. A newpoi_excel_max_file_sizeoption (default 50 MB) now makes POI reject an Excel file that exceeds the limit before the workbook is built. The guard covers both plain and archived (ZIP/TAR/TAR_GZ/GZ) Excel entries, and applies only whenexcel_engine = POI(the default); the streamingexcel_engine = EasyExcelpath is not bound by this limit. - Impact: Existing jobs that read POI-engine Excel files larger than 50 MB - which previously succeeded at the cost of heavy memory pressure - will now fail fast with a
FileConnectorExceptioninstead of potentially OOMing the worker. - Migration Guide: For POI jobs that must read large Excel files and have sufficient worker memory, raise the limit with
poi_excel_max_file_size = <bytes>. Otherwise switch toexcel_engine = EasyExcel, which streams rows lazily and is not subject to the limit.
- Affected component:
Breaking Change: Prometheus Sink
flush_intervaloption removed- Affected component:
seatunnel-connectors-v2/connector-prometheus - Description: The Prometheus Sink no longer starts its own background flush thread. The connector-level
flush_intervaloption has been removed. Timer-based flushing is now driven by the engine throughsink.flush.intervalin the jobenvblock, which is supported only by the Zeta engine. - Impact:
- Spark and Flink lose sub-checkpoint timer-based flushing. The removed
flush_intervalscheduler was a plain connector-owned thread that ran on all engines. Its replacement,sink.flush.interval, is a Zeta engine primitive; the Spark and Flink sink writer contexts do not implement it, so there is no periodic timer flush on those engines. On Spark and Flink the buffer is flushed when it reachesbatch_size, on checkpoint (the sink flushes inprepareCommit()), and when the writer is closed. Buffered points are therefore bounded by the checkpoint interval rather than held until the job stops; for lower latency between checkpoints, tunebatch_sizeaccordingly. - A leftover
flush_intervalkey in thePrometheussink block is rejected only when the config is validated with--check/--dry-run=static/--dry-run=connect(which runvalidateUnknownKeys). A directly submitted job silently ignores the stray key; the connector logs a warning once per sink writer at startup instead (so a job with parallelism N, multiple tables, or replicas logs it multiple times).
- Spark and Flink lose sub-checkpoint timer-based flushing. The removed
- Migration Guide: Remove
flush_intervalfrom thePrometheussink block. To keep timer-based flushing on Zeta, setsink.flush.interval(milliseconds) in the jobenvblock. On Spark and Flink, buffered points are flushed on each checkpoint; tunebatch_sizefor lower latency between checkpoints. Thebatch_sizetrigger and the final flush on writer close are unchanged on all engines.
- Affected component:
Breaking Change: File connectors reject
DOCTYPEdeclarations in XML input (XXE hardening)- Affected component:
seatunnel-connectors-v2/connector-file/connector-file-base(XmlReadStrategy), and every file source built on it: LocalFile, HdfsFile, S3File, OssFile, OssJindoFile, CosFile, FtpFile, SftpFile (file_format_type = xml) - Description: The XML reader previously parsed user-supplied files with a default dom4j
SAXReader, leaving DTD processing and external entity resolution at their JAXP defaults. A craftedDOCTYPE/external-entity payload could disclose local worker-node files, trigger SSRF-style fetches, or exhaust memory via entity expansion ("billion laughs").XmlReadStrategynow routes every parse through a hardened reader that enables JAXP secure processing, rejects any<!DOCTYPE ...>declaration outright, disables external general/parameter entities and external DTD loading, and installs a deny-allEntityResolveras a parser-agnostic backstop. - Impact: XML files that previously parsed successfully only because they carried a
<!DOCTYPE ...>declaration — even a benign one with no externalSYSTEM/PUBLICreference — now fail withFileConnectorException(FILE_READ_FAILED). There is no configuration option to opt back into the previous behavior. - Migration Guide: Remove the
DOCTYPEdeclaration from XML files before ingesting them with SeaTunnel, or pre-process/re-export the file without it. Well-formed XML without aDOCTYPEdeclaration is unaffected. (#11250)
- Affected component:
Transform Changes
Behavior change: AMAZON embedding honors retry options
- Affected component:
Embeddingtransform withmodel_provider = AMAZON. - Description: Configured SeaTunnel retry and backoff options now reach the Bedrock runtime. Previously, the transform ignored these settings and used one SeaTunnel attempt.
- Impact and migration: Configured
model_retry_max_attemptsvalues greater than 1 now enable SeaTunnel retries, which may incur additional model charges; use 1 to retain a single SeaTunnel attempt. The default remains 1. The SDK's own retry and timeout behavior is unchanged;model_request_timeout_msis not currently applied to Bedrock calls.
- Affected component:
[BREAKING] SQL Transform
PARSEDATETIME,TO_DATE, andIS_DATEfunctions now only accept whitelisted datetime format patterns. Custom format patterns that were previously accepted will now fail at runtime. The supported patterns are:- DateTime:
yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss.SSS,yyyy-MM-dd'T'HH:mm:ss,yyyy-MM-dd'T'HH:mm:ss.SSS,yyyy/MM/dd HH:mm:ss,yyyy/MM/dd HH:mm:ss.SSS,yyyyMMddHHmmss - Date:
yyyy-MM-dd,yyyy/MM/dd,yyyyMMdd - Time:
HH:mm:ss,HH:mm:ss.SSS,HHmmss
Exception Type Change: Invalid datetime format patterns now throw
SeaTunnelRuntimeExceptioninstead ofTransformException. If you have error handling or monitoring systems that catchTransformExceptionfor datetime parsing errors, you will need to update them to handleSeaTunnelRuntimeException.Migration Guide: If you are using custom datetime format patterns in
PARSEDATETIME,TO_DATE, orIS_DATEfunctions, you must update your queries to use one of the supported patterns above. If your data uses a different format, you may need to preprocess the input data to match a supported format, or use string manipulation functions to transform the format before parsing.- DateTime:
[BREAKING] Several transform plugins now perform stricter submission-time config validation via declarative
OptionRule. Configs that previously passed submission but failed at runtime will now be rejected at submission time with a descriptiveOptionValidationException:Transform Newly Rejected Config Previous Behavior Migration DefineSinkTypecolumnsentries with null/emptycolumnortypeRuntime NPE or undefined behavior Ensure every entry has non-empty columnandtypefieldsDefineSinkTypecolumnswith duplicate column namesSilent override or runtime conflict Remove duplicate column entries FieldEncryptmax_field_lengthset to ≤ 0Ignored or unexpected truncation Set max_field_lengthto a positive integer, or remove the option to use the defaultDynamicCompilecompile_pattern = SOURCE_CODEwithout a non-blanksource_codeRuntime compilation failure Provide source_codewhen usingSOURCE_CODEpatternDynamicCompilecompile_pattern = ABSOLUTE_PATHwithout a non-blankabsolute_pathRuntime file-read failure Provide absolute_pathwhen usingABSOLUTE_PATHpatternMigration Guide: Review your transform configs against the table above. If any of your existing configs match a "Newly Rejected" pattern, update them before upgrading. The error messages at submission time now clearly identify which option is invalid and why.
[BREAKING] SQL Transform
CEIL/CEILING,FLOORandTRUNC/TRUNCATEnow return the data type of their argument, as their documentation has always specified. PreviouslyCEILandFLOORdeclaredINTandTRUNCdeclaredDOUBLEregardless of the input type, which silently produced wrong values:Expression Input Previous result Current result CEIL(bigint_col)900719925474099319007199254740993FLOOR(double_col)1.0E1821474836471.0E18TRUNC(bigint_col)9007199254740993declared DOUBLE, returned aLong9007199254740993Migration Guide: If a downstream sink column was created against the old
INT/DOUBLEoutput type, widen it to match the source column type (for exampleBIGINTforCEIL(bigint_col)), or wrap the expression in an explicitCAST(... AS INT)to keep the previous schema. Expressions overINTcolumns are unaffected.[BREAKING] SQL Transform
ROUND,TRUNC/TRUNCATEandMODno longer round-trip their arguments throughdouble, soDECIMALand largeBIGINTvalues keep full precision. For exampleROUND(CAST('12345678901234567890.987654321' AS DECIMAL(38,9)), 2)previously returned12345678901234567000.00and now returns12345678901234567890.99, andMOD(9007199254740993, 2)previously returned0and now returns1. Jobs that (intentionally or not) depended on the old lossy values will see different — now correct — output.[BREAKING] SQL Transform arithmetic on
DECIMALcolumns is now exact, and division rounds to nearest:- Operands of
+,-,*and/were previously converted withBigDecimal.valueOf(value.doubleValue()), which collapsed them to adoubleand discarded everything beyond ~17 significant digits. Values now keep full precision — for example, onDECIMAL(38,2)columns123456789012345678.99 + 0.01returns123456789012345679.00instead of123456789012345680.01. - Division now uses
RoundingMode.HALF_UPinstead ofRoundingMode.UP.UPalways rounded away from zero, so at scale 210 / 3returned3.34instead of3.33, and1 / 1000returned0.01instead of0.00. %(MOD) is unaffected; it already delegated to theMODfunction rather than converting operands itself.*now rounds its result to the scale declared for the output column (HALF_UP), the same way/already did. Exact multiplication produces a result whose scale is the sum of the operand scales, while the column is declared asDECIMAL(max(precision), max(scale)); emitting the wider value would break sinks that encode against the declared schema. OnDECIMAL(38,2)columns10.25 * 3.75returns38.44, where the old lossy conversion happened to return38.4375for these particular values.- Dividing by a zero
DECIMALnow fails with aTransformExceptionnaming the operation, where the underlying cause was previouslyjava.lang.ArithmeticException("/ by zero"). The failing expression was already reported either way, since the SQL engine wraps anything thrown while evaluating an expression; only the cause type changed. This matches howMODby zero has always been reported.
Migration Guide: Results that were previously inflated by the old rounding mode, or truncated by the
doubleconversion, will change. Multiplication results may now carry fewer decimal places than before: the old conversion sometimes emitted a value wider than the declared column scale, and that value is now rounded down to it, so a job reading38.4375from aDECIMAL(38,2)column will read38.44after upgrading. Any code that inspects the cause of a division failure and matches onArithmeticExceptionshould be updated to expectTransformException. If a downstream system was reconciled against the old values, re-baseline it after upgrading. Any workaround that compensated for the old behavior (for example subtracting a correction term after a division) should be removed.- Operands of
[BREAKING] SQL Transform
ABS, andROUND/CEIL/CEILING/FLOORwith a negative digit count, now fail with aTransformExceptionwhen the result does not fit the argument's own data type, instead of silently wrapping around to a wrong — usually negative — value:Expression Argument type Previous result Current result ABS(-2147483648)INT-2147483648TransformExceptionABS(-9223372036854775808)BIGINT-9223372036854775808TransformExceptionROUND(2147483647, -1)INT-2147483646TransformExceptionROUND(9223372036854775807, -1)BIGINT-9223372036854775806TransformExceptionCEIL(32767, -1)SMALLINT-32766TransformExceptionFLOOR(-2147483648, -1)INT2147483646TransformExceptionABShas always been documented this way — "ABS(-2147483648) should be 2147483648, but this value is not allowed for this data type. It leads to an exception" — the implementation simply never did it.TRUNC/TRUNCATEround toward zero and so can never grow a value out of its own range; they are unaffected, as areFLOAT,DOUBLEandDECIMALarguments.Migration Guide: A job that previously emitted these wrapped values now fails on the row that overflows. Cast the argument to a wider type to keep the job running —
ABS(CAST(int_col AS BIGINT))orROUND(CAST(int_col AS BIGINT), -1)— or filter the offending rows out upstream. If a downstream system was reconciled against the old wrapped values, re-baseline it after upgrading.[BREAKING] SQL Transform now dispatches
TINYINTandSMALLINTarguments correctly in the numeric functions that previously omitted them.ROUND/CEIL/CEILING/FLOOR/TRUNC/TRUNCATEhad noTINYINTbranch, so aTINYINTargument fell through the type switch and was returned unrounded, with no exception and no log line.ABSandSIGNhad noTINYINTorSMALLINTbranch and rejected those columns outright:Expression Argument type Previous result Current result ROUND(44, -1)TINYINT44, silently not rounded40CEIL(44, -1)TINYINT44, silently not rounded50ROUND(127, -1)TINYINT127, silently not roundedTransformException,130exceedsTINYINTABS(-44)TINYINTTransformException, "Unsupported arg type"44ABS(-300)SMALLINTTransformException, "Unsupported arg type"300SIGN(-44)TINYINTTransformException, "Unsupported arg type"-1The same type switch also gained a
defaultbranch, so any numeric type it does not handle now fails with aTransformExceptioninstead of being returned unrounded.SIGNon aDECIMALargument now usesBigDecimal.signum()rather than adoubleconversion, so a value smaller thanDouble.MIN_VALUEreports its true sign instead of0.Migration Guide: A job with a
TINYINTcolumn that silently skipped rounding now receives the rounded value; if a downstream system was reconciled against the old unrounded output, re-baseline it after upgrading. If a roundedTINYINTno longer fits its own type, cast the argument to a wider type —ROUND(CAST(tiny_col AS INT), -1)— or filter the offending rows out upstream. Queries that worked around theABS/SIGNrejection by casting (ABS(CAST(tiny_col AS INT))) continue to work unchanged and can be simplified at your convenience.
2.3.13
API Changes
Breaking Change: Engine REST table metrics key format
- Affected component: SeaTunnel Engine REST API (job metrics in
/job-info) - Description: To support multiple Sources/Sinks/Transforms processing the same table, the key format of table-level metrics has changed from
{tableName}to{VertexIdentifier}.{tableName}(for example,Sink[0].fake.user_table). - Impact: Existing Grafana dashboards, Prometheus alert rules, and custom monitoring integrations that reference the old keys must be updated.
Before
{
"TableSinkWriteCount": {
"fake.user_table": "15"
}
}After
{
"TableSinkWriteCount": {
"Sink[0].fake.user_table": "10",
"Sink[1].fake.user_table": "5"
}
}- Affected component: SeaTunnel Engine REST API (job metrics in
Transform Changes
- DataValidator transform: In
row_error_handle_way = ROUTE_TO_TABLEmode, the routed error rowtable_idnow includes the upstream database/schema prefix (for example,db1.ffp/db1.schema1.ffpinstead offfp). - Adjusted SQL Transform date & time functions:
DATEDIFF(<start>, <end>, 'MONTH')now returns the total number of months between the two dates across years (for example, from2023-01-01to2024-03-01returns14instead of15).WEEK(<datetime>)now returns the ISO week number directly (previous behavior added an extra+1to the ISO week value).