FilterRowKind

FilterRowKind transform plugin

Description

Filter the data by RowKind

Options

name	type	required	default value
include_kinds	array	no
exclude_kinds	array	no

Exactly one of include_kinds and exclude_kinds must be configured; configuring both or neither fails validation.

include_kinds [array]

The row kinds to include

exclude_kinds [array]

The row kinds to exclude.

You can only config one of include_kinds and exclude_kinds.

common options [string]

Transform plugin common parameters, please refer to Transform Plugin for details

Examples

The RowKind of the data generated by FakeSource is INSERT. If we use the FilterRowKind transform and exclude the INSERT data, zero rows will be written into the sink.

env {
  job.mode = "BATCH"
}

source {
  FakeSource {
    plugin_output = "fake"
    row.num = 100
    schema = {
      fields {
        id = "int"
        name = "string"
        age = "int"
      }
    }
  }
}

transform {
  FilterRowKind {
    plugin_input = "fake"
    plugin_output = "fake1"
    exclude_kinds = ["INSERT"]
  }
}

sink {
  Console {
    plugin_input = "fake1"
  }
}