Skip to main content
Version: 2.1.0

Common Options

Transform Common Options [Spark]

Transform Plugin common parameters​

nametyperequireddefault value
source_table_namestringno-
result_table_namestringno-

source_table_name [string]​

When source_table_name is not specified, the current plug-in processes the data set (dataset) output by the previous plug-in in the configuration file;

When source_table_name is specified, the current plugin is processing the data set corresponding to this parameter.

result_table_name [string]​

When result_table_name is not specified, the data processed by this plugin will not be registered as a data set that can be directly accessed by other plugins, or called a temporary table (table);

When result_table_name is specified, the data processed by this plugin will be registered as a data set (dataset) that can be directly accessed by other plugins, or called a temporary table (table) . The dataset registered here can be directly accessed by other plugins by specifying source_table_name .

Examples​

split {
source_table_name = "source_view_table"
source_field = "message"
delimiter = "&"
fields = ["field1", "field2"]
result_table_name = "result_view_table"
}

The Split plugin will process the data in the temporary table source_view_table and register the processing result as a temporary table named result_view_table. This temporary table can be used by any subsequent Filter or Output plugin by specifying source_table_name .

split {
source_field = "message"
delimiter = "&"
fields = ["field1", "field2"]
}

If source_table_name is not configured, output the processing result of the last Transform plugin in the configuration file