Skip to main content
Version: 2.1.0

Quick start

Let's take an application that receives data through a socket , divides the data into multiple fields, and outputs the processing results as an example to quickly show how to use seatunnel .

If you are familiar with Flink or have prepared the Flink operating environment, you can ignore this step. Flink does not require any special configuration.

Please download Flink first, please choose Flink version >= 1.12.0 and version < version < 1.14.x . The download is complete to install Flink

Step 2: Download seatunnel​

Enter the seatunnel installation package download page and download the latest version of seatunnel-<version>.zip

Or download the specified version directly (take 2.0.4 as an example):

wget https://github.com/apache/incubator-seatunnel/releases/download/v2.0.4/waterdrop-dist-2.0.4-2.11.8-release.zip -O seatunnel-2.0.4.zip

After downloading, unzip:

unzip seatunnel-<version>.zip
ln -s seatunnel-<version> seatunnel

Step 3: Configure seatunnel​

  • Edit config/seatunnel-env.sh , specify the necessary environment configuration such as FLINK_HOME (the directory after Flink downloaded and decompressed in Step 1)

  • Edit config/application.conf , it determines the way and logic of data input, processing, and output after seatunnel is started.

env {
  # You can set flink configuration here
  execution.parallelism = 1
  #execution.checkpoint.interval = 10000
  #execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}

source {
    SocketStream{
          result_table_name = "fake"
          field_name = "info"
    }
}

transform {
  Split{
    separator = "#"
    fields = ["name","age"]
  }
  sql {
    sql = "select * from (select info,split(info) as info_row from fake) t1"
  }
}

sink {
  ConsoleSink {}
}

Step 4: Start the netcat server to send data​

nc -l -p 9999

Step 5: start seatunnel​

cd seatunnel
./bin/start-seatunnel-flink.sh \
--config ./config/application.conf

Step 6: Input at the nc terminal​

xg#1995

It is printed in the TaskManager Stdout log of flink WebUI:

xg#1995,xg,1995

Summary​

If you want to know more seatunnel configuration examples, please refer to:

The above configuration is the default [streaming configuration template] , which can be run directly, the command is as follows:

cd seatunnel
./bin/start-seatunnel-flink.sh \
--config ./config/flink.streaming.conf.template

The above configuration is the default [offline batch configuration template] , which can be run directly, the command is as follows:

cd seatunnel
./bin/start-seatunnel-flink.sh \
--config ./config/flink.batch.conf.template