Set Up with Locally

Prepare

Before you getting start the local run, you need to make sure you already have installed the following software which SeaTunnel required:

• Java (only JDK 8 supported by now) installed and JAVA_HOME set.
• Download the engine, you can choose and download one of them from below as your favour, you could see more information about why we need engine in SeaTunnel
  • Spark: Please download Spark first(required version >= 2 and version < 3.x ). For more information you could see Getting Started: standalone
  • Flink: Please download Flink first(required version >= 1.12.0 and version < version < 1.14.x ). For more information you could see Getting Started: standalone

Installation

Enter the seatunnel download page and download the latest version of distribute package seatunnel-<version>-bin.tar.gz

Or you can download it by terminal

export version="2.1.0"
wget "https://archive.apache.org/dist/incubator/seatunnel/${version}/apache-seatunnel-incubating-${version}-bin.tar.gz"
tar -xzvf "apache-seatunnel-incubating-${version}-bin.tar.gz"

Run SeaTunnel Application

Configure SeaTunnel: Change the setting in config/seatunnel-env.sh, it is base on the path your engine install at prepare step two. Change SPARK_HOME if you using Spark as your engine, or change FLINK_HOME if you're using Flink.

Run Application with Build-in Configure: We already providers and out-of-box configuration in directory config which you could find when you extract the tarball. You could start the application by the following commands

Spark

cd "apache-seatunnel-incubating-${version}"
./bin/start-seatunnel-spark.sh \
--master local[4] \
--deploy-mode client \
--config ./config/spark.streaming.conf.template

Flink

cd "apache-seatunnel-incubating-${version}"
./bin/start-seatunnel-flink.sh \
--config ./config/flink.streaming.conf.template

See The Output: When you run the command, you could see its output in your console or in Flink UI, You can think this is a sign that the command ran successfully or not.

Spark

The SeaTunnel console will prints some logs as below:

Hello World, SeaTunnel
Hello World, SeaTunnel
Hello World, SeaTunnel
...
Hello World, SeaTunnel

Flink

The content printed in the TaskManager Stdout log of flink WebUI, is two columned record just like below(your content maybe different cause we use fake source to create data random):

apache, 15
seatunnel, 30
incubator, 20
...
topLevel, 20

Explore More Build-in Examples

Our local quick start is using one of the build-in example in directory config, and we provider more than one out-of-box example you could and feel free to have a try and make your hands dirty. All you have to do is change the started command option value in running application to the configuration you want to run, we use batch template in config as examples:

Spark

cd "apache-seatunnel-incubating-${version}"
./bin/start-seatunnel-spark.sh \
--master local[4] \
--deploy-mode client \
--config ./config/spark.batch.conf.template

Flink

cd "apache-seatunnel-incubating-${version}"
./bin/start-seatunnel-flink.sh \
--config ./config/flink.batch.conf.template

What's More

For now, you are already take a quick look about SeaTunnel, you could see connector to find all source and sink SeaTunnel supported. Or see deployment if you want to submit your application in other kind of your engine cluster.