Deployment

This section will show you how to submit your SeaTunnel application in all kinds of cluster engine. If you still not installation SeaTunnel you could go to see quick start about how to prepare and change SeaTunnel configuration firstly.

Deployment in All Kind of Engine

Local Mode(Spark Only)

Local mode only support Spark engine for now.

./bin/start-seatunnel-spark.sh \
--master local[4] \
--deploy-mode client \
--config ./config/application.conf

Standalone Cluster

Spark

# client mode
./bin/start-seatunnel-spark.sh \
--master spark://ip:7077 \
--deploy-mode client \
--config ./config/application.conf

# cluster mode
./bin/start-seatunnel-spark.sh \
--master spark://ip:7077 \
--deploy-mode cluster \
--config ./config/application.conf

Flink

bin/start-seatunnel-flink.sh \
--config config-path

# -p 2 specifies that the parallelism of flink job is 2. You can also specify more parameters, use flink run -h to view
bin/start-seatunnel-flink.sh \
-p 2 \
--config config-path

Yarn Cluster

Spark

# client mode
./bin/start-seatunnel-spark.sh \
--master yarn \
--deploy-mode client \
--config ./config/application.conf

# cluster mode
./bin/start-seatunnel-spark.sh \
--master yarn \
--deploy-mode cluster \
--config ./config/application.conf

Flink

bin/start-seatunnel-flink.sh \
-m yarn-cluster \
--config config-path

# -ynm seatunnel specifies the name displayed in the yarn webUI as seatunnel, you can also specify more parameters, use flink run -h to view
bin/start-seatunnel-flink.sh \
-m yarn-cluster \
-ynm seatunnel \
--config config-path

Mesos Cluster(Spark Only)

Mesos cluster deployment only support Spark engine for now.

# cluster mode
./bin/start-seatunnel-spark.sh \
--master mesos://ip:7077 \
--deploy-mode cluster \
--config ./config/application.conf

Run Your Engine in Scaling

(This section is about engine instead of SeaTunnel itself, it is background knowledge for user who not understand engine cluster type). Both Spark and Flink could be run in different kind of cluster and any scale. This guide only show the basic usage of SeaTunnel which build above engine Spark or Flink, if you want to scale your engine cluster see Spark or Flink document.