跳到主要内容
版本:2.3.10

Http

Http 数据接收器

支持引擎

Spark
Flink
SeaTunnel Zeta

主要特性

描述

接收Source端传入的数据,利用数据触发 web hooks。

例如,来自上游的数据为[age: 12, name: tyrantlucifer],则body内容如下:{"age": 12, "name": "tyrantlucifer"}

Tips: Http 接收器仅支持 post json 类型的 web hook,source 数据将被视为 webhook 中的 body 内容。

支持的数据源信息

想使用 Http 连接器,需要安装以下必要的依赖。可以通过运行 install-plugin.sh 脚本或者从 Maven 中央仓库下载这些依赖

数据源支持版本依赖
Http通用下载

接收器选项

名称类型是否必须默认值描述
urlString-Http 请求链接
headersMap-Http 标头
retryInt-如果请求http返回IOException的最大重试次数
retry_backoff_multiplier_msInt100http请求失败,重试回退次数(毫秒)乘数
retry_backoff_max_msInt10000http请求失败,最大重试回退时间(毫秒)
connect_timeout_msInt12000连接超时设置,默认12s
socket_timeout_msInt60000套接字超时设置,默认为60s
common-options-Sink插件常用参数,请参考 Sink常用选项 了解详情

示例

简单示例:

Http {
url = "http://localhost/test/webhook"
headers {
token = "9e32e859ef044462a257e1fc76730066"
}
}

变更日志

Change Log
ChangeCommitVersion
[improve] http connector options (#8969)https://github.com/apache/seatunnel/commit/63ff9f9102.3.10
[Fix][connector-http] fix when post have param (#8434)https://github.com/apache/seatunnel/commit/c1b2675ab2.3.10
[Improve] restruct connector common options (#8634)https://github.com/apache/seatunnel/commit/f3499a6ee2.3.10
[Improve][dist]add shade check rule (#8136)https://github.com/apache/seatunnel/commit/51ef800012.3.9
[Feature][Connector-V2] Add prometheus source and sink (#7265)https://github.com/apache/seatunnel/commit/dde6f9fcb2.3.9
[Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786)https://github.com/apache/seatunnel/commit/6b7c53d032.3.9
[Fix][Connector-V2] Fix http source can not read streaming (#7703)https://github.com/apache/seatunnel/commit/a0ffa7ba02.3.8
[Feature][Connector-V2] Suport choose the start page in http paging (#7180)https://github.com/apache/seatunnel/commit/ed15f0dcf2.3.8
[Improve][Connector] Add multi-table sink option check (#7360)https://github.com/apache/seatunnel/commit/2489f64462.3.7
[Improve][API] Make sure the table name in TablePath not be null (#7252)https://github.com/apache/seatunnel/commit/764d8b0bc2.3.7
[Feature][Core] Support using upstream table placeholders in sink options and auto replacement (#7131)https://github.com/apache/seatunnel/commit/c4ca741222.3.6
[Feature][Kafka] Support multi-table source read (#5992)https://github.com/apache/seatunnel/commit/60104602d2.3.6
[Improve][CDC] Close idle subtasks gorup(reader/writer) in increment phase (#6526)https://github.com/apache/seatunnel/commit/454c339b92.3.6
Fix HttpSource bug (#6824)https://github.com/apache/seatunnel/commit/c3ab84caa2.3.6
[Hotfix] fix http source can not read yyyy-MM-dd HH:mm:ss format bug & Improve DateTime Utils (#6601)https://github.com/apache/seatunnel/commit/19888e7962.3.5
[Improve][Connector-V2]Support multi-table sink feature for httpsink (#6316)https://github.com/apache/seatunnel/commit/e6c51a95c2.3.5
[Improve][HttpConnector]Increase custom configuration timeout. (#6223)https://github.com/apache/seatunnel/commit/fa5b7d3d82.3.4
[Feature][Core] Upgrade flink source translation (#5100)https://github.com/apache/seatunnel/commit/5aabb14a92.3.4
[BUG][Connector-V2][Http] fix bug http config no schema option and improve e2e test add case (#5939)https://github.com/apache/seatunnel/commit/8a71b9e072.3.4
[Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on redis (#5901)https://github.com/apache/seatunnel/commit/e84dcb8c12.3.4
[Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on http (#5816)https://github.com/apache/seatunnel/commit/6f49ec6ea2.3.4
[Improve][Common] Introduce new error define rule (#5793)https://github.com/apache/seatunnel/commit/9d1b2582b2.3.4
[Feature][Transform] add JsonPath transform (#5632)https://github.com/apache/seatunnel/commit/d908f0af42.3.4
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755)https://github.com/apache/seatunnel/commit/8de7408102.3.4
[Feature][Connector-V2] HTTP supports page increase #5477 (#5561)https://github.com/apache/seatunnel/commit/bb180b2982.3.4
[improve][Connector-V2][http] improve http e2e test (#5655)https://github.com/apache/seatunnel/commit/f5867adca2.3.4
Support config column/primaryKey/constraintKey in schema (#5564)https://github.com/apache/seatunnel/commit/eac76b4e52.3.4
[BUG][Connector-V2][http] fix httpheader cover (#5446)https://github.com/apache/seatunnel/commit/cdd8e0a652.3.4
[Feature][Connector][Http] Support multi-line text splits (#4698)https://github.com/apache/seatunnel/commit/6a524981c2.3.2
Merge branch 'dev' into merge/cdchttps://github.com/apache/seatunnel/commit/4324ee1912.3.1
[Improve][Project] Code format with spotless plugin.https://github.com/apache/seatunnel/commit/423b583032.3.1
[Feature][Connector-V2][Github] Adding Github Source Connector (#4155)https://github.com/apache/seatunnel/commit/49d9172b12.3.1
[improve][api] Refactoring schema parse (#4157)https://github.com/apache/seatunnel/commit/b2f573a132.3.1
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd601052.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab166562.3.1
[Feature][Connector-V2][Persistiq]Add Persistiq source connector (#3460)https://github.com/apache/seatunnel/commit/aec3912ed2.3.1
[Feature][Connector] add get source method to all source connector (#3846)https://github.com/apache/seatunnel/commit/417178fb82.3.1
[Feature][Connector-V2][Notion] Add Notion source connector (#3470)https://github.com/apache/seatunnel/commit/46abc6d942.3.0
[Hotfix][seatunnel-connectors-v2] [connector-http] fix http json request error (#3629)https://github.com/apache/seatunnel/commit/54f594d6c2.3.0
[Improve][Connector-V2][Http]Improve json parse option rule for all http connector (#3627)https://github.com/apache/seatunnel/commit/589e4161e2.3.0
[Improve][Connector-V2][OneSignal]Unified exception for OneSignal connector (#3609)https://github.com/apache/seatunnel/commit/97cce8c252.3.0
[Feature][Connector-V2][HTTP] Use json-path parsing (#3510)https://github.com/apache/seatunnel/commit/1807eb6c92.3.0
[Improve][Connector-V2][Http]Unified exception for http source & sink… (#3594)https://github.com/apache/seatunnel/commit/d798cd8672.3.0
[Hotfix][OptionRule] Fix option rule about all connectors (#3592)https://github.com/apache/seatunnel/commit/226dc6a112.3.0
[Improve][Connector-V2][MyHours]Unified exception for MyHours connector (#3538)https://github.com/apache/seatunnel/commit/48ab7c97d2.3.0
[Improve][Connector-V2][Gitlab] Unified excetion for Gitlab connector and improve optione rule (#3533)https://github.com/apache/seatunnel/commit/77f68f1ee2.3.0
[Improve][Connector-V2][Klaviyo]Unified exception for Klaviyo connector (#3555)https://github.com/apache/seatunnel/commit/08f8615072.3.0
[Feature][Connector-V2][Jira]Add Jira source connector (#3473)https://github.com/apache/seatunnel/commit/fb40162c02.3.0
[Improve][Connector-V2][Lemlist] Unified exception for lemlist connector (#3534)https://github.com/apache/seatunnel/commit/705728ebb2.3.0
[Feature][Connector V2] add gitlab source connector (#3408)https://github.com/apache/seatunnel/commit/545595c6d2.3.0
[Feature][Connector-V2][OneSignal]Add OneSignal source conector (#3454)https://github.com/apache/seatunnel/commit/b318b31662.3.0
[Feature][Connector-V2][Klaviyo]Add Klaviyo source connector (#3443)https://github.com/apache/seatunnel/commit/fc00a28662.3.0
[Feature][Connector-V2][Lemlist]Add Lemlist source connector (#3346)https://github.com/apache/seatunnel/commit/12d66b4242.3.0
[HotFix][Core][API] Fix OptionValidation error code (#3439)https://github.com/apache/seatunnel/commit/ace219f372.3.0
[Improve][Connector-V2][My Hours]Add http method enum && Improve My Hours connector option rule (#3390)https://github.com/apache/seatunnel/commit/a86c9d90f2.3.0
[Feature][Connector-V2][Http] Add option rules && Improve Myhours sink connector (#3351)https://github.com/apache/seatunnel/commit/cc8bb60c82.3.0
[Feature][Connector-V2][My Hours] Add My Hours Source Connector (#3228)https://github.com/apache/seatunnel/commit/4104a3e302.3.0
[Improve][all] change Log to @Slf4j (#3001)https://github.com/apache/seatunnel/commit/6016100f12.3.0-beta
[Bug][format][json] Fix jackson package conflict with spark (#2934)https://github.com/apache/seatunnel/commit/1a92b83692.3.0-beta
[Bug][Connector-V2] Fix wechat sink data serialization (#2856)https://github.com/apache/seatunnel/commit/3aee11fc12.3.0-beta
[Improve][Connector-V2] Improve http connector (#2833)https://github.com/apache/seatunnel/commit/5b3957bc52.2.0-beta
[DEV][Api] Replace SeaTunnelContext with JobContext and remove singleton pattern (#2706)https://github.com/apache/seatunnel/commit/cbf82f7552.2.0-beta
[Improve][build] Improved scope of maven-shade-plugin (#2665)https://github.com/apache/seatunnel/commit/93bc8bd112.2.0-beta
[#2606]Dependency management split (#2630)https://github.com/apache/seatunnel/commit/fc047be692.2.0-beta
[chore][connector-common] Rename SeatunnelSchema to SeaTunnelSchema (#2538)https://github.com/apache/seatunnel/commit/7dc2a27382.2.0-beta
[Bug][Connector-V2] Fix the bug that set params by mistake (#2511) (#2513)https://github.com/apache/seatunnel/commit/ead3d68b02.2.0-beta
[Improve][Connector-V2] Http source support user-defined schema (#2439)https://github.com/apache/seatunnel/commit/793933b6b2.2.0-beta
[Feature][Connector-V2] Add Enterprise Wechat sink connector (#2412)https://github.com/apache/seatunnel/commit/3e200e0a32.2.0-beta
[Improve][Connector-V2] Format SeaTunnelRow use seatunnel-format-json (#2435)https://github.com/apache/seatunnel/commit/e4e8f7fbf2.2.0-beta
[Improve][Connector-V2] Make the attribute of http-connector from private to protected (#2418)https://github.com/apache/seatunnel/commit/f3b00ef692.2.0-beta
[Feature][Connector-V2] Add feishu sink (#2381)https://github.com/apache/seatunnel/commit/0fec8ca432.2.0-beta
[Feature][Connector-V2] Add http sink(Webhook) (#2348)https://github.com/apache/seatunnel/commit/4b72074902.2.0-beta
[Improve][Http Connector-V2-Source] Refactor the code and make code more clearly (#2322)https://github.com/apache/seatunnel/commit/a9a797ad82.2.0-beta
[Improve][Connector-V2] Fix the log information (#2317)https://github.com/apache/seatunnel/commit/736983a702.2.0-beta
[Improve][Connector-V2] Http client provider improve (#2312)https://github.com/apache/seatunnel/commit/cc950007c2.2.0-beta
[Improve][Connector-V2] Fix 'Singleton' word error (#2309)https://github.com/apache/seatunnel/commit/12ebcb4a02.2.0-beta
[api-draft][Optimize] Optimize module name (#2062)https://github.com/apache/seatunnel/commit/f79e3112b2.2.0-beta