Skip to main content
Version: 2.3.0-beta

Http

Http source connector

Description​

Used to read data from Http.

Key features​

Options​

nametyperequireddefault value
urlStringYes-
schemaConfigNo-
schema.fieldsConfigNo-
formatStringNojson
methodStringNoget
headersMapNo-
paramsMapNo-
bodyStringNo-
poll_interval_msintNo-
retryintNo-
retry_backoff_multiplier_msintNo100
retry_backoff_max_msintNo10000
common-optionsNo-

url [String]​

http request url

method [String]​

http request method, only supports GET, POST method.

headers [Map]​

http headers

params [Map]​

http params

body [String]​

http body

poll_interval_ms [int]​

request http api interval(millis) in stream mode

retry [int]​

The max retry times if request http return to IOException

retry_backoff_multiplier_ms [int]​

The retry-backoff times(millis) multiplier if request http failed

retry_backoff_max_ms [int]​

The maximum retry-backoff times(millis) if request http failed

format [String]​

the format of upstream data, now only support json text, default json.

when you assign format is json, you should also assign schema option, for example:

upstream data is the following:


{"code": 200, "data": "get success", "success": true}

you should assign schema as the following:


schema {
fields {
code = int
data = string
success = boolean
}
}

connector will generate data as the following:

codedatasuccess
200get successtrue

when you assign format is text, connector will do nothing for upstream data, for example:

upstream data is the following:


{"code": 200, "data": "get success", "success": true}

connector will generate data as the following:

content
{"code": 200, "data": "get success", "success": true}

schema [Config]​

fields [Config]​

the schema fields of upstream data

common options​

Source plugin common parameters, please refer to Source Common Options for details

Example​

simple:

Http {
url = "https://tyrantlucifer.com/api/getDemoData"
schema {
fields {
code = int
message = string
data = string
ok = boolean
}
}
}

Changelog​

2.2.0-beta 2022-09-26​

  • Add Http Source Connector