跳到主要内容
版本:Next

My Hours

My Hours 源连接器

支持这些引擎​

Spark
Flink
SeaTunnel Zeta

关键特性​

描述​

用于通过 My Hours REST API 读取数据。连接器会先使用配置的 email 和 password 登录获取访问令牌, 然后在后续请求中自动携带该令牌。

支持的数据源信息​

为了使用 My Hours 连接器,需要以下依赖项。 可以通过 install-plugin.sh 或从 Maven 中央存储库下载。

数据源支持的版本依赖
My Hoursuniversal下载

源选项​

参数名类型必须默认值描述
urlString是-My Hours API 请求 URL。
emailString是-My Hours 登录邮箱。
passwordString是-My Hours 登录密码。
schemaConfig否-当 format 为 json 时需要配置。更多详情请参考 Schema 特性。
schema.fieldsConfig否-上游数据字段。
json_fieldConfig否-通过 JSONPath 从响应中抽取字段,需要与 schema 一起使用。
content_fieldString否-在解析 schema 前抽取 JSON 响应中的一部分,例如 $.store.book.*。
formatString否text响应格式,支持 json 和 text。使用 schema、json_field 或 content_field 时请设置为 json。
methodString否GETHTTP 请求方法,支持 GET 和 POST。
headersMap否-额外 HTTP 请求头。连接器登录后会自动添加 My Hours Authorization 请求头。
paramsMap否-HTTP 查询参数。
bodyString否-HTTP 请求体。
poll_interval_millisInt否-流模式下请求 HTTP API 的间隔,单位毫秒。
retryInt否-请求抛出 IOException 时的最大重试次数。
retry_backoff_multiplier_msInt否100重试退避倍数,单位毫秒。
retry_backoff_max_msInt否10000最大重试退避时间,单位毫秒。
json_filed_missed_return_nullBoolean否false配置的 JSON 字段缺失时返回 null。
common-options否-源插件通用参数。

如何创建 My Hours 数据同步作业​

env {
parallelism = 1
job.mode = "BATCH"
}

source {
MyHours {
url = "https://api2.myhours.com/api/Projects/getAll"
email = "seatunnel@test.com"
password = "********"
method = "GET"
format = "json"
schema {
fields {
name = string
archived = boolean
dateArchived = string
dateCreated = string
clientName = string
budgetAlertPercent = string
budgetType = int
totalTimeLogged = double
budgetValue = double
totalAmount = double
totalExpense = double
laborCost = double
totalCost = double
billableTimeLogged = double
totalBillableAmount = double
billable = boolean
roundType = int
roundInterval = int
budgetSpentPercentage = double
budgetTarget = int
budgetPeriodType = string
budgetSpent = string
id = string
}
}
}
}

# 控制台打印读取的数据
sink {
Console {
parallelism = 1
}
}

参数解释​

format​

当您指定格式为 json 时,您还应该指定 schema 选项,例如:

上游数据如下:

{
"code": 200,
"data": "get success",
"success": true
}

您应该按如下方式指定 schema:


schema {
fields {
code = int
data = string
success = boolean
}
}

连接器将生成如下数据:

codedatasuccess
200get successtrue

当您指定格式为 text 时,连接器不会对上游数据做任何处理,例如:

上游数据如下:

{
"code": 200,
"data": "get success",
"success": true
}

连接器将生成如下数据:

content
{"code": 200, "data": "get success", "success": true}

content_field​

此参数可以获取响应中的一部分 JSON 数据。如果只需要 book 部分,可配置 content_field = "$.store.book.*"。

如果您的返回数据类似于这样:

{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}

您可以配置 content_field = "$.store.book.*",返回的结果如下所示:

[
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
]

然后您就可以使用更简单的 schema 获得所需的结果,例如:

MyHours {
url = "http://mockserver:1080/contentjson/mock"
email = "seatunnel@test.com"
password = "********"
method = "GET"
format = "json"
content_field = "$.store.book.*"
schema = {
fields {
category = string
author = string
title = string
price = string
}
}
}

json_field​

此参数帮助您配置模式,因此此参数必须与 schema 一起使用。

如果您的数据类似于这样:

{ 
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}

您可以按如下方式配置任务,获取 book 的内容:

source {
MyHours {
url = "http://mockserver:1080/jsonpath/mock"
email = "seatunnel@test.com"
password = "********"
method = "GET"
format = "json"
json_field = {
category = "$.store.book[*].category"
author = "$.store.book[*].author"
title = "$.store.book[*].title"
price = "$.store.book[*].price"
}
schema = {
fields {
category = string
author = string
title = string
price = string
}
}
}
}

变更日志​

Change Log
ChangeCommitVersion
[Feature][connector-http] Parameters support placeholder replacement (#9184)https://github.com/apache/seatunnel/commit/8617014edc2.3.11
[improve] http connector options (#8969)https://github.com/apache/seatunnel/commit/63ff9f910a2.3.10
[Fix][connector-http] fix when post have param (#8434)https://github.com/apache/seatunnel/commit/c1b2675ab02.3.10
[Improve][dist]add shade check rule (#8136)https://github.com/apache/seatunnel/commit/51ef8000162.3.9
[Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on http (#5816)https://github.com/apache/seatunnel/commit/6f49ec6ead2.3.4
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd6010512.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab1665612.3.1
[Improve][Connector-V2][Http]Improve json parse option rule for all http connector (#3627)https://github.com/apache/seatunnel/commit/589e4161ec2.3.0
[Feature][Connector-V2][HTTP] Use json-path parsing (#3510)https://github.com/apache/seatunnel/commit/1807eb6c952.3.0
[Hotfix][OptionRule] Fix option rule about all connectors (#3592)https://github.com/apache/seatunnel/commit/226dc6a1192.3.0
[Improve][Connector-V2][MyHours]Unified exception for MyHours connector (#3538)https://github.com/apache/seatunnel/commit/48ab7c97d52.3.0
[HotFix][Core][API] Fix OptionValidation error code (#3439)https://github.com/apache/seatunnel/commit/ace219f3762.3.0
[Improve][Connector-V2][My Hours]Add http method enum && Improve My Hours connector option rule (#3390)https://github.com/apache/seatunnel/commit/a86c9d90f72.3.0
[Feature][Connector-V2][Http] Add option rules && Improve Myhours sink connector (#3351)https://github.com/apache/seatunnel/commit/cc8bb60c832.3.0
[Feature][Connector-V2][My Hours] Add My Hours Source Connector (#3228)https://github.com/apache/seatunnel/commit/4104a3e30e2.3.0