跳到主要内容
版本：2.3.10

Maxcompute

Maxcompute 源连接器

描述​

用于从 Maxcompute 读取数据.

关键特性​

选项​

名称类型必需默认值
accessIdstring是-
accesskeystring是-
endpointstring是-
projectstring是-
table_namestring是-
partition_specstring否-
split_rowint否10000
read_columnsArray否-
table_listArray否-
common-optionsstring否
schemaconfig否

accessId [string]​

accessId 您的 Maxcompute 密钥 Id, 可以从阿里云访问哪个云.

accesskey [string]​

accesskey Your Maxcompute 密钥, 可以从阿里云访问哪个云.

endpoint [string]​

endpoint 您的 Maxcompute 端点以 http 开头.

project [string]​

project 您在阿里云中创建的Maxcompute项目.

table_name [string]​

table_name 目标Maxcompute表名，例如：fake.

partition_spec [string]​

partition_spec Maxcompute分区表的此规范，例如:ds='20220101'.

split_row [int]​

split_row 每次拆分的行数，默认值: 10000.

read_columns [Array]​

read_columns 要读取的列，如果未设置，则将读取所有列。例如. ["col1", "col2"]

table_list [Array]​

要读取的表列表，您可以使用此配置代替 table_name.

common options​

源插件常用参数, 详见 源通用选项 .

示例​

表读取​

source {
  Maxcompute {
    accessId="<your access id>"
    accesskey="<your access Key>"
    endpoint="<http://service.odps.aliyun.com/api>"
    project="<your project>"
    table_name="<your table name>"
    #partition_spec="<your partition spec>"
    #split_row = 10000
    #read_columns = ["col1", "col2"]
  }
}

使用表列表读取​

source {
  Maxcompute {
    accessId="<your access id>"
    accesskey="<your access Key>"
    endpoint="<http://service.odps.aliyun.com/api>"
    project="<your project>" # default project
    table_list = [
      {
        table_name = "test_table"
        #partition_spec="<your partition spec>"
        #split_row = 10000
        #read_columns = ["col1", "col2"]
      },
      {
        project = "test_project"
        table_name = "test_table2"
        #partition_spec="<your partition spec>"
        #split_row = 10000
        #read_columns = ["col1", "col2"]
      }
    ]
  }
}

变更日志​

Change Log
ChangeCommitVersion
[Fix][Connector-V2] Fix maxcompute read with partition spec (#8896)https://github.com/apache/seatunnel/commit/e62bf6c652.3.10
[Fix][Connector-V2] Fix MaxCompute cannot get project and tableName when use schema (#8865)https://github.com/apache/seatunnel/commit/a24fa8fef2.3.10
[Improve] restruct connector common options (#8634)https://github.com/apache/seatunnel/commit/f3499a6ee2.3.10
[Feature][Connector-V2] Support maxcompute source with multi-table (#8582)https://github.com/apache/seatunnel/commit/0f78242922.3.10
[Fix][Connector-V2] Fixed adding table comments (#8514)https://github.com/apache/seatunnel/commit/edca75b0d2.3.10
[Improve][Connector-V2] MaxComputeSink support create partition in savemode (#8474)https://github.com/apache/seatunnel/commit/0b8f9de462.3.10
[Improve][Transform] Rename sql transform table name from 'fake' to 'dual' (#8298)https://github.com/apache/seatunnel/commit/e6169684f2.3.9
[Feature][Connector-V2] Support MaxCompute save mode (#8277)https://github.com/apache/seatunnel/commit/44ea675f12.3.9
[Improve][dist]add shade check rule (#8136)https://github.com/apache/seatunnel/commit/51ef800012.3.9
[Feature][Core] Rename result_table_name/source_table_name to plugin_input/plugin_output (#8072)https://github.com/apache/seatunnel/commit/c7bbd322d2.3.9
[Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786)https://github.com/apache/seatunnel/commit/6b7c53d032.3.9
[Fix] Fix dead link on seatunnel connectors list url (#7453)https://github.com/apache/seatunnel/commit/62b4f16f42.3.8
[BugFix][Connector-V2][Maxcompute]fix:Maxcompute sink can't map field(#7164) (#7168)https://github.com/apache/seatunnel/commit/d5abf8f502.3.6
[Feature] Add unsupported datatype check for all catalog (#5890)https://github.com/apache/seatunnel/commit/b9791285a2.3.4
FakeSource support generate different CatalogTable for MultipleTable (#5766)https://github.com/apache/seatunnel/commit/a8b93805e2.3.4
[Improve][Common] Introduce new error define rule (#5793)https://github.com/apache/seatunnel/commit/9d1b2582b2.3.4
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755)https://github.com/apache/seatunnel/commit/8de7408102.3.4
[Improve][Connector] Add field name to DataTypeConvertor to improve error message (#5782)https://github.com/apache/seatunnel/commit/ab60790f02.3.4
[Improve][Test] Move MaxCompute test case file (#5786)https://github.com/apache/seatunnel/commit/38132f5152.3.4
[Fix] Fix MaxCompute use not exist SCHEMA option (#5708)https://github.com/apache/seatunnel/commit/ba4782a672.3.4
[Feature] Support catalog in MaxCompute Source (#5283)https://github.com/apache/seatunnel/commit/946d89cb92.3.4
[Bugfix][Connector-V2][maxcompute] sink commit with Block not exsits on server (#4725)https://github.com/apache/seatunnel/commit/2760cae732.3.2
[Bug][Maxcompute] Fix failed to parse some maxcompute type (#3894)https://github.com/apache/seatunnel/commit/642901f0a2.3.1
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd601052.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab166562.3.1
[Feature][Connector] add get source method to all source connector (#3846)https://github.com/apache/seatunnel/commit/417178fb82.3.1
[Feature][API &amp; Connector &amp; Doc] add parallelism and column projection interface (#3829)https://github.com/apache/seatunnel/commit/b9164b8ba2.3.1
[Feature][Connector-V2][Maxcompute] Add Maxcompute source & sink connector (#3640)https://github.com/apache/seatunnel/commit/80cf8f4e42.3.0