跳到主要内容
版本:2.3.10

connector-file-base

Change Log
ChangeCommitVersion
Revert " [improve] update localfile connector config" (#9018)https://github.com/apache/seatunnel/commit/cdc79e13a2.3.10
[improve] update localfile connector config (#8765)https://github.com/apache/seatunnel/commit/def369a852.3.10
[Fix][File]use common-csv to read csv file (#8919)https://github.com/apache/seatunnel/commit/3e64a42832.3.10
[Improve][connector-file-base] Improved multiple table file source allocation algorithm for subtasks (#8878)https://github.com/apache/seatunnel/commit/44a12cc552.3.10
[Fix][Connector-File] Fix conflicting file_format_type requirement (#8823)https://github.com/apache/seatunnel/commit/6e0d630f72.3.10
[Feature][Connector-V2] Add filename_extension parameter for read/write file (#8769)https://github.com/apache/seatunnel/commit/78b23c0ef2.3.10
[Improve][Connector-V2] Improve orc read error message (#8751)https://github.com/apache/seatunnel/commit/d66d9dc9c2.3.10
[Improve] restruct connector common options (#8634)https://github.com/apache/seatunnel/commit/f3499a6ee2.3.10
[Fix][Connector-V2] User selects csv string pattern (#8572)https://github.com/apache/seatunnel/commit/227a11f5a2.3.10
[Fix][Connector-V2] Fix CSV String type write type (#8499)https://github.com/apache/seatunnel/commit/9268f5a252.3.10
[Fix][File] Fix Multi-file with binary format synchronization failed (#8546)https://github.com/apache/seatunnel/commit/6e4ee468a2.3.10
[Feature][Connector-V2] Support create emtpy file when no data (#8543)https://github.com/apache/seatunnel/commit/275db78912.3.10
[Feature][Connector-V2] Support single file mode in file sink (#8518)https://github.com/apache/seatunnel/commit/e893deed52.3.10
[Improve][Connector-file-base] Improved file allocation algorithm for subtasks. (#8453)https://github.com/apache/seatunnel/commit/d61cba2332.3.9
[Bug][connector-file] When the data source field is less than the target (Hive) field,it will throw null pointer exception#8150 (#8200)https://github.com/apache/seatunnel/commit/25b8a02b72.3.9
[Fix] Set all snappy dependency use one version (#8423)https://github.com/apache/seatunnel/commit/3ac977c8d2.3.9
[Improve][Connector][Hive] skip temporary hidden directories (#8402)https://github.com/apache/seatunnel/commit/9fdedc4872.3.9
[Feature][Connector-V2] Support use EasyExcel as read excel engine (#8064)https://github.com/apache/seatunnel/commit/b8e1177fc2.3.9
[BugFix][Excel] Fix read formulas/number cell value of excel (#8316)https://github.com/apache/seatunnel/commit/00c5aed1a2.3.9
[Improve][Transform] gz support excel (#8181)https://github.com/apache/seatunnel/commit/c3ae726ee2.3.9
[Improve][dist]add shade check rule (#8136)https://github.com/apache/seatunnel/commit/51ef800012.3.9
[Feature][File] Support config null format for text file read (#8109)https://github.com/apache/seatunnel/commit/2dbf02df42.3.9
[Improve][Excel] Support read blank string & auto type-cast (#8111)https://github.com/apache/seatunnel/commit/3a54f12532.3.9
[Improve][API] Unified tables_configs and table_list (#8100)https://github.com/apache/seatunnel/commit/84c0b8d662.3.9
[Feature][Connectors] LocalFile Support reading gz (#8025)https://github.com/apache/seatunnel/commit/337aa50f02.3.9
[Fix][Connector-V2] Fix file binary format sync convert directory to file (#7942)https://github.com/apache/seatunnel/commit/86ae9272c2.3.9
[Improve][Connector-V2] Change File Read/WriteStrategy setSeaTunnelRowTypeInfo to setCatalogTable (#7829)https://github.com/apache/seatunnel/commit/6b5f74e522.3.9
[Bug][connectors-v2] The Hadoop Source/Sink fails with Unable to find valid Kerberos Ticket. (#7809)https://github.com/apache/seatunnel/commit/a8bdea24c2.3.9
[Fix][Connector-V2] Fix When reading Excel data, string and date type conversion errors (#7796)https://github.com/apache/seatunnel/commit/749b2fe362.3.9
[Improve][Connector-V2] Support read archive compress file (#7633)https://github.com/apache/seatunnel/commit/3f98cd8a12.3.8
[Improve] Refactor S3FileCatalog and it's factory (#7457)https://github.com/apache/seatunnel/commit/d928e8b112.3.8
[Feature][Connector-V2][Iceberg] Support Iceberg Kerberos (#7246)https://github.com/apache/seatunnel/commit/e3001207c2.3.8
[Feature][Core] Support using upstream table placeholders in sink options and auto replacement (#7131)https://github.com/apache/seatunnel/commit/c4ca741222.3.6
[feature][connector-file-local] add save mode function for localfile (#7080)https://github.com/apache/seatunnel/commit/7b2f538312.3.6
[Hotfix][Hive Connector] Fix Hive hdfs-site.xml and hive-site.xml not be load error (#7069)https://github.com/apache/seatunnel/commit/c23a577f32.3.6
[Feature][Connector-V2] Add Huawei Cloud OBS connector (#4578)https://github.com/apache/seatunnel/commit/d266f4db62.3.6
[Improve][File Connector]Improve xml read code & fix can not use true for a boolean option (#6930)https://github.com/apache/seatunnel/commit/c13a563992.3.6
[Improve][Files] Support write fixed/timestamp as int96 of parquet (#6971)https://github.com/apache/seatunnel/commit/1a48a9c492.3.6
[Feature][Connector-V2] Supports the transfer of any file (#6826)https://github.com/apache/seatunnel/commit/c1401787b2.3.6
[Feature][Doris] Add Doris type converter (#6354)https://github.com/apache/seatunnel/commit/5189991842.3.6
[Improve][Connector-v2] The hive connector support multiple filesystem (#6648)https://github.com/apache/seatunnel/commit/8a4c01fe32.3.6
[Improve] Improve read with parquet type convert error (#6683)https://github.com/apache/seatunnel/commit/6c65805692.3.5
[Hotfix] fix http source can not read yyyy-MM-dd HH:mm:ss format bug & Improve DateTime Utils (#6601)https://github.com/apache/seatunnel/commit/19888e7962.3.5
[Bug] Fix OrcWriteStrategy/ParquetWriteStrategy doesn't login with kerberos (#6472)https://github.com/apache/seatunnel/commit/24441c8762.3.5
[Bug][formats] Fix fail to parse line when content contains the file delimiter (#6589)https://github.com/apache/seatunnel/commit/17e29185f2.3.5
[Improve][Connector-V2] Support read orc with schema config to cast type (#6531)https://github.com/apache/seatunnel/commit/d1599f8ad2.3.5
[Chore] Fix file spell errors (#6606)https://github.com/apache/seatunnel/commit/2599d3b732.3.5
[Feature][Connectors-V2][File]support assign encoding for file source/sink (#6489)https://github.com/apache/seatunnel/commit/d159fbe082.3.5
Add support for XML file type to various file connectors such as SFTP, FTP, LocalFile, HdfsFile, and more. (#6327)https://github.com/apache/seatunnel/commit/ec533ecd92.3.5
[Test][E2E] Add thread leak check for connector (#5773)https://github.com/apache/seatunnel/commit/1f2f3fc5f2.3.4
Fix HiveMetaStoreProxy#enableKerberos will return true if doesn't enable kerberos (#6307)https://github.com/apache/seatunnel/commit/1dad6f7062.3.4
[Feature][Connector]add s3file save mode function (#6131)https://github.com/apache/seatunnel/commit/81c51073b2.3.4
[bugfix][file-execl] Fix the Issue of Abnormal Data Reading from Excel Files (#5932)https://github.com/apache/seatunnel/commit/6a2b05a842.3.4
[Feature][Connectors-v2-file-ftp] FTP source/sink add ftp connection mode (#6077) (#6099)https://github.com/apache/seatunnel/commit/f6bcc4d592.3.4
Disable HDFSFileSystem cache (#6039)https://github.com/apache/seatunnel/commit/135c918182.3.4
[Feature][OssFile Connector] Make Oss implement source factory and sink factory (#6062)https://github.com/apache/seatunnel/commit/1a8e9b4552.3.4
[Improve][Common] Adapt FILE_OPERATION_FAILED to CommonError (#5928)https://github.com/apache/seatunnel/commit/b3dc0bbc22.3.4
[Feature][Connector-V2] Support read .xls excel file (#6066)https://github.com/apache/seatunnel/commit/43787a3dd2.3.4
Add multiple table file sink to base (#6049)https://github.com/apache/seatunnel/commit/085e0e5fc2.3.4
[Refactor][File Connector] Put Multiple Table File API to File Base Module (#6033)https://github.com/apache/seatunnel/commit/c324d663b2.3.4
Support using multiple hadoop account (#5903)https://github.com/apache/seatunnel/commit/d69d88d1a2.3.4
[Feature][Core] Upgrade flink source translation (#5100)https://github.com/apache/seatunnel/commit/5aabb14a92.3.4
[Improve][File] Clean memory buffer of JsonWriteStrategy & ExcelWriteStrategy (#5925)https://github.com/apache/seatunnel/commit/7297a4c952.3.4
[Bug][Connector][FileBase]Parquet reader parsing array type exception. (#4457)https://github.com/apache/seatunnel/commit/5c6b113292.3.4
[Improve]Change System.out.println to log output. (#5912)https://github.com/apache/seatunnel/commit/bbedb07a92.3.4
[Feature] LocalFileSource support multiple tablehttps://github.com/apache/seatunnel/commit/72be6663a2.3.4
[Improve][Common] Introduce new error define rule (#5793)https://github.com/apache/seatunnel/commit/9d1b2582b2.3.4
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755)https://github.com/apache/seatunnel/commit/8de7408102.3.4
[Improve][connector-file] unifiy option between file source/sink and update document (#5680)https://github.com/apache/seatunnel/commit/8d87cf8fc2.3.4
[Improve][LocalFile] parquet use system timezone (#5605)https://github.com/apache/seatunnel/commit/b3e13513a2.3.4
[Bugfix][Connector-v2] fix file sink isPartitionFieldWriteInFile occurred exception when no columns are given (#5508)https://github.com/apache/seatunnel/commit/9fb5499292.3.4
[Feature] Support LZO compress on File Read (#5083)https://github.com/apache/seatunnel/commit/a4a1901092.3.4
[Feature][Connector-V2][File] Support read empty directory (#5591)https://github.com/apache/seatunnel/commit/1f58f224a2.3.4
[Hotfix][File-Connector] Fix WriteStrategy parallel writing thread unsafe issue (#5546)https://github.com/apache/seatunnel/commit/1177d02d52.3.4
[Feature][File Connector] Supports writing column names when the output type is file (CSV) (#5459)https://github.com/apache/seatunnel/commit/f73b372912.3.4
Revert "[fix][hive-source][bug] fix An error occurred reading an empty directory (#5427)" (#5487)https://github.com/apache/seatunnel/commit/0939010682.3.4
[fix][hive-source][bug] fix An error occurred reading an empty directory (#5427)https://github.com/apache/seatunnel/commit/de7b86a5d2.3.4
[Improve][CheckStyle] Remove useless 'SuppressWarnings' annotation of checkstyle. (#5260)https://github.com/apache/seatunnel/commit/51c0d709b2.3.4
[Hotfix] Fix com.google.common.base.Preconditions to seatunnel shade one (#5284)https://github.com/apache/seatunnel/commit/ed5eadcf72.3.3
[Feature][Connector V2][File] Add config of 'file_filter_pattern', which used for filtering files. (#5153)https://github.com/apache/seatunnel/commit/a3c13e59e2.3.3
[bugfix][File Base] Fix Hadoop Kerberos authentication related issues. (#5171)https://github.com/apache/seatunnel/commit/2a85525f42.3.3
[Feature][Connector-V2][File] Add cos source&sink (#4979)https://github.com/apache/seatunnel/commit/1f94676432.3.3
[Improve]Connector[File] Optimize files commit order (#5045)https://github.com/apache/seatunnel/commit/1e18a8c532.3.3
[Feature][E2E][FtpFile] add ftp file e2e test case (#4647)https://github.com/apache/seatunnel/commit/b1b1f5e7e2.3.3
[Bugfix][Connector-V2] [File] Fix read temp file (#4876)https://github.com/apache/seatunnel/commit/5e03d22d62.3.2
[Feature][ConnectorV2]add file excel sink and source (#4164)https://github.com/apache/seatunnel/commit/e3b97ae5d2.3.2
Change file type to file_format_type in file source/sink (#4249)https://github.com/apache/seatunnel/commit/973a2fae32.3.1
[improve][zeta] fix zeta bugshttps://github.com/apache/seatunnel/commit/3a82e8b392.3.1
[chore] Code format with spotless plugin.https://github.com/apache/seatunnel/commit/291214ad62.3.1
Merge branch 'dev' into merge/cdchttps://github.com/apache/seatunnel/commit/4324ee1912.3.1
[Improve][Project] Code format with spotless plugin.https://github.com/apache/seatunnel/commit/423b583032.3.1
[improve][api] Refactoring schema parse (#4157)https://github.com/apache/seatunnel/commit/b2f573a132.3.1
[Imprve][Connector-V2][Hive] Support read text table & Column projection (#4105)https://github.com/apache/seatunnel/commit/717620f542.3.1
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd601052.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab166562.3.1
[Improve][Connector-V2][Hive] Support assign partitions (#3842)https://github.com/apache/seatunnel/commit/6a4a850b42.3.1
[Bug][Connectors] Text And Json WriteStrategy lost the sinkColumnsIndexInRow (#3863)https://github.com/apache/seatunnel/commit/7b5f6f1bc2.3.1
[Feature][Connector-V2][File] Support compress (#3899)https://github.com/apache/seatunnel/commit/55602f6b12.3.1
[Feature][Connector-V2][File] Allow the user to set the row delimiter as an empty string (#3854)https://github.com/apache/seatunnel/commit/84508fcb62.3.1
[Feature][Connector-V2] Support kerberos in hive and hdfs file connector (#3840)https://github.com/apache/seatunnel/commit/055ad9d832.3.1
[Feature][Connector-V2][File] Support skip number when reading text csv files (#3900)https://github.com/apache/seatunnel/commit/243b6a6b22.3.1
[Feature][API & Connector & Doc] add parallelism and column projection interface (#3829)https://github.com/apache/seatunnel/commit/b9164b8ba2.3.1
[Improve][Connector-V2][File] Improve file connector option rule and document (#3812)https://github.com/apache/seatunnel/commit/bd76077662.3.1
[Improve][Connector-V2][File] File Connector add lzo compression way. (#3782)https://github.com/apache/seatunnel/commit/8875d02582.3.1
[Improve][Connector-V2] The log outputs detailed exception stack information (#3805)https://github.com/apache/seatunnel/commit/d0c6217f22.3.1
fix file source connector option rule bug (#3804)https://github.com/apache/seatunnel/commit/cab42f6eb2.3.1
[Feature][Shade] Add seatunnel hadoop3 uber (#3755)https://github.com/apache/seatunnel/commit/5a024bdf82.3.0
[Improve][Connector-V2][HDFS] Support setting hdfs-site.xml (#3778)https://github.com/apache/seatunnel/commit/c8d59ecac2.3.0
[Feature][Connector-V2][File] Optimize filesystem utils (#3749)https://github.com/apache/seatunnel/commit/ac4e880fb2.3.0
[Improve][Connector-V2] Fix Kafka sink can't run EXACTLY_ONCE semantics (#3724)https://github.com/apache/seatunnel/commit/5e3f196e22.3.0
[Connector-V2][File] Fix bug data file name will duplicate when use SeaTunnel Engine (#3717)https://github.com/apache/seatunnel/commit/c96c530042.3.0
[Hotfix][Connector-V2][File] Fix file sink connector npe (#3706)https://github.com/apache/seatunnel/commit/a662a88fd2.3.0
[Feature][Connector-V2][Oss jindo] Add oss jindo source & sink connector (#3456)https://github.com/apache/seatunnel/commit/2507372312.3.0
[Improve][Connector-V2][File] Support split file based on batch size (#3625)https://github.com/apache/seatunnel/commit/f39e3a5312.3.0
[Improve][Connector-V2][File] Unified excetion for file source & sink connectors (#3525)https://github.com/apache/seatunnel/commit/031e8e2632.3.0
[Hotfix][Connector-V2][Hive] Fix npe of getting file system (#3506)https://github.com/apache/seatunnel/commit/e1fc3d1b02.3.0
[Improve][core-v1][seatunnel-core-base] remove seatunnel-core-base (#3480)https://github.com/apache/seatunnel/commit/d6e6a02a32.3.0
[Feature][Connector-V2][File] Add option and factory for file connectors (#3375)https://github.com/apache/seatunnel/commit/db286e8632.3.0
[Improve][Connector-V2][File] Improve code structure (#3238)https://github.com/apache/seatunnel/commit/dd5c353882.3.0
[Hotfix][Connector-V2][Hive] Fix the bug that when write data to hive throws NullPointerException (#3258)https://github.com/apache/seatunnel/commit/777bf6b422.3.0
[Bug]add 3node worker done test and fix some bug (#3115)https://github.com/apache/seatunnel/commit/bc852a4df2.3.0
[Feature][Connector-V2][SFTP] Add SFTP file source & sink connector (#3006)https://github.com/apache/seatunnel/commit/9e496383b2.3.0
[Feature][Connector-V2][S3] Add S3 file source & sink connector (#3119)https://github.com/apache/seatunnel/commit/f27d68ca92.3.0-beta
[Feature][Connector-V2][File] Fix filesystem get error (#3117)https://github.com/apache/seatunnel/commit/7404c180d2.3.0-beta
[Improve][Connector-v2][file] Reuse array type container when read row data (#3123)https://github.com/apache/seatunnel/commit/da0646ac62.3.0-beta
[Hotfix][Connector-V2][File] Fix ParquetReadStrategy get NPE (#3122)https://github.com/apache/seatunnel/commit/ba99de08c2.3.0-beta
[hotfix][engine] Add master node switch test and fix bug (#3082)https://github.com/apache/seatunnel/commit/608be51bc2.3.0-beta
[Improve][Connector-V2][File] Support parse field from file path (#2985)https://github.com/apache/seatunnel/commit/0bc12085c2.3.0-beta
[hotfix][connector][file] Solved the bug of can not parse '\t' as delimiter from config file (#3083)https://github.com/apache/seatunnel/commit/bfde596752.3.0-beta
unify flatten-maven-plugin version (#3078)https://github.com/apache/seatunnel/commit/ed743fddc2.3.0-beta
[Improve][Connector-V2] Improve text write (#2971)https://github.com/apache/seatunnel/commit/0ecd7906c2.3.0-beta
[Improve][connector][file] Support user-defined schema for reading text file (#2976)https://github.com/apache/seatunnel/commit/1c05ee0d72.3.0-beta
[Bug][Connector-V2][File] Fix the bug of incorrect path in windows environment (#2980)https://github.com/apache/seatunnel/commit/2e16161862.3.0-beta
[Improve][Connector] Improve write parquet (#2943)https://github.com/apache/seatunnel/commit/8fd9663942.3.0-beta
[Improve][all] change Log to @Slf4j (#3001)https://github.com/apache/seatunnel/commit/6016100f12.3.0-beta
[Bug][connector-file-base] Fix source split assigning reader to negative number (#2921)https://github.com/apache/seatunnel/commit/0b5a2852f2.3.0-beta
[Improve][Connector-V2] Improve orc write strategy to support all data types (#2860)https://github.com/apache/seatunnel/commit/4d048cc232.3.0-beta
[Fix][Connector-V2-File] Fix file connector bug (#2858)https://github.com/apache/seatunnel/commit/e0459bbab2.2.0-beta
[Fix][Connector-V2] Fix HiveSource Connector read orc table error (#2845)https://github.com/apache/seatunnel/commit/61720306e2.2.0-beta
[Improve][Connector-V2] Improve read parquet (#2841)https://github.com/apache/seatunnel/commit/e19bc82f92.2.0-beta
[Imporve][Connector-V2] Refactor ftp sink & Add ftp file source (#2774)https://github.com/apache/seatunnel/commit/4aacbcdd12.2.0-beta
[Bug][Connector-V2] Fix hive source connector parallelism not work (#2823)https://github.com/apache/seatunnel/commit/9f21d4c762.2.0-beta
[Improve][Connector-V2] Imporve orc read strategy (#2747)https://github.com/apache/seatunnel/commit/af34beda32.2.0-beta
[DEV][Api] Replace SeaTunnelContext with JobContext and remove singleton pattern (#2706)https://github.com/apache/seatunnel/commit/cbf82f7552.2.0-beta
[Improve][build] Improved scope of maven-shade-plugin (#2665)https://github.com/apache/seatunnel/commit/93bc8bd112.2.0-beta
[Feature][Connector-V2] Add oss sink (#2629)https://github.com/apache/seatunnel/commit/bb2ad40482.2.0-beta
[#2606]Dependency management split (#2630)https://github.com/apache/seatunnel/commit/fc047be692.2.0-beta
[Improve][Connector-V2] Refactor the structure of file sink to reduce redundant codes (#2555)https://github.com/apache/seatunnel/commit/6315092932.2.0-beta
[Feature][Connector-V2] Add oss source connector (#2467)https://github.com/apache/seatunnel/commit/712b777442.2.0-beta
[Feature][File connector] Support ftp file sink (#2483)https://github.com/apache/seatunnel/commit/a87e5de802.2.0-beta
[Feature][Connector-V2] Add hdfs file json support (#2451)https://github.com/apache/seatunnel/commit/84f6b17c12.2.0-beta
[Feature][Connector-V2] Add base source connector code for connector-file-base (#2399)https://github.com/apache/seatunnel/commit/1829ddc662.2.0-beta
[Improve][Connector-V2] Refactor the package of local file connector (#2403)https://github.com/apache/seatunnel/commit/a538daed52.2.0-beta
[Feature][Connector-V2] Add json file sink & json format (#2385)https://github.com/apache/seatunnel/commit/dd68c06b02.2.0-beta
[Bug][Connector-V2] Fix the bug that file connector release resources multi times (#2379)https://github.com/apache/seatunnel/commit/58c64aab22.2.0-beta
[Improve][Connector-V2] Optimize the code structure (#2380)https://github.com/apache/seatunnel/commit/7376ec7ab2.2.0-beta
[Feature][Connector-V2] Support orc file format in file connector (#2369)https://github.com/apache/seatunnel/commit/f44fe1e032.2.0-beta
[improve][UT] Upgrade junit to 5.+ (#2305)https://github.com/apache/seatunnel/commit/362319ff32.2.0-beta
Replace plain string with constants (#2308)https://github.com/apache/seatunnel/commit/3c0415e562.2.0-beta
[Connector-V2] Add parquet writer in file connector (#2273)https://github.com/apache/seatunnel/commit/c95cc72cf2.2.0-beta
[Connector-V2] Add Hive sink connector v2 (#2158)https://github.com/apache/seatunnel/commit/23ad4ee732.2.0-beta
[Connector-V2] Add File Sink Connector (#2117)https://github.com/apache/seatunnel/commit/e2283da642.2.0-beta