MatrixOne-v2.0.1

Release date: December 10, 2024
MatrixOne version: v2.0.1

MatrixOne 2.0.1 introduces a series of improvements and bug fixes that enhance system performance, stability, and usability. Below are the major updates and new features.

Key Improvements

• Index and Query Optimization
• Memory and Resource Optimization
• Password Management
• Backup and Restore Enhancements
• Experimental support for full text search

Bugfix

This release addresses several critical bugs that improve the overall stability and reliability of the system. Some of the key bug fixes include:

• Transaction Management:

  • Fixed deadlocks and transaction state inconsistencies in distributed environments.
  • Resolved race conditions in transaction handling and session management.
  • Improved error handling for lock conflicts and concurrent delete operations.

• SQL Execution:

  • Fixed issues with recursive CTEs, view creation, and ON DUPLICATE KEY UPDATE statements.
  • Resolved problems with incorrect row counts and handling of compressed files during LOAD DATA.

• Logtail and Cache:

  • Addressed data race conditions and performance issues in logtail processing.
  • Fixed issues with cache usage and hit status monitoring.

• Full-Text Index:

  • Fixed panics related to full-text index creation and runtime filters.
  • Improved delete operations for tables with full-text indexes.

• Backup and Restore:

  • Fixed missing workspace data during the restore process.
  • Resolved errors when restoring tables or databases with special keywords or case-sensitive names.

• Performance and Stability:

  • Addressed memory allocation issues in batch processing and deduplication operations.
  • Resolved panics and race conditions in various modules, including query execution and system diagnostics.

Additionally, many other minor bug fixes were implemented to further enhance system reliability. For the full list of changes, refer to the What's Changed.

What's Changed

• Fix missing workspace data in restore process by @aptend in #19736
• uses CI release/2.0-dev for 2.0-dev by @Rosyrain in #19692
• refactor: improve checkTableDefChange by @huby2358 in #19681
• sync to 2.0: reduce the comparison during the reader primary key iterator scan by @gouhongshen in #19677
• optimize folding in EvalExpression(2.0-dev) by @zengyan1 in #19686
• [enhancement] proxy: reduce the interval time of refresh cn nodes by @volgariver6 in #19704
• Operator analyze counter set reuse2.0-dev by @qingxinhome in #19708
• abort all active txn on invalid service and make new txn can work to 2.0 by @zhangxu19830126 in #19749
• Fix test global checkpoint5 2.0 by @LeftHandCold in #19742
• fix lockservice dead lock with txn by @zhangxu19830126 in #19773
• fix dedup by @jiangxinmeng1 in #19775
• [bug] return retry error if snapshot ts is smaller than ts of catalog by @volgariver6 in #19780
• malloc: add IgnoreMunmapError by @reusee in #19782
• Password management policy to 2.0-dev by @YANGGMM in #19785
• fix group concat without data by @iamlinjunhong in #19778
• set sink mysql timeout to 10min by @ck89119 in #19791
• Fix TestAppendAndGC for 2.0 by @LeftHandCold in #19795
• handle index table missing table_kind identifier by @qingxinhome in #19796
• Backup adapts to 2.0 by @LeftHandCold in #19768
• Improve cdc send sql performance by @ck89119 in #19797
• fix: fetch result from correct resultset 2.0-dev by @xzxiong in #19809
• fix txn status invalid to 2.0 by @zhangxu19830126 in #19817
• fix ut by @ck89119 in #19836
• check user after open check switch by @YANGGMM in #19830
• fix transfer deletes by @jiangxinmeng1 in #19831
• [bug] logtail client: change the channel to buffered one by @volgariver6 in #19839
• Fix TestAppendAndGC2 by @LeftHandCold in #19824
• refactor: reduce memory alloc in makeBatchRows by @huby2358 in #19793
• refactor DML plan (cp #18406) by @aunjgr in #19848
• objectio: make metadata cache capacity dynamic adjustable by @reusee in #19829
• use customized driver by @ck89119 in #19849
• allow now function pushdown to block filters by @badboynt1 in #19855
• fix a bug cause remote run cannot stop immediately. by @m-schen in #19841
• fix Incorrect time unit for password lock check(#2.0-dev) by @YANGGMM in #19853
• fifocache: add context params to methods by @reusee in #19805
• [fix] : Instead of sending the value scan's batch all at once, split it up to reuse memory. by @jensenojs in #19845
• make ut for multi_update faster (#19861) by @ouyuanning in #19874
• [bug] proxy: fix UT by @volgariver6 in #19880
• metric: redefine duration buckets by @reusee in #19878
• to 2.0: fix the batchApplyTombstone got unsorted rowOffsets by @gouhongshen in #19870
• fix nested recursive cte by @iamlinjunhong in #19899
• [Bug]: use load_file() function to load stage data, mo report 'ERROR 20101 (HY000): internal error: Invalid alloc size 7775731712 by @cpegeric in #19905
• malloc: do not unmap to avoid VMA fragmentation by @reusee in #19886
• bug fix fulltext phrase search by @cpegeric in #19902
• add debug tool for remote run (#19833) by @ouyuanning in #19892
• feat: keep S3IOInput as 6-prec float value. by @xzxiong in #19865
• refactor: remove openSysTable 2.0 by @aptend in #19823
• Add metric for GC by @LeftHandCold in #19887
• cherry-pick optimize performance for filling hashkeys on string vectors (#19881) by @badboynt1 in #19918
• fix bug: sysbench delete pk will flush s3 by @ouyuanning in #19927
• support ip whitelist feat (#2.0-dev) by @YANGGMM in #19919
• cherry pick 'support pushdown cast expression to block filters for date time type' by @badboynt1 in #19938
• ignore bvt case 2.0-dev by @w-zr in #19935
• update log by @jiangxinmeng1 in #19932
• do some lock if lock no rows after lock_op by @ouyuanning in #19943
• [bug] clusterservice: ignore tn store which is expired by @volgariver6 in #19954
• support password_history and password_reuse_interval take effect separately(#2.0-dev) by @YANGGMM in #19950
• enable transfer in metric and cu table on 2.0-dev by @w-zr in #19916
• optimize workspace size by @Wenbin1002 in #19929
• cherry-pick: support observe cache usage and hit status (#19914) by @xzxiong in #19926
• add system variable profiling_history_size(#2.0-dev) by @YANGGMM in #19946
• cherry-pick: couple of updates by @daviszhen in #19968
• update logtail entry type by @jiangxinmeng1 in #19901
• [bug] fix ut, which raises "address already in use" error by @volgariver6 in #19957
• fix panic in fillStrhashmap by @badboynt1 in #19971
• fix create view with hint(#2.0-dev) by @YANGGMM in #19978
• fix bug: panic on create fulltext idx by @ouyuanning in #19975
• Fix TestBackupData2 for 2.0 by @LeftHandCold in #19986
• fix show index panic(#2.0-dev) by @YANGGMM in #19990
• optimize index only scan in some cases by @badboynt1 in #19997
• fix: load data local format bug by @huby2358 in #20004
• reuse txnop by @daviszhen in #20017
• New op add analyze-2.0-dev by @qingxinhome in #20013
• Fix force global checkpoint 2.0 by @LeftHandCold in #20019
• fix password reuse interval（2.0-dev) by @YANGGMM in #20041
• Fix Generate_Series NULL (2.0-dev) by @zengyan1 in #20030
• chore: Compile with the new GoMaxProcs value in 2.0-dev by @xzxiong in #20003
• add some logs in workspace by @Wenbin1002 in #20024
• to 2.0: fix a lot of tombstone objects missing when CN transfer. by @gouhongshen in #20034
• improve mem reuse for serial&serial_full by @ouyuanning in #20429
• Security bvt case by @Songxx-7 in #20428
• fix waiter leak by @iamlinjunhong in #20441
• do not lock table if lock no rows by @ouyuanning in #20431
• fix a panic in multi-CN DEDUP join (2.0-dev) by @aunjgr in #20462
• skip shard and partition embed tests to 2.0 by @zhangxu19830126 in #20464
• fix panic by @daviszhen in #20465
• fix hung due to range lock to 2.0 by @zhangxu19830126 in #20452
• update code of relation exist check2.0 dev by @qingxinhome in #20454
• fix data race in TestDelete3 by @w-zr in #20427
• [bug] add rpc read timeout for logtail client by @volgariver6 in #20472
• fix txn wait active queue leak to 2.0 by @zhangxu19830126 in #20474
• quit cdc Start func when pause by @ck89119 in #20385
• fix ut by @daviszhen in #20478
• chore: cu price allows 0 in 2.0 by @xzxiong in #20482
• remove panic in parsing txn entries by @aptend in #20495
• add backSes Exec lower_case_table_names(#2.0-dev) by @YANGGMM in #20494
• fix bug: update secondary key panic by @ouyuanning in #20506
• Analyzer adds FileService Cache read and hit statistics by @qingxinhome in #20451
• to 2.0 "reduce memory consumption in bvt shuffle test" by @badboynt1 in #20503
• fix data race in ut by @ck89119 in #20508
• to 2.0"optimize index plan to support sysbench random ranges benchmark" by @badboynt1 in #20514
• use mo_table_rows/size in show table status by @ck89119 in #20435
• fix double close atomic batch by @ck89119 in #20515
• delete unsubscribed account in mo_subs by @ck89119 in #20676
• fix restore view with lower case table names equals to 0(#2.0-dev) by @YANGGMM in #20668

New Contributors

• @Songxx-7 made their first contribution in #20346

Full Changelog: v2.0.0...v2.0.1