Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pipe: Intoduce TsFileInsertionScanDataContainer to read data from tsfile sequentially to improve pattern parse performance when filter rate is high #12781

Merged
merged 21 commits into from
Jun 27, 2024
Merged

Pipe: Intoduce TsFileInsertionScanDataContainer to read data from tsfile sequentially to improve pattern parse performance when filter rate is high #12781

merged 21 commits into from
Jun 27, 2024

Conversation

Caideyipi
Copy link
Collaborator

Description

As the title said.

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.
Key changed/added classes (or packages if there are too many classes) in this PR

@Caideyipi Caideyipi marked this pull request as draft June 21, 2024 03:50
@SteveYurongSu SteveYurongSu self-assigned this Jun 21, 2024
@Caideyipi Caideyipi marked this pull request as ready for review June 24, 2024 07:56
Caideyipi and others added 18 commits June 24, 2024 22:09
@Caideyipi
Update TsFileInsertionScanDataContainer.java
9b6a69a
@Caideyipi
Merge branch 'master' of https://github.com/apache/iotdb into add_tsf…
4b24f4c 
…ile_scan
@Caideyipi
Merge remote-tracking branch 'upstream/master' into add_tsfile_scan
a8fd26d 
# Conflicts:
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/container/query/TsFileInsertionQueryDataContainer.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/resource/tsfile/PipeTsFileResource.java
@Caideyipi
Update PipeTsFileResource.java
d4d5bd0
@Caideyipi
bug fix
f7be01c
@Caideyipi
Update TsFileInsertionScanDataContainer.java
43faedd
@Caideyipi
Update TsFileInsertionScanDataContainer.java
f9ede59
@Caideyipi
Merge branch 'master' of https://github.com/apache/iotdb into add_tsf…
5537f9d 
…ile_scan
@Caideyipi
Update TsFileInsertionScanDataContainer.java
cd4d86b
@SteveYurongSu
Merge branch 'master' of https://github.com/apache/iotdb into pr/12781
2fa6360
@Caideyipi
merge
97a5770
@SteveYurongSu
refactor
aba8f52
@SteveYurongSu
Merge branch 'add_tsfile_scan' of https://github.com/Caideyipi/iotdb …
ec69c2d 
…into pr/12781
@SteveYurongSu
Update TsFileInsertionDataContainer.java
f421c4f
@SteveYurongSu
Update TsFileInsertionScanDataContainer.java
f784e64
@Caideyipi
add close logic
8565be5
@SteveYurongSu
Update TsFileInsertionDataContainerProvider.java
d5f71e5
@SteveYurongSu
Update TsFileInsertionScanDataContainer.java
3b14450
@SteveYurongSu SteveYurongSu changed the title Pipe: Implemented tsFile scan container Pipe: Intoduce TsFileInsertionScanDataContainer to read data from tsfile sequentially to improve pattern parse performance when filter rate is high Jun 27, 2024
@SteveYurongSu SteveYurongSu merged commit 2e1ebf4 into apache:master Jun 27, 2024
39 checks passed
JackieTien97 pushed a commit that referenced this pull request Jul 10, 2024
@Caideyipi @JackieTien97
Pipe: Intoduce TsFileInsertionScanDataContainer to read data from tsf…
aee6faa 
…ile sequentially to improve pattern parse performance when filter rate is high (#12781)

Co-authored-by: Steve Yurong Su <[email protected]>
(cherry picked from commit 2e1ebf4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SteveYurongSu SteveYurongSu SteveYurongSu approved these changes

Assignees

@SteveYurongSu SteveYurongSu

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Caideyipi @SteveYurongSu