Fix DELETE on compressed chunk with non-btree operators #7645
+249
−75
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@gayyappan, @fabriziomello: please review this pull request.
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7645 +/- ##
==========================================
+ Coverage 80.06% 81.30% +1.23%
==========================================
Files 190 241 +51
Lines 37181 44730 +7549
Branches 9450 11165 +1715
==========================================
+ Hits 29770 36368 +6598
- Misses 2997 3974 +977
+ Partials 4414 4388 -26 ☔ View full report in Codecov by Sentry. |
svenklemm
force-pushed
the
compressed_delete_non_btree
branch
from
c2f040f to
fb7f931
Compare
svenklemm
force-pushed
the
compressed_delete_non_btree
branch
5 times, most recently
from
1055bbf to
986bac7
Compare
antekresic
approved these changes
Feb 3, 2025
svenklemm
force-pushed
the
compressed_delete_non_btree
branch
2 times, most recently
from
9dbf5d4 to
6302c45
Compare
When deleting from compressed chunk the direct delete optimization would ignore constraints that were not using btree operators leading to constraints of the DELETE not being applied to the direct delete on the compressed chunk, potentially leading to data corruption. This patch disables the direct delete optimization when any of the constraints can not be applied. Fixes timescale#7644
svenklemm
force-pushed
the
compressed_delete_non_btree
branch
from
6302c45 to
b1177af
Compare
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * timescale#6884 Match the Postgres NaN comparison behavior in WHERE clause over compressed tables. * timescale#7600 Fix lock order when dropping index * timescale#7637 Allow EXPLAIN in read-only mode * timescale#7645 Fix DELETE on compressed chunk with non-btree operators * timescale#7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * timescale#7653 Push down orderby scankeys to Hypercore TAM * timescale#7665 Block merging of frozen chunks * timescale#7673 Don't abort additional INSERTs when hitting first conflict **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * @jakehedlund for reporting the incompatible NaN behavior in WHERE clause over compressed tables. * Timescale community members Jacob and pantonis for reporting issues with slow queries.
Merged
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * timescale#6884 Match the Postgres NaN comparison behavior in WHERE clause over compressed tables. * timescale#7600 Fix lock order when dropping index * timescale#7637 Allow EXPLAIN in read-only mode * timescale#7645 Fix DELETE on compressed chunk with non-btree operators * timescale#7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * timescale#7653 Push down orderby scankeys to Hypercore TAM * timescale#7665 Block merging of frozen chunks * timescale#7673 Don't abort additional INSERTs when hitting first conflict **GUCs** * enable_hypercore_scankey_pushdown: Push down qualifiers as scankeys when using Hypercore TAM **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * @jakehedlund for reporting the incompatible NaN behavior in WHERE clause over compressed tables. * Timescale community members Jacob and pantonis for reporting issues with slow queries.
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * timescale#7600 Fix lock order when dropping index * timescale#7637 Allow EXPLAIN in read-only mode * timescale#7645 Fix DELETE on compressed chunk with non-btree operators * timescale#7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * timescale#7653 Push down orderby scankeys to Hypercore TAM * timescale#7665 Block merging of frozen chunks * timescale#7673 Don't abort additional INSERTs when hitting first conflict **GUCs** * enable_hypercore_scankey_pushdown: Push down qualifiers as scankeys when using Hypercore TAM **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
svenklemm
added a commit
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * #7656 Remove limitation of compression policy for continuous aggregates **Bugfixes** * #7600 Fix lock order when dropping index * #7637 Allow EXPLAIN in read-only mode * #7645 Fix DELETE on compressed chunk with non-btree operators * #7649 Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * #7653 Push down orderby scankeys to Hypercore TAM * #7665 Block merging of frozen chunks * #7673 Don't abort additional INSERTs when hitting first conflict **GUCs** * enable_hypercore_scankey_pushdown: Push down qualifiers as scankeys when using Hypercore TAM **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
Merged
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Feb 10, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [timescale#7656](timescale#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [timescale#7600](timescale#7600) Fix lock order when dropping index * [timescale#7637](timescale#7637) Allow EXPLAIN in read-only mode * [timescale#7645](timescale#7645) Fix DELETE on compressed chunk with non-btree operators * [timescale#7649](timescale#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [timescale#7653](timescale#7653) Push down orderby scankeys to Hypercore TAM * [timescale#7665](timescale#7665) Block merging of frozen chunks * [timescale#7673](timescale#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [timescale#7653](timescale#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
svenklemm
added a commit
that referenced
this pull request
Feb 11, 2025
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * [#7656](#7656) Remove limitation of compression policy for continuous aggregates **Bugfixes** * [#7600](#7600) Fix lock order when dropping index * [#7637](#7637) Allow EXPLAIN in read-only mode * [#7645](#7645) Fix DELETE on compressed chunk with non-btree operators * [#7649](#7649) Allow non-btree operator pushdown in UPDATE/DELETE queries on compressed chunks * [#7653](#7653) Push down orderby scankeys to Hypercore TAM * [#7665](#7665) Block merging of frozen chunks * [#7673](#7673) Don't abort additional INSERTs when hitting first conflict **GUCs** * `enable_hypercore_scankey_pushdown`: Push down qualifiers as scankeys when using Hypercore TAM introduced with [#7653](#7653) **Thanks** * @bjornuppeke for reporting a problem with INSERT INTO ... ON CONFLICT DO NOTHING on compressed chunks * @ikalafat for reporting a problem with EXPLAIN in read-only mode * Timescale community members Jacob and pantonis for reporting issues with slow queries.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When deleting from compressed chunk the direct delete optimization
would ignore constraints that were not using btree operators
leading to constraints of the DELETE not being applied to the
direct delete on the compressed chunk, potentially leading to
data corruption. This patch disables the direct delete optimization
when any of the constraints can not be applied.
Fixes #7644