Fix handling for multiple unique index in compressed INSERT #7061
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
svenklemm
force-pushed
the
compressed_insert_key_columns
branch
from
99aa76b
to
2c59e76
Compare
antekresic
approved these changes
Jun 25, 2024
tsl/src/compression/compression.c
Outdated
| } | ||
| index_close(indexDesc, AccessShareLock); | ||
|
|
||
| if (first_iteration) |
There was a problem hiding this comment.
Suggested change
| if (first_iteration) | |
| if (!shared_attrs) |
There was a problem hiding this comment.
that wouldnt handle the case when we have non-intersecting indexes resetting shared_atts
svenklemm
force-pushed
the
compressed_insert_key_columns
branch
from
2c59e76
to
5bf3a41
Compare
akuzm
reviewed
Jun 25, 2024
| * | ||
| * This is based on RelationGetIndexAttrBitmap from postgres with changes | ||
| * to also track unique expression indexes. | ||
| * In case of multiple unique indexes we have to return the shared columns. |
There was a problem hiding this comment.
Trying to understand this better: we only use one index for decompression, so it means we have to decompress all batches that match by shared columns, and we can't add a filter on the other columns, right?
There was a problem hiding this comment.
So this is columns we can safely use for filtering the batches in INSERT to satisfy the UNIQUE checks. The check is based on the unique indexes defined on the uncompressed chunk. These columns might be indexed on the compressed because they are segmentby or have metadata and we can filter on them even if we dont have corresponding compressed index.
akuzm
approved these changes
Jun 25, 2024
svenklemm
force-pushed
the
compressed_insert_key_columns
branch
2 times, most recently
from
1698911
to
76f8912
Compare
svenklemm
force-pushed
the
compressed_insert_key_columns
branch
4 times, most recently
from
adac3fb
to
e575c4c
Compare
Adjust compressed_insert_key_columns to correctly handle multiple unique indexes. This patch changes the function to no longer combine the columns from multiple indexes but instead only return intersecting columns from all the unique indexes. This patch also fixes a couple comments in that function.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7061 +/- ##
==========================================
+ Coverage 80.06% 81.75% +1.68%
==========================================
Files 190 200 +10
Lines 37181 37298 +117
Branches 9450 9725 +275
==========================================
+ Hits 29770 30493 +723
+ Misses 2997 2897 -100
+ Partials 4414 3908 -506 ☔ View full report in Codecov by Sentry. |
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jun 26, 2024
This release contains performance improvements and bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Features** **Bugfixes** * timescale#7061 Fix handling of multiple unique indexes in compressed INSERT **Thanks**
Closed
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jun 27, 2024
This release contains performance improvements and bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Features** **Bugfixes** * timescale#7061 Fix handling of multiple unique indexes in compressed INSERT **Thanks**
Closed
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jul 2, 2024
This release contains performance improvements and bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Features** **Bugfixes** * timescale#7061 Fix handling of multiple unique indexes in compressed INSERT * timescale#7080 Fix `corresponding equivalence member not found` error **Thanks**
Closed
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jul 2, 2024
This release contains bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Migrating from self-hosted TimescaleDB v2.14.x and earlier** After you run `ALTER EXTENSION`, you must run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql). For more details, see the following pull request [timescale#6797](timescale#6797). If you are migrating from TimescaleDB v2.15.0, v2.15.1 or v2.15.2, no changes are required. **Bugfixes** * timescale#7061: Fix handling of multiple unique indexes in compressed INSERT. * timescale#7080: Fix `corresponding equivalence member not found` error.
Merged
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jul 2, 2024
This release contains bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Migrating from self-hosted TimescaleDB v2.14.x and earlier** After you run `ALTER EXTENSION`, you must run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql). For more details, see the following pull request [timescale#6797](timescale#6797). If you are migrating from TimescaleDB v2.15.0, v2.15.1 or v2.15.2, no changes are required. **Bugfixes** * timescale#7061: Fix handling of multiple unique indexes in compressed INSERT. * timescale#7080: Fix `corresponding equivalence member not found` error.
pallavisontakke
added a commit
that referenced
this pull request
Jul 2, 2024
This release contains bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Migrating from self-hosted TimescaleDB v2.14.x and earlier** After you run `ALTER EXTENSION`, you must run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql). For more details, see the following pull request [#6797](#6797). If you are migrating from TimescaleDB v2.15.0, v2.15.1 or v2.15.2, no changes are required. **Bugfixes** * #7061: Fix the handling of multiple unique indexes in a compressed INSERT. * #7080: Fix the `corresponding equivalence member not found` error. * #7088: Fix the leaks in the DML functions. * #7035: Fix the error when acquiring a tuple lock on the OSM chunks on the replica. **Thanks** * @Kazmirchuk for reporting the issue about leaks with the functions in DML.
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jul 2, 2024
This release contains performance improvements and bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Features** **Bugfixes** * timescale#7061 Fix handling of multiple unique indexes in compressed INSERT * timescale#7080 Fix `corresponding equivalence member not found` error * timescale#7088 Fix leaks with functions in DML **Thanks** * @Kazmirchuk for reporting this
Merged
svenklemm
pushed a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Jul 2, 2024
This release contains performance improvements and bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Features** **Bugfixes** * timescale#7061 Fix handling of multiple unique indexes in compressed INSERT * timescale#7080 Fix `corresponding equivalence member not found` error * timescale#7088 Fix leaks with functions in DML **Thanks** * @Kazmirchuk for reporting this
svenklemm
added a commit
that referenced
this pull request
Jul 2, 2024
This release contains bug fixes since the 2.15.2 release. Best practice is to upgrade at the next available opportunity. **Migrating from self-hosted TimescaleDB v2.14.x and earlier** After you run `ALTER EXTENSION`, you must run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql). For more details, see the following pull request [#6797](#6797). If you are migrating from TimescaleDB v2.15.0, v2.15.1 or v2.15.2, no changes are required. **Bugfixes** * #7061: Fix the handling of multiple unique indexes in a compressed INSERT. * #7080: Fix the `corresponding equivalence member not found` error. * #7088: Fix the leaks in the DML functions. * #7035: Fix the error when acquiring a tuple lock on the OSM chunks on the replica. * #7091: Fix ORDER BY/GROUP BY expression not found in targetlist on PG16 **Thanks** * @Kazmirchuk for reporting the issue about leaks with the functions in DML. --------- Signed-off-by: Sven Klemm <31455525+svenklemm@users.noreply.github.com> Co-authored-by: Sven Klemm <31455525+svenklemm@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adjust compressed_insert_key_columns to correctly handle multiple
unique indexes. This patch changes the function to no longer combine
the columns from multiple indexes but instead only return intersecting
columns from all the unique indexes.
This patch also fixes a couple comments in that function.
Disable-check: commit-count