Makes sure KNNVectorValues aren't recreated unnecessarily when #2133

shatejas · 2024-09-20T18:08:53Z

quantization isn't needed

Description

An attempt to address the regression seen in force-merge time for cohere-10m dataset.

KNNVectorValues creation is optimized to skip for non-quantization cases

The perf for force merge showed improvements in 1m dataset

	number of index segments	force merge time (minutes)	force merge segments
2.17 code	75	15.68263	3
Current PR	92	12.51252	3

Related Issues

#2134

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

navneet1v · 2024-09-23T18:04:51Z

@shatejas can you update the changelog

navneet1v · 2024-09-23T18:19:58Z

@shatejas is there a way we can validate how many times we are calling those MergedFloatValues in the unit testing?

jmazanec15 · 2024-09-23T20:14:49Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNBinaryVectorValues.java

@@ -34,7 +34,8 @@ public byte[] getVector() throws IOException {
    @Override
    public byte[] conditionalCloneVector() throws IOException {
        byte[] vector = getVector();
-        if (vectorValuesIterator.getDocIdSetIterator() instanceof ByteVectorValues) {
+        if (vectorValuesIterator instanceof KNNVectorValuesIterator.MergeByteVectorValuesIterator


nit: we could add a wrapper method to unwrap so that we dont always have to check both

Its trivial currently so will prefer keeping it as it is, if it gets complicated we can either make a predicate or stub it in a method

jmazanec15 · 2024-09-23T20:15:41Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNVectorValuesFactory.java

+        return getVectorValues(vectorDataType, new KNNVectorValuesIterator.FieldWriterIteratorValues<>(docIdWithFieldSet, vectors));
+    }
+
+    public static <T> KNNVectorValues<T> getVectorValues(


nit: javadocs for all public?

will address javadocs with unit test

jmazanec15 · 2024-09-23T20:18:01Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+
+public final class KNNMergeVectorValues {
+
+    public static List<KNNVectorValuesSub<FloatVectorValues>> mergeFloatVectorValues(FieldInfo fieldInfo, MergeState mergeState)


jmazanec15 · 2024-09-23T20:18:30Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+
+    public static List<KNNVectorValuesSub<FloatVectorValues>> mergeFloatVectorValues(FieldInfo fieldInfo, MergeState mergeState)
+        throws IOException {
+        assert fieldInfo != null && fieldInfo.hasVectorValues();


Are we sure on this assert?

This piece was taken from lucene with minor edits so kept it as it is, any use case that need to be handled considering this is not used for BinaryDocValues?

jmazanec15 · 2024-09-23T20:19:28Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+
+    private static int computeLiveDocs(final DocIdSetIterator values, Bits liveDocs) throws IOException {
+        int count = 0;
+        if (liveDocs != null) {


Why cant we call length on live docs?

length() counts 0s so cannot use that. cardinality() can only be obtained for FixedBitSet and that is handled before calling this

shatejas · 2024-09-23T20:49:30Z

Separated out liveDocs optimization to #2135

quantization isn't needed Signed-off-by: Tejas Shah <shatejas@amazon.com>

…zation isn't needed (#2133) Signed-off-by: Tejas Shah <shatejas@amazon.com> (cherry picked from commit e33afa5)

opensearch-trigger-bot · 2024-09-24T18:01:39Z

The backport to 2.17 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.17 2.17
# Navigate to the new working tree
cd .worktrees/backport-2.17
# Create a new branch
git switch --create backport/backport-2133-to-2.17
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e33afa5de5f8658ad7fbe71125707436e81cc5b8
# Push it to GitHub
git push --set-upstream origin backport/backport-2133-to-2.17
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.17

Then, create a pull request where the base branch is 2.17 and the compare/head branch is backport/backport-2133-to-2.17.

…zation isn't needed (opensearch-project#2133) Signed-off-by: Tejas Shah <shatejas@amazon.com> (cherry picked from commit e33afa5)

…zation isn't needed (#2133) (#2140) Signed-off-by: Tejas Shah <shatejas@amazon.com> (cherry picked from commit e33afa5)

…n quantization isn't needed (opensearch-project#2133) (opensearch-project#2140)" This reverts commit ca6b03f.

…n quantization isn't needed (opensearch-project#2133) (opensearch-project#2140)" This reverts commit ca6b03f. Signed-off-by: Naveen Tatikonda <navtat@amazon.com>

…n quantization isn't needed (#2133) (#2140)" (#2161) This reverts commit ca6b03f. Signed-off-by: Naveen Tatikonda <navtat@amazon.com>

…en quantization isn't needed (opensearch-project#2133) (opensearch-project#2140)" (opensearch-project#2161) This reverts commit b0d82b7. Signed-off-by: Navneet Verma <navneev@amazon.com>

shatejas marked this pull request as ready for review September 20, 2024 18:46

shatejas requested review from heemin32, navneet1v, VijayanB, vamshin, jmazanec15, naveentatikonda, junqiu-lei, martin-gaievski, ryanbogan and luyuncheng as code owners September 20, 2024 18:46

shatejas mentioned this pull request Sep 20, 2024

[BUG] Regression in cohere-10m force merge latency after switching to NativeEngines990KnnVectorsWriter #2134

Open

shatejas changed the title ~~Makes sure KNNVectorValues aren't recreated unnecessarily when~~ Optimizes NativeEngines990KNNVectorsWriter to make sure KNNVectorValues are not generated unnecessarily and totalLiveDocs calculation is optimal Sep 23, 2024

shatejas force-pushed the force-merge-regression branch from 61d3f71 to 0a280c1 Compare September 23, 2024 19:16

jmazanec15 reviewed Sep 23, 2024

View reviewed changes

shatejas force-pushed the force-merge-regression branch from 0a280c1 to cdcb1eb Compare September 23, 2024 20:28

shatejas changed the title ~~Optimizes NativeEngines990KNNVectorsWriter to make sure KNNVectorValues are not generated unnecessarily and totalLiveDocs calculation is optimal~~ Makes sure KNNVectorValues aren't recreated unnecessarily when Sep 23, 2024

Makes sure KNNVectorValues aren't recreated unnecessarily when

2d88cce

quantization isn't needed Signed-off-by: Tejas Shah <shatejas@amazon.com>

shatejas force-pushed the force-merge-regression branch from cdcb1eb to 2d88cce Compare September 23, 2024 20:53

shatejas requested a review from jmazanec15 September 23, 2024 21:27

navneet1v approved these changes Sep 23, 2024

View reviewed changes

jmazanec15 approved these changes Sep 23, 2024

View reviewed changes

Vikasht34 approved these changes Sep 23, 2024

View reviewed changes

jmazanec15 merged commit e33afa5 into opensearch-project:main Sep 23, 2024
31 checks passed

navneet1v added the backport 2.x label Sep 23, 2024

opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 23, 2024

Makes sure KNNVectorValues aren't recreated unnecessarily when quanti…

a342e7b

…zation isn't needed (#2133) Signed-off-by: Tejas Shah <shatejas@amazon.com> (cherry picked from commit e33afa5)

opensearch-trigger-bot bot mentioned this pull request Sep 23, 2024

[Backport 2.x] Makes sure KNNVectorValues aren't recreated unnecessarily when #2137

Merged

navneet1v added the backport 2.17 label Sep 24, 2024

navneet1v removed the backport 2.17 label Sep 24, 2024

shatejas mentioned this pull request Sep 24, 2024

[Backport 2.17] Makes sure KNNVectorValues aren't recreated unnecessarily when quantization isn't needed (#2133) #2140

Merged

5 tasks

naveentatikonda pushed a commit that referenced this pull request Sep 27, 2024

Makes sure KNNVectorValues aren't recreated unnecessarily when quanti…

ca6b03f

…zation isn't needed (#2133) (#2140) Signed-off-by: Tejas Shah <shatejas@amazon.com> (cherry picked from commit e33afa5)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Makes sure KNNVectorValues aren't recreated unnecessarily when #2133

Makes sure KNNVectorValues aren't recreated unnecessarily when #2133

shatejas commented Sep 20, 2024 •

edited

Loading

navneet1v commented Sep 23, 2024

navneet1v commented Sep 23, 2024

jmazanec15 Sep 23, 2024

shatejas Sep 23, 2024

jmazanec15 Sep 23, 2024

shatejas Sep 23, 2024

jmazanec15 Sep 23, 2024

jmazanec15 Sep 23, 2024

shatejas Sep 23, 2024 •

edited

Loading

jmazanec15 Sep 23, 2024

shatejas Sep 23, 2024

shatejas commented Sep 23, 2024

opensearch-trigger-bot bot commented Sep 24, 2024


		public final class KNNMergeVectorValues {

		public static List<KNNVectorValuesSub<FloatVectorValues>> mergeFloatVectorValues(FieldInfo fieldInfo, MergeState mergeState)

Makes sure KNNVectorValues aren't recreated unnecessarily when #2133

Makes sure KNNVectorValues aren't recreated unnecessarily when #2133

Conversation

shatejas commented Sep 20, 2024 • edited Loading

Description

KNNVectorValues creation is optimized to skip for non-quantization cases

Related Issues

Check List

navneet1v commented Sep 23, 2024

navneet1v commented Sep 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shatejas Sep 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shatejas commented Sep 23, 2024

opensearch-trigger-bot bot commented Sep 24, 2024

shatejas commented Sep 20, 2024 •

edited

Loading

shatejas Sep 23, 2024 •

edited

Loading