Backport for the field sort optimization of LUCENE-9280 [LUCENE-9384]

[asfimport]

Field sort optimization implemented in #10320 is based on an assumption that if a numeric field is indexed with both doc values and points, the same data is stored in these points and doc values.  While there is  a plan in #10374 to enforce this consistency from Lucene 9.0, there is nothing in Lucene 8.x to enforce this assumption.

 

Thus in order to backport the sort optimization to 8.x, we need to make a user to explicitly opt in for it. This could be done by either:

• introducing a special SortField (e.g. DocValuesPointSortField) that will use optimization
• introduce a bool parameter to a SortField which when true will indicate that the sort optimization should be enabled (e.g. SortField("my_field", SortField.Type.LONG, true))

---

Migrated from LUCENE-9384 by Mayya Sharipova (@mayya-sharipova), resolved Jul 31 2020
Linked issues:

• Add ability to skip non-competitive documents on field sort [LUCENE-9280] #10320

Pull requests: apache/lucene-solr#1610

[asfimport]

ASF subversion and git services (migrated from JIRA)

Commit 7b12849 in lucene-solr's branch refs/heads/branch_8x from Mayya Sharipova
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7b12849

LUCENE-9384: Backport for field sort optimization (#1610)

Backport for: #10320: Collectors to skip noncompetitive documents (#1351)

Similar how scorers can update their iterators to skip non-competitive
documents, collectors and comparators should also provide and update
iterators that allow them to skip non-competive documents.

To enable sort optimization for numeric sort fields,
the following needs to be done:

1. the field should be indexed with both doc_values and points, that
   must have the same field name and same data
2. SortField#setCanUsePoints must be set
3. totalHitsThreshold should not be set to max value.

[asfimport]

Atri Sharma (@atris) (migrated from JIRA)

Closing after the 8.7.0 release