Add changes for AVX-512 support in k-NN.

[akashsha1]

Description

This change adds support to speed up vector search and indexing in faiss using AVX512 hardware accelerator.

Related Issues

Resolves #2056

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created. Update documentation to reflect k-NN FAISS AVX512 support  documentation-website#8307

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[assanedi]

Benchmark was run using opensearch-benchmark with cohere dataset(768 dimensions).
Her are some configuration details for indexing:
{
"target_index_name": "target_index",
"target_field_name": "target_field",
"target_index_body": "indices/faiss-index.json",
"target_index_primary_shards": 4,
"target_index_replica_shards": 1,
"target_index_dimension": 768,
"target_index_space_type": "innerproduct",
"target_index_bulk_size": 100,
"target_index_bulk_index_data_set_format": "hdf5",
"target_index_bulk_index_data_set_path": "/mnt/nvme1/documents-1m.hdf5",
"target_index_bulk_indexing_clients": 20,
"target_index_max_num_segments": 1,
"hnsw_ef_search": 256,
"hnsw_ef_construction": 256
}

Her are some configuration details for search:
{
"target_index_name": "target_index",
"target_field_name": "target_field",
"query_k": 100,
"query_body": {
"docvalue_fields" : ["_id"],
"stored_fields" : "none"
},
"query_data_set_format": "hdf5",
"query_data_set_path": "/mnt/nvme1/queries-1m-100k.hdf5",
"query_count": 30000,
"search_clients": 20
}

A forcemerge to reduce the number of max_num_segments to 1 is executed via the API before the seach.

The opensearch cluster was deployed with 2 data nodes (r7i.2xlarges), 1 replica and 4 shards.
Using this setup AVX512 shows 15% improvement over AVX2 on indexing and 7 % on search as shown below:

[naveentatikonda]

Benchmark was run using opensearch-benchmark with cohere dataset(768 dimensions). The opensearch cluster was deployed with 2 data nodes (r7i.2xlarges), 1 replica and 4 shards. Using this setup AVX512 shows 15% improvement over AVX2 on indexing and 7 % on search as shown below:

@assanedi Can you also pls add other configuration details like the indexing clients, query clients, ef_construction, ef_search, etc

[naveentatikonda]

LGTM! Thanks @akashsha1 @assanedi

[naveentatikonda]

"target_index_bulk_indexing_clients": 20,
"target_index_max_num_segments": 10,
"hnsw_ef_search": 256,
"hnsw_ef_construction": 256

@assanedi Isn't the max_num_segments was 1 during forcemerge ?

[assanedi]

"target_index_bulk_indexing_clients": 20,
"target_index_max_num_segments": 10,
"hnsw_ef_search": 256,
"hnsw_ef_construction": 256

@assanedi Isn't the max_num_segments was 1 during forcemerge ?

Yes I run the forcemerge API, here is the results of it:
curl -X POST -k --user admin:admin http://10.0.0.80:9200/_forcemerge?max_num_segments=1
{"_shards":{"total":8,"successful":8,"failed":0}}

[naveentatikonda]

Yes I run the forcemerge API, here is the results of it: curl -X POST -k --user admin:admin http://10.0.0.80:9200/_forcemerge?max_num_segments=1 {"_shards":{"total":8,"successful":8,"failed":0}}

Yes, but in the configuration you mentioned it as 10 instead of 1 for target_index_max_num_segments

[naveentatikonda]

For FP32 we don’t need to make any changes in Faiss as they are using auto-vectorization to achieve the optimization with AVX512. But, for Scalar Quantization Intel have raised a PR to Faiss which is under review
facebookresearch/faiss#3853

[assanedi]

Yes I run the forcemerge API, here is the results of it: curl -X POST -k --user admin:admin http://10.0.0.80:9200/_forcemerge?max_num_segments=1 {"_shards":{"total":8,"successful":8,"failed":0}}

Yes, but in the configuration you mentioned it as 10 instead of 1 for target_index_max_num_segments

I updated the configuration details

[jmazanec15]

Shouldnt these be mutually exclusive?

[akashsha1]

Ideally we want simd (avx2, avx512 - highest of whichever is present) to be enabled by default. The order of checks would be:
if (AVX512 enabled and present) { use avx512 }
else if (AVX2 enabled and present) { use avx2 }
else { use generic version }

by making it mutually exclusive, the intermediate step cannot be achieved.

[naveentatikonda]

I think Jack's point is to update it as something like this ./gradlew build -Davx2.enabled=false -Davx512.enabled=true as we can't use both of them at the same time

[jmazanec15]

Can we do proper null checks here? In general, I think its best to avoid catching all exceptions.

[akashsha1]

Looks like java boolean cannot be null. So null check won't be possible.
Your second point on exceptions is valid, and this code shouldn't throw exceptions as a default value is set. I've removed the try/catch block.

[naveentatikonda]

@akashsha1 as users will manually set this setting in opensearch.yml, there is a chance of getting null. To avoid it shall we change it to
return Booleans.parseBoolean(KNNSettings.state().getSettingValue(KNNSettings.KNN_FAISS_AVX512_DISABLED).toString(), false);

[naveentatikonda]

Booleans.parseBoolean will do the null validation and if it is null, it will return the default value

[akashsha1]

spoke with Naveen on slack, and updated to
return Booleans.parseBoolean(KNNSettings.state().getSettingValue(KNNSettings.KNN_FAISS_AVX512_DISABLED).toString(), KNN_DEFAULT_FAISS_AVX512_DISABLED_VALUE);

[jmazanec15]

LGTM thanks