Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow composite aggregation under filter or reverse_nested parent #11499

Merged
merged 5 commits into from
Feb 2, 2024

Conversation

msfroh
Copy link
Collaborator

@msfroh msfroh commented Dec 6, 2023

Description

Composite aggregations are able to run under a filter aggregation with no change required (other than not throwing an exception). Similarly, they can run under a reverse nested aggregation.

Also cleaned up FilterAggregatorFactory a little.

Related Issues

Resolves #11131

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented Dec 6, 2023

Compatibility status:

Checks if related components are compatible with change 6274d2e

Incompatible components

Incompatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/k-nn.git]

Copy link
Contributor

github-actions bot commented Dec 6, 2023

✅ Gradle check result for 65bbddb: SUCCESS

Copy link

codecov bot commented Dec 6, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (c132db9) 71.33% compared to head (6274d2e) 71.31%.
Report is 58 commits behind head on main.

Files Patch % Lines
.../bucket/composite/CompositeAggregationBuilder.java 75.00% 0 Missing and 1 partial ⚠️
...gations/bucket/filter/FilterAggregatorFactory.java 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #11499      +/-   ##
============================================
- Coverage     71.33%   71.31%   -0.02%     
- Complexity    59300    59304       +4     
============================================
  Files          4921     4921              
  Lines        278989   278989              
  Branches      40543    40543              
============================================
- Hits         199014   198973      -41     
- Misses        63444    63483      +39     
- Partials      16531    16533       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Dec 6, 2023

❕ Gradle check result for a491340: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link
Contributor

github-actions bot commented Jan 3, 2024

❌ Gradle check result for f17a66a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions github-actions bot added enhancement Enhancement or improvement to existing feature or request Search:Aggregations Search:Query Capabilities labels Jan 3, 2024
Copy link
Contributor

github-actions bot commented Jan 3, 2024

❕ Gradle check result for c2d4f83: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link
Contributor

❌ Gradle check result for 99f6ee9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Michael Froh <froh@amazon.com>
Copy link
Contributor

❌ Gradle check result for 6274d2e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 6274d2e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for 6274d2e: SUCCESS

@msfroh msfroh changed the title Allow composite aggregation under filter parent Allow composite aggregation under filter or reverse_nested parent Jan 18, 2024
@msfroh
Copy link
Collaborator Author

msfroh commented Jan 18, 2024

This change modifies 3 lines of code:

  • 2 related to supporting the new parent types, and
  • 1 for a long-standing typo in an exception message.

While I jumped through hoops to add unit test coverage for the 2 relevant lines, codecov/patch is failing because I did not add unit test coverage for the typo fix.

I respectfully reject codecov/patch's opinion on this PR.

@reta
Copy link
Collaborator

reta commented Jan 29, 2024

Do we need any documentation updates to say those composite aggregation are now allowed?

@msfroh
Copy link
Collaborator Author

msfroh commented Jan 31, 2024

Do we need any documentation updates to say those composite aggregation are now allowed?

Right now we have no documentation on composite aggregation, as far as I can tell. I think writing the docs can happen independent of this fix.

@msfroh msfroh added backport 2.x Backport to 2.x branch v2.12.0 Issues and PRs related to version 2.12.0 labels Jan 31, 2024
@reta
Copy link
Collaborator

reta commented Jan 31, 2024

I think writing the docs can happen independent of this fix.

Sure, having an issue here https://github.com/opensearch-project/documentation-website/ for 2.12.0 would be enough

@msfroh msfroh merged commit 24b0cc9 into opensearch-project:main Feb 2, 2024
38 of 39 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 2, 2024
…1499)

* Allow composite aggregation under filter parent

Composite aggregations are able to run under a filter aggregation
with no change required (other than not throwing an exception).

Also cleaned up FilterAggregatorFactory a little.

Signed-off-by: Michael Froh <froh@amazon.com>

* Add changelog entry

Signed-off-by: Michael Froh <froh@amazon.com>

* Add support for reverse nested agg too

Signed-off-by: Michael Froh <froh@amazon.com>

* Add unit test coverage

Signed-off-by: Michael Froh <froh@amazon.com>

* Skip new tests in pre-3.0 mixed cluster

Signed-off-by: Michael Froh <froh@amazon.com>

---------

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit 24b0cc9)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@myronmarston
Copy link

Thanks for getting this across the finish line @msfroh!

andrross pushed a commit that referenced this pull request Feb 14, 2024
…1499)

* Allow composite aggregation under filter parent

Composite aggregations are able to run under a filter aggregation
with no change required (other than not throwing an exception).

Also cleaned up FilterAggregatorFactory a little.

Signed-off-by: Michael Froh <froh@amazon.com>

* Add changelog entry

Signed-off-by: Michael Froh <froh@amazon.com>

* Add support for reverse nested agg too

Signed-off-by: Michael Froh <froh@amazon.com>

* Add unit test coverage

Signed-off-by: Michael Froh <froh@amazon.com>

* Skip new tests in pre-3.0 mixed cluster

Signed-off-by: Michael Froh <froh@amazon.com>

---------

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit 24b0cc9)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross added a commit to andrross/OpenSearch that referenced this pull request Feb 14, 2024
This change was not backported to 2.x in time to make the 2.12 release
and therefore will be pending until the 2.13 release.

Signed-off-by: Andrew Ross <andrross@amazon.com>
reta pushed a commit that referenced this pull request Feb 14, 2024
This change was not backported to 2.x in time to make the 2.12 release
and therefore will be pending until the 2.13 release.

Signed-off-by: Andrew Ross <andrross@amazon.com>
msfroh pushed a commit that referenced this pull request Feb 22, 2024
…1499) (#12151)

* Allow composite aggregation under filter parent

Composite aggregations are able to run under a filter aggregation
or a reverse_nested aggreation, with no change required 
(other than not throwing an exception).

Also cleaned up FilterAggregatorFactory a little.

---------


(cherry picked from commit 24b0cc9)

Signed-off-by: Michael Froh <froh@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Mar 1, 2024
…ensearch-project#11499)

* Allow composite aggregation under filter parent

Composite aggregations are able to run under a filter aggregation
with no change required (other than not throwing an exception).

Also cleaned up FilterAggregatorFactory a little.

Signed-off-by: Michael Froh <froh@amazon.com>

* Add changelog entry

Signed-off-by: Michael Froh <froh@amazon.com>

* Add support for reverse nested agg too

Signed-off-by: Michael Froh <froh@amazon.com>

* Add unit test coverage

Signed-off-by: Michael Froh <froh@amazon.com>

* Skip new tests in pre-3.0 mixed cluster

Signed-off-by: Michael Froh <froh@amazon.com>

---------

Signed-off-by: Michael Froh <froh@amazon.com>
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Mar 1, 2024
…h-project#12311)

This change was not backported to 2.x in time to make the 2.12 release
and therefore will be pending until the 2.13 release.

Signed-off-by: Andrew Ross <andrross@amazon.com>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
…ensearch-project#11499)

* Allow composite aggregation under filter parent

Composite aggregations are able to run under a filter aggregation
with no change required (other than not throwing an exception).

Also cleaned up FilterAggregatorFactory a little.

Signed-off-by: Michael Froh <froh@amazon.com>

* Add changelog entry

Signed-off-by: Michael Froh <froh@amazon.com>

* Add support for reverse nested agg too

Signed-off-by: Michael Froh <froh@amazon.com>

* Add unit test coverage

Signed-off-by: Michael Froh <froh@amazon.com>

* Skip new tests in pre-3.0 mixed cluster

Signed-off-by: Michael Froh <froh@amazon.com>

---------

Signed-off-by: Michael Froh <froh@amazon.com>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
…h-project#12311)

This change was not backported to 2.x in time to make the 2.12 release
and therefore will be pending until the 2.13 release.

Signed-off-by: Andrew Ross <andrross@amazon.com>
@ap-h
Copy link

ap-h commented Mar 26, 2024

Hi all, thanks for the great work! Do you know when this feature will be available in AWS OpenSearch service? Currently the latest supported version is 2.11. Feel free to ignore if it's unrelated, thanks!

shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…ensearch-project#11499)

* Allow composite aggregation under filter parent

Composite aggregations are able to run under a filter aggregation
with no change required (other than not throwing an exception).

Also cleaned up FilterAggregatorFactory a little.

Signed-off-by: Michael Froh <froh@amazon.com>

* Add changelog entry

Signed-off-by: Michael Froh <froh@amazon.com>

* Add support for reverse nested agg too

Signed-off-by: Michael Froh <froh@amazon.com>

* Add unit test coverage

Signed-off-by: Michael Froh <froh@amazon.com>

* Skip new tests in pre-3.0 mixed cluster

Signed-off-by: Michael Froh <froh@amazon.com>

---------

Signed-off-by: Michael Froh <froh@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…h-project#12311)

This change was not backported to 2.x in time to make the 2.12 release
and therefore will be pending until the 2.13 release.

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch enhancement Enhancement or improvement to existing feature or request Search:Aggregations Search:Query Capabilities v2.12.0 Issues and PRs related to version 2.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Composite aggregation cannot be used under a filter aggregation
4 participants