Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search] Add shard delay aggregation #77423

Merged
merged 9 commits into from
Sep 29, 2020
Merged

Conversation

lukasolson
Copy link
Member

@lukasolson lukasolson commented Sep 15, 2020

Summary

Waiting on https://github.com/elastic/elasticsearch/pull/62082/files.

Adds support for the shard_delay agg type for use in future manual and/or functional tests.

In elastic/elasticsearch#62082, support for a new aggregation, shard_delay, is being added. It is only available in snapshots (not in production). It allows slow queries to be simulated, which is helpful in efforts to ensure Kibana runs smoothly in environments where queries are slow. This PR adds a new AggType for this aggregation which allows you to specify the delay for the query to run.

This new AggType is only registered when the kibana.yml setting, data.search.aggs.shardDelay.enabled is set to true. By default, it is false.

Checklist

  • Unit or functional tests were updated or added to match the most common scenarios
  • Add expression function/registration

@lukasolson lukasolson added Feature:Search Querying infrastructure in Kibana v8.0.0 Team:AppArch v7.10.0 labels Sep 15, 2020
@lukasolson lukasolson self-assigned this Sep 15, 2020
@lukasolson lukasolson added the release_note:skip Skip the PR/issue when compiling release notes label Sep 15, 2020
@lukasolson
Copy link
Member Author

/cc @lizozom @lukeelmers @spalger

@lukasolson lukasolson changed the title [Search] [WIP] Add shard delay aggregation [Search] [skip-ci] Add shard delay aggregation Sep 15, 2020
@lukasolson lukasolson changed the title [Search] [skip-ci] Add shard delay aggregation [skip-ci] [Search] Add shard delay aggregation Sep 15, 2020
@lukeelmers
Copy link
Member

So far this makes sense to me based on our discussion, though we'll need to add an expression function for this agg type as well in order for it to work with the near-future version of esaggs. (Not necessary now, but will be necessary in 7.11, so might be worth getting it out of the way at the same time)

@lukasolson lukasolson marked this pull request as ready for review September 22, 2020 22:29
@lukasolson lukasolson requested a review from a team as a code owner September 22, 2020 22:29
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-arch (Team:AppArch)

@lukasolson
Copy link
Member Author

@lukeelmers Added the expression function, could you give this another look?

@lukasolson lukasolson changed the title [skip-ci] [Search] Add shard delay aggregation [Search] Add shard delay aggregation Sep 22, 2020
Copy link
Member

@lukeelmers lukeelmers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a few minor comments but overall this makes sense to me!

It would be nice if we could add a few comments either where the agg type is registered, or in the agg type definition, or both. Just in case someone runs across this later and isn't sure why it is being handled differently.

Copy link
Member

@lukeelmers lukeelmers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes LGTM.

Tested locally with the setting enabled. This is going to be great for testing!

Screen Shot 2020-09-24 at 5 28 46 PM

Copy link
Contributor

@lizozom lizozom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
As this flag doesn't even appear in kibana.yml, I feel we need some internal documentation on this flag.

@@ -105,13 +112,26 @@ export class SearchService implements Plugin<ISearchSetup, ISearchStart> {
registerUsageCollector(usageCollection, this.initializerContext);
}

const aggs = this.aggsService.setup({ registerFunction });

this.initializerContext.config
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we pass config in instead of the whole initializer context?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the general pattern for plugins/services is to pass in the entire initializer context in the constructor. (I didn't change the signature here to add an additional parameter, just specified what the config schema should look like. I did, however, change the signature on the public service, which matches the autocomplete service.)

@lukasolson
Copy link
Member Author

@lizozom I added a comment in the config file explaining what it does. Other than that I'm not sure exactly where we should/would document the setting. Do you have any suggestions?

@lizozom
Copy link
Contributor

lizozom commented Sep 29, 2020

@lukasolson the comment is really clear, thanks!
What do you think about the initializerContext?

@lizozom
Copy link
Contributor

lizozom commented Sep 29, 2020

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

@kbn/optimizer bundle module count

id value diff baseline
data 568 +2 566

distributable file count

id value diff baseline
default 45849 +2 45847
oss 26573 +2 26571

page load bundle size

id value diff baseline
data 1.3MB +11.3KB 1.3MB

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@lizozom lizozom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lukasolson lukasolson merged commit 75e09bf into elastic:master Sep 29, 2020
lukasolson added a commit to lukasolson/kibana that referenced this pull request Sep 29, 2020
* [Search] [WIP] Add shard delay aggregation

* Add expression functions

* Register function

* Fix test

* Add comment

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Search Querying infrastructure in Kibana release_note:skip Skip the PR/issue when compiling release notes review v7.10.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants