diff --git a/docs/apm/images/apm-services-overview.png b/docs/apm/images/apm-services-overview.png index 85b441d47f0c2f..3a56d597abfb79 100644 Binary files a/docs/apm/images/apm-services-overview.png and b/docs/apm/images/apm-services-overview.png differ diff --git a/docs/apm/images/apm-traces.png b/docs/apm/images/apm-traces.png index 97e801606c613d..ed15423b42c511 100644 Binary files a/docs/apm/images/apm-traces.png and b/docs/apm/images/apm-traces.png differ diff --git a/docs/apm/images/apm-transactions-overview.png b/docs/apm/images/apm-transactions-overview.png index 80fca19ff96cc0..1b25668f0fd92a 100644 Binary files a/docs/apm/images/apm-transactions-overview.png and b/docs/apm/images/apm-transactions-overview.png differ diff --git a/docs/apm/images/traffic-transactions.png b/docs/apm/images/traffic-transactions.png index 134bc0e6bcb42c..ef429740ceee31 100644 Binary files a/docs/apm/images/traffic-transactions.png and b/docs/apm/images/traffic-transactions.png differ diff --git a/docs/apm/service-overview.asciidoc b/docs/apm/service-overview.asciidoc index 088791e6098e6d..5fd214e6ce613a 100644 --- a/docs/apm/service-overview.asciidoc +++ b/docs/apm/service-overview.asciidoc @@ -17,8 +17,8 @@ Response times for the service. You can filter the *Latency* chart to display th image::apm/images/latency.png[Service latency] [discrete] -[[service-traffic-transactions]] -=== Traffic and transactions +[[service-throughput-transactions]] +=== Throughput and transactions The *Throughput* chart visualizes the average number of transactions per minute for the selected service. @@ -62,6 +62,9 @@ each dependency. By default, dependencies are sorted by _Impact_ to show the mos If there is a particular dependency you are interested in, click *View service map* to view the related <>. +IMPORTANT: A known issue prevents Real User Monitoring (RUM) dependencies from being shown in the +*Dependencies* table. We are working on a fix for this issue. + [role="screenshot"] image::apm/images/spans-dependencies.png[Span type duration and dependencies] diff --git a/docs/apm/transactions.asciidoc b/docs/apm/transactions.asciidoc index 83ca9e5a10a9b0..8c8da81aa577e1 100644 --- a/docs/apm/transactions.asciidoc +++ b/docs/apm/transactions.asciidoc @@ -17,10 +17,10 @@ If there's a weird spike that you'd like to investigate, you can simply zoom in on the graph - this will adjust the specific time range, and all of the data on the page will update accordingly. -*Transactions per minute*:: -Visualize response codes: `2xx`, `3xx`, `4xx`, etc., -and is useful for determining if you're serving more of one code than you typically do. -Like in the Transaction duration graph, you can zoom in on anomalies to further investigate them. +*Throughput*:: +Visualize response codes: `2xx`, `3xx`, `4xx`, etc. +Useful for determining if more responses than usual are being served with a particular response code. +Like in the latency graph, you can zoom in on anomalies to further investigate them. *Error rate*:: Visualize the total number of transactions with errors divided by the total number of transactions. @@ -157,4 +157,4 @@ and solve problems. [role="screenshot"] image::apm/images/apm-logs-tab.png[APM logs tab] -// To do: link to log correlation \ No newline at end of file +// To do: link to log correlation diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index 7084777cbb6f95..465a3d652046db 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -1,14 +1,30 @@ [[troubleshooting]] -== Troubleshoot common problems +== Troubleshooting ++++ Troubleshooting ++++ -If you have something to add to this section, please consider creating a pull request with -your proposed changes at https://github.com/elastic/kibana. - -Also, check out the https://discuss.elastic.co/c/apm[APM discussion forum]. +This section describes common problems you might encounter with the APM app. +To add to this page, please consider opening a +https://github.com/elastic/kibana/pulls[pull request] with your proposed changes. + +If your issue is potentially related to other components of the APM ecosystem, +don't forget to check our other troubleshooting guides or discussion forum: + +* {apm-server-ref}/troubleshooting.html[APM Server troubleshooting] +* {apm-dotnet-ref}/troubleshooting.html[.NET agent troubleshooting] +* {apm-go-ref}/troubleshooting.html[Go agent troubleshooting] +* {apm-java-ref}/trouble-shooting.html[Java agent troubleshooting] +* {apm-node-ref}/troubleshooting.html[Node.js agent troubleshooting] +* {apm-py-ref}/troubleshooting.html[Python agent troubleshooting] +* {apm-ruby-ref}/debugging.html[Ruby agent troubleshooting] +* {apm-rum-ref/troubleshooting.html[RUM troubleshooting] +* https://discuss.elastic.co/c/apm[APM discussion forum]. + +[discrete] +[[troubleshooting-apm-app]] +== Troubleshoot common APM app problems * <> * <> diff --git a/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md b/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md index 1ba359e81b9c62..a854e5ddad19a6 100644 --- a/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md +++ b/docs/development/core/server/kibana-plugin-core-server.elasticsearchclientconfig.md @@ -9,7 +9,7 @@ Configuration options to be used to create a [cluster client](./kibana-plugin-co Signature: ```typescript -export declare type ElasticsearchClientConfig = Pick & { +export declare type ElasticsearchClientConfig = Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ClientOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ClientOptions['requestTimeout']; ssl?: Partial; diff --git a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.logqueries.md b/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.logqueries.md deleted file mode 100644 index 001fb7bfeeb97d..00000000000000 --- a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.logqueries.md +++ /dev/null @@ -1,13 +0,0 @@ - - -[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [ElasticsearchConfig](./kibana-plugin-core-server.elasticsearchconfig.md) > [logQueries](./kibana-plugin-core-server.elasticsearchconfig.logqueries.md) - -## ElasticsearchConfig.logQueries property - -Specifies whether all queries to the client should be logged (status code, method, query etc.). - -Signature: - -```typescript -readonly logQueries: boolean; -``` diff --git a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md b/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md index 5ec3ce7f41859b..d87ea63d59b8dd 100644 --- a/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md +++ b/docs/development/core/server/kibana-plugin-core-server.elasticsearchconfig.md @@ -27,7 +27,6 @@ export declare class ElasticsearchConfig | [healthCheckDelay](./kibana-plugin-core-server.elasticsearchconfig.healthcheckdelay.md) | | Duration | The interval between health check requests Kibana sends to the Elasticsearch. | | [hosts](./kibana-plugin-core-server.elasticsearchconfig.hosts.md) | | string[] | Hosts that the client will connect to. If sniffing is enabled, this list will be used as seeds to discover the rest of your cluster. | | [ignoreVersionMismatch](./kibana-plugin-core-server.elasticsearchconfig.ignoreversionmismatch.md) | | boolean | Whether to allow kibana to connect to a non-compatible elasticsearch node. | -| [logQueries](./kibana-plugin-core-server.elasticsearchconfig.logqueries.md) | | boolean | Specifies whether all queries to the client should be logged (status code, method, query etc.). | | [password](./kibana-plugin-core-server.elasticsearchconfig.password.md) | | string | If Elasticsearch is protected with basic authentication, this setting provides the password that the Kibana server uses to perform its administrative functions. | | [pingTimeout](./kibana-plugin-core-server.elasticsearchconfig.pingtimeout.md) | | Duration | Timeout after which PING HTTP request will be aborted and retried. | | [requestHeadersWhitelist](./kibana-plugin-core-server.elasticsearchconfig.requestheaderswhitelist.md) | | string[] | List of Kibana client-side headers to send to Elasticsearch when request scoped cluster client is used. If this is an empty array then \*no\* client-side will be sent. | diff --git a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md index 823f34bd7dd237..ed2763d9802797 100644 --- a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md +++ b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient._constructor_.md @@ -9,7 +9,7 @@ Constructs a new instance of the `LegacyClusterClient` class Signature: ```typescript -constructor(config: LegacyElasticsearchClientConfig, log: Logger, getAuthHeaders?: GetAuthHeaders); +constructor(config: LegacyElasticsearchClientConfig, log: Logger, type: string, getAuthHeaders?: GetAuthHeaders); ``` ## Parameters @@ -18,5 +18,6 @@ constructor(config: LegacyElasticsearchClientConfig, log: Logger, getAuthHeaders | --- | --- | --- | | config | LegacyElasticsearchClientConfig | | | log | Logger | | +| type | string | | | getAuthHeaders | GetAuthHeaders | | diff --git a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md index d24aeb44ca86a6..0872e5ba7c2197 100644 --- a/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md +++ b/docs/development/core/server/kibana-plugin-core-server.legacyclusterclient.md @@ -21,7 +21,7 @@ export declare class LegacyClusterClient implements ILegacyClusterClient | Constructor | Modifiers | Description | | --- | --- | --- | -| [(constructor)(config, log, getAuthHeaders)](./kibana-plugin-core-server.legacyclusterclient._constructor_.md) | | Constructs a new instance of the LegacyClusterClient class | +| [(constructor)(config, log, type, getAuthHeaders)](./kibana-plugin-core-server.legacyclusterclient._constructor_.md) | | Constructs a new instance of the LegacyClusterClient class | ## Properties diff --git a/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md b/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md index 78f7bf582d355e..b028a09bee4531 100644 --- a/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md +++ b/docs/development/core/server/kibana-plugin-core-server.legacyelasticsearchclientconfig.md @@ -11,7 +11,7 @@ Signature: ```typescript -export declare type LegacyElasticsearchClientConfig = Pick & Pick & { +export declare type LegacyElasticsearchClientConfig = Pick & Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ConfigOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ConfigOptions['requestTimeout']; sniffInterval?: ElasticsearchConfig['sniffInterval'] | ConfigOptions['sniffInterval']; diff --git a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md new file mode 100644 index 00000000000000..937e20a7a95795 --- /dev/null +++ b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md @@ -0,0 +1,13 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) > [ISearchOptions](./kibana-plugin-plugins-data-public.isearchoptions.md) > [legacyHitsTotal](./kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md) + +## ISearchOptions.legacyHitsTotal property + +Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to something other than `true`, this should be set to `false`. + +Signature: + +```typescript +legacyHitsTotal?: boolean; +``` diff --git a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md index 5acd837495dac4..fc2767cd0231f9 100644 --- a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md +++ b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md @@ -17,6 +17,7 @@ export interface ISearchOptions | [abortSignal](./kibana-plugin-plugins-data-public.isearchoptions.abortsignal.md) | AbortSignal | An AbortSignal that allows the caller of search to abort a search request. | | [isRestore](./kibana-plugin-plugins-data-public.isearchoptions.isrestore.md) | boolean | Whether the session is restored (i.e. search requests should re-use the stored search IDs, rather than starting from scratch) | | [isStored](./kibana-plugin-plugins-data-public.isearchoptions.isstored.md) | boolean | Whether the session is already saved (i.e. sent to background) | +| [legacyHitsTotal](./kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md) | boolean | Request the legacy format for the total number of hits. If sending rest_total_hits_as_int to something other than true, this should be set to false. | | [sessionId](./kibana-plugin-plugins-data-public.isearchoptions.sessionid.md) | string | A session ID, grouping multiple search requests into a single session. | | [strategy](./kibana-plugin-plugins-data-public.isearchoptions.strategy.md) | string | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. | diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md new file mode 100644 index 00000000000000..eaac671b9a1826 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md @@ -0,0 +1,18 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) + +## IScopedSessionService interface + +Signature: + +```typescript +export interface IScopedSessionService +``` + +## Properties + +| Property | Type | Description | +| --- | --- | --- | +| [search](./kibana-plugin-plugins-data-server.iscopedsessionservice.search.md) | <Request extends IKibanaSearchRequest, Response extends IKibanaSearchResponse>(strategy: ISearchStrategy<Request, Response>, ...args: Parameters<ISearchStrategy<Request, Response>['search']>) => Observable<Response> | | + diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md new file mode 100644 index 00000000000000..d58a9fd9f37619 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md @@ -0,0 +1,11 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) > [search](./kibana-plugin-plugins-data-server.iscopedsessionservice.search.md) + +## IScopedSessionService.search property + +Signature: + +```typescript +search: (strategy: ISearchStrategy, ...args: Parameters['search']>) => Observable; +``` diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md new file mode 100644 index 00000000000000..59b8b2c6b446f6 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md @@ -0,0 +1,13 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) > [legacyHitsTotal](./kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md) + +## ISearchOptions.legacyHitsTotal property + +Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to something other than `true`, this should be set to `false`. + +Signature: + +```typescript +legacyHitsTotal?: boolean; +``` diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md index 85847e1c61d25b..9de351b2b90194 100644 --- a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md @@ -17,6 +17,7 @@ export interface ISearchOptions | [abortSignal](./kibana-plugin-plugins-data-server.isearchoptions.abortsignal.md) | AbortSignal | An AbortSignal that allows the caller of search to abort a search request. | | [isRestore](./kibana-plugin-plugins-data-server.isearchoptions.isrestore.md) | boolean | Whether the session is restored (i.e. search requests should re-use the stored search IDs, rather than starting from scratch) | | [isStored](./kibana-plugin-plugins-data-server.isearchoptions.isstored.md) | boolean | Whether the session is already saved (i.e. sent to background) | +| [legacyHitsTotal](./kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md) | boolean | Request the legacy format for the total number of hits. If sending rest_total_hits_as_int to something other than true, this should be set to false. | | [sessionId](./kibana-plugin-plugins-data-server.isearchoptions.sessionid.md) | string | A session ID, grouping multiple search requests into a single session. | | [strategy](./kibana-plugin-plugins-data-server.isearchoptions.strategy.md) | string | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. | diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md index 84c7875c26ce89..27a386a714fc1a 100644 --- a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md @@ -53,6 +53,7 @@ | [IFieldSubType](./kibana-plugin-plugins-data-server.ifieldsubtype.md) | | | [IFieldType](./kibana-plugin-plugins-data-server.ifieldtype.md) | | | [IndexPatternAttributes](./kibana-plugin-plugins-data-server.indexpatternattributes.md) | Interface for an index pattern saved object | +| [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) | | | [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) | | | [ISearchSetup](./kibana-plugin-plugins-data-server.isearchsetup.md) | | | [ISearchStart](./kibana-plugin-plugins-data-server.isearchstart.md) | | diff --git a/docs/maps/maps-getting-started.asciidoc b/docs/maps/maps-getting-started.asciidoc index 32a81c8e65f565..609a133c92ad1c 100644 --- a/docs/maps/maps-getting-started.asciidoc +++ b/docs/maps/maps-getting-started.asciidoc @@ -1,15 +1,11 @@ [role="xpack"] [[maps-getting-started]] -== Create a map with multiple layers and data sources - -++++ -Create a multilayer map -++++ +== Build a map to compare metrics by country or region If you are new to **Maps**, this tutorial is a good place to start. It guides you through the common steps for working with your location data. -You'll learn to: +You will learn to: - Create a map with multiple layers and data sources - Use symbols, colors, and labels to style data values diff --git a/docs/setup/settings.asciidoc b/docs/setup/settings.asciidoc index 26f095c59c6444..ecdb41c897b125 100644 --- a/docs/setup/settings.asciidoc +++ b/docs/setup/settings.asciidoc @@ -59,7 +59,7 @@ To enable SSL/TLS for outbound connections to {es}, use the `https` protocol in this setting. | `elasticsearch.logQueries:` - | Log queries sent to {es}. Requires <> set to `true`. + | *deprecated* This setting is no longer used and will get removed in Kibana 8.0. Instead, set <> to `true` This is useful for seeing the query DSL generated by applications that currently do not have an inspector, for example Timelion and Monitoring. *Default: `false`* diff --git a/packages/kbn-es-archiver/src/actions/load.ts b/packages/kbn-es-archiver/src/actions/load.ts index 8afd7d79a98f73..c32496ad426945 100644 --- a/packages/kbn-es-archiver/src/actions/load.ts +++ b/packages/kbn-es-archiver/src/actions/load.ts @@ -98,9 +98,11 @@ export async function loadAction({ // If we affected the Kibana index, we need to ensure it's migrated... if (Object.keys(result).some((k) => k.startsWith('.kibana'))) { await migrateKibanaIndex({ client, kbnClient }); + log.debug('[%s] Migrated Kibana index after loading Kibana data', name); if (kibanaPluginIds.includes('spaces')) { await createDefaultSpace({ client, index: '.kibana' }); + log.debug('[%s] Ensured that default space exists in .kibana', name); } } diff --git a/packages/kbn-es/src/cluster.js b/packages/kbn-es/src/cluster.js index f554dd8a1b8e59..5948e9ecececc4 100644 --- a/packages/kbn-es/src/cluster.js +++ b/packages/kbn-es/src/cluster.js @@ -246,7 +246,7 @@ exports.Cluster = class Cluster { this._log.info(chalk.bold('Starting')); this._log.indent(4); - const esArgs = ['indices.query.bool.max_nested_depth=100'].concat(options.esArgs || []); + const esArgs = options.esArgs || []; // Add to esArgs if ssl is enabled if (this._ssl) { diff --git a/packages/kbn-es/src/integration_tests/cluster.test.js b/packages/kbn-es/src/integration_tests/cluster.test.js index c1adc84ddc9548..684667355852d6 100644 --- a/packages/kbn-es/src/integration_tests/cluster.test.js +++ b/packages/kbn-es/src/integration_tests/cluster.test.js @@ -264,9 +264,7 @@ describe('#start(installPath)', () => { expect(extractConfigFiles.mock.calls).toMatchInlineSnapshot(` Array [ Array [ - Array [ - "indices.query.bool.max_nested_depth=100", - ], + Array [], undefined, Object { "log": , @@ -342,9 +340,7 @@ describe('#run()', () => { expect(extractConfigFiles.mock.calls).toMatchInlineSnapshot(` Array [ Array [ - Array [ - "indices.query.bool.max_nested_depth=100", - ], + Array [], undefined, Object { "log": , diff --git a/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap b/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap index c836686ec602b9..80e23a32ca5570 100644 --- a/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap +++ b/src/core/public/chrome/ui/header/__snapshots__/collapsible_nav.test.tsx.snap @@ -543,6 +543,7 @@ exports[`CollapsibleNav renders links grouped by category 1`] = ` listItems={ Array [ Object { + "data-test-subj": "homeLink", "href": "/", "iconType": "home", "label": "Home", @@ -564,6 +565,7 @@ exports[`CollapsibleNav renders links grouped by category 1`] = ` > { if (isModifiedOrPrevented(event)) { return; diff --git a/src/core/server/elasticsearch/client/client_config.test.ts b/src/core/server/elasticsearch/client/client_config.test.ts index 57bc7407a9a0fd..768d165d5f8bed 100644 --- a/src/core/server/elasticsearch/client/client_config.test.ts +++ b/src/core/server/elasticsearch/client/client_config.test.ts @@ -15,7 +15,6 @@ const createConfig = ( ): ElasticsearchClientConfig => { return { customHeaders: {}, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, sniffInterval: false, diff --git a/src/core/server/elasticsearch/client/client_config.ts b/src/core/server/elasticsearch/client/client_config.ts index 5762ef16704a5a..01d2222a45e3a6 100644 --- a/src/core/server/elasticsearch/client/client_config.ts +++ b/src/core/server/elasticsearch/client/client_config.ts @@ -22,7 +22,6 @@ import { DEFAULT_HEADERS } from '../default_headers'; export type ElasticsearchClientConfig = Pick< ElasticsearchConfig, | 'customHeaders' - | 'logQueries' | 'sniffOnStart' | 'sniffOnConnectionFault' | 'requestHeadersWhitelist' diff --git a/src/core/server/elasticsearch/client/cluster_client.test.ts b/src/core/server/elasticsearch/client/cluster_client.test.ts index b94bf08f1185b6..1d6d373ec185cb 100644 --- a/src/core/server/elasticsearch/client/cluster_client.test.ts +++ b/src/core/server/elasticsearch/client/cluster_client.test.ts @@ -19,7 +19,6 @@ const createConfig = ( parts: Partial = {} ): ElasticsearchClientConfig => { return { - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, sniffInterval: false, @@ -57,16 +56,25 @@ describe('ClusterClient', () => { it('creates a single internal and scoped client during initialization', () => { const config = createConfig(); - new ClusterClient(config, logger, getAuthHeaders); + new ClusterClient(config, logger, 'custom-type', getAuthHeaders); expect(configureClientMock).toHaveBeenCalledTimes(2); - expect(configureClientMock).toHaveBeenCalledWith(config, { logger }); - expect(configureClientMock).toHaveBeenCalledWith(config, { logger, scoped: true }); + expect(configureClientMock).toHaveBeenCalledWith(config, { logger, type: 'custom-type' }); + expect(configureClientMock).toHaveBeenCalledWith(config, { + logger, + type: 'custom-type', + scoped: true, + }); }); describe('#asInternalUser', () => { it('returns the internal client', () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); expect(clusterClient.asInternalUser).toBe(internalClient); }); @@ -74,7 +82,12 @@ describe('ClusterClient', () => { describe('#asScoped', () => { it('returns a scoped cluster client bound to the request', () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); const request = httpServerMock.createKibanaRequest(); const scopedClusterClient = clusterClient.asScoped(request); @@ -87,7 +100,12 @@ describe('ClusterClient', () => { }); it('returns a distinct scoped cluster client on each call', () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); const request = httpServerMock.createKibanaRequest(); const scopedClusterClient1 = clusterClient.asScoped(request); @@ -105,7 +123,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { foo: 'bar', @@ -130,7 +148,7 @@ describe('ClusterClient', () => { other: 'nope', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({}); clusterClient.asScoped(request); @@ -150,7 +168,7 @@ describe('ClusterClient', () => { other: 'nope', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { authorization: 'override', @@ -175,7 +193,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({}); clusterClient.asScoped(request); @@ -195,7 +213,7 @@ describe('ClusterClient', () => { const config = createConfig(); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ kibanaRequestState: { requestId: 'my-fake-id', requestUuid: 'ignore-this-id' }, }); @@ -223,7 +241,7 @@ describe('ClusterClient', () => { foo: 'auth', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({}); clusterClient.asScoped(request); @@ -249,7 +267,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { foo: 'request' }, }); @@ -276,7 +294,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest(); clusterClient.asScoped(request); @@ -297,7 +315,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { [headerKey]: 'foo' }, }); @@ -321,7 +339,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = httpServerMock.createKibanaRequest({ headers: { foo: 'request' }, kibanaRequestState: { requestId: 'from request', requestUuid: 'ignore-this-id' }, @@ -344,7 +362,7 @@ describe('ClusterClient', () => { }); getAuthHeaders.mockReturnValue({}); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = { headers: { authorization: 'auth', @@ -368,7 +386,7 @@ describe('ClusterClient', () => { authorization: 'auth', }); - const clusterClient = new ClusterClient(config, logger, getAuthHeaders); + const clusterClient = new ClusterClient(config, logger, 'custom-type', getAuthHeaders); const request = { headers: { foo: 'bar', @@ -387,7 +405,12 @@ describe('ClusterClient', () => { describe('#close', () => { it('closes both underlying clients', async () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); await clusterClient.close(); @@ -398,7 +421,12 @@ describe('ClusterClient', () => { it('waits for both clients to close', async (done) => { expect.assertions(4); - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); let internalClientClosed = false; let scopedClientClosed = false; @@ -436,7 +464,12 @@ describe('ClusterClient', () => { }); it('return a rejected promise is any client rejects', async () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); internalClient.close.mockRejectedValue(new Error('error closing client')); @@ -446,7 +479,12 @@ describe('ClusterClient', () => { }); it('does nothing after the first call', async () => { - const clusterClient = new ClusterClient(createConfig(), logger, getAuthHeaders); + const clusterClient = new ClusterClient( + createConfig(), + logger, + 'custom-type', + getAuthHeaders + ); await clusterClient.close(); diff --git a/src/core/server/elasticsearch/client/cluster_client.ts b/src/core/server/elasticsearch/client/cluster_client.ts index 87d59e7417aa9d..7e6a7f8ae53e6a 100644 --- a/src/core/server/elasticsearch/client/cluster_client.ts +++ b/src/core/server/elasticsearch/client/cluster_client.ts @@ -60,10 +60,11 @@ export class ClusterClient implements ICustomClusterClient { constructor( private readonly config: ElasticsearchClientConfig, logger: Logger, + type: string, private readonly getAuthHeaders: GetAuthHeaders = noop ) { - this.asInternalUser = configureClient(config, { logger }); - this.rootScopedClient = configureClient(config, { logger, scoped: true }); + this.asInternalUser = configureClient(config, { logger, type }); + this.rootScopedClient = configureClient(config, { logger, type, scoped: true }); } asScoped(request: ScopeableRequest) { diff --git a/src/core/server/elasticsearch/client/configure_client.test.ts b/src/core/server/elasticsearch/client/configure_client.test.ts index 3486c210de1f99..548dc44aa4965a 100644 --- a/src/core/server/elasticsearch/client/configure_client.test.ts +++ b/src/core/server/elasticsearch/client/configure_client.test.ts @@ -76,14 +76,14 @@ describe('configureClient', () => { }); it('calls `parseClientOptions` with the correct parameters', () => { - configureClient(config, { logger, scoped: false }); + configureClient(config, { logger, type: 'test', scoped: false }); expect(parseClientOptionsMock).toHaveBeenCalledTimes(1); expect(parseClientOptionsMock).toHaveBeenCalledWith(config, false); parseClientOptionsMock.mockClear(); - configureClient(config, { logger, scoped: true }); + configureClient(config, { logger, type: 'test', scoped: true }); expect(parseClientOptionsMock).toHaveBeenCalledTimes(1); expect(parseClientOptionsMock).toHaveBeenCalledWith(config, true); @@ -95,7 +95,7 @@ describe('configureClient', () => { }; parseClientOptionsMock.mockReturnValue(parsedOptions); - const client = configureClient(config, { logger, scoped: false }); + const client = configureClient(config, { logger, type: 'test', scoped: false }); expect(ClientMock).toHaveBeenCalledTimes(1); expect(ClientMock).toHaveBeenCalledWith(parsedOptions); @@ -103,7 +103,7 @@ describe('configureClient', () => { }); it('listens to client on `response` events', () => { - const client = configureClient(config, { logger, scoped: false }); + const client = configureClient(config, { logger, type: 'test', scoped: false }); expect(client.on).toHaveBeenCalledTimes(1); expect(client.on).toHaveBeenCalledWith('response', expect.any(Function)); @@ -122,38 +122,15 @@ describe('configureClient', () => { }, }); } - describe('does not log whrn "logQueries: false"', () => { - it('response', () => { - const client = configureClient(config, { logger, scoped: false }); - const response = createResponseWithBody({ - seq_no_primary_term: true, - query: { - term: { user: 'kimchy' }, - }, - }); - - client.emit('response', null, response); - expect(loggingSystemMock.collect(logger).debug).toHaveLength(0); - }); - - it('error', () => { - const client = configureClient(config, { logger, scoped: false }); - - const response = createApiResponse({ body: {} }); - client.emit('response', new errors.TimeoutError('message', response), response); - expect(loggingSystemMock.collect(logger).error).toHaveLength(0); + describe('logs each query', () => { + it('creates a query logger context based on the `type` parameter', () => { + configureClient(createFakeConfig(), { logger, type: 'test123' }); + expect(logger.get).toHaveBeenCalledWith('query', 'test123'); }); - }); - describe('logs each queries if `logQueries` is true', () => { it('when request body is an object', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody({ seq_no_primary_term: true, @@ -169,23 +146,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly {\\"seq_no_primary_term\\":true,\\"query\\":{\\"term\\":{\\"user\\":\\"kimchy\\"}}}", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is a string', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody( JSON.stringify({ @@ -203,23 +170,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly {\\"seq_no_primary_term\\":true,\\"query\\":{\\"term\\":{\\"user\\":\\"kimchy\\"}}}", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is a buffer', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody( Buffer.from( @@ -239,23 +196,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly [buffer]", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is a readable stream', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody( Readable.from( @@ -275,23 +222,13 @@ describe('configureClient', () => { "200 GET /foo?hello=dolly [stream]", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('when request body is not defined', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createResponseWithBody(); @@ -301,23 +238,13 @@ describe('configureClient', () => { Array [ "200 GET /foo?hello=dolly", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); it('properly encode queries', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ body: {}, @@ -336,23 +263,13 @@ describe('configureClient', () => { Array [ "200 GET /foo?city=M%C3%BCnich", - Object { - "tags": Array [ - "query", - ], - }, ], ] `); }); - it('logs queries even in case of errors if `logQueries` is true', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + it('logs queries even in case of errors', () => { + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ statusCode: 500, @@ -375,7 +292,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "500 @@ -386,40 +303,13 @@ describe('configureClient', () => { `); }); - it('does not log queries if `logQueries` is false', () => { - const client = configureClient( - createFakeConfig({ - logQueries: false, - }), - { logger, scoped: false } - ); - - const response = createApiResponse({ - body: {}, - statusCode: 200, - params: { - method: 'GET', - path: '/foo', - }, - }); - - client.emit('response', null, response); - - expect(logger.debug).not.toHaveBeenCalled(); - }); - - it('logs error when the client emits an @elastic/elasticsearch error', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + it('logs debug when the client emits an @elastic/elasticsearch error', () => { + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ body: {} }); client.emit('response', new errors.TimeoutError('message', response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "[TimeoutError]: message", @@ -428,13 +318,8 @@ describe('configureClient', () => { `); }); - it('logs error when the client emits an ResponseError returned by elasticsearch', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + it('logs debug when the client emits an ResponseError returned by elasticsearch', () => { + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); const response = createApiResponse({ statusCode: 400, @@ -453,7 +338,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "400 @@ -464,12 +349,7 @@ describe('configureClient', () => { }); it('logs default error info when the error response body is empty', () => { - const client = configureClient( - createFakeConfig({ - logQueries: true, - }), - { logger, scoped: false } - ); + const client = configureClient(createFakeConfig(), { logger, type: 'test', scoped: false }); let response = createApiResponse({ statusCode: 400, @@ -484,7 +364,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "400 @@ -493,7 +373,7 @@ describe('configureClient', () => { ] `); - logger.error.mockClear(); + logger.debug.mockClear(); response = createApiResponse({ statusCode: 400, @@ -506,7 +386,7 @@ describe('configureClient', () => { }); client.emit('response', new errors.ResponseError(response), response); - expect(loggingSystemMock.collect(logger).error).toMatchInlineSnapshot(` + expect(loggingSystemMock.collect(logger).debug).toMatchInlineSnapshot(` Array [ Array [ "400 diff --git a/src/core/server/elasticsearch/client/configure_client.ts b/src/core/server/elasticsearch/client/configure_client.ts index 00cbd1958d8171..bac792d1293a6f 100644 --- a/src/core/server/elasticsearch/client/configure_client.ts +++ b/src/core/server/elasticsearch/client/configure_client.ts @@ -15,12 +15,12 @@ import { parseClientOptions, ElasticsearchClientConfig } from './client_config'; export const configureClient = ( config: ElasticsearchClientConfig, - { logger, scoped = false }: { logger: Logger; scoped?: boolean } + { logger, type, scoped = false }: { logger: Logger; type: string; scoped?: boolean } ): Client => { const clientOptions = parseClientOptions(config, scoped); const client = new Client(clientOptions); - addLogging(client, logger, config.logQueries); + addLogging(client, logger.get('query', type)); return client; }; @@ -67,15 +67,13 @@ function getResponseMessage(event: RequestEvent): string { return `${event.statusCode}\n${params.method} ${url}${body}`; } -const addLogging = (client: Client, logger: Logger, logQueries: boolean) => { +const addLogging = (client: Client, logger: Logger) => { client.on('response', (error, event) => { - if (event && logQueries) { + if (event) { if (error) { - logger.error(getErrorMessage(error, event)); + logger.debug(getErrorMessage(error, event)); } else { - logger.debug(getResponseMessage(event), { - tags: ['query'], - }); + logger.debug(getResponseMessage(event)); } } }); diff --git a/src/core/server/elasticsearch/elasticsearch_config.test.ts b/src/core/server/elasticsearch/elasticsearch_config.test.ts index 803733fddb71cb..e76de913a9d912 100644 --- a/src/core/server/elasticsearch/elasticsearch_config.test.ts +++ b/src/core/server/elasticsearch/elasticsearch_config.test.ts @@ -47,7 +47,6 @@ test('set correct defaults', () => { "http://localhost:9200", ], "ignoreVersionMismatch": false, - "logQueries": false, "password": undefined, "pingTimeout": "PT30S", "requestHeadersWhitelist": Array [ diff --git a/src/core/server/elasticsearch/elasticsearch_config.ts b/src/core/server/elasticsearch/elasticsearch_config.ts index b90ae2609f1e3f..afda47ca8851b7 100644 --- a/src/core/server/elasticsearch/elasticsearch_config.ts +++ b/src/core/server/elasticsearch/elasticsearch_config.ts @@ -133,6 +133,10 @@ const deprecations: ConfigDeprecationProvider = () => [ log( `Setting [${fromPath}.ssl.certificate] without [${fromPath}.ssl.key] is deprecated. This has no effect, you should use both settings to enable TLS client authentication to Elasticsearch.` ); + } else if (es.logQueries === true) { + log( + `Setting [${fromPath}.logQueries] is deprecated and no longer used. You should set the log level to "debug" for the "elasticsearch.queries" context in "logging.loggers" or use "logging.verbose: true".` + ); } return settings; }, @@ -164,12 +168,6 @@ export class ElasticsearchConfig { */ public readonly apiVersion: string; - /** - * Specifies whether all queries to the client should be logged (status code, - * method, query etc.). - */ - public readonly logQueries: boolean; - /** * Hosts that the client will connect to. If sniffing is enabled, this list will * be used as seeds to discover the rest of your cluster. @@ -248,7 +246,6 @@ export class ElasticsearchConfig { constructor(rawConfig: ElasticsearchConfigType) { this.ignoreVersionMismatch = rawConfig.ignoreVersionMismatch; this.apiVersion = rawConfig.apiVersion; - this.logQueries = rawConfig.logQueries; this.hosts = Array.isArray(rawConfig.hosts) ? rawConfig.hosts : [rawConfig.hosts]; this.requestHeadersWhitelist = Array.isArray(rawConfig.requestHeadersWhitelist) ? rawConfig.requestHeadersWhitelist diff --git a/src/core/server/elasticsearch/elasticsearch_service.test.ts b/src/core/server/elasticsearch/elasticsearch_service.test.ts index a6d966b346072c..3129ccfb5a67e8 100644 --- a/src/core/server/elasticsearch/elasticsearch_service.test.ts +++ b/src/core/server/elasticsearch/elasticsearch_service.test.ts @@ -92,14 +92,15 @@ describe('#setup', () => { // reset all mocks called during setup phase MockLegacyClusterClient.mockClear(); - const customConfig = { logQueries: true }; + const customConfig = { keepAlive: true }; const clusterClient = setupContract.legacy.createClient('some-custom-type', customConfig); expect(clusterClient).toBe(mockLegacyClusterClientInstance); expect(MockLegacyClusterClient).toHaveBeenCalledWith( expect.objectContaining(customConfig), - expect.objectContaining({ context: ['elasticsearch', 'some-custom-type'] }), + expect.objectContaining({ context: ['elasticsearch'] }), + 'some-custom-type', expect.any(Function) ); }); @@ -267,7 +268,7 @@ describe('#start', () => { // reset all mocks called during setup phase MockClusterClient.mockClear(); - const customConfig = { logQueries: true }; + const customConfig = { keepAlive: true }; const clusterClient = startContract.createClient('custom-type', customConfig); expect(clusterClient).toBe(mockClusterClientInstance); @@ -275,7 +276,8 @@ describe('#start', () => { expect(MockClusterClient).toHaveBeenCalledTimes(1); expect(MockClusterClient).toHaveBeenCalledWith( expect.objectContaining(customConfig), - expect.objectContaining({ context: ['elasticsearch', 'custom-type'] }), + expect.objectContaining({ context: ['elasticsearch'] }), + 'custom-type', expect.any(Function) ); }); @@ -286,7 +288,7 @@ describe('#start', () => { // reset all mocks called during setup phase MockClusterClient.mockClear(); - const customConfig = { logQueries: true }; + const customConfig = { keepAlive: true }; startContract.createClient('custom-type', customConfig); startContract.createClient('another-type', customConfig); diff --git a/src/core/server/elasticsearch/elasticsearch_service.ts b/src/core/server/elasticsearch/elasticsearch_service.ts index 2d97f6e5c31211..fd3d546bb77b95 100644 --- a/src/core/server/elasticsearch/elasticsearch_service.ts +++ b/src/core/server/elasticsearch/elasticsearch_service.ts @@ -126,7 +126,8 @@ export class ElasticsearchService private createClusterClient(type: string, config: ElasticsearchClientConfig) { return new ClusterClient( config, - this.coreContext.logger.get('elasticsearch', type), + this.coreContext.logger.get('elasticsearch'), + type, this.getAuthHeaders ); } @@ -134,7 +135,8 @@ export class ElasticsearchService private createLegacyClusterClient(type: string, config: LegacyElasticsearchClientConfig) { return new LegacyClusterClient( config, - this.coreContext.logger.get('elasticsearch', type), + this.coreContext.logger.get('elasticsearch'), + type, this.getAuthHeaders ); } diff --git a/src/core/server/elasticsearch/legacy/cluster_client.test.ts b/src/core/server/elasticsearch/legacy/cluster_client.test.ts index 97a49cd9eb9f4d..177181608bee99 100644 --- a/src/core/server/elasticsearch/legacy/cluster_client.test.ts +++ b/src/core/server/elasticsearch/legacy/cluster_client.test.ts @@ -31,11 +31,15 @@ test('#constructor creates client with parsed config', () => { const mockEsConfig = { apiVersion: 'es-version' } as any; const mockLogger = logger.get(); - const clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + const clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); expect(clusterClient).toBeDefined(); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type' + ); expect(MockClient).toHaveBeenCalledTimes(1); expect(MockClient).toHaveBeenCalledWith(mockEsClientConfig); @@ -57,7 +61,11 @@ describe('#callAsInternalUser', () => { }; MockClient.mockImplementation(() => mockEsClientInstance); - clusterClient = new LegacyClusterClient({ apiVersion: 'es-version' } as any, logger.get()); + clusterClient = new LegacyClusterClient( + { apiVersion: 'es-version' } as any, + logger.get(), + 'custom-type' + ); }); test('fails if cluster client is closed', async () => { @@ -226,7 +234,7 @@ describe('#asScoped', () => { requestHeadersWhitelist: ['one', 'two'], } as any; - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); jest.clearAllMocks(); }); @@ -237,10 +245,15 @@ describe('#asScoped', () => { expect(firstScopedClusterClient).toBeDefined(); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: true, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: true, + } + ); expect(MockClient).toHaveBeenCalledTimes(1); expect(MockClient).toHaveBeenCalledWith( @@ -261,42 +274,57 @@ describe('#asScoped', () => { test('properly configures `ignoreCertAndKey` for various configurations', () => { // Config without SSL. - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); mockParseElasticsearchClientConfig.mockClear(); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1' } })); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: true, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: true, + } + ); // Config ssl.alwaysPresentCertificate === false mockEsConfig = { ...mockEsConfig, ssl: { alwaysPresentCertificate: false } } as any; - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); mockParseElasticsearchClientConfig.mockClear(); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1' } })); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: true, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: true, + } + ); // Config ssl.alwaysPresentCertificate === true mockEsConfig = { ...mockEsConfig, ssl: { alwaysPresentCertificate: true } } as any; - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); mockParseElasticsearchClientConfig.mockClear(); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1' } })); expect(mockParseElasticsearchClientConfig).toHaveBeenCalledTimes(1); - expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith(mockEsConfig, mockLogger, { - auth: false, - ignoreCertAndKey: false, - }); + expect(mockParseElasticsearchClientConfig).toHaveBeenLastCalledWith( + mockEsConfig, + mockLogger, + 'custom-type', + { + auth: false, + ignoreCertAndKey: false, + } + ); }); test('passes only filtered headers to the scoped cluster client', () => { @@ -345,7 +373,7 @@ describe('#asScoped', () => { }); test('does not fail when scope to not defined request', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); clusterClient.asScoped(); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); expect(MockScopedClusterClient).toHaveBeenCalledWith( @@ -356,7 +384,7 @@ describe('#asScoped', () => { }); test('does not fail when scope to a request without headers', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); clusterClient.asScoped({} as any); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); expect(MockScopedClusterClient).toHaveBeenCalledWith( @@ -367,7 +395,7 @@ describe('#asScoped', () => { }); test('calls getAuthHeaders and filters results for a real request', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, () => ({ + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type', () => ({ one: '1', three: '3', })); @@ -381,7 +409,9 @@ describe('#asScoped', () => { }); test('getAuthHeaders results rewrite extends a request headers', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, () => ({ one: 'foo' })); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type', () => ({ + one: 'foo', + })); clusterClient.asScoped(httpServerMock.createRawRequest({ headers: { one: '1', two: '2' } })); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); expect(MockScopedClusterClient).toHaveBeenCalledWith( @@ -392,7 +422,7 @@ describe('#asScoped', () => { }); test("doesn't call getAuthHeaders for a fake request", async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, () => ({})); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type', () => ({})); clusterClient.asScoped({ headers: { one: 'foo' } }); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); @@ -404,7 +434,7 @@ describe('#asScoped', () => { }); test('filters a fake request headers', async () => { - clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger); + clusterClient = new LegacyClusterClient(mockEsConfig, mockLogger, 'custom-type'); clusterClient.asScoped({ headers: { one: '1', two: '2', three: '3' } }); expect(MockScopedClusterClient).toHaveBeenCalledTimes(1); @@ -431,7 +461,8 @@ describe('#close', () => { clusterClient = new LegacyClusterClient( { apiVersion: 'es-version', requestHeadersWhitelist: [] } as any, - logger.get() + logger.get(), + 'custom-type' ); }); diff --git a/src/core/server/elasticsearch/legacy/cluster_client.ts b/src/core/server/elasticsearch/legacy/cluster_client.ts index 9cac7139203316..64e1382fee201a 100644 --- a/src/core/server/elasticsearch/legacy/cluster_client.ts +++ b/src/core/server/elasticsearch/legacy/cluster_client.ts @@ -121,9 +121,10 @@ export class LegacyClusterClient implements ILegacyClusterClient { constructor( private readonly config: LegacyElasticsearchClientConfig, private readonly log: Logger, + private readonly type: string, private readonly getAuthHeaders: GetAuthHeaders = noop ) { - this.client = new Client(parseElasticsearchClientConfig(config, log)); + this.client = new Client(parseElasticsearchClientConfig(config, log, type)); } /** @@ -186,7 +187,7 @@ export class LegacyClusterClient implements ILegacyClusterClient { // between all scoped client instances. if (this.scopedClient === undefined) { this.scopedClient = new Client( - parseElasticsearchClientConfig(this.config, this.log, { + parseElasticsearchClientConfig(this.config, this.log, this.type, { auth: false, ignoreCertAndKey: !this.config.ssl || !this.config.ssl.alwaysPresentCertificate, }) diff --git a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts index 5dac353c1094c2..6c79f2c568caa8 100644 --- a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts +++ b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.test.ts @@ -22,13 +22,13 @@ test('parses minimally specified config', () => { { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -58,7 +58,6 @@ test('parses fully specified config', () => { const elasticsearchConfig: LegacyElasticsearchClientConfig = { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: [ @@ -84,7 +83,8 @@ test('parses fully specified config', () => { const elasticsearchClientConfig = parseElasticsearchClientConfig( elasticsearchConfig, - logger.get() + logger.get(), + 'custom-type' ); // Check that original references aren't used. @@ -163,7 +163,6 @@ test('parses config timeouts of moment.Duration type', () => { { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, pingTimeout: duration(100, 'ms'), @@ -172,7 +171,8 @@ test('parses config timeouts of moment.Duration type', () => { hosts: ['http://localhost:9200/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -208,7 +208,6 @@ describe('#auth', () => { { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['http://user:password@localhost/elasticsearch', 'https://es.local'], @@ -217,6 +216,7 @@ describe('#auth', () => { requestHeadersWhitelist: [], }, logger.get(), + 'custom-type', { auth: false } ) ).toMatchInlineSnapshot(` @@ -260,7 +260,6 @@ describe('#auth', () => { { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], @@ -268,6 +267,7 @@ describe('#auth', () => { password: 'changeme', }, logger.get(), + 'custom-type', { auth: true } ) ).toMatchInlineSnapshot(` @@ -300,7 +300,6 @@ describe('#auth', () => { { apiVersion: 'v7.0.0', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], @@ -308,6 +307,7 @@ describe('#auth', () => { username: 'elastic', }, logger.get(), + 'custom-type', { auth: true } ) ).toMatchInlineSnapshot(` @@ -342,13 +342,13 @@ describe('#customHeaders', () => { { apiVersion: 'master', customHeaders: { [headerKey]: 'foo' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() + logger.get(), + 'custom-type' ); expect(parsedConfig.hosts[0].headers).toEqual({ [headerKey]: 'foo', @@ -357,62 +357,18 @@ describe('#customHeaders', () => { }); describe('#log', () => { - test('default logger with #logQueries = false', () => { + test('default logger', () => { const parsedConfig = parseElasticsearchClientConfig( { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: false, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], }, - logger.get() - ); - - const esLogger = new parsedConfig.log(); - esLogger.error('some-error'); - esLogger.warning('some-warning'); - esLogger.trace('some-trace'); - esLogger.info('some-info'); - esLogger.debug('some-debug'); - - expect(typeof esLogger.close).toBe('function'); - - expect(loggingSystemMock.collect(logger)).toMatchInlineSnapshot(` - Object { - "debug": Array [], - "error": Array [ - Array [ - "some-error", - ], - ], - "fatal": Array [], - "info": Array [], - "log": Array [], - "trace": Array [], - "warn": Array [ - Array [ - "some-warning", - ], - ], - } - `); - }); - - test('default logger with #logQueries = true', () => { - const parsedConfig = parseElasticsearchClientConfig( - { - apiVersion: 'master', - customHeaders: { xsrf: 'something' }, - logQueries: true, - sniffOnStart: false, - sniffOnConnectionFault: false, - hosts: ['http://localhost/elasticsearch'], - requestHeadersWhitelist: [], - }, - logger.get() + logger.get(), + 'custom-type' ); const esLogger = new parsedConfig.log(); @@ -433,11 +389,6 @@ describe('#log', () => { "304 METHOD /some-path ?query=2", - Object { - "tags": Array [ - "query", - ], - }, ], ], "error": Array [ @@ -465,14 +416,14 @@ describe('#log', () => { { apiVersion: 'master', customHeaders: { xsrf: 'something' }, - logQueries: true, sniffOnStart: false, sniffOnConnectionFault: false, hosts: ['http://localhost/elasticsearch'], requestHeadersWhitelist: [], log: customLogger, }, - logger.get() + logger.get(), + 'custom-type' ); expect(parsedConfig.log).toBe(customLogger); @@ -486,14 +437,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'none' }, }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -527,14 +478,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'certificate' }, }, - logger.get() + logger.get(), + 'custom-type' ); // `checkServerIdentity` shouldn't check hostname when verificationMode is certificate. @@ -576,14 +527,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'full' }, }, - logger.get() + logger.get(), + 'custom-type' ) ).toMatchInlineSnapshot(` Object { @@ -618,14 +569,14 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], requestHeadersWhitelist: [], ssl: { verificationMode: 'misspelled' as any }, }, - logger.get() + logger.get(), + 'custom-type' ) ).toThrowErrorMatchingInlineSnapshot(`"Unknown ssl verificationMode: misspelled"`); }); @@ -636,7 +587,6 @@ describe('#ssl', () => { { apiVersion: 'v7.0.0', customHeaders: {}, - logQueries: true, sniffOnStart: true, sniffOnConnectionFault: true, hosts: ['https://es.local'], @@ -651,6 +601,7 @@ describe('#ssl', () => { }, }, logger.get(), + 'custom-type', { ignoreCertAndKey: true } ) ).toMatchInlineSnapshot(` diff --git a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts index ecd2e300970608..66b6046e4516da 100644 --- a/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts +++ b/src/core/server/elasticsearch/legacy/elasticsearch_client_config.ts @@ -29,7 +29,6 @@ export type LegacyElasticsearchClientConfig = Pick & { +export type ElasticsearchClientConfig = Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ClientOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ClientOptions['requestTimeout']; ssl?: Partial; @@ -859,7 +859,6 @@ export class ElasticsearchConfig { readonly healthCheckDelay: Duration; readonly hosts: string[]; readonly ignoreVersionMismatch: boolean; - readonly logQueries: boolean; readonly password?: string; readonly pingTimeout: Duration; readonly requestHeadersWhitelist: string[]; @@ -1531,7 +1530,7 @@ export interface LegacyCallAPIOptions { // @public @deprecated export class LegacyClusterClient implements ILegacyClusterClient { - constructor(config: LegacyElasticsearchClientConfig, log: Logger, getAuthHeaders?: GetAuthHeaders); + constructor(config: LegacyElasticsearchClientConfig, log: Logger, type: string, getAuthHeaders?: GetAuthHeaders); asScoped(request?: ScopeableRequest): ILegacyScopedClusterClient; // @deprecated callAsInternalUser: LegacyAPICaller; @@ -1553,7 +1552,7 @@ export interface LegacyConfig { } // @public @deprecated (undocumented) -export type LegacyElasticsearchClientConfig = Pick & Pick & { +export type LegacyElasticsearchClientConfig = Pick & Pick & { pingTimeout?: ElasticsearchConfig['pingTimeout'] | ConfigOptions['pingTimeout']; requestTimeout?: ElasticsearchConfig['requestTimeout'] | ConfigOptions['requestTimeout']; sniffInterval?: ElasticsearchConfig['sniffInterval'] | ConfigOptions['sniffInterval']; diff --git a/src/plugins/data/common/es_query/kuery/node_types/node_builder.test.ts b/src/plugins/data/common/es_query/kuery/node_types/node_builder.test.ts new file mode 100644 index 00000000000000..df78d68aaef487 --- /dev/null +++ b/src/plugins/data/common/es_query/kuery/node_types/node_builder.test.ts @@ -0,0 +1,280 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import { nodeBuilder } from './node_builder'; +import { toElasticsearchQuery } from '../index'; + +describe('nodeBuilder', () => { + describe('is method', () => { + test('string value', () => { + const nodes = nodeBuilder.is('foo', 'bar'); + const query = toElasticsearchQuery(nodes); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + + test('KueryNode value', () => { + const literalValue = { + type: 'literal' as 'literal', + value: 'bar', + }; + const nodes = nodeBuilder.is('foo', literalValue); + const query = toElasticsearchQuery(nodes); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + }); + + describe('and method', () => { + test('single clause', () => { + const nodes = [nodeBuilder.is('foo', 'bar')]; + const query = toElasticsearchQuery(nodeBuilder.and(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + + test('two clauses', () => { + const nodes = [nodeBuilder.is('foo1', 'bar1'), nodeBuilder.is('foo2', 'bar2')]; + const query = toElasticsearchQuery(nodeBuilder.and(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "filter": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + + test('three clauses', () => { + const nodes = [ + nodeBuilder.is('foo1', 'bar1'), + nodeBuilder.is('foo2', 'bar2'), + nodeBuilder.is('foo3', 'bar3'), + ]; + const query = toElasticsearchQuery(nodeBuilder.and(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "filter": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo3": "bar3", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + }); + + describe('or method', () => { + test('single clause', () => { + const nodes = [nodeBuilder.is('foo', 'bar')]; + const query = toElasticsearchQuery(nodeBuilder.or(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo": "bar", + }, + }, + ], + }, + } + `); + }); + + test('two clauses', () => { + const nodes = [nodeBuilder.is('foo1', 'bar1'), nodeBuilder.is('foo2', 'bar2')]; + const query = toElasticsearchQuery(nodeBuilder.or(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + + test('three clauses', () => { + const nodes = [ + nodeBuilder.is('foo1', 'bar1'), + nodeBuilder.is('foo2', 'bar2'), + nodeBuilder.is('foo3', 'bar3'), + ]; + const query = toElasticsearchQuery(nodeBuilder.or(nodes)); + expect(query).toMatchInlineSnapshot(` + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo1": "bar1", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo2": "bar2", + }, + }, + ], + }, + }, + Object { + "bool": Object { + "minimum_should_match": 1, + "should": Array [ + Object { + "match": Object { + "foo3": "bar3", + }, + }, + ], + }, + }, + ], + }, + } + `); + }); + }); +}); diff --git a/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts b/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts index a72c7f2db41a8a..6da9c3aa293ef5 100644 --- a/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts +++ b/src/plugins/data/common/es_query/kuery/node_types/node_builder.ts @@ -16,12 +16,10 @@ export const nodeBuilder = { nodeTypes.literal.buildNode(false), ]); }, - or: ([first, ...args]: KueryNode[]): KueryNode => { - return args.length ? nodeTypes.function.buildNode('or', [first, nodeBuilder.or(args)]) : first; + or: (nodes: KueryNode[]): KueryNode => { + return nodes.length > 1 ? nodeTypes.function.buildNode('or', nodes) : nodes[0]; }, - and: ([first, ...args]: KueryNode[]): KueryNode => { - return args.length - ? nodeTypes.function.buildNode('and', [first, nodeBuilder.and(args)]) - : first; + and: (nodes: KueryNode[]): KueryNode => { + return nodes.length > 1 ? nodeTypes.function.buildNode('and', nodes) : nodes[0]; }, }; diff --git a/src/plugins/data/common/search/types.ts b/src/plugins/data/common/search/types.ts index c1293f44154583..38e963591f25ce 100644 --- a/src/plugins/data/common/search/types.ts +++ b/src/plugins/data/common/search/types.ts @@ -84,11 +84,18 @@ export interface ISearchOptions { * An `AbortSignal` that allows the caller of `search` to abort a search request. */ abortSignal?: AbortSignal; + /** * Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. */ strategy?: string; + /** + * Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to + * something other than `true`, this should be set to `false`. + */ + legacyHitsTotal?: boolean; + /** * A session ID, grouping multiple search requests into a single session. */ diff --git a/src/plugins/data/public/public.api.md b/src/plugins/data/public/public.api.md index f533af2db9672e..4f197dd43a83ef 100644 --- a/src/plugins/data/public/public.api.md +++ b/src/plugins/data/public/public.api.md @@ -1641,6 +1641,7 @@ export interface ISearchOptions { abortSignal?: AbortSignal; isRestore?: boolean; isStored?: boolean; + legacyHitsTotal?: boolean; sessionId?: string; strategy?: string; } diff --git a/src/plugins/data/server/index.ts b/src/plugins/data/server/index.ts index 27af11674d061e..370ff180fa5627 100644 --- a/src/plugins/data/server/index.ts +++ b/src/plugins/data/server/index.ts @@ -235,6 +235,7 @@ export { SearchUsage, SessionService, ISessionService, + IScopedSessionService, DataApiRequestHandlerContext, DataRequestHandlerContext, } from './search'; diff --git a/src/plugins/data/server/search/es_search/es_search_strategy.ts b/src/plugins/data/server/search/es_search/es_search_strategy.ts index c176a50627b92e..2d9b16ac8b00bf 100644 --- a/src/plugins/data/server/search/es_search/es_search_strategy.ts +++ b/src/plugins/data/server/search/es_search/es_search_strategy.ts @@ -13,7 +13,7 @@ import type { Logger, SharedGlobalConfig } from 'kibana/server'; import type { ISearchStrategy } from '../types'; import type { SearchUsage } from '../collectors'; import { getDefaultSearchParams, getShardTimeout, shimAbortSignal } from './request_utils'; -import { toKibanaSearchResponse } from './response_utils'; +import { shimHitsTotal, toKibanaSearchResponse } from './response_utils'; import { searchUsageObserver } from '../collectors/usage'; import { getKbnServerError, KbnServerError } from '../../../../kibana_utils/server'; @@ -29,7 +29,7 @@ export const esSearchStrategyProvider = ( * @throws `KbnServerError` * @returns `Observable>` */ - search: (request, { abortSignal }, { esClient, uiSettingsClient }) => { + search: (request, { abortSignal, ...options }, { esClient, uiSettingsClient }) => { // Only default index pattern type is supported here. // See data_enhanced for other type support. if (request.indexType) { @@ -46,7 +46,8 @@ export const esSearchStrategyProvider = ( }; const promise = esClient.asCurrentUser.search>(params); const { body } = await shimAbortSignal(promise, abortSignal); - return toKibanaSearchResponse(body); + const response = shimHitsTotal(body, options); + return toKibanaSearchResponse(response); } catch (e) { throw getKbnServerError(e); } diff --git a/src/plugins/data/server/search/es_search/response_utils.test.ts b/src/plugins/data/server/search/es_search/response_utils.test.ts index 8c973b92c7ffe7..7cb5705ecf8ef2 100644 --- a/src/plugins/data/server/search/es_search/response_utils.test.ts +++ b/src/plugins/data/server/search/es_search/response_utils.test.ts @@ -6,7 +6,7 @@ * Public License, v 1. */ -import { getTotalLoaded, toKibanaSearchResponse } from './response_utils'; +import { getTotalLoaded, toKibanaSearchResponse, shimHitsTotal } from './response_utils'; import { SearchResponse } from 'elasticsearch'; describe('response utils', () => { @@ -55,4 +55,79 @@ describe('response utils', () => { }); }); }); + + describe('shimHitsTotal', () => { + test('returns the total if it is already numeric', () => { + const result = shimHitsTotal({ + hits: { + total: 5, + }, + } as any); + expect(result).toEqual({ + hits: { + total: 5, + }, + }); + }); + + test('returns the total if it is inside `value`', () => { + const result = shimHitsTotal({ + hits: { + total: { + value: 5, + }, + }, + } as any); + expect(result).toEqual({ + hits: { + total: 5, + }, + }); + }); + + test('returns other properties from the response', () => { + const result = shimHitsTotal({ + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + }, + }, + } as any); + expect(result).toEqual({ + _shards: {}, + hits: { + hits: [], + total: 5, + }, + }); + }); + + test('returns the response as-is if `legacyHitsTotal` is `false`', () => { + const result = shimHitsTotal( + { + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + relation: 'eq', + }, + }, + } as any, + { legacyHitsTotal: false } + ); + expect(result).toEqual({ + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + relation: 'eq', + }, + }, + }); + }); + }); }); diff --git a/src/plugins/data/server/search/es_search/response_utils.ts b/src/plugins/data/server/search/es_search/response_utils.ts index d4fa14866fd97d..3417f24cf420ac 100644 --- a/src/plugins/data/server/search/es_search/response_utils.ts +++ b/src/plugins/data/server/search/es_search/response_utils.ts @@ -7,6 +7,7 @@ */ import { SearchResponse } from 'elasticsearch'; +import { ISearchOptions } from '../../../common'; /** * Get the `total`/`loaded` for this response (see `IKibanaSearchResponse`). Note that `skipped` is @@ -31,3 +32,20 @@ export function toKibanaSearchResponse(rawResponse: SearchResponse) { ...getTotalLoaded(rawResponse), }; } + +/** + * Temporary workaround until https://github.com/elastic/kibana/issues/26356 is addressed. + * Since we are setting `track_total_hits` in the request, `hits.total` will be an object + * containing the `value`. + * + * @internal + */ +export function shimHitsTotal( + response: SearchResponse, + { legacyHitsTotal = true }: ISearchOptions = {} +) { + if (!legacyHitsTotal) return response; + const total = (response.hits?.total as any)?.value ?? response.hits?.total; + const hits = { ...response.hits, total }; + return { ...response, hits }; +} diff --git a/src/plugins/data/server/search/index.ts b/src/plugins/data/server/search/index.ts index a3334241100655..301b0989b51839 100644 --- a/src/plugins/data/server/search/index.ts +++ b/src/plugins/data/server/search/index.ts @@ -10,5 +10,4 @@ export * from './types'; export * from './es_search'; export { usageProvider, SearchUsage, searchUsageObserver } from './collectors'; export * from './aggs'; -export { shimHitsTotal } from './routes'; export * from './session'; diff --git a/src/plugins/data/server/search/routes/bsearch.ts b/src/plugins/data/server/search/routes/bsearch.ts index e30b7bdaa84022..ba96726b787c02 100644 --- a/src/plugins/data/server/search/routes/bsearch.ts +++ b/src/plugins/data/server/search/routes/bsearch.ts @@ -6,7 +6,7 @@ * Public License, v 1. */ -import { catchError, first, map } from 'rxjs/operators'; +import { catchError, first } from 'rxjs/operators'; import { CoreStart, KibanaRequest } from 'src/core/server'; import { BfetchServerSetup } from 'src/plugins/bfetch/server'; import { @@ -15,7 +15,6 @@ import { ISearchClient, ISearchOptions, } from '../../../common/search'; -import { shimHitsTotal } from './shim_hits_total'; type GetScopedProider = (coreStart: CoreStart) => (request: KibanaRequest) => ISearchClient; @@ -40,14 +39,6 @@ export function registerBsearchRoute( .search(requestData, options) .pipe( first(), - map((response) => { - return { - ...response, - ...{ - rawResponse: shimHitsTotal(response.rawResponse), - }, - }; - }), catchError((err) => { // Re-throw as object, to get attributes passed to the client // eslint-disable-next-line no-throw-literal diff --git a/src/plugins/data/server/search/routes/call_msearch.ts b/src/plugins/data/server/search/routes/call_msearch.ts index fc30e2f29c3ef2..e6ff5f454079b2 100644 --- a/src/plugins/data/server/search/routes/call_msearch.ts +++ b/src/plugins/data/server/search/routes/call_msearch.ts @@ -12,9 +12,8 @@ import { SearchResponse } from 'elasticsearch'; import { IUiSettingsClient, IScopedClusterClient, SharedGlobalConfig } from 'src/core/server'; import type { MsearchRequestBody, MsearchResponse } from '../../../common/search/search_source'; -import { shimHitsTotal } from './shim_hits_total'; import { getKbnServerError } from '../../../../kibana_utils/server'; -import { getShardTimeout, getDefaultSearchParams, shimAbortSignal } from '..'; +import { getShardTimeout, getDefaultSearchParams, shimAbortSignal, shimHitsTotal } from '..'; /** @internal */ export function convertRequestBody( diff --git a/src/plugins/data/server/search/routes/index.ts b/src/plugins/data/server/search/routes/index.ts index ea20240f6ae19b..25e0353fb4a27a 100644 --- a/src/plugins/data/server/search/routes/index.ts +++ b/src/plugins/data/server/search/routes/index.ts @@ -9,4 +9,3 @@ export * from './call_msearch'; export * from './msearch'; export * from './search'; -export * from './shim_hits_total'; diff --git a/src/plugins/data/server/search/routes/search.ts b/src/plugins/data/server/search/routes/search.ts index 6d2da4c1e63ddb..e556e3ca49ec29 100644 --- a/src/plugins/data/server/search/routes/search.ts +++ b/src/plugins/data/server/search/routes/search.ts @@ -9,7 +9,6 @@ import { first } from 'rxjs/operators'; import { schema } from '@kbn/config-schema'; import { getRequestAbortedSignal } from '../../lib'; -import { shimHitsTotal } from './shim_hits_total'; import { reportServerError } from '../../../../kibana_utils/server'; import type { DataPluginRouter } from '../types'; @@ -27,6 +26,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { body: schema.object( { + legacyHitsTotal: schema.maybe(schema.boolean()), sessionId: schema.maybe(schema.string()), isStored: schema.maybe(schema.boolean()), isRestore: schema.maybe(schema.boolean()), @@ -36,7 +36,13 @@ export function registerSearchRoute(router: DataPluginRouter): void { }, }, async (context, request, res) => { - const { sessionId, isStored, isRestore, ...searchRequest } = request.body; + const { + legacyHitsTotal = true, + sessionId, + isStored, + isRestore, + ...searchRequest + } = request.body; const { strategy, id } = request.params; const abortSignal = getRequestAbortedSignal(request.events.aborted$); @@ -47,6 +53,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { { abortSignal, strategy, + legacyHitsTotal, sessionId, isStored, isRestore, @@ -55,14 +62,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { .pipe(first()) .toPromise(); - return res.ok({ - body: { - ...response, - ...{ - rawResponse: shimHitsTotal(response.rawResponse), - }, - }, - }); + return res.ok({ body: response }); } catch (err) { return reportServerError(res, err); } diff --git a/src/plugins/data/server/search/routes/shim_hits_total.test.ts b/src/plugins/data/server/search/routes/shim_hits_total.test.ts deleted file mode 100644 index 6dcd7c3ff6c70c..00000000000000 --- a/src/plugins/data/server/search/routes/shim_hits_total.test.ts +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { shimHitsTotal } from './shim_hits_total'; - -describe('shimHitsTotal', () => { - test('returns the total if it is already numeric', () => { - const result = shimHitsTotal({ - hits: { - total: 5, - }, - } as any); - expect(result).toEqual({ - hits: { - total: 5, - }, - }); - }); - - test('returns the total if it is inside `value`', () => { - const result = shimHitsTotal({ - hits: { - total: { - value: 5, - }, - }, - } as any); - expect(result).toEqual({ - hits: { - total: 5, - }, - }); - }); - - test('returns other properties from the response', () => { - const result = shimHitsTotal({ - _shards: {}, - hits: { - hits: [], - total: { - value: 5, - }, - }, - } as any); - expect(result).toEqual({ - _shards: {}, - hits: { - hits: [], - total: 5, - }, - }); - }); -}); diff --git a/src/plugins/data/server/search/routes/shim_hits_total.ts b/src/plugins/data/server/search/routes/shim_hits_total.ts deleted file mode 100644 index 4b56d6394e0dbc..00000000000000 --- a/src/plugins/data/server/search/routes/shim_hits_total.ts +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { SearchResponse } from 'elasticsearch'; - -/** - * Temporary workaround until https://github.com/elastic/kibana/issues/26356 is addressed. - * Since we are setting `track_total_hits` in the request, `hits.total` will be an object - * containing the `value`. - * - * @internal - */ -export function shimHitsTotal(response: SearchResponse) { - const total = (response.hits?.total as any)?.value ?? response.hits?.total; - const hits = { ...response.hits, total }; - return { ...response, hits }; -} diff --git a/src/plugins/data/server/search/search_service.ts b/src/plugins/data/server/search/search_service.ts index 63593bbe84a088..e9f0edbd4d6c47 100644 --- a/src/plugins/data/server/search/search_service.ts +++ b/src/plugins/data/server/search/search_service.ts @@ -34,7 +34,7 @@ import { AggsService } from './aggs'; import { FieldFormatsStart } from '../field_formats'; import { IndexPatternsServiceStart } from '../index_patterns'; -import { getCallMsearch, registerMsearchRoute, registerSearchRoute, shimHitsTotal } from './routes'; +import { getCallMsearch, registerMsearchRoute, registerSearchRoute } from './routes'; import { ES_SEARCH_STRATEGY, esSearchStrategyProvider } from './es_search'; import { DataPluginStart } from '../plugin'; import { UsageCollectionSetup } from '../../../usage_collection/server'; @@ -62,7 +62,7 @@ import { } from '../../common/search/aggs/buckets/shard_delay'; import { aggShardDelay } from '../../common/search/aggs/buckets/shard_delay_fn'; import { ConfigSchema } from '../../config'; -import { SessionService, IScopedSessionService, ISessionService } from './session'; +import { IScopedSessionService, ISessionService, SessionService } from './session'; import { KbnServerError } from '../../../kibana_utils/server'; import { registerBsearchRoute } from './routes/bsearch'; @@ -209,7 +209,7 @@ export class SearchService implements Plugin { const searchSourceDependencies: SearchSourceDependencies = { getConfig: (key: string): T => uiSettingsCache[key], search: asScoped(request).search, - onResponse: (req, res) => shimHitsTotal(res), + onResponse: (req, res) => res, legacy: { callMsearch: getCallMsearch({ esClient, diff --git a/src/plugins/data/server/server.api.md b/src/plugins/data/server/server.api.md index 9789f3354e9efb..635428f298ab2b 100644 --- a/src/plugins/data/server/server.api.md +++ b/src/plugins/data/server/server.api.md @@ -310,8 +310,6 @@ export const config: PluginConfigDescriptor; // // @public (undocumented) export interface DataApiRequestHandlerContext extends ISearchClient { - // Warning: (ae-forgotten-export) The symbol "IScopedSessionService" needs to be exported by the entry point index.d.ts - // // (undocumented) session: IScopedSessionService; } @@ -912,6 +910,16 @@ export class IndexPatternsService implements Plugin_3(strategy: ISearchStrategy, ...args: Parameters['search']>) => Observable; +} + // Warning: (ae-missing-release-tag) "ISearchOptions" is exported by the package, but it is missing a release tag (@alpha, @beta, @public, or @internal) // // @public (undocumented) @@ -919,6 +927,7 @@ export interface ISearchOptions { abortSignal?: AbortSignal; isRestore?: boolean; isStored?: boolean; + legacyHitsTotal?: boolean; sessionId?: string; strategy?: string; } @@ -1284,7 +1293,7 @@ export class SessionService implements ISessionService { export const shimAbortSignal: (promise: TransportRequestPromise, signal?: AbortSignal | undefined) => TransportRequestPromise; // @internal -export function shimHitsTotal(response: SearchResponse): { +export function shimHitsTotal(response: SearchResponse, { legacyHitsTotal }?: ISearchOptions): { hits: { total: any; max_score: number; @@ -1293,7 +1302,7 @@ export function shimHitsTotal(response: SearchResponse): { _type: string; _id: string; _score: number; - _source: any; + _source: unknown; _version?: number | undefined; _explanation?: import("elasticsearch").Explanation | undefined; fields?: any; @@ -1426,20 +1435,20 @@ export function usageProvider(core: CoreSetup_2): SearchUsage; // src/plugins/data/server/index.ts:100:26 - (ae-forgotten-export) The symbol "TruncateFormat" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index.ts:126:27 - (ae-forgotten-export) The symbol "isFilterable" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index.ts:126:27 - (ae-forgotten-export) The symbol "isNestedField" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "getRequestInspectorStats" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "getResponseInspectorStats" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "tabifyAggResponse" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "tabifyGetColumns" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:245:1 - (ae-forgotten-export) The symbol "CidrMask" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:246:1 - (ae-forgotten-export) The symbol "dateHistogramInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:255:1 - (ae-forgotten-export) The symbol "InvalidEsCalendarIntervalError" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:256:1 - (ae-forgotten-export) The symbol "InvalidEsIntervalFormatError" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:257:1 - (ae-forgotten-export) The symbol "Ipv4Address" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:261:1 - (ae-forgotten-export) The symbol "isValidEsInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:262:1 - (ae-forgotten-export) The symbol "isValidInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:266:1 - (ae-forgotten-export) The symbol "propFilter" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:269:1 - (ae-forgotten-export) The symbol "toAbsoluteDates" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:270:1 - (ae-forgotten-export) The symbol "calcAutoIntervalLessThan" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "getRequestInspectorStats" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "getResponseInspectorStats" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "tabifyAggResponse" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "tabifyGetColumns" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:246:1 - (ae-forgotten-export) The symbol "CidrMask" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:247:1 - (ae-forgotten-export) The symbol "dateHistogramInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:256:1 - (ae-forgotten-export) The symbol "InvalidEsCalendarIntervalError" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:257:1 - (ae-forgotten-export) The symbol "InvalidEsIntervalFormatError" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:258:1 - (ae-forgotten-export) The symbol "Ipv4Address" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:262:1 - (ae-forgotten-export) The symbol "isValidEsInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:263:1 - (ae-forgotten-export) The symbol "isValidInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:267:1 - (ae-forgotten-export) The symbol "propFilter" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:270:1 - (ae-forgotten-export) The symbol "toAbsoluteDates" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:271:1 - (ae-forgotten-export) The symbol "calcAutoIntervalLessThan" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index_patterns/index_patterns_service.ts:59:14 - (ae-forgotten-export) The symbol "IndexPatternsService" needs to be exported by the entry point index.d.ts // src/plugins/data/server/plugin.ts:79:74 - (ae-forgotten-export) The symbol "DataEnhancements" needs to be exported by the entry point index.d.ts // src/plugins/data/server/search/types.ts:103:5 - (ae-forgotten-export) The symbol "ISearchStartSearchSource" needs to be exported by the entry point index.d.ts diff --git a/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap b/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap index e7136ac8172493..5724d46fca10c4 100644 --- a/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap +++ b/src/plugins/home/public/application/components/add_data/__snapshots__/add_data.test.tsx.snap @@ -32,6 +32,7 @@ exports[`AddData render 1`] = `
= ({ addBasePath, features }) => {
{ return this.tabs.map((tab, index) => ( this.onSelectedTabChanged(tab.id)} isSelected={tab.id === this.state.selectedTabId} key={index} @@ -203,7 +205,7 @@ class TutorialDirectoryUi extends React.Component { }) .map((tutorial) => { return ( - + = ({ = ({ = ({ test setup" source test/scripts/jenkins_test_setup_xpack.sh echo " -> run gatling load testing" +export GATLING_SIMULATIONS="$simulations" node scripts/functional_tests \ - --kibana-install-dir "$KIBANA_INSTALL_DIR" \ - --config test/load/config.ts + --kibana-install-dir "$KIBANA_INSTALL_DIR" \ + --config test/load/config.ts diff --git a/test/tsconfig.json b/test/tsconfig.json index c8e6e69586ca0e..1dc58f7b25c241 100644 --- a/test/tsconfig.json +++ b/test/tsconfig.json @@ -4,7 +4,12 @@ "incremental": false, "types": ["node", "flot"] }, - "include": ["**/*", "../typings/elastic__node_crypto.d.ts", "typings/**/*", "../packages/kbn-test/types/ftr_globals/**/*"], + "include": [ + "**/*", + "../typings/elastic__node_crypto.d.ts", + "typings/**/*", + "../packages/kbn-test/types/ftr_globals/**/*" + ], "exclude": ["plugin_functional/plugins/**/*", "interpreter_functional/plugins/**/*"], "references": [ { "path": "../src/core/tsconfig.json" }, @@ -34,5 +39,7 @@ { "path": "../src/plugins/ui_actions/tsconfig.json" }, { "path": "../src/plugins/url_forwarding/tsconfig.json" }, { "path": "../src/plugins/usage_collection/tsconfig.json" }, + { "path": "../src/plugins/index_pattern_management/tsconfig.json" }, + { "path": "../src/plugins/legacy_export/tsconfig.json" } ] } diff --git a/tsconfig.json b/tsconfig.json index d8fb2804242bc3..21760919c89e93 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -27,6 +27,7 @@ "src/plugins/kibana_react/**/*", "src/plugins/kibana_usage_collection/**/*", "src/plugins/kibana_utils/**/*", + "src/plugins/legacy_export/**/*", "src/plugins/management/**/*", "src/plugins/maps_legacy/**/*", "src/plugins/navigation/**/*", @@ -58,6 +59,7 @@ "src/plugins/vis_type_xy/**/*", "src/plugins/visualizations/**/*", "src/plugins/visualize/**/*", + "src/plugins/index_pattern_management/**/*", // In the build we actually exclude **/public/**/* from this config so that // we can run the TSC on both this and the .browser version of this config // file, but if we did it during development IDEs would not be able to find @@ -85,6 +87,7 @@ { "path": "./src/plugins/kibana_react/tsconfig.json" }, { "path": "./src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "./src/plugins/kibana_utils/tsconfig.json" }, + { "path": "./src/plugins/legacy_export/tsconfig.json" }, { "path": "./src/plugins/management/tsconfig.json" }, { "path": "./src/plugins/maps_legacy/tsconfig.json" }, { "path": "./src/plugins/navigation/tsconfig.json" }, @@ -115,5 +118,6 @@ { "path": "./src/plugins/vis_type_xy/tsconfig.json" }, { "path": "./src/plugins/visualizations/tsconfig.json" }, { "path": "./src/plugins/visualize/tsconfig.json" }, + { "path": "./src/plugins/index_pattern_management/tsconfig.json" }, ] } diff --git a/tsconfig.refs.json b/tsconfig.refs.json index 9a65b385b78202..1d08e764709cad 100644 --- a/tsconfig.refs.json +++ b/tsconfig.refs.json @@ -20,6 +20,7 @@ { "path": "./src/plugins/kibana_react/tsconfig.json" }, { "path": "./src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "./src/plugins/kibana_utils/tsconfig.json" }, + { "path": "./src/plugins/legacy_export/tsconfig.json" }, { "path": "./src/plugins/management/tsconfig.json" }, { "path": "./src/plugins/maps_legacy/tsconfig.json" }, { "path": "./src/plugins/navigation/tsconfig.json" }, @@ -52,5 +53,6 @@ { "path": "./src/plugins/vis_type_xy/tsconfig.json" }, { "path": "./src/plugins/visualizations/tsconfig.json" }, { "path": "./src/plugins/visualize/tsconfig.json" }, + { "path": "./src/plugins/index_pattern_management/tsconfig.json" }, ] } diff --git a/vars/githubPr.groovy b/vars/githubPr.groovy index 546a6785ac2f4f..5224aa7463d79d 100644 --- a/vars/githubPr.groovy +++ b/vars/githubPr.groovy @@ -182,12 +182,14 @@ def getNextCommentMessage(previousCommentInfo = [:], isFinal = false) { ## :green_heart: Build Succeeded * [continuous-integration/kibana-ci/pull-request](${env.BUILD_URL}) * Commit: ${getCommitHash()} + ${getDocsChangesLink()} """ } else if(status == 'UNSTABLE') { def message = """ ## :yellow_heart: Build succeeded, but was flaky * [continuous-integration/kibana-ci/pull-request](${env.BUILD_URL}) * Commit: ${getCommitHash()} + ${getDocsChangesLink()} """.stripIndent() def failures = retryable.getFlakyFailures() @@ -204,6 +206,7 @@ def getNextCommentMessage(previousCommentInfo = [:], isFinal = false) { * Commit: ${getCommitHash()} * [Pipeline Steps](${env.BUILD_URL}flowGraphTable) (look for red circles / failed steps) * [Interpreting CI Failures](https://www.elastic.co/guide/en/kibana/current/interpreting-ci-failures.html) + ${getDocsChangesLink()} """ } @@ -292,6 +295,21 @@ def getCommitHash() { return env.ghprbActualCommit } +def getDocsChangesLink() { + def url = "https://kibana_${env.ghprbPullId}.docs-preview.app.elstc.co/diff" + + try { + // httpRequest throws on status codes >400 and failures + httpRequest([ method: "GET", url: url ]) + return "* [Documentation Changes](${url})" + } catch (ex) { + print "Failed to reach ${url}" + buildUtils.printStacktrace(ex) + } + + return "" +} + def getFailedSteps() { return jenkinsApi.getFailedSteps()?.findAll { step -> step.displayName != 'Check out from version control' diff --git a/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts b/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts index 457079229de94d..569b54f21f906a 100644 --- a/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts +++ b/x-pack/plugins/alerts/server/alerts_client/alerts_client.ts @@ -149,6 +149,7 @@ export interface CreateOptions { | 'executionStatus' > & { actions: NormalizedAlertAction[] }; options?: { + id?: string; migrationVersion?: Record; }; } @@ -226,7 +227,7 @@ export class AlertsClient { data, options, }: CreateOptions): Promise> { - const id = SavedObjectsUtils.generateId(); + const id = options?.id || SavedObjectsUtils.generateId(); try { await this.authorization.ensureAuthorized( diff --git a/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts b/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts index 0424a1295c9b99..2e3dac76f72e50 100644 --- a/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts +++ b/x-pack/plugins/alerts/server/alerts_client/tests/create.test.ts @@ -462,6 +462,73 @@ describe('create()', () => { expect(actionsClient.isActionTypeEnabled).toHaveBeenCalledWith('test', { notifyUsage: true }); }); + test('creates an alert with a custom id', async () => { + const data = getMockData(); + const createdAttributes = { + ...data, + alertTypeId: '123', + schedule: { interval: '10s' }, + params: { + bar: true, + }, + createdAt: '2019-02-12T21:01:22.479Z', + createdBy: 'elastic', + updatedBy: 'elastic', + updatedAt: '2019-02-12T21:01:22.479Z', + muteAll: false, + mutedInstanceIds: [], + actions: [ + { + group: 'default', + actionRef: 'action_0', + actionTypeId: 'test', + params: { + foo: true, + }, + }, + ], + }; + unsecuredSavedObjectsClient.create.mockResolvedValueOnce({ + id: '123', + type: 'alert', + attributes: createdAttributes, + references: [ + { + name: 'action_0', + type: 'action', + id: '1', + }, + ], + }); + taskManager.schedule.mockResolvedValueOnce({ + id: 'task-123', + taskType: 'alerting:123', + scheduledAt: new Date(), + attempts: 1, + status: TaskStatus.Idle, + runAt: new Date(), + startedAt: null, + retryAt: null, + state: {}, + params: {}, + ownerId: null, + }); + const result = await alertsClient.create({ data, options: { id: '123' } }); + expect(result.id).toEqual('123'); + expect(unsecuredSavedObjectsClient.create.mock.calls[0][2]).toMatchInlineSnapshot(` + Object { + "id": "123", + "references": Array [ + Object { + "id": "1", + "name": "action_0", + "type": "action", + }, + ], + } + `); + }); + test('creates an alert with multiple actions', async () => { const data = getMockData({ actions: [ diff --git a/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization.test.ts.snap b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization.test.ts.snap new file mode 100644 index 00000000000000..f9a28dc3eb1197 --- /dev/null +++ b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization.test.ts.snap @@ -0,0 +1,316 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`AlertsAuthorization getFindAuthorizationFilter creates a filter based on the privileged types 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myOtherAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "mySecondAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + ], + "function": "or", + "type": "function", +} +`; diff --git a/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization_kuery.test.ts.snap b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization_kuery.test.ts.snap new file mode 100644 index 00000000000000..de01a7b27ef05b --- /dev/null +++ b/x-pack/plugins/alerts/server/authorization/__snapshots__/alerts_authorization_kuery.test.ts.snap @@ -0,0 +1,448 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`asFiltersByAlertTypeAndConsumer constructs filter for multiple alert types across authorized consumer 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myOtherAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "mySecondAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myAppWithSubFeature", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", + }, + ], + "function": "or", + "type": "function", +} +`; + +exports[`asFiltersByAlertTypeAndConsumer constructs filter for single alert type with multiple authorized consumer 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "alerts", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myOtherApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "or", + "type": "function", + }, + ], + "function": "and", + "type": "function", +} +`; + +exports[`asFiltersByAlertTypeAndConsumer constructs filter for single alert type with single authorized consumer 1`] = ` +Object { + "arguments": Array [ + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.alertTypeId", + }, + Object { + "type": "literal", + "value": "myAppAlertType", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + Object { + "arguments": Array [ + Object { + "type": "literal", + "value": "alert.attributes.consumer", + }, + Object { + "type": "literal", + "value": "myApp", + }, + Object { + "type": "literal", + "value": false, + }, + ], + "function": "is", + "type": "function", + }, + ], + "function": "and", + "type": "function", +} +`; diff --git a/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts b/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts index a7d94210734834..fc895f3e308f47 100644 --- a/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts +++ b/x-pack/plugins/alerts/server/authorization/alerts_authorization.test.ts @@ -6,7 +6,6 @@ import { KibanaRequest } from 'kibana/server'; import { alertTypeRegistryMock } from '../alert_type_registry.mock'; import { securityMock } from '../../../../plugins/security/server/mocks'; -import { esKuery } from '../../../../../src/plugins/data/server'; import { PluginStartContract as FeaturesStartContract, KibanaFeature, @@ -627,11 +626,17 @@ describe('AlertsAuthorization', () => { }); alertTypeRegistry.list.mockReturnValue(setOfAlertTypes); - expect((await alertAuthorization.getFindAuthorizationFilter()).filter).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` - ) - ); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again, instead of toMatchSnapshot(): + // + // expect((await alertAuthorization.getFindAuthorizationFilter()).filter).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` + // ) + // ); + + // This code is the replacement code for above + expect((await alertAuthorization.getFindAuthorizationFilter()).filter).toMatchSnapshot(); expect(auditLogger.alertsAuthorizationSuccess).not.toHaveBeenCalled(); }); diff --git a/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts b/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts index 8249047c0ef399..3d80ff0273db78 100644 --- a/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts +++ b/x-pack/plugins/alerts/server/authorization/alerts_authorization_kuery.test.ts @@ -3,7 +3,6 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ -import { esKuery } from '../../../../../src/plugins/data/server'; import { RecoveredActionGroup } from '../../common'; import { asFiltersByAlertTypeAndConsumer, @@ -30,11 +29,14 @@ describe('asFiltersByAlertTypeAndConsumer', () => { }, ]) ) - ).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(myApp)))` - ) - ); + ).toMatchSnapshot(); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again instead of toMatchSnapshot() + // ).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(myApp)))` + // ) + // ); }); test('constructs filter for single alert type with multiple authorized consumer', async () => { @@ -58,11 +60,14 @@ describe('asFiltersByAlertTypeAndConsumer', () => { }, ]) ) - ).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp)))` - ) - ); + ).toMatchSnapshot(); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again, instead of toMatchSnapshot(): + // ).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp)))` + // ) + // ); }); test('constructs filter for multiple alert types across authorized consumer', async () => { @@ -119,11 +124,14 @@ describe('asFiltersByAlertTypeAndConsumer', () => { }, ]) ) - ).toEqual( - esKuery.fromKueryExpression( - `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` - ) - ); + ).toMatchSnapshot(); + // TODO: once issue https://github.com/elastic/kibana/issues/89473 is + // resolved, we can start using this code again, instead of toMatchSnapshot(): + // ).toEqual( + // esKuery.fromKueryExpression( + // `((alert.attributes.alertTypeId:myAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:myOtherAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)) or (alert.attributes.alertTypeId:mySecondAppAlertType and alert.attributes.consumer:(alerts or myApp or myOtherApp or myAppWithSubFeature)))` + // ) + // ); }); }); diff --git a/x-pack/plugins/alerts/server/routes/create.test.ts b/x-pack/plugins/alerts/server/routes/create.test.ts index fc531821f25b68..d0e21ac99a264c 100644 --- a/x-pack/plugins/alerts/server/routes/create.test.ts +++ b/x-pack/plugins/alerts/server/routes/create.test.ts @@ -82,7 +82,7 @@ describe('createAlertRoute', () => { const [config, handler] = router.post.mock.calls[0]; - expect(config.path).toMatchInlineSnapshot(`"/api/alerts/alert"`); + expect(config.path).toMatchInlineSnapshot(`"/api/alerts/alert/{id?}"`); alertsClient.create.mockResolvedValueOnce(createResult); @@ -125,6 +125,9 @@ describe('createAlertRoute', () => { ], "throttle": "30s", }, + "options": Object { + "id": undefined, + }, }, ] `); @@ -134,6 +137,74 @@ describe('createAlertRoute', () => { }); }); + it('allows providing a custom id', async () => { + const expectedResult = { + ...createResult, + id: 'custom-id', + }; + const licenseState = licenseStateMock.create(); + const router = httpServiceMock.createRouter(); + + createAlertRoute(router, licenseState); + + const [config, handler] = router.post.mock.calls[0]; + + expect(config.path).toMatchInlineSnapshot(`"/api/alerts/alert/{id?}"`); + + alertsClient.create.mockResolvedValueOnce(expectedResult); + + const [context, req, res] = mockHandlerArguments( + { alertsClient }, + { + params: { id: 'custom-id' }, + body: mockedAlert, + }, + ['ok'] + ); + + expect(await handler(context, req, res)).toEqual({ body: expectedResult }); + + expect(alertsClient.create).toHaveBeenCalledTimes(1); + expect(alertsClient.create.mock.calls[0]).toMatchInlineSnapshot(` + Array [ + Object { + "data": Object { + "actions": Array [ + Object { + "group": "default", + "id": "2", + "params": Object { + "foo": true, + }, + }, + ], + "alertTypeId": "1", + "consumer": "bar", + "name": "abc", + "notifyWhen": "onActionGroupChange", + "params": Object { + "bar": true, + }, + "schedule": Object { + "interval": "10s", + }, + "tags": Array [ + "foo", + ], + "throttle": "30s", + }, + "options": Object { + "id": "custom-id", + }, + }, + ] + `); + + expect(res.ok).toHaveBeenCalledWith({ + body: expectedResult, + }); + }); + it('ensures the license allows creating alerts', async () => { const licenseState = licenseStateMock.create(); const router = httpServiceMock.createRouter(); diff --git a/x-pack/plugins/alerts/server/routes/create.ts b/x-pack/plugins/alerts/server/routes/create.ts index 2b6735d9063df2..46151893baef59 100644 --- a/x-pack/plugins/alerts/server/routes/create.ts +++ b/x-pack/plugins/alerts/server/routes/create.ts @@ -45,8 +45,13 @@ export const bodySchema = schema.object({ export const createAlertRoute = (router: AlertingRouter, licenseState: ILicenseState) => { router.post( { - path: `${BASE_ALERT_API_PATH}/alert`, + path: `${BASE_ALERT_API_PATH}/alert/{id?}`, validate: { + params: schema.maybe( + schema.object({ + id: schema.maybe(schema.string()), + }) + ), body: bodySchema, }, }, @@ -59,10 +64,12 @@ export const createAlertRoute = (router: AlertingRouter, licenseState: ILicenseS } const alertsClient = context.alerting.getAlertsClient(); const alert = req.body; + const params = req.params; const notifyWhen = alert?.notifyWhen ? (alert.notifyWhen as AlertNotifyWhenType) : null; try { const alertRes: Alert = await alertsClient.create({ data: { ...alert, notifyWhen }, + options: { id: params?.id }, }); return res.ok({ body: alertRes, diff --git a/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts b/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts index c8bbe599ca44f2..3ff6686138e9a7 100644 --- a/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts +++ b/x-pack/plugins/apm/public/components/app/RumDashboard/VisitorBreakdownMap/useLayerList.ts @@ -16,6 +16,7 @@ import { COLOR_MAP_TYPE, FIELD_ORIGIN, LABEL_BORDER_SIZES, + SOURCE_TYPES, STYLE_TYPE, SYMBOLIZE_AS_TYPES, } from '../../../../../../maps/common/constants'; @@ -29,7 +30,7 @@ import { import { TRANSACTION_PAGE_LOAD } from '../../../../../common/transaction_types'; const ES_TERM_SOURCE_COUNTRY: ESTermSourceDescriptor = { - type: 'ES_TERM_SOURCE', + type: SOURCE_TYPES.ES_TERM_SOURCE, id: '3657625d-17b0-41ef-99ba-3a2b2938655c', indexPatternTitle: 'apm-*', term: 'client.geo.country_iso_code', @@ -46,7 +47,7 @@ const ES_TERM_SOURCE_COUNTRY: ESTermSourceDescriptor = { }; const ES_TERM_SOURCE_REGION: ESTermSourceDescriptor = { - type: 'ES_TERM_SOURCE', + type: SOURCE_TYPES.ES_TERM_SOURCE, id: 'e62a1b9c-d7ff-4fd4-a0f6-0fdc44bb9e41', indexPatternTitle: 'apm-*', term: 'client.geo.region_iso_code', diff --git a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.test.ts b/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.test.ts deleted file mode 100644 index 1f772e0734404e..00000000000000 --- a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.test.ts +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ -import { getTraceUrl } from './ExternalLinks'; - -jest.mock('../../../app/Main/route_config/index.tsx', () => ({ - routes: [ - { - name: 'link_to_trace', - path: '/link-to/trace/:traceId', - }, - ], -})); - -describe('ExternalLinks', () => { - afterAll(() => { - jest.clearAllMocks(); - }); - it('trace link', () => { - expect( - getTraceUrl({ traceId: 'foo', rangeFrom: '123', rangeTo: '456' }) - ).toEqual('/link-to/trace/foo?rangeFrom=123&rangeTo=456'); - }); -}); diff --git a/x-pack/plugins/apm/public/index.ts b/x-pack/plugins/apm/public/index.ts index 40992d7b58e657..d7a8573f2080bf 100644 --- a/x-pack/plugins/apm/public/index.ts +++ b/x-pack/plugins/apm/public/index.ts @@ -22,4 +22,3 @@ export const plugin: PluginInitializer = ( ) => new ApmPlugin(pluginInitializerContext); export { ApmPluginSetup, ApmPluginStart }; -export { getTraceUrl } from './components/shared/Links/apm/ExternalLinks'; diff --git a/x-pack/plugins/apm/server/lib/helpers/calculate_throughput.ts b/x-pack/plugins/apm/server/lib/helpers/calculate_throughput.ts new file mode 100644 index 00000000000000..7fcbe9e798188d --- /dev/null +++ b/x-pack/plugins/apm/server/lib/helpers/calculate_throughput.ts @@ -0,0 +1,16 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { SetupTimeRange } from './setup_request'; + +export function calculateThroughput({ + start, + end, + value, +}: SetupTimeRange & { value: number }) { + const durationAsMinutes = (end - start) / 1000 / 60; + return value / durationAsMinutes; +} diff --git a/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts b/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts index fa4bf6144fb6fb..f844a6ce1c3458 100644 --- a/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts +++ b/x-pack/plugins/apm/server/lib/observability_overview/get_transaction_coordinates.ts @@ -12,6 +12,7 @@ import { rangeFilter } from '../../../common/utils/range_filter'; import { Coordinates } from '../../../../observability/typings/common'; import { Setup, SetupTimeRange } from '../helpers/setup_request'; import { getProcessorEventForAggregatedTransactions } from '../helpers/aggregated_transactions'; +import { calculateThroughput } from '../helpers/calculate_throughput'; export async function getTransactionCoordinates({ setup, @@ -51,12 +52,10 @@ export async function getTransactionCoordinates({ }, }); - const deltaAsMinutes = (end - start) / 1000 / 60; - return ( aggregations?.distribution.buckets.map((bucket) => ({ x: bucket.key, - y: bucket.doc_count / deltaAsMinutes, + y: calculateThroughput({ start, end, value: bucket.doc_count }), })) || [] ); } diff --git a/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts b/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts index 0ac881aeac00e8..2b209f8f6a80a9 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts @@ -13,6 +13,7 @@ import { joinByKey } from '../../../../common/utils/join_by_key'; import { Setup, SetupTimeRange } from '../../helpers/setup_request'; import { getMetrics } from './get_metrics'; import { getDestinationMap } from './get_destination_map'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; export type ServiceDependencyItem = { name: string; @@ -51,7 +52,6 @@ export async function getServiceDependencies({ numBuckets: number; }): Promise { const { start, end } = setup; - const [allMetrics, destinationMap] = await Promise.all([ getMetrics({ setup, @@ -134,8 +134,6 @@ export async function getServiceDependencies({ } ); - const deltaAsMinutes = (end - start) / 60 / 1000; - const destMetrics = { latency: { value: @@ -150,11 +148,18 @@ export async function getServiceDependencies({ throughput: { value: mergedMetrics.value.count > 0 - ? mergedMetrics.value.count / deltaAsMinutes + ? calculateThroughput({ + start, + end, + value: mergedMetrics.value.count, + }) : null, timeseries: mergedMetrics.timeseries.map((point) => ({ x: point.x, - y: point.count > 0 ? point.count / deltaAsMinutes : null, + y: + point.count > 0 + ? calculateThroughput({ start, end, value: point.count }) + : null, })), }, errorRate: { diff --git a/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts b/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts index c5e5269c4409e8..118fbc64146a72 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_instances/get_service_instance_transaction_stats.ts @@ -19,6 +19,7 @@ import { getProcessorEventForAggregatedTransactions, getTransactionDurationFieldForAggregatedTransactions, } from '../../helpers/aggregated_transactions'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; export async function getServiceInstanceTransactionStats({ setup, @@ -104,7 +105,6 @@ export async function getServiceInstanceTransactionStats({ }, }); - const deltaAsMinutes = (end - start) / 60 / 1000; const bucketSizeInMinutes = bucketSize / 60; return ( @@ -128,7 +128,7 @@ export async function getServiceInstanceTransactionStats({ })), }, throughput: { - value: count / deltaAsMinutes, + value: calculateThroughput({ start, end, value: count }), timeseries: timeseries.buckets.map((dateBucket) => ({ x: dateBucket.key, y: dateBucket.doc_count / bucketSizeInMinutes, diff --git a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts index 400c896e380b4b..77642c1f3d65f8 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/get_transaction_groups_for_page.ts @@ -25,6 +25,7 @@ import { getLatencyAggregation, getLatencyValue, } from '../../helpers/latency_aggregation_type'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; export type ServiceOverviewTransactionGroupSortField = | 'name' @@ -64,8 +65,6 @@ export async function getTransactionGroupsForPage({ transactionType: string; latencyAggregationType: LatencyAggregationType; }) { - const deltaAsMinutes = (end - start) / 1000 / 60; - const field = getTransactionDurationFieldForAggregatedTransactions( searchAggregatedTransactions ); @@ -121,7 +120,11 @@ export async function getTransactionGroupsForPage({ latencyAggregationType, aggregation: bucket.latency, }), - throughput: bucket.doc_count / deltaAsMinutes, + throughput: calculateThroughput({ + start, + end, + value: bucket.doc_count, + }), errorRate, }; }) ?? []; diff --git a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts index b0b1cb09dd7845..4b8b1aabbbbcc9 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_transaction_groups/merge_transaction_group_data.ts @@ -6,6 +6,7 @@ import { EVENT_OUTCOME } from '../../../../common/elasticsearch_fieldnames'; import { LatencyAggregationType } from '../../../../common/latency_aggregation_types'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; import { getLatencyValue } from '../../helpers/latency_aggregation_type'; import { TransactionGroupTimeseriesData } from './get_timeseries_data_for_transaction_groups'; import { TransactionGroupWithoutTimeseriesData } from './get_transaction_groups_for_page'; @@ -25,8 +26,6 @@ export function mergeTransactionGroupData({ latencyAggregationType: LatencyAggregationType; transactionType: string; }) { - const deltaAsMinutes = (end - start) / 1000 / 60; - return transactionGroups.map((transactionGroup) => { const groupBucket = timeseriesData.find( ({ key }) => key === transactionGroup.name @@ -52,7 +51,11 @@ export function mergeTransactionGroupData({ ...acc.throughput, timeseries: acc.throughput.timeseries.concat({ x: point.key, - y: point.doc_count / deltaAsMinutes, + y: calculateThroughput({ + start, + end, + value: point.doc_count, + }), }), }, errorRate: { diff --git a/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts b/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts index efbc30169d178a..d7cd13317fd73e 100644 --- a/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts +++ b/x-pack/plugins/apm/server/lib/services/get_services/get_service_transaction_stats.ts @@ -22,6 +22,7 @@ import { getTransactionDurationFieldForAggregatedTransactions, } from '../../helpers/aggregated_transactions'; import { getBucketSize } from '../../helpers/get_bucket_size'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; import { calculateTransactionErrorPercentage, getOutcomeAggregation, @@ -35,16 +36,6 @@ interface AggregationParams { const MAX_NUMBER_OF_SERVICES = 500; -function calculateAvgDuration({ - value, - deltaAsMinutes, -}: { - value: number; - deltaAsMinutes: number; -}) { - return value / deltaAsMinutes; -} - export async function getServiceTransactionStats({ setup, searchAggregatedTransactions, @@ -131,8 +122,6 @@ export async function getServiceTransactionStats({ }, }); - const deltaAsMinutes = (setup.end - setup.start) / 1000 / 60; - return ( response.aggregations?.services.buckets.map((bucket) => { const topTransactionTypeBucket = @@ -171,16 +160,18 @@ export async function getServiceTransactionStats({ ), }, transactionsPerMinute: { - value: calculateAvgDuration({ + value: calculateThroughput({ + start, + end, value: topTransactionTypeBucket.doc_count, - deltaAsMinutes, }), timeseries: topTransactionTypeBucket.timeseries.buckets.map( (dateBucket) => ({ x: dateBucket.key, - y: calculateAvgDuration({ + y: calculateThroughput({ + start, + end, value: dateBucket.doc_count, - deltaAsMinutes, }), }) ), diff --git a/x-pack/plugins/apm/server/lib/services/get_throughput.ts b/x-pack/plugins/apm/server/lib/services/get_throughput.ts index bde826a568da95..15ecc88a019dbd 100644 --- a/x-pack/plugins/apm/server/lib/services/get_throughput.ts +++ b/x-pack/plugins/apm/server/lib/services/get_throughput.ts @@ -16,6 +16,7 @@ import { getProcessorEventForAggregatedTransactions, } from '../helpers/aggregated_transactions'; import { getBucketSize } from '../helpers/get_bucket_size'; +import { calculateThroughput } from '../helpers/calculate_throughput'; import { Setup, SetupTimeRange } from '../helpers/setup_request'; interface Options { @@ -27,16 +28,15 @@ interface Options { type ESResponse = PromiseReturnType; -function transform(response: ESResponse, options: Options) { - const { end, start } = options.setup; - const deltaAsMinutes = (end - start) / 1000 / 60; +function transform(options: Options, response: ESResponse) { if (response.hits.total.value === 0) { return []; } + const { start, end } = options.setup; const buckets = response.aggregations?.throughput.buckets ?? []; - return buckets.map(({ key: x, doc_count: y }) => ({ + return buckets.map(({ key: x, doc_count: value }) => ({ x, - y: y / deltaAsMinutes, + y: calculateThroughput({ start, end, value }), })); } @@ -87,6 +87,6 @@ async function fetcher({ export async function getThroughput(options: Options) { return { - throughput: transform(await fetcher(options), options), + throughput: transform(options, await fetcher(options)), }; } diff --git a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts index 8dfb0a9f65878a..dda6573ea93ef1 100644 --- a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts +++ b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/index.ts @@ -100,9 +100,7 @@ export async function getThroughputCharts({ setup: Setup & SetupTimeRange; searchAggregatedTransactions: boolean; }) { - const { start, end } = setup; - const { bucketSize, intervalString } = getBucketSize({ start, end }); - const durationAsMinutes = (end - start) / 1000 / 60; + const { bucketSize, intervalString } = getBucketSize(setup); const response = await searchThroughput({ serviceName, @@ -117,7 +115,7 @@ export async function getThroughputCharts({ throughputTimeseries: getThroughputBuckets({ throughputResultBuckets: response.aggregations?.throughput.buckets, bucketSize, - durationAsMinutes, + setupTimeRange: setup, }), }; } diff --git a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts index 7e43a0d76f70ae..35d1b0e901deeb 100644 --- a/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts +++ b/x-pack/plugins/apm/server/lib/transactions/get_throughput_charts/transform.ts @@ -7,18 +7,21 @@ import { sortBy } from 'lodash'; import { NOT_AVAILABLE_LABEL } from '../../../../common/i18n'; import { ThroughputChartsResponse } from '.'; +import { calculateThroughput } from '../../helpers/calculate_throughput'; +import { SetupTimeRange } from '../../helpers/setup_request'; type ThroughputResultBuckets = Required['aggregations']['throughput']['buckets']; export function getThroughputBuckets({ throughputResultBuckets = [], bucketSize, - durationAsMinutes, + setupTimeRange, }: { throughputResultBuckets?: ThroughputResultBuckets; bucketSize: number; - durationAsMinutes: number; + setupTimeRange: SetupTimeRange; }) { + const { start, end } = setupTimeRange; const buckets = throughputResultBuckets.map( ({ key: resultKey, timeseries }) => { const dataPoints = timeseries.buckets.map((bucket) => { @@ -38,7 +41,7 @@ export function getThroughputBuckets({ .reduce((a, b) => a + b, 0); // calculate average throughput - const avg = docCountTotal / durationAsMinutes; + const avg = calculateThroughput({ start, end, value: docCountTotal }); return { key, dataPoints, avg }; } diff --git a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts index 43b16cbfad574d..64b1e1a57b4891 100644 --- a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts +++ b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts @@ -23,6 +23,7 @@ import { getTotalLoaded, searchUsageObserver, shimAbortSignal, + shimHitsTotal, } from '../../../../../src/plugins/data/server'; import type { IAsyncSearchOptions } from '../../common'; import { pollSearch } from '../../common'; @@ -69,7 +70,8 @@ export const enhancedEsSearchStrategyProvider = ( ? client.get({ ...params, id }) : client.submit(params); const { body } = await shimAbortSignal(promise, options.abortSignal); - return toAsyncKibanaSearchResponse(body); + const response = shimHitsTotal(body.response, options); + return toAsyncKibanaSearchResponse({ ...body, response }); }; const cancel = async () => { @@ -114,7 +116,7 @@ export const enhancedEsSearchStrategyProvider = ( const esResponse = await shimAbortSignal(promise, options?.abortSignal); const response = esResponse.body as SearchResponse; return { - rawResponse: response, + rawResponse: shimHitsTotal(response, options), ...getTotalLoaded(response), }; } catch (e) { diff --git a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts index 90700f8fa7521b..7ec700607f3e0f 100644 --- a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts +++ b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.test.ts @@ -83,11 +83,11 @@ describe('#create', () => { expect(mockBaseClient.create).toHaveBeenCalledWith('unknown-type', attributes, options); }); - it('fails if type is registered and ID is specified', async () => { + it('fails if type is registered and non-UUID ID is specified', async () => { const attributes = { attrOne: 'one', attrSecret: 'secret', attrThree: 'three' }; await expect(wrapper.create('known-type', attributes, { id: 'some-id' })).rejects.toThrowError( - 'Predefined IDs are not allowed for saved objects with encrypted attributes, unless the ID has been generated using `SavedObjectsUtils.generateId`.' + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.' ); expect(mockBaseClient.create).not.toHaveBeenCalled(); @@ -310,7 +310,7 @@ describe('#bulkCreate', () => { ); }); - it('fails if ID is specified for registered type', async () => { + it('fails if non-UUID ID is specified for registered type', async () => { const attributes = { attrOne: 'one', attrSecret: 'secret', attrThree: 'three' }; const bulkCreateParams = [ @@ -319,7 +319,7 @@ describe('#bulkCreate', () => { ]; await expect(wrapper.bulkCreate(bulkCreateParams)).rejects.toThrowError( - 'Predefined IDs are not allowed for saved objects with encrypted attributes, unless the ID has been generated using `SavedObjectsUtils.generateId`.' + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.' ); expect(mockBaseClient.bulkCreate).not.toHaveBeenCalled(); diff --git a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts index c3008a8e865052..21475f6a4f5d28 100644 --- a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts +++ b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts @@ -59,7 +59,7 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return await this.options.baseClient.create(type, attributes, options); } - const id = getValidId(options.id, options.version, options.overwrite); + const id = this.getValidId(options.id, options.version, options.overwrite); const namespace = getDescriptorNamespace( this.options.baseTypeRegistry, type, @@ -93,7 +93,7 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return object; } - const id = getValidId(object.id, object.version, options?.overwrite); + const id = this.getValidId(object.id, object.version, options?.overwrite); const namespace = getDescriptorNamespace( this.options.baseTypeRegistry, object.type, @@ -307,27 +307,27 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return response; } -} -// Saved objects with encrypted attributes should have IDs that are hard to guess especially -// since IDs are part of the AAD used during encryption, that's why we control them within this -// wrapper and don't allow consumers to specify their own IDs directly unless overwriting the original document. -function getValidId( - id: string | undefined, - version: string | undefined, - overwrite: boolean | undefined -) { - if (id) { - // only allow a specified ID if we're overwriting an existing ESO with a Version - // this helps us ensure that the document really was previously created using ESO - // and not being used to get around the specified ID limitation - const canSpecifyID = (overwrite && version) || SavedObjectsUtils.isRandomId(id); - if (!canSpecifyID) { - throw new Error( - 'Predefined IDs are not allowed for saved objects with encrypted attributes, unless the ID has been generated using `SavedObjectsUtils.generateId`.' - ); + // Saved objects with encrypted attributes should have IDs that are hard to guess especially + // since IDs are part of the AAD used during encryption, that's why we control them within this + // wrapper and don't allow consumers to specify their own IDs directly unless overwriting the original document. + private getValidId( + id: string | undefined, + version: string | undefined, + overwrite: boolean | undefined + ) { + if (id) { + // only allow a specified ID if we're overwriting an existing ESO with a Version + // this helps us ensure that the document really was previously created using ESO + // and not being used to get around the specified ID limitation + const canSpecifyID = (overwrite && version) || SavedObjectsUtils.isRandomId(id); + if (!canSpecifyID) { + throw this.errors.createBadRequestError( + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.' + ); + } + return id; } - return id; + return SavedObjectsUtils.generateId(); } - return SavedObjectsUtils.generateId(); } diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts index 48b8a06b2549ca..5e106a7f42f577 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts @@ -311,3 +311,157 @@ export const SOURCE_NAME_LABEL = i18n.translate( defaultMessage: 'Source name', } ); + +export const ORG_SOURCES_LINK = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.link', + { + defaultMessage: 'Add an organization content source', + } +); + +export const ORG_SOURCES_HEADER_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.title', + { + defaultMessage: 'Organization sources', + } +); + +export const ORG_SOURCES_HEADER_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.description', + { + defaultMessage: + 'Organization sources are available to the entire organization and can be assigned to specific user groups.', + } +); + +export const PRIVATE_LINK_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.link', + { + defaultMessage: 'Add a private content source', + } +); + +export const PRIVATE_CAN_CREATE_PAGE_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.canCreate.title', + { + defaultMessage: 'Manage private content sources', + } +); + +export const PRIVATE_VIEW_ONLY_PAGE_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.vewOnly.title', + { + defaultMessage: 'Review Group Sources', + } +); + +export const PRIVATE_VIEW_ONLY_PAGE_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.vewOnly.description', + { + defaultMessage: 'Review the status of all sources shared with your Group.', + } +); + +export const PRIVATE_CAN_CREATE_PAGE_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.canCreate.description', + { + defaultMessage: + 'Review the status of all connected private sources, and manage private sources for your account.', + } +); + +export const PRIVATE_HEADER_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.header.title', + { + defaultMessage: 'My private content sources', + } +); + +export const PRIVATE_HEADER_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.header.description', + { + defaultMessage: 'Private content sources are available only to you.', + } +); + +export const PRIVATE_SHARED_SOURCES_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.privateShared.header.title', + { + defaultMessage: 'Shared content sources', + } +); + +export const PRIVATE_EMPTY_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.empty.title', + { + defaultMessage: 'You have no private sources', + } +); +export const SHARED_EMPTY_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.shared.empty.title', + { + defaultMessage: 'No content source available', + } +); + +export const SHARED_EMPTY_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.shared.empty.description', + { + defaultMessage: + 'Once content sources are shared with you, they will be displayed here, and available via the search experience.', + } +); + +export const AND = i18n.translate('xpack.enterpriseSearch.workplaceSearch.and', { + defaultMessage: 'and', +}); + +export const LICENSE_CALLOUT_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.licenseCallout.title', + { + defaultMessage: 'Private Sources are no longer available', + } +); + +export const LICENSE_CALLOUT_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.licenseCallout.description', + { + defaultMessage: 'Contact your search experience administrator for more information.', + } +); + +export const SOURCE_DISABLED_CALLOUT_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.title', + { + defaultMessage: 'Content source is disabled', + } +); + +export const SOURCE_DISABLED_CALLOUT_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.description', + { + defaultMessage: + 'Your organization’s license level has changed. Your data is safe, but document-level permissions are no longer supported and searching of this source has been disabled. Upgrade to a Platinum license to re-enable this source.', + } +); + +export const SOURCE_DISABLED_CALLOUT_BUTTON = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.button', + { + defaultMessage: 'Explore Platinum license', + } +); + +export const DOCUMENT_PERMISSIONS_LINK = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.documentPermissionsLink', + { + defaultMessage: 'Learn more about document-level permission configuration', + } +); + +export const UNDERSTAND_BUTTON = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.understandButton', + { + defaultMessage: 'I understand', + } +); diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx index fdb536dd797714..3081301fe0a9f1 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx @@ -12,6 +12,12 @@ import { Link, Redirect } from 'react-router-dom'; import { EuiButton } from '@elastic/eui'; import { ADD_SOURCE_PATH, getSourcesPath } from '../../routes'; +import { + ORG_SOURCES_LINK, + ORG_SOURCES_HEADER_TITLE, + ORG_SOURCES_HEADER_DESCRIPTION, +} from './constants'; + import { Loading } from '../../../shared/loading'; import { ContentSection } from '../../components/shared/content_section'; import { SourcesTable } from '../../components/shared/sources_table'; @@ -21,11 +27,6 @@ import { SourcesLogic } from './sources_logic'; import { SourcesView } from './sources_view'; -const ORG_LINK_TITLE = 'Add an organization content source'; -const ORG_HEADER_TITLE = 'Organization sources'; -const ORG_HEADER_DESCRIPTION = - 'Organization sources are available to the entire organization and can be assigned to specific user groups.'; - export const OrganizationSources: React.FC = () => { const { initializeSources, setSourceSearchability, resetSourcesState } = useActions(SourcesLogic); @@ -40,28 +41,22 @@ export const OrganizationSources: React.FC = () => { if (contentSources.length === 0) return ; - const linkTitle = ORG_LINK_TITLE; - const headerTitle = ORG_HEADER_TITLE; - const headerDescription = ORG_HEADER_DESCRIPTION; - const sectionTitle = ''; - const sectionDescription = ''; - return ( - {linkTitle} + {ORG_SOURCES_LINK} } - description={headerDescription} + description={ORG_SOURCES_HEADER_DESCRIPTION} alignItems="flexStart" /> - + { const { hasPlatinumLicense } = useValues(LicensingLogic); const { initializeSources, setSourceSearchability, resetSourcesState } = useActions(SourcesLogic); @@ -112,7 +119,7 @@ export const PrivateSources: React.FC = () => { - You have no private sources} /> + {PRIVATE_EMPTY_TITLE}} /> @@ -124,13 +131,8 @@ export const PrivateSources: React.FC = () => { No content source available} - body={ -

- Once content sources are shared with you, they will be displayed here, and available - via the search experience. -

- } + title={

{SHARED_EMPTY_TITLE}

} + body={

{SHARED_EMPTY_DESCRIPTION}

} /> @@ -140,16 +142,21 @@ export const PrivateSources: React.FC = () => { const hasPrivateSources = privateContentSources?.length > 0; const privateSources = hasPrivateSources ? privateSourcesTable : privateSourcesEmptyState; - const groupsSentence = `${groups.slice(0, groups.length - 1).join(', ')}, and ${groups.slice( + const groupsSentence = `${groups.slice(0, groups.length - 1).join(', ')}, ${AND} ${groups.slice( -1 )}`; const sharedSources = ( @@ -157,8 +164,8 @@ export const PrivateSources: React.FC = () => { const licenseCallout = ( <> - -

Contact your search experience administrator for more information.

+ +

{LICENSE_CALLOUT_DESCRIPTION}

 diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx index f46743778a1683..67995a4920925d 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx @@ -17,6 +17,12 @@ import { SendWorkplaceSearchTelemetry as SendTelemetry } from '../../../shared/t import { NAV } from '../../constants'; +import { + SOURCE_DISABLED_CALLOUT_TITLE, + SOURCE_DISABLED_CALLOUT_DESCRIPTION, + SOURCE_DISABLED_CALLOUT_BUTTON, +} from './constants'; + import { ENT_SEARCH_LICENSE_MANAGEMENT, REINDEX_JOB_PATH, @@ -80,14 +86,10 @@ export const SourceRouter: React.FC = () => { const callout = ( <> - -

- Your organization’s license level has changed. Your data is safe, but document-level - permissions are no longer supported and searching of this source has been disabled. - Upgrade to a Platinum license to re-enable this source. -

+ +

{SOURCE_DISABLED_CALLOUT_DESCRIPTION}

- Explore Platinum license + {SOURCE_DISABLED_CALLOUT_BUTTON} diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx index 9e6c8f5b7319ee..f8a2d345c85130 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx @@ -8,6 +8,9 @@ import React from 'react'; import { useActions, useValues } from 'kea'; +import { i18n } from '@kbn/i18n'; +import { FormattedMessage } from '@kbn/i18n/react'; + import { EuiButton, EuiLink, @@ -27,6 +30,12 @@ import { SourceIcon } from '../../components/shared/source_icon'; import { EXTERNAL_IDENTITIES_DOCS_URL, DOCUMENT_PERMISSIONS_DOCS_URL } from '../../routes'; +import { + EXTERNAL_IDENTITIES_LINK, + DOCUMENT_PERMISSIONS_LINK, + UNDERSTAND_BUTTON, +} from './constants'; + import { SourcesLogic } from './sources_logic'; interface SourcesViewProps { @@ -59,35 +68,53 @@ export const SourcesView: React.FC = ({ children }) => { - {addedSourceName} requires additional configuration + + {i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sourcesView.modal.heading', + { + defaultMessage: '{addedSourceName} requires additional configuration', + values: { addedSourceName }, + } + )} +

- {addedSourceName} has been successfully connected and initial content synchronization - is already underway. Since you have elected to synchronize document-level permission - information, you must now provide user and group mappings using the  - - External Identities API - - . + + {EXTERNAL_IDENTITIES_LINK} + + ), + }} + />

- Documents will not be searchable from Workplace Search until user and group mappings - have been configured.  - - Learn more about document-level permission configuration - - . + + {DOCUMENT_PERMISSIONS_LINK} + + ), + }} + />

- I understand + {UNDERSTAND_BUTTON} diff --git a/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts b/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts index e1fa2a0b18b593..95f99976451764 100644 --- a/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts +++ b/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts @@ -11,7 +11,6 @@ import { TemplateRef, IndexTemplate, IndexTemplateMappings, - DataType, } from '../../../../types'; import { getRegistryDataStreamAssetBaseName } from '../index'; @@ -26,8 +25,8 @@ interface MultiFields { export interface IndexTemplateMapping { [key: string]: any; } -export interface CurrentIndex { - indexName: string; +export interface CurrentDataStream { + dataStreamName: string; indexTemplate: IndexTemplate; } const DEFAULT_SCALING_FACTOR = 1000; @@ -348,33 +347,31 @@ export const updateCurrentWriteIndices = async ( ): Promise => { if (!templates.length) return; - const allIndices = await queryIndicesFromTemplates(callCluster, templates); + const allIndices = await queryDataStreamsFromTemplates(callCluster, templates); if (!allIndices.length) return; - return updateAllIndices(allIndices, callCluster); + return updateAllDataStreams(allIndices, callCluster); }; -function isCurrentIndex(item: CurrentIndex[] | undefined): item is CurrentIndex[] { +function isCurrentDataStream(item: CurrentDataStream[] | undefined): item is CurrentDataStream[] { return item !== undefined; } -const queryIndicesFromTemplates = async ( +const queryDataStreamsFromTemplates = async ( callCluster: CallESAsCurrentUser, templates: TemplateRef[] -): Promise => { - const indexPromises = templates.map((template) => { - return getIndices(callCluster, template); +): Promise => { + const dataStreamPromises = templates.map((template) => { + return getDataStreams(callCluster, template); }); - const indexObjects = await Promise.all(indexPromises); - return indexObjects.filter(isCurrentIndex).flat(); + const dataStreamObjects = await Promise.all(dataStreamPromises); + return dataStreamObjects.filter(isCurrentDataStream).flat(); }; -const getIndices = async ( +const getDataStreams = async ( callCluster: CallESAsCurrentUser, template: TemplateRef -): Promise => { +): Promise => { const { templateName, indexTemplate } = template; - // Until ES provides a way to update mappings of a data stream - // get the last index of the data stream, which is the current write index const res = await callCluster('transport.request', { method: 'GET', path: `/_data_stream/${templateName}-*`, @@ -382,26 +379,28 @@ const getIndices = async ( const dataStreams = res.data_streams; if (!dataStreams.length) return; return dataStreams.map((dataStream: any) => ({ - indexName: dataStream.indices[dataStream.indices.length - 1].index_name, + dataStreamName: dataStream.name, indexTemplate, })); }; -const updateAllIndices = async ( - indexNameWithTemplates: CurrentIndex[], +const updateAllDataStreams = async ( + indexNameWithTemplates: CurrentDataStream[], callCluster: CallESAsCurrentUser ): Promise => { - const updateIndexPromises = indexNameWithTemplates.map(({ indexName, indexTemplate }) => { - return updateExistingIndex({ indexName, callCluster, indexTemplate }); - }); - await Promise.all(updateIndexPromises); + const updatedataStreamPromises = indexNameWithTemplates.map( + ({ dataStreamName, indexTemplate }) => { + return updateExistingDataStream({ dataStreamName, callCluster, indexTemplate }); + } + ); + await Promise.all(updatedataStreamPromises); }; -const updateExistingIndex = async ({ - indexName, +const updateExistingDataStream = async ({ + dataStreamName, callCluster, indexTemplate, }: { - indexName: string; + dataStreamName: string; callCluster: CallESAsCurrentUser; indexTemplate: IndexTemplate; }) => { @@ -416,53 +415,13 @@ const updateExistingIndex = async ({ // try to update the mappings first try { await callCluster('indices.putMapping', { - index: indexName, + index: dataStreamName, body: mappings, + write_index_only: true, }); // if update fails, rollover data stream } catch (err) { try { - // get the data_stream values to compose datastream name - const searchDataStreamFieldsResponse = await callCluster('search', { - index: indexTemplate.index_patterns[0], - body: { - size: 1, - _source: ['data_stream.namespace', 'data_stream.type', 'data_stream.dataset'], - query: { - bool: { - filter: [ - { - exists: { - field: 'data_stream.type', - }, - }, - { - exists: { - field: 'data_stream.dataset', - }, - }, - { - exists: { - field: 'data_stream.namespace', - }, - }, - ], - }, - }, - }, - }); - if (searchDataStreamFieldsResponse.hits.total.value === 0) - throw new Error('data_stream fields are missing from datastream indices'); - const { - dataset, - namespace, - type, - }: { - dataset: string; - namespace: string; - type: DataType; - } = searchDataStreamFieldsResponse.hits.hits[0]._source.data_stream; - const dataStreamName = `${type}-${dataset}-${namespace}`; const path = `/${dataStreamName}/_rollover`; await callCluster('transport.request', { method: 'POST', @@ -478,10 +437,10 @@ const updateExistingIndex = async ({ if (!settings.index.default_pipeline) return; try { await callCluster('indices.putSettings', { - index: indexName, + index: dataStreamName, body: { index: { default_pipeline: settings.index.default_pipeline } }, }); } catch (err) { - throw new Error(`could not update index template settings for ${indexName}`); + throw new Error(`could not update index template settings for ${dataStreamName}`); } }; diff --git a/x-pack/plugins/grokdebugger/tsconfig.json b/x-pack/plugins/grokdebugger/tsconfig.json new file mode 100644 index 00000000000000..34cf8d74c0024b --- /dev/null +++ b/x-pack/plugins/grokdebugger/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "common/**/*", + "public/**/*", + "server/**/*", + "../../typings/**/*", + ], + "references": [ + { "path": "../../../src/core/tsconfig.json" }, + { "path": "../../../src/plugins/dev_tools/tsconfig.json"}, + { "path": "../../../src/plugins/home/tsconfig.json"}, + { "path": "../../../src/plugins/kibana_react/tsconfig.json" }, + { "path": "../../../src/plugins/es_ui_shared/tsconfig.json" }, + { "path": "../licensing/tsconfig.json" } + ] +} diff --git a/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts b/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts index 87f5408f6ca421..3221054839865d 100644 --- a/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts +++ b/x-pack/plugins/index_management/__jest__/client_integration/home/data_streams_tab.test.ts @@ -71,7 +71,7 @@ describe('Data Streams tab', () => { test('when Fleet is enabled, links to Fleet', async () => { testBed = await setup({ - plugins: { fleet: { hi: 'ok' } }, + plugins: { isFleetEnabled: true }, }); await act(async () => { diff --git a/x-pack/plugins/index_management/public/application/app_context.tsx b/x-pack/plugins/index_management/public/application/app_context.tsx index 91bcfe5ed55c04..4e5164562207de 100644 --- a/x-pack/plugins/index_management/public/application/app_context.tsx +++ b/x-pack/plugins/index_management/public/application/app_context.tsx @@ -8,9 +8,8 @@ import React, { createContext, useContext } from 'react'; import { ScopedHistory } from 'kibana/public'; import { ManagementAppMountParams } from 'src/plugins/management/public'; import { UsageCollectionSetup } from 'src/plugins/usage_collection/public'; -import { CoreSetup, CoreStart } from '../../../../../src/core/public'; -import { FleetSetup } from '../../../fleet/public'; +import { CoreSetup, CoreStart } from '../../../../../src/core/public'; import { UiMetricService, NotificationService, HttpService } from './services'; import { ExtensionsService } from '../services'; import { SharePluginStart } from '../../../../../src/plugins/share/public'; @@ -24,7 +23,7 @@ export interface AppDependencies { }; plugins: { usageCollection: UsageCollectionSetup; - fleet?: FleetSetup; + isFleetEnabled: boolean; }; services: { uiMetricService: UiMetricService; diff --git a/x-pack/plugins/index_management/public/application/mount_management_section.ts b/x-pack/plugins/index_management/public/application/mount_management_section.ts index b94718c14d3aac..f4136a977df1a6 100644 --- a/x-pack/plugins/index_management/public/application/mount_management_section.ts +++ b/x-pack/plugins/index_management/public/application/mount_management_section.ts @@ -9,7 +9,6 @@ import { CoreSetup } from 'src/core/public'; import { ManagementAppMountParams } from 'src/plugins/management/public/'; import { UsageCollectionSetup } from 'src/plugins/usage_collection/public'; -import { FleetSetup } from '../../../fleet/public'; import { UIM_APP_NAME } from '../../common/constants'; import { PLUGIN } from '../../common/constants/plugin'; import { ExtensionsService } from '../services'; @@ -50,7 +49,7 @@ export async function mountManagementSection( usageCollection: UsageCollectionSetup, params: ManagementAppMountParams, extensionsService: ExtensionsService, - fleet?: FleetSetup + isFleetEnabled: boolean ) { const { element, setBreadcrumbs, history } = params; const [core, startDependencies] = await coreSetup.getStartServices(); @@ -80,7 +79,7 @@ export async function mountManagementSection( }, plugins: { usageCollection, - fleet, + isFleetEnabled, }, services: { httpService, notificationService, uiMetricService, extensionsService }, history, diff --git a/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx b/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx index 64d874c76afb3c..07eccd23d9f44e 100644 --- a/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx +++ b/x-pack/plugins/index_management/public/application/sections/home/data_stream_list/data_stream_list.tsx @@ -52,7 +52,7 @@ export const DataStreamList: React.FunctionComponent {' ' /* We need this space to separate these two sentences. */} - {fleet ? ( + {isFleetEnabled ? ( { return { app: 'apm', - hash: getTraceUrl({ traceId, rangeFrom, rangeTo }), + pathname: getApmTraceUrl({ traceId, rangeFrom, rangeTo }), }; }; diff --git a/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx b/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx index ed34a32012bd29..71cfab79ba0cc1 100644 --- a/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx +++ b/x-pack/plugins/infra/public/pages/metrics/inventory_view/components/waffle/stepped_gradient_legend.tsx @@ -16,13 +16,12 @@ interface Props { bounds: InfraWaffleMapBounds; formatter: InfraFormatter; } - +type TickValue = 0 | 1; export const SteppedGradientLegend: React.FC = ({ legend, bounds, formatter }) => { return ( - @@ -39,7 +38,7 @@ export const SteppedGradientLegend: React.FC = ({ legend, bounds, formatt interface TickProps { bounds: InfraWaffleMapBounds; - value: number; + value: TickValue; formatter: InfraFormatter; } diff --git a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts index 49f4b56532936c..9f1c2f90635a31 100644 --- a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts +++ b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.test.ts @@ -37,14 +37,14 @@ describe('calculateBoundsFromNodes', () => { const bounds = calculateBoundsFromNodes(nodes); expect(bounds).toEqual({ min: 0.2, - max: 1.5, + max: 0.5, }); }); it('should have a minimum of 0 for only a single node', () => { const bounds = calculateBoundsFromNodes([nodes[0]]); expect(bounds).toEqual({ min: 0, - max: 1.5, + max: 0.5, }); }); it('should return zero for empty nodes', () => { diff --git a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts index 6eb64971efbd76..ff1093a795a10c 100644 --- a/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts +++ b/x-pack/plugins/infra/public/pages/metrics/inventory_view/lib/calculate_bounds_from_nodes.ts @@ -9,23 +9,17 @@ import { SnapshotNode } from '../../../../../common/http_api/snapshot_api'; import { InfraWaffleMapBounds } from '../../../../lib/lib'; export const calculateBoundsFromNodes = (nodes: SnapshotNode[]): InfraWaffleMapBounds => { - const maxValues = nodes.map((node) => { + const values = nodes.map((node) => { const metric = first(node.metrics); - if (!metric) return 0; - return metric.max; - }); - const minValues = nodes.map((node) => { - const metric = first(node.metrics); - if (!metric) return 0; - return metric.value; + return !metric || !metric.value ? 0 : metric.value; }); // if there is only one value then we need to set the bottom range to zero for min // otherwise the legend will look silly since both values are the same for top and // bottom. - if (minValues.length === 1) { - minValues.unshift(0); + if (values.length === 1) { + values.unshift(0); } - const maxValue = max(maxValues) || 0; - const minValue = min(minValues) || 0; + const maxValue = max(values) || 0; + const minValue = min(values) || 0; return { min: isFinite(minValue) ? minValue : 0, max: isFinite(maxValue) ? maxValue : 0 }; }; diff --git a/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts b/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts index 705d7bf34c1c66..a682500e5af18a 100644 --- a/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts +++ b/x-pack/plugins/infra/server/lib/adapters/framework/adapter_types.ts @@ -14,7 +14,6 @@ import { } from '../../../../../../../src/plugins/data/server'; import { HomeServerPluginSetup } from '../../../../../../../src/plugins/home/server'; import { VisTypeTimeseriesSetup } from '../../../../../../../src/plugins/vis_type_timeseries/server'; -import { APMPluginSetup } from '../../../../../../plugins/apm/server'; import { PluginSetupContract as FeaturesPluginSetup } from '../../../../../../plugins/features/server'; import { SpacesPluginSetup } from '../../../../../../plugins/spaces/server'; import { PluginSetupContract as AlertingPluginContract } from '../../../../../alerts/server'; @@ -28,7 +27,6 @@ export interface InfraServerPluginSetupDeps { usageCollection: UsageCollectionSetup; visTypeTimeseries: VisTypeTimeseriesSetup; features: FeaturesPluginSetup; - apm: APMPluginSetup; alerts: AlertingPluginContract; ml?: MlPluginSetup; } diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.scss b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.scss similarity index 100% rename from x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.scss rename to x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.scss diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.tsx b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.tsx similarity index 98% rename from x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.tsx rename to x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.tsx index 9519d849e5d900..cbbd032f25b3dd 100644 --- a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/add_documents_accordion.tsx +++ b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/add_docs_accordion.tsx @@ -15,7 +15,7 @@ import { useKibana } from '../../../../../../../../shared_imports'; import { useIsMounted } from '../../../../../use_is_mounted'; import { AddDocumentForm } from '../add_document_form'; -import './add_documents_accordion.scss'; +import './add_docs_accordion.scss'; const DISCOVER_URL_GENERATOR_ID = 'DISCOVER_APP_URL_GENERATOR'; diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/index.ts b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/index.ts similarity index 78% rename from x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/index.ts rename to x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/index.ts index cb00ec640b5a65..5f7939690fa55c 100644 --- a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_documents_accordion/index.ts +++ b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/add_docs_accordion/index.ts @@ -4,4 +4,4 @@ * you may not use this file except in compliance with the Elastic License. */ -export { AddDocumentsAccordion } from './add_documents_accordion'; +export { AddDocumentsAccordion } from './add_docs_accordion'; diff --git a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx index 6888f947b86067..dccc343e9359ce 100644 --- a/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx +++ b/x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/test_pipeline/test_pipeline_tabs/tab_documents/tab_documents.tsx @@ -23,7 +23,7 @@ import { Form, } from '../../../../../../../shared_imports'; import { Document } from '../../../../types'; -import { AddDocumentsAccordion } from './add_documents_accordion'; +import { AddDocumentsAccordion } from './add_docs_accordion'; import { ResetDocumentsModal } from './reset_documents_modal'; import './tab_documents.scss'; diff --git a/x-pack/plugins/ingest_pipelines/tsconfig.json b/x-pack/plugins/ingest_pipelines/tsconfig.json new file mode 100644 index 00000000000000..5d78992600e817 --- /dev/null +++ b/x-pack/plugins/ingest_pipelines/tsconfig.json @@ -0,0 +1,28 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "common/**/*", + "public/**/*", + "server/**/*", + "__jest__/**/*", + "../../typings/**/*" + ], + "references": [ + { "path": "../../../src/core/tsconfig.json" }, + { "path": "../licensing/tsconfig.json" }, + { "path": "../features/tsconfig.json" }, + { "path": "../security/tsconfig.json" }, + { "path": "../../../src/plugins/es_ui_shared/tsconfig.json"}, + { "path": "../../../src/plugins/kibana_react/tsconfig.json"}, + { "path": "../../../src/plugins/management/tsconfig.json"}, + { "path": "../../../src/plugins/share/tsconfig.json"}, + { "path": "../../../src/plugins/usage_collection/tsconfig.json"}, + ] +} diff --git a/x-pack/plugins/maps/common/constants.ts b/x-pack/plugins/maps/common/constants.ts index b86d48bfccdabd..c8db433a372350 100644 --- a/x-pack/plugins/maps/common/constants.ts +++ b/x-pack/plugins/maps/common/constants.ts @@ -85,6 +85,7 @@ export enum SOURCE_TYPES { REGIONMAP_FILE = 'REGIONMAP_FILE', GEOJSON_FILE = 'GEOJSON_FILE', MVT_SINGLE_LAYER = 'MVT_SINGLE_LAYER', + TABLE_SOURCE = 'TABLE_SOURCE', } export enum FIELD_ORIGIN { diff --git a/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts b/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts index b67f05cb169fd5..65cc145e20c895 100644 --- a/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts +++ b/x-pack/plugins/maps/common/descriptor_types/layer_descriptor_types.ts @@ -8,11 +8,11 @@ import { Query } from 'src/plugins/data/public'; import { StyleDescriptor, VectorStyleDescriptor } from './style_property_descriptor_types'; import { DataRequestDescriptor } from './data_request_descriptor_types'; -import { AbstractSourceDescriptor, ESTermSourceDescriptor } from './source_descriptor_types'; +import { AbstractSourceDescriptor, TermJoinSourceDescriptor } from './source_descriptor_types'; export type JoinDescriptor = { leftField?: string; - right: ESTermSourceDescriptor; + right: TermJoinSourceDescriptor; }; export type LayerDescriptor = { diff --git a/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts b/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts index b849b42429cf6f..dca7ae766f375f 100644 --- a/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts +++ b/x-pack/plugins/maps/common/descriptor_types/source_descriptor_types.ts @@ -8,7 +8,14 @@ import { FeatureCollection } from 'geojson'; import { Query } from 'src/plugins/data/public'; import { SortDirection } from 'src/plugins/data/common/search'; -import { AGG_TYPE, GRID_RESOLUTION, RENDER_AS, SCALING_TYPES, MVT_FIELD_TYPE } from '../constants'; +import { + AGG_TYPE, + GRID_RESOLUTION, + RENDER_AS, + SCALING_TYPES, + MVT_FIELD_TYPE, + SOURCE_TYPES, +} from '../constants'; export type AttributionDescriptor = { attributionText?: string; @@ -105,6 +112,7 @@ export type ESTermSourceDescriptor = AbstractESAggSourceDescriptor & { term: string; // term field name whereQuery?: Query; size?: number; + type: SOURCE_TYPES.ES_TERM_SOURCE; }; export type KibanaRegionmapSourceDescriptor = AbstractSourceDescriptor & { @@ -156,14 +164,24 @@ export type TiledSingleLayerVectorSourceDescriptor = AbstractSourceDescriptor & tooltipProperties: string[]; }; -export type GeoJsonFileFieldDescriptor = { +export type InlineFieldDescriptor = { name: string; type: 'string' | 'number'; }; export type GeojsonFileSourceDescriptor = { - __fields?: GeoJsonFileFieldDescriptor[]; + __fields?: InlineFieldDescriptor[]; __featureCollection: FeatureCollection; name: string; type: string; }; + +export type TableSourceDescriptor = { + id: string; + type: SOURCE_TYPES.TABLE_SOURCE; + __rows: Array<{ [key: string]: string | number }>; + __columns: InlineFieldDescriptor[]; + term: string; +}; + +export type TermJoinSourceDescriptor = ESTermSourceDescriptor | TableSourceDescriptor; diff --git a/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.test.ts b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.test.ts new file mode 100644 index 00000000000000..c9ab4b00d8923c --- /dev/null +++ b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.test.ts @@ -0,0 +1,45 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { addTypeToTermJoin } from './add_type_to_termjoin'; +import { LAYER_TYPE, SOURCE_TYPES } from '../constants'; +import { LayerDescriptor } from '../descriptor_types'; + +describe('addTypeToTermJoin', () => { + test('Should handle missing type attribute', () => { + const layerListJSON = JSON.stringify(([ + { + type: LAYER_TYPE.VECTOR, + joins: [ + { + right: {}, + }, + { + right: { + type: SOURCE_TYPES.TABLE_SOURCE, + }, + }, + { + right: { + type: SOURCE_TYPES.ES_TERM_SOURCE, + }, + }, + ], + }, + ] as unknown) as LayerDescriptor[]); + + const attributes = { + title: 'my map', + layerListJSON, + }; + + const { layerListJSON: migratedLayerListJSON } = addTypeToTermJoin({ attributes }); + const migratedLayerList = JSON.parse(migratedLayerListJSON!); + expect(migratedLayerList[0].joins[0].right.type).toEqual(SOURCE_TYPES.ES_TERM_SOURCE); + expect(migratedLayerList[0].joins[1].right.type).toEqual(SOURCE_TYPES.TABLE_SOURCE); + expect(migratedLayerList[0].joins[2].right.type).toEqual(SOURCE_TYPES.ES_TERM_SOURCE); + }); +}); diff --git a/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.ts b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.ts new file mode 100644 index 00000000000000..84e13eb6c3947e --- /dev/null +++ b/x-pack/plugins/maps/common/migrations/add_type_to_termjoin.ts @@ -0,0 +1,48 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { MapSavedObjectAttributes } from '../map_saved_object_type'; +import { JoinDescriptor, LayerDescriptor } from '../descriptor_types'; +import { LAYER_TYPE, SOURCE_TYPES } from '../constants'; + +// enforce type property on joins. It's possible older saved-objects do not have this correctly filled in +// e.g. sample-data was missing the right.type field. +// This is just to be safe. +export function addTypeToTermJoin({ + attributes, +}: { + attributes: MapSavedObjectAttributes; +}): MapSavedObjectAttributes { + if (!attributes || !attributes.layerListJSON) { + return attributes; + } + + const layerList: LayerDescriptor[] = JSON.parse(attributes.layerListJSON); + + layerList.forEach((layer: LayerDescriptor) => { + if (layer.type !== LAYER_TYPE.VECTOR) { + return; + } + + if (!layer.joins) { + return; + } + layer.joins.forEach((join: JoinDescriptor) => { + if (!join.right) { + return; + } + + if (typeof join.right.type === 'undefined') { + join.right.type = SOURCE_TYPES.ES_TERM_SOURCE; + } + }); + }); + + return { + ...attributes, + layerListJSON: JSON.stringify(layerList), + }; +} diff --git a/x-pack/plugins/maps/public/classes/fields/geojson_file_field.ts b/x-pack/plugins/maps/public/classes/fields/inline_field.ts similarity index 80% rename from x-pack/plugins/maps/public/classes/fields/geojson_file_field.ts rename to x-pack/plugins/maps/public/classes/fields/inline_field.ts index ae42b09d491c58..287edbd07cce8a 100644 --- a/x-pack/plugins/maps/public/classes/fields/geojson_file_field.ts +++ b/x-pack/plugins/maps/public/classes/fields/inline_field.ts @@ -7,10 +7,9 @@ import { FIELD_ORIGIN } from '../../../common/constants'; import { IField, AbstractField } from './field'; import { IVectorSource } from '../sources/vector_source'; -import { GeoJsonFileSource } from '../sources/geojson_file_source'; -export class GeoJsonFileField extends AbstractField implements IField { - private readonly _source: GeoJsonFileSource; +export class InlineField extends AbstractField implements IField { + private readonly _source: T; private readonly _dataType: string; constructor({ @@ -20,7 +19,7 @@ export class GeoJsonFileField extends AbstractField implements IField { dataType, }: { fieldName: string; - source: GeoJsonFileSource; + source: T; origin: FIELD_ORIGIN; dataType: string; }) { diff --git a/x-pack/plugins/maps/public/classes/joins/inner_join.test.js b/x-pack/plugins/maps/public/classes/joins/inner_join.test.js index ca40ab1ea7db7f..bca5954e73d7b5 100644 --- a/x-pack/plugins/maps/public/classes/joins/inner_join.test.js +++ b/x-pack/plugins/maps/public/classes/joins/inner_join.test.js @@ -5,11 +5,13 @@ */ import { InnerJoin } from './inner_join'; +import { SOURCE_TYPES } from '../../../common/constants'; jest.mock('../../kibana_services', () => {}); jest.mock('../layers/vector_layer/vector_layer', () => {}); const rightSource = { + type: SOURCE_TYPES.ES_TERM_SOURCE, id: 'd3625663-5b34-4d50-a784-0d743f676a0c', indexPatternId: '90943e30-9a47-11e8-b64d-95841ca0b247', indexPatternTitle: 'kibana_sample_data_logs', diff --git a/x-pack/plugins/maps/public/classes/joins/inner_join.ts b/x-pack/plugins/maps/public/classes/joins/inner_join.ts index 32bd767aa94d8f..95e163709dff94 100644 --- a/x-pack/plugins/maps/public/classes/joins/inner_join.ts +++ b/x-pack/plugins/maps/public/classes/joins/inner_join.ts @@ -9,29 +9,51 @@ import { Feature, GeoJsonProperties } from 'geojson'; import { ESTermSource } from '../sources/es_term_source'; import { getComputedFieldNamePrefix } from '../styles/vector/style_util'; import { - META_DATA_REQUEST_ID_SUFFIX, FORMATTERS_DATA_REQUEST_ID_SUFFIX, + META_DATA_REQUEST_ID_SUFFIX, + SOURCE_TYPES, } from '../../../common/constants'; -import { JoinDescriptor } from '../../../common/descriptor_types'; +import { + ESTermSourceDescriptor, + JoinDescriptor, + TableSourceDescriptor, + TermJoinSourceDescriptor, +} from '../../../common/descriptor_types'; import { IVectorSource } from '../sources/vector_source'; import { IField } from '../fields/field'; import { PropertiesMap } from '../../../common/elasticsearch_util'; +import { ITermJoinSource } from '../sources/term_join_source'; +import { TableSource } from '../sources/table_source'; +import { Adapters } from '../../../../../../src/plugins/inspector/common/adapters'; + +function createJoinTermSource( + descriptor: Partial | undefined, + inspectorAdapters: Adapters | undefined +): ITermJoinSource | undefined { + if (!descriptor) { + return; + } + + if ( + descriptor.type === SOURCE_TYPES.ES_TERM_SOURCE && + 'indexPatternId' in descriptor && + 'term' in descriptor + ) { + return new ESTermSource(descriptor as ESTermSourceDescriptor, inspectorAdapters); + } else if (descriptor.type === SOURCE_TYPES.TABLE_SOURCE) { + return new TableSource(descriptor as TableSourceDescriptor, inspectorAdapters); + } +} export class InnerJoin { private readonly _descriptor: JoinDescriptor; - private readonly _rightSource?: ESTermSource; + private readonly _rightSource?: ITermJoinSource; private readonly _leftField?: IField; constructor(joinDescriptor: JoinDescriptor, leftSource: IVectorSource) { this._descriptor = joinDescriptor; const inspectorAdapters = leftSource.getInspectorAdapters(); - if ( - joinDescriptor.right && - 'indexPatternId' in joinDescriptor.right && - 'term' in joinDescriptor.right - ) { - this._rightSource = new ESTermSource(joinDescriptor.right, inspectorAdapters); - } + this._rightSource = createJoinTermSource(this._descriptor.right, inspectorAdapters); this._leftField = joinDescriptor.leftField ? leftSource.createField({ fieldName: joinDescriptor.leftField }) : undefined; @@ -47,8 +69,8 @@ export class InnerJoin { return this._leftField && this._rightSource ? this._rightSource.hasCompleteConfig() : false; } - getJoinFields() { - return this._rightSource ? this._rightSource.getMetricFields() : []; + getJoinFields(): IField[] { + return this._rightSource ? this._rightSource.getRightFields() : []; } // Source request id must be static and unique because the re-fetch logic uses the id to locate the previous request. @@ -77,7 +99,7 @@ export class InnerJoin { if (!feature.properties || !this._leftField || !this._rightSource) { return false; } - const rightMetricFields = this._rightSource.getMetricFields(); + const rightMetricFields: IField[] = this._rightSource.getRightFields(); // delete feature properties added by previous join for (let j = 0; j < rightMetricFields.length; j++) { const metricPropertyKey = rightMetricFields[j].getName(); @@ -106,7 +128,7 @@ export class InnerJoin { } } - getRightJoinSource(): ESTermSource { + getRightJoinSource(): ITermJoinSource { if (!this._rightSource) { throw new Error('Cannot get rightSource from InnerJoin with incomplete config'); } diff --git a/x-pack/plugins/maps/public/classes/layers/layer.test.ts b/x-pack/plugins/maps/public/classes/layers/layer.test.ts index e669ddf13e5acd..d8e6a4906a63ae 100644 --- a/x-pack/plugins/maps/public/classes/layers/layer.test.ts +++ b/x-pack/plugins/maps/public/classes/layers/layer.test.ts @@ -7,7 +7,13 @@ import { AbstractLayer } from './layer'; import { ISource } from '../sources/source'; -import { AGG_TYPE, FIELD_ORIGIN, LAYER_STYLE_TYPE, VECTOR_STYLES } from '../../../common/constants'; +import { + AGG_TYPE, + FIELD_ORIGIN, + LAYER_STYLE_TYPE, + SOURCE_TYPES, + VECTOR_STYLES, +} from '../../../common/constants'; import { ESTermSourceDescriptor, VectorStyleDescriptor } from '../../../common/descriptor_types'; import { getDefaultDynamicProperties } from '../styles/vector/vector_style_defaults'; @@ -73,7 +79,7 @@ describe('cloneDescriptor', () => { indexPatternTitle: 'logs-*', metrics: [{ type: AGG_TYPE.COUNT }], term: 'myTermField', - type: 'joinSource', + type: SOURCE_TYPES.ES_TERM_SOURCE, applyGlobalQuery: true, applyGlobalTime: true, }, diff --git a/x-pack/plugins/maps/public/classes/layers/layer.tsx b/x-pack/plugins/maps/public/classes/layers/layer.tsx index fe13e4f0ac2f6c..1596c392e8d63f 100644 --- a/x-pack/plugins/maps/public/classes/layers/layer.tsx +++ b/x-pack/plugins/maps/public/classes/layers/layer.tsx @@ -20,11 +20,13 @@ import { MB_SOURCE_ID_LAYER_ID_PREFIX_DELIMITER, MIN_ZOOM, SOURCE_DATA_REQUEST_ID, + SOURCE_TYPES, STYLE_TYPE, } from '../../../common/constants'; import { copyPersistentState } from '../../reducers/util'; import { AggDescriptor, + ESTermSourceDescriptor, JoinDescriptor, LayerDescriptor, MapExtent, @@ -158,6 +160,14 @@ export class AbstractLayer implements ILayer { if (clonedDescriptor.joins) { clonedDescriptor.joins.forEach((joinDescriptor: JoinDescriptor) => { + if (joinDescriptor.right && joinDescriptor.right.type === SOURCE_TYPES.TABLE_SOURCE) { + throw new Error( + 'Cannot clone table-source. Should only be used in MapEmbeddable, not in UX' + ); + } + const termSourceDescriptor: ESTermSourceDescriptor = joinDescriptor.right as ESTermSourceDescriptor; + + // todo: must tie this to generic thing const originalJoinId = joinDescriptor.right.id!; // right.id is uuid used to track requests in inspector @@ -166,8 +176,8 @@ export class AbstractLayer implements ILayer { // Update all data driven styling properties using join fields if (clonedDescriptor.style && 'properties' in clonedDescriptor.style) { const metrics = - joinDescriptor.right.metrics && joinDescriptor.right.metrics.length - ? joinDescriptor.right.metrics + termSourceDescriptor.metrics && termSourceDescriptor.metrics.length + ? termSourceDescriptor.metrics : [{ type: AGG_TYPE.COUNT }]; metrics.forEach((metricsDescriptor: AggDescriptor) => { const originalJoinKey = getJoinAggKey({ diff --git a/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx b/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx index 2304bb277da49d..e3a80a4c9eb5d3 100644 --- a/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx +++ b/x-pack/plugins/maps/public/classes/layers/vector_layer/vector_layer.tsx @@ -63,6 +63,7 @@ import { ITooltipProperty } from '../../tooltips/tooltip_property'; import { IDynamicStyleProperty } from '../../styles/vector/properties/dynamic_style_property'; import { IESSource } from '../../sources/es_source'; import { PropertiesMap } from '../../../../common/elasticsearch_util'; +import { ITermJoinSource } from '../../sources/term_join_source'; interface SourceResult { refreshed: boolean; @@ -574,7 +575,7 @@ export class VectorLayer extends AbstractLayer { dynamicStyleProps: this.getCurrentStyle() .getDynamicPropertiesArray() .filter((dynamicStyleProp) => { - const matchingField = joinSource.getMetricFieldForName(dynamicStyleProp.getFieldName()); + const matchingField = joinSource.getFieldByName(dynamicStyleProp.getFieldName()); return ( dynamicStyleProp.getFieldOrigin() === FIELD_ORIGIN.JOIN && !!matchingField && @@ -599,7 +600,7 @@ export class VectorLayer extends AbstractLayer { }: { dataRequestId: string; dynamicStyleProps: Array>; - source: IVectorSource; + source: IVectorSource | ITermJoinSource; sourceQuery?: MapQuery; style: IVectorStyle; } & DataRequestContext) { @@ -679,7 +680,7 @@ export class VectorLayer extends AbstractLayer { fields: style .getDynamicPropertiesArray() .filter((dynamicStyleProp) => { - const matchingField = joinSource.getMetricFieldForName(dynamicStyleProp.getFieldName()); + const matchingField = joinSource.getFieldByName(dynamicStyleProp.getFieldName()); return dynamicStyleProp.getFieldOrigin() === FIELD_ORIGIN.JOIN && !!matchingField; }) .map((dynamicStyleProp) => { @@ -699,7 +700,7 @@ export class VectorLayer extends AbstractLayer { }: { dataRequestId: string; fields: IField[]; - source: IVectorSource; + source: IVectorSource | ITermJoinSource; } & DataRequestContext) { if (fields.length === 0) { return; diff --git a/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts b/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts index 77177dd47a166f..5cb299ac33ff8b 100644 --- a/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts +++ b/x-pack/plugins/maps/public/classes/sources/es_agg_source/es_agg_source.ts @@ -23,7 +23,6 @@ export interface IESAggSource extends IESSource { getAggKey(aggType: AGG_TYPE, fieldName: string): string; getAggLabel(aggType: AGG_TYPE, fieldLabel: string): string; getMetricFields(): IESAggField[]; - hasMatchingMetricField(fieldName: string): boolean; getMetricFieldForName(fieldName: string): IESAggField | null; getValueAggsDsl(indexPattern: IndexPattern): { [key: string]: unknown }; } @@ -74,11 +73,6 @@ export abstract class AbstractESAggSource extends AbstractESSource implements IE throw new Error('Cannot create a new field from just a fieldname for an es_agg_source.'); } - hasMatchingMetricField(fieldName: string): boolean { - const matchingField = this.getMetricFieldForName(fieldName); - return !!matchingField; - } - getMetricFieldForName(fieldName: string): IESAggField | null { const targetMetricField = this.getMetricFields().find((metricField: IESAggField) => { return metricField.getName() === fieldName; diff --git a/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts b/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts index 235e8e3a651ee1..c7107964568c92 100644 --- a/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts +++ b/x-pack/plugins/maps/public/classes/sources/es_term_source/es_term_source.ts @@ -30,6 +30,8 @@ import { import { Adapters } from '../../../../../../../src/plugins/inspector/common/adapters'; import { PropertiesMap } from '../../../../common/elasticsearch_util'; import { isValidStringConfig } from '../../util/valid_string_config'; +import { ITermJoinSource } from '../term_join_source/term_join_source'; +import { IField } from '../../fields/field'; const TERMS_AGG_NAME = 'join'; const TERMS_BUCKET_KEYS_TO_IGNORE = ['key', 'doc_count']; @@ -47,7 +49,7 @@ export function extractPropertiesMap(rawEsData: any, countPropertyName: string): return propertiesMap; } -export class ESTermSource extends AbstractESAggSource { +export class ESTermSource extends AbstractESAggSource implements ITermJoinSource { static type = SOURCE_TYPES.ES_TERM_SOURCE; static createDescriptor(descriptor: Partial): ESTermSourceDescriptor { @@ -79,7 +81,7 @@ export class ESTermSource extends AbstractESAggSource { }); } - hasCompleteConfig() { + hasCompleteConfig(): boolean { return _.has(this._descriptor, 'indexPatternId') && _.has(this._descriptor, 'term'); } @@ -174,4 +176,8 @@ export class ESTermSource extends AbstractESAggSource { } : null; } + + getRightFields(): IField[] { + return this.getMetricFields(); + } } diff --git a/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts b/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts index 69d84dc65d382e..35464b24185d03 100644 --- a/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts +++ b/x-pack/plugins/maps/public/classes/sources/geojson_file_source/geojson_file_source.ts @@ -8,15 +8,15 @@ import { Feature, FeatureCollection } from 'geojson'; import { AbstractVectorSource, BoundsFilters, GeoJsonWithMeta } from '../vector_source'; import { EMPTY_FEATURE_COLLECTION, FIELD_ORIGIN, SOURCE_TYPES } from '../../../../common/constants'; import { - GeoJsonFileFieldDescriptor, + InlineFieldDescriptor, GeojsonFileSourceDescriptor, MapExtent, } from '../../../../common/descriptor_types'; import { registerSource } from '../source_registry'; import { IField } from '../../fields/field'; import { getFeatureCollectionBounds } from '../../util/get_feature_collection_bounds'; -import { GeoJsonFileField } from '../../fields/geojson_file_field'; import { Adapters } from '../../../../../../../src/plugins/inspector/common/adapters'; +import { InlineField } from '../../fields/inline_field'; function getFeatureCollection( geoJson: Feature | FeatureCollection | null | undefined @@ -56,14 +56,14 @@ export class GeoJsonFileSource extends AbstractVectorSource { super(normalizedDescriptor, inspectorAdapters); } - _getFields(): GeoJsonFileFieldDescriptor[] { + _getFields(): InlineFieldDescriptor[] { const fields = (this._descriptor as GeojsonFileSourceDescriptor).__fields; return fields ? fields : []; } createField({ fieldName }: { fieldName: string }): IField { const fields = this._getFields(); - const descriptor: GeoJsonFileFieldDescriptor | undefined = fields.find((field) => { + const descriptor: InlineFieldDescriptor | undefined = fields.find((field) => { return field.name === fieldName; }); @@ -74,7 +74,7 @@ export class GeoJsonFileSource extends AbstractVectorSource { )} ` ); } - return new GeoJsonFileField({ + return new InlineField({ fieldName: descriptor.name, source: this, origin: FIELD_ORIGIN.SOURCE, @@ -84,8 +84,8 @@ export class GeoJsonFileSource extends AbstractVectorSource { async getFields(): Promise { const fields = this._getFields(); - return fields.map((field: GeoJsonFileFieldDescriptor) => { - return new GeoJsonFileField({ + return fields.map((field: InlineFieldDescriptor) => { + return new InlineField({ fieldName: field.name, source: this, origin: FIELD_ORIGIN.SOURCE, diff --git a/x-pack/plugins/maps/public/classes/sources/table_source/index.ts b/x-pack/plugins/maps/public/classes/sources/table_source/index.ts new file mode 100644 index 00000000000000..7258e6b464cd02 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/table_source/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { TableSource } from './table_source'; diff --git a/x-pack/plugins/maps/public/classes/sources/table_source/table_source.test.ts b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.test.ts new file mode 100644 index 00000000000000..9409eefa4ae073 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.test.ts @@ -0,0 +1,200 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { TableSource } from './table_source'; +import { FIELD_ORIGIN } from '../../../../common/constants'; +import { + MapFilters, + MapQuery, + VectorJoinSourceRequestMeta, + VectorSourceSyncMeta, +} from '../../../../common/descriptor_types'; + +describe('TableSource', () => { + describe('getName', () => { + it('should get default display name', async () => { + const tableSource = new TableSource({}); + expect((await tableSource.getDisplayName()).startsWith('table source')).toBe(true); + }); + }); + + describe('getPropertiesMap', () => { + it('should roll up results', async () => { + const tableSource = new TableSource({ + term: 'iso', + __rows: [ + { + iso: 'US', + population: 100, + }, + { + iso: 'CN', + population: 400, + foo: 'bar', // ignore this prop, not defined in `__columns` + }, + { + // ignore this row, cannot be joined + population: 400, + }, + { + // row ignored since it's not first row with key 'US' + iso: 'US', + population: -1, + }, + ], + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const propertiesMap = await tableSource.getPropertiesMap( + ({} as unknown) as VectorJoinSourceRequestMeta, + 'a', + 'b', + () => {} + ); + + expect(propertiesMap.size).toEqual(2); + expect(propertiesMap.get('US')).toEqual({ + population: 100, + }); + expect(propertiesMap.get('CN')).toEqual({ + population: 400, + }); + }); + }); + + describe('getTermField', () => { + it('should throw when no match', async () => { + const tableSource = new TableSource({ + term: 'foobar', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + expect(() => { + tableSource.getTermField(); + }).toThrow(); + }); + + it('should return field', async () => { + const tableSource = new TableSource({ + term: 'iso', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const termField = tableSource.getTermField(); + expect(termField.getName()).toEqual('iso'); + expect(await termField.getDataType()).toEqual('string'); + }); + }); + + describe('getRightFields', () => { + it('should return columns', async () => { + const tableSource = new TableSource({ + term: 'foobar', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const rightFields = tableSource.getRightFields(); + expect(rightFields[0].getName()).toEqual('iso'); + expect(await rightFields[0].getDataType()).toEqual('string'); + expect(rightFields[0].getOrigin()).toEqual(FIELD_ORIGIN.JOIN); + expect(rightFields[0].getSource()).toEqual(tableSource); + + expect(rightFields[1].getName()).toEqual('population'); + expect(await rightFields[1].getDataType()).toEqual('number'); + expect(rightFields[1].getOrigin()).toEqual(FIELD_ORIGIN.JOIN); + expect(rightFields[1].getSource()).toEqual(tableSource); + }); + }); + + describe('getFieldByName', () => { + it('should return columns', async () => { + const tableSource = new TableSource({ + term: 'foobar', + __columns: [ + { + name: 'iso', + type: 'string', + }, + { + name: 'population', + type: 'number', + }, + ], + }); + + const field = tableSource.getFieldByName('iso'); + expect(field!.getName()).toEqual('iso'); + expect(await field!.getDataType()).toEqual('string'); + expect(field!.getOrigin()).toEqual(FIELD_ORIGIN.JOIN); + expect(field!.getSource()).toEqual(tableSource); + }); + }); + + describe('getGeoJsonWithMeta', () => { + it('should throw - not implemented', async () => { + const tableSource = new TableSource({}); + + let didThrow = false; + try { + await tableSource.getGeoJsonWithMeta( + 'foobar', + ({} as unknown) as MapFilters & { + applyGlobalQuery: boolean; + applyGlobalTime: boolean; + fieldNames: string[]; + geogridPrecision?: number; + sourceQuery?: MapQuery; + sourceMeta: VectorSourceSyncMeta; + }, + () => {}, + () => { + return false; + } + ); + } catch (e) { + didThrow = true; + } finally { + expect(didThrow).toBe(true); + } + }); + }); +}); diff --git a/x-pack/plugins/maps/public/classes/sources/table_source/table_source.ts b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.ts new file mode 100644 index 00000000000000..d157c4f5df60af --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/table_source/table_source.ts @@ -0,0 +1,218 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import uuid from 'uuid'; +import { FIELD_ORIGIN, SOURCE_TYPES, VECTOR_SHAPE_TYPE } from '../../../../common/constants'; +import { + MapExtent, + MapFilters, + MapQuery, + TableSourceDescriptor, + VectorJoinSourceRequestMeta, + VectorSourceSyncMeta, +} from '../../../../common/descriptor_types'; +import { Adapters } from '../../../../../../../src/plugins/inspector/common/adapters'; +import { ITermJoinSource } from '../term_join_source'; +import { BucketProperties, PropertiesMap } from '../../../../common/elasticsearch_util'; +import { IField } from '../../fields/field'; +import { Query } from '../../../../../../../src/plugins/data/common/query'; +import { + AbstractVectorSource, + BoundsFilters, + GeoJsonWithMeta, + IVectorSource, + SourceTooltipConfig, +} from '../vector_source'; +import { DataRequest } from '../../util/data_request'; +import { InlineField } from '../../fields/inline_field'; + +export class TableSource extends AbstractVectorSource implements ITermJoinSource, IVectorSource { + static type = SOURCE_TYPES.TABLE_SOURCE; + + static createDescriptor(descriptor: Partial): TableSourceDescriptor { + return { + type: SOURCE_TYPES.TABLE_SOURCE, + __rows: descriptor.__rows || [], + __columns: descriptor.__columns || [], + term: descriptor.term || '', + id: descriptor.id || uuid(), + }; + } + + readonly _descriptor: TableSourceDescriptor; + + constructor(descriptor: Partial, inspectorAdapters?: Adapters) { + const sourceDescriptor = TableSource.createDescriptor(descriptor); + super(sourceDescriptor, inspectorAdapters); + this._descriptor = sourceDescriptor; + } + + async getDisplayName(): Promise { + // no need to localize. this is never rendered. + return `table source ${uuid()}`; + } + + getSyncMeta(): VectorSourceSyncMeta | null { + return null; + } + + async getPropertiesMap( + searchFilters: VectorJoinSourceRequestMeta, + leftSourceName: string, + leftFieldName: string, + registerCancelCallback: (callback: () => void) => void + ): Promise { + const propertiesMap: PropertiesMap = new Map(); + + const fieldNames = await this.getFieldNames(); + + for (let i = 0; i < this._descriptor.__rows.length; i++) { + const row: { [key: string]: string | number } = this._descriptor.__rows[i]; + let propKey: string | number | undefined; + const props: { [key: string]: string | number } = {}; + for (const key in row) { + if (row.hasOwnProperty(key)) { + if (key === this._descriptor.term && row[key]) { + propKey = row[key]; + } + if (fieldNames.indexOf(key) >= 0 && key !== this._descriptor.term) { + props[key] = row[key]; + } + } + } + if (propKey && !propertiesMap.has(propKey.toString())) { + // If propKey is not a primary key in the table, this will favor the first match + propertiesMap.set(propKey.toString(), props); + } + } + + return propertiesMap; + } + + getTermField(): IField { + const column = this._descriptor.__columns.find((c) => { + return c.name === this._descriptor.term; + }); + + if (!column) { + throw new Error( + `Cannot find column for ${this._descriptor.term} in ${JSON.stringify( + this._descriptor.__columns + )}` + ); + } + + return new InlineField({ + fieldName: column.name, + source: this, + origin: FIELD_ORIGIN.JOIN, + dataType: column.type, + }); + } + + getWhereQuery(): Query | undefined { + return undefined; + } + + hasCompleteConfig(): boolean { + return true; + } + + getId(): string { + return this._descriptor.id; + } + + getRightFields(): IField[] { + return this._descriptor.__columns.map((column) => { + return new InlineField({ + fieldName: column.name, + source: this, + origin: FIELD_ORIGIN.JOIN, + dataType: column.type, + }); + }); + } + + getFieldNames(): string[] { + return this._descriptor.__columns.map((column) => { + return column.name; + }); + } + + canFormatFeatureProperties(): boolean { + return false; + } + + createField({ fieldName }: { fieldName: string }): IField { + const field = this.getFieldByName(fieldName); + if (!field) { + throw new Error(`Cannot find field for ${fieldName}`); + } + return field; + } + + async getBoundsForFilters( + boundsFilters: BoundsFilters, + registerCancelCallback: (callback: () => void) => void + ): Promise { + return null; + } + + getFieldByName(fieldName: string): IField | null { + const column = this._descriptor.__columns.find((c) => { + return c.name === fieldName; + }); + + if (!column) { + return null; + } + + return new InlineField({ + fieldName: column.name, + source: this, + origin: FIELD_ORIGIN.JOIN, + dataType: column.type, + }); + } + + getFields(): Promise { + throw new Error('must implement'); + } + + // The below is the IVectorSource interface. + // Could be useful to implement, e.g. to preview raw csv data + async getGeoJsonWithMeta( + layerName: string, + searchFilters: MapFilters & { + applyGlobalQuery: boolean; + applyGlobalTime: boolean; + fieldNames: string[]; + geogridPrecision?: number; + sourceQuery?: MapQuery; + sourceMeta: VectorSourceSyncMeta; + }, + registerCancelCallback: (callback: () => void) => void, + isRequestStillActive: () => boolean + ): Promise { + throw new Error('TableSource cannot return GeoJson'); + } + + async getLeftJoinFields(): Promise { + throw new Error('TableSource cannot be used as a left-layer in a term join'); + } + + getSourceTooltipContent(sourceDataRequest?: DataRequest): SourceTooltipConfig { + throw new Error('must add tooltip content'); + } + + async getSupportedShapeTypes(): Promise { + return []; + } + + isBoundsAware(): boolean { + return false; + } +} diff --git a/x-pack/plugins/maps/public/classes/sources/term_join_source/index.ts b/x-pack/plugins/maps/public/classes/sources/term_join_source/index.ts new file mode 100644 index 00000000000000..1879d64d3b2077 --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/term_join_source/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { ITermJoinSource } from './term_join_source'; diff --git a/x-pack/plugins/maps/public/classes/sources/term_join_source/term_join_source.ts b/x-pack/plugins/maps/public/classes/sources/term_join_source/term_join_source.ts new file mode 100644 index 00000000000000..534ac9f200362e --- /dev/null +++ b/x-pack/plugins/maps/public/classes/sources/term_join_source/term_join_source.ts @@ -0,0 +1,33 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { GeoJsonProperties } from 'geojson'; +import { IField } from '../../fields/field'; +import { Query } from '../../../../../../../src/plugins/data/common/query'; +import { + VectorJoinSourceRequestMeta, + VectorSourceSyncMeta, +} from '../../../../common/descriptor_types'; +import { PropertiesMap } from '../../../../common/elasticsearch_util'; +import { ITooltipProperty } from '../../tooltips/tooltip_property'; +import { ISource } from '../source'; + +export interface ITermJoinSource extends ISource { + hasCompleteConfig(): boolean; + getTermField(): IField; + getWhereQuery(): Query | undefined; + getPropertiesMap( + searchFilters: VectorJoinSourceRequestMeta, + leftSourceName: string, + leftFieldName: string, + registerCancelCallback: (callback: () => void) => void + ): Promise; + getSyncMeta(): VectorSourceSyncMeta | null; + getId(): string; + getRightFields(): IField[]; + getTooltipProperties(properties: GeoJsonProperties): Promise; + getFieldByName(fieldName: string): IField | null; +} diff --git a/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx b/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx index 882247e375ddcd..96494a346e625b 100644 --- a/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx +++ b/x-pack/plugins/maps/public/classes/styles/vector/properties/dynamic_style_property.tsx @@ -97,7 +97,7 @@ export class DynamicStyleProperty } const join = this._layer.getValidJoins().find((validJoin: InnerJoin) => { - return validJoin.getRightJoinSource().hasMatchingMetricField(fieldName); + return !!validJoin.getRightJoinSource().getFieldByName(fieldName); }); return join ? join.getSourceMetaDataRequestId() : null; } diff --git a/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx b/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx index 9bf4cafd66407d..126f19b7012f85 100644 --- a/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx +++ b/x-pack/plugins/maps/public/classes/styles/vector/vector_style.tsx @@ -620,7 +620,7 @@ export class VectorStyle implements IVectorStyle { dataRequestId = SOURCE_FORMATTERS_DATA_REQUEST_ID; } else { const targetJoin = this._layer.getValidJoins().find((join) => { - return join.getRightJoinSource().hasMatchingMetricField(fieldName); + return !!join.getRightJoinSource().getFieldByName(fieldName); }); if (targetJoin) { dataRequestId = targetJoin.getSourceFormattersDataRequestId(); @@ -841,7 +841,7 @@ export class VectorStyle implements IVectorStyle { this._iconOrientationProperty.syncIconRotationWithMb(symbolLayerId, mbMap); } - _makeField(fieldDescriptor?: StylePropertyField) { + _makeField(fieldDescriptor?: StylePropertyField): IField | null { if (!fieldDescriptor || !fieldDescriptor.name) { return null; } @@ -852,10 +852,10 @@ export class VectorStyle implements IVectorStyle { return this._source.getFieldByName(fieldDescriptor.name); } else if (fieldDescriptor.origin === FIELD_ORIGIN.JOIN) { const targetJoin = this._layer.getValidJoins().find((join) => { - return join.getRightJoinSource().hasMatchingMetricField(fieldDescriptor.name); + return !!join.getRightJoinSource().getFieldByName(fieldDescriptor.name); }); return targetJoin - ? targetJoin.getRightJoinSource().getMetricFieldForName(fieldDescriptor.name) + ? targetJoin.getRightJoinSource().getFieldByName(fieldDescriptor.name) : null; } else { throw new Error(`Unknown origin-type ${fieldDescriptor.origin}`); diff --git a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx index d47f130d4ede39..ce5c0ed5fdcade 100644 --- a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx +++ b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/join_editor.tsx @@ -5,7 +5,6 @@ */ import React, { Fragment } from 'react'; -import _ from 'lodash'; import uuid from 'uuid/v4'; import { @@ -24,6 +23,7 @@ import { Join } from './resources/join'; import { ILayer } from '../../../classes/layers/layer'; import { JoinDescriptor } from '../../../../common/descriptor_types'; import { IField } from '../../../classes/fields/field'; +import { SOURCE_TYPES } from '../../../../common/constants'; export interface Props { joins: JoinDescriptor[]; @@ -44,19 +44,25 @@ export function JoinEditor({ joins, layer, onChange, leftJoinFields, layerDispla onChange(layer, [...joins.slice(0, index), ...joins.slice(index + 1)]); }; - return ( - - - - - ); + if (joinDescriptor.right.type === SOURCE_TYPES.TABLE_SOURCE) { + throw new Error( + 'PEBKAC - Table sources cannot be edited in the UX and should only be used in MapEmbeddable' + ); + } else { + return ( + + + + + ); + } }); }; diff --git a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js index 507b32fa39fd87..a46b27b62a19e0 100644 --- a/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js +++ b/x-pack/plugins/maps/public/connected_components/layer_panel/join_editor/resources/join.js @@ -17,6 +17,7 @@ import { GlobalTimeCheckbox } from '../../../../components/global_time_checkbox' import { indexPatterns } from '../../../../../../../../src/plugins/data/public'; import { getIndexPatternService } from '../../../../kibana_services'; +import { SOURCE_TYPES } from '../../../../../common/constants'; export class Join extends Component { state = { @@ -85,6 +86,7 @@ export class Join extends Component { ...restOfRight, indexPatternId, indexPatternTitle, + type: SOURCE_TYPES.ES_TERM_SOURCE, }, }); }; diff --git a/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js b/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js index a3dbf8b1438fa9..d1aa044676e004 100644 --- a/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js +++ b/x-pack/plugins/maps/server/sample_data/ecommerce_saved_objects.js @@ -70,6 +70,7 @@ const layerList = [ { leftField: 'iso2', right: { + type: 'ES_TERM_SOURCE', id: '741db9c6-8ebb-4ea9-9885-b6b4ac019d14', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.country_iso_code', @@ -134,6 +135,7 @@ const layerList = [ { leftField: 'name', right: { + type: 'ES_TERM_SOURCE', id: '30a0ec24-49b6-476a-b4ed-6c1636333695', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.region_name', @@ -198,6 +200,7 @@ const layerList = [ { leftField: 'label_en', right: { + type: 'ES_TERM_SOURCE', id: 'e325c9da-73fa-4b3b-8b59-364b99370826', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.region_name', @@ -262,6 +265,7 @@ const layerList = [ { leftField: 'label_en', right: { + type: 'ES_TERM_SOURCE', id: '612d805d-8533-43a9-ac0e-cbf51fe63dcd', indexPatternTitle: 'kibana_sample_data_ecommerce', term: 'geoip.region_name', diff --git a/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js b/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js index ec445567de21c6..010f06e00ca3f3 100644 --- a/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js +++ b/x-pack/plugins/maps/server/sample_data/web_logs_saved_objects.js @@ -70,6 +70,7 @@ const layerList = [ { leftField: 'iso2', right: { + type: 'ES_TERM_SOURCE', id: '673ff994-fc75-4c67-909b-69fcb0e1060e', indexPatternTitle: 'kibana_sample_data_logs', term: 'geo.src', diff --git a/x-pack/plugins/maps/server/saved_objects/migrations.js b/x-pack/plugins/maps/server/saved_objects/migrations.js index 653f07772ee586..346bc5eff16575 100644 --- a/x-pack/plugins/maps/server/saved_objects/migrations.js +++ b/x-pack/plugins/maps/server/saved_objects/migrations.js @@ -14,6 +14,7 @@ import { migrateUseTopHitsToScalingType } from '../../common/migrations/scaling_ import { migrateJoinAggKey } from '../../common/migrations/join_agg_key'; import { removeBoundsFromSavedObject } from '../../common/migrations/remove_bounds'; import { setDefaultAutoFitToBounds } from '../../common/migrations/set_default_auto_fit_to_bounds'; +import { addTypeToTermJoin } from '../../common/migrations/add_type_to_termjoin'; export const migrations = { map: { @@ -79,6 +80,14 @@ export const migrations = { '7.10.0': (doc) => { const attributes = setDefaultAutoFitToBounds(doc); + return { + ...doc, + attributes, + }; + }, + '7.12.0': (doc) => { + const attributes = addTypeToTermJoin(doc); + return { ...doc, attributes, diff --git a/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts b/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts index 42dee46e71fd6c..dba7d006da282c 100644 --- a/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts +++ b/x-pack/plugins/ml/common/types/anomaly_detection_jobs/combined_job.ts @@ -4,7 +4,6 @@ * you may not use this file except in compliance with the Elastic License. */ -import { cloneDeep } from 'lodash'; import { Datafeed } from './datafeed'; import { DatafeedStats } from './datafeed_stats'; import { Job } from './job'; @@ -25,16 +24,6 @@ export interface CombinedJobWithStats extends JobWithStats { datafeed_config: DatafeedWithStats; } -export function expandCombinedJobConfig(combinedJob: CombinedJob) { - const combinedJobClone = cloneDeep(combinedJob); - const job = combinedJobClone; - const datafeed = combinedJobClone.datafeed_config; - // @ts-expect-error - delete job.datafeed_config; - - return { job, datafeed }; -} - export function isCombinedJobWithStats(arg: any): arg is CombinedJobWithStats { return typeof arg.job_id === 'string'; } diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx index e72af6a0e30c24..2cd7223027566a 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_creation/page.tsx @@ -62,7 +62,7 @@ export const Page: FC = ({ jobId }) => { if (currentIndexPattern) { (async function () { if (jobId !== undefined) { - const analyticsConfigs = await ml.dataFrameAnalytics.getDataFrameAnalytics(jobId); + const analyticsConfigs = await ml.dataFrameAnalytics.getDataFrameAnalytics(jobId, true); if ( Array.isArray(analyticsConfigs.data_frame_analytics) && analyticsConfigs.data_frame_analytics.length > 0 diff --git a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx index b1592d51874c74..72c6f5a9eca997 100644 --- a/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx +++ b/x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/action_clone/clone_action_name.tsx @@ -310,9 +310,6 @@ export type CloneDataFrameAnalyticsConfig = Omit< */ export function extractCloningConfig({ id, - version, - // eslint-disable-next-line @typescript-eslint/naming-convention - create_time, ...configToClone }: DeepReadonly): CloneDataFrameAnalyticsConfig { return (cloneDeep({ diff --git a/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js b/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js index 338222e3ac4a2f..0e2fee70c748a1 100644 --- a/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js +++ b/x-pack/plugins/ml/public/application/jobs/jobs_list/components/utils.js @@ -36,6 +36,23 @@ export function loadFullJob(jobId) { }); } +export function loadJobForCloning(jobId) { + return new Promise((resolve, reject) => { + ml.jobs + .jobForCloning(jobId) + .then((resp) => { + if (resp) { + resolve(resp); + } else { + throw new Error(`Could not find job ${jobId}`); + } + }) + .catch((error) => { + reject(error); + }); + }); +} + export function isStartable(jobs) { return jobs.some( (j) => j.datafeedState === DATAFEED_STATE.STOPPED && j.jobState !== JOB_STATE.CLOSING @@ -180,31 +197,38 @@ function showResults(resp, action) { export async function cloneJob(jobId) { try { - const job = await loadFullJob(jobId); - if (job.custom_settings && job.custom_settings.created_by) { + const [{ job: cloneableJob, datafeed }, originalJob] = await Promise.all([ + loadJobForCloning(jobId), + loadFullJob(jobId, false), + ]); + if (cloneableJob !== undefined && originalJob?.custom_settings?.created_by !== undefined) { // if the job is from a wizards, i.e. contains a created_by property // use tempJobCloningObjects to temporarily store the job - mlJobService.tempJobCloningObjects.job = job; + mlJobService.tempJobCloningObjects.createdBy = originalJob?.custom_settings?.created_by; + mlJobService.tempJobCloningObjects.job = cloneableJob; if ( - job.data_counts.earliest_record_timestamp !== undefined && - job.data_counts.latest_record_timestamp !== undefined && - job.data_counts.latest_bucket_timestamp !== undefined + originalJob.data_counts.earliest_record_timestamp !== undefined && + originalJob.data_counts.latest_record_timestamp !== undefined && + originalJob.data_counts.latest_bucket_timestamp !== undefined ) { // if the job has run before, use the earliest and latest record timestamp // as the cloned job's time range - let start = job.data_counts.earliest_record_timestamp; - let end = job.data_counts.latest_record_timestamp; + let start = originalJob.data_counts.earliest_record_timestamp; + let end = originalJob.data_counts.latest_record_timestamp; - if (job.datafeed_config.aggregations !== undefined) { + if (originalJob.datafeed_config.aggregations !== undefined) { // if the datafeed uses aggregations the earliest and latest record timestamps may not be the same // as the start and end of the data in the index. - const bucketSpanMs = parseInterval(job.analysis_config.bucket_span).asMilliseconds(); + const bucketSpanMs = parseInterval( + originalJob.analysis_config.bucket_span + ).asMilliseconds(); // round down to the start of the nearest bucket start = - Math.floor(job.data_counts.earliest_record_timestamp / bucketSpanMs) * bucketSpanMs; + Math.floor(originalJob.data_counts.earliest_record_timestamp / bucketSpanMs) * + bucketSpanMs; // use latest_bucket_timestamp and add two bucket spans minus one ms - end = job.data_counts.latest_bucket_timestamp + bucketSpanMs * 2 - 1; + end = originalJob.data_counts.latest_bucket_timestamp + bucketSpanMs * 2 - 1; } mlJobService.tempJobCloningObjects.start = start; @@ -212,12 +236,17 @@ export async function cloneJob(jobId) { } } else { // otherwise use the tempJobCloningObjects - mlJobService.tempJobCloningObjects.job = job; + mlJobService.tempJobCloningObjects.job = cloneableJob; + // resets the createdBy field in case it still retains previous settings + mlJobService.tempJobCloningObjects.createdBy = undefined; + } + if (datafeed !== undefined) { + mlJobService.tempJobCloningObjects.datafeed = datafeed; } - if (job.calendars) { + if (originalJob.calendars) { mlJobService.tempJobCloningObjects.calendars = await mlCalendarService.fetchCalendarsByIds( - job.calendars + originalJob.calendars ); } diff --git a/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts b/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts index cedaaa3b5dfaac..18992e5cbf5d8a 100644 --- a/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts +++ b/x-pack/plugins/ml/public/application/jobs/new_job/pages/index_or_search/preconfigured_job_redirect.ts @@ -8,7 +8,7 @@ import { ApplicationStart } from 'kibana/public'; import { IndexPatternsContract } from '../../../../../../../../../src/plugins/data/public'; import { mlJobService } from '../../../../services/job_service'; import { loadIndexPatterns, getIndexPatternIdFromName } from '../../../../util/index_utils'; -import { CombinedJob } from '../../../../../../common/types/anomaly_detection_jobs'; +import { Datafeed, Job } from '../../../../../../common/types/anomaly_detection_jobs'; import { CREATED_BY_LABEL, JOB_TYPE } from '../../../../../../common/constants/new_job'; export async function preConfiguredJobRedirect( @@ -16,11 +16,11 @@ export async function preConfiguredJobRedirect( basePath: string, navigateToUrl: ApplicationStart['navigateToUrl'] ) { - const { job } = mlJobService.tempJobCloningObjects; - if (job) { + const { createdBy, job, datafeed } = mlJobService.tempJobCloningObjects; + if (job && datafeed) { try { await loadIndexPatterns(indexPatterns); - const redirectUrl = getWizardUrlFromCloningJob(job); + const redirectUrl = getWizardUrlFromCloningJob(createdBy, job, datafeed); await navigateToUrl(`${basePath}/app/ml/${redirectUrl}`); return Promise.reject(); } catch (error) { @@ -33,8 +33,8 @@ export async function preConfiguredJobRedirect( } } -function getWizardUrlFromCloningJob(job: CombinedJob) { - const created = job?.custom_settings?.created_by; +function getWizardUrlFromCloningJob(createdBy: string | undefined, job: Job, datafeed: Datafeed) { + const created = createdBy; let page = ''; switch (created) { @@ -55,7 +55,7 @@ function getWizardUrlFromCloningJob(job: CombinedJob) { break; } - const indexPatternId = getIndexPatternIdFromName(job.datafeed_config.indices.join()); + const indexPatternId = getIndexPatternIdFromName(datafeed.indices.join()); return `jobs/new_job/${page}?index=${indexPatternId}&_g=()`; } diff --git a/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx b/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx index 8f7f93763fdd64..a196934610855a 100644 --- a/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx +++ b/x-pack/plugins/ml/public/application/jobs/new_job/pages/new_job/page.tsx @@ -37,7 +37,6 @@ import { useMlContext } from '../../../../contexts/ml'; import { getTimeFilterRange } from '../../../../components/full_time_range_selector'; import { getTimeBucketsFromCache } from '../../../../util/time_buckets'; import { ExistingJobsAndGroups, mlJobService } from '../../../../services/job_service'; -import { expandCombinedJobConfig } from '../../../../../../common/types/anomaly_detection_jobs'; import { newJobCapsService } from '../../../../services/new_job_capabilities_service'; import { EVENT_RATE_FIELD_ID } from '../../../../../../common/types/fields'; import { getNewJobDefaults } from '../../../../services/ml_server_info'; @@ -74,10 +73,11 @@ export const Page: FC = ({ existingJobsAndGroups, jobType }) => { if (mlJobService.tempJobCloningObjects.job !== undefined) { // cloning a job - const clonedJob = mlJobService.cloneJob(mlJobService.tempJobCloningObjects.job); - const { job, datafeed } = expandCombinedJobConfig(clonedJob); + const clonedJob = mlJobService.tempJobCloningObjects.job; + const clonedDatafeed = mlJobService.cloneDatafeed(mlJobService.tempJobCloningObjects.datafeed); + initCategorizationSettings(); - jobCreator.cloneFromExistingJob(job, datafeed); + jobCreator.cloneFromExistingJob(clonedJob, clonedDatafeed); // if we're not skipping the time range, this is a standard job clone, so wipe the jobId if (mlJobService.tempJobCloningObjects.skipTimeRangeStep === false) { diff --git a/x-pack/plugins/ml/public/application/services/job_service.d.ts b/x-pack/plugins/ml/public/application/services/job_service.d.ts index 30b2ec044285a2..0bf40bc0dad775 100644 --- a/x-pack/plugins/ml/public/application/services/job_service.d.ts +++ b/x-pack/plugins/ml/public/application/services/job_service.d.ts @@ -6,7 +6,7 @@ import { SearchResponse } from 'elasticsearch'; import { TimeRange } from 'src/plugins/data/common/query/timefilter/types'; -import { CombinedJob } from '../../../common/types/anomaly_detection_jobs'; +import { CombinedJob, Datafeed } from '../../../common/types/anomaly_detection_jobs'; import { Calendar } from '../../../common/types/calendars'; export interface ExistingJobsAndGroups { @@ -18,6 +18,8 @@ declare interface JobService { jobs: CombinedJob[]; createResultsUrlForJobs: (jobs: any[], target: string, timeRange?: TimeRange) => string; tempJobCloningObjects: { + createdBy?: string; + datafeed?: Datafeed; job: any; skipTimeRangeStep: boolean; start?: number; @@ -26,7 +28,7 @@ declare interface JobService { }; skipTimeRangeStep: boolean; saveNewJob(job: any): Promise; - cloneJob(job: any): any; + cloneDatafeed(datafeed: any): Datafeed; openJob(jobId: string): Promise; saveNewDatafeed(datafeedConfig: any, jobId: string): Promise; startDatafeed( diff --git a/x-pack/plugins/ml/public/application/services/job_service.js b/x-pack/plugins/ml/public/application/services/job_service.js index 5f504e46655003..06f1aea3e6e1e8 100644 --- a/x-pack/plugins/ml/public/application/services/job_service.js +++ b/x-pack/plugins/ml/public/application/services/job_service.js @@ -28,6 +28,8 @@ class JobService { // if populated when loading the job management page, the start datafeed modal // is automatically opened. this.tempJobCloningObjects = { + createdBy: undefined, + datafeed: undefined, job: undefined, skipTimeRangeStep: false, start: undefined, @@ -325,67 +327,15 @@ class JobService { return ml.addJob({ jobId: job.job_id, job }).then(func).catch(func); } - cloneJob(job) { - // create a deep copy of a job object - // also remove items from the job which are set by the server and not needed - // in the future this formatting could be optional - const tempJob = cloneDeep(job); - - // remove all of the items which should not be copied - // such as counts, state and times - delete tempJob.state; - delete tempJob.job_version; - delete tempJob.data_counts; - delete tempJob.create_time; - delete tempJob.finished_time; - delete tempJob.last_data_time; - delete tempJob.model_size_stats; - delete tempJob.node; - delete tempJob.average_bucket_processing_time_ms; - delete tempJob.model_snapshot_id; - delete tempJob.open_time; - delete tempJob.established_model_memory; - delete tempJob.calendars; - delete tempJob.timing_stats; - delete tempJob.forecasts_stats; - delete tempJob.assignment_explanation; - - delete tempJob.analysis_config.use_per_partition_normalization; - - each(tempJob.analysis_config.detectors, (d) => { - delete d.detector_index; - }); + cloneDatafeed(datafeed) { + const tempDatafeed = cloneDeep(datafeed); // remove parts of the datafeed config which should not be copied - if (tempJob.datafeed_config) { - delete tempJob.datafeed_config.datafeed_id; - delete tempJob.datafeed_config.job_id; - delete tempJob.datafeed_config.state; - delete tempJob.datafeed_config.node; - delete tempJob.datafeed_config.timing_stats; - delete tempJob.datafeed_config.assignment_explanation; - - // remove query_delay if it's between 60s and 120s - // the back-end produces a random value between 60 and 120 and so - // by deleting it, the back-end will produce a new random value - if (tempJob.datafeed_config.query_delay) { - const interval = parseInterval(tempJob.datafeed_config.query_delay); - if (interval !== null) { - const queryDelay = interval.asSeconds(); - if (queryDelay > 60 && queryDelay < 120) { - delete tempJob.datafeed_config.query_delay; - } - } - } + if (tempDatafeed) { + delete tempDatafeed.datafeed_id; + delete tempDatafeed.job_id; } - - // when jumping from a wizard to the advanced job creation, - // the wizard's created_by information should be stripped. - if (tempJob.custom_settings && tempJob.custom_settings.created_by) { - delete tempJob.custom_settings.created_by; - } - - return tempJob; + return tempDatafeed; } // find a job based on the id diff --git a/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts b/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts index 7b246e557d7a57..98a8e4c9cbf2d1 100644 --- a/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts +++ b/x-pack/plugins/ml/public/application/services/ml_api_service/data_frame_analytics.ts @@ -52,11 +52,12 @@ interface JobsExistsResponse { } export const dataFrameAnalytics = { - getDataFrameAnalytics(analyticsId?: string) { + getDataFrameAnalytics(analyticsId?: string, excludeGenerated?: boolean) { const analyticsIdString = analyticsId !== undefined ? `/${analyticsId}` : ''; return http({ path: `${basePath()}/data_frame/analytics${analyticsIdString}`, method: 'GET', + ...(excludeGenerated ? { query: { excludeGenerated } } : {}), }); }, getDataFrameAnalyticsStats(analyticsId?: string) { diff --git a/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts b/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts index 10e035103dbecf..67aaf6b5571680 100644 --- a/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts +++ b/x-pack/plugins/ml/public/application/services/ml_api_service/jobs.ts @@ -13,6 +13,8 @@ import { MlJobWithTimeRange, MlSummaryJobs, CombinedJobWithStats, + Job, + Datafeed, } from '../../../../common/types/anomaly_detection_jobs'; import { JobMessage } from '../../../../common/types/audit_message'; import { AggFieldNamePair } from '../../../../common/types/fields'; @@ -48,6 +50,15 @@ export const jobsApiProvider = (httpService: HttpService) => ({ }); }, + jobForCloning(jobId: string) { + const body = JSON.stringify({ jobId }); + return httpService.http<{ job?: Job; datafeed?: Datafeed } | undefined>({ + path: `${basePath()}/jobs/job_for_cloning`, + method: 'POST', + body, + }); + }, + jobs(jobIds: string[]) { const body = JSON.stringify({ jobIds }); return httpService.http({ diff --git a/x-pack/plugins/ml/server/models/job_service/datafeeds.ts b/x-pack/plugins/ml/server/models/job_service/datafeeds.ts index e144da0ae08004..b0c942647227ce 100644 --- a/x-pack/plugins/ml/server/models/job_service/datafeeds.ts +++ b/x-pack/plugins/ml/server/models/job_service/datafeeds.ts @@ -160,11 +160,55 @@ export function datafeedsProvider(mlClient: MlClient) { }, {} as { [id: string]: string }); } + async function getDatafeedByJobId( + jobId: string, + excludeGenerated?: boolean + ): Promise { + async function findDatafeed() { + // if the job was doesn't use the standard datafeedId format + // get all the datafeeds and match it with the jobId + const { + body: { datafeeds }, + } = await mlClient.getDatafeeds( + excludeGenerated ? { exclude_generated: true } : {} + ); + for (const result of datafeeds) { + if (result.job_id === jobId) { + return result; + } + } + } + // if the job was created by the wizard, + // then we can assume it uses the standard format of the datafeedId + const assumedDefaultDatafeedId = `datafeed-${jobId}`; + try { + const { + body: { datafeeds: datafeedsResults }, + } = await mlClient.getDatafeeds({ + datafeed_id: assumedDefaultDatafeedId, + ...(excludeGenerated ? { exclude_generated: true } : {}), + }); + if ( + Array.isArray(datafeedsResults) && + datafeedsResults.length === 1 && + datafeedsResults[0].job_id === jobId + ) { + return datafeedsResults[0]; + } else { + return await findDatafeed(); + } + } catch (e) { + // if assumedDefaultDatafeedId does not exist, ES will throw an error + return await findDatafeed(); + } + } + return { forceStartDatafeeds, stopDatafeeds, forceDeleteDatafeed, getDatafeedIdsByJobId, getJobIdsByDatafeedId, + getDatafeedByJobId, }; } diff --git a/x-pack/plugins/ml/server/models/job_service/jobs.ts b/x-pack/plugins/ml/server/models/job_service/jobs.ts index d47a1d4b4892d0..6ab4af63004b4f 100644 --- a/x-pack/plugins/ml/server/models/job_service/jobs.ts +++ b/x-pack/plugins/ml/server/models/job_service/jobs.ts @@ -18,6 +18,8 @@ import { AuditMessage, DatafeedWithStats, CombinedJobWithStats, + Datafeed, + Job, } from '../../../common/types/anomaly_detection_jobs'; import { MlJobsResponse, @@ -47,7 +49,9 @@ interface Results { export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { const { asInternalUser } = client; - const { forceDeleteDatafeed, getDatafeedIdsByJobId } = datafeedsProvider(mlClient); + const { forceDeleteDatafeed, getDatafeedIdsByJobId, getDatafeedByJobId } = datafeedsProvider( + mlClient + ); const { getAuditMessagesSummary } = jobAuditMessagesProvider(client, mlClient); const { getLatestBucketTimestampByJob } = resultsServiceProvider(mlClient); const calMngr = new CalendarManager(mlClient); @@ -257,6 +261,25 @@ export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { return { jobs, jobsMap }; } + async function getJobForCloning(jobId: string) { + const [{ body: jobResults }, datafeedResult] = await Promise.all([ + mlClient.getJobs({ job_id: jobId, exclude_generated: true }), + getDatafeedByJobId(jobId, true), + ]); + const result: { datafeed?: Datafeed; job?: Job } = { job: undefined, datafeed: undefined }; + if (datafeedResult && datafeedResult.job_id === jobId) { + result.datafeed = datafeedResult; + } + + if (jobResults && jobResults.jobs) { + const job = jobResults.jobs.find((j) => j.job_id === jobId); + if (job) { + result.job = job; + } + } + return result; + } + async function createFullJobsList(jobIds: string[] = []) { const jobs: CombinedJobWithStats[] = []; const groups: { [jobId: string]: string[] } = {}; @@ -265,6 +288,7 @@ export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { const globalCalendars: string[] = []; const jobIdsString = jobIds.join(); + const [ { body: jobResults }, { body: jobStatsResults }, @@ -502,6 +526,7 @@ export function jobsProvider(client: IScopedClusterClient, mlClient: MlClient) { forceStopAndCloseJob, jobsSummary, jobsWithTimerange, + getJobForCloning, createFullJobsList, deletingJobTasks, jobsExist, diff --git a/x-pack/plugins/ml/server/routes/apidoc.json b/x-pack/plugins/ml/server/routes/apidoc.json index 015ec6e4ec9c0a..5dc9a3107af868 100644 --- a/x-pack/plugins/ml/server/routes/apidoc.json +++ b/x-pack/plugins/ml/server/routes/apidoc.json @@ -73,6 +73,7 @@ "CloseJobs", "JobsSummary", "JobsWithTimeRange", + "GetJobForCloning", "CreateFullJobsList", "GetAllGroups", "JobsExist", diff --git a/x-pack/plugins/ml/server/routes/data_frame_analytics.ts b/x-pack/plugins/ml/server/routes/data_frame_analytics.ts index 0abba7a429aea4..4d504f4f2ef203 100644 --- a/x-pack/plugins/ml/server/routes/data_frame_analytics.ts +++ b/x-pack/plugins/ml/server/routes/data_frame_analytics.ts @@ -19,6 +19,7 @@ import { stopsDataFrameAnalyticsJobQuerySchema, deleteDataFrameAnalyticsJobSchema, jobsExistSchema, + analyticsQuerySchema, } from './schemas/data_analytics_schema'; import { GetAnalyticsMapArgs, ExtendAnalyticsMapArgs } from '../models/data_frame_analytics/types'; import { IndexPatternHandler } from '../models/data_frame_analytics/index_patterns'; @@ -102,7 +103,9 @@ export function dataFrameAnalyticsRoutes({ router, mlLicense, routeGuard }: Rout }, routeGuard.fullLicenseAPIGuard(async ({ mlClient, response }) => { try { - const { body } = await mlClient.getDataFrameAnalytics({ size: 1000 }); + const { body } = await mlClient.getDataFrameAnalytics({ + size: 1000, + }); return response.ok({ body, }); @@ -126,6 +129,7 @@ export function dataFrameAnalyticsRoutes({ router, mlLicense, routeGuard }: Rout path: '/api/ml/data_frame/analytics/{analyticsId}', validate: { params: analyticsIdSchema, + query: analyticsQuerySchema, }, options: { tags: ['access:ml:canGetDataFrameAnalytics'], @@ -134,8 +138,11 @@ export function dataFrameAnalyticsRoutes({ router, mlLicense, routeGuard }: Rout routeGuard.fullLicenseAPIGuard(async ({ mlClient, request, response }) => { try { const { analyticsId } = request.params; + const { excludeGenerated } = request.query; + const { body } = await mlClient.getDataFrameAnalytics({ id: analyticsId, + ...(excludeGenerated ? { exclude_generated: true } : {}), }); return response.ok({ body, diff --git a/x-pack/plugins/ml/server/routes/job_service.ts b/x-pack/plugins/ml/server/routes/job_service.ts index c067d9ce0abbc6..a72e942e987aaa 100644 --- a/x-pack/plugins/ml/server/routes/job_service.ts +++ b/x-pack/plugins/ml/server/routes/job_service.ts @@ -272,6 +272,40 @@ export function jobServiceRoutes({ router, routeGuard }: RouteInitialization) { }) ); + /** + * @apiGroup JobService + * + * @api {post} /api/ml/jobs/job_for_cloning Get job for cloning + * @apiName GetJobForCloning + * @apiDescription Get the job configuration with auto generated fields excluded for cloning + * + * @apiSchema (body) jobIdSchema + */ + router.post( + { + path: '/api/ml/jobs/job_for_cloning', + validate: { + body: jobIdSchema, + }, + options: { + tags: ['access:ml:canGetJobs'], + }, + }, + routeGuard.fullLicenseAPIGuard(async ({ client, mlClient, request, response }) => { + try { + const { getJobForCloning } = jobServiceProvider(client, mlClient); + const { jobId } = request.body; + + const resp = await getJobForCloning(jobId); + return response.ok({ + body: resp, + }); + } catch (e) { + return response.customError(wrapError(e)); + } + }) + ); + /** * @apiGroup JobService * diff --git a/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts b/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts index cf52d1cb27433e..0f965cf500b864 100644 --- a/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts +++ b/x-pack/plugins/ml/server/routes/schemas/data_analytics_schema.ts @@ -64,6 +64,13 @@ export const analyticsIdSchema = schema.object({ analyticsId: schema.string(), }); +export const analyticsQuerySchema = schema.object({ + /** + * Analytics Query + */ + excludeGenerated: schema.maybe(schema.boolean()), +}); + export const deleteDataFrameAnalyticsJobSchema = schema.object({ /** * Analytics Destination Index diff --git a/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts b/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts index 583c9c41727ea9..56094a4950a0ce 100644 --- a/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts +++ b/x-pack/plugins/ml/server/routes/schemas/job_service_schema.ts @@ -39,6 +39,11 @@ export const forceStartDatafeedSchema = schema.object({ end: schema.maybe(schema.number()), }); +export const jobIdSchema = schema.object({ + /** Optional list of job IDs. */ + jobIds: schema.maybe(schema.string()), +}); + export const jobIdsSchema = schema.object({ /** Optional list of job IDs. */ jobIds: schema.maybe(schema.arrayOf(schema.maybe(schema.string()))), diff --git a/x-pack/plugins/ml/server/saved_objects/authorization.ts b/x-pack/plugins/ml/server/saved_objects/authorization.ts index 958ee2091f11e6..9afc479c32d7d3 100644 --- a/x-pack/plugins/ml/server/saved_objects/authorization.ts +++ b/x-pack/plugins/ml/server/saved_objects/authorization.ts @@ -10,6 +10,15 @@ import { ML_SAVED_OBJECT_TYPE } from '../../common/types/saved_objects'; export function authorizationProvider(authorization: SecurityPluginSetup['authz']) { async function authorizationCheck(request: KibanaRequest) { + const shouldAuthorizeRequest = authorization?.mode.useRbacForRequest(request) ?? false; + + if (shouldAuthorizeRequest === false) { + return { + canCreateGlobally: true, + canCreateAtSpace: true, + }; + } + const checkPrivilegesWithRequest = authorization.checkPrivilegesWithRequest(request); // Checking privileges "dynamically" will check against the current space, if spaces are enabled. // If spaces are disabled, then this will check privileges globally instead. diff --git a/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts b/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts index 734caa7374686b..3336e65da2b11e 100644 --- a/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts +++ b/x-pack/plugins/monitoring/server/es_client/instantiate_client.ts @@ -30,7 +30,6 @@ export function instantiateClient( const cluster = createClient('monitoring', { ...(isMonitoringCluster ? elasticsearchConfig : {}), plugins: [monitoringBulk, monitoringEndpointDisableWatches], - logQueries: Boolean(elasticsearchConfig.logQueries), } as ESClusterConfig); const configSource = isMonitoringCluster ? 'monitoring' : 'production'; diff --git a/x-pack/plugins/observability/public/index.ts b/x-pack/plugins/observability/public/index.ts index c052541956c139..49dc298d9a9b04 100644 --- a/x-pack/plugins/observability/public/index.ts +++ b/x-pack/plugins/observability/public/index.ts @@ -33,3 +33,4 @@ export * from './typings'; export { useChartTheme } from './hooks/use_chart_theme'; export { useTheme } from './hooks/use_theme'; +export { getApmTraceUrl } from './utils/get_apm_trace_url'; diff --git a/x-pack/plugins/observability/public/utils/get_apm_trace_url.test.ts b/x-pack/plugins/observability/public/utils/get_apm_trace_url.test.ts new file mode 100644 index 00000000000000..14ede0d3a114e9 --- /dev/null +++ b/x-pack/plugins/observability/public/utils/get_apm_trace_url.test.ts @@ -0,0 +1,14 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import { getApmTraceUrl } from './get_apm_trace_url'; + +describe('getApmTraceUrl', () => { + it('returns a trace url', () => { + expect(getApmTraceUrl({ traceId: 'foo', rangeFrom: '123', rangeTo: '456' })).toEqual( + '/link-to/trace/foo?rangeFrom=123&rangeTo=456' + ); + }); +}); diff --git a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.ts b/x-pack/plugins/observability/public/utils/get_apm_trace_url.ts similarity index 69% rename from x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.ts rename to x-pack/plugins/observability/public/utils/get_apm_trace_url.ts index 819c6eafe80b4d..e60ec10e45a868 100644 --- a/x-pack/plugins/apm/public/components/shared/Links/apm/ExternalLinks.ts +++ b/x-pack/plugins/observability/public/utils/get_apm_trace_url.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -export const getTraceUrl = ({ +export function getApmTraceUrl({ traceId, rangeFrom, rangeTo, @@ -12,9 +12,6 @@ export const getTraceUrl = ({ traceId: string; rangeFrom: string; rangeTo: string; -}) => { - return ( - `/link-to/trace/${traceId}?` + - new URLSearchParams({ rangeFrom, rangeTo }).toString() - ); -}; +}) { + return `/link-to/trace/${traceId}?` + new URLSearchParams({ rangeFrom, rangeTo }).toString(); +} diff --git a/x-pack/plugins/reporting/common/constants.ts b/x-pack/plugins/reporting/common/constants.ts index 16e40bab65a46e..882387184ba9c5 100644 --- a/x-pack/plugins/reporting/common/constants.ts +++ b/x-pack/plugins/reporting/common/constants.ts @@ -56,14 +56,19 @@ export const LAYOUT_TYPES = { }; // Export Type Definitions -export const CSV_REPORT_TYPE = 'CSV'; export const PDF_REPORT_TYPE = 'printablePdf'; -export const PNG_REPORT_TYPE = 'PNG'; - export const PDF_JOB_TYPE = 'printable_pdf'; + +export const PNG_REPORT_TYPE = 'PNG'; export const PNG_JOB_TYPE = 'PNG'; -export const CSV_JOB_TYPE = 'csv'; + export const CSV_FROM_SAVEDOBJECT_JOB_TYPE = 'csv_from_savedobject'; + +// This is deprecated because it lacks support for runtime fields +// but the extension points are still needed for pre-existing scripted automation, until 8.0 +export const CSV_REPORT_TYPE_DEPRECATED = 'CSV'; +export const CSV_JOB_TYPE_DEPRECATED = 'csv'; + export const USES_HEADLESS_JOB_TYPES = [PDF_JOB_TYPE, PNG_JOB_TYPE]; // Licenses diff --git a/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx b/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx index bbdc2e1aebe776..bafb5d7a68630a 100644 --- a/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx +++ b/x-pack/plugins/reporting/public/components/reporting_panel_content.tsx @@ -10,7 +10,11 @@ import React, { Component, ReactElement } from 'react'; import { ToastsSetup } from 'src/core/public'; import url from 'url'; import { toMountPoint } from '../../../../../src/plugins/kibana_react/public'; -import { CSV_REPORT_TYPE, PDF_REPORT_TYPE, PNG_REPORT_TYPE } from '../../common/constants'; +import { + CSV_REPORT_TYPE_DEPRECATED, + PDF_REPORT_TYPE, + PNG_REPORT_TYPE, +} from '../../common/constants'; import { BaseParams } from '../../common/types'; import { ReportingAPIClient } from '../lib/reporting_api_client'; @@ -173,7 +177,7 @@ class ReportingPanelContentUi extends Component { case PDF_REPORT_TYPE: return 'PDF'; case 'csv': - return CSV_REPORT_TYPE; + return CSV_REPORT_TYPE_DEPRECATED; case 'png': return PNG_REPORT_TYPE; default: diff --git a/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx b/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx index 9a4832b114e40d..49c0eaaa2960d4 100644 --- a/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx +++ b/x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx @@ -103,7 +103,6 @@ export class GetCsvReportPanelAction implements ActionDefinition const kibanaTimezone = this.core.uiSettings.get('dateFormat:tz'); const id = `search:${embeddable.getSavedSearch().id}`; - const filename = embeddable.getSavedSearch().title; const timezone = kibanaTimezone === 'Browser' ? moment.tz.guess() : kibanaTimezone; const fromTime = dateMath.parse(from); const toTime = dateMath.parse(to, { roundUp: true }); @@ -140,7 +139,7 @@ export class GetCsvReportPanelAction implements ActionDefinition .then((rawResponse: string) => { this.isDownloading = false; - const download = `${filename}.csv`; + const download = `${embeddable.getSavedSearch().title}.csv`; const blob = new Blob([rawResponse], { type: 'text/csv;charset=utf-8;' }); // Hack for IE11 Support diff --git a/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx b/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx index 7126762c0f4ee9..4659952eef720e 100644 --- a/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx +++ b/x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx @@ -10,7 +10,10 @@ import React from 'react'; import { IUiSettingsClient, ToastsSetup } from 'src/core/public'; import { ShareContext } from '../../../../../src/plugins/share/public'; import { LicensingPluginSetup } from '../../../licensing/public'; -import { JobParamsCSV, SearchRequest } from '../../server/export_types/csv/types'; +import { + JobParamsDeprecatedCSV, + SearchRequestDeprecatedCSV, +} from '../../server/export_types/csv/types'; import { ReportingPanelContent } from '../components/reporting_panel_content_lazy'; import { checkLicense } from '../lib/license_check'; import { ReportingAPIClient } from '../lib/reporting_api_client'; @@ -59,12 +62,12 @@ export const csvReportingProvider = ({ return []; } - const jobParams: JobParamsCSV = { + const jobParams: JobParamsDeprecatedCSV = { browserTimezone, objectType, title: sharingData.title as string, indexPatternId: sharingData.indexPatternId as string, - searchRequest: sharingData.searchRequest as SearchRequest, + searchRequest: sharingData.searchRequest as SearchRequestDeprecatedCSV, fields: sharingData.fields as string[], metaFields: sharingData.metaFields as string[], conflictedTypesFields: sharingData.conflictedTypesFields as string[], diff --git a/x-pack/plugins/reporting/server/export_types/csv/create_job.ts b/x-pack/plugins/reporting/server/export_types/csv/create_job.ts index f0f72a0bc99657..e704f9650b7a8d 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/create_job.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/create_job.ts @@ -4,15 +4,19 @@ * you may not use this file except in compliance with the Elastic License. */ -import { CSV_JOB_TYPE } from '../../../common/constants'; +import { CSV_JOB_TYPE_DEPRECATED } from '../../../common/constants'; import { cryptoFactory } from '../../lib'; import { CreateJobFn, CreateJobFnFactory } from '../../types'; -import { IndexPatternSavedObject, JobParamsCSV, TaskPayloadCSV } from './types'; +import { + IndexPatternSavedObjectDeprecatedCSV, + JobParamsDeprecatedCSV, + TaskPayloadDeprecatedCSV, +} from './types'; export const createJobFnFactory: CreateJobFnFactory< - CreateJobFn + CreateJobFn > = function createJobFactoryFn(reporting, parentLogger) { - const logger = parentLogger.clone([CSV_JOB_TYPE, 'create-job']); + const logger = parentLogger.clone([CSV_JOB_TYPE_DEPRECATED, 'create-job']); const config = reporting.getConfig(); const crypto = cryptoFactory(config.get('encryptionKey')); @@ -24,7 +28,7 @@ export const createJobFnFactory: CreateJobFnFactory< const indexPatternSavedObject = ((await savedObjectsClient.get( 'index-pattern', jobParams.indexPatternId - )) as unknown) as IndexPatternSavedObject; // FIXME + )) as unknown) as IndexPatternSavedObjectDeprecatedCSV; return { headers: serializedEncryptedHeaders, diff --git a/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts b/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts index ea65262c090eee..098a90959f8a76 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/execute_job.test.ts @@ -22,7 +22,7 @@ import { LevelLogger } from '../../lib'; import { setFieldFormats } from '../../services'; import { createMockReportingCore } from '../../test_helpers'; import { runTaskFnFactory } from './execute_job'; -import { TaskPayloadCSV } from './types'; +import { TaskPayloadDeprecatedCSV } from './types'; const delay = (ms: number) => new Promise((resolve) => setTimeout(() => resolve(), ms)); @@ -31,7 +31,7 @@ const getRandomScrollId = () => { return puid.generate(); }; -const getBasePayload = (baseObj: any) => baseObj as TaskPayloadCSV; +const getBasePayload = (baseObj: any) => baseObj as TaskPayloadDeprecatedCSV; describe('CSV Execute Job', function () { const encryptionKey = 'testEncryptionKey'; diff --git a/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts b/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts index 6b4dd48583efec..cb321b7573701f 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/execute_job.ts @@ -4,20 +4,20 @@ * you may not use this file except in compliance with the Elastic License. */ -import { CONTENT_TYPE_CSV, CSV_JOB_TYPE } from '../../../common/constants'; +import { CONTENT_TYPE_CSV, CSV_JOB_TYPE_DEPRECATED } from '../../../common/constants'; import { RunTaskFn, RunTaskFnFactory } from '../../types'; import { decryptJobHeaders } from '../common'; import { createGenerateCsv } from './generate_csv'; -import { TaskPayloadCSV } from './types'; +import { TaskPayloadDeprecatedCSV } from './types'; export const runTaskFnFactory: RunTaskFnFactory< - RunTaskFn + RunTaskFn > = function executeJobFactoryFn(reporting, parentLogger) { const config = reporting.getConfig(); return async function runTask(jobId, job, cancellationToken) { const elasticsearch = reporting.getElasticsearchService(); - const logger = parentLogger.clone([CSV_JOB_TYPE, 'execute-job', jobId]); + const logger = parentLogger.clone([CSV_JOB_TYPE_DEPRECATED, 'execute-job', jobId]); const generateCsv = createGenerateCsv(logger); const encryptionKey = config.get('encryptionKey'); diff --git a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts index 4cb8de58105847..0c74e3aa54b0e8 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.test.ts @@ -6,13 +6,13 @@ import expect from '@kbn/expect'; import { fieldFormats, FieldFormatsGetConfigFn, UI_SETTINGS } from 'src/plugins/data/server'; -import { IndexPatternSavedObject } from '../types'; +import { IndexPatternSavedObjectDeprecatedCSV } from '../types'; import { fieldFormatMapFactory } from './field_format_map'; type ConfigValue = { number: { id: string; params: {} } } | string; describe('field format map', function () { - const indexPatternSavedObject: IndexPatternSavedObject = { + const indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV = { timeFieldName: '@timestamp', title: 'logstash-*', attributes: { diff --git a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts index e01fee530fc65a..c05dc7d3fd75f3 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/field_format_map.ts @@ -7,7 +7,7 @@ import _ from 'lodash'; import { FieldFormat } from 'src/plugins/data/common'; import { FieldFormatConfig, IFieldFormatsRegistry } from 'src/plugins/data/server'; -import { IndexPatternSavedObject } from '../types'; +import { IndexPatternSavedObjectDeprecatedCSV } from '../types'; /** * Create a map of FieldFormat instances for index pattern fields @@ -17,7 +17,7 @@ import { IndexPatternSavedObject } from '../types'; * @return {Map} key: field name, value: FieldFormat instance */ export function fieldFormatMapFactory( - indexPatternSavedObject: IndexPatternSavedObject, + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV, fieldFormatsRegistry: IFieldFormatsRegistry, timezone: string | undefined ) { diff --git a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts index 2f6df9cd67a758..ee09f3904678c2 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/generate_csv/index.ts @@ -12,7 +12,7 @@ import { CSV_BOM_CHARS } from '../../../../common/constants'; import { byteSizeValueToNumber } from '../../../../common/schema_utils'; import { LevelLogger } from '../../../lib'; import { getFieldFormats } from '../../../services'; -import { IndexPatternSavedObject, SavedSearchGeneratorResult } from '../types'; +import { IndexPatternSavedObjectDeprecatedCSV, SavedSearchGeneratorResult } from '../types'; import { checkIfRowsHaveFormulas } from './check_cells_for_formulas'; import { createEscapeValue } from './escape_value'; import { fieldFormatMapFactory } from './field_format_map'; @@ -39,7 +39,7 @@ interface SearchRequest { export interface GenerateCsvParams { browserTimezone?: string; searchRequest: SearchRequest; - indexPatternSavedObject: IndexPatternSavedObject; + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; fields: string[]; metaFields: string[]; conflictedTypesFields: string[]; diff --git a/x-pack/plugins/reporting/server/export_types/csv/index.ts b/x-pack/plugins/reporting/server/export_types/csv/index.ts index f7b7ff5709fe69..23f4b879eb1409 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/index.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/index.ts @@ -5,7 +5,7 @@ */ import { - CSV_JOB_TYPE as jobType, + CSV_JOB_TYPE_DEPRECATED as jobType, LICENSE_TYPE_BASIC, LICENSE_TYPE_ENTERPRISE, LICENSE_TYPE_GOLD, @@ -17,11 +17,11 @@ import { CreateJobFn, ExportTypeDefinition, RunTaskFn } from '../../types'; import { createJobFnFactory } from './create_job'; import { runTaskFnFactory } from './execute_job'; import { metadata } from './metadata'; -import { JobParamsCSV, TaskPayloadCSV } from './types'; +import { JobParamsDeprecatedCSV, TaskPayloadDeprecatedCSV } from './types'; export const getExportType = (): ExportTypeDefinition< - CreateJobFn, - RunTaskFn + CreateJobFn, + RunTaskFn > => ({ ...metadata, jobType, diff --git a/x-pack/plugins/reporting/server/export_types/csv/types.d.ts b/x-pack/plugins/reporting/server/export_types/csv/types.d.ts index 78615a0e7b72c6..dd0b37a17a2ffb 100644 --- a/x-pack/plugins/reporting/server/export_types/csv/types.d.ts +++ b/x-pack/plugins/reporting/server/export_types/csv/types.d.ts @@ -8,7 +8,7 @@ import { BaseParams, BasePayload } from '../../types'; export type RawValue = string | object | null | undefined; -export interface IndexPatternSavedObject { +export interface IndexPatternSavedObjectDeprecatedCSV { title: string; timeFieldName: string; fields?: any[]; @@ -18,25 +18,25 @@ export interface IndexPatternSavedObject { }; } -interface BaseParamsCSV { - searchRequest: SearchRequest; +interface BaseParamsDeprecatedCSV { + searchRequest: SearchRequestDeprecatedCSV; fields: string[]; metaFields: string[]; conflictedTypesFields: string[]; } -export type JobParamsCSV = BaseParamsCSV & +export type JobParamsDeprecatedCSV = BaseParamsDeprecatedCSV & BaseParams & { indexPatternId: string; }; // CSV create job method converts indexPatternID to indexPatternSavedObject -export type TaskPayloadCSV = BaseParamsCSV & +export type TaskPayloadDeprecatedCSV = BaseParamsDeprecatedCSV & BasePayload & { - indexPatternSavedObject: IndexPatternSavedObject; + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; }; -export interface SearchRequest { +export interface SearchRequestDeprecatedCSV { index: string; body: | { @@ -66,7 +66,7 @@ export interface SearchRequest { | any; } -type FormatsMap = Map< +type FormatsMapDeprecatedCSV = Map< string, { id: string; diff --git a/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts b/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts index e3631b9c89724c..fa983c5af639c0 100644 --- a/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts +++ b/x-pack/plugins/reporting/server/export_types/csv_from_savedobject/lib/get_data_source.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import { IndexPatternSavedObject } from '../../csv/types'; +import { IndexPatternSavedObjectDeprecatedCSV } from '../../csv/types'; import { SavedObjectReference, SavedSearchObjectAttributesJSON, SearchSource } from '../types'; export async function getDataSource( @@ -12,10 +12,10 @@ export async function getDataSource( indexPatternId?: string, savedSearchObjectId?: string ): Promise<{ - indexPatternSavedObject: IndexPatternSavedObject; + indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; searchSource: SearchSource | null; }> { - let indexPatternSavedObject: IndexPatternSavedObject; + let indexPatternSavedObject: IndexPatternSavedObjectDeprecatedCSV; let searchSource: SearchSource | null = null; if (savedSearchObjectId) { diff --git a/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts b/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts index 7706aa9d650c7c..641ce6e48a1f32 100644 --- a/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts +++ b/x-pack/plugins/reporting/server/routes/lib/get_document_payload.ts @@ -7,7 +7,7 @@ // @ts-ignore import contentDisposition from 'content-disposition'; import { get } from 'lodash'; -import { CSV_JOB_TYPE } from '../../../common/constants'; +import { CSV_JOB_TYPE_DEPRECATED } from '../../../common/constants'; import { ExportTypesRegistry, statuses } from '../../lib'; import { ReportDocument } from '../../lib/store'; import { TaskRunResult } from '../../lib/tasks'; @@ -33,7 +33,7 @@ const getTitle = (exportType: ExportTypeDefinition, title?: string): string => const getReportingHeaders = (output: TaskRunResult, exportType: ExportTypeDefinition) => { const metaDataHeaders: Record = {}; - if (exportType.jobType === CSV_JOB_TYPE) { + if (exportType.jobType === CSV_JOB_TYPE_DEPRECATED) { const csvContainsFormulas = get(output, 'csv_contains_formulas', false); const maxSizedReach = get(output, 'max_size_reached', false); diff --git a/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts b/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts index 30befcf291a549..8d69d75f66212f 100644 --- a/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts +++ b/x-pack/plugins/reporting/server/usage/decorate_range_stats.ts @@ -5,7 +5,7 @@ */ import { uniq } from 'lodash'; -import { CSV_JOB_TYPE, PDF_JOB_TYPE, PNG_JOB_TYPE } from '../../common/constants'; +import { CSV_JOB_TYPE_DEPRECATED, PDF_JOB_TYPE, PNG_JOB_TYPE } from '../../common/constants'; import { AvailableTotal, ExportType, FeatureAvailabilityMap, RangeStats } from './types'; function getForFeature( @@ -54,7 +54,7 @@ export const decorateRangeStats = ( // combine the known types with any unknown type found in reporting data const keysBasic = uniq([ - CSV_JOB_TYPE, + CSV_JOB_TYPE_DEPRECATED, PNG_JOB_TYPE, ...Object.keys(rangeStatsBasic), ]) as ExportType[]; diff --git a/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap b/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap index 99780542b97f48..d6eb4c20b8003a 100644 --- a/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap +++ b/x-pack/plugins/security/public/authentication/login/components/login_form/__snapshots__/login_form.test.tsx.snap @@ -128,6 +128,7 @@ exports[`LoginForm renders as expected 1`] = ` > { {...this.validator.validateUsername(this.state.username)} > { wrapper .find(`[data-test-subj="caseTitle"] input`) @@ -85,7 +99,12 @@ describe('Create case', () => { beforeEach(() => { jest.resetAllMocks(); + postCase.mockResolvedValue({ + id: sampleId, + ...sampleData, + }); usePostCaseMock.mockImplementation(() => defaultPostCase); + usePostPushToServiceMock.mockImplementation(() => defaultPostPushToService); useConnectorsMock.mockReturnValue(sampleConnectorData); useCaseConfigureMock.mockImplementation(() => useCaseConfigureResponse); useGetIncidentTypesMock.mockReturnValue(useGetIncidentTypesResponse); @@ -163,25 +182,6 @@ describe('Create case', () => { ); }); - it('should redirect to new case when caseData is there', async () => { - const sampleId = 'case-id'; - usePostCaseMock.mockImplementation(() => ({ - ...defaultPostCase, - caseData: { id: sampleId }, - })); - - mount( - - - - - - - ); - - await waitFor(() => expect(onFormSubmitSuccess).toHaveBeenCalledWith({ id: 'case-id' })); - }); - it('it should select the default connector set in the configuration', async () => { useCaseConfigureMock.mockImplementation(() => ({ ...useCaseConfigureResponse, @@ -258,12 +258,15 @@ describe('Create case', () => { fillForm(wrapper); wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => expect(postCase).toBeCalledWith(sampleData)); + await waitFor(() => { + expect(postCase).toBeCalledWith(sampleData); + expect(postPushToService).not.toHaveBeenCalled(); + }); }); }); describe('Step 2 - Connector Fields', () => { - it(`it should submit a Jira connector`, async () => { + it(`it should submit and push to Jira connector`, async () => { useConnectorsMock.mockReturnValue({ ...sampleConnectorData, connectors: connectorsMock, @@ -304,7 +307,7 @@ describe('Create case', () => { wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => + await waitFor(() => { expect(postCase).toBeCalledWith({ ...sampleData, connector: { @@ -313,11 +316,27 @@ describe('Create case', () => { type: '.jira', fields: { issueType: '10007', parent: null, priority: '2' }, }, - }) - ); + }); + expect(postPushToService).toHaveBeenCalledWith({ + caseId: sampleId, + caseServices: {}, + connector: { + id: 'jira-1', + name: 'Jira', + type: '.jira', + fields: { issueType: '10007', parent: null, priority: '2' }, + }, + alerts: {}, + updateCase: noop, + }); + expect(onFormSubmitSuccess).toHaveBeenCalledWith({ + id: sampleId, + ...sampleData, + }); + }); }); - it(`it should submit a resilient connector`, async () => { + it(`it should submit and push to resilient connector`, async () => { useConnectorsMock.mockReturnValue({ ...sampleConnectorData, connectors: connectorsMock, @@ -359,7 +378,7 @@ describe('Create case', () => { wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => + await waitFor(() => { expect(postCase).toBeCalledWith({ ...sampleData, connector: { @@ -368,11 +387,29 @@ describe('Create case', () => { type: '.resilient', fields: { incidentTypes: ['19'], severityCode: '4' }, }, - }) - ); + }); + + expect(postPushToService).toHaveBeenCalledWith({ + caseId: sampleId, + caseServices: {}, + connector: { + id: 'resilient-2', + name: 'My Connector 2', + type: '.resilient', + fields: { incidentTypes: ['19'], severityCode: '4' }, + }, + alerts: {}, + updateCase: noop, + }); + + expect(onFormSubmitSuccess).toHaveBeenCalledWith({ + id: sampleId, + ...sampleData, + }); + }); }); - it(`it should submit a servicenow connector`, async () => { + it(`it should submit and push to servicenow connector`, async () => { useConnectorsMock.mockReturnValue({ ...sampleConnectorData, connectors: connectorsMock, @@ -404,7 +441,7 @@ describe('Create case', () => { wrapper.find(`[data-test-subj="create-case-submit"]`).first().simulate('click'); - await waitFor(() => + await waitFor(() => { expect(postCase).toBeCalledWith({ ...sampleData, connector: { @@ -413,8 +450,26 @@ describe('Create case', () => { type: '.servicenow', fields: { impact: '2', severity: '2', urgency: '2' }, }, - }) - ); + }); + + expect(postPushToService).toHaveBeenCalledWith({ + caseId: sampleId, + caseServices: {}, + connector: { + id: 'servicenow-1', + name: 'My Connector', + type: '.servicenow', + fields: { impact: '2', severity: '2', urgency: '2' }, + }, + alerts: {}, + updateCase: noop, + }); + + expect(onFormSubmitSuccess).toHaveBeenCalledWith({ + id: sampleId, + ...sampleData, + }); + }); }); }); }); diff --git a/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx b/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx index 4315011ac8df16..03e03d853878cc 100644 --- a/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx +++ b/x-pack/plugins/security_solution/public/cases/components/create/form_context.tsx @@ -3,9 +3,8 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ - import React, { useCallback, useEffect, useMemo } from 'react'; - +import { noop } from 'lodash/fp'; import { schema, FormProps } from './schema'; import { Form, useForm } from '../../../shared_imports'; import { @@ -14,6 +13,8 @@ import { normalizeActionConnector, } from '../configure_cases/utils'; import { usePostCase } from '../../containers/use_post_case'; +import { usePostPushToService } from '../../containers/use_post_push_to_service'; + import { useConnectors } from '../../containers/configure/use_connectors'; import { useCaseConfigure } from '../../containers/configure/use_configure'; import { Case } from '../../containers/types'; @@ -34,7 +35,9 @@ interface Props { export const FormContext: React.FC = ({ children, onSuccess }) => { const { connectors } = useConnectors(); const { connector: configurationConnector } = useCaseConfigure(); - const { caseData, postCase } = usePostCase(); + const { postCase } = usePostCase(); + const { postPushToService } = usePostPushToService(); + const connectorId = useMemo( () => connectors.some((connector) => connector.id === configurationConnector.id) @@ -50,18 +53,33 @@ export const FormContext: React.FC = ({ children, onSuccess }) => { ) => { if (isValid) { const caseConnector = getConnectorById(dataConnectorId, connectors); + const connectorToUpdate = caseConnector ? normalizeActionConnector(caseConnector, fields) : getNoneConnector(); - await postCase({ + const updatedCase = await postCase({ ...dataWithoutConnectorId, connector: connectorToUpdate, settings: { syncAlerts }, }); + + if (updatedCase?.id && dataConnectorId !== 'none') { + await postPushToService({ + caseId: updatedCase.id, + caseServices: {}, + connector: connectorToUpdate, + alerts: {}, + updateCase: noop, + }); + } + + if (onSuccess && updatedCase) { + onSuccess(updatedCase); + } } }, - [postCase, connectors] + [connectors, postCase, onSuccess, postPushToService] ); const { form } = useForm({ @@ -70,18 +88,10 @@ export const FormContext: React.FC = ({ children, onSuccess }) => { schema, onSubmit: submitCase, }); - const { setFieldValue } = form; - // Set the selected connector to the configuration connector useEffect(() => setFieldValue('connectorId', connectorId), [connectorId, setFieldValue]); - useEffect(() => { - if (caseData && onSuccess) { - onSuccess(caseData); - } - }, [caseData, onSuccess]); - return
{children}
; }; diff --git a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx index 8e8432d0d190c3..bd57f57713e084 100644 --- a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx +++ b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.test.tsx @@ -6,9 +6,9 @@ import { renderHook, act } from '@testing-library/react-hooks'; import { usePostCase, UsePostCase } from './use_post_case'; -import { basicCasePost } from './mock'; import * as api from './api'; import { ConnectorTypes } from '../../../../case/common/api/connectors'; +import { basicCasePost } from './mock'; jest.mock('./api'); @@ -40,7 +40,6 @@ describe('usePostCase', () => { expect(result.current).toEqual({ isLoading: false, isError: false, - caseData: null, postCase: result.current.postCase, }); }); @@ -59,6 +58,16 @@ describe('usePostCase', () => { }); }); + it('calls postCase with correct result', async () => { + await act(async () => { + const { result, waitForNextUpdate } = renderHook(() => usePostCase()); + await waitForNextUpdate(); + + const postData = await result.current.postCase(samplePost); + expect(postData).toEqual(basicCasePost); + }); + }); + it('post case', async () => { await act(async () => { const { result, waitForNextUpdate } = renderHook(() => usePostCase()); @@ -66,7 +75,6 @@ describe('usePostCase', () => { result.current.postCase(samplePost); await waitForNextUpdate(); expect(result.current).toEqual({ - caseData: basicCasePost, isLoading: false, isError: false, postCase: result.current.postCase, @@ -96,7 +104,6 @@ describe('usePostCase', () => { result.current.postCase(samplePost); expect(result.current).toEqual({ - caseData: null, isLoading: false, isError: true, postCase: result.current.postCase, diff --git a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx index 3ca78dfe75c80c..c98446effe47dd 100644 --- a/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx +++ b/x-pack/plugins/security_solution/public/cases/containers/use_post_case.tsx @@ -3,25 +3,17 @@ * or more contributor license agreements. Licensed under the Elastic License; * you may not use this file except in compliance with the Elastic License. */ - -import { useReducer, useCallback } from 'react'; - +import { useReducer, useCallback, useRef, useEffect } from 'react'; import { CasePostRequest } from '../../../../case/common/api'; import { errorToToaster, useStateToaster } from '../../common/components/toasters'; import { postCase } from './api'; import * as i18n from './translations'; import { Case } from './types'; - interface NewCaseState { - caseData: Case | null; isLoading: boolean; isError: boolean; } -type Action = - | { type: 'FETCH_INIT' } - | { type: 'FETCH_SUCCESS'; payload: Case } - | { type: 'FETCH_FAILURE' }; - +type Action = { type: 'FETCH_INIT' } | { type: 'FETCH_SUCCESS' } | { type: 'FETCH_FAILURE' }; const dataFetchReducer = (state: NewCaseState, action: Action): NewCaseState => { switch (action.type) { case 'FETCH_INIT': @@ -35,7 +27,6 @@ const dataFetchReducer = (state: NewCaseState, action: Action): NewCaseState => ...state, isLoading: false, isError: false, - caseData: action.payload ?? null, }; case 'FETCH_FAILURE': return { @@ -47,47 +38,47 @@ const dataFetchReducer = (state: NewCaseState, action: Action): NewCaseState => return state; } }; - export interface UsePostCase extends NewCaseState { - postCase: (data: CasePostRequest) => Promise<() => void>; + postCase: (data: CasePostRequest) => Promise; } export const usePostCase = (): UsePostCase => { const [state, dispatch] = useReducer(dataFetchReducer, { isLoading: false, isError: false, - caseData: null, }); const [, dispatchToaster] = useStateToaster(); - - const postMyCase = useCallback(async (data: CasePostRequest) => { - let cancel = false; - const abortCtrl = new AbortController(); - - try { - dispatch({ type: 'FETCH_INIT' }); - const response = await postCase(data, abortCtrl.signal); - if (!cancel) { - dispatch({ - type: 'FETCH_SUCCESS', - payload: response, - }); + const cancel = useRef(false); + const abortCtrl = useRef(new AbortController()); + const postMyCase = useCallback( + async (data: CasePostRequest) => { + try { + dispatch({ type: 'FETCH_INIT' }); + abortCtrl.current.abort(); + cancel.current = false; + abortCtrl.current = new AbortController(); + const response = await postCase(data, abortCtrl.current.signal); + if (!cancel.current) { + dispatch({ type: 'FETCH_SUCCESS' }); + } + return response; + } catch (error) { + if (!cancel.current) { + errorToToaster({ + title: i18n.ERROR_TITLE, + error: error.body && error.body.message ? new Error(error.body.message) : error, + dispatchToaster, + }); + dispatch({ type: 'FETCH_FAILURE' }); + } } - } catch (error) { - if (!cancel) { - errorToToaster({ - title: i18n.ERROR_TITLE, - error: error.body && error.body.message ? new Error(error.body.message) : error, - dispatchToaster, - }); - dispatch({ type: 'FETCH_FAILURE' }); - } - } + }, + [dispatchToaster] + ); + useEffect(() => { return () => { - abortCtrl.abort(); - cancel = true; + abortCtrl.current.abort(); + cancel.current = true; }; - // eslint-disable-next-line react-hooks/exhaustive-deps }, []); - return { ...state, postCase: postMyCase }; }; diff --git a/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx b/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx index 191c3955caa9ba..c444702312ffb3 100644 --- a/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx +++ b/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx @@ -62,7 +62,7 @@ export const usePrivilegeUser = (): ReturnPrivilegeUser => { setPrivilegeUser({ isAuthenticated: privilege.is_authenticated, hasEncryptionKey: privilege.has_encryption_key, - hasIndexManage: privilege.index[indexName].manage, + hasIndexManage: privilege.index[indexName].manage && privilege.cluster.manage, hasIndexMaintenance: privilege.index[indexName].maintenance, hasIndexWrite: privilege.index[indexName].create || diff --git a/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx b/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx index 92e0ac757cc397..10e8ca42f35aee 100644 --- a/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx +++ b/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx @@ -9,7 +9,6 @@ import { EuiButtonEmpty, EuiFlexGroup, EuiFlexItem, - EuiFocusTrap, EuiScreenReaderOnly, } from '@elastic/eui'; import React, { useCallback } from 'react'; @@ -83,31 +82,29 @@ export const AddNote = React.memo<{ return ( - -
- -

{i18n.YOU_ARE_EDITING_A_NOTE}

- - - - {onCancelAddNote != null ? ( - - - - ) : null} +
+ +

{i18n.YOU_ARE_EDITING_A_NOTE}

+ + + + {onCancelAddNote != null ? ( - - {i18n.ADD_NOTE} - + - -
- + ) : null} + + + {i18n.ADD_NOTE} + + + +
); }); diff --git a/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts b/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts index 9ad094086b6320..de0cec3c06033e 100644 --- a/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts +++ b/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts @@ -5,9 +5,10 @@ */ /* eslint-disable no-console */ import yargs from 'yargs'; +import fs from 'fs'; import { Client, ClientOptions } from '@elastic/elasticsearch'; import { ResponseError } from '@elastic/elasticsearch/lib/errors'; -import { KbnClient, ToolingLog } from '@kbn/dev-utils'; +import { KbnClient, ToolingLog, CA_CERT_PATH } from '@kbn/dev-utils'; import { AxiosResponse } from 'axios'; import { indexHostsAndAlerts } from '../../common/endpoint/index_data'; import { ANCESTRY_LIMIT, EndpointDocGenerator } from '../../common/endpoint/generate_data'; @@ -202,15 +203,41 @@ async function main() { type: 'boolean', default: false, }, + ssl: { + alias: 'ssl', + describe: 'Use https for elasticsearch and kbn clients', + type: 'boolean', + default: false, + }, }).argv; + let ca: Buffer; + let kbnClient: KbnClientWithApiKeySupport; + let clientOptions: ClientOptions; - const kbnClient = new KbnClientWithApiKeySupport({ - log: new ToolingLog({ - level: 'info', - writeTo: process.stdout, - }), - url: argv.kibana, - }); + if (argv.ssl) { + ca = fs.readFileSync(CA_CERT_PATH); + const url = argv.kibana.replace('http:', 'https:'); + const node = argv.node.replace('http:', 'https:'); + kbnClient = new KbnClientWithApiKeySupport({ + log: new ToolingLog({ + level: 'info', + writeTo: process.stdout, + }), + url, + certificateAuthorities: [ca], + }); + clientOptions = { node, ssl: { ca: [ca] } }; + } else { + kbnClient = new KbnClientWithApiKeySupport({ + log: new ToolingLog({ + level: 'info', + writeTo: process.stdout, + }), + url: argv.kibana, + }); + clientOptions = { node: argv.node }; + } + const client = new Client(clientOptions); try { await doIngestSetup(kbnClient); @@ -219,9 +246,6 @@ async function main() { process.exit(1); } - const clientOptions: ClientOptions = { node: argv.node }; - const client = new Client(clientOptions); - if (argv.delete) { await deleteIndices( [argv.eventIndex, argv.metadataIndex, argv.policyIndex, argv.alertIndex], diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts index 3030bd8c52c705..2aa8981cc618bf 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts @@ -70,6 +70,7 @@ export const searchAfterAndBulkCreate = async ({ interval, buildRuleMessage, }); + const tuplesToBeLogged = [...totalToFromTuples]; logger.debug(buildRuleMessage(`totalToFromTuples: ${totalToFromTuples.length}`)); while (totalToFromTuples.length > 0) { @@ -294,5 +295,6 @@ export const searchAfterAndBulkCreate = async ({ } } logger.debug(buildRuleMessage(`[+] completed bulk index of ${toReturn.createdSignalsCount}`)); + toReturn.totalToFromTuples = tuplesToBeLogged; return toReturn; }; diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts index d08ab66af56836..2b0abdfdfa090b 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts @@ -670,6 +670,21 @@ export const signalRulesAlertType = ({ lastLookBackDate: result.lastLookBackDate?.toISOString(), }); } + + // adding this log line so we can get some information from cloud + logger.info( + buildRuleMessage( + `[+] Finished indexing ${result.createdSignalsCount} ${ + !isEmpty(result.totalToFromTuples) + ? `signals searched between date ranges ${JSON.stringify( + result.totalToFromTuples, + null, + 2 + )}` + : '' + }` + ) + ); } else { const errorMessage = buildRuleMessage( 'Bulk Indexing of signals failed:', diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts index 5ae411678aa03d..cb955673a7ea6a 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts @@ -5,7 +5,7 @@ */ import { DslQuery, Filter } from 'src/plugins/data/common'; -import moment from 'moment'; +import moment, { Moment } from 'moment'; import { Status } from '../../../../common/detection_engine/schemas/common/schemas'; import { RulesSchema } from '../../../../common/detection_engine/schemas/response/rules_schema'; import { @@ -263,6 +263,11 @@ export interface SearchAfterAndBulkCreateReturnType { createdSignalsCount: number; createdSignals: SignalHit[]; errors: string[]; + totalToFromTuples?: Array<{ + to: Moment | undefined; + from: Moment | undefined; + maxSignals: number; + }>; } export interface ThresholdAggregationBucket extends TermAggregationBucket { diff --git a/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts b/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts index 8b2cce01cf07a6..d25f1aaccc5e7f 100644 --- a/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts +++ b/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts @@ -37,7 +37,7 @@ export const securitySolutionSearchStrategyProvider = { + const { nextMajor } = contextValue.kibanaVersionInfo; return ( @@ -28,7 +28,7 @@ export const RootComponent = ({ i18n, ...contextValue }: AppDependencies) => { diff --git a/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx b/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx index 11c88a52ea24e0..865f1347137796 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/app_context.tsx @@ -6,10 +6,17 @@ import { DocLinksStart, HttpSetup } from 'src/core/public'; import React, { createContext, useContext } from 'react'; +export interface KibanaVersionContext { + currentMajor: number; + prevMajor: number; + nextMajor: number; +} + export interface ContextValue { http: HttpSetup; isCloudEnabled: boolean; docLinks: DocLinksStart; + kibanaVersionInfo: KibanaVersionContext; } export const AppContext = createContext({} as any); diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx index d9ec1832317391..5d0df54dd532cb 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/latest_minor_banner.tsx @@ -9,15 +9,16 @@ import React from 'react'; import { EuiCallOut, EuiLink } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { CURRENT_MAJOR_VERSION, NEXT_MAJOR_VERSION } from '../../../common/version'; import { useAppContext } from '../app_context'; export const LatestMinorBanner: React.FunctionComponent = () => { - const { docLinks } = useAppContext(); + const { docLinks, kibanaVersionInfo } = useAppContext(); const { ELASTIC_WEBSITE_URL } = docLinks; const esDocBasePath = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference`; + const { currentMajor, nextMajor } = kibanaVersionInfo; + return ( { /> ), - nextEsVersion: `${NEXT_MAJOR_VERSION}.x`, - currentEsVersion: `${CURRENT_MAJOR_VERSION}.x`, + nextEsVersion: `${nextMajor}.x`, + currentEsVersion: `${currentMajor}.x`, }} />

diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx index 6a99bd24ef26b6..600e764afd32bb 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs.test.tsx @@ -5,6 +5,7 @@ */ import React from 'react'; +import SemVer from 'semver/classes/semver'; import { mountWithIntl } from '@kbn/test/jest'; import { httpServiceMock } from 'src/core/public/mocks'; import { UpgradeAssistantTabs } from './tabs'; @@ -16,6 +17,7 @@ import { OverviewTab } from './tabs/overview'; const promisesToResolve = () => new Promise((resolve) => setTimeout(resolve, 0)); const mockHttp = httpServiceMock.createSetupContract(); +const mockKibanaVersion = new SemVer('8.0.0'); jest.mock('../app_context', () => { return { @@ -25,6 +27,11 @@ jest.mock('../app_context', () => { DOC_LINK_VERSION: 'current', ELASTIC_WEBSITE_URL: 'https://www.elastic.co/', }, + kibanaVersionInfo: { + currentMajor: mockKibanaVersion.major, + prevMajor: mockKibanaVersion.major - 1, + nextMajor: mockKibanaVersion.major + 1, + }, }; }, }; diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx index 3a1e042a3aa5f9..65dc9c25dacbdc 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.test.tsx @@ -6,6 +6,7 @@ import { shallow } from 'enzyme'; import React from 'react'; +import SemVer from 'semver/classes/semver'; import { LoadingState } from '../../types'; import AssistanceData from '../__fixtures__/checkup_api_response.json'; @@ -20,6 +21,8 @@ const defaultProps = { setSelectedTabIndex: jest.fn(), }; +const mockKibanaVersion = new SemVer('8.0.0'); + jest.mock('../../../app_context', () => { return { useAppContext: () => { @@ -28,6 +31,11 @@ jest.mock('../../../app_context', () => { DOC_LINK_VERSION: 'current', ELASTIC_WEBSITE_URL: 'https://www.elastic.co/', }, + kibanaVersionInfo: { + currentMajor: mockKibanaVersion.major, + prevMajor: mockKibanaVersion.major - 1, + nextMajor: mockKibanaVersion.major + 1, + }, }; }, }; diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx index 02cbc87483e55d..4fa4dafb55ff52 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/checkup_tab.tsx @@ -18,7 +18,6 @@ import { } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { NEXT_MAJOR_VERSION } from '../../../../../common/version'; import { LoadingErrorBanner } from '../../error_banner'; import { useAppContext } from '../../../app_context'; import { @@ -53,11 +52,13 @@ export const CheckupTab: FunctionComponent = ({ const [search, setSearch] = useState(''); const [currentGroupBy, setCurrentGroupBy] = useState(GroupByOption.message); - const { docLinks } = useAppContext(); + const { docLinks, kibanaVersionInfo } = useAppContext(); const { DOC_LINK_VERSION, ELASTIC_WEBSITE_URL } = docLinks; const esDocBasePath = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference/${DOC_LINK_VERSION}`; + const { nextMajor } = kibanaVersionInfo; + const changeFilter = (filter: LevelFilterOption) => { setCurrentFilter(filter); }; @@ -99,7 +100,7 @@ export const CheckupTab: FunctionComponent = ({ defaultMessage="These {strongCheckupLabel} issues need your attention. Resolve them before upgrading to Elasticsearch {nextEsVersion}." values={{ strongCheckupLabel: {checkupLabel}, - nextEsVersion: `${NEXT_MAJOR_VERSION}.x`, + nextEsVersion: `${nextMajor}.x`, }} />

diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx index 87899271677669..d7a30bf2e6a5e7 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/index.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { Fragment, FunctionComponent } from 'react'; +import React, { FunctionComponent } from 'react'; import { EuiFlexGroup, @@ -17,54 +17,59 @@ import { } from '@elastic/eui'; import { FormattedMessage } from '@kbn/i18n/react'; -import { NEXT_MAJOR_VERSION } from '../../../../../common/version'; +import { useAppContext } from '../../../app_context'; import { LoadingErrorBanner } from '../../error_banner'; import { LoadingState, UpgradeAssistantTabProps } from '../../types'; import { Steps } from './steps'; -export const OverviewTab: FunctionComponent = (props) => ( - - +export const OverviewTab: FunctionComponent = (props) => { + const { kibanaVersionInfo } = useAppContext(); + const { nextMajor } = kibanaVersionInfo; - -

- -

- + values={{ + nextEsVersion: `${nextMajor}.x`, + }} + /> +

+ - + - {props.alertBanner && ( - - {props.alertBanner} + {props.alertBanner && ( + <> + {props.alertBanner} - - - )} + + + )} - - - {props.loadingState === LoadingState.Success && } + + + {props.loadingState === LoadingState.Success && } - {props.loadingState === LoadingState.Loading && ( - - - - - - )} + {props.loadingState === LoadingState.Loading && ( + + + + + + )} - {props.loadingState === LoadingState.Error && ( - - )} - - - -); + {props.loadingState === LoadingState.Error && ( + + )} + + + + ); +}; diff --git a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx index dd392f6d1b2946..d81e7097680659 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx +++ b/x-pack/plugins/upgrade_assistant/public/application/components/tabs/overview/steps.tsx @@ -19,22 +19,21 @@ import { import { i18n } from '@kbn/i18n'; import { FormattedMessage } from '@kbn/i18n/react'; -import { CURRENT_MAJOR_VERSION, NEXT_MAJOR_VERSION } from '../../../../../common/version'; import { UpgradeAssistantTabProps } from '../../types'; import { DeprecationLoggingToggle } from './deprecation_logging_toggle'; import { useAppContext } from '../../../app_context'; // Leaving these here even if unused so they are picked up for i18n static analysis // Keep this until last minor release (when next major is also released). -const WAIT_FOR_RELEASE_STEP = { +const WAIT_FOR_RELEASE_STEP = (majorVersion: number, nextMajorVersion: number) => ({ title: i18n.translate('xpack.upgradeAssistant.overviewTab.steps.waitForReleaseStep.stepTitle', { defaultMessage: 'Wait for the Elasticsearch {nextEsVersion} release', values: { - nextEsVersion: `${NEXT_MAJOR_VERSION}.0`, + nextEsVersion: `${nextMajorVersion}.0`, }, }), children: ( - + <>

- + ), -}; +}); // Swap in this step for the one above it on the last minor release. // @ts-ignore @@ -100,11 +99,13 @@ export const Steps: FunctionComponent = ({ }, {} as { [checkupType: string]: number }); // Uncomment when START_UPGRADE_STEP is in use! - const { docLinks, http /* , isCloudEnabled */ } = useAppContext(); + const { kibanaVersionInfo, docLinks, http /* , isCloudEnabled */ } = useAppContext(); const { DOC_LINK_VERSION, ELASTIC_WEBSITE_URL } = docLinks; const esDocBasePath = `${ELASTIC_WEBSITE_URL}guide/en/elasticsearch/reference/${DOC_LINK_VERSION}`; + const { currentMajor, nextMajor } = kibanaVersionInfo; + return ( = ({ /> ), - nextEsVersion: `${NEXT_MAJOR_VERSION}.0`, + nextEsVersion: `${nextMajor}.0`, }} />

@@ -278,7 +279,7 @@ export const Steps: FunctionComponent = ({ }, // Swap in START_UPGRADE_STEP on the last minor release. - WAIT_FOR_RELEASE_STEP, + WAIT_FOR_RELEASE_STEP(currentMajor, nextMajor), // START_UPGRADE_STEP(isCloudEnabled, esDocBasePath), ]} /> diff --git a/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts b/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts index c0124d52e45d74..906bf8f6c07099 100644 --- a/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts +++ b/x-pack/plugins/upgrade_assistant/public/application/mount_management_section.ts @@ -6,11 +6,13 @@ import { CoreSetup } from 'src/core/public'; import { ManagementAppMountParams } from '../../../../../src/plugins/management/public'; import { renderApp } from './render_app'; +import { KibanaVersionContext } from './app_context'; export async function mountManagementSection( coreSetup: CoreSetup, isCloudEnabled: boolean, - params: ManagementAppMountParams + params: ManagementAppMountParams, + kibanaVersionInfo: KibanaVersionContext ) { const [{ i18n, docLinks }] = await coreSetup.getStartServices(); return renderApp({ @@ -19,5 +21,6 @@ export async function mountManagementSection( http: coreSetup.http, i18n, docLinks, + kibanaVersionInfo, }); } diff --git a/x-pack/plugins/upgrade_assistant/public/plugin.ts b/x-pack/plugins/upgrade_assistant/public/plugin.ts index 98f1b8351b88b8..4bbb79cdd83b86 100644 --- a/x-pack/plugins/upgrade_assistant/public/plugin.ts +++ b/x-pack/plugins/upgrade_assistant/public/plugin.ts @@ -4,13 +4,13 @@ * you may not use this file except in compliance with the Elastic License. */ +import SemVer from 'semver/classes/semver'; import { i18n } from '@kbn/i18n'; import { Plugin, CoreSetup, PluginInitializerContext } from 'src/core/public'; import { CloudSetup } from '../../cloud/public'; import { ManagementSetup } from '../../../../src/plugins/management/public'; -import { NEXT_MAJOR_VERSION } from '../common/version'; import { Config } from '../common/config'; interface Dependencies { @@ -29,10 +29,17 @@ export class UpgradeAssistantUIPlugin implements Plugin { const appRegistrar = management.sections.section.stack; const isCloudEnabled = Boolean(cloud?.isCloudEnabled); + const kibanaVersion = new SemVer(this.ctx.env.packageInfo.version); + + const kibanaVersionInfo = { + currentMajor: kibanaVersion.major, + prevMajor: kibanaVersion.major - 1, + nextMajor: kibanaVersion.major + 1, + }; const pluginName = i18n.translate('xpack.upgradeAssistant.appTitle', { defaultMessage: '{version} Upgrade Assistant', - values: { version: `${NEXT_MAJOR_VERSION}.0` }, + values: { version: `${kibanaVersionInfo.nextMajor}.0` }, }); appRegistrar.registerApp({ @@ -47,8 +54,14 @@ export class UpgradeAssistantUIPlugin implements Plugin { } = coreStart; docTitle.change(pluginName); + const { mountManagementSection } = await import('./application/mount_management_section'); - const unmountAppCallback = await mountManagementSection(coreSetup, isCloudEnabled, params); + const unmountAppCallback = await mountManagementSection( + coreSetup, + isCloudEnabled, + params, + kibanaVersionInfo + ); return () => { docTitle.reset(); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/__fixtures__/version.ts b/x-pack/plugins/upgrade_assistant/server/lib/__fixtures__/version.ts new file mode 100644 index 00000000000000..f08f449bbdae99 --- /dev/null +++ b/x-pack/plugins/upgrade_assistant/server/lib/__fixtures__/version.ts @@ -0,0 +1,20 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import { SemVer } from 'semver'; + +export const MOCK_VERSION_STRING = '8.0.0'; + +export const getMockVersionInfo = (versionString = MOCK_VERSION_STRING) => { + const currentVersion = new SemVer(versionString); + const currentMajor = currentVersion.major; + + return { + currentVersion, + currentMajor, + prevMajor: currentMajor - 1, + nextMajor: currentMajor + 1, + }; +}; diff --git a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts index 2310f993ce27d2..74ec268d71e849 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.test.ts @@ -7,12 +7,16 @@ import { SemVer } from 'semver'; import { IScopedClusterClient, kibanaResponseFactory } from 'src/core/server'; import { xpackMocks } from '../../../../mocks'; -import { CURRENT_VERSION } from '../../common/version'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from './__fixtures__/version'; + import { esVersionCheck, getAllNodeVersions, verifyAllMatchKibanaVersion, } from './es_version_precheck'; +import { versionService } from './version'; + +const { currentMajor, currentVersion } = getMockVersionInfo(); describe('getAllNodeVersions', () => { it('returns a list of unique node versions', async () => { @@ -41,25 +45,25 @@ describe('getAllNodeVersions', () => { describe('verifyAllMatchKibanaVersion', () => { it('detects higher version nodes', () => { - const result = verifyAllMatchKibanaVersion([new SemVer('99999.0.0')]); + const result = verifyAllMatchKibanaVersion([new SemVer('99999.0.0')], currentMajor); expect(result.allNodesMatch).toBe(false); expect(result.allNodesUpgraded).toBe(true); }); it('detects lower version nodes', () => { - const result = verifyAllMatchKibanaVersion([new SemVer('0.0.0')]); + const result = verifyAllMatchKibanaVersion([new SemVer('0.0.0')], currentMajor); expect(result.allNodesMatch).toBe(false); expect(result.allNodesUpgraded).toBe(true); }); it('detects if all are on same major correctly', () => { const versions = [ - CURRENT_VERSION, - CURRENT_VERSION.inc('minor'), - CURRENT_VERSION.inc('minor').inc('minor'), + currentVersion, + currentVersion.inc('minor'), + currentVersion.inc('minor').inc('minor'), ]; - const result = verifyAllMatchKibanaVersion(versions); + const result = verifyAllMatchKibanaVersion(versions, currentMajor); expect(result.allNodesMatch).toBe(true); expect(result.allNodesUpgraded).toBe(false); }); @@ -67,17 +71,21 @@ describe('verifyAllMatchKibanaVersion', () => { it('detects partial matches', () => { const versions = [ new SemVer('0.0.0'), - CURRENT_VERSION.inc('minor'), - CURRENT_VERSION.inc('minor').inc('minor'), + currentVersion.inc('minor'), + currentVersion.inc('minor').inc('minor'), ]; - const result = verifyAllMatchKibanaVersion(versions); + const result = verifyAllMatchKibanaVersion(versions, currentMajor); expect(result.allNodesMatch).toBe(false); expect(result.allNodesUpgraded).toBe(false); }); }); describe('EsVersionPrecheck', () => { + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + }); + it('returns a 403 when callCluster fails with a 403', async () => { const fakeCall = jest.fn().mockRejectedValue({ statusCode: 403 }); @@ -107,8 +115,8 @@ describe('EsVersionPrecheck', () => { info: jest.fn().mockResolvedValue({ body: { nodes: { - node1: { version: CURRENT_VERSION.raw }, - node2: { version: new SemVer(CURRENT_VERSION.raw).inc('major').raw }, + node1: { version: currentVersion.raw }, + node2: { version: new SemVer(currentVersion.raw).inc('major').raw }, }, }, }), @@ -132,8 +140,8 @@ describe('EsVersionPrecheck', () => { info: jest.fn().mockResolvedValue({ body: { nodes: { - node1: { version: new SemVer(CURRENT_VERSION.raw).inc('major').raw }, - node2: { version: new SemVer(CURRENT_VERSION.raw).inc('major').raw }, + node1: { version: new SemVer(currentVersion.raw).inc('major').raw }, + node2: { version: new SemVer(currentVersion.raw).inc('major').raw }, }, }, }), @@ -157,8 +165,8 @@ describe('EsVersionPrecheck', () => { info: jest.fn().mockResolvedValue({ body: { nodes: { - node1: { version: CURRENT_VERSION.raw }, - node2: { version: CURRENT_VERSION.raw }, + node1: { version: currentVersion.raw }, + node2: { version: currentVersion.raw }, }, }, }), diff --git a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts index be6c4f5ff02308..c308334c6cb08d 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/es_version_precheck.ts @@ -13,7 +13,7 @@ import { RequestHandler, RequestHandlerContext, } from 'src/core/server'; -import { CURRENT_VERSION } from '../../common/version'; +import { versionService } from './version'; interface Nodes { nodes: { @@ -39,14 +39,14 @@ export const getAllNodeVersions = async (adminClient: IScopedClusterClient) => { .map((version) => new SemVer(version)); }; -export const verifyAllMatchKibanaVersion = (allNodeVersions: SemVer[]) => { +export const verifyAllMatchKibanaVersion = (allNodeVersions: SemVer[], majorVersion: number) => { // Determine if all nodes in the cluster are running the same major version as Kibana. const numDifferentVersion = allNodeVersions.filter( - (esNodeVersion) => esNodeVersion.major !== CURRENT_VERSION.major + (esNodeVersion) => esNodeVersion.major !== majorVersion ).length; const numSameVersion = allNodeVersions.filter( - (esNodeVersion) => esNodeVersion.major === CURRENT_VERSION.major + (esNodeVersion) => esNodeVersion.major === majorVersion ).length; if (numDifferentVersion) { @@ -83,7 +83,9 @@ export const esVersionCheck = async ( throw e; } - const result = verifyAllMatchKibanaVersion(allNodeVersions); + const majorVersion = versionService.getMajorVersion(); + + const result = verifyAllMatchKibanaVersion(allNodeVersions, majorVersion); if (!result.allNodesMatch) { return response.customError({ // 426 means "Upgrade Required" and is used when semver compatibility is not met. diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts index 9ec06b72f02e2c..2111b77422f3ee 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.test.ts @@ -4,7 +4,9 @@ * you may not use this file except in compliance with the Elastic License. */ -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; +import { versionService } from '../version'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from '../__fixtures__/version'; + import { generateNewIndexName, getReindexWarnings, @@ -12,6 +14,8 @@ import { transformFlatSettings, } from './index_settings'; +const { currentMajor, prevMajor } = getMockVersionInfo(); + describe('transformFlatSettings', () => { it('does not blow up for empty mappings', () => { expect( @@ -56,6 +60,10 @@ describe('transformFlatSettings', () => { }); describe('sourceNameForIndex', () => { + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + }); + it('parses internal indices', () => { expect(sourceNameForIndex('.myInternalIndex')).toEqual('.myInternalIndex'); }); @@ -69,42 +77,46 @@ describe('sourceNameForIndex', () => { expect(sourceNameForIndex('.myInternalIndex-reindexed-v5')).toEqual('.myInternalIndex'); }); - it('replaces reindexed-v${PREV_MAJOR_VERSION} with reindexed-v${CURRENT_MAJOR_VERSION} in newIndexName', () => { - expect(sourceNameForIndex(`reindexed-v${PREV_MAJOR_VERSION}-myIndex`)).toEqual('myIndex'); - expect(sourceNameForIndex(`.reindexed-v${PREV_MAJOR_VERSION}-myInternalIndex`)).toEqual( + it(`replaces reindexed-v${prevMajor} with reindexed-v${currentMajor} in newIndexName`, () => { + expect(sourceNameForIndex(`reindexed-v${prevMajor}-myIndex`)).toEqual('myIndex'); + expect(sourceNameForIndex(`.reindexed-v${prevMajor}-myInternalIndex`)).toEqual( '.myInternalIndex' ); }); }); describe('generateNewIndexName', () => { + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + }); + it('parses internal indices', () => { expect(generateNewIndexName('.myInternalIndex')).toEqual( - `.reindexed-v${CURRENT_MAJOR_VERSION}-myInternalIndex` + `.reindexed-v${currentMajor}-myInternalIndex` ); }); it('parses non-internal indices', () => { - expect(generateNewIndexName('myIndex')).toEqual(`reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`); + expect(generateNewIndexName('myIndex')).toEqual(`reindexed-v${currentMajor}-myIndex`); }); it('excludes appended v5 reindexing string from generateNewIndexName', () => { expect(generateNewIndexName('myIndex-reindexed-v5')).toEqual( - `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex` + `reindexed-v${currentMajor}-myIndex` ); expect(generateNewIndexName('.myInternalIndex-reindexed-v5')).toEqual( - `.reindexed-v${CURRENT_MAJOR_VERSION}-myInternalIndex` + `.reindexed-v${currentMajor}-myInternalIndex` ); }); - it('replaces reindexed-v${PREV_MAJOR_VERSION} with reindexed-v${CURRENT_MAJOR_VERSION} in generateNewIndexName', () => { - expect(generateNewIndexName(`reindexed-v${PREV_MAJOR_VERSION}-myIndex`)).toEqual( - `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex` + it(`replaces reindexed-v${prevMajor} with reindexed-v${currentMajor} in generateNewIndexName`, () => { + expect(generateNewIndexName(`reindexed-v${prevMajor}-myIndex`)).toEqual( + `reindexed-v${currentMajor}-myIndex` ); - expect(generateNewIndexName(`.reindexed-v${PREV_MAJOR_VERSION}-myInternalIndex`)).toEqual( - `.reindexed-v${CURRENT_MAJOR_VERSION}-myInternalIndex` + expect(generateNewIndexName(`.reindexed-v${prevMajor}-myInternalIndex`)).toEqual( + `.reindexed-v${currentMajor}-myInternalIndex` ); }); }); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts index 5722a6c29b68ff..b632bbfa1faec0 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/index_settings.ts @@ -6,7 +6,7 @@ import { flow, omit } from 'lodash'; import { ReindexWarning } from '../../../common/types'; -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; +import { versionService } from '../version'; import { FlatSettings } from './types'; export interface ParsedIndexName { @@ -44,7 +44,10 @@ export const sourceNameForIndex = (indexName: string): string => { // in 5.6 the upgrade assistant appended to the index, in 6.7+ we prepend to // avoid conflicts with index patterns/templates/etc - const reindexedMatcher = new RegExp(`(-reindexed-v5$|reindexed-v${PREV_MAJOR_VERSION}-)`, 'g'); + const reindexedMatcher = new RegExp( + `(-reindexed-v5$|reindexed-v${versionService.getPrevMajorVersion()}-)`, + 'g' + ); const cleanBaseName = baseName.replace(reindexedMatcher, ''); return `${internal}${cleanBaseName}`; @@ -58,7 +61,7 @@ export const sourceNameForIndex = (indexName: string): string => { */ export const generateNewIndexName = (indexName: string): string => { const sourceName = sourceNameForIndex(indexName); - const currentVersion = `reindexed-v${CURRENT_MAJOR_VERSION}`; + const currentVersion = `reindexed-v${versionService.getMajorVersion()}`; return indexName.startsWith('.') ? `.${currentVersion}-${sourceName.substr(1)}` diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts index d059c03bcecb11..9a6ac4030e0513 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.test.ts @@ -17,8 +17,11 @@ import { ReindexStatus, ReindexStep, } from '../../../common/types'; -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; +import { versionService } from '../version'; import { LOCK_WINDOW, ReindexActions, reindexActionsFactory } from './reindex_actions'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from '../__fixtures__/version'; + +const { currentMajor, prevMajor } = getMockVersionInfo(); describe('ReindexActions', () => { let client: jest.Mocked; @@ -47,13 +50,16 @@ describe('ReindexActions', () => { }); describe('createReindexOp', () => { - beforeEach(() => client.create.mockResolvedValue()); + beforeEach(() => { + versionService.setup(MOCK_VERSION_STRING); + client.create.mockResolvedValue(); + }); - it(`prepends reindexed-v${CURRENT_MAJOR_VERSION} to new name`, async () => { + it(`prepends reindexed-v${currentMajor} to new name`, async () => { await actions.createReindexOp('myIndex'); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { indexName: 'myIndex', - newIndexName: `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`, + newIndexName: `reindexed-v${currentMajor}-myIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -65,11 +71,11 @@ describe('ReindexActions', () => { }); }); - it(`prepends reindexed-v${CURRENT_MAJOR_VERSION} to new name, preserving leading period`, async () => { + it(`prepends reindexed-v${currentMajor} to new name, preserving leading period`, async () => { await actions.createReindexOp('.internalIndex'); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { indexName: '.internalIndex', - newIndexName: `.reindexed-v${CURRENT_MAJOR_VERSION}-internalIndex`, + newIndexName: `.reindexed-v${currentMajor}-internalIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -82,12 +88,12 @@ describe('ReindexActions', () => { }); // in v5.6, the upgrade assistant appended to the index name instead of prepending - it(`prepends reindexed-v${CURRENT_MAJOR_VERSION}- and removes reindex appended in v5`, async () => { + it(`prepends reindexed-v${currentMajor}- and removes reindex appended in v5`, async () => { const indexName = 'myIndex-reindexed-v5'; await actions.createReindexOp(indexName); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { indexName, - newIndexName: `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`, + newIndexName: `reindexed-v${currentMajor}-myIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -99,11 +105,11 @@ describe('ReindexActions', () => { }); }); - it(`replaces reindexed-v${PREV_MAJOR_VERSION} with reindexed-v${CURRENT_MAJOR_VERSION}`, async () => { - await actions.createReindexOp(`reindexed-v${PREV_MAJOR_VERSION}-myIndex`); + it(`replaces reindexed-v${prevMajor} with reindexed-v${currentMajor}`, async () => { + await actions.createReindexOp(`reindexed-v${prevMajor}-myIndex`); expect(client.create).toHaveBeenCalledWith(REINDEX_OP_TYPE, { - indexName: `reindexed-v${PREV_MAJOR_VERSION}-myIndex`, - newIndexName: `reindexed-v${CURRENT_MAJOR_VERSION}-myIndex`, + indexName: `reindexed-v${prevMajor}-myIndex`, + newIndexName: `reindexed-v${currentMajor}-myIndex`, reindexOptions: undefined, status: ReindexStatus.inProgress, lastCompletedStep: ReindexStep.created, @@ -291,7 +297,7 @@ describe('ReindexActions', () => { } as RequestEvent); it('returns flat settings', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce( + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce( asApiResponse({ myIndex: { settings: { 'index.mySetting': '1' }, @@ -306,7 +312,7 @@ describe('ReindexActions', () => { }); it('returns null if index does not exist', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce(asApiResponse({})); + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce(asApiResponse({})); await expect(actions.getFlatSettings('myIndex')).resolves.toBeNull(); }); }); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts index 611ab3c92b72b0..653bf8336255b2 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts @@ -236,7 +236,7 @@ export const reindexActionsFactory = ( }, async getFlatSettings(indexName: string) { - const { body: flatSettings } = await esClient.indices.getSettings<{ + const { body: flatSettings } = await esClient.indices.get<{ [indexName: string]: FlatSettings; }>({ index: indexName, diff --git a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts index 8a7033c1594da3..29c8207a5f2848 100644 --- a/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts +++ b/x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts @@ -18,11 +18,12 @@ import { ReindexStatus, ReindexStep, } from '../../../common/types'; -import { CURRENT_MAJOR_VERSION, PREV_MAJOR_VERSION } from '../../../common/version'; import { licensingMock } from '../../../../licensing/server/mocks'; import { LicensingPluginSetup } from '../../../../licensing/server'; +import { MOCK_VERSION_STRING, getMockVersionInfo } from '../__fixtures__/version'; import { esIndicesStateCheck } from '../es_indices_state_check'; +import { versionService } from '../version'; import { isMlIndex, @@ -36,6 +37,8 @@ const asApiResponse = (body: T): RequestEvent => body, } as RequestEvent); +const { currentMajor, prevMajor } = getMockVersionInfo(); + describe('reindexService', () => { let actions: jest.Mocked; let clusterClient: ScopedClusterClientMock; @@ -82,6 +85,8 @@ describe('reindexService', () => { log, licensingPluginSetup ); + + versionService.setup(MOCK_VERSION_STRING); }); describe('hasRequiredPrivileges', () => { @@ -107,7 +112,7 @@ describe('reindexService', () => { cluster: ['manage'], index: [ { - names: ['anIndex', `reindexed-v${CURRENT_MAJOR_VERSION}-anIndex`], + names: ['anIndex', `reindexed-v${currentMajor}-anIndex`], allow_restricted_indices: true, privileges: ['all'], }, @@ -131,7 +136,7 @@ describe('reindexService', () => { cluster: ['manage', 'manage_ml'], index: [ { - names: ['.ml-anomalies', `.reindexed-v${CURRENT_MAJOR_VERSION}-ml-anomalies`], + names: ['.ml-anomalies', `.reindexed-v${currentMajor}-ml-anomalies`], allow_restricted_indices: true, privileges: ['all'], }, @@ -149,9 +154,7 @@ describe('reindexService', () => { asApiResponse({ has_all_requested: true }) ); - const hasRequired = await service.hasRequiredPrivileges( - `reindexed-v${PREV_MAJOR_VERSION}-anIndex` - ); + const hasRequired = await service.hasRequiredPrivileges(`reindexed-v${prevMajor}-anIndex`); expect(hasRequired).toBe(true); expect(clusterClient.asCurrentUser.security.hasPrivileges).toHaveBeenCalledWith({ body: { @@ -159,8 +162,8 @@ describe('reindexService', () => { index: [ { names: [ - `reindexed-v${PREV_MAJOR_VERSION}-anIndex`, - `reindexed-v${CURRENT_MAJOR_VERSION}-anIndex`, + `reindexed-v${prevMajor}-anIndex`, + `reindexed-v${currentMajor}-anIndex`, 'anIndex', ], allow_restricted_indices: true, @@ -188,7 +191,7 @@ describe('reindexService', () => { cluster: ['manage', 'manage_watcher'], index: [ { - names: ['.watches', `.reindexed-v${CURRENT_MAJOR_VERSION}-watches`], + names: ['.watches', `.reindexed-v${currentMajor}-watches`], allow_restricted_indices: true, privileges: ['all'], }, @@ -497,9 +500,9 @@ describe('reindexService', () => { }); it('is true for ML re-indexed indices', () => { - expect(isMlIndex(`.reindexed-v${PREV_MAJOR_VERSION}-ml-state`)).toBe(true); - expect(isMlIndex(`.reindexed-v${PREV_MAJOR_VERSION}-ml-anomalies`)).toBe(true); - expect(isMlIndex(`.reindexed-v${PREV_MAJOR_VERSION}-ml-config`)).toBe(true); + expect(isMlIndex(`.reindexed-v${prevMajor}-ml-state`)).toBe(true); + expect(isMlIndex(`.reindexed-v${prevMajor}-ml-anomalies`)).toBe(true); + expect(isMlIndex(`.reindexed-v${prevMajor}-ml-config`)).toBe(true); }); }); @@ -514,8 +517,8 @@ describe('reindexService', () => { }); it('is true for watcher re-indexed indices', () => { - expect(isWatcherIndex(`.reindexed-v${PREV_MAJOR_VERSION}-watches`)).toBe(true); - expect(isWatcherIndex(`.reindexed-v${PREV_MAJOR_VERSION}-triggered-watches`)).toBe(true); + expect(isWatcherIndex(`.reindexed-v${prevMajor}-watches`)).toBe(true); + expect(isWatcherIndex(`.reindexed-v${prevMajor}-triggered-watches`)).toBe(true); }); }); @@ -829,7 +832,7 @@ describe('reindexService', () => { }); it('fails if create index is not acknowledged', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce( + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce( asApiResponse({ myIndex: settingsMappings }) ); @@ -844,7 +847,7 @@ describe('reindexService', () => { }); it('fails if create index fails', async () => { - clusterClient.asCurrentUser.indices.getSettings.mockResolvedValueOnce( + clusterClient.asCurrentUser.indices.get.mockResolvedValueOnce( asApiResponse({ myIndex: settingsMappings }) ); diff --git a/x-pack/plugins/upgrade_assistant/server/lib/version.ts b/x-pack/plugins/upgrade_assistant/server/lib/version.ts new file mode 100644 index 00000000000000..f33200d2156383 --- /dev/null +++ b/x-pack/plugins/upgrade_assistant/server/lib/version.ts @@ -0,0 +1,33 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import SemVer from 'semver/classes/semver'; + +export class Version { + private version!: SemVer; + + public setup(version: string) { + this.version = new SemVer(version); + } + + public getCurrentVersion() { + return this.version; + } + + public getMajorVersion() { + return this.version?.major; + } + + public getNextMajorVersion() { + return this.version?.major + 1; + } + + public getPrevMajorVersion() { + return this.version?.major - 1; + } +} + +export const versionService = new Version(); diff --git a/x-pack/plugins/upgrade_assistant/server/plugin.ts b/x-pack/plugins/upgrade_assistant/server/plugin.ts index 9ef0f250da8efd..ea3677d01423bf 100644 --- a/x-pack/plugins/upgrade_assistant/server/plugin.ts +++ b/x-pack/plugins/upgrade_assistant/server/plugin.ts @@ -22,6 +22,7 @@ import { LicensingPluginSetup } from '../../licensing/server'; import { CredentialStore, credentialStoreFactory } from './lib/reindexing/credential_store'; import { ReindexWorker } from './lib/reindexing'; import { registerUpgradeAssistantUsageCollector } from './lib/telemetry'; +import { versionService } from './lib/version'; import { registerClusterCheckupRoutes } from './routes/cluster_checkup'; import { registerDeprecationLoggingRoutes } from './routes/deprecation_logging'; import { registerReindexIndicesRoutes, createReindexWorker } from './routes/reindex_indices'; @@ -40,6 +41,7 @@ interface PluginsSetup { export class UpgradeAssistantServerPlugin implements Plugin { private readonly logger: Logger; private readonly credentialStore: CredentialStore; + private readonly kibanaVersion: string; // Properties set at setup private licensing?: LicensingPluginSetup; @@ -48,9 +50,10 @@ export class UpgradeAssistantServerPlugin implements Plugin { private savedObjectsServiceStart?: SavedObjectsServiceStart; private worker?: ReindexWorker; - constructor({ logger }: PluginInitializerContext) { + constructor({ logger, env }: PluginInitializerContext) { this.logger = logger.get(); this.credentialStore = credentialStoreFactory(); + this.kibanaVersion = env.packageInfo.version; } private getWorker() { @@ -98,6 +101,9 @@ export class UpgradeAssistantServerPlugin implements Plugin { licensing, }; + // Initialize version service with current kibana version + versionService.setup(this.kibanaVersion); + registerClusterCheckupRoutes(dependencies); registerDeprecationLoggingRoutes(dependencies); registerReindexIndicesRoutes(dependencies, this.getWorker.bind(this)); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx deleted file mode 100644 index be4f0fc62271d2..00000000000000 --- a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx +++ /dev/null @@ -1,219 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ - -import React, { useContext, useEffect, useState } from 'react'; -import { - EuiButtonIcon, - EuiFlexGroup, - EuiFlexItem, - EuiImage, - EuiSpacer, - EuiText, - EuiLoadingSpinner, -} from '@elastic/eui'; -import useIntersection from 'react-use/lib/useIntersection'; -import moment from 'moment'; -import styled from 'styled-components'; -import { FormattedMessage } from '@kbn/i18n/react'; -import { Ping } from '../../../../../common/runtime_types/ping'; -import { getShortTimeStamp } from '../../../overview/monitor_list/columns/monitor_status_column'; -import { euiStyled } from '../../../../../../../../src/plugins/kibana_react/common'; -import { useFetcher, FETCH_STATUS } from '../../../../../../observability/public'; -import { getJourneyScreenshot } from '../../../../state/api/journey'; -import { UptimeSettingsContext } from '../../../../contexts'; - -const StepImage = styled(EuiImage)` - &&& { - display: flex; - figcaption { - white-space: nowrap; - align-self: center; - margin-left: 8px; - margin-top: 8px; - text-decoration: none !important; - } - } -`; - -const StepDiv = styled.div` - figure.euiImage { - div.stepArrowsFullScreen { - display: none; - } - } - - figure.euiImage-isFullScreen { - div.stepArrowsFullScreen { - display: flex; - } - } - position: relative; - div.stepArrows { - display: none; - } - :hover { - div.stepArrows { - display: flex; - } - } -`; - -interface Props { - timestamp: string; - ping: Ping; -} - -export const PingTimestamp = ({ timestamp, ping }: Props) => { - const [stepNo, setStepNo] = useState(1); - - const [stepImages, setStepImages] = useState([]); - - const intersectionRef = React.useRef(null); - - const { basePath } = useContext(UptimeSettingsContext); - - const imgPath = basePath + `/api/uptime/journey/screenshot/${ping.monitor.check_group}/${stepNo}`; - - const intersection = useIntersection(intersectionRef, { - root: null, - rootMargin: '0px', - threshold: 1, - }); - - const { data, status } = useFetcher(() => { - if (intersection && intersection.intersectionRatio === 1 && !stepImages[stepNo - 1]) - return getJourneyScreenshot(imgPath); - }, [intersection?.intersectionRatio, stepNo]); - - useEffect(() => { - if (data) { - setStepImages((prevState) => [...prevState, data?.src]); - } - }, [data]); - - const imgSrc = stepImages[stepNo] || data?.src; - - const isLoading = status === FETCH_STATUS.LOADING; - const isPending = status === FETCH_STATUS.PENDING; - - const captionContent = `Step:${stepNo} ${data?.stepName}`; - - const ImageCaption = ( - <> -
- {imgSrc && ( - - - { - setStepNo(stepNo - 1); - }} - iconType="arrowLeft" - aria-label="Next" - /> - - - {captionContent} - - - { - setStepNo(stepNo + 1); - }} - iconType="arrowRight" - aria-label="Next" - /> - - - )} -
- {/* TODO: Add link to details page once it's available */} - {getShortTimeStamp(moment(timestamp))} - - - ); - - return ( - - {imgSrc ? ( - - ) : ( - - - {isLoading || isPending ? ( - - ) : ( - - )} - - {ImageCaption} - - )} - - - { - setStepNo(stepNo - 1); - }} - iconType="arrowLeft" - aria-label="Next" - /> - - - { - setStepNo(stepNo + 1); - }} - iconType="arrowRight" - aria-label="Next" - /> - - - - ); -}; - -const BorderedText = euiStyled(EuiText)` - width: 120px; - text-align: center; - border: 1px solid ${(props) => props.theme.eui.euiColorLightShade}; -`; - -export const NoImageAvailable = () => { - return ( - - - - - - ); -}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts new file mode 100644 index 00000000000000..db9c18e30cfc15 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { PingTimestamp } from './ping_timestamp'; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx new file mode 100644 index 00000000000000..c8acfd48a9913d --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx @@ -0,0 +1,88 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NavButtons, NavButtonsProps } from './nav_buttons'; + +describe('NavButtons', () => { + let defaultProps: NavButtonsProps; + + beforeEach(() => { + defaultProps = { + maxSteps: 3, + stepNumber: 2, + setStepNumber: jest.fn(), + setIsImagePopoverOpen: jest.fn(), + }; + }); + + it('labels prev and next buttons', () => { + const { getByLabelText } = render(); + + expect(getByLabelText('Previous step')); + expect(getByLabelText('Next step')); + }); + + it('increments step number on next click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(3); + }); + }); + + it('decrements step number on prev click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Previous step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(1); + }); + }); + + it('disables `next` button on final step', () => { + defaultProps.stepNumber = 3; + + const { getByLabelText } = render(); + + // getByLabelText('Next step'); + expect(getByLabelText('Next step')).toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).not.toHaveAttribute('disabled'); + }); + + it('disables `prev` button on final step', () => { + defaultProps.stepNumber = 1; + + const { getByLabelText } = render(); + + expect(getByLabelText('Next step')).not.toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).toHaveAttribute('disabled'); + }); + + it('opens popover when mouse enters', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.mouseEnter(nextButton); + + await waitFor(() => { + expect(defaultProps.setIsImagePopoverOpen).toHaveBeenCalledTimes(1); + expect(defaultProps.setIsImagePopoverOpen).toHaveBeenCalledWith(true); + }); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx new file mode 100644 index 00000000000000..1c24caba6a9171 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx @@ -0,0 +1,56 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiButtonIcon, EuiFlexItem, EuiFlexGroup } from '@elastic/eui'; +import React from 'react'; +import { nextAriaLabel, prevAriaLabel } from './translations'; + +export interface NavButtonsProps { + maxSteps?: number; + setIsImagePopoverOpen: React.Dispatch>; + setStepNumber: React.Dispatch>; + stepNumber: number; +} + +export const NavButtons: React.FC = ({ + maxSteps, + setIsImagePopoverOpen, + setStepNumber, + stepNumber, +}) => ( + setIsImagePopoverOpen(true)} + style={{ position: 'absolute', bottom: 0, left: 30 }} + > + + { + setStepNumber(stepNumber - 1); + }} + iconType="arrowLeft" + aria-label={prevAriaLabel} + /> + + + { + setStepNumber(stepNumber + 1); + }} + iconType="arrowRight" + aria-label={nextAriaLabel} + /> + + +); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx new file mode 100644 index 00000000000000..17e679846a66df --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx @@ -0,0 +1,17 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NoImageAvailable } from './no_image_available'; + +describe('NoImageAvailable', () => { + it('renders expected text', () => { + const { getByText } = render(); + + expect(getByText('No image available')); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx new file mode 100644 index 00000000000000..2498e07969f111 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx @@ -0,0 +1,29 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiText } from '@elastic/eui'; +import { FormattedMessage } from '@kbn/i18n/react'; +import React from 'react'; +import { euiStyled } from '../../../../../../../../../src/plugins/kibana_react/common'; + +const BorderedText = euiStyled(EuiText)` + width: 120px; + text-align: center; + border: 1px solid ${(props) => props.theme.eui.euiColorLightShade}; +`; + +export const NoImageAvailable = () => { + return ( + + + + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx new file mode 100644 index 00000000000000..24080e2f4061d0 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx @@ -0,0 +1,44 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NoImageDisplay, NoImageDisplayProps } from './no_image_display'; +import { imageLoadingSpinnerAriaLabel } from './translations'; + +describe('NoImageDisplay', () => { + let defaultProps: NoImageDisplayProps; + beforeEach(() => { + defaultProps = { + imageCaption:
test caption
, + isLoading: false, + isPending: false, + }; + }); + + it('renders a loading spinner for loading state', () => { + defaultProps.isLoading = true; + const { getByText, getByLabelText } = render(); + + expect(getByLabelText(imageLoadingSpinnerAriaLabel)); + expect(getByText('test caption')); + }); + + it('renders a loading spinner for pending state', () => { + defaultProps.isPending = true; + const { getByText, getByLabelText } = render(); + + expect(getByLabelText(imageLoadingSpinnerAriaLabel)); + expect(getByText('test caption')); + }); + + it('renders no image available when not loading or pending', () => { + const { getByText } = render(); + + expect(getByText('No image available')); + expect(getByText('test caption')); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx new file mode 100644 index 00000000000000..185f488d5acd29 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx @@ -0,0 +1,39 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiFlexItem, EuiFlexGroup, EuiLoadingSpinner } from '@elastic/eui'; +import React from 'react'; +import { NoImageAvailable } from './no_image_available'; +import { imageLoadingSpinnerAriaLabel } from './translations'; + +export interface NoImageDisplayProps { + imageCaption: JSX.Element; + isLoading: boolean; + isPending: boolean; +} + +export const NoImageDisplay: React.FC = ({ + imageCaption, + isLoading, + isPending, +}) => { + return ( + + + {isLoading || isPending ? ( + + ) : ( + + )} + + {imageCaption} + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx similarity index 70% rename from x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx rename to x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx index 1baeb8a69d34cc..a934f6fa39b22a 100644 --- a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx @@ -5,16 +5,17 @@ */ import React from 'react'; +import { fireEvent, waitFor } from '@testing-library/react'; import { PingTimestamp } from './ping_timestamp'; -import { mockReduxHooks } from '../../../../lib/helper/test_helpers'; -import { render } from '../../../../lib/helper/rtl_helpers'; -import { Ping } from '../../../../../common/runtime_types/ping'; -import * as observabilityPublic from '../../../../../../observability/public'; +import { mockReduxHooks } from '../../../../../lib/helper/test_helpers'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { Ping } from '../../../../../../common/runtime_types/ping'; +import * as observabilityPublic from '../../../../../../../observability/public'; mockReduxHooks(); -jest.mock('../../../../../../observability/public', () => { - const originalModule = jest.requireActual('../../../../../../observability/public'); +jest.mock('../../../../../../../observability/public', () => { + const originalModule = jest.requireActual('../../../../../../../observability/public'); return { ...originalModule, @@ -92,4 +93,26 @@ describe('Ping Timestamp component', () => { const { container } = render(); expect(container.querySelector('img')?.src).toBe(src); }); + + it('displays popover image when mouse enters img caption, and hides onLeave', async () => { + const src = 'http://sample.com/sampleImageSrc.png'; + jest.spyOn(observabilityPublic, 'useFetcher').mockReturnValue({ + status: FETCH_STATUS.SUCCESS, + data: { src }, + refetch: () => null, + }); + const { getByAltText, getByText, queryByAltText } = render( + + ); + const caption = getByText('Nov 26, 2020 10:28:56 AM'); + fireEvent.mouseEnter(caption); + + const altText = `A larger version of the screenshot for this journey step's thumbnail.`; + + await waitFor(() => getByAltText(altText)); + + fireEvent.mouseLeave(caption); + + await waitFor(() => expect(queryByAltText(altText)).toBeNull()); + }); }); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx new file mode 100644 index 00000000000000..6d605f25f6f683 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx @@ -0,0 +1,120 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { useContext, useEffect, useState } from 'react'; +import useIntersection from 'react-use/lib/useIntersection'; +import styled from 'styled-components'; +import { Ping } from '../../../../../../common/runtime_types/ping'; +import { useFetcher, FETCH_STATUS } from '../../../../../../../observability/public'; +import { getJourneyScreenshot } from '../../../../../state/api/journey'; +import { UptimeSettingsContext } from '../../../../../contexts'; +import { NavButtons } from './nav_buttons'; +import { NoImageDisplay } from './no_image_display'; +import { StepImageCaption } from './step_image_caption'; +import { StepImagePopover } from './step_image_popover'; +import { formatCaptionContent } from './translations'; + +const StepDiv = styled.div` + figure.euiImage { + div.stepArrowsFullScreen { + display: none; + } + } + + figure.euiImage-isFullScreen { + div.stepArrowsFullScreen { + display: flex; + } + } + position: relative; + div.stepArrows { + display: none; + } + :hover { + div.stepArrows { + display: flex; + } + } +`; + +interface Props { + timestamp: string; + ping: Ping; +} + +export const PingTimestamp = ({ timestamp, ping }: Props) => { + const [stepNumber, setStepNumber] = useState(1); + const [isImagePopoverOpen, setIsImagePopoverOpen] = useState(false); + + const [stepImages, setStepImages] = useState([]); + + const intersectionRef = React.useRef(null); + + const { basePath } = useContext(UptimeSettingsContext); + + const imgPath = `${basePath}/api/uptime/journey/screenshot/${ping.monitor.check_group}/${stepNumber}`; + + const intersection = useIntersection(intersectionRef, { + root: null, + rootMargin: '0px', + threshold: 1, + }); + + const { data, status } = useFetcher(() => { + if (intersection && intersection.intersectionRatio === 1 && !stepImages[stepNumber - 1]) + return getJourneyScreenshot(imgPath); + }, [intersection?.intersectionRatio, stepNumber]); + + useEffect(() => { + if (data) { + setStepImages((prevState) => [...prevState, data?.src]); + } + }, [data]); + + const imgSrc = stepImages[stepNumber] || data?.src; + + const captionContent = formatCaptionContent(stepNumber, data?.maxSteps); + + const ImageCaption = ( + + ); + + return ( + setIsImagePopoverOpen(true)} + onMouseLeave={() => setIsImagePopoverOpen(false)} + ref={intersectionRef} + > + {imgSrc ? ( + + ) : ( + + )} + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx new file mode 100644 index 00000000000000..ef1d0cb388a189 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx @@ -0,0 +1,89 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { StepImageCaption, StepImageCaptionProps } from './step_image_caption'; + +describe('StepImageCaption', () => { + let defaultProps: StepImageCaptionProps; + + beforeEach(() => { + defaultProps = { + captionContent: 'test caption content', + imgSrc: 'http://sample.com/sampleImageSrc.png', + maxSteps: 3, + setStepNumber: jest.fn(), + stepNumber: 2, + timestamp: '2020-11-26T15:28:56.896Z', + }; + }); + + it('labels prev and next buttons', () => { + const { getByLabelText } = render(); + + expect(getByLabelText('Previous step')); + expect(getByLabelText('Next step')); + }); + + it('increments step number on next click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(3); + }); + }); + + it('decrements step number on prev click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Previous step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(1); + }); + }); + + it('disables `next` button on final step', () => { + defaultProps.stepNumber = 3; + + const { getByLabelText } = render(); + + // getByLabelText('Next step'); + expect(getByLabelText('Next step')).toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).not.toHaveAttribute('disabled'); + }); + + it('disables `prev` button on final step', () => { + defaultProps.stepNumber = 1; + + const { getByLabelText } = render(); + + expect(getByLabelText('Next step')).not.toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).toHaveAttribute('disabled'); + }); + + it('renders a timestamp', () => { + const { getByText } = render(); + + getByText('Nov 26, 2020 10:28:56 AM'); + }); + + it('renders caption content', () => { + const { getByText } = render(); + + getByText('test caption content'); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx new file mode 100644 index 00000000000000..c5da98bacc431c --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx @@ -0,0 +1,68 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiButtonIcon, EuiFlexGroup, EuiFlexItem, EuiSpacer, EuiText } from '@elastic/eui'; +import React from 'react'; +import moment from 'moment'; +import { nextAriaLabel, prevAriaLabel } from './translations'; +import { getShortTimeStamp } from '../../../../overview/monitor_list/columns/monitor_status_column'; + +export interface StepImageCaptionProps { + captionContent: string; + imgSrc?: string; + maxSteps?: number; + setStepNumber: React.Dispatch>; + stepNumber: number; + timestamp: string; +} + +export const StepImageCaption: React.FC = ({ + captionContent, + imgSrc, + maxSteps, + setStepNumber, + stepNumber, + timestamp, +}) => { + return ( + <> +
+ {imgSrc && ( + + + { + setStepNumber(stepNumber - 1); + }} + iconType="arrowLeft" + aria-label={prevAriaLabel} + /> + + + {captionContent} + + + { + setStepNumber(stepNumber + 1); + }} + iconType="arrowRight" + aria-label={nextAriaLabel} + /> + + + )} +
+ {/* TODO: Add link to details page once it's available */} + {getShortTimeStamp(moment(timestamp))} + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx new file mode 100644 index 00000000000000..184794c1465aa4 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx @@ -0,0 +1,60 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { StepImagePopover, StepImagePopoverProps } from './step_image_popover'; +import { render } from '../../../../../lib/helper/rtl_helpers'; + +describe('StepImagePopover', () => { + let defaultProps: StepImagePopoverProps; + + beforeEach(() => { + defaultProps = { + captionContent: 'test caption', + imageCaption:
test caption element
, + imgSrc: 'http://sample.com/sampleImageSrc.png', + isImagePopoverOpen: false, + }; + }); + + it('opens displays full-size image on click, hides after close is closed', async () => { + const { getByAltText, getByLabelText, queryByLabelText } = render( + + ); + + const closeFullScreenButton = 'Close full screen test caption image'; + + expect(queryByLabelText(closeFullScreenButton)).toBeNull(); + + const caption = getByAltText('test caption'); + fireEvent.click(caption); + + await waitFor(() => { + const closeButton = getByLabelText(closeFullScreenButton); + fireEvent.click(closeButton); + }); + + await waitFor(() => { + expect(queryByLabelText(closeFullScreenButton)).toBeNull(); + }); + }); + + it('shows the popover when `isOpen` is true', () => { + defaultProps.isImagePopoverOpen = true; + + const { getByAltText } = render(); + + expect(getByAltText(`A larger version of the screenshot for this journey step's thumbnail.`)); + }); + + it('renders caption content', () => { + const { getByRole } = render(); + const image = getByRole('img'); + expect(image).toHaveAttribute('alt', 'test caption'); + expect(image).toHaveAttribute('src', 'http://sample.com/sampleImageSrc.png'); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx new file mode 100644 index 00000000000000..fd7b7e6a886bb3 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx @@ -0,0 +1,61 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiImage, EuiPopover } from '@elastic/eui'; +import React from 'react'; +import styled from 'styled-components'; +import { fullSizeImageAlt } from './translations'; + +const POPOVER_IMG_HEIGHT = 360; +const POPOVER_IMG_WIDTH = 640; + +const StepImage = styled(EuiImage)` + &&& { + display: flex; + figcaption { + white-space: nowrap; + align-self: center; + margin-left: 8px; + margin-top: 8px; + text-decoration: none !important; + } + } +`; +export interface StepImagePopoverProps { + captionContent: string; + imageCaption: JSX.Element; + imgSrc: string; + isImagePopoverOpen: boolean; +} + +export const StepImagePopover: React.FC = ({ + captionContent, + imageCaption, + imgSrc, + isImagePopoverOpen, +}) => ( + + } + isOpen={isImagePopoverOpen} + > + + +); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts new file mode 100644 index 00000000000000..ad49143a680572 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts @@ -0,0 +1,35 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { i18n } from '@kbn/i18n'; + +export const prevAriaLabel = i18n.translate('xpack.uptime.synthetics.prevStepButton.airaLabel', { + defaultMessage: 'Previous step', +}); + +export const nextAriaLabel = i18n.translate('xpack.uptime.synthetics.nextStepButton.ariaLabel', { + defaultMessage: 'Next step', +}); + +export const imageLoadingSpinnerAriaLabel = i18n.translate( + 'xpack.uptime.synthetics.imageLoadingSpinner.ariaLabel', + { + defaultMessage: 'An animated spinner indicating the image is loading', + } +); + +export const fullSizeImageAlt = i18n.translate('xpack.uptime.synthetics.thumbnail.fullSize.alt', { + defaultMessage: `A larger version of the screenshot for this journey step's thumbnail.`, +}); + +export const formatCaptionContent = (stepNumber: number, stepName?: number) => + i18n.translate('xpack.uptime.synthetics.pingTimestamp.captionContent', { + defaultMessage: 'Step: {stepNumber} {stepName}', + values: { + stepNumber, + stepName, + }, + }); diff --git a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx index bdc6dbf3f6de22..3d9b646931e7d3 100644 --- a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx @@ -32,7 +32,7 @@ export const StepDetailContainer: React.FC = ({ checkGroup, stepIndex }) useEffect(() => { if (checkGroup) { - dispatch(getJourneySteps({ checkGroup })); + dispatch(getJourneySteps({ checkGroup, syntheticEventTypes: ['step/end'] })); } }, [dispatch, checkGroup]); diff --git a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx index 3efcff196b55fd..716e877c50943c 100644 --- a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx @@ -98,7 +98,7 @@ export const StepScreenshotDisplay: FC = ({ closePopover={() => setIsImagePopoverOpen(false)} isOpen={isImagePopoverOpen} > - { = ({ } ) } - src={imgSrc} + url={imgSrc} style={{ width: POPOVER_IMG_WIDTH, height: POPOVER_IMG_HEIGHT, objectFit: 'contain' }} /> diff --git a/x-pack/plugins/uptime/public/state/actions/journey.ts b/x-pack/plugins/uptime/public/state/actions/journey.ts index 0d35559d97fc38..5931980c569473 100644 --- a/x-pack/plugins/uptime/public/state/actions/journey.ts +++ b/x-pack/plugins/uptime/public/state/actions/journey.ts @@ -9,6 +9,7 @@ import { SyntheticsJourneyApiResponse } from '../../../common/runtime_types'; export interface FetchJourneyStepsParams { checkGroup: string; + syntheticEventTypes?: string[]; } export interface GetJourneyFailPayload { diff --git a/x-pack/plugins/uptime/public/state/api/journey.ts b/x-pack/plugins/uptime/public/state/api/journey.ts index 1aeeb485e481f1..684056b197f938 100644 --- a/x-pack/plugins/uptime/public/state/api/journey.ts +++ b/x-pack/plugins/uptime/public/state/api/journey.ts @@ -16,7 +16,7 @@ export async function fetchJourneySteps( ): Promise { return (await apiService.get( `/api/uptime/journey/${params.checkGroup}`, - undefined, + { syntheticEventTypes: params.syntheticEventTypes }, SyntheticsJourneyApiResponseType )) as SyntheticsJourneyApiResponse; } diff --git a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts new file mode 100644 index 00000000000000..8c432ff6f1e0fe --- /dev/null +++ b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts @@ -0,0 +1,196 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { getJourneySteps, formatSyntheticEvents } from './get_journey_steps'; +import { getUptimeESMockClient } from './helper'; + +describe('getJourneySteps request module', () => { + describe('formatStepTypes', () => { + it('returns default steps if none are provided', () => { + expect(formatSyntheticEvents()).toMatchInlineSnapshot(` + Array [ + "step/end", + "stderr", + "cmd/status", + "step/screenshot", + ] + `); + }); + + it('returns provided step array if isArray', () => { + expect(formatSyntheticEvents(['step/end', 'stderr'])).toMatchInlineSnapshot(` + Array [ + "step/end", + "stderr", + ] + `); + }); + + it('returns provided step string in an array', () => { + expect(formatSyntheticEvents('step/end')).toMatchInlineSnapshot(` + Array [ + "step/end", + ] + `); + }); + }); + + describe('getJourneySteps', () => { + let data: any; + beforeEach(() => { + data = { + body: { + hits: { + hits: [ + { + _id: 'o6myXncBFt2V8m6r6z-r', + _source: { + '@timestamp': '2021-02-01T17:45:19.001Z', + synthetics: { + package_version: '0.0.1-alpha.8', + journey: { + name: 'inline', + id: 'inline', + }, + step: { + name: 'load homepage', + index: 1, + }, + type: 'step/end', + }, + monitor: { + name: 'My Monitor', + id: 'my-monitor', + check_group: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + type: 'browser', + }, + }, + }, + { + _id: 'IjqzXncBn2sjqrYxYoCG', + _source: { + '@timestamp': '2021-02-01T17:45:49.944Z', + synthetics: { + package_version: '0.0.1-alpha.8', + journey: { + name: 'inline', + id: 'inline', + }, + step: { + name: 'hover over products menu', + index: 2, + }, + type: 'step/end', + }, + monitor: { + name: 'My Monitor', + timespan: { + lt: '2021-02-01T17:46:49.945Z', + gte: '2021-02-01T17:45:49.945Z', + }, + id: 'my-monitor', + check_group: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + type: 'browser', + }, + }, + }, + ], + }, + }, + }; + }); + + it('formats ES result', async () => { + const { esClient: mockEsClient, uptimeEsClient } = getUptimeESMockClient(); + + mockEsClient.search.mockResolvedValueOnce(data as any); + const result: any = await getJourneySteps({ + uptimeEsClient, + checkGroup: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + }); + expect(mockEsClient.search).toHaveBeenCalledTimes(1); + const call: any = mockEsClient.search.mock.calls[0][0]; + + // check that default `synthetics.type` value is supplied, + expect(call.body.query.bool.filter[0]).toMatchInlineSnapshot(` + Object { + "terms": Object { + "synthetics.type": Array [ + "step/end", + "stderr", + "cmd/status", + "step/screenshot", + ], + }, + } + `); + + // given check group is used for the terms filter + expect(call.body.query.bool.filter[1]).toMatchInlineSnapshot(` + Object { + "term": Object { + "monitor.check_group": "2bf952dc-64b5-11eb-8b3b-42010a84000d", + }, + } + `); + + // should sort by step index, then timestamp + expect(call.body.sort).toMatchInlineSnapshot(` + Array [ + Object { + "synthetics.step.index": Object { + "order": "asc", + }, + }, + Object { + "@timestamp": Object { + "order": "asc", + }, + }, + ] + `); + + expect(result).toHaveLength(2); + // `getJourneySteps` is responsible for formatting these fields, so we need to check them + result.forEach((step: any) => { + expect(['2021-02-01T17:45:19.001Z', '2021-02-01T17:45:49.944Z']).toContain(step.timestamp); + expect(['o6myXncBFt2V8m6r6z-r', 'IjqzXncBn2sjqrYxYoCG']).toContain(step.docId); + expect(step.synthetics.screenshotExists).toBeDefined(); + }); + }); + + it('notes screenshot exists when a document of type step/screenshot is included', async () => { + const { esClient: mockEsClient, uptimeEsClient } = getUptimeESMockClient(); + + data.body.hits.hits[0]._source.synthetics.type = 'step/screenshot'; + data.body.hits.hits[0]._source.synthetics.step.index = 2; + mockEsClient.search.mockResolvedValueOnce(data as any); + + const result: any = await getJourneySteps({ + uptimeEsClient, + checkGroup: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + syntheticEventTypes: ['stderr', 'step/end'], + }); + + const call: any = mockEsClient.search.mock.calls[0][0]; + + // assert that filters for only the provided step types are used + expect(call.body.query.bool.filter[0]).toMatchInlineSnapshot(` + Object { + "terms": Object { + "synthetics.type": Array [ + "stderr", + "step/end", + ], + }, + } + `); + + expect(result).toHaveLength(1); + expect(result[0].synthetics.screenshotExists).toBe(true); + }); + }); +}); diff --git a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts index c330e1b66fe93c..60d2a97c99f7d1 100644 --- a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts +++ b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts @@ -9,11 +9,23 @@ import { Ping } from '../../../common/runtime_types'; interface GetJourneyStepsParams { checkGroup: string; + syntheticEventTypes?: string | string[]; } +const defaultEventTypes = ['step/end', 'stderr', 'cmd/status', 'step/screenshot']; + +export const formatSyntheticEvents = (eventTypes?: string | string[]) => { + if (!eventTypes) { + return defaultEventTypes; + } else { + return Array.isArray(eventTypes) ? eventTypes : [eventTypes]; + } +}; + export const getJourneySteps: UMElasticsearchQueryFn = async ({ uptimeEsClient, checkGroup, + syntheticEventTypes, }) => { const params = { query: { @@ -21,7 +33,7 @@ export const getJourneySteps: UMElasticsearchQueryFn checkGroup: schema.string(), _debug: schema.maybe(schema.boolean()), }), + query: schema.object({ + // provides a filter for the types of synthetic events to include + // when fetching a journey's data + syntheticEventTypes: schema.maybe( + schema.oneOf([schema.arrayOf(schema.string()), schema.string()]) + ), + }), }, handler: async ({ uptimeEsClient, request }): Promise => { const { checkGroup } = request.params; + const { syntheticEventTypes } = request.query; const result = await libs.requests.getJourneySteps({ uptimeEsClient, checkGroup, + syntheticEventTypes, }); const details = await libs.requests.getJourneyDetails({ diff --git a/x-pack/test/accessibility/apps/home.ts b/x-pack/test/accessibility/apps/home.ts index 280769bc09bc9a..6857383b7db5bd 100644 --- a/x-pack/test/accessibility/apps/home.ts +++ b/x-pack/test/accessibility/apps/home.ts @@ -9,12 +9,10 @@ import { FtrProviderContext } from '../ftr_provider_context'; export default function ({ getService, getPageObjects }: FtrProviderContext) { const PageObjects = getPageObjects(['common', 'home']); const a11y = getService('a11y'); - const retry = getService('retry'); - const globalNav = getService('globalNav'); const testSubjects = getService('testSubjects'); + const find = getService('find'); - // FLAKY: https://github.com/elastic/kibana/issues/80929 - describe.skip('Kibana Home', () => { + describe('Kibana Home', () => { before(async () => { await PageObjects.common.navigateToApp('home'); }); @@ -23,64 +21,74 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { await a11y.testAppSnapshot(); }); - it('all plugins view page meets a11y requirements', async () => { - await PageObjects.home.clickAllKibanaPlugins(); + it('Kibana overview page meets a11y requirements ', async () => { + await testSubjects.click('homSolutionPanel homSolutionPanel_kibana'); await a11y.testAppSnapshot(); }); - it('visualize & explore details tab meets a11y requirements', async () => { - await PageObjects.home.clickVisualizeExplorePlugins(); + it('toggle side nav meets a11y requirements', async () => { + await testSubjects.click('toggleNavButton'); await a11y.testAppSnapshot(); }); - it('administrative detail tab meets a11y requirements', async () => { - await PageObjects.home.clickAdminPlugin(); + it('Enterprise search overview page meets a11y requirements ', async () => { + await testSubjects.click('homeLink'); + await testSubjects.click('homSolutionPanel homSolutionPanel_enterpriseSearch'); await a11y.testAppSnapshot(); }); - it('navigating to console app from administration tab meets a11y requirements', async () => { - await PageObjects.home.clickOnConsole(); - // wait till dev tools app is loaded (lazy loading the bundle) - await retry.waitFor( - 'switched to dev tools', - async () => (await globalNav.getLastBreadcrumb()) === 'Dev Tools' - ); + it('Observability overview page meets a11y requirements ', async () => { + await testSubjects.click('toggleNavButton'); + await testSubjects.click('homeLink'); + await testSubjects.click('homSolutionPanel homSolutionPanel_observability'); await a11y.testAppSnapshot(); }); - it('navigating back to home page from console meets a11y requirements', async () => { - await PageObjects.home.clickOnLogo(); + it('Security overview page meets a11y requirements ', async () => { + await testSubjects.click('toggleNavButton'); + await testSubjects.click('homeLink'); + await testSubjects.click('homSolutionPanel homSolutionPanel_securitySolution'); await a11y.testAppSnapshot(); }); - it('click on Add logs panel to open all log examples page meets a11y requirements ', async () => { - await PageObjects.home.clickOnAddData(); + it('Add data page meets a11y requirements ', async () => { + await testSubjects.click('toggleNavButton'); + await testSubjects.click('homeLink'); + await testSubjects.click('homeAddData'); await a11y.testAppSnapshot(); }); - it('click on ActiveMQ logs panel to open tutorial meets a11y requirements', async () => { - await PageObjects.home.clickOnLogsTutorial(); + it('Sample data page meets a11y requirements ', async () => { + await testSubjects.click('homeTab-sampleData'); await a11y.testAppSnapshot(); }); - it('click on cloud tutorial meets a11y requirements', async () => { - await PageObjects.home.clickOnCloudTutorial(); + it('click on Add logs panel to open all log examples page meets a11y requirements ', async () => { + await testSubjects.click('sampleDataSetCardlogs'); + await a11y.testAppSnapshot(); + }); + + it('click on ActiveMQ logs panel to open tutorial meets a11y requirements', async () => { + await testSubjects.click('homeTab-all'); + await testSubjects.click('homeSynopsisLinkactivemqlogs'); await a11y.testAppSnapshot(); }); - it('click on side nav to see all the side nav menu', async () => { - await PageObjects.home.clickOnLogo(); - await PageObjects.home.clickOnToggleNavButton(); + it('click on cloud tutorial meets a11y requirements', async () => { + await testSubjects.click('onCloudTutorial'); await a11y.testAppSnapshot(); }); it('Dock the side nav', async () => { + await testSubjects.click('toggleNavButton'); await PageObjects.home.dockTheSideNav(); await a11y.testAppSnapshot(); }); it('click on collapse on observability in side nav to test a11y of collapse button', async () => { - await PageObjects.home.collapseObservabibilitySideNav(); + await find.clickByCssSelector( + '[data-test-subj="collapsibleNavGroup-observability"] .euiCollapsibleNavGroup__title' + ); await a11y.testAppSnapshot(); }); @@ -91,8 +99,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { }); it('passes with searchbox open', async () => { - await PageObjects.common.navigateToApp('home'); - await testSubjects.click('header-search'); + await testSubjects.click('nav-search-popover'); await a11y.testAppSnapshot(); }); }); diff --git a/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts b/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts index 8ed979a1711694..e1502b496f77ed 100644 --- a/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts +++ b/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts @@ -10,7 +10,8 @@ import { setupSpacesAndUsers, tearDown } from '..'; // eslint-disable-next-line import/no-default-export export default function alertingTests({ loadTestFile, getService }: FtrProviderContext) { describe('Alerts', () => { - describe('legacy alerts', () => { + // FLAKY: https://github.com/elastic/kibana/issues/86952 + describe.skip('legacy alerts', () => { before(async () => { await setupSpacesAndUsers(getService); }); diff --git a/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts b/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts index 8bf0a2a0f034fd..7e25707c10c78a 100644 --- a/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts +++ b/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/create.ts @@ -111,6 +111,75 @@ export default function createAlertTests({ getService }: FtrProviderContext) { }); }); + it('should allow providing custom saved object ids (uuid v1)', async () => { + const customId = '09570bb0-6299-11eb-8fde-9fe5ce6ea450'; + const response = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()); + + expect(response.status).to.eql(200); + objectRemover.add(Spaces.space1.id, response.body.id, 'alert', 'alerts'); + expect(response.body.id).to.eql(customId); + // Ensure AAD isn't broken + await checkAAD({ + supertest, + spaceId: Spaces.space1.id, + type: 'alert', + id: customId, + }); + }); + + it('should allow providing custom saved object ids (uuid v4)', async () => { + const customId = 'b3bc6d83-3192-4ffd-9702-ad4fb88617ba'; + const response = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()); + + expect(response.status).to.eql(200); + objectRemover.add(Spaces.space1.id, response.body.id, 'alert', 'alerts'); + expect(response.body.id).to.eql(customId); + // Ensure AAD isn't broken + await checkAAD({ + supertest, + spaceId: Spaces.space1.id, + type: 'alert', + id: customId, + }); + }); + + it('should not allow providing simple custom ids (non uuid)', async () => { + const customId = '1'; + const response = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()); + + expect(response.status).to.eql(400); + expect(response.body).to.eql({ + statusCode: 400, + error: 'Bad Request', + message: + 'Predefined IDs are not allowed for saved objects with encrypted attributes unless the ID is a UUID.: Bad Request', + }); + }); + + it('should return 409 when document with id already exists', async () => { + const customId = '5031f8f0-629a-11eb-b500-d1931a8e5df7'; + const createdAlertResponse = await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()) + .expect(200); + objectRemover.add(Spaces.space1.id, createdAlertResponse.body.id, 'alert', 'alerts'); + await supertest + .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert/${customId}`) + .set('kbn-xsrf', 'foo') + .send(getTestAlertData()) + .expect(409); + }); + it('should handle create alert request appropriately when consumer is unknown', async () => { const response = await supertest .post(`${getUrlPrefix(Spaces.space1.id)}/api/alerts/alert`) diff --git a/x-pack/test/api_integration/apis/maps/migrations.js b/x-pack/test/api_integration/apis/maps/migrations.js index b634e7117e6076..9f9082c959ca52 100644 --- a/x-pack/test/api_integration/apis/maps/migrations.js +++ b/x-pack/test/api_integration/apis/maps/migrations.js @@ -41,7 +41,7 @@ export default function ({ getService }) { type: 'index-pattern', }, ]); - expect(resp.body.migrationVersion).to.eql({ map: '7.10.0' }); + expect(resp.body.migrationVersion).to.eql({ map: '7.12.0' }); expect(resp.body.attributes.layerListJSON.includes('indexPatternRefName')).to.be(true); }); }); diff --git a/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts b/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts index 574ff6dd615adc..a43f51a1655e57 100644 --- a/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts +++ b/x-pack/test/fleet_api_integration/apis/epm/data_stream.ts @@ -12,8 +12,6 @@ export default function (providerContext: FtrProviderContext) { const { getService } = providerContext; const supertest = getService('supertest'); const es = getService('es'); - const dockerServers = getService('dockerServers'); - const server = dockerServers.get('registry'); const pkgName = 'datastreams'; const pkgVersion = '0.1.0'; const pkgUpdateVersion = '0.2.0'; @@ -21,6 +19,7 @@ export default function (providerContext: FtrProviderContext) { const pkgUpdateKey = `${pkgName}-${pkgUpdateVersion}`; const logsTemplateName = `logs-${pkgName}.test_logs`; const metricsTemplateName = `metrics-${pkgName}.test_metrics`; + const namespaces = ['default', 'foo', 'bar']; const uninstallPackage = async (pkg: string) => { await supertest.delete(`/api/fleet/epm/packages/${pkg}`).set('kbn-xsrf', 'xxxx'); @@ -35,86 +34,105 @@ export default function (providerContext: FtrProviderContext) { describe('datastreams', async () => { skipIfNoDockerRegistry(providerContext); + beforeEach(async () => { await installPackage(pkgKey); - await es.transport.request({ - method: 'POST', - path: `/${logsTemplateName}-default/_doc`, - body: { - '@timestamp': '2015-01-01', - logs_test_name: 'test', - data_stream: { - dataset: `${pkgName}.test_logs`, - namespace: 'default', - type: 'logs', - }, - }, - }); - await es.transport.request({ - method: 'POST', - path: `/${metricsTemplateName}-default/_doc`, - body: { - '@timestamp': '2015-01-01', - logs_test_name: 'test', - data_stream: { - dataset: `${pkgName}.test_metrics`, - namespace: 'default', - type: 'metrics', - }, - }, - }); + await Promise.all( + namespaces.map(async (namespace) => { + const createLogsRequest = es.transport.request({ + method: 'POST', + path: `/${logsTemplateName}-${namespace}/_doc`, + body: { + '@timestamp': '2015-01-01', + logs_test_name: 'test', + data_stream: { + dataset: `${pkgName}.test_logs`, + namespace, + type: 'logs', + }, + }, + }); + const createMetricsRequest = es.transport.request({ + method: 'POST', + path: `/${metricsTemplateName}-${namespace}/_doc`, + body: { + '@timestamp': '2015-01-01', + logs_test_name: 'test', + data_stream: { + dataset: `${pkgName}.test_metrics`, + namespace, + type: 'metrics', + }, + }, + }); + return Promise.all([createLogsRequest, createMetricsRequest]); + }) + ); }); + afterEach(async () => { - if (!server.enabled) return; - await es.transport.request({ - method: 'DELETE', - path: `/_data_stream/${logsTemplateName}-default`, - }); - await es.transport.request({ - method: 'DELETE', - path: `/_data_stream/${metricsTemplateName}-default`, - }); + await Promise.all( + namespaces.map(async (namespace) => { + const deleteLogsRequest = es.transport.request({ + method: 'DELETE', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + const deleteMetricsRequest = es.transport.request({ + method: 'DELETE', + path: `/_data_stream/${metricsTemplateName}-${namespace}`, + }); + return Promise.all([deleteLogsRequest, deleteMetricsRequest]); + }) + ); await uninstallPackage(pkgKey); await uninstallPackage(pkgUpdateKey); }); + it('should list the logs and metrics datastream', async function () { - const resLogsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${logsTemplateName}-default`, - }); - const resMetricsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${metricsTemplateName}-default`, + namespaces.forEach(async (namespace) => { + const resLogsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + const resMetricsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${metricsTemplateName}-${namespace}`, + }); + expect(resLogsDatastream.body.data_streams.length).equal(1); + expect(resLogsDatastream.body.data_streams[0].indices.length).equal(1); + expect(resMetricsDatastream.body.data_streams.length).equal(1); + expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); - expect(resLogsDatastream.body.data_streams.length).equal(1); - expect(resLogsDatastream.body.data_streams[0].indices.length).equal(1); - expect(resMetricsDatastream.body.data_streams.length).equal(1); - expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); it('after update, it should have rolled over logs datastream because mappings are not compatible and not metrics', async function () { await installPackage(pkgUpdateKey); - const resLogsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${logsTemplateName}-default`, - }); - const resMetricsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${metricsTemplateName}-default`, + namespaces.forEach(async (namespace) => { + const resLogsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + const resMetricsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${metricsTemplateName}-${namespace}`, + }); + expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); + expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); - expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); - expect(resMetricsDatastream.body.data_streams[0].indices.length).equal(1); }); + it('should be able to upgrade a package after a rollover', async function () { - await es.transport.request({ - method: 'POST', - path: `/${logsTemplateName}-default/_rollover`, - }); - const resLogsDatastream = await es.transport.request({ - method: 'GET', - path: `/_data_stream/${logsTemplateName}-default`, + namespaces.forEach(async (namespace) => { + await es.transport.request({ + method: 'POST', + path: `/${logsTemplateName}-${namespace}/_rollover`, + }); + const resLogsDatastream = await es.transport.request({ + method: 'GET', + path: `/_data_stream/${logsTemplateName}-${namespace}`, + }); + expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); }); - expect(resLogsDatastream.body.data_streams[0].indices.length).equal(2); await installPackage(pkgUpdateKey); }); }); diff --git a/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts b/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts new file mode 100644 index 00000000000000..0a153aecec323e --- /dev/null +++ b/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts @@ -0,0 +1,72 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import expect from '@kbn/expect'; +import { FtrProviderContext } from '../../ftr_provider_context'; + +export default function ({ getPageObjects, getService }: FtrProviderContext) { + const PageObjects = getPageObjects(['common', 'dashboard', 'visualize', 'lens']); + + const find = getService('find'); + const esArchiver = getService('esArchiver'); + const dashboardPanelActions = getService('dashboardPanelActions'); + const dashboardVisualizations = getService('dashboardVisualizations'); + + describe('dashboard lens by value', function () { + before(async () => { + await esArchiver.loadIfNeeded('logstash_functional'); + await esArchiver.loadIfNeeded('lens/basic'); + await PageObjects.common.navigateToApp('dashboard'); + await PageObjects.dashboard.preserveCrossAppState(); + await PageObjects.dashboard.clickNewDashboard(); + }); + + it('can add a lens panel by value', async () => { + await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); + await PageObjects.lens.createAndAddLensFromDashboard({}); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(1); + }); + + it('edits to a by value lens panel are properly applied', async () => { + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.switchToVisualization('donut'); + await PageObjects.lens.saveAndReturn(); + await PageObjects.dashboard.waitForRenderComplete(); + + const pieExists = await find.existsByCssSelector('.lnsPieExpression__container'); + expect(pieExists).to.be(true); + }); + + it('editing and saving a lens by value panel retains number of panels', async () => { + const originalPanelCount = await PageObjects.dashboard.getPanelCount(); + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.switchToVisualization('treemap'); + await PageObjects.lens.saveAndReturn(); + await PageObjects.dashboard.waitForRenderComplete(); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(originalPanelCount); + }); + + it('updates panel on dashboard when a by value panel is saved to library', async () => { + const newTitle = 'look out library, here I come!'; + const originalPanelCount = await PageObjects.dashboard.getPanelCount(); + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.save(newTitle, false, true); + await PageObjects.dashboard.waitForRenderComplete(); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(originalPanelCount); + const titles = await PageObjects.dashboard.getPanelTitles(); + expect(titles.indexOf(newTitle)).to.not.be(-1); + }); + }); +} diff --git a/x-pack/test/functional/apps/dashboard/index.ts b/x-pack/test/functional/apps/dashboard/index.ts index 1ba87f89762a1b..d6c0c4394e24a4 100644 --- a/x-pack/test/functional/apps/dashboard/index.ts +++ b/x-pack/test/functional/apps/dashboard/index.ts @@ -15,5 +15,6 @@ export default function ({ loadTestFile }: FtrProviderContext) { loadTestFile(require.resolve('./drilldowns')); loadTestFile(require.resolve('./sync_colors')); loadTestFile(require.resolve('./_async_dashboard')); + loadTestFile(require.resolve('./dashboard_lens_by_value')); }); } diff --git a/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js b/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js index bd35374643e9b8..9408bee7dc8685 100644 --- a/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js +++ b/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js @@ -7,7 +7,6 @@ import expect from '@kbn/expect'; export default function ({ getPageObjects, getService }) { - const log = getService('log'); const testSubjects = getService('testSubjects'); const esArchiver = getService('esArchiver'); const dashboardVisualizations = getService('dashboardVisualizations'); @@ -27,40 +26,12 @@ export default function ({ getPageObjects, getService }) { await PageObjects.dashboard.gotoDashboardLandingPage(); }); - async function createAndAddLens(title, saveAsNew = false, redirectToOrigin = true) { - log.debug(`createAndAddLens(${title})`); - const inViewMode = await PageObjects.dashboard.getIsInViewMode(); - if (inViewMode) { - await PageObjects.dashboard.switchToEditMode(); - } - await PageObjects.visualize.clickLensWidget(); - await PageObjects.lens.goToTimeRange(); - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension', - operation: 'date_histogram', - field: '@timestamp', - }); - - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension', - operation: 'avg', - field: 'bytes', - }); - - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_splitDimensionPanel > lns-empty-dimension', - operation: 'terms', - field: 'ip', - }); - await PageObjects.lens.save(title, saveAsNew, redirectToOrigin); - } - it('adds Lens visualization to empty dashboard', async () => { const title = 'Dashboard Test Lens'; await testSubjects.exists('addVisualizationButton'); await testSubjects.click('addVisualizationButton'); await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); - await createAndAddLens(title); + await PageObjects.lens.createAndAddLensFromDashboard({ title, redirectToOrigin: true }); await PageObjects.dashboard.waitForRenderComplete(); await testSubjects.exists(`embeddablePanelHeading-${title}`); }); @@ -118,7 +89,7 @@ export default function ({ getPageObjects, getService }) { await testSubjects.exists('dashboardAddNewPanelButton'); await testSubjects.click('dashboardAddNewPanelButton'); await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); - await createAndAddLens(title, false, false); + await PageObjects.lens.createAndAddLensFromDashboard({ title }); await PageObjects.lens.notLinkedToOriginatingApp(); await PageObjects.common.navigateToApp('dashboard'); }); diff --git a/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts b/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts index 532de930bc1a16..91ca0e6f32fd8a 100644 --- a/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts +++ b/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts @@ -13,7 +13,8 @@ export default function ({ getService }: FtrProviderContext) { const esArchiver = getService('esArchiver'); const ml = getService('ml'); - describe('jobs cloning supported by UI form', function () { + // Failing ES promotion, see https://github.com/elastic/kibana/issues/89980 + describe.skip('jobs cloning supported by UI form', function () { const testDataList: Array<{ suiteTitle: string; archive: string; diff --git a/x-pack/test/functional/page_objects/lens_page.ts b/x-pack/test/functional/page_objects/lens_page.ts index 17f9fb036129a7..292e91866d6903 100644 --- a/x-pack/test/functional/page_objects/lens_page.ts +++ b/x-pack/test/functional/page_objects/lens_page.ts @@ -16,7 +16,7 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont const find = getService('find'); const comboBox = getService('comboBox'); const browser = getService('browser'); - const PageObjects = getPageObjects(['header', 'timePicker', 'common', 'visualize']); + const PageObjects = getPageObjects(['header', 'timePicker', 'common', 'visualize', 'dashboard']); return logWrapper('lensPage', log, { /** @@ -586,5 +586,49 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont // TODO: target dimensionTrigger color element after merging https://github.com/elastic/kibana/pull/76871 await testSubjects.getAttribute('colorPickerAnchor', color); }, + + /** + * Creates and saves a lens visualization from a dashboard + * + * @param title - title for the new lens. If left undefined, the panel will be created by value + * @param redirectToOrigin - whether to redirect back to the dashboard after saving the panel + */ + async createAndAddLensFromDashboard({ + title, + redirectToOrigin, + }: { + title?: string; + redirectToOrigin?: boolean; + }) { + log.debug(`createAndAddLens${title}`); + const inViewMode = await PageObjects.dashboard.getIsInViewMode(); + if (inViewMode) { + await PageObjects.dashboard.switchToEditMode(); + } + await PageObjects.visualize.clickLensWidget(); + await this.goToTimeRange(); + await this.configureDimension({ + dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension', + operation: 'date_histogram', + field: '@timestamp', + }); + + await this.configureDimension({ + dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension', + operation: 'avg', + field: 'bytes', + }); + + await this.configureDimension({ + dimension: 'lnsXY_splitDimensionPanel > lns-empty-dimension', + operation: 'terms', + field: 'ip', + }); + if (title) { + await this.save(title, false, redirectToOrigin); + } else { + await this.saveAndReturn(); + } + }, }); } diff --git a/x-pack/test/load/runner.ts b/x-pack/test/load/runner.ts index e9a1cadfddc558..52448a6b32a998 100644 --- a/x-pack/test/load/runner.ts +++ b/x-pack/test/load/runner.ts @@ -7,23 +7,57 @@ import { withProcRunner } from '@kbn/dev-utils'; import { resolve } from 'path'; import { REPO_ROOT } from '@kbn/utils'; +import Fs from 'fs'; +import { createFlagError } from '@kbn/dev-utils'; import { FtrProviderContext } from './../functional/ftr_provider_context'; +const baseSimulationPath = 'src/test/scala/org/kibanaLoadTest/simulation'; +const simulationPackage = 'org.kibanaLoadTest.simulation'; +const simulationFIleExtension = '.scala'; +const gatlingProjectRootPath: string = + process.env.GATLING_PROJECT_PATH || resolve(REPO_ROOT, '../kibana-load-testing'); +const simulationEntry: string = process.env.GATLING_SIMULATIONS || 'DemoJourney'; + +if (!Fs.existsSync(gatlingProjectRootPath)) { + throw createFlagError( + `Incorrect path to load testing project: '${gatlingProjectRootPath}'\n + Clone 'elastic/kibana-load-testing' and set path using 'GATLING_PROJECT_PATH' env var` + ); +} + +const dropEmptyLines = (s: string) => s.split(',').filter((i) => i.length > 0); +const simulationClasses = dropEmptyLines(simulationEntry); +const simulationsRootPath = resolve(gatlingProjectRootPath, baseSimulationPath); + +simulationClasses.map((className) => { + const simulationClassPath = resolve( + simulationsRootPath, + className.replace('.', '/') + simulationFIleExtension + ); + if (!Fs.existsSync(simulationClassPath)) { + throw createFlagError(`Simulation class is not found: '${simulationClassPath}'`); + } +}); + +/** + * + * GatlingTestRunner is used to run load simulation against local Kibana instance + * + * Use GATLING_SIMULATIONS to pass comma-separated class names + * Use GATLING_PROJECT_PATH to override path to 'kibana-load-testing' project + */ export async function GatlingTestRunner({ getService }: FtrProviderContext) { const log = getService('log'); - const gatlingProjectRootPath = resolve(REPO_ROOT, '../kibana-load-testing'); await withProcRunner(log, async (procs) => { - await procs.run('gatling', { + await procs.run('mvn: clean compile', { cmd: 'mvn', args: [ - 'clean', - '-q', + '-Dmaven.wagon.http.retryHandler.count=3', '-Dmaven.test.failure.ignore=true', - 'compile', - 'gatling:test', '-q', - '-Dgatling.simulationClass=org.kibanaLoadTest.simulation.DemoJourney', + 'clean', + 'compile', ], cwd: gatlingProjectRootPath, env: { @@ -31,5 +65,20 @@ export async function GatlingTestRunner({ getService }: FtrProviderContext) { }, wait: true, }); + for (const simulationClass of simulationClasses) { + await procs.run('gatling: test', { + cmd: 'mvn', + args: [ + 'gatling:test', + '-q', + `-Dgatling.simulationClass=${simulationPackage}.${simulationClass}`, + ], + cwd: gatlingProjectRootPath, + env: { + ...process.env, + }, + wait: true, + }); + } }); } diff --git a/x-pack/test/tsconfig.json b/x-pack/test/tsconfig.json index 5232af0dd304b2..a9d76eea80d8fd 100644 --- a/x-pack/test/tsconfig.json +++ b/x-pack/test/tsconfig.json @@ -24,6 +24,7 @@ { "path": "../../src/plugins/kibana_react/tsconfig.json" }, { "path": "../../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../../src/plugins/kibana_utils/tsconfig.json" }, + { "path": "../../src/plugins/legacy_export/tsconfig.json" }, { "path": "../../src/plugins/navigation/tsconfig.json" }, { "path": "../../src/plugins/newsfeed/tsconfig.json" }, { "path": "../../src/plugins/saved_objects/tsconfig.json" }, @@ -35,9 +36,10 @@ { "path": "../../src/plugins/usage_collection/tsconfig.json" }, { "path": "../../src/plugins/ui_actions/tsconfig.json" }, { "path": "../../src/plugins/url_forwarding/tsconfig.json" }, + { "path": "../../src/plugins/index_pattern_management/tsconfig.json" }, - { "path": "../plugins/actions/tsconfig.json"}, - { "path": "../plugins/alerts/tsconfig.json"}, + { "path": "../plugins/actions/tsconfig.json" }, + { "path": "../plugins/alerts/tsconfig.json" }, { "path": "../plugins/console_extensions/tsconfig.json" }, { "path": "../plugins/data_enhanced/tsconfig.json" }, { "path": "../plugins/enterprise_search/tsconfig.json" }, @@ -60,7 +62,10 @@ { "path": "../plugins/cloud/tsconfig.json" }, { "path": "../plugins/saved_objects_tagging/tsconfig.json" }, { "path": "../plugins/global_search_bar/tsconfig.json" }, + { "path": "../plugins/ingest_pipelines/tsconfig.json" }, { "path": "../plugins/license_management/tsconfig.json" }, + { "path": "../plugins/snapshot_restore/tsconfig.json" }, + { "path": "../plugins/grokdebugger/tsconfig.json" }, { "path": "../plugins/painless_lab/tsconfig.json" }, { "path": "../plugins/watcher/tsconfig.json" } ] diff --git a/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js b/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js index f57d2184d4b8db..c70b90b384d139 100644 --- a/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js +++ b/x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js @@ -71,6 +71,8 @@ export default function ({ getService }) { expect(indexSummary[newIndexName]).to.be.an('object'); // The original index name is aliased to the new one expect(indexSummary[newIndexName].aliases.dummydata).to.be.an('object'); + // Verify mappings exist on new index + expect(indexSummary[newIndexName].mappings.properties).to.be.an('object'); // The number of documents in the new index matches what we expect expect((await es.count({ index: lastState.newIndexName })).body.count).to.be(3); diff --git a/x-pack/tsconfig.json b/x-pack/tsconfig.json index 1be6b5cf84cda3..7f64a552a51699 100644 --- a/x-pack/tsconfig.json +++ b/x-pack/tsconfig.json @@ -40,9 +40,12 @@ "plugins/cloud/**/*", "plugins/saved_objects_tagging/**/*", "plugins/global_search_bar/**/*", + "plugins/ingest_pipelines/**/*", "plugins/license_management/**/*", + "plugins/snapshot_restore/**/*", "plugins/painless_lab/**/*", "plugins/watcher/**/*", + "plugins/grokdebugger/**/*", "test/**/*" ], "compilerOptions": { @@ -67,6 +70,7 @@ { "path": "../src/plugins/kibana_react/tsconfig.json" }, { "path": "../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../src/plugins/kibana_utils/tsconfig.json" }, + { "path": "../src/plugins/legacy_export/tsconfig.json" }, { "path": "../src/plugins/management/tsconfig.json" }, { "path": "../src/plugins/navigation/tsconfig.json" }, { "path": "../src/plugins/newsfeed/tsconfig.json" }, @@ -82,6 +86,7 @@ { "path": "../src/plugins/ui_actions/tsconfig.json" }, { "path": "../src/plugins/url_forwarding/tsconfig.json" }, { "path": "../src/plugins/usage_collection/tsconfig.json" }, + { "path": "../src/plugins/index_pattern_management/tsconfig.json" }, { "path": "./plugins/actions/tsconfig.json"}, { "path": "./plugins/alerts/tsconfig.json"}, { "path": "./plugins/beats_management/tsconfig.json" }, @@ -110,12 +115,19 @@ { "path": "./plugins/searchprofiler/tsconfig.json" }, { "path": "./plugins/security/tsconfig.json" }, { "path": "./plugins/spaces/tsconfig.json" }, - { "path": "./plugins/stack_alerts/tsconfig.json"}, + { "path": "./plugins/stack_alerts/tsconfig.json" }, { "path": "./plugins/task_manager/tsconfig.json" }, { "path": "./plugins/telemetry_collection_xpack/tsconfig.json" }, { "path": "./plugins/translations/tsconfig.json" }, { "path": "./plugins/triggers_actions_ui/tsconfig.json"}, + { "path": "./plugins/stack_alerts/tsconfig.json"}, + { "path": "./plugins/snapshot_restore/tsconfig.json" }, + { "path": "./plugins/grokdebugger/tsconfig.json" }, + { "path": "./plugins/ingest_pipelines/tsconfig.json"}, + { "path": "./plugins/license_management/tsconfig.json" }, + { "path": "./plugins/painless_lab/tsconfig.json" }, + { "path": "./plugins/triggers_actions_ui/tsconfig.json" }, { "path": "./plugins/ui_actions_enhanced/tsconfig.json" }, - { "path": "./plugins/watcher/tsconfig.json" }, + { "path": "./plugins/watcher/tsconfig.json" } ] } diff --git a/x-pack/tsconfig.refs.json b/x-pack/tsconfig.refs.json index ed209cd2415861..43a488e8727cc0 100644 --- a/x-pack/tsconfig.refs.json +++ b/x-pack/tsconfig.refs.json @@ -36,6 +36,19 @@ { "path": "./plugins/telemetry_collection_xpack/tsconfig.json" }, { "path": "./plugins/translations/tsconfig.json" }, { "path": "./plugins/triggers_actions_ui/tsconfig.json"}, + { "path": "./plugins/spaces/tsconfig.json" }, + { "path": "./plugins/security/tsconfig.json" }, + { "path": "./plugins/stack_alerts/tsconfig.json"}, + { "path": "./plugins/encrypted_saved_objects/tsconfig.json" }, + { "path": "./plugins/beats_management/tsconfig.json" }, + { "path": "./plugins/cloud/tsconfig.json" }, + { "path": "./plugins/saved_objects_tagging/tsconfig.json" }, + { "path": "./plugins/global_search_bar/tsconfig.json" }, + { "path": "./plugins/snapshot_restore/tsconfig.json" }, + { "path": "./plugins/grokdebugger/tsconfig.json" }, + { "path": "./plugins/ingest_pipelines/tsconfig.json" }, + { "path": "./plugins/license_management/tsconfig.json" }, + { "path": "./plugins/painless_lab/tsconfig.json" }, { "path": "./plugins/ui_actions_enhanced/tsconfig.json" }, { "path": "./plugins/watcher/tsconfig.json" } ]