[Logs UI] Reimplement log source configuration routes in plain HTTP+JSON #64021

weltenwort · 2020-04-20T23:43:11Z

Summary

This reimplements the parts of the source configuration GraphQL API relevant for logs in HTTP+JSON. The source configuration UI components specific for the Logs UI are moved to the pages/logs/settings directory.

The storage layer is not changed, so the saved object is still shared with the Metrics UI. But this separation on the route level prepares for an eventual split of the source configuration storage mechanisms.

The source status is implemented as separate route to allow for lazy field cap querying (see phase one of #53138).

It leaves the previous GraphQL resolvers in place for use by the Metrics UI. They will be removed along with the whole GraphQL endpoint in a later PR (see phase two of #53138).

New routes

`GET /api/infra/log_source_configurations/{sourceId}`

Get a log source configuration from the saved objects or the defaults.

`PATCH /api/infra/log_source_configurations/{sourceId}`

Update a log source configuration's saved object.

`GET /api/infra/log_source_configurations/{sourceId}/status`

Get the index names and field list pointed to by a source configuration.

Testing

Some points to have an eye on:

The UX should not have changed at all except that log source loading might be a bit faster.
The Logs UI should not perform any GraphQL requests anymore. They should all be to the /api/infra/log_source_configurations/... endpoints.
The "create" route was not implemented separately, but subsumed in the "update" route. Nevertheless it should be possible to store the initial configuration as well as update it later via the settings tab.
Changes to the source name and timestamp field should be reflected in the Metrics UI, but other source configuration properties (such as the metrics indices) should be preserved (and not be reset to the default).
Source configurations saved before should remain usable.

Follow-up tasks

Reimplement a similar API for metrics sources
Remove and dependencies of the Metrics UI on GraphQL types
Remove the GraphQL infra endpoint
Remove the GraphQL type generation script

elasticmachine · 2020-04-20T23:43:14Z

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

weltenwort · 2020-04-23T09:34:01Z

@elasticmachine merge upstream

weltenwort · 2020-04-23T09:44:32Z

There's a significant collision of intended goals with #64126, which I'll try to resolve.

weltenwort · 2020-04-23T13:59:03Z

@elasticmachine merge upstream

Kerry350

Looks / works great 👍 This will make the alerting stuff much easier 🙏

Just a few small comments.

Kerry350 · 2020-04-24T08:28:06Z

x-pack/plugins/infra/common/runtime_types.ts

+type ValdidationResult<Value> = ReturnType<RouteValidationFunction<Value>>;
+
+export const createValidationFunction = <A, O, I>(
+  runtimeType: Type<A, O, I>


Small nit: I personally find generics easier / quicker to parse with meaningful type variables, could we change these from A, O, I?

Good point, I absolutely agree. I stuck with the io-ts naming scheme during experimentation and forgot to clean it up afterwards.

x-pack/plugins/infra/public/pages/logs/page_content.tsx

x-pack/plugins/infra/public/pages/logs/stream/page_providers.tsx

x-pack/test/api_integration/apis/infra/log_sources.ts

weltenwort · 2020-04-27T10:36:24Z

@elasticmachine merge upstream

elasticmachine · 2020-04-27T10:36:25Z

merge conflict between base and head

weltenwort · 2020-04-27T13:48:04Z

@Kerry350 I had to adapt the mount location of the view-in-context-related component hierarchy to make sure the providers are available. I'd appreciate a second pair of eyes on that to ensure I haven't broken the view-in-context modal if you have the time 🤔

Kerry350 · 2020-04-27T15:14:49Z

@weltenwort

LGTM 👍

kibanamachine · 2020-04-28T00:09:35Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 8cc81b5

History

💔 Build #43334 failed fff15e1
💚 Build #43140 succeeded 241a9ed
💔 Build #43122 failed df3eadb
💔 Build #43068 failed fee6041
💔 Build #42767 failed 1e9fe00

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…SON (elastic#64021)

…HTTP+JSON (#64021) (#64626) Backports the following commits to 7.x: - [Logs UI] Reimplement log source configuration routes in plain HTTP+JSON (#64021)

weltenwort added 17 commits April 15, 2020 11:22

Add initial draft of http+json log source route

4b54c1b

Add api test for log source read route

9dec483

Extract common log source api types

2eccbe4

Add support for PATCH method in framework adapter

0426212

Add route to update source config

12945ae

Add tests for source config PATCH route

8633729

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

5cceac0

Add log source status route

e9d4f89

Start moving Logs UI over to the new source API (WIP)

308eb59

Correctly concatenate index names in fields domain

bc6c2e2

Only query field caps when at least one index exists

cbe5a55

Provide more values from source hook to stay compatible

7441b24

Switch more pages to the new log source container

a4d554a

Load data source on log rate and categories pages

a8d8a0e

Add update operation to log source hook

1a7369e

Move source configuration ui code to logs page dir

19a79c2

Trigger log source loading on a higher page level

8b23fe3

weltenwort added v8.0.0 Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services release_note:skip Skip the PR/issue when compiling release notes v7.8.0 labels Apr 20, 2020

weltenwort self-assigned this Apr 20, 2020

weltenwort added 2 commits April 21, 2020 01:56

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

3cb1126

Make redirect tests more stable

bac844b

weltenwort marked this pull request as ready for review April 21, 2020 18:50

weltenwort requested a review from a team as a code owner April 21, 2020 18:50

Kerry350 self-requested a review April 23, 2020 07:51

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

90795b9

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

1e9fe00

Kerry350 mentioned this pull request Apr 23, 2020

[Logs UI] Alerting #62806

Merged

Kerry350 approved these changes Apr 24, 2020

View reviewed changes

weltenwort added 4 commits April 24, 2020 12:41

Improve generic argument names

2476c50

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

fee6041

Use scopedHistory-compatible Prompt component

df3eadb

Fix custom prompt usage

241a9ed

weltenwort mentioned this pull request Apr 24, 2020

[Logs UI] Add dataset filter to ML module setup screen #64470

Merged

4 tasks

weltenwort added 3 commits April 27, 2020 12:42

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

18d303f

Use the new source provider in the context provider

fff15e1

Move "view in context" modal into the page content

e1f753c

weltenwort added 2 commits April 28, 2020 00:09

Merge branch 'master' into logs-ui-reimplement-source-routes-as-http

e7c0806

Adapt to new source config getter signature

8cc81b5

weltenwort merged commit 2fba7ed into elastic:master Apr 28, 2020

weltenwort deleted the logs-ui-reimplement-source-routes-as-http branch April 28, 2020 09:12

weltenwort mentioned this pull request Apr 28, 2020

[7.x] [Logs UI] Reimplement log source configuration routes in plain HTTP+JSON (#64021) #64626

Merged

weltenwort added a commit to weltenwort/kibana that referenced this pull request Apr 28, 2020

[Logs UI] Reimplement log source configuration routes in plain HTTP+J…

4d4dafb

…SON (elastic#64021)

weltenwort mentioned this pull request Apr 28, 2020

[Logs UI, Metrics UI] Data errors can block access to the settings page #64552

Closed

Kerry350 mentioned this pull request Apr 29, 2020

[Logs UI] [Alerting] Alerts management page enhancements #64654

Merged

weltenwort mentioned this pull request May 5, 2020

[Metrics UI] Move source API metrics routes source to HTTP #53138

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Logs UI] Reimplement log source configuration routes in plain HTTP+JSON #64021

[Logs UI] Reimplement log source configuration routes in plain HTTP+JSON #64021

weltenwort commented Apr 20, 2020 •

edited

Loading

elasticmachine commented Apr 20, 2020

weltenwort commented Apr 23, 2020

weltenwort commented Apr 23, 2020

weltenwort commented Apr 23, 2020

Kerry350 left a comment

Kerry350 Apr 24, 2020

weltenwort Apr 24, 2020

weltenwort commented Apr 27, 2020

elasticmachine commented Apr 27, 2020

weltenwort commented Apr 27, 2020

Kerry350 commented Apr 27, 2020

kibanamachine commented Apr 28, 2020

[Logs UI] Reimplement log source configuration routes in plain HTTP+JSON #64021

[Logs UI] Reimplement log source configuration routes in plain HTTP+JSON #64021

Conversation

weltenwort commented Apr 20, 2020 • edited Loading

Summary

New routes

GET /api/infra/log_source_configurations/{sourceId}

PATCH /api/infra/log_source_configurations/{sourceId}

GET /api/infra/log_source_configurations/{sourceId}/status

Testing

Follow-up tasks

elasticmachine commented Apr 20, 2020

weltenwort commented Apr 23, 2020

weltenwort commented Apr 23, 2020

weltenwort commented Apr 23, 2020

Kerry350 left a comment

Choose a reason for hiding this comment

Kerry350 Apr 24, 2020

Choose a reason for hiding this comment

weltenwort Apr 24, 2020

Choose a reason for hiding this comment

weltenwort commented Apr 27, 2020

elasticmachine commented Apr 27, 2020

weltenwort commented Apr 27, 2020

Kerry350 commented Apr 27, 2020

kibanamachine commented Apr 28, 2020

💚 Build Succeeded

History

weltenwort commented Apr 20, 2020 •

edited

Loading

`GET /api/infra/log_source_configurations/{sourceId}`

`PATCH /api/infra/log_source_configurations/{sourceId}`

`GET /api/infra/log_source_configurations/{sourceId}/status`