Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Query] Use a minimal index pattern interface for es query #102364

Merged
merged 24 commits into from
Jun 23, 2021
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
148af16
Move JSON utils to utils package
Jun 14, 2021
0b79d06
Imports from tests
Jun 14, 2021
e5f9f80
Merge branch 'master' of github.com:elastic/kibana into es-kuery/move…
Jun 14, 2021
83645da
delete
Jun 14, 2021
33f7a87
Merge branch 'master' into es-kuery/move-json-utils
kibanamachine Jun 14, 2021
6b51a9a
Merge branch 'master' of github.com:elastic/kibana into es-kuery/move…
Jun 15, 2021
04206fb
split package
Jun 15, 2021
3f895f4
Merge branch 'master' of github.com:elastic/kibana into es-kuery/move…
Jun 16, 2021
4248be7
Merge branch 'es-kuery/move-json-utils' of github.com:lizozom/kibana …
Jun 16, 2021
b1ccf5c
docs
Jun 16, 2021
e02bd98
test
Jun 16, 2021
9ab64da
test
Jun 16, 2021
6347118
imports
Jun 16, 2021
508ed0e
minimal index pattern
Jun 16, 2021
25884bb
move some functions out and use miniaml ip in all es-kuery
Jun 16, 2021
51c490b
Merge branch 'master' of github.com:elastic/kibana into es-kuery/new-…
Jun 16, 2021
cdb1f82
docs
Jun 16, 2021
c49630b
docs
Jun 21, 2021
e611902
Merge branch 'master' of github.com:elastic/kibana into es-kuery/new-…
Jun 21, 2021
1bab47e
Merge branch 'master' of github.com:elastic/kibana into es-kuery/new-…
Jun 23, 2021
308ec01
Merge branch 'master' into es-kuery/new-interface
kibanamachine Jun 23, 2021
03ca054
rename
Jun 23, 2021
903b8f4
Merge branch 'master' of github.com:elastic/kibana into es-kuery/new-…
Jun 23, 2021
68a7d27
Merge branch 'es-kuery/new-interface' of github.com:lizozom/kibana in…
Jun 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/es_query/build_es_query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ import { groupBy, has, isEqual } from 'lodash';
import { buildQueryFromKuery } from './from_kuery';
import { buildQueryFromFilters } from './from_filters';
import { buildQueryFromLucene } from './from_lucene';
import { IIndexPattern } from '../../index_patterns';
import { Filter } from '../filters';
import { Query } from '../../query/types';
import { MinimalIndexPattern } from './types';

export interface EsQueryConfig {
allowLeadingWildcards: boolean;
Expand All @@ -36,7 +36,7 @@ function removeMatchAll<T>(filters: T[]) {
* config contains dateformat:tz
*/
export function buildEsQuery(
indexPattern: IIndexPattern | undefined,
indexPattern: MinimalIndexPattern | undefined,
queries: Query | Query[],
filters: Filter | Filter[],
config: EsQueryConfig = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@
* Side Public License, v 1.
*/

import { IIndexPattern, IFieldType } from '../../index_patterns';
import { IFieldType } from '../../index_patterns';
import { Filter } from '../filters';
import { MinimalIndexPattern } from './types';

/*
* TODO: We should base this on something better than `filter.meta.key`. We should probably modify
* this to check if `filter.meta.index` matches `indexPattern.id` instead, but that's a breaking
* change.
*/
export function filterMatchesIndex(filter: Filter, indexPattern?: IIndexPattern | null) {
export function filterMatchesIndex(filter: Filter, indexPattern?: MinimalIndexPattern | null) {
if (!filter.meta?.key || !indexPattern) {
return true;
}
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/es_query/from_filters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { isUndefined } from 'lodash';
import { migrateFilter } from './migrate_filter';
import { filterMatchesIndex } from './filter_matches_index';
import { Filter, cleanFilter, isFilterDisabled } from '../filters';
import { IIndexPattern } from '../../index_patterns';
import { MinimalIndexPattern } from './types';
import { handleNestedFilter } from './handle_nested_filter';

/**
Expand Down Expand Up @@ -45,7 +45,7 @@ const translateToQuery = (filter: Filter) => {

export const buildQueryFromFilters = (
filters: Filter[] = [],
indexPattern: IIndexPattern | undefined,
indexPattern: MinimalIndexPattern | undefined,
ignoreFilterIfFieldNotInIndex: boolean = false
) => {
filters = filters.filter((filter) => filter && !isFilterDisabled(filter));
Expand Down
6 changes: 3 additions & 3 deletions src/plugins/data/common/es_query/es_query/from_kuery.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
*/

import { fromKueryExpression, toElasticsearchQuery, nodeTypes, KueryNode } from '../kuery';
import { IIndexPattern } from '../../index_patterns';
import { MinimalIndexPattern } from './types';
import { Query } from '../../query/types';

export function buildQueryFromKuery(
indexPattern: IIndexPattern | undefined,
indexPattern: MinimalIndexPattern | undefined,
queries: Query[] = [],
allowLeadingWildcards: boolean = false,
dateFormatTZ?: string
Expand All @@ -24,7 +24,7 @@ export function buildQueryFromKuery(
}

function buildQuery(
indexPattern: IIndexPattern | undefined,
indexPattern: MinimalIndexPattern | undefined,
queryASTs: KueryNode[],
config: Record<string, any> = {}
) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,14 @@
import { handleNestedFilter } from './handle_nested_filter';
import { fields } from '../../index_patterns/mocks';
import { buildPhraseFilter, buildQueryFilter } from '../filters';
import { IFieldType, IIndexPattern } from '../../index_patterns';
import { MinimalIndexPattern } from './types';
import { IFieldType } from '../../index_patterns';

describe('handleNestedFilter', function () {
const indexPattern: IIndexPattern = ({
const indexPattern: MinimalIndexPattern = {
id: 'logstash-*',
fields,
} as unknown) as IIndexPattern;
};

it("should return the filter's query wrapped in nested query if the target field is nested", () => {
const field = getField('nestedField.child');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
*/

import { getFilterField, cleanFilter, Filter } from '../filters';
import { IIndexPattern } from '../../index_patterns';
import { MinimalIndexPattern } from './types';

export const handleNestedFilter = (filter: Filter, indexPattern?: IIndexPattern) => {
export const handleNestedFilter = (filter: Filter, indexPattern?: MinimalIndexPattern) => {
if (!indexPattern) return filter;

const fieldName = getFilterField(filter);
Expand Down
1 change: 1 addition & 0 deletions src/plugins/data/common/es_query/es_query/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ export { buildQueryFromFilters } from './from_filters';
export { luceneStringToDsl } from './lucene_string_to_dsl';
export { decorateQuery } from './decorate_query';
export { getEsQueryConfig } from './get_es_query_config';
export { MinimalIndexPattern } from './types';
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/es_query/migrate_filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import { get, omit } from 'lodash';
import { getConvertedValueForField } from '../filters';
import { Filter } from '../filters';
import { IIndexPattern } from '../../index_patterns';
import { MinimalIndexPattern } from './types';

export interface DeprecatedMatchPhraseFilter extends Filter {
query: {
Expand All @@ -28,7 +28,7 @@ function isDeprecatedMatchPhraseFilter(filter: any): filter is DeprecatedMatchPh
return Boolean(fieldName && get(filter, ['query', 'match', fieldName, 'type']) === 'phrase');
}

export function migrateFilter(filter: Filter, indexPattern?: IIndexPattern) {
export function migrateFilter(filter: Filter, indexPattern?: MinimalIndexPattern) {
if (isDeprecatedMatchPhraseFilter(filter)) {
const fieldName = Object.keys(filter.query.match)[0];
const params: Record<string, any> = get(filter, ['query', 'match', fieldName]);
Expand Down
14 changes: 14 additions & 0 deletions src/plugins/data/common/es_query/es_query/types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { IFieldType } from '../../index_patterns';

export interface MinimalIndexPattern {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattkime do you have a better name for this?
And what do you think in general?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine. Its a bit hacky but it solves the problem without increasing complexity.

fields: IFieldType[];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the dependency on IFieldType[] going to be a problem too, whenever this gets split out into a package? I'd imagine you'd need to relocate that as well.

FWIW, if it starts to feel weird having all of this stuff crammed in a @kbn/es-query package, someone had proposed creating something like @kbn/data-types as a type-specific package just for the data plugin, so that these things could live there instead (and be imported by @kbn/es-query)

Neither option feels great, just wanted to mention it as something to consider.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I intend to handle IFieldType in a followup PR, before moving both into a package

id?: string;
}
6 changes: 3 additions & 3 deletions src/plugins/data/common/es_query/filters/build_filters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
* Side Public License, v 1.
*/

import { IIndexPattern, IFieldType } from '../..';
import { IFieldType, MinimalIndexPattern } from '../..';
import {
Filter,
FILTERS,
Expand All @@ -19,7 +19,7 @@ import {
} from '.';

export function buildFilter(
indexPattern: IIndexPattern,
indexPattern: MinimalIndexPattern,
field: IFieldType,
type: FILTERS,
negate: boolean,
Expand Down Expand Up @@ -59,7 +59,7 @@ export function buildCustomFilter(
}

function buildBaseFilter(
indexPattern: IIndexPattern,
indexPattern: MinimalIndexPattern,
field: IFieldType,
type: FILTERS,
params: any
Expand Down
5 changes: 3 additions & 2 deletions src/plugins/data/common/es_query/filters/exists_filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
*/

import { Filter, FilterMeta } from './meta_filter';
import { IIndexPattern, IFieldType } from '../../index_patterns';
import { IFieldType } from '../../index_patterns';
import { MinimalIndexPattern } from '..';

export type ExistsFilterMeta = FilterMeta;

Expand All @@ -26,7 +27,7 @@ export const getExistsFilterField = (filter: ExistsFilter) => {
return filter.exists && filter.exists.field;
};

export const buildExistsFilter = (field: IFieldType, indexPattern: IIndexPattern) => {
export const buildExistsFilter = (field: IFieldType, indexPattern: MinimalIndexPattern) => {
return {
meta: {
index: indexPattern.id,
Expand Down
2 changes: 0 additions & 2 deletions src/plugins/data/common/es_query/filters/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,8 @@ export * from './custom_filter';
export * from './exists_filter';
export * from './geo_bounding_box_filter';
export * from './geo_polygon_filter';
export * from './get_display_value';
export * from './get_filter_field';
export * from './get_filter_params';
export * from './get_index_pattern_from_filter';
export * from './match_all_filter';
export * from './meta_filter';
export * from './missing_filter';
Expand Down
5 changes: 3 additions & 2 deletions src/plugins/data/common/es_query/filters/phrase_filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
import type { estypes } from '@elastic/elasticsearch';
import { get, isPlainObject } from 'lodash';
import { Filter, FilterMeta } from './meta_filter';
import { IIndexPattern, IFieldType } from '../../index_patterns';
import { IFieldType } from '../../index_patterns';
import { MinimalIndexPattern } from '..';

export type PhraseFilterMeta = FilterMeta & {
params?: {
Expand Down Expand Up @@ -60,7 +61,7 @@ export const getPhraseFilterValue = (filter: PhraseFilter): PhraseFilterValue =>
export const buildPhraseFilter = (
field: IFieldType,
value: any,
indexPattern: IIndexPattern
indexPattern: MinimalIndexPattern
): PhraseFilter => {
const convertedValue = getConvertedValueForField(field, value);

Expand Down
5 changes: 3 additions & 2 deletions src/plugins/data/common/es_query/filters/phrases_filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
import { Filter, FilterMeta } from './meta_filter';
import { getPhraseScript } from './phrase_filter';
import { FILTERS } from './index';
import { IIndexPattern, IFieldType } from '../../index_patterns';
import { IFieldType } from '../../index_patterns';
import { MinimalIndexPattern } from '../es_query';

export type PhrasesFilterMeta = FilterMeta & {
params: string[]; // The unformatted values
Expand All @@ -34,7 +35,7 @@ export const getPhrasesFilterField = (filter: PhrasesFilter) => {
export const buildPhrasesFilter = (
field: IFieldType,
params: any[],
indexPattern: IIndexPattern
indexPattern: MinimalIndexPattern
) => {
const index = indexPattern.id;
const type = FILTERS.PHRASES;
Expand Down
5 changes: 3 additions & 2 deletions src/plugins/data/common/es_query/filters/range_filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
import type { estypes } from '@elastic/elasticsearch';
import { map, reduce, mapValues, get, keys, pickBy } from 'lodash';
import { Filter, FilterMeta } from './meta_filter';
import { IIndexPattern, IFieldType } from '../../index_patterns';
import { IFieldType } from '../../index_patterns';
import { MinimalIndexPattern } from '..';

const OPERANDS_IN_RANGE = 2;

Expand Down Expand Up @@ -93,7 +94,7 @@ const format = (field: IFieldType, value: any) =>
export const buildRangeFilter = (
field: IFieldType,
params: RangeFilterParams,
indexPattern: IIndexPattern,
indexPattern: MinimalIndexPattern,
formattedValue?: string
): RangeFilter => {
const filter: any = { meta: { index: indexPattern.id, params: {} } };
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/kuery/ast/ast.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ import { JsonObject } from '@kbn/common-utils';
import { nodeTypes } from '../node_types/index';
import { KQLSyntaxError } from '../kuery_syntax_error';
import { KueryNode, DslQuery, KueryParseOptions } from '../types';
import { IIndexPattern } from '../../../index_patterns/types';

// @ts-ignore
import { parse as parseKuery } from './_generated_/kuery';
import { MinimalIndexPattern } from '../..';

const fromExpression = (
expression: string | DslQuery,
Expand Down Expand Up @@ -65,7 +65,7 @@ export const fromKueryExpression = (
*/
export const toElasticsearchQuery = (
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config?: Record<string, any>,
context?: Record<string, any>
): JsonObject => {
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/kuery/functions/and.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
*/

import * as ast from '../ast';
import { IIndexPattern, KueryNode } from '../../..';
import { MinimalIndexPattern, KueryNode } from '../../..';

export function buildNodeParams(children: KueryNode[]) {
return {
Expand All @@ -17,7 +17,7 @@ export function buildNodeParams(children: KueryNode[]) {

export function toElasticsearchQuery(
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config: Record<string, any> = {},
context: Record<string, any> = {}
) {
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/kuery/functions/exists.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

import { get } from 'lodash';
import * as literal from '../node_types/literal';
import { IIndexPattern, KueryNode, IFieldType } from '../../..';
import { KueryNode, IFieldType, MinimalIndexPattern } from '../../..';

export function buildNodeParams(fieldName: string) {
return {
Expand All @@ -18,7 +18,7 @@ export function buildNodeParams(fieldName: string) {

export function toElasticsearchQuery(
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config: Record<string, any> = {},
context: Record<string, any> = {}
) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import _ from 'lodash';
import { nodeTypes } from '../node_types';
import * as ast from '../ast';
import { IIndexPattern, KueryNode, IFieldType, LatLon } from '../../..';
import { MinimalIndexPattern, KueryNode, IFieldType, LatLon } from '../../..';

export function buildNodeParams(fieldName: string, params: any) {
params = _.pick(params, 'topLeft', 'bottomRight');
Expand All @@ -26,7 +26,7 @@ export function buildNodeParams(fieldName: string, params: any) {

export function toElasticsearchQuery(
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config: Record<string, any> = {},
context: Record<string, any> = {}
) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

import { nodeTypes } from '../node_types';
import * as ast from '../ast';
import { IIndexPattern, KueryNode, IFieldType, LatLon } from '../../..';
import { MinimalIndexPattern, KueryNode, IFieldType, LatLon } from '../../..';
import { LiteralTypeBuildNode } from '../node_types/types';

export function buildNodeParams(fieldName: string, points: LatLon[]) {
Expand All @@ -25,7 +25,7 @@ export function buildNodeParams(fieldName: string, points: LatLon[]) {

export function toElasticsearchQuery(
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config: Record<string, any> = {},
context: Record<string, any> = {}
) {
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/kuery/functions/is.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { getPhraseScript } from '../../filters';
import { getFields } from './utils/get_fields';
import { getTimeZoneFromSettings } from '../../utils';
import { getFullFieldNameNode } from './utils/get_full_field_name_node';
import { IIndexPattern, KueryNode, IFieldType } from '../../..';
import { MinimalIndexPattern, KueryNode, IFieldType } from '../../..';

import * as ast from '../ast';

Expand Down Expand Up @@ -39,7 +39,7 @@ export function buildNodeParams(fieldName: string, value: any, isPhrase: boolean

export function toElasticsearchQuery(
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config: Record<string, any> = {},
context: Record<string, any> = {}
) {
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/common/es_query/kuery/functions/nested.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

import * as ast from '../ast';
import * as literal from '../node_types/literal';
import { IIndexPattern, KueryNode } from '../../..';
import { MinimalIndexPattern, KueryNode } from '../../..';

export function buildNodeParams(path: any, child: any) {
const pathNode =
Expand All @@ -20,7 +20,7 @@ export function buildNodeParams(path: any, child: any) {

export function toElasticsearchQuery(
node: KueryNode,
indexPattern?: IIndexPattern,
indexPattern?: MinimalIndexPattern,
config: Record<string, any> = {},
context: Record<string, any> = {}
) {
Expand Down
Loading