Skip to content

Commit

Permalink
[7.x] add SavedObject export hooks (#87807) (#88973)
Browse files Browse the repository at this point in the history
* add SavedObject export hooks (#87807)

* initial POC

* fix spaces UT

* address POC feedback, add tests for applyExportTransforms

* add sorting for transforms

* add type validation in SOTR

* add FTR tests

* update documentation

* add explicit so type export for client-side

* update generated doc

* add exporter test

* update license headers

* update generated doc

* fix so import... imports

* update generated doc

* nits

* update generated doc

* rename test plugins

* adding FTR tests on export failures

* fix data for 7.x
  • Loading branch information
pgayvallet authored Jan 21, 2021
1 parent b368d35 commit a8c21e0
Show file tree
Hide file tree
Showing 53 changed files with 2,080 additions and 41 deletions.
2 changes: 2 additions & 0 deletions docs/development/core/server/kibana-plugin-core-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [SavedObjectsExportByObjectOptions](./kibana-plugin-core-server.savedobjectsexportbyobjectoptions.md) | Options for the [export by objects API](./kibana-plugin-core-server.savedobjectsexporter.exportbyobjects.md) |
| [SavedObjectsExportByTypeOptions](./kibana-plugin-core-server.savedobjectsexportbytypeoptions.md) | Options for the [export by type API](./kibana-plugin-core-server.savedobjectsexporter.exportbytypes.md) |
| [SavedObjectsExportResultDetails](./kibana-plugin-core-server.savedobjectsexportresultdetails.md) | Structure of the export result details entry |
| [SavedObjectsExportTransformContext](./kibana-plugin-core-server.savedobjectsexporttransformcontext.md) | Context passed down to a [export transform function](./kibana-plugin-core-server.savedobjectsexporttransform.md) |
| [SavedObjectsFindOptions](./kibana-plugin-core-server.savedobjectsfindoptions.md) | |
| [SavedObjectsFindOptionsReference](./kibana-plugin-core-server.savedobjectsfindoptionsreference.md) | |
| [SavedObjectsFindResponse](./kibana-plugin-core-server.savedobjectsfindresponse.md) | Return type of the Saved Objects <code>find()</code> method.<!-- -->\*Note\*: this type is different between the Public and Server Saved Objects clients. |
Expand Down Expand Up @@ -297,6 +298,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [SavedObjectsClientFactory](./kibana-plugin-core-server.savedobjectsclientfactory.md) | Describes the factory used to create instances of the Saved Objects Client. |
| [SavedObjectsClientFactoryProvider](./kibana-plugin-core-server.savedobjectsclientfactoryprovider.md) | Provider to invoke to retrieve a [SavedObjectsClientFactory](./kibana-plugin-core-server.savedobjectsclientfactory.md)<!-- -->. |
| [SavedObjectsClientWrapperFactory](./kibana-plugin-core-server.savedobjectsclientwrapperfactory.md) | Describes the factory used to create instances of Saved Objects Client Wrappers. |
| [SavedObjectsExportTransform](./kibana-plugin-core-server.savedobjectsexporttransform.md) | Transformation function used to mutate the exported objects of the associated type.<!-- -->A type's export transform function will be executed once per user-initiated export, for all objects of that type. |
| [SavedObjectsFieldMapping](./kibana-plugin-core-server.savedobjectsfieldmapping.md) | Describe a [saved object type mapping](./kibana-plugin-core-server.savedobjectstypemappingdefinition.md) field.<!-- -->Please refer to [elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) For the mapping documentation |
| [SavedObjectsImportHook](./kibana-plugin-core-server.savedobjectsimporthook.md) | A hook associated with a specific saved object type, that will be invoked during the import process. The hook will have access to the objects of the registered type.<!-- -->Currently, the only supported feature for import hooks is to return warnings to be displayed in the UI when the import succeeds. The only interactions the hook can have with the import process is via the hook's response. Mutating the objects inside the hook's code will have no effect. |
| [SavedObjectsImportWarning](./kibana-plugin-core-server.savedobjectsimportwarning.md) | Composite type of all the possible types of import warnings.<!-- -->See [SavedObjectsImportSimpleWarning](./kibana-plugin-core-server.savedobjectsimportsimplewarning.md) and [SavedObjectsImportActionRequiredWarning](./kibana-plugin-core-server.savedobjectsimportactionrequiredwarning.md) for more details. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,5 @@ export interface SavedObjectExportBaseOptions
| [excludeExportDetails](./kibana-plugin-core-server.savedobjectexportbaseoptions.excludeexportdetails.md) | <code>boolean</code> | flag to not append [export details](./kibana-plugin-core-server.savedobjectsexportresultdetails.md) to the end of the export stream. |
| [includeReferencesDeep](./kibana-plugin-core-server.savedobjectexportbaseoptions.includereferencesdeep.md) | <code>boolean</code> | flag to also include all related saved objects in the export stream. |
| [namespace](./kibana-plugin-core-server.savedobjectexportbaseoptions.namespace.md) | <code>string</code> | optional namespace to override the namespace used by the savedObjectsClient. |
| [request](./kibana-plugin-core-server.savedobjectexportbaseoptions.request.md) | <code>KibanaRequest</code> | The http request initiating the export. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectExportBaseOptions](./kibana-plugin-core-server.savedobjectexportbaseoptions.md) &gt; [request](./kibana-plugin-core-server.savedobjectexportbaseoptions.request.md)

## SavedObjectExportBaseOptions.request property

The http request initiating the export.

<b>Signature:</b>

```typescript
request: KibanaRequest;
```
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ Constructs a new instance of the `SavedObjectsExporter` class
<b>Signature:</b>

```typescript
constructor({ savedObjectsClient, exportSizeLimit, }: {
constructor({ savedObjectsClient, typeRegistry, exportSizeLimit, }: {
savedObjectsClient: SavedObjectsClientContract;
typeRegistry: ISavedObjectTypeRegistry;
exportSizeLimit: number;
});
```
Expand All @@ -19,5 +20,5 @@ constructor({ savedObjectsClient, exportSizeLimit, }: {

| Parameter | Type | Description |
| --- | --- | --- |
| { savedObjectsClient, exportSizeLimit, } | <code>{</code><br/><code> savedObjectsClient: SavedObjectsClientContract;</code><br/><code> exportSizeLimit: number;</code><br/><code> }</code> | |
| { savedObjectsClient, typeRegistry, exportSizeLimit, } | <code>{</code><br/><code> savedObjectsClient: SavedObjectsClientContract;</code><br/><code> typeRegistry: ISavedObjectTypeRegistry;</code><br/><code> exportSizeLimit: number;</code><br/><code> }</code> | |

Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ export declare class SavedObjectsExporter

| Constructor | Modifiers | Description |
| --- | --- | --- |
| [(constructor)({ savedObjectsClient, exportSizeLimit, })](./kibana-plugin-core-server.savedobjectsexporter._constructor_.md) | | Constructs a new instance of the <code>SavedObjectsExporter</code> class |
| [(constructor)({ savedObjectsClient, typeRegistry, exportSizeLimit, })](./kibana-plugin-core-server.savedobjectsexporter._constructor_.md) | | Constructs a new instance of the <code>SavedObjectsExporter</code> class |

## Properties

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsExportError](./kibana-plugin-core-server.savedobjectsexporterror.md) &gt; [invalidTransformError](./kibana-plugin-core-server.savedobjectsexporterror.invalidtransformerror.md)

## SavedObjectsExportError.invalidTransformError() method

Error returned when a [export tranform](./kibana-plugin-core-server.savedobjectsexporttransform.md) performed an invalid operation during the transform, such as removing objects from the export, or changing an object's type or id.

<b>Signature:</b>

```typescript
static invalidTransformError(objectKeys: string[]): SavedObjectsExportError;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| objectKeys | <code>string[]</code> | |

<b>Returns:</b>

`SavedObjectsExportError`

Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,7 @@ export declare class SavedObjectsExportError extends Error
| Method | Modifiers | Description |
| --- | --- | --- |
| [exportSizeExceeded(limit)](./kibana-plugin-core-server.savedobjectsexporterror.exportsizeexceeded.md) | <code>static</code> | |
| [invalidTransformError(objectKeys)](./kibana-plugin-core-server.savedobjectsexporterror.invalidtransformerror.md) | <code>static</code> | Error returned when a [export tranform](./kibana-plugin-core-server.savedobjectsexporttransform.md) performed an invalid operation during the transform, such as removing objects from the export, or changing an object's type or id. |
| [objectFetchError(objects)](./kibana-plugin-core-server.savedobjectsexporterror.objectfetcherror.md) | <code>static</code> | |
| [objectTransformError(objects, cause)](./kibana-plugin-core-server.savedobjectsexporterror.objecttransformerror.md) | <code>static</code> | Error returned when a [export tranform](./kibana-plugin-core-server.savedobjectsexporttransform.md) threw an error |
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsExportError](./kibana-plugin-core-server.savedobjectsexporterror.md) &gt; [objectTransformError](./kibana-plugin-core-server.savedobjectsexporterror.objecttransformerror.md)

## SavedObjectsExportError.objectTransformError() method

Error returned when a [export tranform](./kibana-plugin-core-server.savedobjectsexporttransform.md) threw an error

<b>Signature:</b>

```typescript
static objectTransformError(objects: SavedObject[], cause: Error): SavedObjectsExportError;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| objects | <code>SavedObject[]</code> | |
| cause | <code>Error</code> | |

<b>Returns:</b>

`SavedObjectsExportError`

Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsExportTransform](./kibana-plugin-core-server.savedobjectsexporttransform.md)

## SavedObjectsExportTransform type

Transformation function used to mutate the exported objects of the associated type.

A type's export transform function will be executed once per user-initiated export, for all objects of that type.

<b>Signature:</b>

```typescript
export declare type SavedObjectsExportTransform = <T = unknown>(context: SavedObjectsExportTransformContext, objects: Array<SavedObject<T>>) => SavedObject[] | Promise<SavedObject[]>;
```

## Remarks

Trying to change an object's id or type during the transform will result in a runtime error during the export process.

## Example 1

Registering a transform function changing the object's attributes during the export

```ts
// src/plugins/my_plugin/server/plugin.ts
import { myType } from './saved_objects';

export class Plugin() {
setup: (core: CoreSetup) => {
core.savedObjects.registerType({
...myType,
management: {
...myType.management,
onExport: (ctx, objects) => {
return objects.map((obj) => ({
...obj,
attributes: {
...obj.attributes,
enabled: false,
}
})
}
},
});
}
}

```
## Example 2
Registering a transform function adding additional objects to the export
```ts
// src/plugins/my_plugin/server/plugin.ts
import { myType } from './saved_objects';

export class Plugin() {
setup: (core: CoreSetup) => {
const savedObjectStartContractPromise = getStartServices().then(
([{ savedObjects: savedObjectsStart }]) => savedObjectsStart
);

core.savedObjects.registerType({
...myType,
management: {
...myType.management,
onExport: async (ctx, objects) => {
const { getScopedClient } = await savedObjectStartContractPromise;
const client = getScopedClient(ctx.request);

const depResponse = await client.find({
type: 'my-nested-object',
hasReference: objs.map(({ id, type }) => ({ id, type })),
});

return [...objs, ...depResponse.saved_objects];
}
},
});
}
}

```
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsExportTransformContext](./kibana-plugin-core-server.savedobjectsexporttransformcontext.md)

## SavedObjectsExportTransformContext interface

Context passed down to a [export transform function](./kibana-plugin-core-server.savedobjectsexporttransform.md)

<b>Signature:</b>

```typescript
export interface SavedObjectsExportTransformContext
```

## Properties

| Property | Type | Description |
| --- | --- | --- |
| [request](./kibana-plugin-core-server.savedobjectsexporttransformcontext.request.md) | <code>KibanaRequest</code> | The request that initiated the export request. Can be used to create scoped services or client inside the [transformation](./kibana-plugin-core-server.savedobjectsexporttransform.md) |

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsExportTransformContext](./kibana-plugin-core-server.savedobjectsexporttransformcontext.md) &gt; [request](./kibana-plugin-core-server.savedobjectsexporttransformcontext.request.md)

## SavedObjectsExportTransformContext.request property

The request that initiated the export request. Can be used to create scoped services or client inside the [transformation](./kibana-plugin-core-server.savedobjectsexporttransform.md)

<b>Signature:</b>

```typescript
request: KibanaRequest;
```
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ export interface SavedObjectsTypeManagementDefinition
| [getTitle](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.gettitle.md) | <code>(savedObject: SavedObject&lt;any&gt;) =&gt; string</code> | Function returning the title to display in the management table. If not defined, will use the object's type and id to generate a label. |
| [icon](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.icon.md) | <code>string</code> | The eui icon name to display in the management table. If not defined, the default icon will be used. |
| [importableAndExportable](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.importableandexportable.md) | <code>boolean</code> | Is the type importable or exportable. Defaults to <code>false</code>. |
| [onExport](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.onexport.md) | <code>SavedObjectsExportTransform</code> | An optional export transform function that can be used transform the objects of the registered type during the export process.<!-- -->It can be used to either mutate the exported objects, or add additional objects (of any type) to the export list.<!-- -->See [the transform type documentation](./kibana-plugin-core-server.savedobjectsexporttransform.md) for more info and examples. |
| [onImport](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.onimport.md) | <code>SavedObjectsImportHook</code> | An optional [import hook](./kibana-plugin-core-server.savedobjectsimporthook.md) to use when importing given type.<!-- -->Import hooks are executed during the savedObjects import process and allow to interact with the imported objects. See the [hook documentation](./kibana-plugin-core-server.savedobjectsimporthook.md) for more info. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsTypeManagementDefinition](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.md) &gt; [onExport](./kibana-plugin-core-server.savedobjectstypemanagementdefinition.onexport.md)

## SavedObjectsTypeManagementDefinition.onExport property

An optional export transform function that can be used transform the objects of the registered type during the export process.

It can be used to either mutate the exported objects, or add additional objects (of any type) to the export list.

See [the transform type documentation](./kibana-plugin-core-server.savedobjectsexporttransform.md) for more info and examples.

<b>Signature:</b>

```typescript
onExport?: SavedObjectsExportTransform;
```

## Remarks

`importableAndExportable` must be `true` to specify this property.

Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ Import hooks are executed during the savedObjects import process and allow to in
onImport?: SavedObjectsImportHook;
```

## Remarks

`importableAndExportable` must be `true` to specify this property.

## Example

Registering a hook displaying a warning about a specific type of object
Expand Down Expand Up @@ -48,5 +52,4 @@ export class Plugin() {
}

```
messages returned in the warnings are user facing and must be translated.

7 changes: 7 additions & 0 deletions src/core/public/public.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { ApiResponse } from '@elastic/elasticsearch/lib/Transport';
import Boom from '@hapi/boom';
import { ConfigDeprecationProvider } from '@kbn/config';
import { ConfigPath } from '@kbn/config';
import { DetailedPeerCertificate } from 'tls';
import { EnvironmentMode } from '@kbn/config';
import { EuiBreadcrumb } from '@elastic/eui';
import { EuiButtonEmptyProps } from '@elastic/eui';
Expand All @@ -18,27 +19,33 @@ import { EuiGlobalToastListToast } from '@elastic/eui';
import { History } from 'history';
import { Href } from 'history';
import { IconType } from '@elastic/eui';
import { IncomingHttpHeaders } from 'http';
import { KibanaClient } from '@elastic/elasticsearch/api/kibana';
import { Location } from 'history';
import { LocationDescriptorObject } from 'history';
import { Logger } from '@kbn/logging';
import { LogMeta } from '@kbn/logging';
import { MaybePromise } from '@kbn/utility-types';
import { ObjectType } from '@kbn/config-schema';
import { Observable } from 'rxjs';
import { PackageInfo } from '@kbn/config';
import { Path } from 'history';
import { PeerCertificate } from 'tls';
import { PublicMethodsOf } from '@kbn/utility-types';
import { PublicUiSettingsParams as PublicUiSettingsParams_2 } from 'src/core/server/types';
import React from 'react';
import { RecursiveReadonly } from '@kbn/utility-types';
import { Request } from '@hapi/hapi';
import * as Rx from 'rxjs';
import { SchemaTypeError } from '@kbn/config-schema';
import { TransportRequestOptions } from '@elastic/elasticsearch/lib/Transport';
import { TransportRequestParams } from '@elastic/elasticsearch/lib/Transport';
import { TransportRequestPromise } from '@elastic/elasticsearch/lib/Transport';
import { Type } from '@kbn/config-schema';
import { TypeOf } from '@kbn/config-schema';
import { UiCounterMetricType } from '@kbn/analytics';
import { UnregisterCallback } from 'history';
import { URL } from 'url';
import { UserProvidedValues as UserProvidedValues_2 } from 'src/core/server/types';

// @internal (undocumented)
Expand Down
2 changes: 2 additions & 0 deletions src/core/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,8 @@ export {
SavedObjectsExportByObjectOptions,
SavedObjectsExportByTypeOptions,
SavedObjectsExportError,
SavedObjectsExportTransform,
SavedObjectsExportTransformContext,
SavedObjectsImporter,
ISavedObjectsImporter,
SavedObjectsImportError,
Expand Down
Loading

0 comments on commit a8c21e0

Please sign in to comment.