Provide a clear migration path when removing kibana.index

[romain-chanu]

Kibana version: 8.x

Elasticsearch version: 8.x

Server OS version: N.A

Browser version: N.A

Browser OS version: N.A

Original install method (e.g. download page, yum, from source, etc.): N.A

Describe the bug:

kibana.index setting has been removed in version 8.0 (c.f #112773). Users upgrading from 7.x to version 8.x will face the following challenges:

1. After removing kibana.index and upgrading to version 8.x, saved objects migration will not be triggered since the kibana.index was not referring to the default .kibana alias/index name. Users will observe a loss of all their saved objects when Kibana 8.x starts-up (since a new .kibana index is created).

2. Elasticsearch roles (where Kibana privileges have been configured) will not be updated. Users will report that the affected roles have lost their Kibana privileges.

3. The upgrade documentation does not provide any steps to safely remove kibana.index and ensure that all saved objects / privileges are "migrated" to version 8.x

Steps to reproduce:

1. Spin-up a cluster version 7.17.x and configure kibana.index to let's say .kibana-test
2. Load the Kibana sample data and create a couple of Elasticsearch roles with custom Kibana privileges.
3. Remove kibana.index and upgrade the cluster
4. Observe the following:

• Saved objects created in version 7.x are no longer accessible in version 8.x
• Elasticsearch roles configured with Kibana privileges in version 7.x no longer work.

Expected behavior:

1. Ideally, we should have an automated way to migrate the saved objects (and privileges) after removing kibana.index

2. If the above is not possible, we should document a clear migration path when removing kibana.index.

2.1) For the saved objects migration, the steps are fairly simple. The upgrade process would be the same as our upgrade guide with one extra step (i.e adding the .kibana alias to point to the custom Kibana index):

a) Stop all Kibana instances and upgrade the Kibana binaries to version 8.x

b) Modify kibana.yml as needed (e.g removing the kibana.index setting).

c) Add the .kibana alias to point to the Kibana index used pre-upgrade. For example: if Kibana was previously using version 7.17.9 with kibana.index set to .kibana-test, users would need to configure the .kibana alias to point to index .kibana-test_7.17.9_001:

POST /_aliases
{
    "actions" : [
        { "add" : { "index" : ".kibana-test_7.17.9_001", "alias" : ".kibana" } }
    ]
}

d) Start Kibana instances using the new version.

2.2) For the roles / privileges, this is a bit more complex.

a) Users will need to retrieve the list of Elasticsearch roles (c.f Get roles API) and identify all the affected roles. In each role, users will find an applications.application field that references the customer Kibana index (in our earlier example, this would be kibana-.kibana-test).

b) User will then need to update the affected roles (c.f Create or update roles API) and edit the applications.application field to match the new Kibana index name (in the above example, replacing kibana-.kibana-test with kibana-.kibana)

Errors in browser console (if relevant): N.A

Provide logs and/or server output (if relevant): N.A

Any additional context: N.A

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[rudolf]

We should also document the upgrade path for the other indices which are no longer configurable #101964

[rudolf]

@elastic/kibana-security Would Kibana ever create roles with privileges to the .kibana (or in this case the configured kibana.index) index? Or might this have been something custom the user added? My understanding is that Kibana always uses the internal user when accessing .kibana.

[thomheymann]

@elastic/kibana-security Would Kibana ever create roles with privileges to the .kibana (or in this case the configured kibana.index) index? Or might this have been something custom the user added? My understanding is that Kibana always uses the internal user when accessing .kibana.

I am not aware of Kibana creating roles with access to the kibana system index so these must have been something manually added by the user.

[romain-chanu]

@thomheymann - I just want to highlight that the "roles" mentioned in this issue are simply Elasticsearch roles that users would configure for RBAC. Each role could be configured with certain Kibana privileges. As I stated earlier, one of the problem is that Elasticsearch references the old Kibana index in the role definition (c.f issue description, with the applications.application field).

[thomheymann]

@thomheymann - I just want to highlight that the "roles" mentioned in this issue are simply Elasticsearch roles that users would configure for RBAC. Each role could be configured with certain Kibana privileges. As I stated earlier, one of the problem is that Elasticsearch references the old Kibana index in the role definition (c.f issue description, with the applications.application field).

Sorry, ignore me, I thought you were talking about cluster index privileges rather than application privileges.

You're right, if the Kibana system index changes as part of the v7.x > v8.x upgrade that would render all roles that contain Kibana application privileges useless:

{
  "test_role": {
    "cluster": [],
    "indices": [],
    "applications": [
      {
        "application": "kibana-.kibana",
        "privileges": [
          "feature_discover.read"
        ],
        "resources": [
          "space:default"
        ]
      }
    ]
  }
}