Tag Management vs Saved-Object Management

[kobelb]

Background

As part of the effort to remove legacy multitenancy in 8.0, I've been pushing for us to allow users to export/import all of their Kibana data using saved-object management. This would allow us to continue to use the approach originally recommended when Spaces was first released in 6.5 to use saved-object management export all of their data from a legacy tenant and import it into a Space.

However, @pgayvallet brought it to my attention that we were planning on no longer listing tags on the saved-object management screen. Without any fundamental changes, this would mean that tags would not be included in exports unless the user selected to "include related objects" and the tag happens to be referenced by one of the other saved-objects:

This is also in contrast to the way that index-patterns behave, where they show up in both the index-pattern management screens and the saved-object management screens:

Discussion topics

I don't intend to mandate that we stick to these topics or forcefully frame the discussion in this manner, so please feel free to break from this format if you so choose.

1. Where should we allow users to export/import all of their Kibana specific data? Requiring users to use the dashboard app to export/import dashboards, the visualization app to export/import visualizations, etc. seems like a non-starter from the user-experience perspective. Should we continue to use saved-object management for this?
2. Are we alright with the existing behavior where the tags are only included in the export in some situations?
3. Are we strictly opposed to listing the tags in saved-object management?

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[kobelb]

/cc @alexfrancoeur @MichaelMarcialis @ryankeairns

[ryankeairns]

We also discussed having that setting (to include related) on by default during the export process. Perhaps that could be even more explicitly written as 'include tags' or 'include metadata'.

It's an assumption, but I feel like this would meet expectations. We know tags as saved objects, internally, but conceptually that is probably not what people are expecting.

[kobelb]

A large part of me thinks this awkwardness is because of our term "saved objects". If instead of "saved objects" we called them "Kibana resources", "Kibana entities" or "Kibana data", would we still have these same concerns? In my mental model, a tag is still a Kibana resource/entity/data, it just happens to be used in a particular manner.

[ryankeairns]

I hear ya, we could both be right, hard to say :)

At the end of the day, we want to have a simple, single-click sort of thing. Perhaps we could offer a stripped down interface focused on exporting data and avoid all the nomenclature confusion.

[kobelb]

At the end of the day, we want to have a simple, single-click sort of thing. Perhaps we could offer a stripped down interface focused on exporting data and avoid all the nomenclature confusion.

Agreed. When I've gone through a similar exercise in my head it's ended up roughly looking like saved-object management though because I figured we should be listing the "things" that we're going to export before doing so.

[pgayvallet]

As already mentioned in our recent SO export discussions, I think the saved object management section is just doing too much at the moment. We mixed privileged features such as export to space, and 'backup / maintenance' features such as the ability to export/import, aka dump/load, or manually edit some SO's data.

In my honest opinion, the existing saved objects management section should become something that is explicitly only ('true') admin/expert level level. That would only include (the current version of) import/export + delete + edit. All, non maintenance related features, should move elsewhere.

We would have another page with features such as export to space, copy to space and so on. Ideally, as already discussed, these 'non-expert' (understand: safe) features should probably (imho) also be made available from each individual apps.

Now for the specific issue we are discussing about: I really agree with @kobelb observation about 'SO' vs 'resources. I think that mid term, SO export/import should display ALL of Kibana's data, and by all I mean even hidden types, and, ideally, data that are not technically saved objects. This page needs to be a way to backup your Kibana data. All your relevant Kibana data. Period. This will be required for the 8.0 multi-tenancy meta issue anyway Which is why I think we need to keep the tag SO type visible, and exportable, from there (with the current behavior: exporting tag only export the tag object, to assigned objects)

That being said, I also think that we need to dissociate 'dump/backup' export (main use case being: I want to export all the things because 8.0 is removing multi tenancy and I need a need a way to export all my stuff, or, I want to backup my data before another upgrade, or I need to move my Kibana data to another ES cluster), versus the 'move some specific stuff between env' use case. AKA: I created some dashboards in staging with the prod-ready tag, an I now want to export all these objects to production.

This is currently possible via SO management: I filter by tag:prod-ready, and I export my objects. This already works. Now, would, from a design or product point of view, be a better user experience to also allow this 'export by tag' workflow to be accessible from the tag management section? If so, I'm not opposed per said to add this alternative, use case specific, workflow. But we have to be aware that this will set a precedent, as that would be, to my knowledge, the first time we allow to create export files from elsewhere than SO management. If we go down that road, shouldn't we also allow to export dashboards or visualizations from their respective applications?

[ryankeairns]

If I'm following correctly, it seems we're tracking toward adding tags to the SO view; however, we're doing so under a newly proposed admin context. Further out, we should continue exploring the notion of separating out copy and import/export capabilities.

Coupled with the upcoming user profiles, this also has me thinking about how things might fit together (or, more accurately, be separated out) once those capabilities exist. For example, does a standard user need access to Stack Management? If so, which parts? and/or does some of that functionality move out into applications (as Pierre suggests)?

For example, selecting a set of dashboards or visualizations and tagging them is a likely use case, one which may negate the need for a standard user to access Stack Management > Tags.

So many thoughts :) It seems we can arrive at a short term path forward, but we ought to simultaneously consider our long term vision.

[MichaelMarcialis]

We also discussed having that setting (to include related) on by default during the export process. Perhaps that could be even more explicitly written as 'include tags' or 'include metadata'.

Yes, the mockups default to having those options checked during the export process, as shown in the following screenshots.

Export Tag from Tag Management Page

Export Saved Object from Saved Object Management Page

Now, would, from a design or product point of view, be a better user experience to also allow this 'export by tag' workflow to be accessible from the tag management section? If so, I'm not opposed per said to add this alternative, use case specific, workflow.

I believe we should be offering export options from both tag and saved object management pages, as shown in the mockups (and screenshots above).

If I'm following correctly, it seems we're tracking toward adding tags to the SO view;

Earlier mockups explored this direction. After reviewing the designs and discussing further, the idea was scrapped due to concerns about tag management discoverability from @alexfrancoeur. Just wanted to mention that before we make any decisions to retread that direction.

[kobelb]

I agree that we should be treating saved-object management as an "admin level" feature, and we shouldn't be only exposing the ability to copy/share to space here. However, I'm not opposed to us continuing to allow users to copy/share to space here, as long as it doesn't introduce additional complexity.

Allowing saved-objects to be exported and imported has some complexities imposed by the Kibana privileges model that I failed to elaborate on previously, as my immediate concern was exposing tags in saved-object management... Kibana's saved-objects are related to each other in an increasingly complicated manner. For example, we have Dashboards that can reference Visualizations that can reference Index Patterns; Dashboards can reference other Dashboards; and each of these saved-objects can be tagged:

Behind the scenes, when we grant the user the "Dashboard all" privilege, we grant the user the ability to CRUD the Dashboard and read all of the related saved-objects, so that they can effectively create and view dashboards:

This privileges model doesn't impose any issues for export, as the user is able to read the dashboard and all of the related saved-objects. However, it does cause issues on import. If the user is only able to create and update some of the saved-objects, they won't be able to import a Dashboard and all of its related saved-objects. We can handle this by either failing the import entirely or partially through, neither of which is great.

When faced with this dilemma when dealing with saved-object management's import/export, we decided that when a user was granted all access to saved-object management, they get CRUD access to every single saved-object that can be imported/exported.

This allowed us to side-step this complexity and allow users a more seamless import/export process. We can take a similar approach with tag management; however, that would result in this UI being an "admin level" feature. This is problematic because we only allow for tags to be created using tag management, and I don't think we want to severely restrict the users who can create tags.

[joshdover]

This is problematic because we only allow for tags to be created using tag management, and I don't think we want to severely restrict the users who can create tags.

Not sure I'm following this last part. Users can create tags inline when assigning them to an object, such as in Dashboard's save modal. If we adopted a similar approach as SOM to the Tag Management UI, couldn't we still give users write access to Tags without giving them write access to all of the Tag Management UI?

[alexfrancoeur]

I've been meaning to get to this issue all week. Let me start by providing some thoughts on the original questions and work my way down. Sorry in advance for the length of this.

Where should we allow users to export/import all of their Kibana specific data? Requiring users to use the dashboard app to export/import dashboards, the visualization app to export/import visualizations, etc. seems like a non-starter from the user-experience perspective. Should we continue to use saved-object management for this?

I think we should have SO management or the equivalent for import / export (backed by the appropriate API's).

Are we alright with the existing behavior where the tags are only included in the export in some situations?

I think we should be consistent as possible, looking at your awesome brah example (😆 ) there seems to be a clear gap in expected functionality. SO management (as it stands today) should probably have an option to export the tags themselves.

Are we strictly opposed to listing the tags in saved-object management?

I'd be fine with adding here as well. It might be a little strange, but if you're able to click into a tag and say export all saved objects with that tag - I think it's fine. When we were originally defining requirements, that's how I had thought we'd approach exporting by tag.

A large part of me thinks this awkwardness is because of our term "saved objects". If instead of "saved objects" we called them "Kibana resources", "Kibana entities" or "Kibana data", would we still have these same concerns? In my mental model, a tag is still a Kibana resource/entity/data, it just happens to be used in a particular manner.

Long live Kibana assets in 8.0? But seriously, let's consider it.

If we go down that road, shouldn't we also allow to export dashboards or visualizations from their respective applications?

I think that's worth exploring, especially as we start walking towards that OLS route. If I have access to certain things in multiple spaces, should I really need permissions to SO management to copy them over? Also, I pretty much agree with everything @pgayvallet said in this comment.

If I'm following correctly, it seems we're tracking toward adding tags to the SO view; however, we're doing so under a newly proposed admin context. Further out, we should continue exploring the notion of separating out copy and import/export capabilities.

Short term, it feels like we should have both. Filter by tag to export, and export tag and all related objects. I know we went back and forth on this a bit, but it's beginning to feel necessary.

So many thoughts :) It seems we can arrive at a short term path forward, but we ought to simultaneously consider our long term vision.

Yes, I agree. And I think the general theme of content management will play into this as well. I've been hearing some interesting ideas on the topic lately.

Earlier mockups explored this direction. After reviewing the designs and discussing further, the idea was scrapped due to concerns about tag management discoverability from @alexfrancoeur. Just wanted to mention that before we make any decisions to retread that direction.

In addition to discoverability, you may not need access to all that SO management has too offer to create, modify and assign tags.

I agree that we should be treating saved-object management as an "admin level" feature, and we shouldn't be only exposing the ability to copy/share to space here. However, I'm not opposed to us continuing to allow users to copy/share to space here, as long as it doesn't introduce additional complexity.

+1, and +1 to stick figure diagrams.

When faced with this dilemma when dealing with saved-object management's import/export, we decided that when a user was granted all access to saved-object management, they get CRUD access to every single saved-object that can be imported/exported.

I understand why we do this, but it's always felt wrong to me. Especially as once we added feature controls.

To me, it sounds like there is one immediate decision. Do we add tags to SO management and treat as any other SO. I lean towards yes while still keeping the functionality we built with tags and tag management. It also seems that we may want to explore this approach for other types of SO's, which makes sense to me. SO's are also evolving in complexity and there are more "things" that are kind of SO's but not really, and it's probably time for a rebrand, SOv2 (I've heard whispers of this) and rethinking how we handle import / export across Kibana.

[kobelb]

I understand why we do this, but it's always felt wrong to me. Especially as once we added feature controls.

Fair enough :) If we want to change the privileges model around saved-object management, we can investigate that. Whether or not we change the saved-object management privileges model or add import/export to tag management with a different privileges model, we'll have to figure out how to make import/export behave correctly when the user is only partially authorized. Until then, we can continue to rely on the ability to import/export using saved-object management as long as we can list tags here, and it sounds like we've come to an agreement that this is a tolerable experience.

[kobelb]

Thanks, everyone for participating in these conversations. It's been quite productive in determining the immediate path forward and it's clarified the future direction.

I've attempted to summarize the takeaways below, and unless there's an objection this issue will be closed in 1 week on Monday, November 30th 2020:

• We're alright listing tags on saved-object management for now and continuing to use this as the UI to import/export everything.
• Renaming "saved objects" to Kibana assets would clarify some of our concerns about "what is a saved-object", and is an option we should explore
• Saved-object management privileges model should potentially be revisited to no longer grant the user privileges to all saved-objects. At a minimum, it's not a privileges model that we should replicate everywhere, particularly with tag management
• We should continue to have tag management and it shouldn't follow the saved-object management privileges model . Supporting import/export saved-objects using tag management should be added in the future; however, it requires us to address the complications with partial import/export
• We should provide copy-to-space and share-to-space features outside of saved-object management in the future

[alexfrancoeur]

Overall, this looks good to me 👍

One question that remains for me is the behavior of exporting a tag from the SO management UI. I would expect that we have a similar experience as a dashboard with referenced objects. If I click into a tag in SO management, exporting would result in all dashboards, visualization and index patterns either tagged or related to those that are tagged. Right now, even with include related objects switched on - I believe we're only exporting the tag. Maybe this is a bug that can be addressed separately, but it's probably worth touching upon expected behavior.

[kobelb]

Right now, even with include related objects switched on - I believe we're only exporting the tag.

This is true, and it's a result of the way that the relationships are being modeled... Currently, relationships are modeled by having a saved-object have references to other saved-objects. This reference is in one-direction and is represented by the direction of the arrow in the following diagram.

Dashboards have references to visualizations, visualization have references to index-patterns, and all of these saved-objects have references to tags. When you export a saved-object and choose "include related objects" the export will include all outbound references. For example, when you export a Dashboard it will export the Visualizations, Index-Patterns and Tags:

However, when you export a Visualization and choose to "include related objects", it will export the referenced Index-Patterns, Tags, but not the Dashboards:

And when you export a Tag, all you get is the tag even when choosing "include related objects":

If we were to borrow the parent/child terminology that we use in the relationships flyout, we should really rename "include related objects" to "include child objects".

The only way to export all tagged saved-objects currently is to use the "tag filter" to filter the saved-objects to be for a specific tag, and then we can use Export with "include related objects" to get all of the saved-objects: