Discuss: How to retrieve the latest package

[jfsiii]

EPM currently does this locally by calling /search and returning the first value because it "knows" that how that endpoint works https://github.com/elastic/kibana/blob/74bf87721f70d0c71738c209613f290020fed100/x-pack/plugins/ingest_manager/server/services/epm/registry/index.ts#L44

It seems like the registry is best suited to answer what's the latest version. Can it provide a more explicit way to get that?

[ruflin]

To be more specific here, I assume this is to get latest package which is compatible with the current Kibana version? One thing missing today in the url is the kibana version which should always be passed to the /search endpoint.

One thing we could do is having a /packages/{package}/latest endpoint which would be dynamic. For example https://epr.elastic.co/package/aws/latest?kibana=7.9.1 would return the latest package for aws which is compatible with kibana:7.9.1. Would this solve the problem?

[ph]

@jfsiii Can we solve it in the registry vs EPM?

[jfsiii]

One thing missing today in the url is the kibana version which should always be passed to the /search endpoint.

Good point. Thanks for bringing that up. I'll open an issue and/or PR. I'll talk about it there, but I believe the kibana version is included in an HTTP header if use use the core client.

https://epr.elastic.co/package/aws/latest?kibana=7.9.1 would return the latest package for aws which is compatible with kibana:7.9.1. Would this solve the problem?

/cc @neptunian for more 👀 🧠 but, yes, that sounds great to me.

[neptunian]

I thought /search was modified to return the latest single package unless otherwise specified with the all option:

https://epr.elastic.co/search?package=nginx&all=true&kibana=7.3.0

[ruflin]

The Kibana issue to send it as a query param is here: elastic/kibana#64868

@neptunian I think there is one key difference between the two endpoints. The /search endpoint returns a short summary of a package. The */latest endpoint I would suspect to return the full info for a package like we have here: https://epr.elastic.co/package/aws/0.0.3/

[neptunian]

Ah, right.

That url looks fine to me. I was thinking of doing something like /packages/aws for EPM API and returning the version the user has installed. So we can also do /packages/aws/latest for latest. Currently the EPM API is still using the package-version in the routes and I think we should separate it like the above.

[ruflin]

@neptunian @jfsiii Is the conclusion that it is worth having such an API endpoint in the registry and I should follow up with a PR to get it in?

[neptunian]

I'm not sure where we need this endpoint atm. If all we need to do right now is answer the question of what is the latest version of a package given a package name, the search route is enough and EPM could create an endpoint using it for @paul-tavares elastic/kibana#66777.

[paul-tavares]

The need for Endpoint will be to get the version installed - which might actually be different from what I had posted in the use case on elastic/kibana#66777 . I say it this way (installed -vs- latest) because I don't yet clearly understand how installation of new package version.

The installation of a new package version may require data migrations - especially if updates to the packages will be done out-of-band with the version of Kibana -thus getting information on what is the latest package might not be desired approach (for quick context: we will use the version of the returned information to build a URL to #/integrations/endpoint-0.1.0/add-datasource so that a user can create a new Endpoint "policy")

[neptunian]

@paul-tavares EPM could provide that or you can query epm-packages saved objects to see what packages they have installed

[jfsiii]

@neptunian I've wanted this a few times now (installed packages, filter, etc) but wasn't sure of the endpoint. We use /packages for the available ones (from EPR). I'm not sure what to use for installed ones (our Saved Object)

[ruflin]

@paul-tavares @neptunian @jfsiii Is this still something needed? If not, I suggest we close the issue and reopen if we find a need for it.

[paul-tavares]

The need is still there - currently we (endpoint) are using a work around by pulling the list filtered by a category of security and then trying to find the endpoint package, which is not ideal. This also reminds me that with an increase in the number of packages, we might not even get the endpoint package back on the first page (and I don’t think we’re paving through 😦 will need to check (cc/ @kevinlog)

[neptunian]

@paul-tavares would querying the saved objects for the installed package meet your needs?
http://localhost:5601/saved_objects/_find?type=epm-packages&filter=epm-packages.attributes.name:endpoint

Or are you needing something that returns both latest available and installed?

[paul-tavares]

I realized that this Issue here is under the Package Registry, so I'm not clear if it is valid or not. Our needs are captured under Kibana issue that is opened - here: elastic/kibana#66777

I think our need is for the latest (supported on current Kibana version), so that we can build the URL to the EPM page for user to create a new endpoint package policy integration

[neptunian]

Cool, i'll close this one and we can move the conversation there.