-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fleet] Speed up fleet setup #96026
Comments
Pinging @elastic/fleet (Team:Fleet) |
@kevinlog FYI |
Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt) |
I did some benchmarking and this function is where most of the time goes. I removed
I'm not sure why endpoint is installed at all, since it doesn't seem to be used. Maybe we can indeed remove it. Another thing we could do is to call |
It will be great, but currently it's not really doable as we need a lot of permission that the |
Following offline discussion with @kevinlog, it seems that we can indeed remove Endpoint from the list of default integrations. |
@jen-huang Agree, I think we don't need the endpoint package anymore by default. It would be good to document the reason here as we might come back to this. |
Another solution that we can do is improve the UX of the first load itself, similar to the elevator waiting time problem. Instead of only showing a spinner on first load we can explain to the user what we are doing (the initial setup) and change the spinner with a progress bar, to remove that feeling of the UI being frozen. |
Alright, a small update regarding this. These are the benchmarks for all different steps for the default packages. I tried my best to represent what happens in parallel and what sequentially
Most of the time is spent in the last two packages, installing kibana assets, pipelines and templates. Take into account that the
One thing that was proposed was to call The second group happens sequentially because there are dependencies between data streams, pipelines, templates and transforms. We could potentially create a dependency graph of which transform depends on which template, who depends on which pipeline, who depends on with ILM, and install those things walking the graph, but I'm not sure how easy it is, or if the dependencies are clearly stated in the package's manifests. This might be beneficial for packages with lots of pipelines/templates/transforms. Potentially can collapse the timing of the So, I think for now we can remove |
Thanks for putting together all this data, this is great! One thing that caught my attention is the time it takes to install all the pipelines and templates. I'm aware there are a few pipelines and templates to be loaded but the number seems higher than what I would expect. It would be interesting to know what the time of just loading a single pipeline into ES with the direct ES call and how much it is when going through our and Kibana code. Basically my question is, is it ES that is slow or some code in between? |
@ruflin good questions. I haven't digged into each function, and to be fair all these timings are running against a local ES instance, so that might affect the speed as well. I will take a look at what the functions are doing nonetheless |
When the user visits Fleet for the first time, Fleet runs setup to install some required packages and many setup. This currently takes (too) long. When setup was initially built, it contained very few steps but packages grew over time and also the required packages. Along the way also things change. We should evaluate if the way setup is built is still the right way to go today and if things can be optimised / changed. Some ideas:
cc @jen-huang
The text was updated successfully, but these errors were encountered: