Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Advanced Lens documentation #81780

Open
KOTungseth opened this issue Oct 27, 2020 · 14 comments · Fixed by #86821 or #97142
Open

Advanced Lens documentation #81780

KOTungseth opened this issue Oct 27, 2020 · 14 comments · Fixed by #86821 or #97142
Labels
documentation Feature:Lens impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. Team:Docs Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@KOTungseth
Copy link
Contributor

KOTungseth commented Oct 27, 2020

As we make Lens more complex, there will be more areas that would benefit from more technical descriptions.

For example:

  • How is the moving average calculated?
  • How are pie chart slices sized exactly?
  • How is the percentage chart calculating its percentages?
  • How are the data summaries sourced?
  • When is a field shown as "empty"?
  • What operation can you do with what types of field?
  • How the x axes of multiple layers merged in a chart?

For all of these things there are obvious, intuitive answers, which are right most of the time, but as with almost everything, there's depth to them and it would be cool to have an official answer for them (right now most of these are answered in blog posts or forum questions, but it's not really structured)

@flash1293 @wylieconlon @timroes @mbondyra

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-docs (Team:Docs)

@flash1293
Copy link
Contributor

More things similar to that:

  • Whats "granularity" for histograms?
  • How does the "auto" date histogram work?
  • Maybe a description what "group by this field first" does (on Elasticsearch aggregation level)

@wylieconlon
Copy link
Contributor

It seems easiest to discover some of this documentation directly in the product, but we don't currently have a design for this. This is pretty closely related to the idea of in-product help text about operations.

@wylieconlon
Copy link
Contributor

After discussing some of these ideas with @KOTungseth and @MichaelMarcialis we have identified basically three possible types of reference content for Lens, mixing both in-product and external documentation.

Type Example How is it presented?
Inline help that is tied to a form element Whats "granularity" for histograms? Best presented in-product, such as a flyout or popover
Inline help that is not part of a form When is a field shown as "empty"? Can be presented in-product or using external docs
Higher-level help that explains multiple parts of the UI What operation can you do with what types of field? Mostly done using external docs

I think the next steps are:

  1. Identify the things that need documentation- should we use this github issue or a spreadsheet @KOTungseth?
  2. Categorize and decide what belongs in-product and what belongs external
  3. Decide how to present any in-product docs @MichaelMarcialis (modal, flyout, popover, etc?)

@KOTungseth
Copy link
Contributor Author

KOTungseth commented Nov 20, 2020

We've identified the following things need documentation.

Content Target location Notes
How are data summaries sourced? Kibana Guide Devs to write ( #85575 )
What's granularity in histograms? Lens UI Devs to write for popover testing
How does the auto date histogram work? Lens UI Devs to write for popover testing
What is each fitting function doing? Kibana Guide Devs to write later
Describe what "group by this field first" does (on es agg level) Lens UI Talk to Michael about putting animations in the product. We should not try to describe this with more text.
Limitations Kibana Guide To include JSON input, percentages, pie charts, and x-axes of multiple layers merged in a chart
When do fields show as empty? Tooltip on Lens UI #83321
How is the moving average calculated? Kibana Guide Targeted for 7.12
What operations can you do with what types of fields? TBD Do we need this?

@wylieconlon
Copy link
Contributor

wylieconlon commented Nov 23, 2020

It sounds like @mbondyra or @dej611 will write the first few items that we want to test out in a popover.

@MichaelMarcialis
Copy link
Contributor

For visibility, here are the design concepts that I shared with @wylieconlon and @KOTungseth in our last meeting.

@dej611
Copy link
Contributor

dej611 commented Jan 28, 2021

Migrating here as single source of truth from now on.

Content Topic/Section Target location Issue/PR Source Notes
How are pie chart slices sized exactly? Visualizations / Pie Chart Kibana Guide / Lens Limitations -- -- --
How is the percentage chart calculating its percentages? Visualizations / XY Chart Kibana Guide / Lens Limitations -- -- --
How the x axes of multiple layers are merged in a chart? Visualizations / XY Chart Kibana Guide / Lens Limitations -- -- --
What is each fitting function doing? Visualizations / XY Chart Lens UI #86821 -- --
How can I configure the axis position? Visualizations / XY Chart Lens UI? -- -- Probably addressed in the UI should be enough?
Can I show value labels for my charts? Visualizations / XY Chart Lens UI? / Lens Limitations #89615 #82954 There's currently some help in the UI
Can I sort by multiple columns? Visualizations / Datatable Kibana Guide / Lens Limitations #89615 -- --
How are IPs sorted? Visualizations / Datatable Kibana Guide / Lens Limitations -- #85431 --
Can I use a custom color palette? Visualizations Kibana Guide #95217 -- --
Is it possible to reduce/increase the number of suggestions in Lens? Visualizations Kibana Guide #89615 #85518 --
How is the moving average calculated? Data Aggregation in Lens Lens UI #86821 -- --
How are the data summaries sourced? Lens UI Lens UI #85575 -- --
What does "group by this field first" do (on Elasticsearch aggregation level)? Data Aggregation in Lens Lens UI? -- -- Talk to Michael about putting animations in the product. We should not try to describe this with more text.
What is the “Records” special field? Lens UI? Lens UI? -- #50420 --
What Elasticsearch functions are renamed in Lens? Lens UI Kibana Guide / Lens page -- -- Something like the new TSVB page with a "translation" page
How are empty (null or NaN) values affecting the moving average window computation? Data Aggregation in Lens Lens UI + Kibana Guide / Lens Limitations #86821 (Lens UI) -- --
How can I include documents without the field in the operation? Data Aggregation in Lens Kibana Guide / Lens Limitations #89615 -- --
What is the "Other" category Data Aggregation in Lens Lens UI?/ Kibana Guide / Lens Limitations #89615 -- --
What’s "granularity" for histograms? Data Aggregation in Lens Lens UI #86821 -- --
How does the "auto" date histogram work Data Aggregation in Lens Lens UI #86821 -- --
Is it possible to build a histogram using two date fields from the same index pattern? Data Aggregation in Lens Kibana Guide -- -- --
When is a field shown as "empty"? Lens UI Lens UI #86821 -- --
How does the last value work Data Aggregation in Lens Lens UI -- -- ES reference
Can the entire dashboard be exported as CSV? Share / Export / Reporting Kibana Guide / Dashboard -- -- Perhaps a dedicated Export page for Kibana can be useful to address this
What can (or can't) be exported? Share / Export / Reporting Kibana Guide / Dashboard or Lens Limitations -- -- Perhaps a dedicated Export page for Kibana can be useful to address this
Is there a way to automatically export using the Kibana API? Share / Export / Reporting Kibana Guide -- Kibana forum thread Perhaps a dedicated Export page for Kibana can be useful to address this
What happens with very large CSVs? Share / Export / Reporting Kibana Guide / Dashboard or Lens Limitations -- -- Perhaps a dedicated Export page for Kibana can be useful to address this
Are multiple layers exported in CSV? Share / Export / Reporting Kibana Guide / Dashboard or Lens Limitations -- -- Perhaps a dedicated Export page for Kibana can be useful to address this
How can I compare two dates? Data Aggregation in Lens Kibana Guide / Lens Limitations -- -- --
Is it possible to inspect the Elasticsearch query in Lens? Data Aggregation in Lens Kibana Guide / Lens Limitations #111601 -- --
Is it possible to pick/select color for bars or lines? Lens UI Lens UI? #68405 (Lens UI) / #89615 (Limitations page) -- Partially addressed by the PR (how to "clear" it).
Is it possible to sort dimensions in a chart? Visualizations / XY Chart Kibana Guide / Lens Limitations #89615 #86184 --
Is it possible to configure decimal point in number format in Lens Visualizations Lens UI? -- -- --
Is it possible to "split up" a metric visualization by terms? Visualizations / Metric Kibana Guide / Lens Limitations -- -- --
How does the "Order by" feature works in Top values? Data Aggregation in Lens Lens UI #90749 -- --
Why field X is shown as empty when there's data? Data Aggregation in Lens Lens limitations + Lens UI Lens UI: #85544 / Limitation page #89615 #89716 --
How to do Math calculations between the output of two series? Data aggregations in Lens Lens Tutorials #102430 Elastic forum thread --
Can I use saved searches in Lens? Data aggregations in Lens Kibana Guide / Lens Limitations #89615 Original discussion for #93600 --
Is it possible to use different index patterns in a single visualization? Data aggregations in Lens Kibana Guide / Lens Limitations #97142 Kibana forum thread --
Why is my field X missing from the fields list in Lens? Data aggregations in Lens Kibana Guide / Lens Limitations Part I (#97142~~), Part II (#105057)~~ Kibana forum thread --
How to handle gaps in time series visualizations? Visualizations / XY Chart Kibana Guide / Lens Limitations #97142 Kibana forum thread 1 Kibana forum thread 2 Kibana forum thread 3 Kibana forum thread 4 --
Is it possible to change the scale of the Y axis in a visualization? Visualizations / XY Chart Kibana Guide / Lens Limitations #97142 Kibana forum thread --
Is it possible to have more than one Y axis scale in visualization? Visualizations / XY Chart Kibana Guide / Lens Limitations #97142 Kibana forum thread --
Is it possible to show icons in datatable? Visualizations/Datatable Kibana Guide #105057 Kibana forum thread --
Is it possible to filter tooltip entries? Visualizations/XY Chart Kibana Guide/Lens Limitations -- Kibana forum thread --
How to isolate a single series in a chart? Visualizations/XY Chart Kibana Guide/Lens Limitations #111601 Kibana forum thread --
Can I pick a specific color for a specific term/category name? Visualization/XY Chart Kibana Guide/Lens Limitations -- Kibana forum thread, Kibana forum thread 2 --

@dej611 dej611 reopened this Jan 28, 2021
@KOTungseth
Copy link
Contributor Author

8-Jan-2021

  • Where should the Lens limitations live? We're going to need to reformat the docs.
    Kaarina to open a draft PR for reference content and Dashboard reformat, then team can push to PR.

  • What release should we plan to merge these changes?
    Reference content to target 7.12, additional UI content to target 7.13.

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app (Team:KibanaApp)

@flash1293
Copy link
Contributor

@dej611 @KOTungseth does it still make sense to keep this open?

@dej611
Copy link
Contributor

dej611 commented Jun 29, 2021

I'm using this issue to keep track of old and new Lens questions.
If closed I would have to create something similar to keep track of the same information.

@govindrai
Copy link

Hi folks, I've been looking for documentation on "Group by this field First". I haven't really been able to find anything in the docs. Could someone point me to it or post an official description of what that switch is supposed to do? Thank you!

@flash1293
Copy link
Contributor

Hi @govindrai , thanks for raising this. I think we actually don't document this well (@KOTungseth do you think we can get it into some of the existing guides?). A short explanation here:

"Group by this field first" is about the order of multiple grouping dimensions. Let's say you have an xy chart with a date histogram on the horizontal axis and a "top 5 values of x" as breakdown. By default, your whole dataset is grouped by the top 5 values first, so you will get the top 5 values of x for the whole data set, then see their trends over time. However if you uncheck "group by this field first" for the breakdown dimension, it means it will do the date histogram first (grouping your data into time intervals), and then afterwards apply the "top 5 values of x" grouping separately per time bucket. This means you don't get the top 5 values overall over time, you get the top 5 values per time interval (e.g. per day). So your chart could show many more than 5 different values of x because there might be a different top 5 for each day. The same applies in other cases (e.g. if you have two "top values" dimensions on different fields). In some situations there is no effect at all, e.g. if there are less than 5 values in total - as all of them will show up either way, the grouping order doesn't change the result.

See this example:

Top 3 geo.dest overall, then do the time grouping:
Screenshot 2022-10-19 at 10 31 35

Time grouping, then top 3 geo.dest per 3 hour interval separately:
Screenshot 2022-10-19 at 10 31 42

@stratoula stratoula added the impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. label Sep 6, 2023
@KOTungseth KOTungseth removed their assignment Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Feature:Lens impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. Team:Docs Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet