Skip to content

Commit

Permalink
Add Dashboard exercise
Browse files Browse the repository at this point in the history
  • Loading branch information
Starefossen committed Sep 1, 2024
1 parent 3691550 commit 3a30f9b
Show file tree
Hide file tree
Showing 11 changed files with 100 additions and 4 deletions.
4 changes: 2 additions & 2 deletions workshop/02-setup.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Lab 2: Local setup
# Exercise 2: Local setup

In this section, we will set up the environment for the workshop. We will use Docker and Docker Compose to run the services required for the workshop.

Expand Down Expand Up @@ -34,4 +34,4 @@ docker-compose pull

Now that you have the repository cloned and the Docker images pulled, you are ready to start the workshop. In the next section, we will cover the basics of instrumentation with OpenTelemetry.

Continue to [Lab 3: Instrumentation with JAVA](./03-instrumentation.md)
Continue to [Exercise 3: Instrumentation with JAVA](./03-instrumentation.md)
2 changes: 1 addition & 1 deletion workshop/03-instrumentation.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Lab 3: Instrumentation with JAVA
# Exercise 3: Instrumentation with JAVA

In this section, we will instrument a simple Java application with OpenTelemetry. We will use the OpenTelemetry Java SDK to add tracing to the application.
2 changes: 1 addition & 1 deletion workshop/04-grafana.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Lab 4: Exploring data in Grafana
# Exercise 4: Exploring data in Grafana

In this section, we will explore the OpenTelemetry data (logs, metrics, and traces) in Grafana. We will use the Grafana dashboard to visualize the data and understand the performance of the application.

Expand Down
96 changes: 96 additions & 0 deletions workshop/05-dashboard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Exercise 5: Create a Dashboard

In this section, we will create a dashboard to visualize the data collected for our OpenTelemetry instrumented application.

## Prerequisites

Make sure you have started the services required for the workshop. If you haven't done so, follow the instructions in the [Local setup](./02-setup.md) section.

## Create a dashboard

1. Open [localhost:8080/grafana](http://localhost:8080/grafana) in your browser.
2. In the left menu, and click on the [Dasbhaords](http://localhost:8080/grafana/dashboards) item.
3. Click on the [New](http://localhost:8080/grafana/dashboard/new) button to create a new dashboard.

![Create a new dashboard](./assets/05-dashboard-new.png)

Start by clicking the gear icon in on the top navigation bar. This will open the settings for the dashboard. Give the dasshboard a descriptive name, such as `OpenTelemetry Workshop`.

![Dashboard settings](./assets/05-dashboard-settings.png)

Click on the `Variables` tab and add a new variable. Name the variable `service` and set the type to `Query`.

In the Query Options section, set the following values:

- Data source: `Prometheus`
- Query type: `Label values`
- Label: `service`
- Metric: `traces_spanmetrics_latency_count`

You should see a list of services in the Preview values section to confirm that the query is working correctly.

Click on the `Apply` button to save the variable and click on the `Save dashboard` button in the top navigation bar to save your changes.

![Dashboard variables](./assets/05-dashboard-variable.png)

## Our first panel

Now we are ready to add some panels to our dashboard, start by clicking the prominent `+ Add visualization` button.

Let's start by visualizing span latency for our application.

Select the `Prometheus` data source, this will bring up the query builder that we will use to create our panel.

Select the following parameters:

- Metric: `traces_spanmetrics_latency_bucket`
- Label filters: `service` `=` `${service}`

Click on the `hint: add histogram_quantile` link to add the quantile function to the query.

!!! note
A histogram is a type of metric that samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.

The `histogram_quantile` function is built into the PromQL query language and calculates the selected quantile over time from a histogram.

:question: What histogram quantile are we using to visualize the span latency?

Click on the `Run query` button to see the results of the query.

![Dashboard panel](./assets/05-dashboard-query.png)

!!! note
:question: What is the prometheus query that we are using to visualize the span latency?

<details>
<summary>Hint</summary>

The PromQL query is displayed below the query builder filters. It should look like this:

```promql
histogram_quantile(0.95, sum by(le) (rate(traces_spanmetrics_latency_bucket{service="$service"}[$__rate_interval])))
```
</details>

Whenever you cange the service variable, the panel will update to show the span latency for the selected service. Try it for yourself!

Now let's add the other quantiles to the panel. Either click ont the `Add query` button below the query builder or click on the `Duplicate query` button to create a copy of the current query and change the quantile value.

Add the following qunatiles: `0.5`, `0.9`, and `0.99`. Click on the `Run query` button to see the results.

Lets add some better legends for the queries. Click on the `Options` tab and set the `Legend` field to the following values:

- `0.5`: `quantile50`
- `0.9`: `quantile90`
- `0.95`: `quantile95`
- `0.99`: `quantile99`

On the right hand side of the panel, you can set the panel title to `Latency for ${service}`.

The last thing we are going to do is enable exemplars in our panel. Exemplars are a powerful feature of Prometheus that allow you to see the exact trace that contributed to a specific data point in order to jump directly to trace view.

In the `Options` tab for one of the queries, enable the `Exemplars` option.

![Dashboard panel queries](./assets/05-dashboard-queries.png)

Click ont eh `Apply` button to save the changes to the panel and you should see the first panel in your dashboard. Click on the floppy disk icon in the top navigation bar to save the dashboard.
File renamed without changes.
File renamed without changes.
Binary file added workshop/assets/05-dashboard-new.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/assets/05-dashboard-queries.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/assets/05-dashboard-query.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/assets/05-dashboard-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/assets/05-dashboard-variable.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 3a30f9b

Please sign in to comment.