[Docker] Add dimensions to metrics data streams, except for events

[constanca-m]

What does this PR do?

Add dimensions to metrics data streams, except for events.

Details

For all data streams, there exists these set of dimensions:

- external: ecs
  name: cloud.account.id
  dimension: true
- external: ecs
  name: cloud.provider
  dimension: true
- external: ecs
  name: cloud.region
  dimension: true
- external: ecs
  name: cloud.availability_zone
  dimension: true
- external: ecs
  name: cloud.instance.id
  dimension: true

In case it is running on cloud.More info on this can be found in #5193 (comment).

The dimensions set for each data stream are container.id and service.address, since the ID is unique per Docker daemon. Some data streams did not hold value for container.id, so a similar field was selected (see below).

Container: No change in the number of documents with TSDB disabled v enabled

CPU: No change in the number of documents with TSDB disabled v enabled

Diskio: No change in the number of documents with TSDB disabled v enabled

Event: Does not have any metrics

Healthcheck: No change in the number of documents with TSDB disabled v enabled

Image: No change in the number of documents with TSDB disabled v enabled. Instead of container.id, we use docker.image.id.current.

Also changed:

To:

Info: docker.info.id instead of container.id is used (last one holds no value).

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• No change in the number of documents after enabling TSDB

How to test this PR locally

1. Clone this branch.
2. Update the service registry to include this version of Docker and enable TSDB.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-06-05T13:47:31.128+0000

• Duration: 20 min 51 sec

Test stats 🧪

Test	Results
Failed	0
Passed	38
Skipped	0
Total	38

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (0/0)	💚
Files	100.0% (0/0)	💚 3.846
Classes	100.0% (0/0)	💚 3.846
Methods	96.667% (29/30)	👍 8.899
Lines	100.0% (0/0)	💚 13.666
Conditionals	100.0% (0/0)	💚

[constanca-m]

Hey @gizas , I added agent.id (dimension as well) to all metrics data streams. Reasoning can be found in the thread following this comment #5193 (comment)

[tetianakravchenko]

to align with #5193 (comment) I think that host.name should be added. I see that cloud.* are not defined in docker package, so lets skip it.

[tetianakravchenko]

this data_stream does not contain container.id or it is just missing in fields definition?

[constanca-m]

Exactly, container.id has no value for this data stream

[tetianakravchenko]

container.id for image data_stream is it not set as a dimension on purpose?

[constanca-m]

Yes @tetianakravchenko The field doesn't hold any value for that data stream, that's why docker.image.id.current is a dimension.

[constanca-m]

to align with #5193 (comment) I think that host.name should be added. I see that cloud.* are not defined in docker package, so lets skip it.

I think service.address should be enough for different hosts. What do you think @tetianakravchenko ?

[tetianakravchenko]

to align with #5193 (comment) I think that host.name should be added. I see that cloud.* are not defined in docker package, so lets skip it.

I think service.address should be enough for different hosts. What do you think @tetianakravchenko ?

from my understanding there should be either the ip address or the socket, which might not be unique enough, if we run in different cloud providers/different cloud accounts instance with the docker service running, isn't it?

in the sample file -

integrations/packages/docker/data_stream/info/sample_event.json

Lines 24 to 25 in c8e5b36

	"service": {
	"address": "/var/run/docker.sock",

, so it might be the same for different instances, so we need to add some information about the host to separate data from different hosts

[constanca-m]

from my understanding there should be either the ip address or the socket, which might not be unique enough, if we run in different cloud providers/different cloud accounts instance with the docker service running, isn't it?

in the sample file -

integrations/packages/docker/data_stream/info/sample_event.json

Lines 24 to 25 in c8e5b36

	"service": {
	"address": "/var/run/docker.sock",

, so it might be the same for different instances, so we need to add some information about the host to separate data from different hosts

You're right. In my example they were different, but now that I think of it, they were only different because I made them be. I will fix it.

[elasticmachine]

Package docker - 2.5.1 containing this change is available at https://epr.elastic.co/search?package=docker