[Integration][Datadog] Add metrics availabity data on service and hosts #867
Conversation
phalbert
changed the title
phalbert
changed the title
phalbert
changed the title
|
A few other improvements I could make. cc @PeyGis , @Tankilevitch
|
phalbert
force-pushed
the
PORT-9179-datadog-add-metrics-on-service-and-maybe-hosts
branch
from
5fa6fe1
to
dd6dbf9
Compare
There was a problem hiding this comment.
I think the implementation is too specific for the user use-case.
We have multiple resources touched to support that capability. And I think they needs to be separated.
Service
Host
Dashboard
Dashboard-Metrics
IMO we need to approach it by taking advantage of port builder capabilities, relations, mirrors, aggregation properties.
Service -> Dashboard Metrics or Service -> Dashboard -> Metrics
Or
Metrics -> Service
This change now adds available metrics as relational array of metric ids that are available for a host or service |
Co-authored-by: Matan <51418643+matan84@users.noreply.github.com>
Co-authored-by: Matan <51418643+matan84@users.noreply.github.com>
Co-authored-by: Matan <51418643+matan84@users.noreply.github.com>
Co-authored-by: Matan <51418643+matan84@users.noreply.github.com>
…mits using a semaphore This change refactors the `_fetch_metrics_for_services` method to introduce concurrency using `asyncio.gather`. Previously, the fetch operations were performed sequentially, even though the code utilized `asyncio`. By fetching metrics concurrently, we aim to improve performance by potentially overlapping delays caused by network latency or Datadog API response times. However, we still maintain control over the rate of requests using a semaphore to ensure we stay within Datadog's API rate limits. Key changes: * Fetch operations are now performed concurrently within each service iteration using `asyncio.gather`. * A semaphore is used to control concurrency and prevent exceeding Datadog's rate limits. * Error handling and further optimizations may be considered in future iterations.
integrations/datadog/client.py
Outdated
| Returns: | ||
| The metric name, or None if no metric name is found. | ||
|
|
||
| Examples: | ||
| - "sum:container.memory.usage" -> "container.memory.usage" | ||
| - "avg:system.cpu.used{service:my-service}" -> "system.cpu.used" | ||
| """ | ||
| match = re.search(r"(?:\w+:)?(\w+\.\w+\.\w+)", query_string) |
There was a problem hiding this comment.
why we are doing that? as eventually I do care about the avg in avg:system.cpu.used?
is there a different naming terminology between avg:system.cpu.used to system.cpu.used in datadog?
There was a problem hiding this comment.
this was just metadata info. we still have the full metric name in the title of the entity and query property
{
"identifier": "avg:system.mem.used/service:mayicard-mayicore/env:staging",
"title": "avg:system.mem.used{service:mayicard-mayicore,env:staging}",
"icon": null,
"blueprint": "datadogServiceMetric",
"team": [],
"properties": {
"query": "avg:system.mem.used",
"series": [],
"res_type": "time_series",
"from_date": "2024-08-16T09:15:34Z",
"to_date": "2024-08-16T09:25:34Z",
"metric_name": "system.mem.used",
"env": "staging"
},
"relations": {
"service": "mayicard-mayicore"
}
}
Co-authored-by: Tom Tankilevitch <59158507+Tankilevitch@users.noreply.github.com>
…s' of https://github.com/port-labs/ocean into PORT-9179-datadog-add-metrics-on-service-and-maybe-hosts
Tankilevitch
deleted the
PORT-9179-datadog-add-metrics-on-service-and-maybe-hosts
branch
Description
metricandserviceMetricto get specified metricsType of change
Please leave one option from the following and delete the rest:
Screenshots
API Documentation
Provide links to the API documentation used for this integration.