Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus exporter emits invalid characters in target_info metric #3563

Closed
Jdsleppy opened this issue Nov 30, 2023 · 1 comment
Closed

Prometheus exporter emits invalid characters in target_info metric #3563

Jdsleppy opened this issue Nov 30, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@Jdsleppy
Copy link

Describe your environment

$ python
Python 3.11.2 (main, Feb 17 2023, 09:28:16) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.platform()
'Linux-6.5.6-76060506-generic-x86_64-with-glibc2.28'

Prometheus version 2.46.0

opentelemetry-distro = "0.42b0"
opentelemetry-exporter-prometheus = "==0.42b0"
...

Steps to reproduce
Allow autoinstrumentation to set up the Prometheus metrics exporter

OTEL_SERVICE_NAME=myservice
OTEL_METRICS_EXPORTER=prometheus
OTEL_EXPORTER_PROMETHEUS_PORT=9464
OTEL_EXPORTER_PROMETHEUS_HOST=0.0.0.0

Verify that metrics are exposed:

$ http http://127.0.0.1:9464 | head
# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 1.025664e+06
python_gc_objects_collected_total{generation="1"} 277619.0
python_gc_objects_collected_total{generation="2"} 27443.0
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
...

Have Prometheus pull metrics from port 9464.

What is the expected behavior?
I expect all the metrics to be pulled into Prometheus without error.

What is the actual behavior?
Prometheus errors when parsing the target_info metric

expected equal, got "." ("INVALID") while parsing: "target_info{service."
$ http http://127.0.0.1:9464 | grep target_info
# HELP target_info Target metadata
# TYPE target_info gauge
target_info{service.name="myservice",telemetry.auto.version="0.42b0",telemetry.sdk.language="python",telemetry.sdk.name="opentelemetry",telemetry.sdk.version="1.21.0"} 1.0

Additional context
The regex given by Prometheus to describe valid label names does not allow for . characters like this exporter emits (service.name="...").

Support for target_info was added in #3073 and there is no environment variable to disable it.

@Jdsleppy Jdsleppy added the bug Something isn't working label Nov 30, 2023
@Jdsleppy Jdsleppy changed the title Prometheus exporter emits invalid characters in metric labels Prometheus exporter emits invalid characters in target_info metric Dec 1, 2023
@Jdsleppy
Copy link
Author

Duplicate of #3571 and is on the way to being fixed.

@Jdsleppy Jdsleppy closed this as not planned Won't fix, can't repro, duplicate, stale Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant