Send metrics via any protocol in the OTLP Registry

[kasparkivistik]

Please describe the feature request.
The idea is to make it available for any consumer of the micrometer-otlp-registry to provide their own metrics sending implementation. The OTLP Registry can still only support HTTP/protobuf as it does today but, for example, its implementation can be described by a public interface.

Rationale
The purpose of the change is to allow for metrics sending to be more flexible on the consumer side. I'm not advocating any changed to the OTLP protocols, rather my intention would be to open metrics sending up.
And example on my end would be that I desire to push all metrics to a Kafka topic before they are sent via HTTP/protobuf to OTLP collector. This way I have greater control over my metrics that might be coming from a number of microservices.

Additional context
Slack discussion: https://micrometer-metrics.slack.com/archives/C01C4Q8FHC2/p1731676707725909

[jonatan-ivanov]

People who give this issue a 👍🏼 or 🚀, what is your use-case?

Right now, the use-case seems to be sending the data through Kafka which is not supported by the specs if I remember correctly. But I'm curious if people have other use-cases or you are co-workers and has the same use-case.

[kasparkivistik]

@jonatan-ivanov they are co-workers 😅

For the time being I did a local workaround for the issue. Let's see what the community might think of it

[schewara]

Personally I am not a big fan of adding too much complexity at this level, when it might be better placed on the collector level, but switching from the prometheus-registry to the opentelemetry-registry without losing any functionality by allowing to configure a prometheus exporter would allow for a much simpler migration path.

We are currently migrating away from prometheus metrics to otlp metrics and being able to switch between prometheus pull to otlp push on a switchover would be extremly helpful and would allow for fast fallback in case of problems.

[jonatan-ivanov]

@schewara I'm not sure how your use-case is connected. You can use the Prometheus and the OTLP registry simultaneously, you can use as many registries as you want. If you use Spring Boot just add both dependencies, if you do things manually, register both prometheus and otlp registries into one CompositeMeterRegistry and use the composite to record data.

[schewara]

@jonatan-ivanov You are absolutely correct.
I was thinking more on a lower level, trying to avoid any of these error prone compatibility and normalization issues between the different formats, where it can make a difference which of the respective library created the actual metrics including metadata and of course the additional maintenance effort for both libraries instead of a single one.

For more background you can checkout the following docs

• https://opentelemetry.io/docs/specs/otel/compatibility/prometheus_and_openmetrics/
• https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/translator/prometheus