Skip to content

Commit

Permalink
OpenCensus metric exporter bridge (#1444)
Browse files Browse the repository at this point in the history
* add OpenCensus metric exporter bridge

* Update bridge/opencensus/README.md

Co-authored-by: Eric Sirianni <[email protected]>

Co-authored-by: Eric Sirianni <[email protected]>
Co-authored-by: Tyler Yahn <[email protected]>
  • Loading branch information
3 people authored Mar 11, 2021
1 parent 77aa218 commit a1539d4
Show file tree
Hide file tree
Showing 10 changed files with 1,319 additions and 9 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- Added codeql worfklow to GitHub Actions (#1428)
- Added Gosec workflow to GitHub Actions (#1429)
- Add new HTTP driver for OTLP exporter in `exporters/otlp/otlphttp`. Currently it only supports the binary protobuf payloads. (#1420)
- Add an OpenCensus exporter bridge. (#1444)

### Changed

Expand Down
58 changes: 54 additions & 4 deletions bridge/opencensus/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

The OpenCensus Bridge helps facilitate the migration of an application from OpenCensus to OpenTelemetry.

## The Problem: Mixing OpenCensus and OpenTelemetry libraries
## Tracing

### The Problem: Mixing OpenCensus and OpenTelemetry libraries

In a perfect world, one would simply migrate their entire go application --including custom instrumentation, libraries, and exporters-- from OpenCensus to OpenTelemetry all at once. In the real world, dependency constraints, third-party ownership of libraries, or other reasons may require mixing OpenCensus and OpenTelemetry libraries in a single application.

Expand Down Expand Up @@ -44,10 +46,12 @@ The bridge implements the OpenCensus trace API using OpenTelemetry. This would

### User Journey

Starting from an application using entirely OpenCensus APIs:

1. Instantiate OpenTelemetry SDK and Exporters
2. Override OpenCensus' DefaultTracer with the bridge
3. Migrate libraries from OpenCensus to OpenTelemetry
4. Remove OpenCensus Exporters
3. Migrate libraries individually from OpenCensus to OpenTelemetry
4. Remove OpenCensus exporters and configuration

To override OpenCensus' DefaultTracer with the bridge:
```golang
Expand All @@ -63,10 +67,56 @@ octrace.DefaultTracer = opencensus.NewTracer(tracer)

Be sure to set the `Tracer` name to your instrumentation package name instead of `"bridge"`.

### Incompatibilities
#### Incompatibilities

OpenCensus and OpenTelemetry APIs are not entirely compatible. If the bridge finds any incompatibilities, it will log them. Incompatibilities include:

* Custom OpenCensus Samplers specified during StartSpan are ignored.
* Links cannot be added to OpenCensus spans.
* OpenTelemetry Debug or Deferred trace flags are dropped after an OpenCensus span is created.

## Metrics

### The problem: mixing libraries without mixing pipelines

The problem for monitoring is simpler than the problem for tracing, since there
are no context propagation issues to deal with. However, it still is difficult
for users to migrate an entire applications' monitoring at once. It
should be possible to send metrics generated by OpenCensus libraries to an
OpenTelemetry pipeline so that migrating a metric does not require maintaining
separate export pipelines for OpenCensus and OpenTelemetry.

### The Exporter "wrapper" solution

The solution we use here is to allow wrapping an OpenTelemetry exporter such
that it implements the OpenCensus exporter interfaces. This allows a single
exporter to be used for metrics from *both* OpenCensus and OpenTelemetry.

### User Journey

Starting from an application using entirely OpenCensus APIs:

1. Instantiate OpenTelemetry SDK and Exporters.
2. Replace OpenCensus exporters with a wrapped OpenTelemetry exporter from step 1.
3. Migrate libraries individually from OpenCensus to OpenTelemetry
4. Remove OpenCensus Exporters and configuration.

For example, to swap out the OpenCensus logging exporter for the OpenTelemetry stdout exporter:
```golang
import (
"go.opencensus.io/metric/metricexport"
"go.opentelemetry.io/otel/bridge/opencensus"
"go.opentelemetry.io/otel/exporters/stdout"
"go.opentelemetry.io/otel"
)
// With OpenCensus, you could have previously configured the logging exporter like this:
// import logexporter "go.opencensus.io/examples/exporter"
// exporter, _ := logexporter.NewLogExporter(logexporter.Options{})
// Instead, we can create an equivalent using the OpenTelemetry stdout exporter:
openTelemetryExporter, _ := stdout.NewExporter(stdout.WithPrettyPrint())
exporter := opencensus.NewMetricExporter(openTelemetryExporter)

// Use the wrapped OpenTelemetry exporter like you normally would with OpenCensus
intervalReader, _ := metricexport.NewIntervalReader(&metricexport.Reader{}, exporter)
intervalReader.Start()
```
180 changes: 180 additions & 0 deletions bridge/opencensus/aggregation.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package opencensus

import (
"errors"
"fmt"
"time"

"go.opencensus.io/metric/metricdata"

"go.opentelemetry.io/otel/metric/number"
"go.opentelemetry.io/otel/sdk/export/metric/aggregation"
)

var (
errIncompatibleType = errors.New("incompatible type for aggregation")
errEmpty = errors.New("points may not be empty")
errBadPoint = errors.New("point cannot be converted")
)

// aggregationWithEndTime is an aggregation that can also provide the timestamp
// of the last recorded point.
type aggregationWithEndTime interface {
aggregation.Aggregation
end() time.Time
}

// newAggregationFromPoints creates an OpenTelemetry aggregation from
// OpenCensus points. Points may not be empty and must be either
// all (int|float)64 or all *metricdata.Distribution.
func newAggregationFromPoints(points []metricdata.Point) (aggregationWithEndTime, error) {
if len(points) == 0 {
return nil, errEmpty
}
switch t := points[0].Value.(type) {
case int64:
return newExactAggregator(points)
case float64:
return newExactAggregator(points)
case *metricdata.Distribution:
return newDistributionAggregator(points)
default:
// TODO add *metricdata.Summary support
return nil, fmt.Errorf("%w: %v", errIncompatibleType, t)
}
}

var _ aggregation.Aggregation = &ocExactAggregator{}
var _ aggregation.LastValue = &ocExactAggregator{}
var _ aggregation.Points = &ocExactAggregator{}

// newExactAggregator creates an OpenTelemetry aggreation from OpenCensus points.
// Points may not be empty, and must only contain integers or floats.
func newExactAggregator(pts []metricdata.Point) (aggregationWithEndTime, error) {
points := make([]aggregation.Point, len(pts))
for i, pt := range pts {
switch t := pt.Value.(type) {
case int64:
points[i] = aggregation.Point{
Number: number.NewInt64Number(pt.Value.(int64)),
Time: pt.Time,
}
case float64:
points[i] = aggregation.Point{
Number: number.NewFloat64Number(pt.Value.(float64)),
Time: pt.Time,
}
default:
return nil, fmt.Errorf("%w: %v", errIncompatibleType, t)
}
}
return &ocExactAggregator{
points: points,
}, nil
}

type ocExactAggregator struct {
points []aggregation.Point
}

// Kind returns the kind of aggregation this is.
func (o *ocExactAggregator) Kind() aggregation.Kind {
return aggregation.ExactKind
}

// Points returns access to the raw data set.
func (o *ocExactAggregator) Points() ([]aggregation.Point, error) {
return o.points, nil
}

// LastValue returns the last point.
func (o *ocExactAggregator) LastValue() (number.Number, time.Time, error) {
last := o.points[len(o.points)-1]
return last.Number, last.Time, nil
}

// end returns the timestamp of the last point
func (o *ocExactAggregator) end() time.Time {
_, t, _ := o.LastValue()
return t
}

var _ aggregation.Aggregation = &ocDistAggregator{}
var _ aggregation.Histogram = &ocDistAggregator{}

// newDistributionAggregator creates an OpenTelemetry aggreation from
// OpenCensus points. Points may not be empty, and must only contain
// Distributions. The most recent disribution will be used in the aggregation.
func newDistributionAggregator(pts []metricdata.Point) (aggregationWithEndTime, error) {
// only use the most recent datapoint for now.
pt := pts[len(pts)-1]
val, ok := pt.Value.(*metricdata.Distribution)
if !ok {
return nil, fmt.Errorf("%w: %v", errBadPoint, pt.Value)
}
bucketCounts := make([]uint64, len(val.Buckets))
for i, bucket := range val.Buckets {
if bucket.Count < 0 {
return nil, fmt.Errorf("%w: bucket count may not be negative", errBadPoint)
}
bucketCounts[i] = uint64(bucket.Count)
}
if val.Count < 0 {
return nil, fmt.Errorf("%w: count may not be negative", errBadPoint)
}
return &ocDistAggregator{
sum: number.NewFloat64Number(val.Sum),
count: uint64(val.Count),
buckets: aggregation.Buckets{
Boundaries: val.BucketOptions.Bounds,
Counts: bucketCounts,
},
endTime: pts[len(pts)-1].Time,
}, nil
}

type ocDistAggregator struct {
sum number.Number
count uint64
buckets aggregation.Buckets
endTime time.Time
}

// Kind returns the kind of aggregation this is.
func (o *ocDistAggregator) Kind() aggregation.Kind {
return aggregation.HistogramKind
}

// Sum returns the sum of values.
func (o *ocDistAggregator) Sum() (number.Number, error) {
return o.sum, nil
}

// Count returns the number of values.
func (o *ocDistAggregator) Count() (uint64, error) {
return o.count, nil
}

// Histogram returns the count of events in pre-determined buckets.
func (o *ocDistAggregator) Histogram() (aggregation.Buckets, error) {
return o.buckets, nil
}

// end returns the time the histogram was measured.
func (o *ocDistAggregator) end() time.Time {
return o.endTime
}
Loading

0 comments on commit a1539d4

Please sign in to comment.