OpenTelemetry Collector failing to export to Jaeger via OTLP

Question

As the title states, I have some client applications sending metrics, traces and logs to opentelemtry collector, and the collector should then export mentioned signals to a jaeger instance.

For some reason It is failing with a not very clear error and i'm stuck.

Here is the Docker compose:

version: '3.9'
    x-default-logging: &logging
      driver: "json-file"
      options:
        max-size: "5m"
        max-file: "2"
    
    services:
      # ********************
      # Telemetry Components
      # ********************
      
      # Jaeger
      jaeger:
        image: jaegertracing/all-in-one:1.50
        container_name: jaeger
        deploy:
          resources:
            limits:
              memory: 500M
        restart: unless-stopped
        ports:
          - "16686:16686"                    # Jaeger UI
          - "4317"                           # OTLP gRPC default port
          - "4318"                           # OTLP HTTP default port (protobuf)
          - "14250"
        environment:
          - COLLECTOR_OTLP_ENABLED=true
        logging: *logging
    
      # OpenTelemetry Collector
      otelcol:
        image: otel/opentelemetry-collector-contrib:0.87
        container_name: otel-col
        deploy:
          resources:
            limits:
              memory: 525M
        restart: unless-stopped
        command: [ "--config=/etc/otelcol-config.yml" ]
        volumes:
          - ./otel/otelcol-config.yml:/etc/otelcol-config.yml
        ports:
          - "4317:4317"     # OTLP over gRPC receiver
          - "4318:4318"     # OTLP over HTTP receiver
          - "9464"          # Prometheus exporter
          - "8888"          # metrics endpoint
        depends_on:
          - jaeger
        logging: *logging

And here is otelcol-config.yml:

receivers:
      otlp:
        protocols:
          grpc:
          http:
            cors:
              allowed_origins:
                - "http://*"
                - "https://*"
    
    exporters:
      debug:
        verbosity: detailed
      otlp:
        endpoint: "jaeger:4317"
        tls:
          insecure: true
    
    processors:
      batch:
    
    service:
      telemetry:
        logs:
          level: debug
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch]
          exporters: [debug, otlp]
        metrics:
          receivers: [otlp]
          processors: [batch]
          exporters: [debug, otlp]
        logs:
          receivers: [otlp]
          processors: [batch]
          exporters: [debug, otlp]

I can confirm the collector succesfully receives all the signals (it exports to stdout of collector), but when exporting to Jaeger I get the following error:

2023-10-17T01:44:36.404Z    error   exporterhelper/retry_sender.go:145      Exporting failed. The error is not retryable. Dropping data.    {"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "Permanent error: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.metrics.v1.MetricsService", "dropped_items": 35}
otel-col  | go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
otel-col  |     go.opentelemetry.io/collector/[email protected]/exporterhelper/retry_sender.go:145
otel-col  | go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send
otel-col  |     go.opentelemetry.io/collector/[email protected]/exporterhelper/metrics.go:176
otel-col  | go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1
otel-col  |     go.opentelemetry.io/collector/[email protected]/exporterhelper/queue_sender.go:126
otel-col  | go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1
otel-col  |     go.opentelemetry.io/collector/[email protected]/exporterhelper/internal/bounded_memory_queue.go:52

It seems that the function is not implemented on Jaeger side ? I've looked trough documentation and examples on both Jaeger and OpenTelemetry website / Github and they all do it this way, but I can't get it to work.

Answer 1

Jaeger can only receive traces, not metrics or logs. If you remove otlp from the exporter pipelines in metrics and logs, the error should go away.

If you do want to export metrics and logs, you'll need to install compatible backends such as Prometheus and Loki, and use the appropriate exporters.