jaegertracing / jaeger

CNCF Jaeger, a Distributed Tracing Platform

Home Page:https://www.jaegertracing.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Bug]: Monitor tab page error with error 501 Not Implemented

irnurzaman opened this issue · comments

What happened?

I'm trying to use otel spanmetrics connector and Jaeger monitor feature but the monitor tab shows error 501 Not Implemented in the console and the page doesn't show anything.

Steps to reproduce

  1. Run Jaeger using docker compose
version: '3.7'

services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib
    container_name: otel-collector
    volumes:
      - ./telemetry/otel-collector-config.yml:/etc/otelcol/otel-collector-config.yml
    command: --config /etc/otelcol/otel-collector-config.yml
    ports:
      - 8889:8889
      - 4318:4318
    networks:
      - backend

  prometheus:
    image: prom/prometheus
    container_name: prometheus
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    command: --config.file=/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090
    networks:
      - backend

  jaeger-collector:
    image: jaegertracing/jaeger-collector
    container_name: jaeger-collector
    ports:
      - 14269:14269
    command: ["--cassandra.keyspace=jaeger_v1_dc1"]
    environment:
      - SPAN_STORAGE_TYPE=cassandra
      - CASSANDRA_SERVERS=cassandra
      - CASSANDRA_KEYSPACE=jaeger_v1_dc1
    networks:
      - backend
    depends_on:
      - otel-collector

  jaeger-query:
    image: jaegertracing/jaeger-query
    container_name: jaeger-query
    volumes:
      - "./jaeger/jaeger-ui.json:/etc/jaeger/jaeger-ui.json"
    ports:
      - 16686:16686
    command: --query.ui-config /etc/jaeger/jaeger-ui.json
    environment:
      - SPAN_STORAGE_TYPE=cassandra
      - CASSANDRA_SERVERS=cassandra
      - CASSANDRA_KEYSPACE=jaeger_v1_dc1
      - METRIC_STORAGE_TYPE=prometheus
      - PROMETHEUS_SERVERS=http://prometheus:9090
      - PROMETHEUS_QUERY_SUPPORT_SPANMETRICS_CONNECTOR=true
      - PROMETHUES_QUERY_NORMALIZE_CALLS=true
      - PROMETHEUS_QUERY_NORMALIZE_DURATION=true
    depends_on:
      - otel-collector
    networks:
      - backend

networks:
  backend:
    name: backend
    external: true
  1. Mount jaeger-ui.json config to jaeger-query config
{
  "monitor": {
    "menuEnabled": true
  },
  "dependencies": {
    "menuEnabled": true
  }
}
  1. Go to monitor tab and get this error
    Screenshot 2024-04-22 053714

Expected behavior

The monitor tab shows metrics data as shown in monitor examples

Relevant log output

No response

Screenshot

No response

Additional context

No response

Jaeger backend version

docker image jaeger-query:latest

SDK

Jaeger Go client SDK

Pipeline

OTEL SDK -> OTEL Collector -> Jaeger Collector & Prometheus -> Cassandra -> Jaeger Query

Stogage backend

docker image cassandra:latest

Operating system

Windows 11

Deployment model

Docker

Deployment configs

Docker Compose
version: '3.7'

services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib
    container_name: otel-collector
    volumes:
      - ./telemetry/otel-collector-config.yml:/etc/otelcol/otel-collector-config.yml
    command: --config /etc/otelcol/otel-collector-config.yml
    ports:
      - 8889:8889
      - 4318:4318
    networks:
      - backend

  prometheus:
    image: prom/prometheus
    container_name: prometheus
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    command: --config.file=/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090
    networks:
      - backend

  jaeger-collector:
    image: jaegertracing/jaeger-collector
    container_name: jaeger-collector
    ports:
      - 14269:14269
    command: ["--cassandra.keyspace=jaeger_v1_dc1"]
    environment:
      - SPAN_STORAGE_TYPE=cassandra
      - CASSANDRA_SERVERS=cassandra
      - CASSANDRA_KEYSPACE=jaeger_v1_dc1
    networks:
      - backend
    depends_on:
      - otel-collector

  jaeger-query:
    image: jaegertracing/jaeger-query
    container_name: jaeger-query
    volumes:
      - "./jaeger/jaeger-ui.json:/etc/jaeger/jaeger-ui.json"
    ports:
      - 16686:16686
    command: --query.ui-config /etc/jaeger/jaeger-ui.json
    environment:
      - SPAN_STORAGE_TYPE=cassandra
      - CASSANDRA_SERVERS=cassandra
      - CASSANDRA_KEYSPACE=jaeger_v1_dc1
      - METRIC_STORAGE_TYPE=prometheus
      - PROMETHEUS_SERVERS=http://prometheus:9090
      - PROMETHEUS_QUERY_SUPPORT_SPANMETRICS_CONNECTOR=true
      - PROMETHUES_QUERY_NORMALIZE_CALLS=true
      - PROMETHEUS_QUERY_NORMALIZE_DURATION=true
    depends_on:
      - otel-collector
    networks:
      - backend

networks:
  backend:
    name: backend
    external: true

Otel Collector Config
receivers:
  otlp:
    protocols:
      http: 
        endpoint: "0.0.0.0:4318"

exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"

  otlphttp:
    endpoint: http://jaeger-collector:4318
    tls:
      insecure: true

connectors:
  spanmetrics:

processors:
  batch:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [spanmetrics, otlphttp]
    # The exporter name in this pipeline must match the spanmetrics.metrics_exporter name.
    # The receiver is just a dummy and never used; added to pass validation requiring at least one receiver in a pipeline.
    metrics/spanmetrics:
      receivers: [spanmetrics]
      exporters: [prometheus]

Prometheus Config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

scrape_configs:
  - job_name: aggregated-trace-metrics
    static_configs:
    - targets: ['otel_collector:8889']

Jaeger Query UI Config
{
  "monitor": {
    "menuEnabled": true
  },
  "dependencies": {
    "menuEnabled": true
  }
}

When I check the detailed API response, there is error response like this
Screenshot 2024-04-22 094614