open-telemetry / opentelemetry-collector

OpenTelemetry Collector

Home Page:https://opentelemetry.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Panic when exporting using otlphttp

icyphox opened this issue · comments

The collector crashes when trying to export traces using otlphttp. I'm running v0.51.0. Here is the stacktrace:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x4d3077]

goroutine 88 [running]:
io.copyBuffer({0x387b920, 0xc00085e580}, {0x0, 0x0}, {0x0, 0x0, 0x0})
        io/io.go:423 +0x197
io.Copy(...)
        io/io.go:382
go.opentelemetry.io/collector/config/confighttp.(*compressRoundTripper).RoundTrip(0xc0002e8ed0, 0xc0009a3500)
        go.opentelemetry.io/collector@v0.51.0/config/confighttp/compression.go:85 +0xf9
net/http.send(0xc0009a3400, {0x3881f40, 0xc0002e8ed0}, {0x31c7280, 0x4d2d01, 0x547e340})
        net/http/client.go:252 +0x5d8
net/http.(*Client).send(0xc0002e8f00, 0xc0009a3400, {0x7f802d581680, 0xc0003f00e0, 0x547e340})
        net/http/client.go:176 +0x9b
net/http.(*Client).do(0xc0002e8f00, 0xc0009a3400)
        net/http/client.go:725 +0x908
net/http.(*Client).Do(...)
        net/http/client.go:593
go.opentelemetry.io/collector/exporter/otlphttpexporter.(*exporter).export(0xc00025a990, {0x38e7c18, 0xc00089c8a0}, {0xc0009721e0, 0x25}, {0xc0009da000, 0x76aa, 0x76aa})
        go.opentelemetry.io/collector@v0.51.0/exporter/otlphttpexporter/otlp.go:136 +0x3d1
go.opentelemetry.io/collector/exporter/otlphttpexporter.(*exporter).pushTraces(0xc00025a990, {0x38e7c18, 0xc00089c8a0}, {0xc0008f6300})
        go.opentelemetry.io/collector@v0.51.0/exporter/otlphttpexporter/otlp.go:105 +0x8a
go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesRequest).export(0xc00089c7e0, {0x38e7c18, 0xc00089c8a0})
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/traces.go:70 +0x34
go.opentelemetry.io/collector/exporter/exporterhelper.(*timeoutSender).send(0xc0007c70c8, {0x3939ac0, 0xc00089c7e0})
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/common.go:225 +0x96
go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send(0xc00025aab0, {0x3939ac0, 0xc00089c7e0})
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/queued_retry.go:176 +0x5eb
go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesExporterWithObservability).send(0xc00097efc0, {0x3939ac0, 0xc00089c7e0})
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/traces.go:134 +0x88
go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1({0x2f58800, 0xc00089c7e0})
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/queued_retry_inmemory.go:118 +0x5c
go.opentelemetry.io/collector/exporter/exporterhelper/internal.consumerFunc.consume(0x0, {0x2f58800, 0xc00089c7e0})
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:82 +0x2c
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func2()
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:69 +0xcd
created by go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers
        go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:63 +0x9d

And this is the configuration I'm using:

    exporters:
      logging:
        loglevel: "debug"
      otlphttp:
        endpoint: "http://x.x.x.x/tempo"
    processors:
      attributes:
        actions:
          - action: "insert"
            key: "changeme-gateway"
            value: "changemetoo-gateway"
      batch: {}
    receivers:
      otlp:
        protocols:
          grpc: {}
          http: {}
    service:
      pipelines:
        traces:
          exporters:
            - "otlphttp"
            - "logging"
          processors:
            - "batch"
            - "attributes"
          receivers:
            - "otlp"
      telemetry:
        metrics:
          address: "0.0.0.0:8888"
          level: "detailed"

I think the problem maybe this line.

_, copyErr := io.Copy(compressWriter, req.Body)

req.Body may be nil when the request is from client with Get Action

https://github.com/golang/go/blob/527ace0ffa81d59698d3a78ac3545de7295ea76b/src/net/http/request.go#L170-L172

Looks like it's a great candidate for an initial contribution to the project.