Skip to content

Bug: gRPC Context incorrectly propagated to async threads via OpenTelemetry Context (with reproduction code) #15596

@153952402

Description

@153952402

When using OpenTelemetry's gRPC instrumentation library with Context.current().wrap(runnable) to propagate OpenTelemetry context to async threads, the gRPC context is incorrectly propagated along with it. This causes errors when making gRPC calls from async threads because the gRPC context from the original request thread is incorrectly activated in the async thread.

To Reproduce:

##reproduction code repository : https://github.com/153952402/grpc-otel-context-bug-reproduction
cd grpc-otel-context-bug-reproduction
mvn spring-boot:run

In another terminal:

grpcurl -plaintext \
  -import-path ./src/main/resources/proto \
  -proto downstream_service.proto \
  -d '{"id":"test-123","data":"test data"}' \
  127.0.0.1:9090 \
  com.example.reproduction.downstream.grpc.DownstreamService/Handle

What did you expect to see?

  1. The gRPC Context should NOT be propagated to async threads - only OpenTelemetry tracing context should be propagated.
  2. When checking io.grpc.Context.current() in async threads, it should return Context.ROOT.
  3. Async threads should be able to make gRPC calls independently without errors.

What did you see instead?

  1. The gRPC Context IS incorrectly propagated to async threads via OpenTelemetry Context.
  2. When checking io.grpc.Context.current() in async threads, it returns a non-ROOT context (the context from the original request thread).
  3. When making gRPC calls from async threads, errors occur because:
    • The gRPC context from the original request is still active
    • This can cause deadline/cancellation states to be incorrectly propagated
    • The context state may be inconsistent for the new gRPC call

What version and what artifacts are you using?

Artifacts:

  • opentelemetry-api version 1.32.0
  • opentelemetry-sdk version 1.32.0
  • opentelemetry-grpc-1.6 (instrumentation library) version 2.1.0-alpha
  • grpc-netty-shaded version 1.60.1
  • grpc-protobuf version 1.60.1
  • grpc-stub version 1.60.1
  • spring-boot-starter-web version 3.2.0
  • grpc-server-spring-boot-starter version 2.15.0.RELEASE

Version:

  • Java: 21
  • Spring Boot: 3.2.0
  • OpenTelemetry: 1.32.0
  • OpenTelemetry Instrumentation (gRPC 1.6): 2.1.0-alpha
  • gRPC: 1.60.1

Thank you for your attention to this issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions