X Tutup
Skip to content

Add distributed tracing using opentelemetry#119

Open
andystaples wants to merge 11 commits intomicrosoft:mainfrom
andystaples:andystaples/add-distributed-tracing
Open

Add distributed tracing using opentelemetry#119
andystaples wants to merge 11 commits intomicrosoft:mainfrom
andystaples:andystaples/add-distributed-tracing

Conversation

@andystaples
Copy link
Contributor

@andystaples andystaples commented Mar 3, 2026 •
edited
Loading

Adds support for Distributed Tracing by passing the trace information for orchestrations passed in the gRPC spec to activitites/entities/sub-orchestrations

Compare to the Dotnet SDK: https://github.com/microsoft/durabletask-dotnet

Test orchestrator:

def fan_out_orchestrator(ctx, names: list[str]):
    """Timer → 3 sequential activities → entity signal → entity call."""
    yield ctx.create_timer(timedelta(milliseconds=250))
    print("  Timer fired.")

    results = []
    for name in names:
        result = yield ctx.call_activity(say_hello, input=name)
        results.append(result)

    # Entity signal (fire-and-forget)
    entity_id = EntityInstanceId(entity="Counter", key="myCounter")
    ctx.signal_entity(entity_id, "add", input=10)
    print("  Entity signaled.")

    # Entity call (waits for result)
    count = yield ctx.call_entity(entity_id, "get")
    print(f"  Entity call returned: {count}")

    return {"greetings": results, "counter": count}

Jaeger profile for fan-out orchestration:
image

image

DTS view of the same orchestration:
image

Entity traces are present but unlinked, until the protobuf changes here are merged and incorporated into backends:
microsoft/durabletask-protobuf#64

Resolves #97

@andystaples andystaples marked this pull request as ready for review March 3, 2026 19:21
@torosent torosent mentioned this pull request Mar 3, 2026
Merged
4 tasks
Copilot AI reviewed Mar 3, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds OpenTelemetry-based distributed tracing to the Durable Task Python SDK by propagating W3C trace context through the gRPC protocol and emitting spans for orchestration, activity, sub-orchestration, timer, entity, and event operations.

Changes:

  • Introduces durabletask.internal.tracing with optional OpenTelemetry helpers for context extraction/injection and span lifecycle management.
  • Updates worker + client code paths to propagate TraceContext over gRPC and to create/finish spans across multi-dispatch orchestration replays.
  • Adds a comprehensive tracing test suite and updates dependency metadata to include OpenTelemetry packages/extras.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/durabletask/test_tracing.py New unit/integration-style tests covering tracing utilities and replay/lifecycle behavior
durabletask/internal/tracing.py New tracing utility module (optional OpenTelemetry dependency)
durabletask/worker.py Server-side span creation/propagation; persists orchestration + client spans across dispatches
durabletask/client.py Producer spans + parent trace context injection for orchestration creation and entity signals
durabletask/internal/helpers.py Adds parent_trace_context to schedule/sub-orchestration action helpers
pyproject.toml Adds opentelemetry optional dependency extra
requirements.txt Adds OpenTelemetry packages to the dev/test requirements list
CHANGELOG.md Notes improved distributed tracing support

You can also share your feedback on Copilot code review. Take the survey.

@andystaples andystaples marked this pull request as draft March 4, 2026 00:05
@andystaples andystaples marked this pull request as ready for review March 4, 2026 21:07
@andystaples andystaples requested a review from bachuv March 6, 2026 18:32
bachuv
bachuv reviewed Mar 7, 2026
View reviewed changes
tests/durabletask/test_tracing.py
Copy link

@bachuv bachuv Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a sample for distributed tracing with a README and include screenshots? Can you also add an example of a suborchestration in the sample and screenshots?

Copy link
Contributor Author

@andystaples andystaples Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

durabletask/internal/tracing.py Outdated

def create_timer_span_name(orchestration_name: str) -> str:
"""Build a timer span name: ``orchestration:<name>:timer``."""
return f"orchestration:{orchestration_name}:timer"
Copy link

@bachuv bachuv Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put the strings "orchestration", "timer", etc. as constants?

Copy link
Contributor Author

@andystaples andystaples Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bachuv bachuv bachuv left review comments

Copilot code review Copilot Copilot left review comments

@cgillum cgillum Awaiting requested review from cgillum

@torosent torosent Awaiting requested review from torosent

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add support for Distributed Tracing

3 participants

@andystaples @bachuv
X Tutup