Free learning / 50-minute transport proof workshop
Trace context: a successful request can still have a broken trace
Execute real HTTP propagation with pinned OpenTelemetry. Compare dropped and reused parents, valid non-recording contexts and a bounded baggage policy without confusing trace evidence with task execution.
No account needed. Use your own copy for writing and keep the original as a reference.
Engineering lab library / Run locally
Choose your next agent lab.
Start with the durable workflow reference, then choose the boundary you want to test. Open its walkthrough for setup, commands and expected observations.
Synthetic records. No provider key or model inference. Package setup needs package-registry access.
Start here / Real library, SDK or framework · Scripted inputs
HTTP trace propagation and sampled execution
Inspect actual HTTP carriers, SDK parent links, non-recording contexts and separately observed handler execution.
Python 3.11+ · Existing pinned OpenTelemetry packages · Owned loopback HTTP · No remote collector
Original LockedIn Labs workshop · 50 minutes · Python 3.11+ and the existing pinned OpenTelemetry environment. Seven cases use two owned HTTP listeners on 127.0.0.1, separate SDK providers and an in-memory exporter. The listeners are in one process, on fresh ephemeral ports, with separate request threads. No collector, model, TLS, external service, patient/customer data or durable business operation is involved.
1. Keep delivery and the trace relationship separate
A gateway can receive HTTP 200 from a worker while the worker's trace is disconnected or attached to the wrong parent. Response status answers a transport/application question; it does not certify the instrumentation. A trace ID groups related spans. A span ID names one operation. A propagated parent ID tells the next receiver which span it should regard as its parent.
The gateway extracts incoming context, creates its own server span, then injects the active context into the worker request. The worker extracts that actual received header and creates another server span. Our teaching code deliberately omits a separate HTTP client span. In an instrumented HTTP client, the outbound carrier would normally identify that client span instead. Always follow the actual instrumentation topology.
OpenTelemetry provides the SDK, context handling and W3C propagator. It is not a hosted monitoring product. The Python process emits the local records; this page's diagram is an authored explanation, not live telemetry.
Illustrative learning artifact
Reliability belongs to the user task
Two beyond the allowance. Investigate the failed work.
2. Read the carrier before drawing the parent chain
For the version-00 fixture, traceparent has this shape:
00-1234567890abcdef1234567890abcdef-1234567890abcdef-01
version 32 hex trace ID 16 hex parent ID flagsThe final 01 has the sampled bit set; 00 clears it. The trace and parent IDs cannot be all zero. The SDK handles extraction rather than a home-grown validator. This workshop exercises selected version-00 cases, not every W3C version, malformed carrier or forward-compatibility rule. Consult the standard and the pinned propagator for the full contract.
The receiving server span has a new span ID, keeps a valid incoming trace ID and identifies the extracted parent as remote. When this gateway injects its active context, the outgoing parent ID becomes the gateway span ID. Copying the incoming header unchanged keeps the trace ID but names the upstream parent again: the worker becomes the gateway's apparent sibling despite the observed gateway-to-worker call.
The supplied upstream context is synthetic. The caller's span is not created or exported here. A remote parent reference does not prove that a corresponding record exists in a backend, nor that the caller is trusted.
UNDERSTAND THE SYSTEM
Carry the active context across the actual request
Six authored frames explain seven real loopback HTTP comparisons using pinned OpenTelemetry providers. Response status, wire context, SDK parentage and exported observations answer different questions. The browser is not sending these requests or displaying a live collector.
Watch one example, step by step.
Read the full worked example
- 011 · Continue the active server span
Correct extraction retains the supplied trace ID while the gateway creates a new span ID. Injection names that gateway span. The worker's remote parent equals it. Both handlers execute once, HTTP returns 200 and two finished spans reach the local exporter. A real instrumented client span would add another parent boundary.
ACTUAL LOCAL COMPARISON: Same trace · worker parent = gateway span · two exports
- 022 · A missing header splits the trace
Dropped propagation deliberately sends no outgoing traceparent. The worker still returns the same successful task response, but starts a root on another trace. Two exported records remain; a count of spans does not repair their missing relationship.
SUCCESSFUL REQUEST, LOST RELATIONSHIP: HTTP 200 · two traces · no worker parent
- 033 · Reusing the old parent draws the wrong edge
The reused-parent case copies the incoming header instead of injecting active context. Both endpoint spans share the upstream parent and trace ID. HTTP succeeds, yet the worker appears as the gateway's sibling. Common trace identity alone cannot establish the observed gateway-to-worker edge.
ACTUAL WRONG-PARENT COMPARISON: Same trace · worker parent ≠ gateway span
- 044 · Missing or invalid context starts a new root
Missing input has no incoming context. Invalid-zero carries an all-zero trace ID that the SDK declines. Both gateway runs start new roots, then inject their active context correctly. Worker parentage remains valid inside the newly created trace; no caller identity has been established.
OBSERVED SDK EXTRACTION: New gateway root · correctly linked worker
- 055 · Unsampled work still executes
The unsampled remote parent makes both endpoint spans non-recording under ParentBased(ALWAYS_ON). They still have valid linked contexts and both handlers return the successful synthetic result. The in-memory exporter receives zero finished spans. This observed policy does not turn absent telemetry into evidence of absent work.
ACTUAL SAMPLER COMPARISON: Two handler entries · HTTP 200 · zero exports
- 066 · Trace context does not carry authority
The baggage case supplies synthetic metadata to the gateway. This trace-only propagator sends no baggage to the worker. That is the exercise's explicit adapter choice, not a universal baggage rule. Trace IDs and baggage cannot authorize a caller, select an allowed tenant or deduplicate business work.
BOUNDED PROPAGATION POLICY: Baggage received at gateway · absent at worker
Read the connections in plain English
- 01Incoming context → Gateway server span
Continue valid context or create a root when extraction yields no valid parent.
- 02Gateway server span → Actual HTTP carrier
The outgoing header names the newly active gateway span in this topology.
- 03Actual HTTP carrier → Worker server span
The actual received carrier determines the worker's remote parent.
- 04Gateway server span → In-memory exporter
A recording gateway span is exported when it ends.
- 05Worker server span → In-memory exporter
A recording worker span is exported separately from its HTTP reply.
- 06In-memory exporter → Receiving evidence
Reconcile exported parentage without treating order or timestamps as causality.
- 07Worker server span → Receiving evidence
The direct handler count and observed reply are separate from sampled export.
Check your understanding: Do HTTP 200 and one common trace ID prove the trace is correct?
No. The reused-parent case returns 200 with one shared trace ID while linking the worker to the wrong parent. Compare the actual outgoing/received carrier and span IDs. Separately retain sampling, execution and receiving evidence; trace metadata supplies no authorization or business receipt.
3. Execute seven transport comparisons
Download the runner and the existing pinned requirements into a disposable folder. Use a fresh environment or the unchanged environment from the first tracing lesson:
python3 -m venv .venv-trace
. .venv-trace/bin/activate
python -m pip install -r fde-first-trace-requirements.txt
python fde-http-trace.py > http-trace-original.json
python fde-http-trace.py --case correct --sampled 0 > http-trace-unsampled.jsonOn Windows Command Prompt, use py -3 -m venv .venv-trace and .venv-trace\Scripts\activate.bat; the remaining commands are unchanged. Installation needs internet access. Execution connects only to the two loopback listeners it allocates. If local socket policy blocks them, retain that setup failure; reading a report is a different practice mode.
The runner checks the exact existing pins: OpenTelemetry API/SDK 1.45.1, semantic conventions 0.66b1 and typing_extensions 4.16.0. It sets no global tracer provider and loads no remote exporter. Each endpoint has its own TracerProvider, service name and ParentBased(ALWAYS_ON) sampler. Only the in-memory exporter is shared for inspection.
Predict before executing:
| Case | Actual HTTP/task observation | Trace evidence to inspect |
|---|---|---|
correct | Both handlers run once; client receives 200 and SYNTHETIC-OK | Same trace; worker parent equals gateway span; two finished spans |
unsampled | The same two handlers and successful result remain | Valid linked contexts; neither server span records; zero finished spans |
dropped | Both requests still succeed | Gateway sends no trace header; worker starts another trace |
reused-parent | Both requests still succeed | Trace ID agrees, but worker parent is the original upstream ID, not the gateway span |
invalid-zero | Both requests still succeed | SDK declines the all-zero trace ID; gateway starts a new root and correctly links the worker |
missing | Both requests still succeed | Gateway has no incoming context; it starts a root and forwards its own active context |
baggage | Both requests still succeed | Gateway receives synthetic baggage; the trace-only propagator sends no baggage to the worker |
Use one comparison at a time:
python fde-http-trace.py --case dropped
python fde-http-trace.py --case reused-parent
python fde-http-trace.py --case baggageKeep original reports before rerunning; shell redirection overwrites an existing file of the same name. The program accepts no URL, file, tenant or credential input. Its fixed GET performs no business mutation. Request success here establishes only the supplied local reply.
4. Reconcile wire context with exported spans
Start with gateway.received, gateway.sent and worker.received. These are the selected headers actually supplied to and read by the owned HTTP endpoints. Then compare each endpoint's parent, current span, recording flag and independently incremented handler_counts.
For the correctly linked case:
gateway.span.trace_id == worker.span.trace_id
gateway.span.span_id == worker.parent.span_id
worker.parent.remote == trueRecompute those relationships instead of trusting the report's comparison summary. For recorded cases, reconcile them with finished_spans: actual SDK names, service resources, contexts and parents. SDK-generated IDs change on every run; compare relationships inside one report, not literal generated IDs across files. Span export order is not a causal-order contract.
For reused-parent, both spans still name the same supplied upstream parent. Two exported records and one common trace ID therefore do not establish the right causal edge. For dropped, the worker creates a separate root even though the gateway actually called it. Nearby timestamps and similar names cannot restore the lost parent.
The process retains each handler's direct observations separately from exported telemetry. These counters and wire records are teaching evidence, not evidence that production instrumentation always has an independent audit channel.
5. Change sampling without changing the task
Compare correct with --sampled 1 and --sampled 0. This changes the incoming sampled bit for that case; the separately named unsampled case always uses zero. Under this lab's parent-based sampler, the unsampled remote parent yields valid non-recording contexts. Injection still passes the new active span's context onward. Both handlers execute and return the same synthetic result while the exporter receives zero finished spans.
Absence from this exporter cannot mean the task did not run. Nor can a sampled trace collection automatically provide all-request counts for a service-level indicator. Different sampler or pipeline configurations can behave differently; this is one observed SDK policy, not a universal absence rule.
Explain the boundary: the lab does not send OTLP, test collector delivery/retention, align clocks across hosts, instrument real services or prove an enterprise's sampling policy. An installed SDK and a correct local carrier are necessary evidence for their own boundaries, not a complete observability rollout.
6. Receive trace evidence as a delivery pod
The trace-only propagator carries trace context, not baggage. In baggage, the gateway sees synthetic.department=practice; the worker does not. That is this adapter's explicit propagation choice, not a requirement that all applications discard all baggage. Baggage is separately defined metadata and is not automatically a span attribute. Any real propagation policy needs approved keys, destinations, limits and retention evidence.
Trace IDs and metadata can correlate activity and still require privacy review. Never place secrets or private person/customer content in a carrier. A caller can supply a valid-looking trace ID or baggage value. Neither authenticates the caller, grants access, selects an authorized tenant or supplies an idempotency key. This lab implements none of those controls.
Work in three roles:
- Builder: retain the exact runner, package versions, original/changed command and report. Locate the actual outgoing and received carrier, not just the friendly summary.
- Receiver: reproduce the unchanged-header bug, identify the incorrect parent and run the sampling change. Record your own action and assistance separately from the builder's account.
- Application/platform owners: agree the intended service/client span topology, propagation boundary, sampling policy and next collector or receiving-service probe. Keep identity, tenant authorization and business receipts with their accountable owners.
Each participant keeps a separate record:
Synthetic alias / role:
Exact runner / package versions / command:
Prediction before execution:
Gateway received / sent carrier evidence path:
Worker received carrier / actual parent evidence path:
Own observation / finished spans / handler counts:
Own contribution / assistance:
Changed sampled bit / prediction / separately retained result:
Receiving action actually attempted / assistance:
Missing evidence / next bounded probe / accountable role:No practice record establishes qualification, independence, placement or permission to deploy. Continue with operational ownership to connect trace evidence to user promises and receiving responsibilities.
7. Defend the evidence in an interview
Explain how reused-parent differs from dropped despite the same successful task. Sketch the intended trace chain and the actual wrong-parent chain using the IDs in one report. Then explain why an actual HTTP client span would change the expected parent.
Explain how the unsampled case has valid span IDs without finished exported spans, and which observation establishes that the handlers ran. Name the sampler configuration before generalizing. Describe why a backend with only sampled traces cannot automatically supply the complete request denominator.
Finally, describe one real receiving-service test and one collector-retention test still needed. State why a valid header or a baggage tenant label must never replace application authorization. An operational answer names the missing boundary and its owner instead of making a tracing success carry unrelated claims.
Sources checked 28 October 2026: W3C Trace Context Recommendation, 23 November 2021, OpenTelemetry Python propagation, OpenTelemetry parent-based sampling, and OpenTelemetry baggage. These primary references explain the protocols and SDK concepts. The fixture, comparisons, diagrams and receiving exercise are original.