# Coalesce cache misses without sharing the wrong promise

Original LockedIn Labs workshop · 40 minutes. Python 3.11+ standard library.
Follow function calls, dictionary keys and exceptions first. The program starts
owned threads against synthetic in-memory inputs, holds their builders with
explicit events, and joins every worker before reporting completion. No network,
model, credentials, files, database or service deployment is involved.

The [application caching guide](/training/guides/caching) introduces the reuse
contract. This workshop executes a different question: what happens when callers
ask for the same work before the first build finishes?

## 1. Name the unfinished work

A **cache stampede** is overlapping misses causing repeated work for the same
candidate. **Request coalescing**, sometimes called singleflight, lets callers
join an existing in-flight computation for a suitable key. The first participant
starts the work; a **waiter** observes its eventual result or failure.

The [Go singleflight package](https://pkg.go.dev/golang.org/x/sync/singleflight)
documents a practical duplicate-call suppression API. This exercise implements
its own small Python test fixture; it does not run that package or claim the
same complete API. It shares unfinished work only, retaining no completed-result
cache. Registering an in-flight operation, caching its completed output and
checking whether that output is valid now are separate jobs.

A Python **Future** represents a completion that can return a value or raise an
exception. A bounded `result(timeout=...)` wait can raise `TimeoutError` while
the work remains unfinished. Python's [Future documentation](https://docs.python.org/3.12/library/concurrent.futures.html#future-objects)
separates waiting from cancellation. The fixture constructs Futures and manages
their state as executor-like teaching machinery. In application code, use a
maintained concurrency implementation rather than copying this small manager.

## 2. Follow one registration and two waits

The registry key is a tuple: supplied workspace, document and renderer version.
Default callers both ask for North's `guide-7` through `renderer-1`. These are
fixture labels chosen by the driver, not authenticated callers or real tenants.

In `FlightGroup.join`, one lock covers finding an existing Future and registering
a new one. The worker captures the source value under the input lock, then waits
on a release event. The driver registers all callers while the build is held.
This provides a reproducible overlap without using a sleep to guess which
thread ran first. Thread scheduling still varies; do not infer a general
ordering or CPU parallelism from this selected interleaving.

The visual explains six independently reset cases. It does not execute Python
inside the browser. Predict both the build count and each caller's observation.

## 3. Run six owned-thread cases

Download and read [the runner](/templates/fde-cache-coalescing.py). Keep an
unchanged reference. Write predictions before reading its JSON:

```sh
python3 --version
python3 fde-cache-coalescing.py > coalescing-two.json
python3 fde-cache-coalescing.py --callers 3 > coalescing-three.json
```

The `--callers` argument accepts two or three. Separate-keys always uses two
workspaces. Each case creates a fresh input set and registry. Gates and waits
are bounded; the report is printed only after all owned builder threads join.
These are actual function executions, not simulated arrival counts.

| Case | First-wave builds with two callers | Observation to defend |
| --- | --- | --- |
| `naive` | 2 | Both misses execute the builder before either completes |
| `singleflight` | 1 | Second caller joins; a later same-key call starts build 2 |
| `leader-error` | 1 | Both receive the injected failure; key clears and an explicit later call succeeds |
| `waiter-timeout` | 1 | Caller 2 stops waiting while shared work remains running |
| `source-changed` | 1 | Captured r1 is returned although the source has advanced to r2 |
| `separate-keys` | 2 | North and South start separate work despite the same document ID |

`joined_existing` records which registration reused an existing Future. It is
not an authorization verdict or cache-hit rate. `before_release` describes the
held first wave. `after_first_wave` precedes any separate later call.
`captured_builds` retains the actual captured key and source body for each build.
The event log orders registration/join/completion observations under one local
lock; the captured inputs supply the separate build evidence.

## 4. Read the completion without erasing an earlier failure

Keep this ledger beside your predictions:

```text
singleflight:
  joined_existing: [false, true]
  before_release.builds_started: 1
  after_first_wave.pending_keys: []
  later_call.shared: false
  later_call.builds_started: 2

leader-error:
  caller_outcomes: builder-error, builder-error
  first error: injected_builder_failure
  after_first_wave.pending_keys: []
  later_call.result.state: value

waiter-timeout:
  caller_outcomes[1].state: wait-timeout
  before_release.builder_still_running_after_waiter_timeout: true
  fixture_completion_observer[1].state: value
  caller_outcomes[1].value: null

source-changed:
  captured_builds[0].input.revision: 1
  caller_outcomes: revision 1, revision 1
  source_after.north.revision: 2
```

The timeout case executes a real short `Future.result` wait while the builder
is held. The driver releases the worker only after observing `TimeoutError`.
The timeout duration is a fixture setting, not a service latency measurement or
a universal recommended deadline. The builder does not become cancelled because
that wait ended.

After release, the fixture driver can inspect completion independently. It keeps
that evidence in `fixture_completion_observer`; it does not rewrite the timed-out
caller's `caller_outcomes` entry. A late response in a deployed system also needs
an explicit receiving and reconciliation contract. Successful background work
does not prove that the original caller received or accepted a result.

The failure case shares the authored `RuntimeError` with all first-wave callers.
The registry removes completed work, including failure, before delivering its
result. A separate healthy call then starts fresh work. It is deliberately one
explicit new attempt; the program has no recursive retry policy. Permanent
retention of a failed Future would strand later requests on a poisoned entry.

The fixture copies values when recording caller observations so one report does
not mutate another's evidence. It is not a general mutable-object isolation API.
Production sharing also needs an agreed result ownership/immutability contract.

## 5. Change the burst and challenge the boundary

With three callers, naive starts three builders and singleflight starts one
first-wave builder. Both extra callers join in the coalesced case. The later
independent singleflight call still makes total builds two. Leader-error now
produces three first-wave errors. These execution counts establish no throughput,
latency, model-token saving or hardware sizing result.

Use focused comparisons when receiving the work:

```sh
python3 fde-cache-coalescing.py --case waiter-timeout --callers 3
python3 fde-cache-coalescing.py --case source-changed
python3 fde-cache-coalescing.py --case separate-keys
```

In the three-caller timeout case, caller 2 keeps its timed-out observation;
caller 3 receives the value after release. One caller ending its wait does not
make the other waiter's decision for it. Cancellation policy must distinguish
ending an individual wait, stopping shared work and cancelling a downstream
operation. This exercise induces only the first of those.

Source-changed shares captured revision 1 with both callers after the driver
advances the current source to revision 2. Coalescing did its narrow job: one
build. It did not refresh input or make publication correct. Continue with the
[late-publication proof](/training/guides/caching#7-run-the-late-publication-proof-lab)
to inspect an authoritative publication fence. Current resource authorization
still requires a separate trusted check before disclosure.

Separate-keys keeps North and South work apart. A document ID alone would not
define this fixture's result. In a real application, derive scope in trusted
code and identify every input that changes the result. Matching labels do not
authenticate a workspace, and coalescing is not a tenant-isolation test.

The registry lives in one process. A second process or replica has its own
registry; this lab supplies no cross-process suppression, durable lease,
crash recovery, distributed lock or restart evidence. It also supplies no
production admission bound on distinct keys, dependency execution timeout or
resource authorization. The driver limits its own workload to two or three
callers, and releases/joins its workers. Do not deploy this teaching manager.

## 6. Receive it as a delivery pod

Rotate three roles across another run:

- **Builder:** preserve runtime, exact command, captured input and the event log.
  Identify the registration lock, cleanup boundary and one intentional failure.
- **Receiver:** run the three-caller burst without author takeover. Explain one
  joined call, one failed or timed-out caller and one later independent attempt.
- **Application/platform owner:** name trusted key derivation, current validity,
  wait ownership, dependency cancellation, distinct-key admission and the actual
  process/replica boundary. Choose one bounded receiving-environment experiment.

Keep individual observations and assistance separate from the team's explanation:

```text
Learner / run / runtime / exact command:
Prediction before the run:
Requested key / joined existing work:
Actual first-wave builds / later builds:
My caller outcome / independent completion evidence:
Captured input revision / current source revision:
My contribution / assistance received:
Changed caller count / prediction / observed result:
Receiving action actually attempted / assistance:
Unresolved production boundary / accountable owner / next bounded experiment:
```

A receiver might next test an approved adapter with representative synthetic
requests, one unavailable dependency and a separately defined wait policy.
Document who owns each unresolved request after its caller stops waiting. Do
not test real customer work merely because this local exercise completes.

## 7. Defend the mechanism in an interview

**Explain:** Why is singleflight not a cache? Use the empty post-wave registry
and the later independent build as evidence.

**Debug:** A waiter times out, but another caller succeeds. Identify their shared
work, separate observations and actual stop boundary before recommending a retry.

**Design:** Specify a trusted result key, current source/access checks, bounded
waits, failure cleanup, mutable-result ownership and distinct-key admission.
Explain the additional evidence needed across replicas and process failure.

Primary references checked 28 October 2026:
[Go singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight) and
[Python concurrent futures](https://docs.python.org/3.12/library/concurrent.futures.html).
The Python fixture, comparisons, visual frames and receiving exercise are
original instruction. No Go package or production cache is activated.