Caching 101: when can you reuse earlier work?

A cache stores a copy of data or earlier work for reuse. Caching is a technique, not a model or programming language. A hit finds an eligible entry; a miss requires the application to fetch or compute the result again.

If you know build systems, compare a saved excerpt with a compiled artifact. When an input changes, a dependent output can become obsolete. The analogy has a limit: access to AI context can also change independently of its content. A content hash cannot tell you whether a person may read the copy now.

Before choosing a cache product, name the object. Here it is a formatted source excerpt. It is neither an answer generated by an AI model nor proof that the excerpt is sufficient to answer a person's question.

The fictional operator has access to both North and South. Both workspaces have a document called guide-7, but their text differs. The lab's cache key includes the active tenant, principal, document and renderer version. Each entry also records its source revision and expiry.

The baseline checks current permission and source metadata on every request. Only then does it look for a matching entry. A missing, obsolete or expired entry triggers an actual local render and database write. Permission loss withholds the result and deletes that caller's entry. Missing current evidence withholds without reading the saved body.

The rendering function only formats a string with its source reference. This makes the mechanism inspectable without a model. A hit skips that function; it does not skip the current metadata lookup or establish an end-to-end speedup.

Download fde-caching-lab.py into a new folder. Predict the result for every row below before running it:

python3 --version
python3 fde-caching-lab.py

These are supplied logical seconds, not measured runtime or sleeps. The five-second TTL uses an inclusive expiry boundary: now >= expires is expired. The unavailable-evidence case uses an explicit failure flag; it does not take a database or cloud service offline. The restoration request is deliberately visible. Without it, the previous revocation would leave no warm entry for the unavailable-evidence case.

Inspect cache, decision, body, current_version, returned_version and render_calls_this_request. Explain each observation in a sentence before checking the reasoning below.

The baseline produces:

• cold: miss-empty, North version 1, one render.
• warm: hit, North version 1, no render.
• other-tenant: miss-empty, South's amber version 1, one render.
• source-changed: miss-source-changed, North's green version 2, one render.
• permission-revoked: withheld, no body and no cache read.
• access-restored: miss-empty, North version 2, one render.
• evidence-unavailable: withheld, no body and no cache read.
• expired: miss-expired, North version 2, one render.

There are five renders, one warm hit and two withheld results. This is an operation count for one synthetic sequence. It is not a measured improvement in latency, token cost or learner performance.

Make two separate fresh copies of the original file:

1. In the first copy change CHECK_SOURCE_VERSION = True to False. Run it. At source-changed, the entry is still young, so the broken copy returns blue version 1 even though current_version is 2. The stale result identifies the dependency that the check protected.
2. In the other fresh copy change CHECK_CURRENT_ACCESS = True to False. Run it. At permission-revoked, the broken copy returns the warm green excerpt despite the removed grant. The key and TTL did not enforce authority.

Keep the baseline and both reports separately. Re-enable the check in each copy and confirm the corresponding defect disappears. These mutations demonstrate bugs in an isolated exercise; they are not configuration recommendations.

An application might cache a source excerpt or a completed answer. Reusing an answer additionally requires a contract for the task and its model, prompt, source and policy dependencies. Similar wording does not establish equivalent permission or an equivalent answer.

Claude's prompt caching can reuse processing of eligible repeated input prefixes. It does not turn a saved final answer into a current one. Use the provider's current eligibility and usage documentation when measuring that feature; this exercise implements no provider cache or token billing.

A transformer's key-value (KV) cache is a different inference mechanism for reusing attention-related state. Continue through the NVIDIA memory material for the model-runtime layer: GPU memory 101. Our SQLite excerpt cache does not model GPU memory.

Before optimizing any layer, identify the repeated work and what evidence would show a useful improvement. Include misses, validation and invalidation overhead in the comparison. A high hit count alone says nothing about answer quality.

Complete this template for one source cache in your proposed AI assistant:

Cached object:
Trusted caller and active workspace:
Key fields:
Authoritative source and permission dependencies:
Maximum age and expiry boundary:
Events that invalidate reuse:
Behavior when current evidence is missing:
Allowed result and refused result:
Work actually avoided; work still performed:
Concurrent update/revocation test still needed:
Receiving owner and rollback condition:

The local program uses one sequential process. It does not prove atomic revocation across concurrent workers, distributed cache coherence, durability, memory bounds or provider behavior. If many workers miss at once, they can duplicate expensive work; describe how you would test and bound that load. For sensitive data, decide whether a cache is appropriate before designing it.

Explain the baseline to another engineer without reading the code: what was cached, what changed, what was returned, and why? Then let them choose one dependency you missed. That discussion is the handoff artifact.

Further reading, checked 22 October 2026:

• AWS: what is caching?
• Microsoft: cache-aside pattern and consistency considerations
• Anthropic: prompt caching
• OWASP: check authorization for each request

These references support further study. The fictional case, experiment and teaching diagram are original LockedIn Labs material; no provider affiliation or certification is implied.