# Caching 101: when can you reuse earlier work?

An operator opens the same guide twice. Rebuilding its excerpt every time feels
wasteful. Saving the first result seems obvious—until the guide changes or the
operator loses access. How does the application know when reuse is still valid?

This original 45-minute workshop starts with that question. You will predict
eight decisions, inspect real local SQLite rows and create two separate defects
in disposable copies. All organizations, documents and text are fictional.
Python 3.11 or later is sufficient: no packages, model, credentials, network or
cloud account. The program uses an in-memory database and writes no files.

## 1. Name the object and the technique

A **cache** stores a copy of data or earlier work for reuse. Caching is a
technique, not a model or programming language. A **hit** finds an eligible
entry; a **miss** requires the application to fetch or compute the result again.

If you know build systems, compare a saved excerpt with a compiled artifact.
When an input changes, a dependent output can become obsolete. The analogy has
a limit: access to AI context can also change independently of its content.
A content hash cannot tell you whether a person may read the copy now.

| Term | Plain meaning | Question to ask |
| --- | --- | --- |
| Cache key | Identifier used to find a saved entry | Could two requests share this key but require different results? |
| Source of truth | Authoritative value or permission record | Where can I establish what is current? |
| Time to live (TTL) | Allowed entry lifetime | When does age alone force a miss? |
| Invalidation | Making an earlier entry unusable | Which source, permission or configuration change triggers it? |
| Eviction | Removing entries to reclaim capacity | Can the application still work correctly after removal? |

Before choosing a cache product, name the object. Here it is a formatted source
excerpt. It is neither an answer generated by an AI model nor proof that the
excerpt is sufficient to answer a person's question.

## 2. Follow the dependencies before the saved copy

The fictional operator has access to both North and South. Both workspaces have
a document called `guide-7`, but their text differs. The lab's cache key includes
the active tenant, principal, document and renderer version. Each entry also
records its source revision and expiry.

The baseline checks current permission and source metadata on every request.
Only then does it look for a matching entry. A missing, obsolete or expired
entry triggers an actual local render and database write. Permission loss
withholds the result and deletes that caller's entry. Missing current evidence
withholds without reading the saved body.

The rendering function only formats a string with its source reference. This
makes the mechanism inspectable without a model. A hit skips that function; it
does not skip the current metadata lookup or establish an end-to-end speedup.

## 3. Predict and run eight requests

Download [fde-caching-lab.py](/templates/fde-caching-lab.py) into a new folder.
Predict the result for every row below before running it:

```shell
python3 --version
python3 fde-caching-lab.py
```

| Case | Logical second | Change to inspect |
| --- | --- | --- |
| cold | 0 | North asks for guide-7 with no saved entry |
| warm | 1 | Same workspace, caller, guide and current revision |
| other-tenant | 2 | Same operator selects South and its guide-7 |
| source-changed | 3 | North's guide changes from blue version 1 to green version 2 |
| permission-revoked | 4 | North's grant is removed while the entry is young |
| access-restored | 5 | North's grant is restored; make a fresh request |
| evidence-unavailable | 6 | Current evidence cannot be obtained despite the saved entry |
| expired | 10 | Current evidence is available; the entry reaches its age limit |

These are supplied logical seconds, not measured runtime or sleeps. The
five-second TTL uses an inclusive expiry boundary: `now >= expires` is expired.
The unavailable-evidence case uses an explicit failure flag; it does not take a
database or cloud service offline.
The restoration request is deliberately visible. Without it, the previous
revocation would leave no warm entry for the unavailable-evidence case.

Inspect `cache`, `decision`, `body`, `current_version`, `returned_version` and
`render_calls_this_request`. Explain each observation in a sentence before
checking the reasoning below.

## 4. Compare the evidence and change one rule

The baseline produces:

- `cold`: `miss-empty`, North version 1, one render.
- `warm`: `hit`, North version 1, no render.
- `other-tenant`: `miss-empty`, South's amber version 1, one render.
- `source-changed`: `miss-source-changed`, North's green version 2, one render.
- `permission-revoked`: `withheld`, no body and no cache read.
- `access-restored`: `miss-empty`, North version 2, one render.
- `evidence-unavailable`: `withheld`, no body and no cache read.
- `expired`: `miss-expired`, North version 2, one render.

There are five renders, one warm hit and two withheld results. This is an
operation count for one synthetic sequence. It is not a measured improvement
in latency, token cost or learner performance.

Make two **separate fresh copies** of the original file:

1. In the first copy change `CHECK_SOURCE_VERSION = True` to `False`. Run it.
   At `source-changed`, the entry is still young, so the broken copy returns
   blue version 1 even though `current_version` is 2. The stale result identifies
   the dependency that the check protected.
2. In the other fresh copy change `CHECK_CURRENT_ACCESS = True` to `False`.
   Run it. At `permission-revoked`, the broken copy returns the warm green
   excerpt despite the removed grant. The key and TTL did not enforce authority.

Keep the baseline and both reports separately. Re-enable the check in each copy
and confirm the corresponding defect disappears. These mutations demonstrate
bugs in an isolated exercise; they are not configuration recommendations.

## 5. Separate application reuse from model caching

An application might cache a source excerpt or a completed answer. Reusing an
answer additionally requires a contract for the task and its model, prompt,
source and policy dependencies. Similar wording does not establish equivalent
permission or an equivalent answer.

Claude's prompt caching can reuse processing of eligible repeated input
prefixes. It does not turn a saved final answer into a current one. Use the
provider's current eligibility and usage documentation when measuring that
feature; this exercise implements no provider cache or token billing.

A transformer's **key-value (KV) cache** is a different inference mechanism for
reusing attention-related state. Continue through the NVIDIA memory material
for the model-runtime layer: [GPU memory 101](/training/fde-101?topic=gpu-memory#concept).
Our SQLite excerpt cache does not model GPU memory.

Before optimizing any layer, identify the repeated work and what evidence would
show a useful improvement. Include misses, validation and invalidation overhead
in the comparison. A high hit count alone says nothing about answer quality.

## 6. Write a contract another engineer can challenge

Complete this template for one source cache in your proposed AI assistant:

```text
Cached object:
Trusted caller and active workspace:
Key fields:
Authoritative source and permission dependencies:
Maximum age and expiry boundary:
Events that invalidate reuse:
Behavior when current evidence is missing:
Allowed result and refused result:
Work actually avoided; work still performed:
Concurrent update/revocation test still needed:
Receiving owner and rollback condition:
```

The local program uses one sequential process. It does not prove atomic
revocation across concurrent workers, distributed cache coherence, durability,
memory bounds or provider behavior. If many workers miss at once, they can
duplicate expensive work; describe how you would test and bound that load.
For sensitive data, decide whether a cache is appropriate before designing it.

Explain the baseline to another engineer without reading the code: what was
cached, what changed, what was returned, and why? Then let them choose one
dependency you missed. That discussion is the handoff artifact.

Further reading, checked 22 October 2026:

- [AWS: what is caching?](https://aws.amazon.com/caching/)
- [Microsoft: cache-aside pattern and consistency considerations](https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside)
- [Anthropic: prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
- [OWASP: check authorization for each request](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html)

These references support further study. The fictional case, experiment and
teaching diagram are original LockedIn Labs material; no provider affiliation
or certification is implied.