# First reliability review: defend the evidence

LockedIn Labs · Edition 2026-10-30.1 · Separate 45-minute foundation route

This route prepares you to receive a technical claim and explain what its
observations support. Use Python 3.11+ and these two existing files in the same
disposable folder, retaining their filenames:

- [Agent evaluation companion](https://training.lockedinlabs.ai/templates/fde-agent-evaluation-lab.py)
- [Durable workflow companion](https://training.lockedinlabs.ai/templates/fde-agent-engineering-lab.py)

They use the standard library and temporary fictional SQLite records. The draft
is deterministic. There are no packages, accounts, keys, model or network calls.
This route does not require the larger guide's tracing or framework workshops.
Your redirected JSON files are local evidence; keep their first observations.

## 1. Declare what should happen — 10 minutes

Find `CONTRACT`, `observe` and `evaluate` in the evaluator. Explain which names
an expectation, which executes the candidate and which compares the result.
The contract is `field-desk-regression-v2` with seven planned cases:

| Case | Obligation to explain in your own words | First prediction |
| --- | --- | --- |
| authorized-draft | Useful drafting from the current source | |
| foreign-source | No source read or run from another tenant | |
| foreign-run | No inspection of another tenant's run | |
| changed-source | No approval based on an older source revision | |
| withdrawn-source | No approval after the source is inactive | |
| same-version-source | No approval after content changes without a version advance | |
| restart-replay | Reopen the record; preserve the original run and repeated decision | |

The tenant names in these domain methods are supplied fixtures, not verified
sessions. `withdrawn-source` changes source activity, not a reviewer's grant.
Read the actual expected fields beside each case before running candidates.
Do not silently change expectations to match the observed result.

My expectation / observation / decision distinction:
My selected file revision or hash and Python version:
My first predictions, prior reading and help:

## 2. Run and retain five packets — 15 minutes

Predict failed case IDs first. Run each command separately so a deliberately
blocked candidate does not prevent you from observing the next one:

```sh
python3 --version
python3 fde-agent-evaluation-lab.py run --candidate reference > reference.json
python3 fde-agent-evaluation-lab.py run --candidate refuse-all > refuse-all.json
python3 fde-agent-evaluation-lab.py run --candidate stale-review > stale-review.json
python3 fde-agent-evaluation-lab.py run --candidate version-only > version-only.json
python3 fde-agent-evaluation-lab.py run --candidate version-only --omit-case same-version-source > omitted.json
```

The reference exits 0 when its local contract is met. The other four comparisons
intentionally exit 2 with a JSON `blocked` decision. A missing file, Python error
or interrupted process is different: retain that actual condition or mark it
unfinished. Do not fill in the expected report as if you observed it. Redirection
does not change the exit status. The optional existing `test` command checks the
companion itself; it does not replace your five candidate observations.

| Packet | Actual command / exit | Utility | Boundary | Recovery | Failed IDs | Unrun IDs |
| --- | --- | --- | --- | --- | --- | --- |
| reference | | | | | | |
| refuse-all | | | | | | |
| stale-review | | | | | | |
| version-only | | | | | | |
| omitted | | | | | | |

Inspect `contract`, `planned`, `executed`, `slices`, `cases` and `decision`.
Compare the useful authorized draft with the foreign-source denial. Why would
counting only denials hide a system that refuses all useful work?

My first mismatch with the prediction and how I investigated it:
A useful case retained alongside the negative controls:
A report I could not execute / actual help received:

## 3. Distinguish failure from missing evidence — 10 minutes

In the full `version-only` packet, read `same-version-source` field by field.
The fixture changes the body while retaining version 1. What do `error`,
`status`, `version_unchanged` and `after_reopen` actually say? Compare them with
the expectation in `CONTRACT`. Reopening checks persistence of this sequential
result; it does not test concurrent source edits or distributed consistency.

Now read the same row in `omitted.json`. Record its state and whether an
observation exists. Keep the declared seven-case denominator when explaining
six passing observations. A failed case and an unrun case support different
statements, even though both prevent this complete contract from passing.

My full-packet observation and violated expectation:
My omitted-packet planned / executed / state / observation:
My decision, named case and remaining uncertainty:
A repair I would investigate without weakening the expectation:

Only compare with this interpretation after retaining your own writing: the
full version-only report has six passes and one content-related failure;
acceptance persists after reopening. Omission has six passes and one unrun row.
Both preserve seven planned cases, boundary 4/5 and a blocked decision. The
reference is green only for this local contract. None of these counts measures
model quality, human acceptance, production readiness or someone's capability.

## 4. Receive a changed requirement — 10 minutes

Give a receiving colleague the unchanged files and your packets. Before reading
your explanation, they run the full version-only comparison and locate the
content check. Each person retains their own prediction, actual command, fields,
explanation and assistance. A shared report cannot supply another person's run.

Change the requirement: a person's review access can be revoked, and the draft
will now come from a model. Keep the earlier observations; do not relabel source
withdrawal as reviewer revocation. Agree expected useful and refused behavior
before implementing or executing those new cases.

| Receiving responsibility | Actual person / evidence still required | Observed or unrun |
| --- | --- | --- |
| Identity/service owner: verified reviewer, current action/resource permission, allowed/revoked controls | | |
| Model/evaluation owner: exact configuration, source revisions, supported-answer criteria, fresh cases outside tuning | | |
| Receiving engineer: own run and changed-condition explanation | | |
| Delivery lead: unresolved dependencies, responsible owner and delivery decision | | |

The original foundation also asks for ordinary, missing-data, conflicting-identity,
stale, hostile-instruction and provider-failure reports. Map each to a real case
above or mark the gap. This program does not exercise model inference, hostile
model input, provider errors, authenticated reviewer revocation or provider
costs. Those conditions need different observations, not renamed rows.

My separate receiving account and help:
My proposed next case / agreed expected behavior / implementation still needed:
What the current evidence supports, and what remains unrun:

Continue in the [protected foundation lesson](https://training.lockedinlabs.ai/app/academy?course=architecture-foundations&lesson=context-evaluation&view=focus)
or the [larger free agent-engineering guide](https://training.lockedinlabs.ai/training/guides/agent-engineering).
Practice records do not grant production authority, independent delivery,
client placement or qualification.