Investigate retrieval before trusting the answer

The sponsor says, “Every answer has a citation, so let's release.” Your job is to show which citations actually support an authorized, current instruction. Produce one reviewable decision with the cases that changed it, then explain the smallest repair you would investigate next.

Spend 8 minutes establishing source authority, 12 classifying the eight requests, 10 calculating separate measures and 10 defending a repair. Keep your first judgment and corrected version. These are learning artifacts, not review approvals or qualification evidence.

For this exercise the trusted session permits tenant-a, location-a only. The task may draft parking-access, form-location and desk-hours guidance. It cannot send a message. The fixture clock is 09:00 on 15 October 2026. These are supplied rules for this exercise, not a production policy recommendation.

An eligible source must match the trusted tenant, permitted location and request kind, be approved, be effective at the fixture clock and not be withdrawn. A newer version or a higher search score does not override these conditions. Conflicting eligible instructions require the procedure owner's resolution; do not blend them.

The following teaching packet deliberately shows foreign-source metadata so you can recognize an isolation defect. A real unauthorized caller must not receive the foreign source or its existence through an error message.

Q4 and Q5 below modify only their own source snapshot. Reset the packet before the next row. Search rank is deliberately omitted: no ranking can grant authority.

Before reading the observed run, mark each request as answerable, manual or denied. For answerable cases, name the eligible source. For the others, identify the missing authority or unresolved fact and the responsible next step.

Use a row with these fields: case ID; expected outcome; eligible source or stop reason; observed retrieval; observed answer; defect; operator next action. A fluent answer is not a reason to change the expected outcome.

The rows below are authored observations, not output generated by this platform or a provider. “Retrieved” means content reached the drafting context, not merely that an internal index contained a document.

Describe Q2 and Q7 separately. One crosses a tenant boundary; the other selects an instruction before its effective date. Q3 demonstrates a different failure even though retrieval found the intended source. Do not reduce all three to “prompt needs improvement.”

Show the case IDs in every numerator and denominator so another engineer can reconstruct your result.

1. Eligible-source coverage: among answerable requests, how many retrieved at least one eligible source? This measure can pass a row that also retrieved an ineligible source. It is not isolation or answer quality.
2. Acceptable emitted drafts: among requests that emitted a draft, how many stayed within the authorized, effective source instruction without an unsupported claim or unresolved conflict? Do not exclude failed drafts from the denominator.
3. Correct stopping: among requests whose expected result was manual or denied, how many stopped before drafting with the appropriate next action?
4. Cross-scope exposure: which requests passed another tenant's content into the drafting context? Record the incident even if no answer is later shown. Never average it away inside an overall quality score.

Also identify the future-effective and withdrawn documents that reached the drafting context. They are lifecycle defects even when no cross-tenant access occurred. Report the counts as properties of eight supplied fixtures, not estimates of production reliability.

Choose the earliest enforcing boundary for each defect: trusted authorization and scoped retrieval, lifecycle/conflict resolution, or output support. Specify what happens when the relevant metadata cannot be resolved. A missing verdict cannot silently become permission to continue.

For a repair proposal, record the change, its owner, the cases it should affect, and the new observation needed. Predict the effect on each measure before claiming improvement. Re-running these authored examples after a repair would still be regression evidence, not unseen performance evidence.

Have the receiver change one fact: the operator loses location-a access after retrieval but before the answer returns. Identify the check and consistency contract required at that boundary. A cached source reference and an earlier successful read do not establish current access.

Spend another twenty-five minutes investigating five independent pagination snapshots. A single eligible result does not establish that it is the only eligible instruction. An empty first page does not establish that the source set is empty. This is a tabletop design exercise; it does not add pagination to the starter or query a live source store.

For these cases only, the procedure owner offers an authoritative catalog for one trusted tenant, location and request kind. Its declared protocol provides a stable snapshot, opaque continuation cursors and an explicit complete result. That guarantee must be demonstrated by an actual adapter before it can be used. Completing a relevance-ranked search's pages would not by itself establish an exhaustive authoritative catalog. Keep those two retrieval contracts distinct.

Use the original fixture clock and authority rules. Before comparing with the worked interpretation, write observed pages, coverage known or incomplete, eligible instructions, allowed output, stop reason and next owner for each row. All page contents and failures below are synthetic supplied observations.

Compare the interpretation with your first decision

P1 must not return the first instruction merely because that page arrived first. The complete snapshot reveals conflicting eligible records. Use the manual path and assign the dispute to the procedure owner. Receiving every page establishes coverage under this fixture contract; it does not resolve disagreement. Keep both observations rather than choosing whichever instruction looks familiar.

P2 has incomplete coverage, not an established missing-source result. Preserve the transport failure and use the manual path with a source-lookup-unavailable explanation. A useful interface can let the operator consult the established trusted procedure route without falsely saying that no procedure exists. Do not record an unobserved page as empty or call the draft supported by a complete set.

P3 shows a repeated cursor and no complete snapshot. Stop the cycle immediately; the three-attempt cap is a maximum, not a reason to make a known-repeating third request. Do not draft from the repeated partial result or consume an unbounded retry budget. Record the two actual attempts, repeated cursor and unavailable coverage for the adapter owner. In a real implementation, keep continuation tokens out of ordinary operator logs; this packet uses only synthetic C1.

P4 cannot fetch or return another page on the remembered cursor alone. Recheck trusted authorization at that boundary and bind continuation state to its original scope. A cursor is navigation state, not an access credential. Stop the affected path and route the permission question to the access owner; do not change the request's tenant or reuse a different account to finish the packet.

P5 supports the reception instruction under the supplied complete-snapshot contract. The empty first page was an intermediate observation. Preserve both pages, their common scope and snapshot, the completion evidence and the exact eligible source. Do not generalize this result into a claim that any successful search response establishes complete source coverage.

Give the receiver an adapter contract

Write a short contract naming the trusted scope, snapshot identity, continuation rule, completion evidence, page/time budget and repeated-cursor stop. State what happens if an adapter mixes snapshot identities or loses the completion marker. Fail incomplete rather than silently treating a partial result as a complete source set. This design also needs the existing lifecycle and conflict checks; pagination cannot replace them.

Have a receiving partner select P2 or P5 without announcing which. Explain the first empty page before they reveal the next observation. Revise the outcome only when that evidence arrives, then ask them to reproduce the distinction between incomplete and complete coverage. Keep your first decision, changed observation and revised record. If working alone, label it self-review and leave the partner's observation unverified. This exercise establishes no live adapter guarantee or learner qualification.

Run the catalog fixture yourself (another 15 minutes)

Download the offline catalog-coverage lab. Keep the reference unchanged and use Node.js 24; no package installation, credentials, model account or network connection is required by the program. It returns scripted proposals and metadata, never sends or persists anything. Its fixed rows stand in for an authorized catalog's eligible records. It does not implement real identity, lifecycle filtering or a production snapshot.

Predict each result before running its command:

node fde-catalog-coverage-lab.mjs lookup-interrupted
node fde-catalog-coverage-lab.mjs empty-then-supported
node fde-catalog-coverage-lab.mjs cursor-loop
node fde-catalog-coverage-lab.mjs access-revoked

lookup-interrupted makes two page attempts and returns manual / source-lookup-unavailable with incomplete coverage. empty-then-supported also makes two attempts, but complete coverage permits the reception proposal for human review. A two-attempt count alone cannot establish the same outcome. cursor-loop stops after two attempts; access-revoked stops before a second fetch. Locate the predicate responsible for each difference in the small file. The JSON trace contains no source instruction; only the supported proposal contains text. A manual result does not publish a partial source as a draft.

Three additional cases make the design easier to challenge: snapshot-changed rejects a completed second page from a different snapshot; page-budget stops after three fetch attempts without requesting its fourth supplied page; complete-empty establishes missing-source from a completed empty snapshot. later-conflict is P1. Use --all if you want all eight authored observations. The program's exit status reports whether a selected exercise ran, not whether the fictional service is ready to release. Manual outcomes are intended results.

In a separate copy, replace snapshot !== result.snapshot with false and run snapshot-changed. Compare the original manual result with the changed proposal: the code has combined two different snapshots and no longer establishes its declared coverage contract. Restore the check and retain both observations. This is a deliberately weakened teaching copy; it is not a proposed repair. Explain why a plausible final source cannot validate how that source set was assembled. Give your receiver the file revision, command, actual JSON, changed line and unresolved production assumptions. Mark anything you did not run as expected, not executed.

Your record should include the fixed clock and authority rules, all eight decisions, four measures with case IDs, three distinct failure causes, the release decision and an explicit manual next step. Include your first mistaken judgment if the exercise changed it.

Ask the receiver to locate the evidence behind one rejected draft without your narration. If they cannot, improve the record before expanding the system. Do not turn a completed worksheet into a claim that an engineer is qualified or that a customer system is safe.

Continue with the client-delivery workbook. Guided interpretation and review questions sit in the Academy lesson “Design a minimal administrative service with enforced boundaries,” under existing training access.