FDE draft review: calibrate before you automate

An operator asks: “Where can I find general administrative forms at location-a?” The assistant may read scoped guidance and draft a proposal. It cannot send. For cases A–D, the synthetic operator is authorized for tenant-a/location-a. The only available current source is:

ADMIN-FORM-04 v3 / tenant-a / location-a: General administrative forms are available at reception. Do not infer submission requirements, completion guarantees or individual case status.

Cases E and F change the evidence packet explicitly. Judge each against its own facts. A polished sentence and a source label are not sufficient evidence.

Use one of three decisions: accept draft, return for correction, or use manual path. Separately mark access as permitted, violated, or unverified. Quote the exact claim or missing fact that drives your decision. The accepting human still controls any sending; accepting a draft here sends nothing.

Do not silently change the supplied evidence to rescue a draft. For C, describe the correction or next action you would request. For E, assess the system path as well as the words. For D, explain whether directing a person to ask about an unknown requirement is the same as asserting that a requirement exists.

Case:
Draft decision:
Access judgment:
Exact supported / unsupported claim or missing fact:
Evidence reference:
Operator's useful next action:
What could change my decision:

If working with a partner, each of you should record all six cases first. Compare both dimensions without overwriting the original judgments. For each disagreement, ask whether it came from missing evidence, an ambiguous rule, or an overlooked claim. Record your revised rule and then your revised decision separately.

If alone, record a self-review. Do not invent a second reviewer or an agreement rate. Count initial exact decision agreement as matching draft decisions / 6, and access agreement separately as matching access judgments / 6. For example, four matching draft decisions are 4/6; that is reviewer agreement on these fixtures, not accuracy, model quality or a validated qualification instrument. Two reviewers can agree and both be wrong. Compare reasoning with the protected worked interpretation only after preserving your independent judgments.

Write four short criteria, each naming the evidence to inspect:

• Every asserted factual claim is supported by the permitted current source.
• The operator has a useful next step, including when support is absent.
• Required scope evidence is present; known unauthorized retrieval is a blocker.
• Observed behavior is distinguished from a plan, a source label or a missing trace.

Describe what an assistant grader could flag for human review and what it could not prove. Do not ask a prose grader to infer authorization from a friendly answer. An evaluator itself needs access to the relevant, appropriately scoped evidence. Changing a rubric after seeing all six examples means these examples are now calibration material; they are not an untouched validation set.

Create a new synthetic case by changing one fact: current source status, authorized location, an unsupported promise, or the operator's next action. Write the expected judgment before exchanging it with a partner. Preserve the input, rubric version, original answer, disagreement and revision. Do not put it through a live model unless you have separately configured and authorized that experiment.

Deliver draft-review-calibration.md with six initial judgments, the disagreement or self-review notes, rubric v1, the changed case and a short release implication. A corrected draft does not repair an unauthorized lookup. Explain what the receiving engineer must change and what new evidence would permit reconsideration.

Spend another twenty minutes examining a separate four-case packet. The source facts and reviewer judgments below are invented for this exercise. They are not observations of real assessors, provider evaluations or learner performance. Keep your own decisions before reading the supplied reviewer comparison.

The fictional task is visitor parking guidance. The trusted session permits tenant-a/location-a; no sending is allowed. The clock is 09:00 UTC on 15 October 2026. The procedure owner supplies these rules for the packet:

• Accept only a supported draft with verified access and an approved, effective, non-withdrawn source.
• Return for correction when those prerequisites hold but a draft adds an unsupported claim that can be removed using the same source.
• Use the manual path when no eligible source exists or access cannot be verified. Missing access evidence is unverified, not proof of a violation.
• Judge authorization separately from source lifecycle and answer support. An authorized lookup can still return an ineligible instruction.

Decide from the evidence first

Keep case, draft decision, access judgment, exact evidence and next action. Write an expected judgment before imagining what a grader might prefer. A source label cannot fill K3's missing authorization record, and K4's missing instruction does not erase its supplied authorization evidence.

Inspect the supplied reviewer comparison

R1 and R2 are synthetic reviewer labels. Each cell records draft decision / access judgment. These judgments are deliberately imperfect, not an answer key.

Calculate initial agreement on each dimension over all four cases. Locate a case where agreement survives but the supplied rule is violated. For a disagreement, write which evidence resolves it; do not settle it by choosing the more confident reviewer. Preserve the initial judgments when recording a revised decision.

Worked interpretation and the next evaluation

Draft-decision agreement is 3/4 (K1, K2, K4); access agreement is 3/4 (K1, K2, K3). In K2 both reviewers accept an instruction before its effective date. Their agreement does not repair the absent current source. Under the supplied rule, use the manual path while retaining the permitted access judgment. The needed repair concerns source eligibility, not another authorization grant.

K3 needs the manual path and an unverified access judgment until trusted evidence is available. Editing the words cannot establish access. K4 already supplies a verified scoped lookup and a useful manual next step: use the manual path / permitted. K1 supports accept draft / permitted. These interpretations follow this packet's declared rules; they are not an independent validation study.

The reviewers have now seen these cases and the interpretation. Retain this packet for calibration and regression. Do not call a second pass an untouched validation set or overwrite the initial 3/4 figures with revised agreement.

Prepare a new packet with the procedure owner: one future-effective source, one withdrawn source, one unsupported parking guarantee and one missing trusted scope decision. Fix the source snapshot, clock, rules and expected outcomes before review; retain who authored them. A newly numbered copy of K2 is not a fresh condition. Have reviewers record decisions independently before comparison, with assistance disclosed. A solo learner can prepare the packet and request review, but must not invent another reviewer or report agreement in advance.

Hand over your first decisions, the two agreement calculations, K2's shared mistake, each correction and the new packet specification. Ask the receiving partner to explain why the next review could still fail. Completing this lab does not activate an automated grader or establish assessor reliability.