Forward-deployed engineering / Free interactive practice
Choose the route you can defend.
Compare two AI configurations against quality, deadlines and a shared attempt budget. Keep your first decision and the evidence that changes it.
Start the decisionYour engagement
A faster draft still has to work.
A fictional front-desk team needs supported administrative drafts. One route is cheaper; another produces different arrival times. Your job is to decide which meets the agreed service contract.
Download the paired case packetMake a decision before you compare.
Use the CSV to make your first prediction. Then inspect the packet, change the deadline and explain what still needs an owner’s decision. The full assignment is below if you want the calculation steps.
Interactive field practice / Synthetic packet
Work the route decision.
Predict first. Inspect the evidence, change one requirement and keep your explanation.
- Draft arrives
- Before 1,000 ms
- Useful drafts
- At least 9 of 12
- Unsupported drafts
- None admitted
- Shared attempt limit
- 5 teaching credits
Use the supplied CSV and rules below. These are authored observations; no model runs here.
Preparing the interactive practice…
Read the full thirty-minute assignment
A sponsor asks whether the cheaper route or the more expensive route should handle the administrative drafting task. Decide from the complete operating contract, not a model label or a single speed score. Download the paired route-selection packet and open it in a spreadsheet or text editor. A calculator is enough.
The 24 rows are authored synthetic observations, paired across twelve requests for fictional configurations route-a-v1 and route-b-v1. They are not results from commercial models, a live router or this platform's production providers. All requests have supplied permitted lookup traces for tenant-a/location-a and the same current ADMIN-FORM-04 v3: “General administrative forms are available at reception.” The source establishes no identification requirement. A valid reference is not evidence that every sentence is supported.
supplied_outcome records each fictional candidate's disposition: every timely proposal is forwarded for operator review, including any unsupported claim. Neither candidate checks claim support before that step. Late proposals are discarded and manual handling begins at the deadline. Judge these supplied observations against your frozen rule; do not silently repair the candidate while counting its results. The CSV describes authored behavior, not an execution trace from a real adapter.
Freeze the operating envelope (6 minutes)
Record these supplied exercise rules before inspecting the CSV:
- Each request has one global deadline at 1,000 virtual milliseconds. An arrival at or after it cannot enter review; the trusted manual route is available at that deadline. A returned packet and an accepted draft are different events.
- At most five teaching credits may be reserved per request. Each admitted route-A call consumes two; route B consumes four, including failed or late attempts. These invented credits are fixed accounting units, not prices, tokens or measurements of provider billing. Unused capacity on another request cannot enlarge this request's limit.
- This packet compares one admitted call per request per configuration; each arm starts afresh. Only supported drafts received before the deadline are useful drafts. A late proposal is discarded even if its text is correct.
- The synthetic workflow owner requires at least nine useful drafts across these twelve requests and no unsupported draft admitted for operator review. A manual result is allowed behavior, but does not count toward that nine-draft requirement. These are task-specific teaching criteria, not a universal SLA.
- No route may broaden source scope, expose a forbidden record or send a reply. The supplied permitted traces cover this packet only; they do not establish revocation, cache isolation or another source's behavior.
Write the complete candidate identity: code/prompt/schema and route revision, authorized source, deadline, attempt policy and case set. If an identity field is not supplied, leave it unresolved rather than inventing a configuration value.
Reconstruct the decision from every row (10 minutes)
Keep both route records for each request. Add columns for timely arrival, claim support, accepted-for-review draft, manual outcome and credits consumed. Inspect the wording before labeling support. Mark which unsupported text was actually admitted before the deadline; do not mistake a discarded late proposal for an emitted draft.
| Observation to calculate | Route A | Route B |
|---|---|---|
| Requests in this arm | [count] | [count] |
| Timely proposals | [count and case IDs] | [count and case IDs] |
| Unsupported timely proposals | [count and exact added claim] | [count and exact added claim] |
| Useful drafts / all requests | [numerator / denominator] | [numerator / denominator] |
| Manual outcomes at the deadline | [count and case IDs] | [count and case IDs] |
| Consumed teaching credits, including late work | [sum] | [sum] |
| Supplied draft and boundary criteria met? | [decision and reason] | [decision and reason] |
Do not discard timed-out requests from the denominator or refund their admitted work. A cheap supported output and a useful complete service are different claims. Retain the evidence behind a recommendation to withhold both candidates if neither meets the declared contract; the task does not require a winner.
Challenge two proposed shortcuts (8 minutes)
- “Try A, then switch to B whenever A is unsuitable.” Reconstruct the shared reservation: A's two credits plus B's four. Does it fit five? If A times out, how much of the same 1,000-ms deadline remains? A fallback is a new attempt within the original scope, source and resource limits, not a free reset. An unsupported proposal already shown to the operator is not repaired by silently changing routes afterward.
- “Change the deadline to 1,001 ms and call B a pass.” Recalculate the one boundary row this changes. Does a numerical match give the engineer authority to extend the operator's agreed wait? Identify the owner, the changed requirement and the new candidate evidence needed before reconsideration. Keep the original 1,000-ms result intact.
Propose one small next candidate, with a prediction and a new counterexample. A source or output validator can address a particular unsupported claim; a prompt tweak alone does not establish that repair. A revised model, route policy, source or deadline changes the candidate. Reuse relevant regression cases while obtaining fresh cases before claiming transfer.
Let a receiver choose the next evidence (6 minutes)
Give a partner your table and a five-sentence decision note without explaining it first. Ask them to find ROUTE-09's unsupported addition, the exact-deadline refusal and the fallback's shared-budget problem. Retain their actual objection and one revision. Solo work is self-review. End with the receiving role that accepts the operator's wait and the observation needed to reconsider your next candidate. No real route is selected or activated by this exercise.
For an optional ten-minute extension, open Defend a changed source in the hosted route-selection lab. Write a first decision and explanation for the new request before consulting its authored reasoning. Revisions keep the original attempt in your local comparison record; neither the selection nor your writing is an independent assessment. A timely proposal can still be unusable when its source was withdrawn.
Keep the calculation, original criteria, changed-deadline comparison and receiver note with portfolio section 04. The protected evaluation lesson offers worked interpretation and a targeted fallback-budget check. This small public packet cannot establish performance, commercial value or enterprise capacity.
Continue the engineering work
Turn the comparison into a release decision.
The evaluation lesson connects the route packet to expected behavior, evidence, review and a scoped recommendation. Bring your own first attempt and the requirement you would challenge.
Open the evaluation lessonRequires existing Academy access. This free exercise keeps temporary work in your tab; download your comparison before leaving.