Before you blame the model, inspect the contract
A composite failure scenario shows how an ambiguous tool schema can turn a capable model into an unreliable operator — and why a common protocol is plumbing, not proof of contract quality.
The failure surface the model cannot explain
Agent evaluations often concentrate on the model while giving less attention to the seam between the model and its tools. That seam can create production failures even when the underlying model is capable. The model's job there is narrow: select a tool and serialize arguments into its schema. When the contract is ambiguous, the model has to infer. Fluent, confident, well-formatted inferences can still be wrong.
Model capability is easy to benchmark; contract quality is harder to summarize on one dashboard. A useful diagnostic order is therefore: inspect the contract first, the prompt second, and the model third. It is a review heuristic, not a claim that every agent incident has the same cause or resolution time.
Composite postmortem: the enum that wasn't
Consider a synthetic composite assembled from common contract failures; it is not a postmortem from a LockedIn deployment. A reconciliation agent tries to close cases with status 'done'. The tool's schema declares an enum — resolved, cancelled, duplicate — but its description says 'mark the case as done when complete'. The model follows the prose and violates the schema. Every call fails.
In the scenario, the tool returns errors as free text — 'invalid status value' — and the harness maps every tool error to a generic retry. Same arguments, same result, forty-one times, until a rate limiter ends the loop. The case stays open and the queue backs up. Three small contract defects compose into an outage: prose that contradicts the schema, an error the model cannot parse, and a retry policy that never inspects the error.
The corrective design is equally concrete: make the description mirror the enum, return structured error codes with remediation hints, and stop retrying on INVALID_ARGUMENT. No model upgrade is required to remove the constructed failure path. That is the point of the example — not a claim about an incident or result in our own stack.
The model wasn't hallucinating. It was guessing at a contract that contradicted itself.
Anatomy of a good tool contract
A tool contract is an API designed for a counterparty that reads literally and fills blanks confidently. Start with this review checklist:
None of this is exotic. It's the discipline good API teams have always practiced, reapplied to a consumer that never gets embarrassed and never asks clarifying questions.
- Name it like a verb with no synonyms. If close_case and resolve_case both exist, the contract forces an unnecessary choice and invites a third interpretation.
- Strict schemas. Where an enum will do, never accept a free-form string. Where a number will do, never accept a sentence.
- Enumerated, machine-readable errors — a code, a message, and a remediation hint the model can act on.
- Idempotency keys on anything that mutates. Assume an agent may retry; design so a retry is safe.
- Documented side effects. If calling the tool sends email, moves money, or mutates state, the schema says so — loudly.
MCP is table stakes, not a cure
MCP standardizes an interface for exposing context and tools, and that is useful progress. But a protocol does not make descriptions precise or errors enumerable. A sloppy contract served over clean plumbing is still a sloppy contract. A server can conform to the protocol while exposing vague descriptions and permissive schemas; the model is still left to infer what the contract failed to state.
A common protocol can reduce bespoke integration work and make discovery more consistent. It does not prove that a model can use the discovered tool safely. LockedIn does not claim blanket MCP deployment or operating results from it. The design obligation remains the same whether a tool arrives through MCP or another interface: constrain the action, describe side effects, and make failure recoverable.
Contracts get reviewed like code
The operating rule that follows: schemas are code. Changes should get pull requests. Descriptions should be diffed, because a one-word edit can reroute model behavior. Error paths should be tested with an actual agent in the loop — not just unit tests asserting response shape, but controlled drills where the agent hits the failure and has to recover. Contracts should be versioned, with compatibility handled deliberately instead of assuming an agent read the migration notes. The goal is an agent system boring enough that its on-call rotation has little to say.