AI native, under regulation: a field guide
Healthcare and financial services aren't anti-AI — they're anti-unaccountable. Treated as a design constraint rather than a blocker, regulation sharpens the system instead of slowing it.
Regulation is a constraint, not a blocker
A common objection in regulated-industry conversations is quiet and absolute: this is useful, but we can't do any of it. The useful answer is narrower. Regulation constrains unaccountable automation — systems that act on people, at scale, with no one able to explain, review, or reverse the action. That is not an anti-AI position. It is a specification, and specifications are what engineers build against.
In healthcare and financial services, the practical pattern is slower to authorize and easier to expand once the evidence is trusted. The controls that make the first scope harder to approve are also what make a later scope legible. Constraint, properly handled, becomes part of the system design rather than a waiver around it.
Accountability, concretely, means three things: every agent action is logged with its inputs and context; every output can be traced to a human disposition; and every system can be explained to an auditor without a translator. None of those is exotic. All of them have to be designed in from the start, because retrofitting accountability is where AI programs go to die.
Start where the work is operational
The winning pattern is boring on purpose: automate the operational penumbra before the core judgment. The workflows that work share a shape — high volume, structured inputs, reversible outputs, and a professional who already owns the decision.
Notice what's absent: diagnosis, treatment decisions, trading, credit decisions. The agent drafts, routes, checks, and queues. The licensed professional disposes. That split isn't a compromise with regulators — it's genuinely good system design, because it keeps accountability legible.
- Intake triage — routing and summarizing inbound requests before a human prioritizes.
- Documentation drafting — first drafts of notes and summaries that a professional edits and signs.
- Prior authorization preparation — assembling the packet, checking completeness, flagging what's missing.
- Reconciliation and exception queues — matching, diffing, and surfacing the five items that need eyes.
Sign-off is the load-bearing gate
A robust regulated deployment uses a human sign-off gate that is real, fast, and reversible. Real means a named professional approves the output before it acts on the world. Fast means the review surface is designed — diffs, highlights, one-screen decisions — so review takes seconds, not minutes; the reviewer sees the three things that changed, not the forty that didn't. Reversible means the action can be undone, so the cost of a mistake is a correction, not an incident.
The gate only works if review is fast. A slow gate trains people to rubber-stamp — and then you have automation with extra steps and less accountability.
Staged validation: retrospective, shadow, supervised, scaled
Trust is earned in stages, and skipping stages is how programs die. The sequence that works:
Each stage has a written exit criterion agreed with compliance before the stage starts. That sentence sounds like overhead. It's actually the accelerant: when the criterion is met, nobody relitigates whether to proceed.
- Retrospective — run the agent on historical cases and measure it against what humans actually decided.
- Shadow — run it live, in parallel, acting on nothing. Compare its outputs to the human's in real time.
- Supervised — the agent acts, but only through the sign-off gate, with every decision logged.
- Scaled — scope expands gate by gate, each with its own exit criteria and rollback plan.
Governance that ships: green, amber, red
The governance artifact that survives contact with an actual organization is a one-page zone map. Green: low-stakes, reversible, high-volume — the agent acts alone. Amber: meaningful stakes — the agent drafts, a professional signs. Red: never the agent, full stop. Every workflow in the system is labeled, every label is enforced in the tooling, and the map is reviewed on a cadence. This is not glamorous. It's what AI native under regulation looks like when it ships: less manifesto, more zone map.
The zone map isn't a policy PDF — it's enforced in the tooling. An amber workflow literally cannot send without a signature; a red one isn't wired to the agent at all. Governance that depends on everyone remembering the rules is governance that fails on a busy Friday.