Skip to main content
Back to Insights
AI & Technology

The model proposes. The system decides.

Russ Reeder8 min read

When a company calls us in because an AI system is producing answers nobody trusts, the diagnosis is almost always the same, and it is almost never the model. Somebody asked a probabilistic component to make a decision that had to be right every time.

The line I keep coming back to is this: the model proposes, the system decides. Get that boundary in the right place and most of the rest follows. Get it wrong and no amount of prompt engineering rescues it, because you are trying to make something reliable by asking it nicely.

In practice that means the model produces structured proposals and deterministic code applies the rules, does the arithmetic, checks the limits and commits. Anything that must always be true lives in code the model cannot reach. A rule enforced inside a prompt holds most of the time, and the exceptions are invisible, which is the worst possible failure mode. A rule enforced in code holds every time and can be tested. If your margin floor, your approval limit or your eligibility check lives in a prompt, you do not have a control, you have a preference.

The second pattern is one model gateway. Every call goes through a single internal service where versions are pinned, cost is tagged per call and per unit of work, and prompts are versioned. Calls scattered through a codebase make cost unknowable and a provider change expensive. Model retirement notices are short, retired models fail outright rather than degrading politely, and version boundaries carry hard breaks. With a gateway, a vendor change is a scheduled afternoon. Without one, it is an outage during your busiest week.

Third, the schema is the single source of truth, held in one place and generating the data layer and the interfaces rather than being described in a document that drifts from the code within a month. This is the cheapest discipline available and the one whose absence costs most in month six, when the front end and the back end have quietly diverged and nobody can say which is right.

Fourth, and this is the one almost everybody skips: stamp full lineage on every output. Model version, prompt version, the version of every reference dataset that produced it, and a hash of the input, retained as long as the output is retained. The first challenged number becomes an investigation. Without lineage the answer to why did it say that is a guess, and once your prompt retention window has passed it is not even that. I have watched a team spend two weeks trying to reconstruct why a system produced a figure four months earlier, and fail.

Fifth, human review by tier rather than by habit. Actions get classified as free, approved or forbidden, and the routing is enforced in code. Anything irreversible or externally visible starts in the approved tier and earns its way down on measured evidence, never on optimism. Review applied everywhere is theatre and gets rubber-stamped within a fortnight. Review applied nowhere is uninsurable. Tiering puts human attention where the consequence actually is.

Sixth, reuse first and deviate on the record. Start from the patterns and platforms the organisation already runs, state conformity in three sentences, then list only the exceptions with a reason against each. Reuse is the highest-value architecture decision available and the one least often made deliberately. Making deviations visible is what stops an estate fragmenting one build at a time.

Seventh, evaluation as a running system rather than a test written once. A regression set held as files in the repository, executed on every release and on every model or prompt change, plus a periodic sample of production output scored the same way. Silent degradation has no symptom except a measurement you were already taking. A model gets retired, a prompt gets edited, quality drops a few percent, every individual output still looks plausible, and six weeks later the trust is gone with no incident to point at.

The rest follow the same logic. A designed swap path, rehearsed, with a named owner. Least-privilege service identity for anything that acts, never a person's borrowed credentials, and included in access review like any other account. Degrade to useful rather than to nothing, so that when the provider is unavailable the generation stops but the validation, the logging and the manual path keep running, because the business process existed before the build and has to survive its outage. And minimise at the prompt boundary, so the model receives the least data that lets it do the job, which is the one control entirely within your own hands regardless of what the vendor terms turn out to say.

None of these are exotic. What is striking is how consistently they are absent, and I think the reason is that AI builds get treated as experiments right up until the moment they are load-bearing. The pilot does not need lineage. The pilot does not need a swap path. Then the pilot quietly becomes the system, and nobody goes back.

The useful question in a design review is never whether a team followed the patterns. It is which ones they left, and what they got for it. A design that conforms needs three sentences and a list of exceptions. A design that deviates on half of them is not necessarily wrong, but it is carrying risk that somebody should be choosing on purpose rather than discovering later. Operations first. AI second. The model proposes; the system decides.

Russ Reeder, Founder & CEO of KeyDelta

Russ Reeder

Founder & CEO, KeyDelta | Forbes Technology Council

30+ years scaling technology companies as a CEO, COO, and operator across Oracle, GoDaddy, OVHcloud, Infrascale, Netrix Global, and XTIUM. Founder of Rightsline (Disney+, Hulu, Sony). Forbes Technology Council member. HBS Executive Education. Russ advises CEOs, PE-backed leadership, and management teams on execution clarity through the VOOCS operating system.

Want to discuss these ideas?

If your team is navigating execution challenges, we should talk.

Book a 30-minute call