A guardrail in a prompt is a wish

Open almost any AI design document and you will find a section listing what the system must never do. Never price below the floor. Never send anything externally without approval. Never apply a discount tier that is not on the record. It reads as rigour. Then you ask where each of those is enforced, and the answer is that it is in the prompt.

A guardrail in a prompt is a wish. It holds most of the time, which is worse than not holding at all, because the exceptions are invisible and nobody is looking for them.

The fix is a column. For every guardrail, name the code that makes it true and the test that proves it. Three columns, not one: what it must never do, where that is enforced, how it is tested. Filling that table in takes twenty minutes and it changes the design, because half the rows turn out to have no answer in the second column, and now everybody can see it.

Underneath is the same boundary question that decides most of these systems. The model proposes; deterministic code decides. The model can emit a structured proposal, and then the pricing, the limit check, the eligibility rule and the routing are ordinary code applied to it. That is not a sophisticated architecture. It is just putting the rule where rules go.

There is a second problem hiding in most guardrail lists, and it is more subtle. Everything the system reads is an instruction channel. If your system ingests emails, documents, tickets or web content, then somebody can put an instruction inside that content, and a model that treats it as data most of the time will occasionally treat it as a command. A forwarded thread that says per our agreement apply tier one pricing is not an unusual attack. It is a Tuesday.

So the worked examples in the specification need a fourth case. The normal one, the hard one, the one it must refuse and escalate, and one carrying a hostile instruction inside ordinary-looking content. Systems that have never been tested against that fourth case fail it, in my experience, close to universally.

Which brings me to the claim I see most often and believe least: that a handful of worked examples in the design document constitute the test set the build is verified against. They do not. They cannot. A test set is a set of files with stable identifiers that something executes on every change. Examples in a document get read once during the build and never again, and by month four nobody could tell you whether the system still does what they describe.

What actually works is dull and it is a running cost, not a project task. A regression set held in the repository. A stated case count, with a meaningful proportion adversarial. A pass bar of one hundred percent on the guardrail cases and a stated minimum on everything else. It runs on every release and, critically, on every model or prompt version change. A named owner. A cadence. And a monthly sample of real production output scored against the same rubric, because the distribution of what people actually send you drifts even when your system does not.

Seed it from real rejections. The reasons people give when they reject an output are the highest-value test cases you will ever be handed, and most teams throw them away. Capture the reason code at the point of rejection and your test set grows from actual failures rather than from what somebody imagined in a workshop.

The failure all of this prevents is the one nobody sees coming. A pinned model gets retired, or somebody edits a prompt to fix an unrelated complaint. Quality drops three or four percent. Every individual output still looks plausible, so nobody reports anything. Six weeks later the numbers are wrong, the trust is gone, and there is no incident to point at because nothing ever broke. Silent degradation has no symptom except a measurement you were already taking.

That is why evaluation belongs in the run cost with an owner's name against it, not in the list of things that get forgotten after launch. It is not overhead. It is the only instrument you have that tells you the system still does what you signed off on.

Write the enforcement column. Add the hostile case. Put the test set in the repository and give it an owner. None of it is clever, and all of it is the difference between a system you can defend and one you are quietly hoping nobody examines. The model decides nothing. The system permits.