We turned off BoardPath's deterministic safeguards on purpose and ran the Q&A and Ask Steward paths naked. No tool constraints. No hooks. Just the context window and the system prompt, doing what a well-written system prompt is supposed to do.
It didn't take long. Confident, fabricated answers started showing up in the eval runs. Not hedged. Not flagged. Governance answers about provisions that did not exist, delivered in the same tone as the correct ones.
The prompt was not weak. It said, in plain language, to abstain when the corpus does not address the question. It said it clearly, it said it early, and it said it more than once. The model read it. Then, under enough retrieval pressure, it did something else.
That is the thing worth understanding. A system prompt is not a constraint. It is a request — a very well-phrased one, sitting in the same context window as everything else, competing for attention with the retrieved chunks, the conversation history, and the user's own framing. It has no special status. It is text. When the pressure gets high enough, it loses.
So I stopped treating prompts as the control layer.
There are four places you can put a guardrail, and they are not equivalent. The ordering is not about elegance. It is about where the enforcement actually lives — and whether the model's reasoning is in a position to argue with it.
PreToolUse and PostToolUse interception. The model proposes; the hook decides. A retrieval call goes out, comes back with a confidence score below threshold, and the hook kills the generation before the model ever sees the chunks. The model does not know it was stopped. There is nothing to reason around.Run each layer's violation through your head and the ordering stops being a preference and starts being a consequence.
A violated tool permission fails loudly, at the boundary, with a stack trace. A violated hook fails loudly. A violated workflow step fails loudly — the graph just stops.
A violated prompt fails silently. The model produces a fluent, confident, well-formed answer that happens to be invented, and nothing in the system raises its hand. You find out when a board member acts on it.
Every layer above the prompt converts a silent failure into a loud one. That is not a nice-to-have in an environment where the output is going to be read as governance guidance by a volunteer board with fiduciary duty. A loud failure costs you a debugging session. A silent one costs somebody else a decision they cannot take back.
The ablation test is the part I'd push anyone building in a liability-bearing domain to actually run. Not because you'll be surprised that prompts alone are insufficient — everyone says they know that. But because there is a difference between knowing it and watching your own system, with your own carefully written prompt, confidently fabricate a provision in front of you.
Do it in an eval harness, on purpose, where the only thing that gets hurt is your assumptions.
Build the version that breaks first. Then build the layers that make the break loud.