Insights · AI Architecture · 11 min read

Let the AI veto. Never let it authorise.

Most teams ask whether an AI is accurate enough to be trusted with a decision. That is the wrong question, and it produces systems that are fragile in a specific and avoidable way. Here is the better question, and what it looks like in a system that risks real money.

In short

  • The question is not whether the AI is accurate enough. It is whether anything irreversible happens on its say-so alone.
  • In a system that risks real money, the AI is allowed to veto a trade and to lower its confidence. It cannot place one, size one, or overrule a limit.
  • Every rule that actually grants permission is ordinary code reading ordinary numbers: a loss ceiling, a position cap, a count of open trades.
  • When the AI layer errors, the system approves by default. That reads as reckless until you notice the AI was never the thing granting permission.
  • Find your own seam by asking one question of every component: if this returned confident nonsense, could money move?

There is a conversation that happens in almost every organisation adopting AI, and it always takes the same shape. Somebody asks whether the model is good enough to be trusted with a particular decision. Everybody debates accuracy. Someone quotes a benchmark. A pilot is run, the numbers look encouraging, and the decision gets handed over.

The question sounds rigorous. It is not, because accuracy is the wrong axis entirely.

A better question, and one that turns out to be much easier to answer, is this: when this component is confidently wrong, what happens next? If the answer is "something irreversible", no accuracy figure is high enough. If the answer is "a deterministic check catches it and nothing moves", then a surprisingly mediocre model is perfectly safe to deploy.

That reframing changes what you build. It stops being an argument about model quality and becomes an argument about where the permission lives.

The system this is drawn from

I run a personal trading system. It watches a watchlist, evaluates strategies against price data, decides whether to open a position, and places the order with a broker. It runs against a paper account and a live one. When it is wrong, it loses actual money, which concentrates the mind considerably better than a demo does.

It uses AI heavily. Claude synthesises research from several sources, writes a lesson after every losing trade, and gives a second opinion on every signal before anything is placed. None of that is decorative. It genuinely improves the quality of what the system does.

And yet the AI cannot open a position. It cannot size one. It cannot raise a limit, extend a budget, or overrule a circuit breaker. It has no path to any of those things, not because it is distrusted, but because those capabilities were never wired into that layer in the first place.

The five gates a trade signal must pass A horizontal pipeline. A deterministic strategy raises a signal, then it passes AI validation, a budget check, the risk manager and a human approval gate before an order is placed. Every gate can block. Only the deterministic strategy can originate a trade, and the AI gate is marked as the only one that cannot grant permission. A signal passes five gates before it becomes an order Each gate can stop a trade. None of them except the first can start one PROBABILISTIC gate shown in outline. DETERMINISTIC gates shown filled. Strategy raises the signal CODE AI validation veto or downgrade AI Budget check per symbol ceiling CODE Risk manager loss, size, count CODE Approval human, live only ORDER bracketed Every gate can say no cannot say yes
Figure 1. A signal passes five gates before it becomes an order. Only one of them is the AI, and it is the only one that cannot say yes.

What the AI is actually allowed to do

A deterministic strategy raises the signal. Plain code, reading price data and indicators, produces a candidate trade with a stop loss and a target already attached.

That candidate then goes to the AI, which is given the research context, the macro picture, and the upcoming high-impact calendar, and asked one narrow question: should this be vetoed? It returns a boolean and a confidence figure it is permitted to revise downward.

Read that again, because the shape of it is the whole point. The AI's output space is a veto and a reduction. There is no value it can return that causes a trade to happen. A trade was already going to happen when the deterministic strategy raised the signal. All the AI can do is stop it, or make it smaller.

After the AI has had its say, the candidate still has to clear a budget check, then the risk manager, then, on the live account, a human approval queue with a five minute expiry that rejects by default. Only then does an order get placed, and it goes out as a bracket order with the take profit and stop loss attached atomically, so even the exit does not depend on anything staying up.

What the model owns and what the code owns Two columns divided by a vertical seam. The left column, judgement, lists reading research, weighing context, vetoing a signal, lowering confidence and writing a lesson after a loss. The right column, permission, lists sizing the position, checking the budget, enforcing the loss ceiling, capping open positions and placing the order. A note states that nothing crosses the seam. The seam: judgement on one side, permission on the other Both are necessary. Neither is allowed to do the other's job JUDGEMENT probabilistic, best model you can justify PERMISSION deterministic, readable in one sitting Read the research and the news Raise the signal in the first place Weigh the macro calendar Size the position Veto a candidate trade Enforce the daily loss ceiling Lower a confidence score Cap the number of open trades Write a lesson after a loss Place and bracket the order nothing crosses this line
Figure 2. The seam. Judgement on the left, permission on the right. Nothing crosses.

The risk manager is worth dwelling on, because it is the least glamorous file in the project and it is doing the most important work. It is under a hundred lines. It runs three checks: has the day's realised loss passed a ceiling, is this position larger than the cap, are there already too many positions open. Each one is a number compared against another number, read from a database and an environment variable.

There is no cleverness in it at all. That is the feature. It behaves identically at three in the morning on a volatile day as it does in a calm test run, it can be reasoned about completely by reading it once, and it cannot be talked into anything.

The most important component in an AI system is usually the least interesting one to build. If your risk layer is fun to write, it is probably doing too much.

The property that looks like a bug

Here is the part that makes people uncomfortable when I describe it, and it is the part I would most want a team to sit with.

When the AI validation step fails, the system approves by default. If the API errors or the response cannot be parsed, the code logs a warning and lets the signal through unvalidated. If the model is not configured at all, it passes through even more quietly.

Every instinct says this is backwards. Surely a safety component should fail closed. Surely losing your validation layer should stop trading, not wave everything past.

It should, if that component were a safety component. It is not. It is a second opinion.

Nothing downstream of it got weaker when it failed. The budget check still runs. The risk manager still runs, with the same loss ceiling and the same position cap. The human approval gate on live trades still runs, and still rejects on timeout. Losing the AI costs you a filter that would have caught some bad trades that the deterministic gates will now let through at their own limits. It does not cost you a rail.

The same outage in two architectures Two stacked scenarios. In the first, the AI authorises, so when it fails every control is lost at once and trading must halt. In the second, the AI only vetoes, so when it fails the deterministic budget, risk and approval gates all still stand and the system keeps running with one filter missing. What an AI outage costs you depends on where permission lives Same model, same accuracy, very different Tuesday AI authorises the model grants permission AI layer unavailable Budget check never existed Risk manager never existed Approval gate never existed Halt trading AI vetoes only code grants, the model objects AI layer unavailable Budget check still standing Risk manager still standing Approval gate still standing Carry on, one filter down
Figure 3. The same outage in two architectures. Where permission lives determines whether a degraded AI is an inconvenience or an incident.

Now imagine the common alternative, the one that gets built when nobody has drawn this line. The AI decides whether to trade, how much to commit, and whether the position is within tolerance. The reasoning is that the model sees more context than any hard-coded rule, which is true. In that system, an AI outage or a malformed response is not a degraded filter. It is the loss of every constraint at once, and the honest response is to halt trading entirely until it comes back.

Both systems use the same model with the same accuracy. One treats an outage as a reduced-quality day. The other treats it as an emergency. The difference is not the model. It is a decision about wiring that was made long before the model was chosen.

Fail-open is not a value judgement about your AI. It is only safe when authorisation lives somewhere else. Move the permission into the model and the same code becomes reckless.

Finding the seam in your own system

The pattern generalises past trading, and the test for it is a single question you can ask of any component in any system you are building.

If this returned confident nonsense, could anything irreversible happen?

Not "would it be wrong". Confidently wrong is the interesting case, because a model that fails loudly is easy to handle. It is the plausible, well-formatted, entirely incorrect answer that walks through a system unchallenged.

Work through your components with that question and they sort themselves into two piles quickly. Reading, extracting, summarising, ranking, drafting, judging, explaining: all of these are safe for a model to own, because being wrong produces a bad suggestion, and a bad suggestion is recoverable. Placing, paying, sending, deleting, publishing, approving, granting: none of these are, because being wrong produces a fact.

Once the piles exist, the rule is mechanical. Everything in the second pile is implemented as ordinary code with explicit limits, and where the consequence is severe enough, a human confirms it. Everything in the first pile can use the best model you can afford, because the cost of it being wrong is bounded by design.

A few things fall out of this that are worth stating plainly.

  • Your gates should be boring and readable. If somebody cannot hold the whole risk layer in their head after one read, it is too clever to be trusted with the job it has.
  • Constrain the output space, not just the prompt. Asking a model nicely to stay in scope is not a control. Giving it a return type that has no way to express "yes, proceed" is.
  • A veto layer degrades gracefully. An authorising layer does not. This is the single most useful property to design for, and it is nearly free if you decide early.
  • Log the vetoes, not just the failures. The record of what the AI stopped is how you find out whether it is earning its place, and it is the first thing you will want when a trade goes badly.
  • Put the human gate on the irreversible edge only. Approval on everything trains people to click through. Approval on the handful of actions that produce facts stays meaningful.

What it costs

I would rather not present this as free, because it is not.

You pay in latency, since every signal now waits on a model call before it can proceed. You pay in a certain amount of duplicated judgement, because the deterministic gates sometimes block things the AI had already sensibly approved, and reconciling the two takes thought. You pay in the ongoing temptation to let the model do a little more, which arrives every time you watch it correctly reason about something your hard-coded rule handles crudely.

That last one is the real cost, and it is worth naming because it does not feel like a cost at the time. It feels like an improvement. The model is genuinely better at weighing an unusual situation than a threshold comparison is, and every instance where you watch it be right builds a case for handing it a little more authority.

The answer is not that the model is untrustworthy. It is that the value of the arrangement comes from the boundary itself, not from any particular judgement made on either side of it. Sound reasoning and a hard limit are not competing approaches. The limit is what makes it safe to let the reasoning be ambitious.

The useful question

Accuracy debates are comfortable because they feel technical and they can be deferred to a benchmark. They are also close to useless for deciding what to build, because they produce a percentage and no architecture.

The question that produces an architecture is the other one. For every component, ask what happens when it is confidently wrong, and be honest about the answer. The components where the honest answer is "a fact gets created" are the ones that must not be probabilistic, regardless of how good the model gets.

Let the AI read everything, weigh everything, and object to anything. Give it the best model you can justify and the widest context you can assemble, because that is where it earns its keep. Then make sure that the word "yes" belongs to something you can read in one sitting.

Dave Carr
Dave Carr MBCS CITP
Founder and Principal Consultant, DCCO Ltd

DCCO is an independent technology consultancy working across the UK and Europe. Founded in 2009, it covers AI adoption, rapid software development, and vendor-neutral advice on digital signage, DOOH, and IT and AV transformation.

Next step

Finding that seam in your own systems is what the AI Readiness Review does: which components could move money or create a fact on their own say-so, and where the permission boundary actually belongs.

Talk it through, 30 minutes, no charge More insights