AI governance under Consumer Duty

This article is a practitioner’s view of how to design for the Consumer Duty. It is not legal or regulatory advice. Check your own position with your compliance team and advisers.

Firms regulated by the FCA often ask what the rules on AI are. The honest answer is that there is no separate set. The FCA has said its approach is outcomes-focused and largely technology-neutral, and that it will rely on existing frameworks. Its AI Update sets that out.

For retail business, the most important of those frameworks is the Consumer Duty. It does not mention AI and does not need to. It asks whether customers get good outcomes, and it asks the firm to show its evidence. A decision made with AI is held to the same standard as one made without.

That has a practical consequence. You do not need a new governance regime for AI. You need to be able to answer the Duty’s existing questions about anything AI touches.

What the Duty asks

The Duty requires firms to act to deliver good outcomes for retail customers. It is organised around four outcomes: products and services, price and value, consumer understanding, and consumer support. Alongside them sit cross-cutting rules: act in good faith, avoid causing foreseeable harm, and enable and support customers to pursue their financial objectives.

Firms are expected to monitor the outcomes customers receive, and boards are expected to review an assessment of them at least once a year.

Put AI into that frame and the governance questions more or less write themselves.

Four outcomes, four sets of questions

Products and services. If AI shapes who is offered a product, or how it is configured, does it still reach the customers it was designed for? Could it steer some groups towards something unsuitable?

Price and value. If AI influences pricing or retention offers, can you show that the price remains fair value for different groups of customers, including those who do not shop around?

Consumer understanding. If AI writes or personalises communications, are they still clear, and tested? Does a customer know when they are dealing with an automated system, where that matters to their decisions?

Consumer support. If AI handles service, can a customer still do what they came to do, including complain, cancel or reach a person? Support that is easy when buying and hard when leaving is a problem the Duty names directly.

Then the cross-cutting rule that carries most weight for AI: avoiding foreseeable harm. Errors, bias against particular groups and poor outcomes for customers in vulnerable circumstances are all foreseeable with these systems. Foreseeable means you are expected to have thought about them beforehand.

Design it in

Most governance trouble comes from sequence. A team builds something. It works. Then risk and compliance are asked to approve it, and the design is reopened.

It is cheaper to ask the Duty’s questions at the start, when they are design inputs. For each workflow AI will touch, I would want five things written down before a build begins.

1. Who is accountable. A named person for the decision, not a committee. Under the Senior Managers regime someone already holds the responsibility. The design should make clear who.
2. What stays with a person. Which decisions a human makes, which AI assists, and which AI takes on within agreed limits.
3. How a decision is reviewed and reversed. The route for a customer or a colleague to challenge an outcome, and for the firm to correct it.
4. How outcomes will be monitored. Which measures will show whether customers are getting good outcomes, broken down by the groups that matter, and who looks at them.
5. What would make you stop. The threshold at which the system is paused or withdrawn.

This is the Duty applied to a workflow. Done first, it shapes a better product and removes the late surprise.

What boards should ask

A board does not need to understand how a model works. It needs to be able to govern the outcomes. Five questions do most of that.

• Where does AI touch a customer outcome in this firm today?
• For each of those, who is accountable?
• What evidence do we have that outcomes are good, and for which groups?
• What has been stopped or changed because of that evidence?
• What are the assumptions behind the business case, and how firm are they?

If the answer to the first question is a list of committees, the governance is not yet real.

What I learned doing it

Before I started Neurally I led the launch of Compare the Market’s first consumer AI assistant, and the first ChatGPT app for insurance, inside an FCA and Consumer Duty environment.

The lesson I took from it is about timing. When compliance and risk colleagues are in the room while the product is being shaped, their questions improve it. When they see it for the first time at the end, the same questions feel like obstacles, and everyone loses weeks.

That is career experience, not a Neurally client result. But it is why I treat governance as part of the design work in every engagement, and why the AI Transformation Deep Dive includes a governance and risk view next to the plan a board is asked to approve.

Other rules still apply

The Duty is not the whole picture. UK GDPR governs how personal data is used, retained and shared with third parties, and has specific provisions on automated decisions. The EU AI Act can reach UK firms whose products or services reach the EU. Both are worth flagging early, while a design can still change, and both are questions for your legal advisers.

Where to start

Pick one customer-facing workflow where AI is in use or planned. Write down the five things above. If any of them is hard to answer, you have found the work that matters most.

There is more on how this fits into a wider programme on the AI governance page, and on what it means for financial services and insurance firms in particular.