Part 1, Understanding AI agents
Part 3, Introducing an agent
Principle 4
Review
Scope and limitations of this guide
This guide provides general information on organising work with AI agents, not advice tailored to a particular organisation, system or decision. It is not legal or regulatory advice.
Its examples, checklists and templates do not establish that a proposed use is safe, lawful or suitable. You need to assess and test the proposed use in your own circumstances. Following the guide does not by itself meet the requirements of any law or standard.
The guide is provided in good faith, using information available at the time of publication or revision. AI systems, provider terms and legal requirements can change. Before acting, check the current requirements that apply to your organisation and country, confirm what the system can do and test the proposed use. You are responsible for how you apply the guide and should obtain professional advice where appropriate.
Reading, downloading or using the guide does not by itself create an adviser–client relationship with Tom Clowry or Considered Ideas Ltd.
To the fullest extent permitted by law, Tom Clowry and Considered Ideas Ltd accept no liability for loss or damage arising from use of, or reliance on, this guide. Nothing in this notice excludes or limits liability for death or personal injury caused by negligence, fraud or fraudulent misrepresentation, or any other liability that cannot lawfully be excluded or limited.
How will you judge whether the agent’s work is accurate, relevant and appropriate?
Check the agent’s work at the points where your judgement is needed. Anyone checking it needs the evidence, time and authority to act on what they find.
- What to decide
- Who checks what, with which evidence and how much time.
- Key point
- Approval requires access to the evidence needed to judge the work.
Why it matters#
When approval is not a meaningful check#
Approval is not a meaningful check if you cannot examine the evidence needed to make the decision. Agree what needs to be checked, by whom, and allow enough time to do it.
Oversight without time or visibility#
Madeleine Clare Elish describes a “moral crumple zone” in which a person bears responsibility or blame for an automated system despite having limited control over it.
Assigning someone responsibility for an agent’s work is therefore insufficient unless they also have the information, time and authority to intervene.
The information
The time
The authority to intervene
Source Elish, M. C. (2019), Moral crumple zones, Engaging Science, Technology, and Society, 5, 40–60.
Background#
From psychology
Automation bias#
A review of research by Raja Parasuraman and Dietrich Manzey found that people using automated aids often fail to notice the aid’s errors, or follow its incorrect advice. This is known as automation bias. It was found in experts as well as beginners, and training or instructions did not prevent it.
In a related effect known as complacency, a reviewer who sees mostly correct work can stop looking closely. Principle 4 therefore asks you to decide what needs your attention, to provide the evidence and to check apparently routine work as well as the problems the agent reports.
Source Parasuraman, R. and Manzey, D. H. (2010), Complacency and bias in human use of automation: an attentional integration.
Fairness#
Fairness and discrimination#
This section is not legal advice. Its examples are drawn from UK law at the time of writing. The law can change, and the rules may differ in the EU and elsewhere. Ask the person who advises your organisation which rules apply to you.
An agent can treat people differently because of patterns in its training or in the records it uses. In Great Britain, the Equality Act 2010 protects people from direct and indirect discrimination, whether a decision is made by a person, by an automated system or by both. Northern Ireland and other countries have their own equality legislation.
Indirect discrimination arises where a rule or practice that applies to everyone puts people who share a protected characteristic at a particular disadvantage, and it cannot be shown to be a proportionate means of achieving a legitimate aim.
Age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex and sexual orientation. Indirect discrimination does not cover pregnancy and maternity.
In Great Britain, public bodies also have a public sector equality duty. They must have due regard to the need to eliminate discrimination, advance equality of opportunity and foster good relations when they decide whether and how to use an agent.
Sources Equality Act 2010, sections 4, 19 and 149. ICO, What about fairness, bias and discrimination?, under review following the Data (Use and Access) Act 2025.
Checking for unfair results#
Do not let an agent contribute to decisions about people until you have checked whether its results differ between groups.
Test results across groups
Compare the agent’s decisions for different groups and examine whether any differences are justified by the task. Check for errors and unjustified differences. Where there is no single correct answer, have qualified reviewers assess the reasoning and outcomes.
Check what the agent relies on
Information such as a postcode can stand in for a protected characteristic even where nobody intended it.
Keep decisions about people open to review
For significant decisions, begin with a suitably qualified person reviewing the evidence and deciding the outcome. Any proposal to remove that review needs a separate legal and risk assessment.
Check again after changes
Repeat the tests when the model, instructions or records change.
In practice#
5 steps
01Decide what needs your attentionYou cannot check everything, so choose the conclusions and actions that need your judgement.
You cannot check everything, so decide which conclusions and actions need your attention, and allow the time and information to assess those carefully.
Decide which situations always need attention, such as conflicting sources or a proposed action outside the Brief The instructions given to the agent, including the purpose, permitted information and actions, checks and arrangements for seeking advice..
02Place checks where they can make a differenceA single approval at the end can come too late.
Place checks where they can prevent an error or change a decision. A single approval at the end can come too late.
Where checks can make a difference:
Before the agent begins
Examine the instructions.
During the work
Decide unusual cases as they arise.
Before the result is used
Check calculations and important claims.
Afterwards
Inspect a selection of apparently routine work.
03Provide the evidence for reviewYou should be able to examine the original evidence.
You should be able to examine the original evidence rather than depend entirely on the agent’s account of its own work.
For a recommendation, provideThe sourcesImportant assumptionsAlternatives consideredFor an action, retainWhat the agent was asked to doWhich permission authorised the actionWhat changedAgent workOutputs and records→Checks and questionsChanges and exceptions→Human decisionEvidence and authority↑↑Use the findings to improve instructions and checksCheck a sampleOriginal sources and routine workReviewing the agent’s work before a decision Checks must also be capable of finding errors that the agent has not flagged.
04Use a defined standard where one existsSome work can be checked against a defined standard before you see it.
Some work can be checked against a defined standard. Software can be run against its tests, and a calculation can be compared with a known result.
Where such a check exists, an agent can run it, correct its work and run it again before you see the result. This can reduce what you need to review.
Set a limit on repeated attempts and require the agent to report unresolved failures. Changes to the agreed acceptance criteria need separate approval.
A person sets the check↓The agent does the work→The agent runs the checkPasses→A person inspects a selection↑ Does not pass: the agent corrects the work and runs the check againChecking work against a defined standard before a person reviews it What you still need to do#
The check does not showWhether the standard is the right oneAnything the check leaves outYou still need toDecide who sets the checkInspect a selection of the resultsOther judgements, such as whether a recommendation suits the people affected, cannot be settled by an automated check alone.
05Account for everyone’s timeA task can finish sooner while requiring more human effort.
A task can finish sooner while requiring more human effort. Include time spent briefing, answering questions, checking, correcting and maintaining sources when you judge whether the change is worthwhile. Look at whose time is being used as well as the total.
Usual processComplete the task→Wait for review→Review→Use the resultWith an agentBrief the agent→Agent works→Check and correct→Use the resultPeople’s timeWaiting or the agent’s workCompare the total time people spend on each approach.
Comparing human effort in two illustrative approaches Consider the mix of work left with each person#
If the agent handles the easy cases and one colleague receives only the difficult exceptions, their work can become harder even though they handle fewer cases.
Discuss with them what support and variety the role needs.
See also · 1.3 How an agent works through a task · 1.3 How agents go wrong
Examples#
The opening hours example
Checks before the decision#
A local service is open from 9 am to 5 pm on weekdays. Some service users have asked for evening appointments, and the team must decide whether longer hours would improve access. An agent examines twelve months of request records and compares the costs of the options. See chapter 1.5.
Before the options discussion, checks that relevant records have been included, the calculations are correct and the assumptions are appropriate.
Examine whether the proposed hours could be covered as described.
If a cost assumption changes, identifies which conclusions depend on it and prepares an updated comparison.
The people making the decision can see what has changed and why.
Illustrative example
Why specialist review remains necessary#
A team uses AI to draft professional learning materials. Drafting becomes quicker, but passages that read fluently can still misstate a professional requirement.
The team therefore allows for specialist review when estimating the time saved.
Illustrative example
Comparing AI with simpler process changes#
A membership organisation compares AI-assisted drafting with clearer standard answers and simpler enquiry routing. Its initial estimates suggest similar time savings. Under more cautious assumptions, however, reviewing and correcting AI drafts could add work. The organisation therefore tests the simpler process changes first.
Illustrative estimates. The cautious scenario assumes less time saved and more time spent checking and correcting AI drafts.
In other work
Routine customer enquiries#
Risks#
What about mistakes?#
Agents can make errors, and an early error may carry into later steps. Before delegating, consider how serious an error could be, who could be affected, how widely it could be repeated and whether its effects could be corrected.
Let the agent continue within its agreed permissions, with checks of the final result or a sample of the work.
Require approval before the relevant action, or enforce limits that prevent it from exceeding its authority.
Test these arrangements on your own work before relying on them.
What the reviewer does not see#
A reviewer who sees only the agent’s output may miss relevant information that the agent omitted.
From time to time, carry out a piece of the work without the agent and compare the results.
Exercise#
Ask someone to review a real result#
Ask a colleague to review a completed piece of the agent’s work. Can they establish what it did, inspect the supporting evidence and identify anything that needs correction? Check whether they have enough information and time to make that assessment.
If they have to repeat the work to assess it, improve the records and explanations provided. If the review happens after someone has already relied on the result, consider which checks need to take place earlier.
State what each reviewer must assess, the evidence they will receive and the time allowed. Explain what happens if they disagree or cannot respond, and how you will check work in which the agent has reported no problems.
Further reading · What is an AI agent harness? · clowryai.com/thinking
© 2026 Considered Ideas Ltd. Licensed under the Creative Commons Attribution 4.0 International licence (CC BY 4.0). This guide provides general information and is not legal or regulatory advice. The guide’s home page gives the suggested credit.
Scope and limitations of this guide
This guide provides general information on organising work with AI agents, not advice tailored to a particular organisation, system or decision. It is not legal or regulatory advice.
Its examples, checklists and templates do not establish that a proposed use is safe, lawful or suitable. You need to assess and test the proposed use in your own circumstances. Following the guide does not by itself meet the requirements of any law or standard.
The guide is provided in good faith, using information available at the time of publication or revision. AI systems, provider terms and legal requirements can change. Before acting, check the current requirements that apply to your organisation and country, confirm what the system can do and test the proposed use. You are responsible for how you apply the guide and should obtain professional advice where appropriate.
Reading, downloading or using the guide does not by itself create an adviser–client relationship with Tom Clowry or Considered Ideas Ltd.
To the fullest extent permitted by law, Tom Clowry and Considered Ideas Ltd accept no liability for loss or damage arising from use of, or reliance on, this guide. Nothing in this notice excludes or limits liability for death or personal injury caused by negligence, fraud or fraudulent misrepresentation, or any other liability that cannot lawfully be excluded or limited.

