How to verify AI output in a law firm or audit practice

Written byEleazer Fordjour QuaysonFounder, Asle Data Labs · 23 October 2026

In June 2025, the High Court in London heard two cases together. In the first, a pupil barrister had put five authorities before the court that do not exist. In the second, 18 of the 45 citations in witness statements filed by a client and his solicitor were fictitious. The client had produced them with AI tools, and the solicitor had relied on them without checking. In a single judgment on both, R (Ayinde) v London Borough of Haringey; Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin), the court said that anyone who uses AI for legal research has “a professional duty therefore to check the accuracy of such research by reference to authoritative sources”.

Notice what the court asked for. It didn’t ask lawyers to read more carefully, because reading is exactly what failed. AI has broken the oldest signal in professional work, that a draft which reads well was done well. It is on this basis that, in a law firm or an audit practice, reliance on an AI draft should not be contingent merely on the fact that it reads well. It should wait until every fact in it has been traced to its source.

Polish used to be proof of work

Partners have always judged work partly by how it reads, and they were right to, because polish used to cost effort. Complete citations in the right form usually meant someone had pulled the reports, since there was no other way to get them right. Rushed work gave itself away, in a missing page reference or a total that didn’t agree with the schedule, and an experienced reviewer knew where to look.

AI produces the look without the work. Its drafts arrive with complete citations and confident prose whether or not anything behind them is real. The distance between that first draft and what a professional would actually sign is the first-output deficit, and its real danger is that you can’t see it. The draft reads as though the gap were closed.

That invisibility explains a reflex researchers have measured. In 2015, Berkeley Dietvorst, Joseph Simmons and Cade Massey of the Wharton School in the US ran five experiments in which people chose between a forecasting algorithm and a human forecaster. People who had watched the algorithm err trusted it less and chose it less often, even when they’d seen it beat the human. The studies concerned forecasts, not chatbots, but I read the pattern like this. A junior’s mistakes usually show, in a missing reference or a figure that doesn’t add up, while software’s look just like its correct answers. Once you’ve seen one, you can’t trust any of it by reading, so people stop using it.

Better tools shrink the deficit without making it visible. In 2024, researchers at Stanford and Yale in the US tested the AI legal research tools then sold by LexisNexis and Thomson Reuters. Built for lawyers, they made fewer errors than a general chatbot, and each still made things up between 17% and 33% of the time. Damien Charlotin’s public database of court decisions in which someone relied on made-up AI content held 2,202 cases on 19 October 2026, eleven of them from South Africa, Tanzania, Kenya, Uganda and Zimbabwe. None is listed from Ghana yet. Yet.

A case that reads right is not a case that is right

The reason runs deeper than any one product. Legal research tools such as those in the Stanford study fetch passages before they answer, usually by turning text into numbers and comparing them. This is text vectorisation, and there are several ways to do it, from counting how often each word appears to methods that try to capture what a passage means. Those numbers measure similarity, which is how alike two pieces of text read. They can’t tell whether a case is binding, still good law or real. For instance, an appeal court that reverses a judgment often sets out the facts and the lower court’s reasoning at length before disagreeing with it, so the two judgments share much of their wording. To a similarity search they’re close neighbours, though only one of them is good law.

The problem is sharper with Ghanaian judgments, because they’re written in highly uniform language. Two judgments can share most of their wording and still decide different things. That uniformity is what made Ghanaian cases hard to separate by wording alone when I tested four of these methods on them for my MSc. A tool that retrieves by similarity will therefore find the judgment that reads closest to your question, and the model will cite it with complete confidence, whether or not it’s good law or supports your point.

Every fact goes back to its source

If reading can’t catch the error, the check has to stop being reading. A firm needs three checks, in a fixed order, with a way back when one fails.

The first is what went into the tool. Client confidentiality covers everything typed or pasted in, and far more than personal data. Even an audit client’s ledgers, which may hold few names, are confidential. Where personal data is involved, such as client names, ID numbers, salary schedules and bank details, the Data Protection Act, 2012 (Act 843) applies as well. Section 28 asks for “appropriate, reasonable, technical and organisational measures” against unauthorised processing, and a provider that handles personal data on your behalf is a data processor, which section 30 says must be bound by a written contract to keep it confidential and secure. Section 30(4) then makes it your job to see that a processor outside Ghana follows Ghanaian law, and most well-known AI tools are run from outside Ghana. Take an associate who pastes a client’s staff salary schedule into a free chatbot to tidy up a table. The names, salaries and bank details have then gone to a provider outside Ghana that the firm has no contract with. If client data went into a tool without that agreement, the draft stops there, however good it looks. Your AI policy should say which tools may hold which information. If you don’t have one yet, start with our free AI use policy template.

The second check, tracing every fact, is the one that closes the deficit. Pull out every name, date, amount, citation and figure in the draft and find each one somewhere you trust, whether that’s the matter file, the engagement letter, the trial balance, the bank confirmation or the judgment itself on GhaLII or in the law reports. Asking another AI doesn’t count, and neither do the citations the first one supplied. Consider a summary that gives the monthly rent as GHS 4,500. That figure is checked against the tenancy agreement on the file, not against the AI’s earlier answer or anyone’s memory of the deal. The Act has a stake here too, because section 26 says a data controller “shall ensure that the data is complete, accurate, up to date and not misleading”, so an AI summary filed with the wrong date of birth or the wrong amount owed fails that test, whoever typed it.

The third is a sign-off by a named person, and it changes meaning once the second check exists. A signature used to mean “I’ve read this and it looks right”, the very judgement AI has made unreliable. Now it means “I’ve seen the sources”. Software can help by flagging a citation it can’t find or a figure that doesn’t match the ledger, but it can’t make the work right or compliant, and it can’t sign.

When any check fails, the draft goes back to whoever asked the AI, with the correction written down, and starts the route again. That written record is where most firms fall short. They leave checking to each person’s judgement, so one associate traces every citation and another only the ones that look odd, and the ones that look odd are exactly the ones AI doesn’t produce. The record also keeps people using the tool. In a 2018 follow-up, the same Wharton team found people “considerably more likely” to use an imperfect algorithm when they could modify its forecasts, even slightly. Kept for a few months, those corrections will show you which kinds of work the tool gets wrong in your firm.

Illustrative scenario. Say an audit senior used AI to draft the findings in a management letter. At the second check, the senior would tick every figure against the trial balance and the bank confirmations, and a receivables balance that didn’t match the ledger would go back corrected. The manager would then sign a letter with a source beside every number, rather than one that simply read well.

Automation can carry the draft, but people have to judge it

Automation is good at moving work along and bad at judgement. It can stop an AI-drafted letter being marked as sent until someone has signed it off, and it can set every citation and figure beside the place it should be checked. It can’t tell you whether a case supports your argument or whether a balance is fairly stated. That’s the split we design AI Workflow Automation around for law firms and tax and audit firms. In practice, we map how a draft travels from the person who asks the AI to the person who signs, and then build the checks into that path.

And so, how does this resolve? Through better models, or through better systems around them. My bet is on the systems, and Asle will be working even harder to help firms here bridge this gap. What remains of the first-output deficit will then be a remnant to look back on, from a time when Ghanaian professionals no longer have to choose between using AI and putting their name to the work.

Sources

• R (Ayinde) v London Borough of Haringey; Al-Haroun v Qatar National Bank [2025] EWHC 1383 (Admin), 6 June 2025, paragraphs 2, 6, 7, 36 to 39, 51, 74, 76 and 77. The National Archives
• Data Protection Act, 2012 (Act 843), sections 26, 28 and 30. Data Protection Commission
• Dietvorst, Simmons and Massey, “Algorithm aversion: people erroneously avoid algorithms after seeing them err”, Journal of Experimental Psychology: General, 2015. The Wharton School, University of Pennsylvania, US. doi.org/10.1037/xge0000033
• Dietvorst, Simmons and Massey, “Overcoming algorithm aversion: people will use imperfect algorithms if they can (even slightly) modify them”, Management Science, 2018. US. doi.org/10.1287/mnsc.2016.2643
• Magesh and others, “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools”, 2024. Stanford and Yale, US. arXiv 2405.20362
• Damien Charlotin, AI Hallucination Cases database, last updated 19 October 2026. damiencharlotin.com/hallucinations