Briefing
Episode 31 · Primer and discussion · About 15 minutes
Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.
Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.
Checked against the lab, government and regulator pages on 3 November 2026. The 58% and 70% figures have no published method, and the eight-to-four-weeks claim has no published data. Check the original sources before quoting a figure in a board paper.
Before we record, every claim the hosts make is checked against the sources (this table was last updated 3 November 2026). Confirmed means a primary source or two independent reports; Reported means one party says so; Partly right, Disputed, Unverified and Not yet known mean what they say, and the hosts say so on air. This is the table as we left it.
Sign up for our podcast at agile-insights.com.au.
Linda Here is the idea for today. When A I drafts a decision and a person signs it off, that sign off is only a safeguard if you can show the person is still checking. So a high share of drafts approved with no edits is a question to test, not a score to celebrate. By the end you will know what England is doing with A I in planning, and how to tell a real human check from a rubber stamp.
Leonard Let me set the scene, because most of us do not follow English planning. If you want a loft conversion or an extension in England, you usually need permission from your local council. A planning officer checks your application against the rules. On the thirty first of October, Google DeepMind and the U K government announced the next stage of a tool called PlanAssist. It is built on Gemini, with Google Cloud and a company called Faculty. It checks the documents and drawings, heritage limits such as tree preservation orders, local and national policy, and earlier objections. Then it drafts a report with citations. A qualified officer approves every decision.
Linda Very few of us do.
Linda And it matters now because of scale. PlanAssist was piloted in three councils, Barnet, Camden and Dorset. It now goes into a twelve month trial in fifteen councils, used alongside the normal process. The Budget commits to offering it to every council in England by the end of twenty twenty seven, subject to further testing. More important for us, this is the most common shape of A I in a decision. The machine drafts and a person decides. Insurance claims, credit, complaints and staff cases will all look like this. So whether the human step works is everyone's question, not a planning one.
Leonard Before we go further, what did the pilots actually show? I have seen some big numbers quoted, and I would like to know what they are numbers of.
Linda Good, because that is step one: read the numbers carefully. There are three, and they measure different things. DeepMind says average officer time on assessment tasks fell by fifty eight per cent. It says officers approved more than seventy per cent of the drafted reports with no edits. And the government says routine householder cases have gone from eight weeks to four. The first is time on a task. The second is agreement. The third is how long the family waits. Picture a barista... who gets twice as quick while the queue stays long, because the till is slow. No method has been published for these figures, and no data for the eight weeks to four...
Leonard Then let me ask the obvious newcomer question. Seventy per cent approved with no edits sounds like very good news. If trained officers changed nothing, isn't that proof the drafts are good?
Linda It might be, and that is step two. An edit rate measures agreement. It does not measure whether the draft was right. Seventy per cent untouched could mean the drafts are excellent. It could also mean a busy officer has stopped reading closely, because the last fifty were fine. Think of proofreading a colleague whose work is nearly always clean. By the twentieth page you are skimming. Nobody has published how many untouched reports were checked by someone else and found correct. So from the outside, the two stories look the same.
Leonard We have all skimmed that page.
Linda We have.
Leonard That sounds a bit unfair on the officers, though. These are trained professionals who know the planning rules well. Surely they would notice when a draft in front of them is wrong?
Linda That is step three, and it is not a slur on anyone. Researchers call it automation bias. A review by Raja Parasuraman and Dietrich Manzey, in the journal Human Factors in twenty ten, pulled the studies together. When a decision aid is imperfect, people miss problems the aid missed. They also follow its advice when it is wrong. It happens to experts as well as beginners, and to teams. The review found it cannot be prevented by training or instructions. You have probably followed a sat nav past a turn you knew was better. Same pull.
Leonard Guilty.
Leonard So if telling people to be careful does not work, what would an organisation actually do? How would you tell a good draft from a rubber stamp in practice?
Linda Step four. You test the reviewer, not just the tool. Plant a few drafts with known errors and see whether they are caught. Send a sample of approved cases to a second reviewer who has not seen the A I's view. And watch the no edit rate over time. A rate that keeps climbing while the team gets busier is a warning, not a win. Our own rule of thumb is that once more than about two thirds of drafts go through untouched, it is time to run those tests. That is our heuristic, not research...
Leonard Here is the misunderstanding I came in with. I assumed that because a person signs every decision, the rules on automated decision making do not really apply. The machine is just helping, surely?
Linda That is the common one, and in Australia it is risky. From the tenth of December twenty twenty six, a privacy policy has to cover certain computer programs. They are programs that make a decision, or do something substantially and directly related to making it. The decision has to be one that could significantly affect a person, using their personal information. That wording reaches a drafting tool even when a person signs. So... a human in the loop is a design you have to evidence, not an exemption. Check your own case with your lawyers.
Leonard That is not what I assumed.
Leonard Let me say the to do list back, then. First, for every A I that drafts a decision, keep three numbers separately: time on the task, time for the customer, and the error rate found by independent review. Second, treat a high no edit rate as a reason to test the reviewer. Third, run the tool alongside the old process first, as England is doing, and say in advance what result would stop the rollout. Fourth, record it as a system that influences decisions about people, signature or not.
Linda That is it exactly. In Part 2, we argue about what the numbers really tell us. If you'd like to know more, sign up for our podcast at agile-insights.com.au.
Leonard Thanks for listening.
Linda Here is the idea again. When A I drafts a decision and a person signs it off, the safeguard is only as real as your evidence that the person is still checking. Today we argue about one case. England's PlanAssist drafts planning assessments, and officers approved more than seventy per cent of its reports with no edits. Is that a model to copy, or a number to worry about? By the end you will know where the evidence settles, where it does not, and what to ask of your own tools.
Leonard A quick recap for anyone who skipped Part 1. PlanAssist is built by Google DeepMind with the U K government, Google Cloud and Faculty, under an eight point two million pound government contract. It checks a householder planning application against documents, heritage limits and policy, and drafts a cited report. A qualified officer approves every decision. In pilots at three councils, DeepMind reports officer time on assessment tasks down fifty eight per cent and more than seventy per cent of reports approved unedited. Automation bias is the tendency to accept an aid's advice even when it is wrong. It affects experts too, and training does not prevent it.
Linda So the question for today is simple to say. If you were advising a board on a tool like this, would you hold it up as the pattern to follow? Leonard, I think you want to make the case for it first, and I think it deserves a fair hearing before I start poking at the numbers.
Leonard Happy to go first.
Leonard I do, because a lot here is done well. Householder cases are routine and rule bound, which is where drafting help is safest. The report cites its sources, so an officer can check a claim rather than trust it. The officer keeps the decision. Then look at the rollout. It went from three councils to fifteen, for twelve months, on live applications, with officers using it alongside their normal process. The Budget says every council by the end of twenty twenty seven, but subject to further testing. And the government is not mandating the tool. It is mandating open publication of planning documents, so the data improves first. That is staged, reversible, and data led. Most boards would love to say that about their first A I rollout.
Linda A strong opening.
Linda I agree with nearly all of that, and the parallel run is the part I would copy tomorrow. My worry is narrower. It is how the success is being described. There are three numbers, and they are doing different jobs. Fifty eight per cent is officer time on tasks. Seventy per cent is agreement with the draft. And the government's release says the tool is helping halve routine processing from eight weeks to four. That last one is what a family feels, and it is the one with no published data. Even the baseline... wobbles. The June release says householder cases are nearly seventy per cent of applications. The October notes call them seventy per cent of officers' workload. Those are different measures...
Leonard Fair, but let me push back on the seventy per cent no edit figure, because I think you are about to treat it as a red flag. For simple cases, a good draft should often need no changes. If a junior planner wrote a clean report and the senior signed it unchanged, we would call that competence, not complacency. A tool that needed heavy edits on seventy per cent of routine cases would be the real failure. So a high number is what success looks like on easy work. Why should a governance lead treat it as suspicious?
Linda Not suspicious. Unexplained. You are right that a good tool on easy work should produce a high rate. The trouble is that a tired reviewer produces the same rate. The number cannot tell those two stories apart, and the research says the second one is not rare. With an imperfect aid, people miss what it missed and follow it when it is wrong, experts included. So I would not call seventy per cent bad. I would call it the moment to start testing the reviewer. Our own rule of thumb is about two thirds. That is our heuristic, not a finding.
Leonard That is a fair distinction.
Leonard Then I think we have landed in the same place. The rate is fine as a speed signal and useless as a quality signal on its own. What turns it into evidence is a second measure that does not depend on the reviewer: errors found by someone independent. So, concretely, what would you want the fifteen council trial to publish at the end...
Linda Four things. First, an error rate from a sample of approved cases, reviewed by an officer who did not see the A I's draft. Second, the result of planted errors: drafts seeded with a known mistake, and how many officers caught them. Third, elapsed time from application to decision, against a baseline, not task time. Fourth, how the no edit rate moved over the year. If it climbs steadily while caseloads rise, that tells you something different from a rate that holds steady while seeded errors keep getting caught. None of that is exotic. It is the same evidence you would ask of any process that leans on a sign off.
Leonard Let me bring this home to Australia, because there is a legal edge here. From the tenth of December twenty twenty six, Australian privacy policies have to cover certain computer programs. They are programs that make decisions, or do something substantially and directly related to making them. The decision has to be one that could significantly affect a person, using their personal information. A tool that drafts the assessment a person then signs fits that wording. So the comfortable story, that a human signs and therefore it is not automated decision making, does not hold. That changes how you classify these tools on day one, not just how you measure them later.
Linda That surprises a lot of people.
Linda So where does the evidence settle? What the tool does, where it is being trialled, and that a person decides every case, all come from the government's own releases. The fifty eight and the seventy are the lab's and the government's figures, with no method published. The eight weeks to four is a claim without data so far. No independent evaluation has been published, and the twelve month trial is where that should come from. What we do not know... is the error rate. That is the number everything else depends on.
Leonard And here is what to do with all that. For any A I that drafts a decision a person signs, report three separate numbers: task time, customer time, and the error rate found by independent checking. Never let one stand in for another. When the no edit rate passes about two thirds, test the reviewer with planted errors and blind second reviews. Run the tool alongside the old process first, and write down in advance what result stops the rollout. And register it as a system that influences decisions about people, signature or not, because in Australia the privacy policy will need to say so.
Linda If you'd like to know more, sign up for our podcast at agile-insights.com.au.
Leonard Thanks for listening.
Every episode tagged #oversight
All of the briefings (38) · Follow Briefing in Apple Podcasts or any podcast app
Everything from Agile Insights: briefings, Making You Think and the reference series