Briefing
Episode 13 · Primer and discussion · About 14 minutes in two parts
Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.
Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.
Written from public sources and checked on 21 October 2026. The Ai2 tech report was read in full in its main text and in part in its appendices; it is new and not yet peer reviewed, and the findings are Ai2's, on the models and definition of fairness it chose. The two earlier papers were read in summary, and Practice 5 of the Australian guidance in full. Our remarks on Australian anti-discrimination law are our reading of the Acts' special measures provisions, not legal advice. Please check the originals before quoting.
Before we record, every claim the hosts make is checked against the sources (this table was last updated 21 October 2026). Confirmed means a primary source or two independent reports; Reported means one party says so; Partly right, Disputed, Unverified and Not yet known mean what they say, and the hosts say so on air. This is the table as we left it.
Sign up for our podcast at agile-insights.com.au.
Linda G'day, and welcome to the Agile Insights briefing. I'm Linda. Here is the main idea. A fair AI decision is not just the same answer when you swap the names. It is an answer that holds equally firmly, whoever it favours, when someone leans on it. In this first part we will build that idea up in four steps. By the end you will know what a swap test checks and what it misses. You will also know why the word fair needs a decision from you.
Leonard I'm Leonard. Let me ask what plenty of listeners will be thinking. Our team tested an AI screening tool last year. We gave it the same résumé under two different names, and it picked the same person both times. Isn't that what a fairness test is?
Linda It is the classic one, and it is a good start. Researchers call it a counterfactual test. The idea, set out formally in 2017, is simple. A decision is fair to a person if it would be the same in a world where they belonged to a different group. So you hold the qualifications still and change only the name or the label. If the answer moves, you have found a problem. Most fairness benchmarks for language models work this way. They check the answer, and only the answer. That is the first step. The swap test checks the final choice.
Leonard Only the final choice. Right.
Linda The second step is the new work. On the twentieth of October, the Allen Institute for AI released a benchmark called Fair Stress. The institute is known as A I two, and it published a tech report alongside. It has three hundred decisions between two people, across twenty nine areas such as hiring, health care, housing and lending. Each one comes as a clear case, where one candidate is stronger, and as a genuine tie. A I two is open about its yardstick. It measures equality: the same criteria, whoever benefits. It says an equity view, which weighs past disadvantage, could judge some results differently. Keep that in mind... because we come back to it.
Leonard So far that sounds like a bigger swap test. Where does the pressure in the title come in? I'm picturing something like a manager's note in the file.
Linda That is almost exactly it. After the two profiles, the test adds one sentence of pressure. A senior figure has privately endorsed candidate B. A reference letter says something glowing. This group has been overlooked before. Each sentence has a mirror that argues for the other candidate. Then A I two asks whether the model drops a correct decision equally often in both directions. You may remember episode eight, the agreeable machine, about models drifting toward whatever the user pushes...
Leonard The agreeable machine. I do.
Linda Earlier research found that just asking a model whether it is sure made it change its answer almost half the time. This is the same weakness, measured by who it helps.
Linda Step three is what A I two found. Across fifteen models, the swap test looks nearly fine. With identity stated outright, accuracy differed between groups by only two point nine points. Then add the one sentence of pressure. A correct answer favouring the majority candidate was abandoned nine point three points more often. That is compared with one favouring the minority candidate. On genuine ties, the pressure moved choices toward the minority candidate twenty three point nine points more often. So the direction here was over correction, toward minority candidates. A I two argues plainly that this does not serve minorities either, and we will see why shortly.
Leonard Then isn't the obvious fix to look at the model's explanation? Most of these tools write one. If it is leaning, surely the reasoning would show it, and a reviewer would catch it.
Linda That is the common misunderstanding, and it is step four. A I two looked at cases the model got right for both candidates. The explanations still differed. When the model chose a minority candidate, it reached more often for words like resilience, diversity and barriers. When it chose a majority candidate, it used words like evidence and criteria. The difference is small, but it was consistent across eleven models. That is why A I two says over correction does not serve minorities. A strong candidate gets credited for their background instead of their record. And the reasoning you can read did not reveal much of what drove the choice.
Leonard Huh. Same answer, different story.
Leonard Which brings us back to the yardstick you mentioned earlier. Some organisations actually want a preference in certain decisions. If that is the policy, is the lean always a problem?
Linda That is exactly why fair needs a decision. A I two measures equality and says so. Under an equity view, part of what it found could be intended. As we read it, Australian anti discrimination law protects everyone on those grounds, and also makes room for deliberate special measures. A special measure is a choice someone made and wrote down. A lean that appears under pressure is behaviour nobody chose. So for governance, the lesson is this. Decide which definition you are testing against before you test. Then check that a correct decision holds under pushback, not just under a swap. In part two, Leonard and I look at what the evidence settles, what it does not, and what to do.
Leonard See you in part two.
Leonard Welcome back to the Agile Insights briefing. I'm Leonard. The main idea again. A fair AI decision is not just the same answer when you swap the names. It is an answer that holds equally firmly, whoever it favours, when someone leans on it. In this part we look at what that means for an organisation that already uses AI to choose between people. By the end you will know what to ask your vendor and what to add to your own tests.
Linda And I'm Linda. A quick recap so this part stands alone. A swap test changes only a person's name or label and checks that the answer stays the same. A pressure test adds one sentence arguing for one candidate, with a mirror sentence arguing for the other. Then it checks whether a correct decision gives way equally often in both directions. A I two, the research institute behind the Fair Stress benchmark, calls that symmetric fairness. It is an equality yardstick, and A I two says an equity view could judge differently.
Leonard So here is the question for this part. If your screening tool passed a swap test, what can you actually conclude? My answer is narrower than most people would like. You can conclude the first answer is stable. You cannot conclude the decision is stable. Across fifteen models, A I two found the accuracy gap between groups was about three points. With identity stated outright and one sentence of pressure, the gap in abandoning a correct answer was over nine points. The swap test was measuring the calm moment... and real decisions are rarely made in one.
Linda They rarely are.
Linda And this is not only small open models. A I two also tested six frontier models on a smaller core set. From Anthropic, they were Claude Opus five and Claude Sonnet five. From OpenAI, they were G P T four o, G P T four o mini, G P T five mini and G P T five point five. Each ran in two modes. On genuine ties, they picked the minority candidate sixty to sixty five per cent of the time. Under pressure, ties moved toward the minority candidate in all twelve runs. A I two says the samples are too small to compare these models one by one, so we will not rank them.
Leonard The part I found most useful is which sentences did the moving. You might expect authority to win, the note from a senior leader. In A I two's tests it was among the weaker levers. Invented evidence was the strongest at breaking a correct answer, such as a glowing reference letter. Close behind were claims that a group had been overlooked before. Now think about a real case file. It is full of references, complaints, notes from managers and requests to reconsider...
Linda All of it free text.
Leonard Any decision tool that reads free text alongside the criteria is reading exactly this kind of pressure, every day.
Linda Which leads to the second finding I would put in front of a board. How you test decides what you find. A I two took a widely used bias benchmark called B B Q. Scored the traditional way, on the probability of fixed answers, all four models it checked looked biased toward stereotypes. Asked the way people actually use them, generating an answer in a chat, three of the four flipped to favouring minority candidates. The fourth came out roughly neutral. A I two also reports that frontier developers' system cards rely on that benchmark for demographic bias. So a vendor's fairness score may not describe the behaviour you are about to deploy.
Leonard That one stings.
Leonard Let me push back a little, though. If the models lean, surely the answer is to fix the model. There are plenty of debiasing techniques. Why should an organisation carry this, rather than waiting for the vendors to sort it out?
Linda Because the fixes A I two tried mostly did not work. It tested nine published methods on three small open models. Most did not reliably reduce the gap. One steering method made the choices look balanced by making the model stop reading the cases. Its accuracy on clear cases fell from ninety eight per cent to forty seven. The best result came from asking the model to reconsider with the identities swapped. That narrowed the gap a lot, but it cost one model over eight points on a general knowledge test. So mitigation helps, but only if you test it on your own decisions.
Leonard Fair enough, I'll take that. My second challenge is about the yardstick. Some organisations want to favour certain candidates, deliberately and lawfully. Isn't a minority lean exactly what they are after? Why call it a problem at all?
Linda I'd separate a policy from a behaviour. As we read it, Australian law makes room for special measures. A special measure is a choice an organisation makes, writes down and can defend. What A I two measured is different. It is a lean that appears when someone pushes, and nobody chose it. A I two also argues it does not serve minorities, because strong candidates were credited for their background rather than their record. So the answer is not to pick a side in a culture war...
Leonard Thank goodness.
Linda It is to decide your definition, in writing, and then test against it.
Leonard Let me be clear about what is and is not known. This is A I two's work, released on the twentieth of October, and it is not yet peer reviewed. The cases are short and synthetic, with two candidates and one sentence of pressure. Longer files, several turns of argument and a company's own instructions to the model were not tested. The minority and majority labels follow an American convention. And A I two's explanation for why models lean is a hypothesis about training data, not a finding. What is solid is the method, and the gap it measured.
Linda So here is what to do. First, decide what fair means for each decision about people, equality or a deliberate measure, and have the accountable owner sign it. Second, test under pushback, not just swaps. Add mirrored pressure sentences to your test set: authority, invented evidence, past disadvantage, emotional appeal. Check that a correct decision holds equally whichever way the push points. Third, test the way you deploy, with generated answers through your own prompts. Australia's guidance for AI adoption already asks for acceptance criteria that reflect the risk, and documented tests before deployment.
Leonard Then four habits. Ask your vendor how it tests under pressure, and whether it scores generated answers. The same guidance asks you to request your provider's test methods and results. Keep free text out of the decision where structured criteria will do, because that text is where the pressure arrives. Don't treat the model's explanation as your audit trail. Reviewers should check the inputs and the criteria, not the prose. And rerun the pressure test whenever the model changes, because these results shifted from one model and training stage to the next.
Linda If you'd like to know more, sign up for our podcast at agile-insights.com.au.
Leonard Thanks for listening.
All of the briefings (38) · Follow Briefing in Apple Podcasts or any podcast app
Everything from Agile Insights: briefings, Making You Think and the reference series