Agile InsightsBriefing
Black and white photograph of a long dark corridor ending at tall double doors, with a yellow line and a chip reading An exit door. Title: What if we got AI right?

Making You Think 8 · Primer and discussion · About 15 minutes

What If We Got AI Right?, by Eleanor Drage

AI doom and AI hype are two halves of the same sales pitch.

#people

Part 1: Primer · About 6 minutes

Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.

Part 2: Discussion · About 8 minutes

Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.

Cover of What If We Got AI Right?

This episode is about What If We Got AI Right?, by Eleanor Drage. Find a copy on Amazon Australia, or ask your local bookshop or library.

Based on essays and our own research.

In this episode

01Doom and hype, one pitchDrage argues AI is sold at once as catastrophe and saviour; one reviewer calls it a panic economy that feeds the biggest firms and hides present harm.
02AI is human and materialAI is made by people in mines, chip factories and data-labelling centres; a "hallucination" is a system error or labelling mistake, not a mind.
03The Triangle of DoomHer name for utilitarianism, effective altruism and existential-risk thinking fused together, so that a speculative future can outweigh real people harmed today.
04Power, not only biasShe urges "shifting power, rather than removing bias"; critics say she is strong on why but weak on how, enforcement and the benefits she leaves out.
05Test the exit doorFor any AI tool, check that a person can get out or reach a human, who can switch it off, and when that was last actually tested.

Your turn

Three questions to ask yourself

Think of the last AI story that made you anxious, or excited: who was telling it, and what did they stand to gain if you believed it?
Which AI benefit would you add to her book, and how would you check that it is real?
If you wanted out of the AI tool you use most at work, where exactly is the door?

Two things to try, alone, with a pen

5–10 minutesSort the storySort one AI headline into doom, hype, harm now and who gains.
5–10 minutesFind the exit doorCheck whether one AI tool has a real way out.
Open the activity sheets (print or save as PDF)

About Eleanor Drage

Senior Research Fellow at the Leverhulme Centre for the Future of Intelligence, University of Cambridge. Her doctorate, from Bologna and Granada, was on science fiction. Co-hosts The Good Robot podcast with Kerry McInerney.

Other writing (1)
What If We Got AI Right? (2026) · Profile Books
the case against AI doom and hype, and for ordinary ethical work
Sources and links (11)
Based on essays and our own research · The book on Amazon Australia
Where this episode comes from.
Profile Books, publisher page · profilebooks.com
Sophie McBain, The Guardian, 27 Jul 2026 (syndicated copy) · aol.co.uk
mixed review: "intellectual heft", but rushed in places
Valerie O'Riordan, Bookmunch, 22 Jul 2026 · bookmunch.wordpress.com
positive review; AI "yoked to capitalism"
Kato Mivule, "Beyond the Triangle of Doom", 20 Sep 2026 · mivuletech.wordpress.com
chapter-by-chapter review; the exit door; the enforcement gap
Jessica Burrell, Evening Standard interview, 15 Jul 2026 (syndicated copy) · aol.co.uk
The Apocalypse Fetish; "why are we using this?"
The Bookseller, IPG Autumn Conference keynote, 10 Sep 2026 · thebookseller.com
"shifting power, rather than removing bias"
Intelligence Squared, "Should We Stop Catastrophising About AI?" with Carl Miller, 25 Jul 2026 · feeds.megaphone.fm
the long-form interview
Eleanor Drage, CV · eleanordrage.com
2020 exam grading controversy · en.wikipedia.org
background on the A-level algorithm
Gender Shades (Buolamwini and Gebru, 2018) · proceedings.mlr.press
the face-analysis study used as our own illustration

Based on essays and our own research, checked on 21 October 2026. The author's argument is paraphrased, not reproduced. Check the original before quoting.

Our fact check · 38 claims: 28 confirmed, 9 reported, 1 unverified

Before we record, every claim the hosts make is checked against the sources (this table was last updated 21 October 2026). Confirmed means a primary source or two independent reports; Reported means one party says so; Partly right, Disputed, Unverified and Not yet known mean what they say, and the hosts say so on air. This is the table as we left it.

  1. ConfirmedProfile Books, 16 Jul 2026, paperback £14.99, ISBN 9781805225447Profile page; Storizen says 23 Jul; Hachette audio 23 Jul
  2. Reported288 ppRetailer listing
  3. ConfirmedSenior Research Fellow, Leverhulme CFI, CambridgeProfile; Evening Standard; Intelligence Squared; Guardian body text says "assistant research professor"
  4. ConfirmedPhD Bologna and Granada; on science fiction; book on women's SFHer CV; Bookmunch; Routledge title; Exact thesis title not checked
  5. ConfirmedCo-hosts The Good Robot with Kerry McInerneyHer CV; Bookmunch
  6. ConfirmedFeminist AI, OUP 2023, co-editedHer CV
  7. ConfirmedWanted to call it The Apocalypse Fetish; publisher refusedEvening Standard, 15 Jul 2026
  8. Reported"Self-reinforcing panic economy"Mivule; Said on air as "one reviewer calls"
  9. ReportedShe examines the March 2023 pause letterMivule
  10. ConfirmedPause letter, March 2023, six months, most powerful systemsFuture of Life Institute letter (public record)
  11. ConfirmedMusk building AI while calling for pausesEvening Standard; Her point in interview
  12. ConfirmedBigger risks are systems, incentives, power structuresEvening Standard
  13. ConfirmedAI the product of labour in mines, factories, data-labelling centresGuardian (McBain); McBain's summary
  14. Confirmed"The cloud" is someone else's computer; hallucination a system error or labelling mistakeGuardian
  15. ReportedTriangle of Doom = utilitarianism + EA + x-risk, among tech entrepreneursMivule; Single reviewer; "one reviewer describes"
  16. Confirmed"AI yoked to capitalism is the problem"Bookmunch; Paraphrased on air (one-quote rule)
  17. ReportedBostrom's "crunch" theory conceals eugenicist assumptionsMivule, as Drage's argument; Left out (single source; cannot state fairly in one line)
  18. Confirmed"Shifting power, rather than removing 'bias'"The Bookseller, IPG keynote 10 Sep 2026
  19. ConfirmedGender Shades (2018): commercial face analysis worst on darker-skinned women; firms improvedBuolamwini and Gebru 2018; Raji and Buolamwini 2019; Hosts' illustration, said so on air; not verified as in the book
  20. UnverifiedGender Shades fix improved police surveillance as Drage's exampleBook note only; Not attributed to her on air
  21. Confirmed"The process of working together to develop a fairer world"Guardian
  22. ConfirmedEU AI Act toolkit she co-designed, "cornerstone of utopianism"Guardian; her CV
  23. ConfirmedPrescriptions: purpose-specific AI, consult stakeholders, real ethics policiesBookmunch
  24. ReportedConsent beyond tick boxes; whistleblower support; ethics trainingMivule
  25. Confirmed"Why are we using this?"Evening Standard interview; Attributed to the interview on air
  26. ConfirmedMumkin (FGM conversations, India); Kuini (Māori women quitting smoking)Guardian
  27. ConfirmedPriya Goswami: documentary filmmaker, co-founder of MumkinSahiyo; The Wire; Humboldt Foundation
  28. ReportedExit door is Goswami's idea, used in the A-level chapterMivule; "One reviewer reports"
  29. Confirmed2020 A-level algorithm used schools' past results; ~39% lower than teacher grades; state schools hit harder; reversed 17 AugWikipedia (36% + 3%); Evesham Observer; Ofqual figures via press
  30. ReportedDrage: domain knowledge and lived context exposed the flawMivule
  31. ConfirmedGuardian: "intellectual heft"; critique of big tech largely convincingGuardian
  32. ConfirmedGuardian: rushed, sloppy; projects underwhelming; omits drug discovery, disease detection; ends unsureGuardian
  33. ConfirmedBookmunch: reframes AI as a choice; maybe too optimistic; light on adoption pressureBookmunch
  34. ConfirmedMivule: no resourcing or enforcementMivule; Of the review's text
  35. ConfirmedDavies' fundamental law of accountabilityMYT 6 pack
  36. ReportedTurkle calls the moment "a state of emergency"Turkle-artificial-intimacy; Check against MYT 9 pack
  37. Confirmed"Ask your boss which tools are mandatory and the productivity evidence"Book note; ES partial; Partly confirmed; ES: little evidence AI improves sustainable productivity; ask "why are we using this?" Only the latter used
  38. ConfirmedIntelligence Squared, 25 Jul 2026, with Carl Miller, first of two partsRSS feed; Audio not heard

Want to know more?

Sign up for our podcast at agile-insights.com.au.

Read the transcript · Part 1: Primer

Linda Here is the idea for today. A I doom and A I hype are two halves of the same sales pitch. Both pull our attention away from the harm happening now, and from the question of who holds the power. That is the argument of What If We Got A I Right?, by Eleanor Drage. By the end of this part you will know what she means by a panic economy and a Triangle of Doom. And you will know why she thinks getting A I right is ordinary work, not a destination. This is Making You Think, episode eight.

Leonard And the author comes at A I from an unusual direction. Eleanor Drage is a Senior Research Fellow at the Leverhulme Centre for the Future of Intelligence, at the University of Cambridge. Her doctorate, from the universities of Bologna and Granada, was on science fiction, and she has written a book on women's science fiction. With Kerry McInerney she co-hosts The Good Robot podcast, and they co-edited Feminist A I, from Oxford University Press in 2023. This book came out from Profile Books in July 2026. In an interview with the Evening Standard, she said she had wanted to call it The Apocalypse Fetish... Her publisher said no.

Linda Fair enough, really. That rejected title is a good way into the first step. Drage describes what one reviewer calls a self reinforcing panic economy. A I is sold at the same time as a catastrophe and as a saviour, and both stories draw attention and money to the biggest firms. Her example is the open letter of March 2023, which called for a six month pause on the most powerful A I systems. In the Evening Standard she pointed to the contradiction of tech leaders like Elon Musk building A I while calling for pauses. If the danger were really that great, you might expect them to stop building.

Leonard So is she saying the risks are made up, that nobody should worry at all?

Linda No. She told the Evening Standard that the bigger risks are the systems, incentives and power structures shaping A I today. That leads to her second step. A I is human and material. As the Guardian's reviewer, Sophie McBain, sums it up, A I is the product of human labour in mines, chip factories and data labelling centres. The language hides this. The cloud is just someone else's computer. And when a chatbot hallucinates, Drage's point is that this is a system error or a labelling mistake... A word borrowed from the human mind makes a software fault sound like a personality.

Leonard I like that, because it changes who you would complain to. You cannot hold a hallucination to account, but you can ask who shipped the error. So where does the Triangle of Doom come in?

Linda That is the third step, and it is her coinage. One reviewer describes the Triangle of Doom as three ideas fused together among tech entrepreneurs. Utilitarianism, the ethics of adding up the greatest good. Effective altruism, the movement that tries to give in the most efficient way. And existential risk thinking, which ranks the survival of humanity above everything else...

Leonard Which sounds noble, on paper.

Linda Put the three together, she argues, and a speculative future can outweigh real people harmed today. Her conclusion, in the Bookmunch reviewer's account, is that A I itself is not the enemy. The problem, she argues, is A I harnessed to capitalism.

Leonard Then surely the fix is the familiar one. Find the bias in a system and take it out. Is that not what A I ethics already does?

Linda That is the fourth step, and the one she pushes hardest. At a publishing conference in September 2026, she urged people to talk about "shifting power, rather than removing bias". Here is an illustration of our own, not hers. In 2018 a study called Gender Shades found that commercial face analysis tools made the most errors on darker skinned women. The firms improved their systems. That was a real gain. But a fairer face recogniser can still be pointed at a crowd of protesters... Fixing the bias does not answer who is watching whom, and why. That second question is about power.

Leonard Hard to argue with that. So if the problem is power, what does getting it right actually look like? Her title promises an answer.

Linda Her answer is that utopia is a practice, not a place. The Guardian quotes her describing it as "the process of working together to develop a fairer world". It is ordinary, incremental work. She calls a toolkit she helped design, to help companies meet the European Union's A I rules, a cornerstone of utopianism. Reviewers report her prescriptions. Build A I for a specific purpose, not a general chatbot by default. Consult the people affected. Make consent real, not a tick box. Protect whistleblowers. Give people a clear exit door from systems they do not trust. And ask at work, as she put it in an interview, "why are we using this?"

Leonard Before Part 2, a question for you. Think of the last A I story that made you anxious, or excited. Who was telling it to you, and what did they stand to gain if you believed it?

Linda In Part 2: where the argument is strong, where the critics push back, and what an exit door looks like in practice. See you there.

Read the transcript · Part 2: Discussion

Linda The idea again, in one sentence. A I doom and A I hype are two halves of the same sales pitch. Both pull attention away from the harm happening now, and from who holds the power. Eleanor Drage's answer is that getting A I right is ordinary, incremental work, not a destination. Today, where her argument is strong, where the critics push back, and what it means for anyone who has to govern A I inside an organisation. We will finish with two short activities you can do with a pen and paper.

Leonard A quick recap of the terms from Part 1. The panic economy is A I sold as catastrophe and saviour at once, with both stories feeding the biggest firms. The Triangle of Doom is her name for utilitarianism, effective altruism and existential risk thinking fused together, so that a speculative future outweighs present harm. Ethics as power means asking who a system serves, not only whether it is biased. Utopia as a practice means the everyday process of working together toward a fairer world. And an exit door is a clear way to get out of a system you do not trust.

Linda Let me start with the strength. Drage puts the people back into the story. Once you see A I as mines, chip factories, data labellers and executives making choices, the question changes. It stops being what the machine will do to us. It becomes what these people are doing... and who can stop them. Even the book's mixed reviewer at the Guardian credits the book with "intellectual heft", and found her critique of big tech's power largely convincing. The Bookmunch reviewer, Valerie O'Riordan, says it reframes A I as a choice we can shape, rather than a fate. For anyone who feels helpless about A I, that is a real gift.

Leonard That is the part that stayed with me. The second strength is her case about power, and the clearest example is one she uses herself. In August 2020, with exams cancelled, an algorithm in England adjusted A level grades to fit each school's past results. Around thirty nine percent of grades came out lower than teachers had predicted. State school students were marked down more than those at private schools. Within days the government reversed it...

Linda That fast.

Leonard One reviewer reports Drage's point about that case. It was domain knowledge and lived experience that exposed the flaw, not data science. A model can be statistically tidy and still be unjust to the people it sorts.

Linda Now the critics, and they are serious. Sophie McBain, in the Guardian in July, found some arguments rushed, and at times sloppy. She thought the community projects, like the Mumkin app for conversations about female genital mutilation in India, and Kuini, a chatbot helping Māori women quit smoking, were underwhelming. What does A I add, she asked. She noted the book leaves out benefits like drug discovery and disease detection. And she ended unsure what getting A I right actually means. O'Riordan, though positive, thought it may be too optimistic, and light on the pressure workers face to adopt A I.

Leonard That last one lands. I want to push back on Drage herself here. Bundling doom and hype into one sales pitch is neat, but perhaps too neat. Plenty of people who study catastrophic A I risk are not selling anything. Some are researchers who think a small chance of a very bad outcome deserves serious work. If you file all of them under the Triangle of Doom, you risk the same mistake she accuses others of. You stop listening to a whole group because of who else is in the room.

Linda That is fair, and I think the honest reading lands in between. Her target is a ranking. She objects when a speculative future is allowed to outweigh people harmed now, especially by those who profit from the story. You can accept that and still fund risk research. An organisation does not get to pick one list. It has a biased hiring tool today and a powerful model on next year's roadmap. Good governance holds both. And she is an advocate, not a neutral surveyor, so it is our job to put back the benefits she leaves out. Here is a question to sit with... Which A I benefit would you add to her book, and how would you check that it is real?

Leonard Which brings us to governing A I, and this part is our extension, not hers. Reviewers keep saying roughly the same thing. She is strong on why, and weak on how. One reviewer, Kato Mivule, says she names the destination without addressing resourcing or enforcement. That gap is exactly what A I governance is for. Take the exit door. One reviewer reports that she takes the idea from Priya Goswami, the documentary filmmaker behind Mumkin. Workers and users get a clear way to reject a system they cannot trust. That is not a slogan...

Linda It is a control.

Leonard It is a control, and controls can be tested.

Linda And it connects straight back to episode six. Dan Davies' fundamental law says you are only accountable for a decision as far as you can change it. An exit door is that law made practical. Can a customer reach a human? Can a team switch the tool off, and who has the authority? How long would it take? Has anyone ever actually done it? A door that has never been opened may be painted on the wall. So, one more question for you. If you wanted out of the A I tool you use most at work, where exactly is the door?

Leonard Two short activities, each about five to ten minutes, with a pen and paper. The first is called Sort the story. Find one A I headline or post from this week. Write it at the top of the page. Then draw four boxes, labelled Doom, Hype, Harm now, and Who gains. Put each claim in the piece into a box. Then, in the last box, write who benefits if you believe it. Then notice which box on the page stays emptiest.

Linda The second is called Find the exit door. Pick one A I tool you or your customers rely on. Write down three things. How a person would get out of it, or reach a human instead. Who has the authority to switch it off. And when that was last tested. Then add Drage's own question at the bottom, why are we using this, and write the honest answer. If you cannot fill in the second or third line, you have found a door that may only be painted on.

Leonard So what to carry away. First, when you hear an A I story, doom or hype, ask who is selling it and what it pulls your attention from. Second, A I is made and run by people, so every harm has an owner you can name. Third, an exit door only counts if someone has opened it. Next time, in episode nine, Sherry Turkle's Artificial Intimacy. She also finds the harm in human choices and business models, but at the opposite temperature. She calls this moment... "a state of emergency".

Linda If you'd like to know more, sign up for our podcast at agile-insights.com.au.

Leonard Thanks for listening.

More on #people

Episode 34 · Primer and discussionWhen the test stops telling you anything: hunger beats tenureEpisode 32 · Primer and discussionAgentic video is real: what to try, and what to checkEpisode 31 · Primer and discussionSpeeding up planning decisions with AI: who checks the checker?Making You Think 12 · Primer and discussionThe Game Changer, by Jason FoxEpisode 13 · Primer and discussionFairness under pressure: a fair AI decision has to hold when someone leans on itMaking You Think 9 · Primer and discussionArtificial Intimacy, by Sherry Turkle

Every episode tagged #people

All of Making You Think (13) · Follow Making You Think in Apple Podcasts or any podcast app

Everything from Agile Insights: briefings, Making You Think and the reference series