Agile InsightsBriefing
A black and white workbench with pliers and a screwdriver on a plan, a box of index cards and a clipboard checklist, labelled in yellow Embedded, Skills and Evals, beside a studio microphone.

Episode 23 · Primer and discussion · About 14 minutes in two parts

What FDE means at OpenAI: skills, not custom code.

When an AI lab's engineers build your AI well, what you keep is a shared library of skills and the evals that prove them. Four terms, the case for taking the help, and what to own before the engineers arrive.

#agents#leadership

Part 1: Primer · About 7 minutes

Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.

Part 2: Discussion · About 7 minutes

Download the MP3 · Linda and Leonard, AI voices, read a script written from the sources below.

In this episode

01Forward deployed engineers explainedA forward deployed engineer is the vendor's engineer working inside your team. OpenAI says its team exists to close the gap between what models can do and what people reliably use.
02Custom code becomes shared skillsOpenAI says custom tools for one chip maker were rebuilt as skills and plugins on one harness, removing roughly ninety per cent of the custom code. It does not say how that was measured.
03Evals decide what goes liveEvals built with the customer's own experts decide what reaches production. In one unnamed telecoms case, OpenAI says it told the customer the model scored about sixty per cent, not yet good enough.
04Take the help, own the libraryEvery figure in the post is OpenAI's own, and the sources do not say what a customer keeps if it leaves the harness. Decide who owns skills, evals and traces before the engineers arrive.
05Register skills and rerun evalsPut shared skills in your AI register with an owner and a version, rerun evals before any change, and keep a person approving any change an agent proposes.
Sources and links (6)
What FDE means at OpenAI · openai.com
OpenAI engineering post by Colin Jarvis, 22 October 2026. The account this episode is about.
Building self-improving tax agents with Codex · openai.com
OpenAI and Thrive Holdings, 27 May 2026. The correction to eval to fix loop, with engineers reviewing changes.
Introducing OpenAI Presence · openai.com
OpenAI product launch, 22 July 2026. Deployed only by OpenAI's engineers and selected integrators.
Build skills · learn.chatgpt.com
OpenAI documentation defining skills and plugins and their enterprise controls.
What are Forward Deployed Engineers, and why are they so in demand? · newsletter.pragmaticengineer.com
Gergely Orosz, The Pragmatic Engineer, 12 August 2025. Background on the role.
Anthropic partners with Blackstone, Hellman and Friedman and Goldman Sachs · blackstone.com
Blackstone press release, 4 May 2026, on the new enterprise AI services firm.

Written from OpenAI's own posts and documentation, checked on 23 October 2026. Most customer results come from OpenAI and name no customer, so treat them as claims, not independent evidence. Check the originals before quoting.

Our fact check · 12 claims: 3 confirmed, 4 reported, 2 partly right, 1 disputed, 2 unverified

Before we record, every claim the hosts make is checked against the sources (this table was last updated 23 October 2026). Confirmed means a primary source or two independent reports; Reported means one party says so; Partly right, Disputed, Unverified and Not yet known mean what they say, and the hosts say so on air. This is the table as we left it.

  1. ConfirmedJarvis founded the FDE team in early 2025; role originated at PalantirOpenAI post; The Pragmatic Engineer
  2. ReportedCustom apps rebuilt as skills and plugins on a common Codex harnessOpenAI’s account; skills and plugins are documented product objects
  3. Unverified~90% of custom code removedNo measure, unnamed customer
  4. ReportedTwo months for the first team, one day for the secondSingle data point; One unnamed example
  5. Partly rightSkills “centrally controlled by the AI CoE”Docs name enterprise skill and plugin controls but do not describe them
  6. DisputedVerification is ~70% of chip project effortSurvey mean 53–57%
  7. Partly rightSee & Spray cuts chemicals “by up to 70%”Deere averages 59% (2024), ~50% (2025)
  8. ReportedTax AI: 7,000 returns, ~50% more throughput; corrections → evals → Codex PR → engineer reviewTwo OpenAI posts by the builders; OpenAI reportedly holds equity in Thrive Holdings (unchecked)
  9. UnverifiedTelecoms evals ~60% → 90%+ over six monthsUnnamed customer, unpublished evals; the behaviour is the point
  10. ConfirmedPresence deployed only by OpenAI FDEs and select SIs; limited GA from 22 Jul 2026Product terms. Design partners include IAG (exploring severe-weather surge support)
  11. ReportedPresence: Codex proposes changes; teams test against production and approve a controlled rolloutOpenAI’s design claim
  12. ConfirmedOpenAI Deployment Company and Tomoro deal (May 2026); Anthropic services firm (4 May 2026)Several reports; Blackstone release. Tomoro closing not yet known

Want to know more?

Sign up for our podcast at agile-insights.com.au.

Read the transcript · Part 1: Primer

Linda G'day, and welcome to the Agile Insights briefing. I'm Linda. Here is the main idea. When an AI lab's engineers build your AI well, what you keep is not a custom app. It is a shared library of skills, and the tests that prove them. That library is now the thing to govern. By the end of this part you will know four terms, and why one of them changes your controls. One disclosure. This script was drafted with Claude, Anthropic's model, and checked by a person.

Leonard I'm Leonard, and I'm the newcomer today. I keep hearing that the AI labs now send their own engineers into big companies. On the twenty second of October, OpenAI published a post on what forward deployed engineering means there. So start me at the beginning. What is a forward deployed engineer, and how is that different from the consultants we already hire?

Linda A forward deployed engineer is the vendor's engineer working inside the customer's team. They write code on the customer's own systems. The role began at Palantir in the early twenty tens. It differs from a solutions architect, who advises and builds demos. Colin Jarvis started OpenAI's team in early twenty twenty five, and he still runs it. He says its job is closing the deployment gap. That is the distance between what models appear able to do, and what people can reliably use at work.

Leonard So, the opposite of a slide deck.

Linda Here is his main example. In mid twenty twenty five, about fifteen OpenAI engineers worked with a large chip maker. OpenAI does not name it. Some had never worked on chips. They sat with the customer's verification engineers, the people who check a chip design before it is built. They built a separate tool for each of more than ten uses. Two were writing tests and finding the cause of a failure. OpenAI says every one reached production within three to six months...

Leonard So far that sounds like a very good consulting job. Where do the other terms come in? You promised me four of them, and so far I've only got one.

Linda Fair enough.

Linda The second term is the harness. A harness is the common frame an AI agent runs in. It handles the model, the tools and the screen, so each use does not rebuild them. Here the harness was Codex, OpenAI's coding agent. The third term is a skill. A skill is a package of instructions, scripts and reference files that the agent loads when a task matches. A plugin bundles skills with connections to other systems, so another team can install the lot. OpenAI's own documentation describes both.

Linda Now watch what happened to the chip work. Once Codex improved, the team rebuilt each custom tool as skills and plugins on that one harness. OpenAI says this removed roughly ninety per cent of the custom code from each project. It does not say how that was measured. It also says the customer's AI centre of excellence could control the skills centrally. In one example from August, the first team took two months to get going. The second team took one day. That is one example, and the post does not say what the day included.

Leonard That is quite a drop.

Leonard Which brings me to the fourth term, I assume. You keep saying the tests that prove them. In my world, a test is something a student sits. So what is an eval, and who writes it?

Linda An eval is a set of test cases that says whether the AI did the job. The good ones are written with the people who do the work. In the chip case, OpenAI's engineers built evals with the customer's own verification staff. OpenAI built a tax product with Thrive Holdings. Its eval counts how many fields on a return were right without correction. The builders say that at launch, a quarter of returns had three quarters of their fields right. Six weeks later, eighty six per cent did.

Leonard An exam with a proper answer key.

Linda Evals also tell you when to say not yet. OpenAI describes a telecoms customer, again unnamed, that wanted AI on its phone lines. At scoping, the model scored about sixty per cent on the key evals. OpenAI says it told the customer the model was not good enough, and brought in its researchers. Over six months of new model versions, the score rose to eighty, then above ninety. The numbers are OpenAI's own. The behaviour is the lesson: a stated gap, measured on the customer's own tests...

Leonard Then here is the misunderstanding I'd have walked in with. Ninety per cent less custom code sounds like ninety per cent less to govern. I'm guessing that's wrong?

Linda It is wrong, and it is the point of this part. Less custom code means less to maintain. It does not mean less to control. A skill is shared. When one team changes it, every team that uses it gets the change, and any mistake in it. So the thing to control moves from the app to the skill library. Each skill needs an owner and a version number. Its evals need to run again before a change reaches everyone. A register that lists only use cases will miss all of that.

Leonard Right. That makes sense.

Leonard Let me say it back, then. A forward deployed engineer is the vendor's engineer working inside your team. The harness is the common frame the agent runs in. Skills and plugins are packaged know how that teams share. Evals are the tests, written with your own experts, that decide what goes live. And what you really own at the end is the skill library, and its evals.

Linda That's it. So, what to do. First, find the shared skills, plugins and connectors your AI work already depends on. Put them in your AI register beside the use cases. Second, give each one an owner and a version, and rerun its evals before any change goes out. Third, ask for evals built with your own experts before anything is approved for production. In Part 2, Leonard and I argue the bigger question. If the lab's engineers build it, what do you own, and who governs it?

Leonard See you in Part 2.

Read the transcript · Part 2: Discussion

Linda Welcome back to the Agile Insights briefing. I'm Linda. Here is the main idea of this part. If an AI lab's engineers build your AI, take the help. But own the evals, the skill library and the change process yourself. Write that into the deal before they arrive. A quick recap for anyone starting here. Forward deployed engineers are a vendor's engineers working inside your team. Skills are packaged instructions an AI agent loads, and teams share. Evals are the tests, built with your experts, that decide what goes live.

Leonard And the question we're arguing is easy to say. If the lab's engineers build it, what do you end up owning, and who is in charge of it? I'll take the friendly side first, and Linda takes the careful side. But some context first, because this is a trend, not one blog post, and Linda has the facts.

Linda OpenAI Presence, its product for voice and chat agents, launched in July. OpenAI says it is not self serve. Its own engineers, or selected global integrators, lead every deployment. The insurer I A G is one design partner, exploring support during severe weather. In May, OpenAI launched a separate deployment company, with more than four billion US dollars committed. It also agreed to buy a firm of about a hundred and fifty deployment specialists. The same month, Anthropic announced an enterprise AI services firm with Blackstone, Hellman and Friedman, and Goldman Sachs. Anthropic, remember, makes the model that helped draft this script.

Leonard Busy year for the labs.

Leonard So here's the case for taking the help. The labs know their models better than anyone, and they can change them. The telecoms example from Part 1 shows it. The model scored about sixty per cent on the customer's evals, and OpenAI said so. Then its researchers worked on the gap until the score passed ninety, by OpenAI's account. It says that voice model now serves other customers too. No outside consultant can do that. And look at the tax product's loop. Practitioner corrections become evals. Codex drafts a narrow fix, and engineers review it before it ships. Unclear cases go back to people. That's a control loop, with the human gates written down.

Linda I will give you that.

Leonard There's one more thing in their favour. Jarvis says success is making his team unnecessary for a class of problem. If they make themselves obsolete every couple of months, he writes, the team is doing its job. That's an unusual promise from a services team. And being honest about limits is good governance in itself. A pilot that quietly fails costs far more than a vendor who tells you not yet.

Linda I agree with most of that, and I still want to be careful, for three reasons. First, every number in the post is OpenAI's own, mostly for customers it does not name. Ninety per cent less code, two months to one day, sixty to ninety. None of them can be checked from outside. And one industry figure in the post doesn't match the industry's own survey. It says verification is about seventy per cent of a chip project's effort. The long running survey of that industry puts the average at a bit over half.

Leonard Huh. Noted.

Linda Second, the result lives on one vendor's harness. The skills and plugins in the chip story run on Codex. None of the sources say what the customer keeps if it moves to another platform. Third, the post says the customer's centre of excellence could control the skills centrally. OpenAI's documentation names enterprise controls for skills and plugins, but does not describe what they do. So who approves a change to a shared skill? How is it tested, and how is it rolled back? The sources are silent on all three...

Leonard I'll push back on your second point, though. Lock in isn't new. Every platform choice carries some, from your cloud to your finance system. And a skill is a package of instructions, scripts and reference files. That sounds easier to move than a custom app, not harder.

Linda That's fair, and it's where we meet. The files may move. What may not move is the harness behaviour they rely on, and the traces and evals that prove they work. So the answer is not to refuse the help. It's to decide up front who owns the skills, the evals and the production traces. Then put it in the contract. Treat the vendor's harness like any important supplier, with an exit plan you have actually tested.

Leonard I can live with that.

Linda Here is where the evidence settles, and where it doesn't. Confirmed: the role is real, the labs are building services arms, and Presence is sold only with their engineers. Reported by OpenAI and its partner: the tax loop, with engineers reviewing every proposed change. Unverified: the customer results, because the customers are unnamed. Not known: what central control of a skill means in practice. Nor what you keep if you leave, or what the engineers' time costs. Those are the questions to ask before you sign.

Leonard So let's finish with what to do. First, put shared skills, plugins and connectors in your AI register, beside the use cases they serve. Give each one an owner and a version. Second, rerun the evals before any change to a shared skill goes out, and again on each new model release. Third, ask for evals built with your own experts, and a stated capability gap, before anything is promoted to production. If a vendor never tells you the model scores sixty per cent, ask why.

Linda Fourth, when an agent proposes changes from production data, keep a person approving the rollout. Ask to see that gate working, not just described. Fifth, before the engineers arrive, write down who owns the skills, the evals and the traces at the end. Ask whether they would run on another platform. And read every vendor number as a claim until you can reproduce it. If you'd like to know more, sign up for our podcast at agile-insights.com.au.

Leonard Thanks for listening.

More on #agents

Episode 38 · Primer and discussionBudgeted for seats, billed for workEpisode 37 · PrimerWhat is an AI skill (and why you should care)Episode 36 · Primer and discussionChecking agents in production, even when you can’t read their workEpisode 33 · Primer and discussionAgents in production: the platform runs it, you still own itEpisode 29 · Primer and discussionCrowdStrike Uncovers New Agentic AI CampaignsEpisode 26 · Primer and discussionThe agent that speaks first: what to decide before proactive agents reach work accounts.

Every episode tagged #agents

All of the briefings (38) · Follow Briefing in Apple Podcasts or any podcast app

Everything from Agile Insights: briefings, Making You Think and the reference series