She was good company for the first half hour. Open, quick, funny about her flatmates, filling the pauses before I could.
Then I asked something about borrowing and the answers came back in one word. She sat back in the chair. She stopped filling the pauses and left them to me.
This was a research session for a bank, years ago, before government. She had come in to talk about overdrafts and short-term loans, and had told me several times, in several ways, that she needed money at short notice. Written down, that is a clean requirement.
Nothing she said told me what had happened. The change told me.
So I stopped asking about borrowing and told her I had noticed she had gone quiet. She gave me the rest herself. She had been in short-term debt before and talked about it as something to be ashamed of. What she wanted from a bank was to be told that this was ordinary, and that needing money at her age did not make her a different kind of person.
What got cheap
I keep coming back to that hour because of what the industry decided last year.
In June 2025 Sean Grove of OpenAI gave a talk called The New Code. Code, he said, was a small share of the value a programmer brings. The specification behind it is the rest, and the most valuable programmer will be whoever states intent most clearly.
That July Amazon shipped Kiro, a coding tool built around specifications. GitHub followed in September with Spec Kit. Within a year, spec-driven had replaced vibe as the word you were meant to use.
They are right about the change. When an agent builds most of what you describe, the slow part becomes the describing.
The serious version of this already knows the rest. Grove did not describe a document handed over once. He described a living thing, revised whenever what got built showed the words had been wrong. Kiro asks for the spec before it plans, and Spec Kit later added a step that does nothing but surface the underspecified parts.
Every one of those loops is real, and every one of them closes against the build. A failing test tells you the spec was wrong about the software.
Nothing in that loop can tell you the spec was wrong about her, because she is not in it.
The industry had this argument once before and settled it. In 1987 Fred Brooks wrote that the hardest single part of building a software system is deciding precisely what to build. His paper was subtitled Essence and Accidents of Software Engineering.
The accidents are the labour of getting a thing into code. The essence is what the thing is for, which is a fact about a person and has never been available by inspection. Forty years on, the accidents have all but gone.
What is left is the essence, and hers was the part she had not said in the first minute.
Why she could not have told me
Plenty of what somebody asks for is exactly what they want. A person chasing a refund wants the refund. A person cancelling wants to cancel. They have settled the matter, and a specification that says so is a good specification.
She had settled nothing.
In 1977 Richard Nisbett and Timothy Wilson reviewed the experiments in which people were asked why they had done something. They answered fluently and confidently, and their answers had little to do with what had moved them.
They were not lying. They had no access to the process, so they built a plausible account and believed it. What their paper describes is the introspection illusion: you feel you can see your own reasons, and what you can see is a story.
Preference goes the same way. What a person wants gets made while you wait for them to say it, out of your question, the setting, and what they think you can do about it. Paul Slovic called this constructed preference in 1995, reviewing two decades of the research.
So the want was not sitting in her, waiting to be specified. It was being made while we talked.
What told me instead
Nothing she said was the evidence. The evidence was that she stopped saying things.
A transcript of that session would show somebody answering questions about borrowing, briefly, then talking about something more personal towards the end.
It would not show the moment the room changed. It would not show her sitting back, or the thirty seconds I decided not to fill, or that I had already changed what I was going to ask next before she said another word.
Most of what passed between us in that hour was not language.
And I nearly missed it. If the session had run fifty minutes rather than sixty, or if she had typed her answers into a box, or if I had filled that silence the way you do when a session is going badly, I would have written down that she needed access to short-term credit.
Which was true. And I would have gone home.
What gets built now
A model would have had the transcript and nothing else.
Ask it afterwards why it read her that way and it will answer, fluently. That answer is written after the fact rather than recalled.
Miles Turpin and three colleagues tested those accounts in 2023. They reordered the options in multiple-choice questions so the right answer always sat first. The models picked up the pattern and started choosing first regardless, and accuracy fell by as much as 36 per cent.
Then the researchers read the reasoning the models had written to justify those answers.
Across 426 explanations defending a biased choice, one mentioned the bias.
Turpin's paper cites Nisbett and Wilson directly, so the parallel is theirs rather than mine. The machine's half of it goes by unfaithful reasoning.
The unreliable account is the smaller problem. The larger one is that the transcript was never where the answer was.
Then there is what gets built from it. The software works, it runs, the tests pass. Whether it works for her is a different question, and once it exists nobody on your team is paid to ask it.
A wrong reading used to sit in a document where somebody could argue with it. Now it sits in a service, and the service is the argument for the spec it came from.
It is easy to get drunk on building when the tokens are this cheap. Every prompt returns something, and something always feels like progress.
Stopping to say we do not know yet what she meant used to cost a fortnight. Now it costs an afternoon you would rather spend building.
Her first sentence, handed to something that builds, would have produced an overdraft product by teatime. A good one. She would have used it, and gone on feeling exactly the way she felt about needing it.
Writing her into the spec
A transcript is what gets pasted into a model now. It holds her words and nothing else.
So the spec takes three lines rather than one.
Said. She needs money at short notice, some months more than others.
Did. Went quiet when I asked about borrowing. Sat back. Answered in one word.
Read. The need underneath is permission rather than credit. Ours, from one hour, untested on anyone else.
The middle line is the one no recording holds, and it is the line that changed what got built. Anything a spec claims about a person has to lead back to a person who said it or did it.
What we built instead
What went forward was not an overdraft product.
It was a commercial proposition for people at her stage, built around support, education and something closer to a community than a credit facility. What people in her position actually do about money. What the options cost. A way of talking about all of it that did not make using any of it feel like a confession.
The job of the whole thing was to make what she was ashamed of look ordinary, which is what she had come in for and could not ask for.
It shipped. It is still running.
Her first sentence had none of that in it. The hour did, and most of the hour was not words.