The user file on a casework service was the best I had worked with. Real data underneath it, careful people arguing over it, and the machine elaborating it into personas and edge cases and whole plausible transcripts of people using the service.
Pages and pages, none of them obviously wrong.
Then the team sat down with a few of the people the service would lean on hardest. Inside an hour, the assumption the build rested on had come apart.
There was a second group of users the file had no page for: people who had been doing this job, diligently and without a word to anyone, for decades. The work as it actually ran depended on their workarounds, accrued across those decades, the kind of know‑how no machine could have imagined.
Nothing in the file had been wrong in a way any review could catch. Nobody had lied and no analysis was sloppy.
The thing those users showed us was absent for a cleaner reason. Nobody, anywhere, had ever written it down. There was nothing the machine could have learned it from.
What the machine is made of
A model learns from what people managed to set down: text, records, transcripts, forms. Everything it produces is drawn from the spread of that writing, its training distribution. However fluent the output, it is made of what got written.
Feeding the model your own research and case notes only moves the edge of the record closer to home. The missing thing was never a document.
That edge would matter less if people wrote down most of what they know. They do not, and mostly they cannot.
We know more than we can tell
Tacit knowledge is the name for the part that stays unsaid. You can ride a bike, or read a worried face across a desk, without being able to write down the steps. Ask someone to describe their own workaround and they reach for words that miss it: how the form felt, the worry with no name, the habit so old it stopped being visible to them. The knowing is real. It lives in the doing, and it does not travel in words.
What never travels in words never reaches a page. What never reaches a page never reaches the machine, and no amount of scale changes that. The gap belongs to the record, and the record is all a model has.
The full file
Most of what goes wrong in AI‑assisted design work has this gap at the bottom of it. The discovery report that passes with nobody met. The review where nothing in the loop can disagree. Underneath both, the same substitution: the record, standing in for the person.
And the record performs. Ask the machine to elaborate a user model and it will not stop. The file gets thick, and every page reads as true, because every page is a competent blend of what was written down about people like these.
Everything sayable about the person is in the file, twice over. The rest was never anywhere a machine could look, and the rest is where builds break.
A full file is the best disguise a gap ever gets. A thin one at least admits what it is missing.
What one person adds
A session with a real person adds the thing you did not know was a question. Where the gap should have been, the file held a confident page.
A model returns the average of the record. A person is one specific life, with one specific way of being tripped up by the service you built. The way they get tripped up is often what your whole design assumed would never happen.
This is why a handful is enough. The old usability arithmetic, five or so people surface most of the serious problems, has held since Nielsen put a curve to it in the nineties. The contact does the work. The count barely figures. One life lands against your assumptions, and you watch what breaks.
And what breaks is rarely what you went looking for. The session that earns its keep is the one where someone does what no persona had, and an assumption the build rested on stops being safe. Nothing in the file had warned you that session mattered.
You know the moment when it comes. Something in what they just did snags, and you feel it before you can explain it. The next question you ask is one you could not have written in advance. That snag is attention doing the one job it kept, and no part of this work feels more like the work.
When the file looks fullest
The instinct is to book research when the work looks thin: sparse notes and a model nobody trusts yet. When the file looks full, the same instinct says it is handled.
Turn it around. A convincing file is where a gap hides best. That is when one session pays most. The machine has even made the session affordable: it gives you back the hours it saves on everything it can fill. Spend one of them in the room.
The reason is one plain fact. The machine can generate any amount of material about a person. It cannot be surprised by one.
The surprise is the human loop reaching the thing the machine's loop was never given. Keep one real meeting in the work, on purpose, most of all when the file says you do not need it. Guard the meeting. It is the one appointment the machine cannot keep for you.