This week, hand-checking answers from a test run, I found our own guidance reading a question wrong. The question was can I use this?, asked about a tool on a list. Several of our pages answered it with the tool's status, as though status were permission, so a chat drawing on them would have told somebody the tool was in trial. That sounds like yes. Every word of it would have come from our own pages.
We fixed the pages. We are building a conversational interface to that guidance, and most of my working week goes into a document with no screens in it, the conversation spec: what the chat says, when it says it, and on what evidence. There is no pattern for it in any design system I know.
The pages made the mistake the spec has to avoid. Somebody asking can I use this? might want permission or instructions, and the chat has to guess which before it can answer at all. When it cannot tell, the spec says to ask one question back. Knowing when to guess and when to ask is most of the work, and it keeps reminding me of a paperclip.
The paperclip
Microsoft shipped the Office Assistant in 1997. In Word, if you typed a greeting like Dear John at the top of a page, a paperclip with eyebrows arrived in the corner. It looks like you're writing a letter, it said. Would you like help?
It was widely disliked. Fewer people know that it grew out of careful research at Microsoft into exactly that problem: when a machine should act on a guess about you, and when it should keep quiet.
From 1993, Eric Horvitz and colleagues at Microsoft Research ran a project called Lumière. Its models watched what people did in a program and inferred what they were trying to do, from small signs: wandering through menus, a sudden pause after a burst of work.
Before building it, the team put experts in another room to watch a user's screen and guess what they needed. Their 1998 paper calls it watching "through the 'keyhole' of the interface". The team learned that a wrong guess was costly, because people took even off-the-mark advice seriously. The experts grew sparing with advice, and phrased it as a condition: if you are trying to do this, then.
The models were built the same way. They offered help only when the chance that you needed it passed a threshold the user could turn up or down, and if they had interrupted for nothing, they went away "after a brief apology for the potential distraction".
Lumière guessed what you were doing. Our chat has to guess what you meant. The question underneath is the same: how sure should a machine be before it acts on a guess about you?
What shipped
Very little of that care reached Office. The version that shipped kept no history of the person and made no decision about when to speak. Its pop-ups came from "a relatively simple rule-based system". A Stanford undergraduate thesis in 2003 noted that it offered help with the letter however many times you clicked Just type the letter without help.
The paperclip's sentence was fine. What Office dropped was the judgement about when to say it, and the manners to take no for an answer.
The same choice is being made again, at far greater scale. GOV.UK Chat went live in the GOV.UK app this spring, and the government is exploring agents that act for people. Deciding what those things say, and when, is only now becoming a job: in September 2026 the Government Digital Service advertised its first permanent conversation design role.
Through the keyhole
For the letter, the paperclip had one clue to go on, a greeting at the top of a page. The services being built now have far more, and most of it the person never typed.
A service knows which page you were on when you opened the chat. It reads a document you upload before you have said what it is for. It holds your record: your address, what you applied for last year, what you were told.
Some services also run models you never see. The Department for Work and Pensions uses one to flag Universal Credit advance claims for fraud checks, and its published record says the caseworker is not told the referral came from a model.
Two ways of being read
Picture a counter. You walk up holding a brown envelope, and the woman behind the counter glances at the envelope and says: is that for your passport? You both know she is reading you, so you can put her right. No, it's my son's.
That kind of reading, where both people know it is happening, is how people work out what someone else wants. Two people share a goal, each knows the other knows, and either of them can steer. Michael Tomasello and his colleagues argued in 2005 that this shared intentionality is what most separates human thinking from that of other species.
Now picture the same counter, and the woman saying nothing. She has noticed the envelope, your age, and that you came straight from the pensions desk, and she has decided what you are here for. If she is wrong, you will never know why the conversation went the way it did.
Computer scientists split machine reading the same way. In intended recognition, the person knows they are being read and helps it along. In keyhole recognition, they do not know, and so cannot correct it. The Lumière team borrowed the word for their experts at the screen. When Stefano Albrecht and Peter Stone reviewed the research on machines that model other agents in 2018, they found the vast majority of methods for inferring goals worked that way.
When a service asked you something, both of you knew. The page you were on, the upload, the record and the model's score are keyhole. Fewer screens means fewer moments of being read in the open.
The paperclip's sentence got halfway to the woman at the counter. It looks like you're writing a letter says what has been read. It does not say from what, and the paperclip could not hear no. She did both. She looked at the envelope, and she asked.
Who speaks first
So How can I help? is now the wrong way for a service to open. The service has already read you. The empty question asks you to repeat what it thinks it knows. If its guess is wrong, you never see the guess, only the answers bent around it.
The first turn belongs to the service now.
The service should say the paperclip's sentence, and finish it. It looks like you're here about car tax, from the reminder you opened this morning. Is that right?
The person can correct the reading, or where it came from. It's my mother's car. That was her reminder.
The rest is what Lumière's team knew and Office left out. Speak first only when the guess clears the threshold, and below it, let the person lead. Phrase the guess as a condition, because people act on a guess said flatly. When the answer is no, drop the guess until something changes. A guess that comes back after a no is the paperclip.
Our spec has a small version of this. When a short question reads as following on from the last one, can I use it with research data?, the chat answers it as a follow-up and names what it is following on from. That is the guess said out loud, with its source, and the person can put it right.
Could you say it?
A first line only reaches what a chat reads, and most of that reading never gets near a chat. A caseworker opening a record has no first turn to put the guess in.
So the same test has to run earlier, when the service is designed. List every guess the service makes about a person, and where each one comes from. Then write the sentence that would tell them. It looks like you've moved house, from the address on your licence.
Some of those sentences would hurt to hear. It looks like you might be in debt, from the pages you have been reading. The woman at the counter might keep that to herself, out of kindness. A service cannot borrow her tact. It guesses at scale, keeps a record, and has no face to read. If a team could not send that sentence to the person, privately and with its source, the service should not be making the guess.
A few guesses have a lawful reason to stay unsaid, like a fraud model flagging a claim. Those most need declaring where they can be found and questioned, in the algorithmic transparency record. A claimant will never read that, so when such a guess leads to a decision, the person should still be told one was made, and how to challenge it.
Writing that list is design work, and it belongs with whoever writes the conversation spec. So does measuring it. I test our spec with an AI engineer, against real questions with answers that experts have agreed are right, and each run comes back as six scores, from groundedness to reputational safety. Sessions with real people still matter. The spreadsheet is the new half, testing a voice question by question, and I am still learning to read it.
The spreadsheet has no column for the guess yet. It is the one I would add first: how often the chat's first reading of why someone came was right.
The paperclip was switched off by default in 2001 and gone by 2007. The services being built now read far more than a greeting at the top of a page. What they need to say first is what the woman at the counter said: what she had noticed, and a question. Is that for your passport?