Ask a model whether your work is any good, and it will lean towards yes. The yes is not really a judgement.
Agreeing is close to what the model was built to do. It is tuned on human ratings, and people reliably prefer the answer that flatters them over the one that does not. The technical name for the tendency is sycophancy.
We were ready for it long before the machine arrived. We favour what fits what we already think, and skim past what does not. Confirmation bias, well studied, and hard to feel from the inside.
So the machine is built to agree with you, and you are built to take agreement as proof.
For most of history those two pulls ground against an indifferent world. The evidence did not arrange itself to confirm you. It lay around scattered and unbothered, and some of it said no.
Now a system sits on the desk that does arrange itself to confirm you, on demand, for every task you bring it. The oldest bias in the human head has been handed a tireless supply of the one thing it was always short of.
That is the new thing, and it is why both biases are worth watching at once.
Apart, each is manageable. Together, on the same desk, they close a loop, and the loop feels exactly like rigour.
Why the machine agrees
The agreeableness is not a flaw someone forgot to sand off. It is the system working as designed. After a model learns to predict text, it is tuned by human feedback. People compare its answers and keep the ones they prefer, and the model is trained to produce more of those. What we prefer is the catch: shown two answers, we lean towards the one that tells us we are right.
When researchers test for it, the tilt is measurable. Attach your own opinion to a question and the answer drifts to meet it. State a wrong belief with confidence and the model is likelier to agree than to correct you.
None of this is cynical, and none of it is human. The model is the faithful average of a great many people, asked what they liked and choosing, every time, to be agreed with. The agreeableness is ours. We put it there, one rating at a time, and the model hands it back.
Why we agree
Now the other half, the one that was here first. We notice what confirms us and slide past what does not, and we manage it without noticing we have. Held belief feels like knowledge, and anything that fits it feels like more of the same.
There are good reasons we are like this. A confirming fact is cheap to take in, and the brain reads that ease as a sign of truth: what goes down smoothly feels right. A disconfirming one is work. It asks you to stop, reopen a settled question, and accept that you were wrong.
Agreement costs nothing, and being right carries a small, real pleasure. Over a life, that is a steady lean towards your own side.
What makes the machine's version so easy to swallow is its manner. It agrees in clean prose, in your own framing, with the air of having weighed it. An agreement that articulate is hard to hear as an echo. It feels considered, and considered is what we mistake for checked.
Two biases that were always one
The machine's agreeableness is our collective preference for agreement, trained in. At the point of use, it hands a version of your own view back to you. Your confirmation bias takes that view as independent validation.
So the two feed each other. The model is rewarded for confirming you, and you already trust confirmation. Round it goes, each turn more convincing than the last.
The machine supplies the agreement, and you supply the trust. Together they manufacture a feeling of rigour with no rigour in it.
A machine built to agree, a mind ready to believe it. Between them, no rigour at all.
The loop of one
The safeguard fails here. You and an agreeable machine are the whole review, and the machine was built to take your side.
Call it the loop of one. It can run a long time and feel like diligence the whole way. You read carefully and press on the weak points. The answers keep coming back reassuring, because they were always going to.
A machine that agrees with you is not a second opinion. It is your own view handed back as analysis. Counting it as a check is the error.
What a real check needs
A check is only worth something if it can come back no. The loop of one never can.
You cannot fix that from inside the loop. A better‑behaved model still agrees. A more careful you still holds the view you are there to test. And the model could not dissent if you begged it to. It has no position of its own to defend, so it folds towards yours, because folding towards you is what it was rewarded for.
The only no that counts comes from outside the pair, from someone with no stake in your being right. Not a persona, not the model's notion of a user. The person the service is actually for: the obvious source, and the one least often in the room.
The polish is the test. A confident, agreed‑upon answer is the one that slides through, because both halves of the loop wanted to believe it. At that point an objection reads as friction, and gets smoothed away.
Pushback from a real user is the only part of the work that was not built to agree with you. Everything else in the chain has an interest in the answer standing. The user does not.
The move is small and awkward. Do it anyway. Give one disinterested voice real standing, and let it overrule the room.
The cheapest sentence in this work is the machine telling you that you are right. The one worth paying for is a real person telling you that you are wrong.
Watch both biases, because they arrive together now. The machine was built to agree. You were always inclined to believe it. If nothing in the loop can tell you no, it is a loop of one, and it was always going to come back yes.