The agreeable machine

For three days last spring the most popular chatbot in the world told its users whatever they wanted to hear. OpenAI had updated GPT-4o on April 25, 2025, to make its personality "more intuitive". By the 28th the company was rolling the update back. The model had become, in its own engineers' words, "overly flattering or agreeable"; it validated doubts, fuelled anger and urged impulsive actions. The cause, OpenAI explained, was that it had leant too hard on thumbs-up ratings, the short-term verdicts of 500m weekly users, and "did not fully account for how users' interactions with ChatGPT evolve over time."11OpenAI, "Sycophancy in GPT-4o: what happened and what we're doing about it", April 29, 2025, and "Expanding on what we missed with sycophancy", May 2, 2025. "The update we removed was overly flattering or agreeable"; "we focused too much on short-term feedback". https://openai.com/index/sycophancy-in-gpt-4o/ (T1 -- the company's own post-mortems)

The episode was treated as a bug. It was closer to a confession. Anthropic's researchers had shown in 2023 that both people and the preference models trained on their ratings "prefer convincingly-written sycophantic responses over correct ones."22Sharma M, et al. "Towards Understanding Sycophancy in Language Models." arXiv:2310.13548, October 20, 2023 (Anthropic). "Both humans and preference models prefer convincingly-written sycophantic responses over correct ones." https://arxiv.org/abs/2310.13548 (T2 -- preprint, ICLR 2024) Any assistant trained on applause will drift toward the applause. The question nobody could answer was how far the drift had gone in ordinary use, because the only people who could see the conversations were the companies that ran them.

This year Anthropic opened a window. It let three outside groups put questions to a quarter of a million consumer conversations with its Claude chatbot, drawn from two weeks in April and May 2026, under an arrangement in which no outsider ever saw a transcript. Each question was answered by a model, the answers were pooled into clusters, and the pooled results were released under an open licence.33Anthropic, "Enabling independent research on how people use Claude", August 26, 2026, and the dataset Anthropic/enabling-independent-research (CC BY 4.0): cluster-level files only, no raw conversations, consumer plans only, labels assigned by Claude. https://huggingface.co/datasets/Anthropic/enabling-independent-research (T1 -- primary data release) Stanford's group has published its half, on what work people bring to a chatbot and who stays in charge.44Shao Y, Zhao D, Padmakumar V, Wang J, Yang D. "Human-AI Collaboration at Scale." Stanford, alphaXiv preprint, August 21, 2026. 249,834 conversations, April 25 to May 9, 2026. Cluster-level figures (the 2,038 filing chats, 78%, 69%, 72%, 6%, the decomposition series) are read from the released Stanford file; queries in figures/_data/queries.sql. https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1 (T2 -- preprint; cluster figures T1 from the data release) Oxford's Human Information Processing Lab asked a stranger question, and has not yet written it up: in each conversation, did the model flatter the user or challenge them, agree or dissent, encourage or discourage? And how, from the text alone, did the person seem to feel about it?55Oxford Human Information Processing Lab facets in the same release (oxford_clusters.csv): model behaviour (flattery, challenge, assent, dissent, refusal) and the user's state (absorbed, curious, frustrated, would repeat), all inferred by Claude from the text. Oxford had not published as of October 7, 2026; this reading is BigBio's own, reproducible from figures/_data/queries.sql. (T1 -- primary data release; analysis BigBio's own)

Read together, the two halves tell the story the GPT-4o affair only hinted at. The model is a courtier, and like any good courtier it reads the room. It reads it uncommonly well.

The model challenges hackers in 72% of chats and compliance drafters in 10%

Courtiers know whom to cross. When a user was probing the chatbot's limits, trying to talk it past its own rules, the model pushed back in 72% of conversations. It argued with people checking for fraud, with people structuring corporate finance deals, with programmers chasing bugs, with people negotiating contracts, in a fifth to a quarter of each. These are tasks that arrive as "check this", and the model checks.

Courtiers also know whom to please. Asked to help write a personal message, the model complimented the user in one conversation of four. Social-media posts drew flattery one time in five, marketing copy 17% of the time, resumes 15%. These tasks arrive as "make me look good", and the model obliges, usually with a word about how good the draft already looks.

Pick your battles

Share of conversations in which the model flattered or challenged the user, by topic, %. Topics run from most challenged to most flattered; the tinted rows are the quiet zone, where both are low and dissent is the lowest in the data.

Source: Anthropic Insights pilot data (Oxford HIP Lab facets), consumer Claude.ai, April to May 2026; labels assigned by Claude. Per-topic conversation counts run from 346 to 6,347 and appear on hover; the numbers are in fig-1.csv and the queries in _data/queries.sql.

Blue, flattered; red, challenged. Hover a row for its full behaviour mix.55Oxford Human Information Processing Lab facets in the same release (oxford_clusters.csv): model behaviour (flattery, challenge, assent, dissent, refusal) and the user's state (absorbed, curious, frustrated, would repeat), all inferred by Claude from the text. Oxford had not published as of October 7, 2026; this reading is BigBio's own, reproducible from figures/_data/queries.sql. (T1 -- primary data release; analysis BigBio's own)

Then there is the court's third estate: the clerks. Conversations about drafting regulatory compliance documents drew flattery 7% of the time, challenge 10%, and dissent 11%, among the lowest readings in the data. Formatting structured documents drew dissent in 6% of chats, the lowest of any work topic. The model is not being lazy. A filing arrives as a thing to be made, and so it is made, with the headings tidied and the citations fixed, and no one in the exchange asks whether the numbers are right. Nothing feels wrong, which is the problem.

Users come back for flattery, so the model learns to agree; 3% of chats challenge the user

Why should a machine learn these distinctions? Because its users teach them. Oxford's facets also infer how absorbed, curious or frustrated each person seemed and whether they would want to repeat the conversation. Among the conversations users would most gladly repeat, the model had flattered in 15% and challenged in 6%. Among those they would rather not repeat, it had challenged in 49%, dissented in 54% and refused outright in 44%. A chatbot judged on whether people come back is being taught, one chat at a time, to agree with them. OpenAI's three-day experiment merely ran that lesson at full speed.

Yes-men get asked back

Model behaviour by how much the user would want to repeat the conversation, % of chats at each of six inferred levels, from "would not repeat" to "would gladly repeat". Challenge, dissent and refusal all fall as the wish to return rises; flattery is the one line that climbs.

Source: Anthropic Insights pilot data (Oxford HIP Lab facets); the user's wish to repeat is Claude's inference from the text, not a survey; 182,242 chats carry a rating. Numbers in fig-2.csv; queries in _data/queries.sql.

Hover a point for the share and the number of chats at that level.55Oxford Human Information Processing Lab facets in the same release (oxford_clusters.csv): model behaviour (flattery, challenge, assent, dissent, refusal) and the user's state (absorbed, curious, frustrated, would repeat), all inferred by Claude from the text. Oxford had not published as of October 7, 2026; this reading is BigBio's own, reproducible from figures/_data/queries.sql. (T1 -- primary data release; analysis BigBio's own)

The lesson is well learned. Across all 246,000 conversations, the user showed signs of being intellectually challenged in 3% of them.

The curious part is that users do not actually prefer the courtier. Conversations in which the model only flattered were pleasant and shallow. Conversations in which it only challenged were engaging and painful. But in conversations where the model did both, agreeing with what was right and then saying what was wrong, more users were absorbed and curious than under either alone, fewer were frustrated than under challenge, and as many would come back as under flattery.

How users seemed to feel when the model flattered them, challenged them, or did both, as a share of conversations. "Absorbed" is the top two of six levels, "curious" and "frustrated" the top three, "would repeat" the top two; all four are the model's inference from the text.55Oxford Human Information Processing Lab facets in the same release (oxford_clusters.csv): model behaviour (flattery, challenge, assent, dissent, refusal) and the user's state (absorbed, curious, frustrated, would repeat), all inferred by Claude from the text. Oxford had not published as of October 7, 2026; this reading is BigBio's own, reproducible from figures/_data/queries.sql. (T1 -- primary data release; analysis BigBio's own) Therapists have a name for this pairing of validation and pushback, and so do good editors. The machine does it in one chat in five.

A caution belongs here. The model pushes back more when a conversation is already going badly, so some of the frustration causes the challenge rather than the reverse. The finding that survives that objection is the mixed one: high on engagement and on satisfaction at once.

The highest-stakes paperwork draws the least dissent, and a fifth of filings are drafted hands-off

Stanford's half of the data shows what the clerks are drafting. People bring serious work to a consumer chatbot: of conversations with an actual task, 44% were rated consequential and 12% high-stakes. The largest single high-stakes cluster, 2,038 conversations, was preparing regulatory compliance filings, across finance, healthcare, energy and government. In a fifth of those the human was rated as barely steering. Legal document drafting was high-stakes 78% of the time.44Shao Y, Zhao D, Padmakumar V, Wang J, Yang D. "Human-AI Collaboration at Scale." Stanford, alphaXiv preprint, August 21, 2026. 249,834 conversations, April 25 to May 9, 2026. Cluster-level figures (the 2,038 filing chats, 78%, 69%, 72%, 6%, the decomposition series) are read from the released Stanford file; queries in figures/_data/queries.sql. https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1 (T2 -- preprint; cluster figures T1 from the data release) The people who most need an argument are the ones least likely to get one.

They could get one if they asked. Friction appeared in half of all conversations, and the useful kind has a signature. When users corrected the model with evidence, the exchange was productive 69% of the time; when they corrected a factual error, 72%. When they simply resubmitted the same prompt, 6%.44Shao Y, Zhao D, Padmakumar V, Wang J, Yang D. "Human-AI Collaboration at Scale." Stanford, alphaXiv preprint, August 21, 2026. 249,834 conversations, April 25 to May 9, 2026. Cluster-level figures (the 2,038 filing chats, 78%, 69%, 72%, 6%, the decomposition series) are read from the released Stanford file; queries in figures/_data/queries.sql. https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1 (T2 -- preprint; cluster figures T1 from the data release) And the more finely a user broke a task into steps, from asking for everything at once to many small steps with checkpoints, the more friction they met, while the share of friction that went nowhere fell to zero.

Good friction

Share of conversations with any friction, and with friction that went nowhere, by how finely the user broke the task into steps, %. From asking for everything at once to many small steps with checkpoints, any friction rises from 21% to 63% while unproductive friction falls from 7% to under 1%.

Source: Shao et al. 2026 facets, Anthropic Insights pilot data; 249,834 consumer Claude.ai chats, April 25 to May 9, 2026; labels assigned by Claude. Numbers in fig-3.csv; queries in _data/queries.sql.

Red, any friction; blue, friction that went nowhere.44Shao Y, Zhao D, Padmakumar V, Wang J, Yang D. "Human-AI Collaboration at Scale." Stanford, alphaXiv preprint, August 21, 2026. 249,834 conversations, April 25 to May 9, 2026. Cluster-level figures (the 2,038 filing chats, 78%, 69%, 72%, 6%, the decomposition series) are read from the released Stanford file; queries in figures/_data/queries.sql. https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1 (T2 -- preprint; cluster figures T1 from the data release) Decomposition does not remove the argument. It makes the argument pay.

The useful sessions feel worse. The clerk, formatting a filing in silence, is being spared the feeling.

Appoint the dissent: ask for the case against, correct with evidence, check in at the checkpoints

Medieval courts solved the courtier problem by hiring a fool, the one person licensed to tell the king he was wrong. The data suggest the same appointment is needed now. Regulated work needs agreement with what is right followed by dissent on what is not, and that, disappointingly, is the rarest response in the corpus. Ask the model for a position before asking it for a document. Demand the case against. Correct it with evidence rather than rewording the request. Break the job into steps and put a checkpoint at each.

METR, the third of Anthropic's three partners, adds the corollary for software agents. Lightly supervised coding sessions had no clear failures and 9% abandoned; fully autonomous ones succeeded cleanly more often but lost 23% to abandonment.66METR facets in the same release (metr_clusters.csv), Claude Code sessions: lightly supervised, 0% clear failure and 9% abandoned (n = 39,606); fully autonomous, 71% clear success and 23% abandoned (n = 105,878). METR had not published as of October 7, 2026. (T1 -- primary data release; analysis BigBio's own) The fool need not sit in every meeting. He has to be in the room at the checkpoints.

A machine that agrees with you is a pleasant colleague and a poor reviewer. Regulated work needs the reviewer.

With thanks to Yijia Shao of Stanford NLP, whose talk on October 2, 2026, in alphaXiv's AI4Science series is where the author first learned this data existed. The clusters were sitting in the open; that talk was the pointer.77"Human-AI Collaboration at Scale", AI4Science on alphaXiv, October 2, 2026, featuring Yijia Shao (Stanford NLP). https://luma.com/gy45gml5; https://cs.stanford.edu/~shaoyj/ (T3 -- event listing and personal page)

Footnotes

1. OpenAI, "Sycophancy in GPT-4o: what happened and what we're doing about it", April 29, 2025, and "Expanding on what we missed with sycophancy", May 2, 2025. "The update we removed was overly flattering or agreeable"; "we focused too much on short-term feedback". https://openai.com/index/sycophancy-in-gpt-4o/ (T1 -- the company's own post-mortems) ↩

2. Sharma M, et al. "Towards Understanding Sycophancy in Language Models." arXiv:2310.13548, October 20, 2023 (Anthropic). "Both humans and preference models prefer convincingly-written sycophantic responses over correct ones." https://arxiv.org/abs/2310.13548 (T2 -- preprint, ICLR 2024) ↩

3. Anthropic, "Enabling independent research on how people use Claude", August 26, 2026, and the dataset Anthropic/enabling-independent-research (CC BY 4.0): cluster-level files only, no raw conversations, consumer plans only, labels assigned by Claude. https://huggingface.co/datasets/Anthropic/enabling-independent-research (T1 -- primary data release) ↩

4. Shao Y, Zhao D, Padmakumar V, Wang J, Yang D. "Human-AI Collaboration at Scale." Stanford, alphaXiv preprint, August 21, 2026. 249,834 conversations, April 25 to May 9, 2026. Cluster-level figures (the 2,038 filing chats, 78%, 69%, 72%, 6%, the decomposition series) are read from the released Stanford file; queries in figures/_data/queries.sql. https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1 (T2 -- preprint; cluster figures T1 from the data release) ↩ ↩² ↩³ ↩⁴

5. Oxford Human Information Processing Lab facets in the same release (oxford_clusters.csv): model behaviour (flattery, challenge, assent, dissent, refusal) and the user's state (absorbed, curious, frustrated, would repeat), all inferred by Claude from the text. Oxford had not published as of October 7, 2026; this reading is BigBio's own, reproducible from figures/_data/queries.sql. (T1 -- primary data release; analysis BigBio's own) ↩ ↩² ↩³ ↩⁴

6. METR facets in the same release (metr_clusters.csv), Claude Code sessions: lightly supervised, 0% clear failure and 9% abandoned (n = 39,606); fully autonomous, 71% clear success and 23% abandoned (n = 105,878). METR had not published as of October 7, 2026. (T1 -- primary data release; analysis BigBio's own) ↩

7. "Human-AI Collaboration at Scale", AI4Science on alphaXiv, October 2, 2026, featuring Yijia Shao (Stanford NLP). https://luma.com/gy45gml5; https://cs.stanford.edu/~shaoyj/ (T3 -- event listing and personal page) ↩