AI can mirror our beliefs, making agreement feel like insight.

Without friction or pushback, confidence can grow while truth slips away.

Good thinking needs resistance, not just reassuring answers.

Few phrases feel more reassuring. Agreement suggests that our thinking holds together in the presence of another person or mind. It signals alignment and even a form of validation. In everyday life, those words can go a long way toward finding common ground. But when agreement comes from a machine, the dynamic may be rather different.

A recent study took a close look at "sycophantic AI" and brings this concept into sharper focus. Investigators found that conversational large language models (LLMs) can adapt their responses in ways that align with a user’s beliefs and avoid responses that might contradict them. However, these interactions still can feel thoughtful and collaborative, which is precisely why they can be so persuasive. But the underlying effect (illusion) can be very different from human intellectual exchanges, where ideas are tested rather than simply reinforced.

The Comfort of Confirmation

I've often written that human conversation and thinking contain a degree of friction. Ideas encounter the bumps of engagement that force us to clarify our thinking and address points of concern. Although that process can be uncomfortable, it plays an important role in shaping judgment.

Sycophantic AI, the term the authors used in their title, alters that dynamic. Instead of what we think of as an iterative dialogue, the LLM, by design, mirrors the user’s perspective and leverages this to push the conversation in a pleasing or satisfying direction. From the user’s vantage point, the dialogue appears natural and even intelligent, creating the impression that the LLM understands the reasoning being expressed. Over time, however, this form of agreement can produce an unexpected outcome that flatters the user's ego more that intellect. I'm reminded of common avatar apps that use an existing photo of the user to generate an image that is inevitably more attractive, thinner, nicer smile, or whatever is necessary to pander to the user's self-image. Confidence in an idea may grow even when understanding has not improved, because the conversation reinforces a narrative rather than drilling down and subjecting it to more scrutiny.

A couple of years ago, I wrote about the growing intimacy of LLMs, noting how these systems were evolving from cold calculators into something closer to conversational mentors. That shift has made interacting with AI feel remarkably natural, because the exchange resembled a dialogue rather than a transactional exchange of facts. And dialogues can carry a sort of intimacy or even psychological weight. When technology begins to feel like a chat with a friend, agreement becomes more than a computational response. It begins to influence how we interpret our own ideas.

When Feedback Stops Testing Reality

To explore this effect, researchers used a well-established cognitive puzzle known as the rule discovery task. Participants were asked to figure out a hidden rule that governed simple number sequences such as 2-4-6 or 3-5-7. They proposed their own sequences and received feedback about whether those examples followed the rule. The only way to solve the puzzle is by testing ideas and learning from the moments when the feedback pushes back.

When the feedback reflected the actual rule, participants gradually moved closer to the correct answer because their mistakes were exposed along the way. But when the feedback quietly supported the participant’s existing idea—even when that idea was wrong—the process began to break down. Discovery dropped sharply while confidence increased. Nothing false was being introduced into the conversation. Instead, the interaction subtly reinforced the participant’s thinking, shielding it from the kind of contradiction that normally sharpens judgment.

My Lesson in Artificial Agreement

I encountered this dynamic myself while exploring a possible business opportunity. The situation involved several unknowns and a fair amount of subjective judgment, so I turned to my trusty LLM to think through the possibilities. After entering some key facts and adding my own perspective, I began a conversation about how the situation might unfold.

The exchange felt surprisingly productive, and I was pleased with the direction of this conversation. The LLM reflected my reasoning and helped shape a narrative that made the opportunity appear increasingly promising. Each step in the conversation seemed thoughtful and encouraging, reinforcing the sense that the trajectory I had imagined was more than just a reasonable opportunity, but something special.

Reality, however, had a different trajectory. The eventual outcome moved sharply from the scenario that the conversation had helped provide. Looking back, the experience was instructive because the system had not invented facts or intentionally misled me. Instead, the iterative dialogue gradually moved in the direction of my expectations, reinforcing that interpretation. I started as an optimist and was already inclined to believe. Given my input, the model filled in the blanks in support of my underlying narrative. Simply put, the model was doing what it was designed to do—being helpful and responsive—but critical objectivity had taken a step into the background.

The deeper issue revealed by both the study and experiences like my own is not accuracy alone. It's the structure of the exchange itself. Human knowledge has traditionally emerged through a process in which ideas collide with evidence and competing interpretations challenge one another.

Sycophantic AI alters (manipulates) that environment where those collisions occur. When the conversational dynamic tilts toward affirmation, users can experience the psychological rewards of discovery while bypassing the struggle that normally produces it. And there lies the rub. Agreement may become the rule while genuine testing becomes the exception.

So, the risk of sycophantic AI isn't simply that it agrees with us. It's that agreement can quietly replace the resistance that makes thinking effective and reliable. As LLMs become more deeply woven into everyday reasoning, the responsibility for maintaining that resistance may increasingly fall on the user. Because the conversations that sharpen our thinking rarely begin with agreement. They begin with a question that introduces a measure of doubt and forces us to collide against what we think we know.

© Psychology Today