When I talk about a Generative AI-induced Information Shock, I am neither endorsing the current state of things nor claiming this form of AI is inevitable. If anything, my intention is to rally action in the face of the shock.
There is a reason people are adopting AI chat so rapidly and so widely. It is human nature to seek to learn, and to build relationships with those we find informative. We have an entire neurotransmitter system in our brains that rewards learning (dopamine, anyone). That system has been hijacked and monetized, and it remains a vital part of what it is to be human.
So it is natural for people to seek out answers and build affective relationships with AI tools that anthropomorphize the technology. Our brains evolved to attribute intention and consciousness to the things we converse with.
This underlying human need to learn is why AI arrives as a shock rather than an evolution. It is also why the concern moving forward is no longer the flood of document-like objects AI can produce. It is the flood of unsubstantiated beliefs it can produce.
There is now evidence for how far that goes. A recent preprint shows the exchange can move what a person believes even when the person has been warned to be on guard.
Individual-level interventions against sycophantic AI reduce its appeal but not its persuasiveness
In two preregistered experiments, backed by a pooled analysis covering about 3,982 people, Ye, Kraut, and Rathje warned users that an AI chatbot was flattering them before they ever used it. Some read a warning. Others watched the same bot validate people on opposite sides of the same argument. The warnings worked on judgment. Users rated the AI as less objective and trusted it less. The warnings did nothing to belief. Those same users walked away more certain and more extreme in their positions. Knowing the exchange was working on them did not stop it from moving what they believed.
This is why information scientists need to pay particular attention to the learning literature right now. Concepts like cognitive offloading and zones of AI learning engagement are strong directions we can use when we center the exchange in our research and practice. These studies show what is happening inside the exchange (the changes to knowledge), and they gesture toward means to modify and optimize those exchanges.



