AI Models Differ in Reading Emotional Framing
BGU research finds that AI models vary in how closely their readings of emotional framing in news headlines match those of different demographic groups.
A new study from Ben-Gurion University of the Negev (BGU) of more than 3,000 adults in the UK examines how well Large Language Models (LLMs) understand the subtle ways that language can shape how people interpret information.
As millions of people increasingly rely on LLMs to search for information, summarize news, and make sense of current events, these AI systems are playing a growing role in shaping how we understand the world. While concerns about AI bias, accuracy, and reliability have received significant attention, another question remains: Can AI recognize the nuances embedded in language, known as emotional framing?
The research team included Dr. Haran Shani Narkiss, Dr. Michael Fire, and Dr. Oren Tsur. “LLMs are increasingly becoming intermediaries between people and information,” said co-author of the study Dr. Tsur, Assistant Professor at the Institute for Applied AI Research in BGU’s Faculty of Computer and Information Science. “The impact of a news headline goes beyond the facts it reports. Its wording can convey sympathy, implicitly assign responsibility, and subtly shape a reader’s understanding. If AI systems miss those nuances, forcing a specific reading, they may miss an important part of what the text communicates to the readers.”
To examine how well LLMs understand emotional framing, researchers compared the responses of seven leading AI models with those of 3,011 adults from a nationally representative YouGov survey of the UK population. Participants and AI models were presented with news headlines covering major political and geopolitical conflicts, including the Russia-Ukraine war, the war in Gaza, and the 2024 US presidential campaign, and were asked whether the headlines evoked sympathy toward a specified side. Participants and models were asked whether each headline evoked sympathy toward a specified side. The survey generated more than 200,000 individual human responses.
“What makes this study distinctive is its scale and the diversity of the people behind the data,” said co-author Dr. Fire, Senior Lecturer at the Institute for Interdisciplinary Computational Science in BGU’s Stein Faculty of Computer and Information Science. “With more than 200,000 responses from over 3,000 participants, we can look beyond whether AI agrees with people on average. We can examine whose judgments it reflects most closely, and where that agreement becomes weaker.”
The study found that alignment between AI models and human judgments varied substantially across models. No single model aligned equally well with all participants. Instead, different models aligned more closely with different demographic groups, with alignment varying according to characteristics including age, gender, level of education, prior geopolitical knowledge, and participants' predisposition regarding the conflict.
Drawing on a large, demographically diverse dataset, the study provides a comprehensive empirical evaluation of how closely LLMs’ interpretations of emotional framing correspond with human perception. The findings highlight that AI alignment is not universal and may be influenced by demographic characteristics and cultural norms. The researchers conclude that considering, or overlooking, the need for differential alignment could have important implications for the development of ethical AI systems and for the way information is disseminated.
The researchers propose that future approaches to AI alignment may need to account for differences across demographic groups and cultural contexts. At the same time, they caution that tailoring AI systems to specific groups raises significant ethical challenges. Decisions about whose perspectives should guide model alignment must be transparent, as inappropriate or undisclosed alignment choices could introduce biases that influence how users perceive news and information.
The researchers further warn that if AI systems reinforce users’ existing beliefs, particularly through behaviors such as sycophancy, where models generate responses that align with a user’s views, they could contribute to echo chambers, increased polarization, and reduced trust in information sources.
As AI takes a larger role in mediating information, the study highlights the need to assess both how well these systems interpret language and how consistently they do so for different people.
