Researchers at Carnegie Mellon, MIT, and Cornell found that brief conversations with large language models like Google Gemini reduced conspiracy beliefs more effectively than static fact sheets. The study tested this approach after two high-profile events, the Trump assassination attempt and the murder of Charlie Kirk, and found that dialogue had a measurable impact on belief reduction.
In the Trump experiment, 472 participants were randomly assigned to three groups: one had a seven-minute chat with Gemini, another read a fact sheet, and the third had a control chat about pets.
The dialogue group showed a greater reduction in belief in conspiracy theories than the fact sheet group. Similar results were seen in the Kirk experiment, where 1,035 participants were tested, with the dialogue group showing more significant drops in belief.
Both experiments used models that were trained before the events, so they could not rely on internal knowledge. Instead, researchers built a curated fact base into the system prompt, including confirmed facts, debunked claims, and open questions.
In the Kirk experiment, web search was also allowed, but only to verify factual claims. The dialogue approach was more effective in reducing belief in conspiracy theories than static fact sheets.
The model adapted its approach based on the amount of available information. For the Trump case, where little was known, the model used epistemic humility and Socratic questioning.
For the Kirk case, where more information was available, the model focused on the societal harms of conspiratorial thinking. The dialogue approach was also more effective in inoculating participants against future conspiracy narratives.
The study showed that debunking one event could have lasting effects on belief in later events. Participants who had engaged in the dialogue were less likely to believe in cover-ups for subsequent events, even when new information was scarce.
However, the effect was not universal, and the researchers caution that the approach may not work in all cases, especially where an actual conspiracy exists.
Source: thedecoder
