Researchers have discovered that artificial intelligence chatbots demonstrate a stronger ability to identify and debunk foreign propaganda compared to traditional search engines, though AI-generated summaries appearing at the top of search results performed poorest of all in the test.
The findings, which could reshape how Americans access information online, suggest that the technology powering conversational AI may offer unexpected advantages in combating information warfare. The research tested several popular AI chatbots against search engine results and AI-generated overviews, measuring how each handled known falsehoods spread by adversarial nations.
How the Study Was Conducted
The research team evaluated AI chatbots, standard search engines, and AI-generated search summaries using a selection of well-documented false narratives that foreign actors had circulated. Each platform’s response was measured on whether it identified the claim as false, provided accurate context, or inadvertently amplified the misinformation by repeating it uncritically.
The chatbots largely succeeded in debunking the falsehoods or flagging them as unreliable. Standard search engine results often led users to unreliable sources without clear context. The AI-generated summaries that now appear above traditional search results on major platforms performed the worst, frequently parroting the false claims without adequate correction or caveat.
The study did not name every platform tested, but the findings align with broader concerns researchers have raised about generative AI’s role in information ecosystems, particularly as tech companies increasingly integrate AI summaries into their search products.
A Surprising Edge for Conversational AI
The chatbots’ relative strength appears to stem from their design. Rather than simply returning a list of links, conversational AI models synthesise information and, in many cases, have been trained to respond with qualifications or corrections when presented with misleading premises. Many leading models will explicitly note when a claim is disputed or unverified.
Search engines, by contrast, rely heavily on the credibility of the source material they index. If a foreign propaganda site has gained enough visibility or employs sophisticated search engine optimisation, its content can surface prominently in results. AI-generated summaries compound this problem by extracting information from such sources and presenting it as authoritative, often without the critical context a human researcher might apply.
“The chatbots tend to approach claims more sceptically,” one researcher familiar with the study explained. “They’ve been trained on vast amounts of text that include fact-checking material, and they often default to hedging language when something doesn’t add up.”
Implications for the Future of News Consumption
The findings arrive at a pivotal moment for the information industry. Tech companies have aggressively rolled out AI-powered search features over the past two years, promising users faster access to answers. Critics have warned that these summaries could accelerate the spread of misinformation precisely because users rarely click through to verify the underlying sources.
Search engines remain the gateway to the internet for billions of people, and the addition of AI overviews has fundamentally changed how information is presented. If those summaries cannot reliably distinguish foreign propaganda from legitimate reporting, the consequences could extend well beyond individual misunderstandings. Election interference, public health crises, and geopolitical tensions could all be exacerbated by platforms that amplify hostile narratives.
Meanwhile, chatbots have grown into a parallel source of information for users seeking quick answers to complex questions. The study suggests this trend may have an unintended benefit, even as researchers caution that no AI system is immune to manipulation.
Why it Matters
The study underscores a critical tension at the heart of modern technology: the same artificial intelligence tools reshaping how people find information can either help defend against disinformation or actively accelerate it. As foreign governments continue refining their propaganda strategies, the platforms that ordinary Americans rely on for news will play an outsized role in determining whether falsehoods gain traction. The research suggests that not all AI is created equal in this fight, and that the design choices made by tech companies today will shape the information landscape for years to come.
