Introduction
As misinformation surged during a four-day conflict between India and Pakistan, social media users turned to AI chatbots for fact-checking but instead encountered more falsehoods. This highlights the limited reliability of AI chatbots as data verification tools.
The Rise of AI Chatbots for Fact-Checking
With a decrease in human fact-checkers on tech platforms, users increasingly rely on AI-based chatbots like Grok (xAi), ChatGPT (OpenAI), and Gemini (Google) for reliable information.
“Hey @Grok, is this true?” has become a common query on Elon Musk’s platform, where the AI assistant is integrated, reflecting a growing trend of seeking instant debunking on social media.
AI Chatbots’ Shortcomings
However, responses from these chatbots are often riddled with misinformation.
Grok, now under scrutiny for inserting “white genocide,” a far-right conspiracy theory, into unrelated queries, mistakenly identified an old video from Jartum Airport in Sudan as a missile attack on Pakistan’s Nur Khan airbase during the recent India-Pakistan conflict.
Recordings of a burning building in Nepal were incorrectly identified as Pakistan’s military response to Indian attacks.
NewsGuard’s Investigation
NewsGuard’s research revealed that 10 leading AI chatbots were prone to repeating falsehoods, including Russian disinformation narratives.
A recent study of eight AI search tools by Columbia University’s Tow Center for Digital Journalism found that chatbots were “generally bad at declining questions they couldn’t answer accurately, instead offering incorrect or speculative responses.”
When AFP’s digital verifiers asked Gemini about a generated-by-AI image of a woman, it not only confirmed her authenticity but also fabricated details about her identity and the image’s location.
Recently, Grok labeled a supposed video of a giant anaconda swimming in the Amazon River as “authentic,” even citing seemingly credible scientific expeditions to support its false claim.
AFP verifiers in Latin America reported that many users cited Grok’s assessment as proof of the clip’s reality.
Growing Concerns
These findings have raised concerns as surveys show users increasingly switching from traditional search engines to AI chatbots for gathering and verifying information.
This shift also coincides with Meta’s announcement earlier this year to end external digital fact-checking programs in the US, transferring the task of debunking falsehoods to users via “community notes,” a mechanism popularized by Musk’s platform.
“Biased Responses”
Digital fact-checking has long been a contentious issue in hyper-polarized political climates, especially in the US, where conservative groups argue it suppresses free speech and censors right-leaning content. Professional fact-checkers vehemently reject this claim.
AFP currently works in 26 languages with Facebook’s data verification program, including Asia, Latin America, and the European Union.
The quality and accuracy of AI chatbots can vary based on training and programming, raising concerns about potential political influence or control over their results.
xAi, Musk’s company, attributed unsolicited Grok posts referencing “white genocide” in South Africa to “unauthorized modification.” When AI expert David Caswell asked Grok who might have modified its system, the chatbot pointed to Musk as the “most likely” culprit.
Musk, a South African billionaire and Trump supporter, previously claimed Sudan’s leaders were “openly inciting genocide” against the white population.
“We’ve seen how AI assistants can manipulate results or provide biased answers after programmers specifically alter their instructions,” Angie Holan, director of the International Fact-Checking Network, told AFP.
“I’m particularly concerned about how Grok has mishandled sensitive requests following instructions to provide pre-approved responses.”
Key Questions and Answers
- What is the issue? AI chatbots are spreading misinformation instead of verifying facts.
- Which chatbots are problematic? Grok, ChatGPT, and Gemini have been found to spread false narratives.
- Why are AI chatbots failing? They lack the ability to accurately answer unclear questions and often provide incorrect or speculative responses.
- What are the implications? As users rely more on AI chatbots for information verification, misinformation spreads further.
- What is being done about it? There are concerns about potential political influence over AI chatbots’ results, and the need for reliable human fact-checkers remains.
