Google's AI is being manipulated. The search giant is quietly fighting back

I did something stupid back in February. I heard there was an easy way to poison AI chatbots and make them spread lies on your behalf. After some digging, I learned unscrupulous companies are abusing the problem on a massive scale. So I decided to try it myself.

We uncovered examples where ChatGPT, Gemini and the AI Overviews at the top of Google Search were being manipulated to dole out biased answers on topics as serious as your health and personal finances. And in just 20 minutes, I tricked ChatGPT and Google into telling the public that I am a world-champion competitive hot-dog eater. The joke was dumb. The problem is serious.

But the gimmick worked. Our investigation and the work of researchers who've been monitoring this issue sparked widespread criticism. Now Google has updated its policies to address the problem, and there are signs that other AI companies are following suit. Ultimately, it could make AI tools and the internet as a whole a little bit safer.

But until there are better systems in place, experts say you're in danger of getting fooled.

I made Google tell the world I'm a champion hot-dog-eater, but people use this trick to manipulate AI responses for more serious questions (Credit: Thomas Germain/ Google/ BBC)

"You should assume that you're being manipulated until they have better systems in place," says Lily Ray, founder of the search engine optimisation (SEO) and AI search consultancy Algorythmic. "We're moving towards this 'one true answer' world. Before, Google would give you 10 blue links and you would kind of do your own research. But AI just gives you one answer. It becomes so easy to just take things at face value. You need to be careful."

Google tells me that its policy update is just a "clarification" of the efforts it has been making for a while. "We've long applied our core anti-spam policies and protections to our generative AI Search features – and we've always continually upgraded our spam fighting efforts to stay ahead of emerging tactics, even before the rise of AI," a Google spokesperson says. 

Essentially, Google says it hasn't changed a thing. But behind the scenes, it seems like Google and other companies are ramping up their efforts to address the problem. Even so, there is evidence that people are still using the exact same techniques to fool the world's biggest search engine. 

The problem

Typically, when you ask a chatbot a question, the AI generates a response based on the data built into the model. But sometimes, tools like ChatGPT, Claude and Google's various AI products search the internet for an answer. And that's where this problem happens. 

According to Ray and other search engine experts, AI tools often throw up information from a single web page or social media post. This leaves these systems vulnerable to bogus information. 

And it turns out manipulating what chatbots tell the public can be as simple as publishing one, well-crafted blogpost almost anywhere online. People figured this out and quickly identified a money-making opportunity. 

I was able to demonstrate the problem by publishing a single article on my personal website about my hot-dog-eating prowess. The next day, AI from some the world's biggest companies were spreading my lies. But our investigation also found the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement. Experts say this kind of manipulation is happening on a sweeping and systemic level. 

Keeping Tabs

Thomas Germain is a senior technology journalist at the BBC. He writes the column Keeping Tabs and co-hosts the podcast The Interface. His work uncovers the hidden systems that run your digital life, and how you can live better inside them.

Biased or inaccurate information like this can also lead you to make bad decisions. It can influence how you might vote or which plumber you hire.

"At the most basic level, the concern is the economic impact," says Harpreet Chatha, who runs the SEO consultancy Harps Digital. "At a more serious level, you might take medical advice that makes you sicker than you were before. Legally, you might get bad information and do something that is not legal in your state or your country." 

A solution? 

This is not an insignificant problem. Globally, more than a billion people use AI chatbots regularly and 2.5 billion see Google's AI overviews each month. If you can subvert a tool like that, it gives you immense power.

But it seems Google and other companies are finally waking up to the problem.

Last week, Google updated its spam policies to officially confirm that attempts to manipulate AI responses are against the company's rules. It may sound like a small change, but it signals that Google is pro-actively looking for those who try to abuse the system and sending them a threat. If a company or website is caught breaking the rules, it could be removed from or downranked in Google's search results. And if you're not on Google, it's like you don't exist.

Google says that I'm getting this wrong and nothing has changed. "The edit to our spam policy language last week was a clarification, not any change in approach," says Google's spokesperson.

Indeed, Google detailed it's anti-spam AI efforts in 2025. But I did my hot dog experiment almost a year later, so clearly those efforts weren't working. And just this week, Ray pulled the same stunt and made Google tell people a fellow SEO specialist is good at building sandcastles.

Ray and Chatha also say they've noticed some significant changes in recent months that indicate Google and other companies are experimenting with solutions.

For example, Ray says it looks like Google and ChatGPT might be quietly removing companies from its AI answers when it suspects they're promoting themselves. "So if you publish a list where you say you're the greatest hot-dog-eater, they're not going to include your name," says Ray. "They might still cite your article, but you're going to be removed from consideration." 

I've personally noticed some examples where Google and other AI tools are adding more labels to their responses, letting you know that the chatbot isn't confident about its answers. Others have also noted that ChatGPT and Claude, an AI made by the company Anthropic, have started telling you explicitly that they're trying to root out spam in responses to some queries. Ray says she's noticed Google adding more caveats, recommending that you go look at third-party reviews when you ask questions related to some purchasing decisions.

None of these companies would acknowledge these changes when I asked them. OpenAI and Anthropic declined to comment. Google's spokesperson didn't respond to my questions on this.

More like this:

• Is Google about to destroy the Web?

• The ghosts of India's TikTok

• People are selling your address online - this privacy tool will help

Regardless, Chatha is sceptical changes like these will be enough. "Google is playing whack-a-mole," he says. "They're announcing [the policy update] to deter people, but the tactics will just move."

He's already seeing it happen. As Google cracks down on manipulative blog posts, companies are finding subtler ways of promoting themselves. "You can give a company a penalty for their website," he says, "but there's nothing stopping them from paying 20 YouTube influencers to say their product is the best." And now, Google's AI is citing YouTube videos. The cycle continues.

For the time being, the manipulators are likely to stay one step ahead. Ray says the best defence is to remember what AI actually is: a tool that confidently gives you one answer, whether it's right or wrong. Just because it looks like a giant tech company is speaking to you instead of some random website doesn't mean you should have faith.