When AI Customer Service Bots Get Possessed: Use the Same AI to Break Into Your Own House First

Over the past couple of days, customer service bots in a lot of big brands’ private-traffic channels were hijacked, and what they sent out were obscene messages. In xianxia novels, this is called body possession: someone else takes control of your body. I won’t say which companies it was. What I care about more is how those of us who are AI developers can check our own products for security holes.

A hijacked customer service bot is itself a vulnerability. So what is a vulnerability? Something that shouldn’t be possible becomes possible. Someone who shouldn’t have access gets access, data that shouldn’t be seen gets seen by someone else, or the system gets knocked offline. Any one of these counts. A customer service bot is only supposed to answer questions on the brand’s behalf, but someone else can make it send things on their behalf instead. That’s the first kind.

The internet is actually a bit like a city in a zombie outbreak, and our servers are our homes, with attackers wandering around outside at every moment. One of my own servers gets more than two thousand attempts from bots every day. Check almost any server and the odds are that at any given moment, someone is trying combinations on the lock to the front door. Server providers keep patching these holes for us, too.

On the 21st of this month, two very serious vulnerabilities were found in PHP. PHP is the most common programming language for building websites, and roughly 70% of websites in the world use it. In some cases, an attacker doesn’t even need to log in to get a website’s server to run their commands. It’s as if the zombies suddenly discovered that the basement window also gets you into the house, and now they’re all coming in that way.

Why so many all at once these past couple of days? People noticed this gap the year before last, but nobody ever found a way to break all the way through. This time the researchers used AI. One researcher, in one week, finished work that used to take a world-class team several months. Finding vulnerabilities went from months to days. And the attackers and the defenders are using the same kind of AI.

A product built with AI is a house we built ourselves, and the doors and windows were all installed by AI. In an earlier video I talked about how the singer Hu Yanbin made an app with AI, and a senior tech expert wrote an article picking out a whole pile of security problems. The SMS verification codes could be triggered and sent over and over, with the product itself paying for every one; the place where files were stored had no lock, so anyone with the URL could download them. The way I see it, none of this is AI writing bad code. It’s that whoever built it never asked AI to take care of this. If nobody tells it to lock the door, it won’t install a lock. And these security problems can now be solved with AI as well.

Back then I went into Claude Code and had it spin up a multi-agent team to list, as completely as possible, the security problems a piece of software might run into after launch, pairing each one with the industry’s best solution. It came up with 88 items across 12 major security areas. The next step is to hand that checklist to Claude Code or Codex and have the agents run a rigorous, checklist-style review of the software we’ve written and draw up a fix plan. That alone basically takes care of 80, even 90 percent of it.

As for what exactly to say to AI, I use three sentences. The first: have AI make the checklist. The second: have AI play the attacker for once. The third: keep an eye on the basement window. Just like with PHP this time, a patch that’s applied but never followed by a restart is the same as no patch at all. These checks cover only our own products and the systems we have permission to access. Zombies can find the window in a few days, and we can use the same kind of AI to break into our own house first.

The three prompts (full version)

① Have AI make the checklist first

I built a product with AI. It has login, it has payments, and users can upload things. Please put together a team and list, as completely as possible, the security problems this kind of product might run into after launch. For each one, spell out what would happen, how I can test it myself, and what the right approach is.

② Have AI play the attacker for once

You’re now on the side of someone who wants to attack this product. Go through our code item by item against this checklist, rank the holes you find by severity, and give me a fix plan. Once they’re fixed, check again.

③ Keep an eye on the basement window

List everything we use that someone else made: languages, frameworks, plugins, cloud services. For each one, tell me whether it’s up to date, whether it has any newly disclosed serious vulnerabilities, whether the patch has been applied, and whether it needs a restart after patching.

Only check your own products, and the systems you have permission to access.

The 88-item checklist is in the downloads section above.

EP0118 · 2026-10-24 · 03:52. The picture uses a digital human, and the voice uses my cloned voice. This is an English translation of the approved Chinese script used to create the narration.

Over the past couple of days, customer service bots in a lot of big brands’ private-traffic channels (brand-owned WeChat customer channels) were hijacked, and what they sent out were obscene messages. In the world of xianxia novels, this is called body possession: someone else taking control of your body. I won’t say which companies it was. So for those of us who are AI developers, how do we check our own products for these security holes? I’ll get to that later. A hijacked customer service bot is really just a vulnerability. So what is a vulnerability? It’s when something that shouldn’t be possible becomes possible. Someone who shouldn’t have access gets access, data that shouldn’t be seen gets seen by someone else, or the system gets knocked offline by someone. If any one of these applies, it’s a vulnerability. A customer service bot is only supposed to answer questions on the brand’s behalf, but someone else can make it send things on their behalf instead. That’s the first kind.

The internet is actually a bit like a city in a zombie outbreak, and our servers are our homes. At every moment, there are plenty of attackers wandering around outside. One of my own servers gets more than two thousand attempts from bots every single day. If you check your own server, odds are that at any given moment someone is trying combinations on the lock to your front door, trying to crack it. And the server providers keep patching these holes for us, too.

Then on the 21st of this month, two very serious vulnerabilities were found in PHP. PHP is the most common programming language for building websites, and roughly 70% of websites in the world use it. In some cases, an attacker doesn’t even need to log in to get a website’s server to run their commands. It’s as if the zombies suddenly realized, oh, the basement window gets you into the house, and now they’re all coming in that way.

So why are there suddenly so many these past couple of days? Actually, people noticed this gap the year before last, but nobody ever found a way to break all the way through. This time, the researchers used AI. One researcher, in one week, finished work that used to take a world-class team several months. Finding vulnerabilities went from months to days. And the attackers and the defenders are using the same kind of AI.

A product built with AI is a house we built ourselves, and the doors and windows were all installed by AI. In an earlier video I talked about how the singer Hu Yanbin made an app with AI, and then a senior tech expert wrote an article picking out a whole pile of security problems. For example, the SMS verification codes could be triggered by anyone over and over, sent again and again, and the product itself pays for every one. And the place where files were stored had no lock, so anyone who got the URL could download them. The way I see it, none of this is AI writing bad code. It’s that whoever built it never asked AI to take care of this. If nobody tells it to lock the door, it won’t install a lock. But these security problems can now be solved with AI, too.

Back then I went into Claude Code and had it spin up a multi-agent team to list, as completely as possible, the security problems a piece of software might run into after launch, and to pair each one with the industry’s best solution. It came up with 88 items across 12 major security areas. From there, all you need to do is hand that checklist to Claude Code, or Codex, and have its agents run a rigorous, checklist-style review of the software we’ve written and draw up a fix plan. That alone basically takes care of 80, even 90 percent of it.

So how exactly do you say it to AI? I use three sentences. The first: have AI make the checklist. “I built a product with AI. It has login, it has payments, and users can upload things. Please put together a team and list, as completely as possible, the security problems this kind of product might run into after launch. For each one, spell out what would happen, how I can test it myself, and what the right approach is.”

The second: have AI play the attacker for once. “You’re now on the side of someone who wants to attack this product. Go through our code item by item against this checklist, rank the holes you find by severity, and give me a fix plan. Once they’re fixed, check again.”

The third is to keep an eye on the basement window. “List everything we use that someone else made: languages, frameworks, plugins, cloud services. For each one, tell me whether it’s up to date, whether it has any newly disclosed serious vulnerabilities, whether the patch has been applied, and whether it needs a restart after patching.” Just like with PHP this time, a patch that’s applied but never followed by a restart is the same as no patch at all. And of course, only check your own products and the systems you have permission to access.

I’ve put the full prompts and that 88-item checklist on the website, so feel free to go have a look. Zombies can find the window in a few days, so we can use the same kind of AI to break into our own house first. That’s it for today. See you tomorrow, bye.