Scared of AI, but Not the Skynet Kind

Lately I’ve been a little scared of AI. Not the Skynet kind of scared, where the machines wake up one day and decide to run the world. Mine is a much smaller fear.

I don’t actually know a lot of basic tech stuff, so there are probably caveats I don’t even know exist. Something I build with AI might look totally fine and still be full of security problems inside. That’s why I think AI security courses should be taught way more widely.

I said all this to Kally on Slack around lunchtime today, and Kally’s first reply was that there were plenty of security problems before AI too XD

Fair enough. But there must be way more now, right? AI makes everything so accessible.

That’s when Kally got serious. According to Kally, it comes down to how strong each side is, the red team that attacks versus the blue team that defends, and right now the red team is clearly ahead. GLM 5.3 can already hunt for vulnerabilities completely on its own.

Before AI, attackers mostly had two ways in. The first was to read through the code of an open source project, find a hole, and then use search engines like Shodan and Censys to find every site running that project and attack them all in one go. The second was to pick one big target and scan its IP ranges and subdomains for a service that would let them in.

With AI, reading code and scanning services happen on a completely different scale. Plenty of companies and developers haven’t caught up yet, either. The risk was always there. It’s just that for most targets, breaking in used to be a terrible return on investment for the attacker.

I warned Kally that a long message was coming, and then I typed for twelve minutes XD

I agreed with Kally completely. But since I know nothing about tech, my take came from the social science side.

There’s an idea in psychology called means restriction. Some suicide and crime rates rise and fall with how easy it is to get hold of the means. In some towns, the gas piped into homes used to be full of carbon monoxide, and suicide rates there were higher simply because it was so easy to reach. Gun ownership is linked to suicide and crime rates in the same way. Make the means harder to reach, and the rates come down. My favorite writer, Malcolm Gladwell, comes back to this idea again and again in his books, so Kally, go read him if you’re curious.

People are a little opportunistic, which is exactly what Kally meant by return on investment. If something is affordable and the opportunity is right there, people will use it, even for things it was never designed to do. Finding a way into a system used to take social engineering or a long chain of steps. Now AI can point straight at the way in.

So my conclusion was that Kally and I were saying the same thing all along. I still don’t know why I needed that many words to say it XD