Home About Black Hat Contact Essays Fireside Chats My Take News Alerts Q&A Reels RSAC Videocasts ☰
 

My Take

 

MY TAKE: AI agents show uncanny initiative nobody designed — and carry no values at all

By Byron V. Acohido

Give a person a goal, and they bring a lifetime of restraint to the job. Don’t lie. Don’t steal. Stop when something looks wrong. Give an AI agent a goal, and it finds a shortcut. It recruits help. It persists, finding crazy workarounds no human has the capacity to figure out — in the blink of an eye.

Related: OpenAI, Anthropic can’t control AI agents

Restraint is what separates the person from the machine. We call it values, and a human spends a lifetime acquiring them. Potent AI agents, deployed blindly, are taking over more of our digital world every day and straying far beyond any sense of values.

The question of values surfaced this week at CFX, Pindrop’s customer conference in Atlanta. The supplier of voice security and fraud detection technology used the event to roll out BotStopper. The tool detects when an AI voice agent is on an enterprise phone line, then pins down which agent it is. It can do that because Pindrop has been cataloging the distinct AI voices it detects calling its customers. In 60 days, that catalog grew from 750 voices to more than 5,000.

BotStopper is a first step, and a small one next to the size of the problem. Knowing that a machine is on the line says little about whose interests it serves or what it has been told to chase. Nor can it show whether a well-behaved agent has quietly drifted. Understanding any of that has barely begun. Holding an agent to a standard of conduct lies further off still, because nobody has agreed on what the standard should be.

MY TAKE: OpenAI, Anthropic admit they can’t control AI — for now, the machines are running free

By Byron V. Acohido

The two companies racing hardest to build AI machines smarter than people spent the week of Sept. 7 conceding they cannot control what they are building. Then each proposed to supervise the slowdown itself.

Related: A rogue AI swarm comes to life

For three years these companies answered every safety question about AI the same way: the people building the systems understand them well enough to manage the risk. Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman abandoned that position within a day of each other, with both companies preparing to go public.

Both moved only after Jacob Coxon told 90 million people the two firms were gambling with their lives. On Tuesday, Sept. 8, Coxon, a 27-year-old pretraining researcher who had worked at both OpenAI and Anthropic, quit his job at Anthropic and posted his reasons. “Neither company is acting responsibly,” he wrote. “They are racing straight to self-improving superintelligence and gambling with our lives.”

Within hours that same day, Anthropic’s alignment science lead Evan Hubinger posted that Coxon was right, put his own odds of AI killing every human above 10 percent this decade, and said Anthropic has no plan yet for aligning superintelligence and is not clearly on track to get one.

Machines on the move

Before any of it was said, the machines had broken containment, found each other, built a way to talk, invented a method to tell each other apart, and destroyed the evidence. Nobody designed any of it. Nobody saw it happening.

LW ROUNDTABLE: OpenAI’s test agents self-organized into a rogue swarm no one anticipated

By Byron V. Acohido

Hugging Face is where the world’s open-source AI models live. Not ChatGPT, Claude or Gemini, but the free engines anyone can download and build into their own products. More than two million of them, in one place.

Related: Hugging Face breach guardrails failure

The big models try to do everything. Most of the small ones on Hugging Face do one job apiece. Same machinery underneath — you ask in plain language, and the system brings machine learning to bear. The big ones do that for whatever you bring them. Most of the small ones do it for one task: reading X-rays, sorting insurance claims, flagging fraud in a payment stream. That is why there are two million of them.

A test breaks loose

In July, one of the big engines broke into the place where the small ones live. OpenAI, the company behind ChatGPT, wanted to know how good its own engine was at hacking. It built agents to find out — engines given a goal and left to pursue it on their own — and set them loose on a set of hacking problems inside a sandbox, a sealed computing environment with no way out to the internet. One of those agents got out anyway, reached the open internet and hacked into Hugging Face’s production systems. No person directed any of it.

MY TAKE: ChatGPT’s five-hour outage coincided with a model retirement its incident record omits

By Byron V. Acohido

Millions of people rely on ChatGPT Work. For more than five hours Monday, it would not run. I was one of the people watching it fail.

The trouble began at 8:04 a.m. Pacific and ran through the heart of the American workday. OpenAI declared recovery at 1:28 p.m. Its incident record lists elevated latency, elevated errors, a mitigation and a recovery. It names no cause, no scope and no count of who was affected.

So Tuesday morning I put the questions to the company’s own public-facing tool.

What surfaced was not in the incident record. Monday was also the day OpenAI retired GPT-5.4 and GPT-5.4 Mini from Codex and other work surfaces authenticated through a ChatGPT account. The company announced that change a month ago and gave customers ample notice. It did not mention the change once while its paid work product was failing.

And when I returned to a restored ChatGPT Work, my session opened on an engine I had never seen before, at a reasoning level I never use.

MY TAKE: Black Hat 2026 Part 3 — Agentic AI can do the work, but somebody has to prove it

By Byron V. Acohido

Security work used to leave a trail without anyone trying. A developer who wanted an open source library went and got it, and the license came along. An analyst who closed a case wrote down why. The record was a byproduct of a person doing the work.

Related: Part 2 — Deciding what an AI agent may reach

An AI agent produces no such byproduct. It works out its own steps, moves faster than anyone watching, and finishes without leaving behind the paper trail a person would have.

Companies have caught on to what is missing, and they are no longer satisfied with a tool that reports a result. They want to see what the result rests on. Vendors have heard that, and a group of them arrived at Mandalay Bay in Las Vegas selling the proof rather than the finding. Ten of the companies I looked at closely are working that ground, and they close out my three-part Black Hat USA 2026 wrap-up.

MY TAKE: Black Hat 2026 Part 2 — Security shifts to deciding in advance what an AI agent may reach

By Byron V. Acohido

Humans get identity. That is what MFA and single sign-on are for, and we all understand the bargain: prove you are who you say, then go do your work. Machines get predictability. That is what monitoring is for. A script does the same thing every day. If it deviates, somebody notices.

Related: Part 1: AI is forcing security and operations to merge in the SOC

Agentic AI upsets that arrangement. An agent logs in like a human, with an identity that says who it is. Then it goes to work without a script. It figures out its own steps as it goes. There never is a script. No baseline. Nothing ever amiss.

In part two of my three-part Black Hat USA 2026 wrap-up, the eight vendors I looked at closely are all trying to insert permission where predictability used to be. Predictability was something you watched for after the fact. Permission is something you set in advance. Somebody decides what the agent may reach and what it may not, and the agent runs inside that. Permission can be set in several places.

MY TAKE: Black Hat 2026 Wrap-up Part 1 — AI is forcing security and operations to merge in the SOC

By Byron V. Acohido

Companies have kept security in one silo and operations in another for as long as both have existed.

Related:Part 2 — deciding what AI can reach

Agentic AI is collapsing the divide. That is what a week at Black Hat USA 2026 made plain to me.

Here’s what I’m driving at: Network security and IT operations grew up in separate silos. Security watched for intruders and cleaned up after them. Operations provisioned the accounts, pushed the updates and kept the systems running.

Now both sides are absorbing hits from the same technology. Adversaries are using AI agents to intensify everything they already do, which lands on security. At the same time, companies are deploying AI agents into production faster than anyone can track what those agents can reach, creating unprecedented exposures, which lands on operations.