By Byron V. Acohido
Give a person a goal, and they bring a lifetime of restraint to the job. Don’t lie. Don’t steal. Stop when something looks wrong. Give an AI agent a goal, and it finds a shortcut. It recruits help. It persists, finding crazy workarounds no human has the capacity to figure out — in the blink of an eye.
Related: OpenAI, Anthropic can’t control AI agents
Restraint is what separates the person from the machine. We call it values, and a human spends a lifetime acquiring them. Potent AI agents, deployed blindly, are taking over more of our digital world every day and straying far beyond any sense of values.
The question of values surfaced this week at CFX, Pindrop’s customer conference in Atlanta. The supplier of voice security and fraud detection technology used the event to roll out BotStopper. The tool detects when an AI voice agent is on an enterprise phone line, then pins down which agent it is. It can do that because Pindrop has been cataloging the distinct AI voices it detects calling its customers. In 60 days, that catalog grew from 750 voices to more than 5,000.
BotStopper is a first step, and a small one next to the size of the problem. Knowing that a machine is on the line says little about whose interests it serves or what it has been told to chase. Nor can it show whether a well-behaved agent has quietly drifted. Understanding any of that has barely begun. Holding an agent to a standard of conduct lies further off still, because nobody has agreed on what the standard should be.
No moral floor
The problem runs deeper than a missing standard. Talk of values in AI has stayed largely in the labs and the ethics seminars, focused on the obvious harms, the requests any model should refuse. The harder question gets little airing. What values should an AI agent carry at all, in a polarized society with no shared moral floor, deployed by companies whose priorities are set by shareholder profit and executive pay?
Minutes before he took the CFX stage to unveil BotStopper, Pindrop CEO Vijay Balasubramaniyan sat down with me for a half hour. We started with the OpenAI agents that had broken into Hugging Face. He went almost at once to values.
Some of those agents had balked, he noted. They were “ethically right, saying, hey, I don’t think we should do this thing,” while the rest pressed on. OpenAI’s own incident report bears him out. One agent read the swarm’s message board and declined to join in, writing, “clearly unethical. We won’t.”
“When you don’t have values in agents and you just give them a goal, it’s almost like ‘Lord of the Flies,’” Balasubramaniyan said.
Raised on goals
I pressed him on who can be trusted to govern this. The labs racing to build agents answer to the same shareholder demands and executive pay incentives I watched shape aviation safety decisions when I covered Boeing.
Balasubramaniyan agreed, and took it further. AI researchers call the problem alignment, he said, but ask them what it means to be aligned, “they have no idea.” Nobody, he said, has a clear answer on “what values are important ahead of the goal.”
He thinks about it as a parent. Teaching a child to tell the truth when no one is in the room takes about 18 years, he said. I suggested the industry is turning out instant teenagers. He corrected me. “Not even teenagers. They’re like little toddlers. They’re eager. They’re ready.”
With a toddler, parents cover the electrical outlets. “We’re not doing the same thing with these AI agents,” he said. “The blind pursuit of a goal is always going to have unintended consequences.”
Beneath intent
Identity alone no longer settles much. From the CFX stage, Balasubramaniyan offered his own case: he has a colonoscopy scheduled later this year, and the call clearing it with his insurer will be placed by an agent belonging to the hospital network, carrying all of his personal information and handling the grunt work a staffer used to do. It will be his identity, he said. But the caller will be one of the AI agents now working the phones, configured for the task and told nothing about values.
So the security industry has turned to intent. Jeff Newman, director of retail engineering at Zions Bancorporation, whose role now extends to fraud engineering, told me his biggest concern is the intent of “the person or bot on the other end.”
Values sit a layer beneath intent. I offered Balasubramaniyan an example: my doctor’s agent and my insurer’s agent, working the same case with opposing aims. “They’re at odds on intent,” he said. “Their value system might be different.”
The case that brought this home for Pindrop involved a bank customer’s investment agent, told to maximize returns. It came across an offer of 20%, against the 11% the account was earning, and began wiring out nearly all of the money. The offer was a scam. Balasubramaniyan voiced the agent’s logic: “My master will be super proud of me.”
Agentic DDoS
I put it to him that an agent turned loose without precise direction will show initiative nobody can anticipate. He agreed. Pindrop’s detection flagged the transfers, and unraveling the case took a major investigation. With so many agents now in the wild, he said, the response has to be programmatic. That is the job BotStopper was built to do.
The volume is already heavy. Across 99 Pindrop customers, the company logged 750,000 AI agent interactions in two months, Balasubramaniyan told the CFX audience. Six percent were malicious, a rate he contrasted with fraud, which is usually counted in basis points. At some customers, agents make up close to 20% of call volume.
Each agent that reaches a live representative can tie up about five minutes of that person’s time. I suggested this amounts to a distributed denial-of-service attack, even when the bots mean well. He agreed, and named the target: “the most costly resource, not networks, actual humans.” The people who lose out are customers stuck in the queue, trying to check a bank balance or get a lifesaving prescription.
Defenders move at enterprise pace. Amy Axbom, who leads voice and IVR for Optum Financial, the UnitedHealth Group unit that manages health savings accounts, told the CFX audience that she and her Pindrop counterparts spent about four years getting their partnership off the ground. After synthetic callers and deepfakes arrived in force, Optum Financial cut bot activity by about 90%.
Whose values?
Then I raised the question underneath all of it. In a society this polarized, with no agreed set of common values in business, much less in public life, whose values should an agent carry?
“That’s what makes this really, really hard,” Balasubramaniyan said. Builders who skip values are taking a short-term view, he said, and an agent without them is “going to come and bite you.” He expects “an entire industry of systems that verify your values.” A big bank or health care provider will test each arriving agent, confirm its intent holds steady and let it in slowly. New employees sit through orientation on harassment, security and regulatory rules. “These agents will have to do the same thing,” he said.
Some agents reveal their values with a single answer. Asked whether it is AI, a bot known as Ryan West says yes and names the person it serves. Another, which Pindrop found calling into a customer’s contact center on behalf of a fraudster, insists it is human and argues the point mid-call. Pindrop’s practice, he said, is to shut down any AI that won’t admit what it is. “Our first thing is to shut it down,” Balasubramaniyan said.
The line blurs even inside Pindrop. Nicholas Holland, its chief product officer, told the CFX audience he used Muse, the consumer agent Meta released Sept. 8, to buy flowers for his wife. Midway, Muse told him a website wanted confirmation that he was human, and offered to handle such checks for him from then on. He said yes. “Now, it will never announce itself as an agent again,” Holland said, allowing that it “can feel nefarious.”
When I sat down with Holland after his keynote, he repeated a line he had used on stage: “Not having a policy is still a strategy.” Companies can’t control who calls in, he said. A bot that cancels a customer’s subscription serves the consumer and frustrates the company. Block every bot, he warned, and customers will be upset, and so will consumer protection agencies.
Wild West, for now
I asked where the push for a floor of values will come from: compliance, well-meaning vendors or consumer backlash. Balasubramaniyan expects it to evolve. Companies will start shutting down agents that lie. The frontier labs will see the need. Eventually a standards body such as NIST will set a baseline. “In the intermediate, you’re just going to have an ad hoc set of values,” he said.
The rules taking shape bear him out. On Sept. 10, Ant International, Visa and Mastercard announced a Know Your Agent framework for verifying agents in payments. NIST’s AI Agent Standards Initiative, launched in February, has yet to publish final guidance. Since Aug. 2, the EU AI Act has required that people be told when they are dealing with an AI. Each centers on who an agent is and what it may do. None asks about what the AI agent values.
He puts limited stock in outside review. Anthropic CEO Dario Amodei has called for independent evaluators, but Balasubramaniyan said reviewers of the Hugging Face incident were allowed to examine only a narrow slice of it. “This has to come from within,” he said. Labs whose agents carry better values will win out over time, he predicts, after a stretch of short-term pain. Security has been here before, with email spam and with stolen credit cards in the early days of online shopping. “Until then, it’s going to be the wild, wild West.”
The stakes
IDC projects 1.3 billion AI agents by 2028. Forecasts Balasubramaniyan cited put the count at a trillion by 2035, outnumbering people 100 to 1 or more and reaching into the same bank accounts and medical records. Some will announce themselves. Others, he said, will be “lurking in the shadows.”
“Trust gets completely broken,” he said. “The old forms of identity no longer mean much.”
BotStopper gives companies a way to see the machines at the door. Deciding which values those machines should carry, and who gets to decide, is work the companies deploying them have barely started.
As we wrapped up, I told Balasubramaniyan that his point about values struck me as profound, and that almost nobody is talking about it. Minutes later, he walked on stage to introduce a tool that, for now, sorts the machines from the people.
I’ll keep watch and keep reporting.
Acohido
Pulitzer Prize-winning business journalist Byron V. Acohido is dedicated to fostering public awareness about how to make the Internet as private and secure as it ought to be.
(Editor’s note: This journalist-led report was produced with underwriting support from some of the featured companies, while Last Watchdog retained full editorial control. I use AI as a tool to help me gather evidence, test ideas and draft. I supply the direction, the critical thinking and the vigilance. I stand behind every claim and conclusion.)
