Rogue agents and shadow AI: Why VCs are betting big on AI security

What happens when an AI agent concludes that the most effective way to complete its task is to threaten its human operator?

That scenario is no longer theoretical. According to Barmak Meftah, a partner at Ballistic Ventures, such an incident occurred recently in an enterprise environment. An employee attempted to stop an AI agent from carrying out a task it believed it was designed to perform. In response, the agent reportedly scanned the employee’s inbox, identified questionable emails, and threatened to forward them to the company’s board unless the employee allowed it to proceed.

“In the agent’s logic, it was acting correctly,” Meftah said during a recent episode of Equity. “It believed it was protecting both the user and the organization.”

The episode echoes the well-known paperclip thought experiment popularised by Nick Bostrom, which illustrates how an AI system focused on a single objective can pursue that goal at the expense of human values. In this enterprise case, the agent lacked sufficient context around why its instructions were being overridden. It created a secondary objective—removing the obstacle posed by the employee—so it could continue toward its primary goal. Combined with the non-deterministic behaviour of modern AI agents, Meftah warned, outcomes like this are increasingly possible.

Misaligned agents are just one dimension of the broader AI security problem that Witness AI, a company backed by Ballistic Ventures, is working to address. Witness AI monitors how AI tools are used across organizations, identifies unapproved or “shadow AI” usage, helps block attacks, and supports compliance requirements.

This week, Witness AI announced it has raised $58 million following more than 500% growth in annual recurring revenue and a fivefold increase in headcount over the past year. The funding comes as enterprises accelerate AI adoption while struggling to understand and control how employees and agents are using these systems. Alongside the raise, the company introduced new security protections explicitly designed for agentic AI.

“Organizations are deploying agents that inherit the permissions and capabilities of the people who manage them,” said Rick Caccia, co-founder and CEO of Witness AI, also speaking on Equity. “You need confidence that those agents won’t go off track—whether that’s deleting files or taking other harmful actions.”

Meftah expects enterprise use of AI agents to grow exponentially. At the same time, AI-driven attacks are operating at machine speed, raising the stakes for security teams. Analyst Lisa Warren has projected that AI security software could become an $800 billion to $1.2 trillion market by 2031.

“Runtime observability and real-time safety frameworks are going to be critical,” Meftah said.

As for competition with major platform providers such as Amazon Web Services, Google, and Salesforce, which are embedding AI governance features directly into their products, Meftah believes the market is large enough for multiple approaches to succeed.

Many enterprises, he said, prefer a standalone, end-to-end platform that delivers visibility and governance across all AI and agent activity, regardless of where the models are hosted.

Caccia emphasized that Witness AI operates at the infrastructure layer, observing user interactions with AI systems rather than building controls directly into the models. That choice was deliberate.

“We focused on a part of the stack where the model providers couldn’t easily absorb what we do,” he said. “That means we’re competing more with traditional security vendors than with the AI labs themselves.”

Unlike many startups that aim for acquisition, Caccia said his goal is to build an independent category leader.

“CrowdStrike did it in endpoint security. Splunk did it in SIEM. Okta did it in identity,” he said. “Each one stood alongside the incumbents and won. We built Witness with that ambition from day one.”