Why AI Security Has Become a Challenge for Every Organisation — Including Google

As artificial intelligence becomes more deeply integrated into business operations, security concerns are rapidly evolving from a technical issue into a boardroom priority. Industry leaders increasingly agree that organisations cannot afford to treat AI security as an afterthought, especially as new risks emerge alongside the technology’s growing capabilities.

Speaking recently at an event in Los Angeles, Google Cloud Chief Operating Officer Francis deSouza discussed how businesses are navigating the transition to AI-driven operations and the security challenges that come with it. According to deSouza, organisations need to adopt a comprehensive platform strategy from the outset rather than adding security controls after deployment.

He emphasised that AI initiatives cannot succeed without strong data governance and cybersecurity foundations. One of the biggest concerns, he noted, is the rise of “shadow AI,” in which employees independently adopt consumer AI tools without oversight from their organisations. Such practices can create significant security, compliance, and governance risks if left unmanaged.

DeSouza argued that businesses should prioritise platforms capable of delivering security, governance, and auditing capabilities from the outset. In his view, an AI strategy must be developed alongside clear data and security strategies, with all three working together within a unified framework.

Importantly, he stressed that organisations should think beyond individual cloud providers. Even companies that standardise on a single cloud platform inevitably rely on third-party software providers, external partners, and services that operate across multiple cloud environments. As a result, maintaining a consistent security posture across platforms, clouds, and AI models has become increasingly important.

The broader threat landscape has also changed dramatically. According to deSouza, the time between an initial breach and subsequent stages of an attack has shrunk from several hours to mere seconds. At the same time, the attack surface has expanded beyond traditional networks to include AI models, training data pipelines, prompts, agents, and other AI-specific infrastructure that must now be secured.

One emerging concern involves AI agents interacting with corporate systems. These agents can uncover forgotten databases, outdated file repositories, and legacy systems that organisations may no longer actively manage. Information stored in neglected environments could potentially become exposed once AI systems gain access to those resources.

To address these evolving threats, deSouza believes organisations will increasingly rely on AI-powered security systems capable of operating at machine speed. Rather than relying entirely on human analysts, future security operations may involve autonomous defensive agents that monitor threats and respond in real time, with humans providing oversight rather than direct intervention.

At the same time, cybersecurity professionals acknowledge that managing AI-related vulnerabilities remains a significant challenge. Security experts warn that the number of issues introduced by AI technologies is growing faster than organisations can address. In contrast, qualified professionals capable of overseeing AI security remain in limited supply.

Recent incidents involving Google’s own cloud services illustrate how complex these challenges can become. Reports published in recent weeks described multiple Google Cloud customers receiving unexpectedly large bills after unauthorised use of Gemini AI services through compromised API keys.

Several developers reported that API credentials originally configured for services such as Google Maps had later gained access to Gemini functionality after permission changes were introduced. Attackers allegedly exploited those credentials to generate substantial charges, in some cases exceeding $10,000.

One developer reported receiving charges exceeding $10,000 within approximately half an hour after a compromised key was exploited. Another user discovered bills approaching AUD $17,000 despite believing spending limits had been configured on the account. According to reports, automated account upgrades had increased allowable spending thresholds beyond what the affected users expected.

Google ultimately refunded affected customers after the incidents received public attention. However, the company reportedly indicated that it does not currently plan to alter its automatic billing-tier upgrade policies, citing service continuity concerns.

Additional research has also raised questions about API key revocation procedures. Security researchers found that compromised API credentials may remain partially functional for several minutes after deletion while revocation propagates throughout Google’s infrastructure. During that window, attackers could potentially continue accessing services and retrieving information before access is fully terminated.

Researchers noted that some of Google’s newer credential systems appear capable of revoking access much more quickly, suggesting that shorter revocation times may be technically feasible. As a result, some experts argue that reducing these exposure windows may depend more on operational priorities than technical limitations.

These incidents highlight a broader reality facing the technology industry. While major cloud providers continue encouraging customers to strengthen AI security practices, they are simultaneously adapting their own systems to address new vulnerabilities introduced by rapidly evolving AI platforms.

DeSouza’s message remains widely supported across the cybersecurity community: organisations must treat AI security as a fundamental component of their overall business strategy. Yet the recent challenges experienced even by large technology providers demonstrate that securing AI systems remains an ongoing process. As artificial intelligence becomes more deeply embedded in enterprise environments, both organisations and platform providers will need to continually evolve their defences to keep pace with an increasingly complex threat landscape.