Google Cloud unveils two new AI chips to rival Nvidia

Google Cloud on Wednesday announced that its eighth generation of custom-built artificial intelligence chips, known as tensor processing units (TPUs), will now be split into two separate versions. One chip, called the TPU 8t, is designed for model training, while the other, the TPU 8i, is focused on inference tasks.

Inference refers to the ongoing use of AI models, meaning what happens after users submit prompts and the system generates responses.

As expected, Google highlighted strong performance improvements over previous generations of its chips, including up to 3x faster AI model training, around 80% better performance per dollar, and the ability to scale to more than 1 million TPUs working together in a single cluster. The company says this should enable significantly more computing power at lower energy consumption and reduced cost for customers compared to earlier versions. Google refers to these processors as TPUs rather than GPUs, since its custom low-power chips were originally branded as Tensor Processing Units.

However, Google’s latest chips are not positioned as a direct replacement for Nvidia’s technology—at least not yet. Similar to other major cloud providers such as Microsoft and Amazon, Google is using its own chips to complement Nvidia-based infrastructure rather than fully replacing it. The company has also confirmed that Nvidia’s latest generation chip, Vera Rubin, will be available on Google Cloud later this year.

Over time, hyperscalers developing in-house AI chips—including Amazon, Microsoft, and Google—could potentially reduce reliance on Nvidia as enterprises shift more AI workloads to cloud platforms and adapt their applications to run on these custom processors.

Despite this, Nvidia remains dominant in the current market. Chip industry analyst Patrick Moorhead previously noted on social media that he had forecast potential competitive pressure on Nvidia and Intel from Google’s TPUs back in 2016 when the first TPU was introduced. Since then, Nvidia has grown into a company with a nearly $5 trillion market capitalisation, underscoring that the prediction did not play out as expected at the time.

In practice, Nvidia may still benefit from the broader expansion of AI infrastructure. If Google’s cloud business continues to grow, it could actually increase demand for Nvidia’s hardware, even as some workloads run on Google’s own chips.

Google also announced that it is collaborating with Nvidia to improve networking performance for Nvidia-based systems running on Google Cloud. The partnership focuses on enhancing the software-defined networking technology known as Falcon, which Google originally developed and open-sourced in 2023 as part of the Open Compute Project, a major open-source initiative for data centre hardware.