Google Cloud unveils TPU 8t and 8i chips to triple AI training performance

Google Cloud has unveiled its eighth-generation Tensor Processing Units (TPUs), introducing two specialized AI chips—TPU 8t for massive model training and TPU 8i for high-speed inference—to power the growing demands of autonomous AI agents and complex workloads. Announced at Google Cloud Next, these chips aim to deliver superior performance and efficiency, positioning Google to challenge Nvidia's dominance in the AI hardware market without fully abandoning the industry leader. According to TechCrunch, the new TPUs are up to three times faster for AI model training than previous versions, with 80% better performance per dollar and the capability to scale to over one million chips in a single cluster.

The TPU 8t stands out as a training powerhouse, optimized for trillion-parameter frontier models like those behind Google's Gemini. It boasts 121 exaflops of native FP4 compute power—using 4-bit floating point precision to slash memory bandwidth bottlenecks—and two petabytes of shared high-bandwidth memory within a 9,600-chip superpod, marking a threefold peak performance increase over the prior generation. Google's technical deep dive highlights how this design eliminates data stalls through innovations like integrated storage with 10 terabytes per second throughput and direct-to-chip access, all while incorporating Arm-based Axion CPUs to remove host bottlenecks. Meanwhile, the TPU 8i targets inference, the phase where trained models respond to user prompts in real time, featuring three times more on-chip SRAM for larger key-value caches and a new Collectives Acceleration Engine to cut latency during long-context tasks essential for AI agents that reason, plan, and execute multi-step workflows.

This launch underscores a critical shift in AI infrastructure, where electricity and compute resources are rationed amid explosive demand from frontier labs. As VentureBeat reports, most AI developers pay Nvidia's "steep gross margins" for graphics processing units (GPUs), fueling the chipmaker's trillion-dollar valuation, but Google sidesteps this "Nvidia tax" with its in-house TPUs. The chips promise lower energy use and costs for customers, enabling more compute for tasks like agentic AI—autonomous systems that act on users' behalf—while maintaining compatibility with Google's AI Hypercomputer architecture. TechCrunch notes that Google still offers Nvidia hardware in its cloud for now, blending custom silicon with ecosystem flexibility.

The stakes are high for cloud providers and AI developers worldwide. These TPUs could accelerate innovation in fields like drug discovery, climate modeling, and personalized services by making massive AI training and deployment more accessible and sustainable. Enterprises reliant on Google Cloud, from startups to hyperscalers, stand to benefit from reduced costs and faster agent performance, potentially leveling the playing field against Nvidia-dependent rivals. General availability is slated for later this year, with developers able to request details now to prepare. As AI workloads evolve toward multi-agent collaboration, Google's specialized hardware signals a broader industry trend: purpose-built chips over one-size-fits-all solutions, promising efficiency gains that could reshape compute economics.