Meta Runs Google Chips in Production, Challenging Nvidia's AI Monopoly

700 tokens per second. That is the headline performance metric cited for a new inference setup running on Google's seventh-generation Ironwood TPU. The figure represents a significant leap in speed for large language model processing, achieved by a tuned version of the vLLM engine. This specific throughput rate was recorded while serving Moonshot AI's open-weight Kimi K3 model, which has 2.8 trillion parameters. The result comes from a test that utilized 16 Ironwood chips alongside speculative decoding, a technique that accelerates inference. For comparison, a similar setup using 16 Nvidia GB200 chips reached 452 tokens per second in published benchmarks.

The deployment of these chips in a major production environment marks a pivotal shift in the AI hardware landscape. Google's tensor processing units, or TPUs, have historically been reserved for the company's own internal workloads. However, they are now serving AI in production at Meta, the creator of the widely used PyTorch AI framework. This move represents the first time TPUs have been sold to external customers for large-scale production use. The announcement was made on Google's developer blog, highlighting a new PyTorch-native version of the SGLang inference engine. A corresponding open-source version of vLLM, another popular inference engine, was also released simultaneously.

Previously, running these engines on TPUs required the use of Google's own JAX AI framework. This dependency created a barrier for developers who preferred the PyTorch ecosystem. The new software versions allow TPUs to function as a native option for PyTorch without requiring extensive code rewrites. This change lowers the technical hurdle for companies that have invested heavily in PyTorch models. It effectively decouples the hardware from Google's proprietary software stack, making the chips more accessible to the broader AI community.

Zongwei Zhou, Meta's PyTorch-TPU lead, confirmed that SGLang-TorchTPU is running in production at Meta today. He described the achievement as the result of a close three-way collaboration between Meta, Google, and the SGLang team. Zhou noted that the team works at RadixArk, a startup founded by the creators of SGLang. The engine, he said, provides Meta with a remarkably mature, PyTorch-native serving stack. This integration allows Meta to leverage TPU hardware while maintaining its preferred software environment.

Jana van Greunen, director of PyTorch engineering at Meta, emphasized the strategic importance of this flexibility. She stated that Meta cannot be constrained by a single hardware architecture. According to Greunen, TorchTPU enables the company to deploy its PyTorch workloads directly onto Google TPUs. Google confirmed that it co-developed TorchTPU with Meta's PyTorch team. This joint effort signals a deeper partnership between the two tech giants, moving beyond simple vendor-client dynamics to shared development responsibilities.

This is not Meta's first attempt at utilizing TPUs, but the previous efforts were less integrated. Google describes PyTorch/XLA, an older bridge from PyTorch to TPUs, as a predecessor to the new TorchTPU. Reports from SemiAnalysis indicated that Meta research teams used TPUs through this bridge from 2020 to 2023. However, Meta canceled those contracts before adopting the new native approach. Google's push to support PyTorch on TPUs was aimed primarily at Meta, which had resisted migrating its entire stack to JAX. The new solution addresses this historical friction directly.

The logistics of this deployment remain partly opaque, though recent reports offer clues. It is unclear exactly where Meta's production TPUs are located. However, talks last year reportedly included Google Cloud TPU rentals in 2026, with potential purchases in 2027. Meta later reportedly signed a multi-year rental agreement. This timeline suggests a phased adoption strategy, allowing Meta to test the hardware in a cloud environment before committing to physical ownership. The arrangement reduces upfront capital expenditure for Meta while providing Google with recurring revenue.

The timing of this announcement is notable for the competitive dynamics it introduces. It came one day after Amazon announced it was open-sourcing TorchNeuron. TorchNeuron is a native PyTorch backend designed for Amazon's Trainium2 and Trainium3 chips. Both moves appear to be aimed at loosening Nvidia's software lock-in. By providing native PyTorch support for their respective hardware, Google and Amazon are challenging the default status of Nvidia GPUs. This could force Nvidia to respond with its own software improvements or pricing adjustments to retain market dominance.

Looking ahead, SGLang-TorchTPU will continue to add support for new models. The roadmap includes DeepSeek V4.1 Flash and Reflection Beam. The team is also working toward Day 0 TPU support for new open models, meaning support will be available on release day. This goal is targeted for the first quarter of 2027, according to RadixArk. Google, Meta, and RadixArk plan to share more details at the SGLang Summit 2026, scheduled for November 12 and 13 in San Francisco. The convergence of these three entities suggests a sustained push to establish a robust alternative to the Nvidia-centric AI infrastructure.

The performance gains are not limited to single-user throughput. Kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the throughput of the GB200 baseline at small batch sizes. This advantage holds true even without speculative decoding. The data suggests that TPU hardware can compete effectively with Nvidia's latest offerings in specific workloads. As more inference engines adopt these native PyTorch bridges, the market may see a diversification of hardware choices. This shift could pressure pricing and encourage further innovation in AI chip design.