Google and Meta have developed TorchTPU, an open-source PyTorch backend for Google TPUs. The project is designed to run existing PyTorch workloads with limited code changes, supporting eager execution, compiled execution and distributed computing. It is still at an early stage.
Google has supported PyTorch on TPUs through PyTorch/XLA since 2020. According to Google, that approach requires explicit changes to define compilable regions, can incur substantial compilation delays, does not support PyTorch’s native eager execution mode and has lagged in support for newer distributed-computing features.
For workloads moving from GPUs to TPUs, the changes described for TorchTPU include replacing the cuda device with tpu and the nccl distributed backend with tpu_dist. Developers can also use torch.accelerator.current_accelerator to identify an available accelerator.
TorchTPU offers three eager-execution modes: Strict, Debug and DeferAndFuse. The last combines eager and compiled execution. Google claims DeferAndFuse can run more than 50% faster than Strict Eager while reducing peak high-bandwidth memory use; that comparison is a Google claim, not an independently verified result in the source.
Google says it built a testing pipeline to evaluate approximately 5,000 Hugging Face models spanning more than 500 architectures, in eager and compiled modes, across TPU generations starting with v5e. For a 30-billion-parameter Muse Glimmer model on one TPUv7 chip, Google reports inference throughput of 25,700 tokens per second at a sequence length of 5,120 and 22,400 at 10,240. Its reported training throughput on four TPUv7 chips was 5,600 and 4,600 tokens per second at those respective lengths. Google also experimented with scaling a Qwen3 MoE 235B training workload from 1,024 to 8,192 TPUv7 chips.
The roadmap includes work on dynamic shapes, compilation performance, operator coverage, reinforcement learning and quantization. The reported benchmarks describe specific workloads and hardware configurations, rather than performance across all PyTorch models.
Original article · Published November 12, 2026.