Google's TPU 8tx and 8i Max: Confidential AI Inference and 2x Memory Bandwidth Explained

Google on November 9, 2026 added two new systems to its eighth-generation Tensor Processing Unit (TPU) family: TPU 8tx, built for confidential AI inference, and TPU 8i Max, which packs two inference chips into one package to double compute, on-chip memory and memory bandwidth. It is the first time Google has introduced four TPU systems in a single year.

The launch matters because it targets two problems that are now slowing enterprise AI projects: keeping sensitive data and model weights protected while they are being processed, and feeding enough memory to agents that work across very long contexts. Google did not give pricing, regions or a general availability date in its announcement, so buyers should treat this as a roadmap signal rather than something they can order today.

Key facts at a glance

• Two new systems, not new chips: TPU 8tx reuses the TPU 8t silicon, and TPU 8i Max combines two TPU 8i chips on one package.
• Confidential computing on a cloud TPU: AMD says TPU 8tx is the first cloud TPU system with hardware-enforced confidential computing, using 5th Gen AMD EPYC host processors.
• Latency claim: Google says TPU 8tx delivers up to 2x lower latency than TPU 8t with a near-zero performance penalty for the security features.
• Density claim: TPU 8i Max delivers double the compute, double the SRAM and double the HBM bandwidth of TPU 8i, and fits twice the silicon in the same rack footprint.
• Software: Google also highlighted open-source TorchTPU for PyTorch, a new XLA Serving Engine and TPU Sync, which it says cuts time to first token by more than 90% in some workloads.

How Google got here

Google first designed TPUs more than a decade ago for its own services, such as voice recognition and translation. The company now says TPUs are a fast-growing business with nearly 100 customers, including frontier AI labs, trading firms, retailers and universities, and that some customers can now deploy TPUs directly inside their own data centers.

At its Cloud Next event in April 2026, Google split the eighth generation into two specialized parts. TPU 8t targets large-scale training, and TPU 8i targets low-latency reasoning. Google said at the time that TPU 8t offers up to 2.7 times better training performance and TPU 8i up to 80% better inference price-performance than the previous generation. It also says TPU 8i cuts inter-chip communication delays by up to half.

The November additions build on that split rather than replacing it. Amin Vahdat, Google's senior vice president and chief technologist for AI and infrastructure, framed the difference as "a new chip" versus "a new system". A chip takes years to design and manufacture. A system is the packaging, memory, networking, storage and security wrapped around that chip, which Google says it can reconfigure in months.

Vahdat said the change in demand comes from customers moving AI agents into production. Businesses are asking for accelerators that can reason across very large context windows, and some highly regulated customers want specialized encryption for unique workloads.

TPU 8tx: confidential AI for regulated workloads

What it is for. TPU 8tx is aimed at regulated enterprises and governments that want to run frontier or proprietary models, and autonomous agents, next to their most sensitive data. Google says it is optimized for secure inference and is also suitable for small-scale training.

The problem it addresses. Data is usually encrypted at rest and in transit, but it has to be decrypted in memory while a computer works on it. Confidential computing closes that gap with hardware-enforced isolation, so data and model weights stay protected even while in use. Google says TPU 8tx builds this protection into the hardware layer.

How AMD fits in. Each TPU 8tx system is paired with 5th Gen AMD EPYC host processors. AMD says the design uses its Secure Encrypted Virtualization (SEV) technology and extends that protection beyond the CPU to the connected accelerators. Before an accelerator gets direct access to protected memory, its identity and security configuration are verified. AMD says customer prompts, model weights and intermediate data stay encrypted in CPU memory and in transit between the host and the TPUs, with hardware isolation while the accelerator runs.

"When teams move AI agents into production, they need confidence that their data and intellectual property stay private, even from the underlying infrastructure."

— Mark Lohmeyer, vice president and general manager for AI and Computing Infrastructure, Google Cloud

Performance trade-off. Security features often cost speed. Google claims TPU 8tx avoids most of that cost and, because it uses the low-latency Boardfly network built for TPU 8i, delivers up to 2x lower latency than TPU 8t. Google also says teams can fine-tune on private data, distill large models into smaller ones and run reinforcement learning on the same system, without moving sensitive data between clusters.

TPU 8i Max: aimed at the memory wall

The bottleneck. When many agents work across a whole codebase or hours of video, memory bandwidth, not raw compute, becomes the limit. Engineers call this the memory wall. A larger context window means the model has to read and reread more data, which puts pressure on on-chip memory and the links between chips.

Google's answer. TPU 8i Max places two TPU 8i chips on a single package. Google says that doubles compute, SRAM (fast on-chip memory) and HBM (high-bandwidth memory stacked next to the processor) bandwidth compared with a single TPU 8i. The aim is to keep data close to the processors so long-context jobs do not stall.

Density for data centers. Because TPU 8i Max fits twice the silicon into the existing rack footprint, Google says operators can deploy significantly more compute per square foot and per watt. Google has not published absolute power figures for the new package.

Source: Google's eighth-generation TPU announcements, April and November 2026.

The software Google shipped alongside

Hardware only helps if developers can use it, and Google spent much of the announcement on software:

1. TorchTPU, open-sourced last month, lets PyTorch code and Hugging Face models run natively on TPUs, with TPU backends for the vLLM and SGLang inference engines.
2. A TPU agent suite, now in preview, plugs into coding tools such as Claude Code, Cursor and Google Antigravity to help onboard workloads, find bottlenecks and tune kernels.
3. Agent Compute Engine suspends idle agent sessions and moves their working memory off the chip within milliseconds, so paused agents do not hold expensive accelerator capacity.
4. TPU Sync, being open-sourced, treats chip memory, host memory, shared storage and the network as one memory hierarchy. Google says it cuts KV cache transfer latency and time to first token by more than 90%.
5. GKE dynamic slicing, now generally available on Google Kubernetes Engine, reshapes TPU topologies on demand. Google says it speeds job startup by up to 5x and fault recovery by up to 4.5x.

Another perspective: trust still has to be verified

Confidential computing reduces how much you have to trust the cloud operator, but it moves that trust to the chip makers' firmware and to attestation, the process that proves a machine is running the expected, patched configuration.

That chain has failed before. In September 2024, Google's own security researchers reported a flaw to AMD, tracked as CVE-2024-56161 with a severity score of 7.2, in how AMD processors verified microcode signatures. The researchers said an attacker with local administrator access could load malicious microcode and compromise workloads protected by AMD SEV-SNP. AMD shipped fixes from December 2024, public disclosure followed on February 3, 2025, and Zen 5 processors, the design behind 5th Gen EPYC, were added to the affected list in April 2025 after their own fixes.

None of this means TPU 8tx is unsafe; AMD notes that Google security experts have reviewed its security processor firmware for years. It does mean buyers should check attestation reports and firmware levels rather than treat "confidential" as a guarantee.

Investors reacted calmly. Alphabet shares rose more than 2% in premarket trading on Monday, after falling 9% so far this year through Friday's close.

What it means for you

• For AI and platform teams in regulated sectors such as banking, health care and the public sector in the US, UK, Canada and Australia, TPU 8tx could make it easier to argue that fine-tuning on customer or patient data in a public cloud meets internal security rules. You will still need to map it to your own compliance requirements.
• For teams running long-context agents, TPU 8i Max is aimed at jobs where memory, not compute, limits throughput, such as agents working across large codebases or long video.
• For PyTorch shops, the larger news may be the software. Running existing PyTorch and Hugging Face code on TPUs lowers the switching cost from GPUs and gives you a second supplier to compare on price.
• For data center and procurement teams, twice the silicon per rack footprint changes capacity planning, but wait for power, cooling and pricing details before committing.

What happens next

Google scheduled a broadcast on its new TPU systems for 9 a.m. Pacific time on November 9. The open questions are practical ones:

• Pricing and availability: Google's announcement did not list prices, regions or a general availability date for either system.
• Attestation details: How customers will verify a TPU 8tx machine's configuration, and which evidence they can feed into their own audits, will matter more than the headline claim.
• Independent benchmarks: The 2x latency and 2x bandwidth figures are Google's own. Third-party tests on real workloads will show how they hold up.

Takeaways

1. Treat TPU 8tx as an option, not an answer: if you are blocked from using cloud AI on sensitive data, add it to your shortlist and ask your Google Cloud contact for attestation documentation and availability dates.
2. Check your bottleneck before buying: if long context is what slows your agents, measure memory bandwidth use first, then compare TPU 8i Max with current GPU options.
3. Test the software path now: try TorchTPU with a representative PyTorch model so you know your porting cost before capacity decisions are due.
4. Keep firmware on your checklist: for any confidential computing platform, require patched firmware levels and verified attestation as a deployment gate.