Google TPU v8: two chips a year, and a 9,600-chip training domain
Speakers: Norm Jouppi, Google Fellow and TPU tech lead since 2013, and Shridhar, senior director of AI infrastructure leading the TPU chip architecture team
Summary: Google used its Hot Chips slot to introduce the eighth TPU generation and, with it, a change in cadence — two chips a year instead of one, split into a dedicated inference part and a dedicated training part. The talk covered why the two now need different network topologies, a 9,600-chip scale-up domain with two petabytes of shared memory, a purpose-built scale-out fabric reaching 134,400 chips, and the reliability engineering underneath all of it.