Title: 

Amir Yazdanbakhsh (Google DeepMind), Suvinay Subramanian (Google)

Teams Link

Abstract:

The rapid advancement of artificial intelligence (AI) has ushered in an era of unprecedented computational demands, necessitating continuous innovation in computing systems. In this talk, we will highlight how codesign has been a key paradigm in enabling innovative solutions and state-of-the-art performance in Google's AI computing systems, namely Tensor Processing Units (TPUs). We present several codesign case studies across different layers of the stack, spanning hardware, systems, software, algorithms, all the way up to the datacenter. We discuss how TPUs have made judicious, yet opinionated bets in our design choices, and how these design choices have not only kept pace with the blistering rate of change, but also enabled many of the breakthroughs in AI.

Bio:

Amir Yazdanbakhsh received his Ph.D. degree in computer science from the Georgia Institute of Technology. His Ph.D. work has been recognized by various awards, including Microsoft PhD Fellowship and Qualcomm Innovation Fellowship. Amir is currently a Research Scientist at Google DeepMind where he is the co-founder and co-lead of the Machine Learning for Computer Architecture team. His work focuses on leveraging the recent machine learning methods and advancements to innovate and design better hardware accelerators. He is also interested in designing large-scale distributed systems for training machine learning applications, and led the development of a massively large-scale distributed reinforcement learning system that scales to TPU Pod and efficiently manages thousands of actors to solve complex, real-world tasks. The work of our team has been covered by media outlets, including WIRED, ZDNet, AnalyticsInsight, InfoQ. Amir was inducted into the ISCA Hall of Fame in 2023.

Suvinay Subramanian is a Staff Software Engineer at Google, where he works on the architecture and codesign for Google's ML supercomputers, Tensor Processing Units (TPUs). His work has directly impacted innovative architecture and systems features in multiple generations of TPUs, and empowered performant training and serving of Google's research and production AI workloads. Suvinay received a Ph.D. from MIT, and a B.Tech from the Indian Institute of Technology Madras. He also co-hosts the Computer Architecture Podcast that spotlights cutting-edge developments in computer architecture and systems.