After the success of the last compiler social, our research group is hosting another compiler social at the University of Cambridge’s Computer Laboratory.
Date: 12.02.2025
Time: 15:00-20:00 (2 talks followed by 3h socializing)
Location: William Gates Building, 15 JJ Thomson Ave, Cambridge CB3 0FD 1
Rooms: LT1 (Talks), Foyer (Social)
Hosts: Emma Urquhart, Luisa Cicolini, Tobias Grosser
If you’d like to attend please make sure to register.
There will be a talk by George Constantinides at 15:00, a talk by Alex Zinenko at 16:00 followed by socializing in the hallway with food and drinks from 17:00 onward.
Here the talks’ and speakers’ details:
Hardware Datapath: For Machine Learning and Beyond - George Constantinides
We will explore a couple of topics of recent interest to me: how to best bridge the Boolean world of hardware design with the continuous world of modern machine learning, and how to utilise recent advances in graph rewriting for datapath optimization. The former will lead us to explore how the fundamental computational units of deep learning might be reconsidered with hardware efficiency in mind. The latter will allow us to optimize hardware for computer arithmetic, including automated (re-)discovery of well-known manual tricks, and is now in use by Intel. This talk summarises the outcomes of joint work with Marta Andronic, Peter Cheung, Sam Coward, James Davis, Theo Drane and Erwei Wang.
George A. Constantinides received the Ph.D. degree from Imperial College London in 2001. Since 2002, he has been with the faculty at Imperial College London, where he is currently Professor of Digital Computation and Director of the Imperial Early Career Researcher Institute, having previously served as Associate Dean of Engineering and Head of Circuits and Systems. He has been TPC chair of the FPGA, FPL and FPT conferences. He currently serves on several program committees and has published over 200 research papers in peer refereed journals and international conferences. Prof Constantinides enjoys family, espresso and general geekery.
Equality Saturation in a Real-World Machine Learning Compiler - Alex Zinenko
Machine learning (ML) compilers rely on graph-level transformations to enhance the runtime performance of ML models. However, these program transformations are often driven by manually-tuned compiler heuristics, which are quickly rendered obsolete by new hardware and model architectures. Instead, we propose the use of equality saturation. We replace such heuristics with a more robust global performance model, which accounts for downstream transformations. While this approach still requires a global performance model to evaluate the profitability of transformations,it holds significant promise for increased automation and adaptability. We address challenges in applying equality saturation on real-world ML compute graphs and state-of-the-art hardware, study different cost modeling approaches to deal with fusion and layout optimization, and tackle scalability issues that arise from considering a very wide range of algebraic optimizations. Our implementation builds on and improves the XLA compilation pipeline for CPU and GPU.
Alex is a compiler researcher working primarily on compilers for machine learning, who currently holds a leadership position in a stealth start-up company. Until early 2024, he was a part of Google DeepMind research lab working on compilers for ML and ML for compilers. He obtained his PhD from the University Paris Saclay (Paris Sud XI) for his work on “Interactive Program Restructuring”. His research interests span from compilation to high-performance systems, to interactive software visualization united for the common goal of making programming efficient programs effectively.