Professor Chengnian Sun and his collaborators Vu Le and Zhendong Su have received the Most Influential Paper Award at OOPSLA 2025, part of the ACM SIGPLAN SPLASH conference. Their paper, Finding Deep Compiler Bugs via Guided Stochastic Program Mutation, presented originally at OOPSLA 2015, was recognized for introducing a novel Equivalence Modulo Inputs mutation strategy that exposed previously undetected bugs in production compilers.
The research was conducted when Professor Sun was a postdoctoral researcher and Vu Le a PhD student, both advised by Professor Su at the University of California, Davis. The award recognizes a paper from ten years earlier that has made a lasting impact on the field of programming languages.
In its citation, the OOPSLA award committee wrote, “This paper has made substantial contributions in establishing systematic and automated test generation as a widely-used technique for ensuring compiler robustness in practice, through an extension of the Equivalence Modulo Inputs (EMI) technique. EMI-based compiler testing observes which code is unexecuted for a given program and input to that program, modifies the unexecuted code, and then checks that the resulting program behaves identically to the original program on the same input. The key innovation of the paper was to insert new unexecuted code into the program, leveraging a Markov-Chain-Monte-Carlo search to generate diverse mutants more likely to expose compiler bugs. Compiler testing techniques building on this new EMI-based technique have exposed over 1,000 bugs in the GCC and LLVM compilers that were reported and fixed by the developers, as well as bugs in compilers for a variety of other languages.”
Chengnian Sun is an Associate Professor and Cheriton Faculty Fellow at the Cheriton School of Computer Science. His research interests lie at the intersection of software engineering and programming languages, with a focus on improving software quality and enhancing developer productivity through better tools, techniques and methodologies.
He has a PhD in Computer Science from the National University of Singapore. Before joining the Cheriton School of Computer Science, he worked as a software engineer at Google in Mountain View, California, and as a postdoctoral researcher at the University of California, Davis, where he conducted this award-winning research.
More about compilers and this research
A compiler is a program that translates code written in a high-level programming language into a lower-level language such as machine code. Compiler bugs are particularly serious because they can affect all programs. These bugs may cause compilers to silently miscompile a program, leading to its incorrect executions and even security vulnerabilities in the miscompiled program. Because bugs in production compilers are relatively rare and occur only under specific circumstances, they can go undetected during software development and surface only after deployment.
This award-winning research advanced a then-emerging compiler validation approach known as Equivalence Modulo Inputs. EMI works by generating new test programs from existing ones that should behave identically under specific inputs. By comparing how a compiler processes these equivalent variants, researchers can detect miscompilations.
Earlier tools such as Orion had demonstrated EMI’s promise, uncovering numerous bugs in production compilers. But despite Orion’s success in finding bugs, its effectiveness was limited by its simple, blind mutation strategy.
To address these limitations, Professor Sun and his collaborators developed Athena, a new realization of EMI that uses guided mutation rather than random variation. By combining code insertion and deletion in unexecuted regions of a program, and applying Markov Chain Monte Carlo optimization guided by control-flow diversity, Athena generates a large number of variants for testing. This guided search allows Athena to uncover deep, complex compiler behaviours that simpler fuzzing tools cannot expose.
Over 19 months of testing, Athena discovered 72 previously unknown bugs in GCC and LLVM, two of the most widely used compilers. All 72 bugs were confirmed by developers. Seventeen of the 40 GCC bugs were marked as P1, the most severe kind of bugs that must be fixed before a new GCC release can be made.
Their experiments showed that Athena detected both shallow and deep bugs, the latter of which require sophisticated mutation sequences that could not be done using Orion. Importantly, the study demonstrated the effectiveness of the general EMI technique and provided insight into developing effective EMI mutation-based compiler testing strategies.
To learn more about the research on which this article is based, please see Vu Le, Chengnian Sun, Zhendong Su. Finding deep compiler bugs via guided stochastic program mutation. Proceedings of ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), 2015.