PhD student Yuzhe You has won the Michael A. J. Sweeney Award for Best Student Paper at Graphics Interface 2025. Held annually by the Canadian Human-Computer Communications Society, GI is the nation’s top conference on computer graphics and visualization, and human-computer interaction.
“I’m truly honored that our paper received such recognition. With AI being adopted in so many areas of life, it’s easy to get swept up in its potential without thinking about the risks. I’m glad our work is helping to raise awareness about the security vulnerabilities that can come with AI and encouraging more developers to think critically about model safety before putting these systems into the world,” says Yuzhe.
The award recognizes Yuzhe’s paper, Exploring Comparative Visual Approaches for Understanding Model Trade-offs in Adversarial Machine Learning, co-authored with Professor Jian Zhao, her supervisor.
“This study tackles the limitations of adversarial training by empowering practitioners with visual tools to evaluate model fairness and robustness. It proposes a visual analytics tool, VATRA, which offers a new way to interpret and balance competing performance metrics in AI systems. It highlights how comparative visualizations can support better decision-making in robust model development,” explains Professor Zhao.
In machine learning, it is crucial to safeguard models against cyberattacks and data privacy breaches. The most effective technique is adversarial training (AT), where a model is exposed to adversarial examples — data designed to mislead the model. As a result, the model can identify and thwart malicious attacks. However, AT can trade-off accuracy for robustness. The resulting model may underperform on clean datasets or be less effective than a natural model. ML researchers are then faced with a dilemma: should they prioritize robustness? Accuracy? Or should they strike a balance between both?
This issue prompted Yuzhe and Professor Zhao to investigate whether visual analytics tools could support model trade-off comparisons. For example, it could depict the accuracy rate among clean and manipulated datasets. The visualizations could help researchers link patterns between the model’s design features and performance.
“Through this research, we were excited to see how visualizations can meaningfully support machine learning practitioners in evaluating model robustness and trade-offs. With the help of VATRA, we observed a shift in their workflow—from an unstructured, linear process to a more flexible and iterative workflow. We hope the design insights from this work can inspire the development of other comparative visualizations across different areas of machine learning,” says Yuzhe.
In the first half of their study, the duo interviewed five adversarial machine learning (AML) experts about their experiences. Surprisingly, none of the interviewees use visual analytical tools due to limited capability. Instead, they created tables and bar charts to examine AT comparisons, which they expressed as tedious and cumbersome.
Based on these interviews, the team developed a visual analytics design probe, VATRA. It has two main components: a backend analytics pipeline and a frontend user interface. The backend analytic pipeline can generate adversarial examples, which the user can adjust and define. The frontend interface has six panel views that provide visualizations of the models’ performance. For example, the summary view can display metrics on accuracy and robustness based on defined benchmarks. The global dual projection view can reveal similarities and differences between the natural and adversarial models, especially between different classes.
Professor Jian Zhao at GI's official award ceremony, which took place in Kelowna, British Columbia. He received this award on Yuzhe's behalf.
Yuzhe and Professor Zhao conducted a user study with six AML experts to examine VATRA’s impact on their workflows. Typically, AML practitioners lacked a clear strategy or starting point during their review. However, VATRA’s innovative and engaging visualizations can facilitate their analysis, leading to a more structured and goal-oriented workflow.
VATRA can also iterate the workflow, allowing users to uncover insights from multiple viewpoints. For example, some AML practitioners adopted a high-low structure approach, where they “began with high-level metrics to identify trade-offs, then move down into low-level metrics for embedding and instance comparisons,” says Yuzhe.
By streamlining and accelerating workflows, VATRA can empower AML practitioners to focus on designing stronger and more efficient models — leading to a new wave of cybersecurity tools.