Please note: This PhD defence will take place in DC 2314 and online.
Ronak Pradeep, PhD candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Jimmy Lin
Modern information access now hinges on large language model (LLM)–centric pipelines, from traditional multi-stage ranking to end-to-end retrieval-augmented generation (RAG), fundamentally reshaping how users consume information. Yet this shift has surfaced three core obstacles: an overreliance on proprietary rerankers that hinders reproducibility (the Component Challenge), the lack of a standardized ecosystem for building and comparing RAG systems (the Benchmarking Challenge), and the difficulty of reliably evaluating RAG responses (the Evaluation Challenge).
This thesis addresses these challenges. We first push supervised ranking and generative retrieval paradigms to their limits, culminating in RankZephyr. This 7B-parameter open-source zero-shot listwise reranker matches or surpasses proprietary frontier models like GPT-4 while remaining transparent, efficient, and widely adopted. We then introduce Ragnarok, a reusable end-to-end RAG framework that standardizes system building and powers the TREC 2024 RAG Track, enabling genuinely reproducible benchmarking at scale. Finally, we refactor nugget-based evaluation for the LLM era via the AutoNuggetizer framework, automating scalable and reliable, fine-grained assessment of generative answers.
In building powerful open rerankers, a shared benchmark, and a scalable evaluation methodology, we directly addressed the three challenges. Their widespread adoption across the research community underscores their impact.
To attend this PhD defence in person, please go to DC 2314. You can also attend virtually on Zoom.