Markelle Rösti

I’m Markelle Rösti (née Kelly), a (soon-to-graduate!) computer science Ph.D. candidate in the DataLab group at the University of California, Irvine, advised by Dr. Padhraic Smyth. I’m interested in understanding and improving how machine learning models are used in practice—going beyond standard evaluation metrics. In particular, I’m excited about:

  • Human-AI interaction: This line of research aims to understand and facilitate collaboration between humans and AI agents, a joint effort with researchers from cognitive science. One project investigated humans’ mental models of their AI teammates, leveraging human-subjects experiments. I’ve also developed methodology for modeling expert predictions in a human-AI setting.
  • Holistic model evaluation: In my internships, I’ve created specialized processes and tools to help detect and characterize undesirable LLM behaviors. I’ve also published work on methods for more fine-grained assessment of model calibration, with an eye towards fairness, and on evaluating LLMs’ abilities to interpret linguistic uncertainty.
  • Data practices in ML: Drawing from my experience as a curator for the UCI ML Repository, I’m working to push back against the under-valuing of “data work” in machine learning. This includes an upcoming ICLR workshop on improving data and benchmarking practices in ML, with an emphasis on the role of data repositories.

I’m a member of the Steckler Center for Responsible, Ethical, and Accessible Technology, the Irvine Initiative in AI, Law, and Society, and the HPI Research Center in Machine Learning and Data Science at UCI. I have previously interned with eBay and Apple, working broadly on tools and processes for evaluating LLMs, and at Project Jupyter, where I was involved in the development and design of JupyterLab. I received my Bachelor’s degree in statistics from California Polytechnic State University, San Luis Obispo in 2020 and my Master’s degree in computer science from UCI in 2022.

Markelle Rösti

Selected Publications

Variable-Based Calibration for Machine Learning Classifiers
Markelle Kelly and Padhraic Smyth
37th AAAI Conference on Artificial Intelligence (AAAI), 2023
[pdf] [code]

Capturing Humans' Mental Models of AI: An Item Response Theory Approach
Markelle Kelly, Aakriti Kumar, Padhraic Smyth, and Mark Steyvers
ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2023
[pdf] [code]

Perceptions of Linguistic Uncertainty by Language Models and Humans
Catarina Belem*, Markelle Kelly*, Sameer Singh, and Padhraic Smyth
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
[pdf] [code]

Benchmark Data Repositories for Better Benchmarking
Rachel Longjohn*, Markelle Kelly*, Sameer Singh, and Padhraic Smyth
Conference on Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2024
[pdf]

News

02/14/2025   UCI ICS News: "André and Markelle Rösti: Match Made in… ICS!"

12/19/2024   I'm co-organizing an ICLR 2025 Workshop: "The Future of Machine Learning Data Practices and Repositories"

10/15/2022   UCI ICS News: "Markelle Kelly Highlights the Human Side of Computer Science"

03/29/2022   UCI Statistics News: "Culture of Collaboration Leads to Cutting-Edge Work in AI"