Vedant Khatri

B.S. Computer Science, University of California, Irvine, expected June 2027 · GPA 3.781

Researching when we can trust an LLM's judgment about students, in grading, tutoring, and learning analytics.

[email protected] · GitHub · LinkedIn · Irvine, CA

Portrait of Vedant Khatri

About

I study when we can trust a large language model's judgment about a student, and how to tell why it went wrong when we can't. I'm a computer science senior at UC Irvine, applying to PhD programs for Fall 2027.

My research started in Professor Renzhe Yu's lab at Teachers College, Columbia. Our LAK 2026 paper split LLM annotation errors into the ones the model caused and the ones that came from an unclear task. It was nominated for Best Paper. In my first-author ICQE 2026 paper, I used Epistemic Network Analysis to see why a five-agent essay scorer reasoned its way to wrong scores. Revising the agents' prompts based on what we found raised exact scoring accuracy from 28% to 40%.

This summer I built a pipeline with Professor Conrad Borchers at Vanderbilt in which two LLM agents code tutoring data and revise the codebook wherever they disagree. That paper is under review at LAK 2027. I also work with Professor René Kizilcec's lab at Cornell on a coding framework for an AI tutor that teaches clinical reasoning, and with Professor Zachary Pardos at UC Berkeley on estimating question difficulty with simulated students.

I'm also the course manager for ICS 31, UC Irvine's intro programming course, which has up to 340 students a term.

If you work on LLM evaluation or learning analytics, I'd be glad to talk. Email me at [email protected].

Publications

Peer-reviewed conference proceedings

Xu, Z., Khatri, V., Dai, Y., Liu, X., Li, S., Zhang, X., & Yu, R. (2026). Enhancing LLM-Based Data Annotation with Error Decomposition. In Proceedings of LAK '26: 16th International Learning Analytics and Knowledge Conference (pp. 325–335). Best Paper Nominee. doi:10.1145/3785022.3785070

Khatri, V., Cusimano, A., Swiecki, Z., Xu, Z., Liu, X., & Yu, R. (2026). From Diagnosis to Redesign: Using Quantitative Ethnography to Improve Multi-Agent LLM Reasoning. Accepted as a full paper at the International Conference on Quantitative Ethnography (ICQE 2026), to appear in Springer CCIS. arXiv:2608.22566

Peer-reviewed journal articles

Khatri, V., Sinha, A., & Kumari, R. (2022). A study of the impact of e-learning on the health of school going adolescents in Bareilly city: A cross-sectional study. Indian Journal of Forensic and Community Medicine, 9(3), 112–116.

Sinha, A., Khatri, V., & Nath, B. (2022). Relevance of health education to e-learning-associated problems among the school-going adolescents in Bareilly city: An interventional study. Journal of Family Medicine and Primary Care, 11, 6863–6868.

Under review

First-author short paper under review at the 17th International Learning Analytics and Knowledge Conference (LAK 2027). Title withheld during double-blind review.

In preparation

Large Language Models for Educational Data Annotation at Scale. Manuscript in preparation for AERA Open.

Two LAK 2027 posters in preparation: question difficulty estimates in PromptHive (UC Berkeley), and student questions to an AI clinical reasoning tutor (Cornell).

Research

Data Science Research Assistant

Teachers College, Columbia University · Prof. Renzhe Yu · Fall 2024 – Present

  • Research on language models for educational data annotation, resulting in a paper accepted to LAK 2026.
  • Ran LLM annotation experiments across four datasets (Bloom, MathDial, Uptake, and GUG), five prompting strategies, and two model versions.
  • Contributed to the framework separating task-inherent ambiguity from model-based annotation error.
  • Building an LLM pipeline that classifies course assignments into 12 types, refined over several prompt and codebook versions and evaluated against 300 human-labeled samples.
  • First author of the ICQE 2026 paper. Used Epistemic Network Analysis to find why a five-agent LLM essay scorer reached wrong scores, then revised the agents' prompts, which raised exact scoring accuracy from 27.78% to 40.28%.

Research Collaborator

Vanderbilt University · Prof. Conrad Borchers · Summer 2026

  • Built a two-agent LLM pipeline that codes tutoring data and revises the codebook where the agents disagree, without human labels.
  • Tested codebook revisions with open-weight (Qwen, Llama) and frontier (GPT-5.5, Claude Haiku) models.
  • First author of the resulting short paper, now under review at LAK 2027.

Research Collaborator

Cornell University · Future of Learning Lab · Prof. René Kizilcec · Summer 2026 – Present

  • Developed the codebook and qualitative coding scheme for analyzing an LLM-based clinical-reasoning tutoring system, working with Yann Hicke.
  • Coded real tutor-learner transcripts, showing that coding the tutor's moves alone cannot separate effective from ineffective tutoring, a finding that reshaped the study's direction.
  • Identified corpus-level data-quality issues that would have corrupted automated coding.
  • Preparing a LAK 2027 poster on the questions students ask the tutor.

Research Collaborator

University of California, Berkeley · CAHLR Lab · Prof. Zachary Pardos · Summer 2026 – Present

  • Integrated PromptHive, which helps instructors generate questions through prompt engineering, with LLMRespondent, which estimates question difficulty using synthetic LLM respondents and Item Response Theory calibration, to build a difficulty filter for generated questions.
  • Investigated why these difficulty estimates are unreliable: a single LLM produces a narrow proficiency distribution and can confidently misrate questions.
  • Preparing a LAK 2027 poster on question difficulty estimates in PromptHive.

Undergraduate Research Fellow

University of California, Irvine · Prof. Shannon Alfaro · Dec 2023 – May 2024

  • Ran a cross-sectional study of social presence and satisfaction in online learning among UC Irvine CS students.
  • Found a strong, statistically significant link between the two and presented it at the UROP Symposium 2024.
  • Selected as a UROP Fellow for 2023–2024 (UROP recognition award).

Research Assistant

Rohilkhand Medical College, Bareilly, India · Apr 2020 – Apr 2022

  • Conducted cross-sectional and interventional studies examining adolescent health, including data collection and statistical analysis.

Teaching

Learning Assistant

ICS 46: Data Structure Implementation and Analysis · Prof. Thomas Yeh · UC Irvine · Fall 2026 – Present

  • Support students in labs and office hours as they learn to build and analyze data structures.

Course Manager

ICS 31: Introduction to Programming · UC Irvine · Winter 2025 – Present

  • Run the operational side of one of UC Irvine's largest introductory computer science courses, with roughly 175 to 340 students per term.
  • Train and coordinate 3–4 TAs and graders each term.
  • Manage roster syncing, attendance sheets, retake-ticket eligibility and delivery, secure per-student credentials for in-lab exams, and grading for lab exams and zyBooks assignments.
  • Operate and extend the course's Python automation pipeline (Canvas, zyBooks, and Google Sheets APIs), adding features on request such as lecture-attendance tracking and automated retake-ticket delivery.

Learning Assistant

ICS 45C · Prof. Nadia Ahmed · UC Irvine · Winter 2026

  • Supported the C++ portion of the introductory sequence while serving concurrently as Course Manager for ICS 31, with office hours, one-on-one debugging support, lab sections, and exam and assignment materials.

Grader

ICS 31 · UC Irvine · Fall 2024

  • Graded programming assignments, projects, and exams, and held office hours.

Learning Assistant

ICS 32 · Prof. Mustafa Ibrahim · UC Irvine · Spring 2024

  • Supported intermediate Python instruction, with weekly office hours, lab sections, and assignment and exam materials.

Learning Assistant

ICS 31 · Prof. Shannon Alfaro · UC Irvine · Winter 2024

  • Supported introductory Python instruction in the same course I later managed, with the same weekly office-hour and lab duties.

Service & Leadership

LAK 2027 Program Committee · 17th International Learning Analytics and Knowledge Conference · 2026

Program committee member, reviewing submissions.

ICQE 2026 Reviewer · International Conference on Quantitative Ethnography · 2026

Reviewed two submissions.

LAK 2026 · Bergen, Norway · April 27 – May 1, 2026

Presented our paper at the 16th International Learning Analytics and Knowledge Conference.

ISLS 2026 · International Society of the Learning Sciences · Irvine, CA · June 15 – 19, 2026

Conference volunteer, assisting with operations and attendee support.

ISAC Intern · Associated Students of UC Irvine (ASUCI) · Fall 2024

Advocated for international students, addressing concerns and organizing transition support events.

School Council · GD Goenka Public School, Bareilly, India · 2020 – 2023

Head Boy (2022–2023) and House Captain (2020–2022), leading career fairs, cultural events, mentorship programs, and academic workshops for 800+ participants.

Skills

Languages: Python, C++, SQL, JavaScript, Bash

LLMs and NLP: OpenAI API, Anthropic API, Hugging Face, vLLM, Ollama, DSPy, Sentence Transformers, spaCy, NLTK

Machine learning: PyTorch, TensorFlow, Keras, scikit-learn, XGBoost

Data and statistics: Pandas, NumPy, SciPy, Statsmodels, Matplotlib, Seaborn, Item Response Theory, Epistemic Network Analysis

Tools: Git/GitHub, Jupyter, Linux, Flask, React, Streamlit, MongoDB, OpenCV

Awards & Funding

UROP Fellow, Summer Undergraduate Research Program (SURP), UC Irvine, Summer 2026. The funded project became my first-author ICQE 2026 paper.

ICS Honors Program, UC Irvine, 2026.

Best Paper Nominee, LAK 2026.

UROP Recognition, UC Irvine, 2023–2024.

Dean's Honor List, UC Irvine.