Research & teaching
Research talks and a short course on statistical inference, large language models, and reinforcement learning.
The collection
Open the slides, explore the abstract,
or read the accompanying paper.
How language models are built and trained: Transformer foundations, pre-training, supervised fine-tuning and preference optimization, followed by inference-time reasoning and reinforcement learning with verifiable rewards.
Presented at
From PFNs and predictive uncertainty to confidence intervals for regression functions. This talk explains TabCon's residual heads, attention-based routing, supervised training and reinforcement learning, with interactive illustrations of coverage, interval length and computation.
BASIS estimates advantage baselines by sharing reward information across prompts in a training batch. It combines an offline value signal with online calibration to support LLM reasoning with one rollout per prompt.
Presented at
VRPO uses an auxiliary preference model to reduce variance in reward and policy estimation when the primary preference model is misspecified. The talk develops the method from RLHF and DPO, then examines sentiment, summarization and dialogue experiments.
Presented at