AdaptMI: Adaptive Skill-based In-context Math Instruction for Small Language Models
Yinghui He, Abhishek Panigrahi, Yong Lin, Sanjeev Arora
Download Paper
COLM 2025; ICML 2025 Workshop on Test-Time Adaptation; ICML 2025 Methods and Opportunities at Small Scale Workshop, 2025
Kids improve when a good teacher offers adaptive, targeted feedback. Can a small LLM benefit if a large LLM provide helpful feedback, in-context?? Naive ideas fail here. We propose AdaptMI: adaptive, skill-based in-context supervision that boosts 1B models by 6% on challenging math tasks.