SOD：让小模型智能体学会用工具的逐步蒸馏 SOD: Step-wise On-policy Distillation for Small Language Model Agents

精读摘要

工具集成推理（TIR）难以扩展到小语言模型：长程工具交互不稳定、模型容量有限。强化学习方法如分组相对策略优化只提供稀疏的结果级奖励；近期流行的在策略蒸馏（OPD）虽能提供密集 token 级监督，但实验表明直接用于 TIR 会出现严重失败模式——错误工具调用会在轨迹中传播。SOD 提出逐步在策略蒸馏，用教师信号分步修正学生生成的轨迹，让小型智能体也能稳定使用工具。 Tool-integrated reasoning is hard to scale to small language models due to instability in long-horizon tool interactions and limited capacity. On-policy distillation supplies dense token-level supervision, but applied to tool-integrated reasoning it exhibits a critical failure mode where erroneous tool calls propagate through trajectories. SOD proposes step-wise on-policy distillation, correcting student-generated trajectories step by step.