Original title: Selective Agent Guidance via Entropy: Learning Autonomous Policies from Imperfect VLM Teachers
Matteo Merler, Giovanni Bonetta, Davide Zago, Rossella Cancelliere +1 · September 2026
Vision-Language Models can see and understand text, making them helpful for decision-making tasks. However, running them directly as decision-makers is expensive, fragile, and does not learn from trial and error. To fix this, researchers created SAGE, a system that trains a smaller, independent reinforcement learning policy using a vision-language model as an imperfect guide. Instead of asking the guide constantly, the student agent asks for help only when it is uncertain. It executes the suggested move during training and learns from it. Crucially, SAGE uses environment feedback to score whether the advice actually helped, rather than trusting every suggestion blindly. In visual reasoning and navigation tests, the trained student learned to act entirely on its own without calling the large model. It outperformed regular unguided training and even beat its own teacher. SAGE also cut costs by asking for help on only a fraction of training steps.
Imagine learning to cook a complex meal by following a detailed cookbook. The cookbook is thorough but sometimes has errors or requires rare ingredients, so you can't rely on it entirely. In this paper, the agent is like the cook, and the VLM is the cookbook. The agent checks the cookbook only when unsure, practices the recipes, and eventually learns to cook the meal without needing the book.
For developers building AI agents, this means they can use powerful but costly vision-language models as temporary teachers without needing them forever. Agents learn to make decisions on their own, reducing computational expense and avoiding repeated errors.
The abstract doesn't specify the exact entropy calculation or the detailed environments beyond sparse-reward visual reasoning and navigation tasks. It also doesn't address how the method performs in dense-reward or non-visual settings.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Selective Agent Guidance via Entropy: Learning Autonomous Policies from Imperfect VLM Teachers — Matteo Merler, Giovanni Bonetta, Davide Zago et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.