Turning thousands of GitHub repos into reusable skills more than doubles AI research performance.

Computer programs called autonomous agents are starting to do machine-learning research on their own. These agents have tools to plan, run code, and check their work, but they lack operational knowledgeoperational knowledgeThe practical know-how needed to make machine learning methods work in real scenarios.. Operational knowledge is the practical know-how needed to make methods actually work in practice. This practical information sits in software repositories and research papers, but it is written for people and is too big to load during a project. To fix this, researchers built DisCo, an agent that distills code repositories into small, verified skills it can reuse. DisCo extracts broad, reusable skills from popular repositories and creates specific skills needed for a single task. The researchers used this method to build a library of over 5,000 skills. Adding these skills to an agent boosted its performance on four research tests, including a 134.3% score increase on MLE-benchMLE-benchA benchmark test for evaluating machine learning engineering skills. compared to running the same agent without any skills.

Why it matters

Autonomous AI agents for research often miss the practical know-how that makes methods work. By distilling existing code into quick-use skills, these agents can apply proven techniques without starting from scratch, helping AI systems solve complex problems faster.

Questions answered