Original title: CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?
Damien Sileo, Dimitri Kachler · September 2026
AI agents can now modify the software tools they run on while operating. However, changing one plugin can cause chain reactions through connected parts and cleanup steps. Researchers created CordisBench, a test with 1,200 questions, to see if language models understand these software lifecycles. The benchmark tests whether models can track affected tools, predict final states after components shut down, and pick successful system updates. Models run tests against Cordis, an engine that handles component dependenciesConnections between components where one part relies on another to function correctly.. The researchers found that models do well on small systems. But as more interactionsThe number of points where components influence each other, increasing system complexity. matter, models struggle, especially at predicting final states and figuring out shutdown orders. Giving models more time to think brings clear gains, but it costs a lot. GPT-5.6 Luna uses nearly 3,000 reasoning tokens per question at medium effort on a 16-interaction subset.
When you're cleaning a closet, moving one box can shift others and jam the door. The problem is that the effect depends on which box you move first, and you can't see how all boxes are connected without pulling them out. In this paper, the AI must predict how changing one software component affects the entire system, especially when components are removed in different orders during cleanup.
This affects anyone relying on AI that modifies its own code, like in automated software updates. If models can't track these ripples, systems might crash unpredictably, causing downtime or security issues.
The study uses a controlled formal setting with specific models at low reasoning effort, so results may not generalize to all AI systems or messy real-world code. It is based on the Cordis runtime, which might not represent all dynamic environments. Inference costs are high, with models using many tokens for accurate predictions.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses? — Damien Sileo, Dimitri Kachler
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.