Original title: Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuation
Yingmao Miao, Pengfei Zhang, Chaoran Xu, Meng Yu +4 · September 2026
When artificial intelligence makes long videos, it usually links short clips together. To do this, it looks at earlier pictures to remember what happened. However, current video generators struggle to track the actual state of the world across these clips. A past picture shows what was visible before, but it cannot show hidden changes or what should happen under a new instruction. To study this, researchers created Statebench, a test covering three areas: past-visible states, occluded-process statesStates that must be inferred from hidden or non-visible changes rather than directly observed frames., and complex-transition statesStates involving multiple or intricate changes that are not straightforward to track from direct observation.. They also built Stateagent. Instead of just copying past pictures, this system tracks entities and their conditions. When given a new prompt, it updates those states, predicts what the final frame should look like after the action, and then creates the video clip. The researchers found that Stateagent raised a key state score from 45.2 to 69.3, and it improved story generation at the one-minute scale.
Imagine you are telling a story where each chapter depends on the last, but you only remember the most recent events. It's easy to forget a character's location or what they were carrying if it wasn't mentioned lately. In this paper, video generators struggle similarly when making long videos, especially with things that change off-screen. Stateagent works like constantly updating a character list, so when you write the next chapter, you know exactly where everything stands.
When AI makes long videos, it often misses details that aren't directly visible, making videos inconsistent. This affects creators who use AI for storytelling, as broken continuity reduces realism and usability.
The abstract doesn't explicitly state limitations, but the work is limited to evaluating three state categories and improving the SCS-All metric. It may not generalize to all video generation tasks or handle states beyond those tested.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuation — Yingmao Miao, Pengfei Zhang, Chaoran Xu et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.