Anonymous Institution
World models are commonly trained and evaluated through next-step prediction, yet standard metrics such as FVD and LPIPS assess only local generation quality. Whether a model maintains compositional consistency—inverse reversibility, loop closure, and path equivalence—remains largely unmeasured, because existing datasets lack the structural annotations needed to define and compute such metrics.
We introduce Reasoning-Structured Videos, a large-scale synthetic video benchmark built in Unreal Engine 5 that fills this evaluation gap. Our dataset organizes trajectories as rooted action-conditioned graphs. Each graph starts from a shared initial state and contains annotated structural relations—inverse paths, loops, and equivalent paths—encoding the compositional algebra of the environment dynamics. These annotations enable a family of consistency metrics (Inv-Err, Loop-Err, Equiv-Err) that quantify aspects of model behavior invisible to standard per-frame metrics.
Large-scale UE5 benchmark with rooted trajectory graphs annotated with three structural relations: inverse, loop, and equivalence.
Three new metrics—Inv-Err, Loop-Err, Equiv-Err—that quantify compositional consistency, provably orthogonal to FVD / LPIPS.
We prove that zero next-step prediction risk does not imply consistency, justifying the metrics as principled diagnostic tools.
A latent-space encoder-decoder model supporting 20-second contexts (81 VAE temporal tokens), evaluated on all three consistency metrics.
Ground-truth trajectories rendered in Unreal Engine 5. The keyboard HUD on each video shows the current action in real time. Paired relations (Equivalence) display Path A and Path B side-by-side with a shared playback bar.