GenDSR: Transferring Representations from Video Generation Models for Dynamic Spatial Reasoning
Organizations: Nanjing University
Abstract
Dynamic spatial reasoning is essential for connecting visual intelligence to thephysical world, yet remains challenging for vision-language models (VLMs), which must preserve scene state while tracking how spatial relations evolve over time. Meanwhile, video generation models (VGMs), trained to synthesize temporally coherent world dynamics, offer a rich source of spatiotemporal representations. We present GenDSR, a framework that transfers intermediate representations from VGM into VLM for dynamic spatial reasoning. Rather than treating a VGM feature as a monolithic prior, GenDSR constructs two complementary branches: a raw branch preserving holistic generative context and a delta branch emphasizing temporal variation. Task-conditioned tokenwise gates then fuse both signals and inject the result into the native visual representation. On DSR-Bench, GenDSR achieves 67.2%, surpassing the previous best result and establishing a new state of the art.
Figures & tables
| Models | A.Dis | A.Dir | A.Ori | A.Spd | A.SpdC | A.DirP | R.Dis | R.Dir | R.Ori | R.Spd | R.SpdC | R.DirP | N-Temp | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Proprietary Models | ||||||||||||||
| GPT-5 | 21.1 | 41.5 | 48.7 | 34.5 | 33.3 | 34.7 | 17.2 | 44.3 | 41.9 | 21.2 | 25.0 | 30.9 | 26.7 | 30.8 |
| Gemini-2.5-Pro | 20.0 | 44.6 | 53.6 | 27.3 | 38.7 | 30.5 | 23.2 | 32.9 | 43.2 | 17.1 | 28.5 | 27.9 | 34.3 | 31.7 |
| Spatial Reasoning Models | ||||||||||||||
| VLM-3R | 28.2 | 27.6 | 31.7 | 42.8 | 38.7 | 33.0 | 34.4 | 23.8 | 30.8 | 22.2 | 26.4 | 29.1 | 35.0 | 31.4 |
| VG-LLM | 55.2 | 32.3 | 58.5 | 57.1 | 51.6 | 32.2 | 56.0 | 36.3 | 32.0 | 30.3 | 32.1 | 29.1 | 27.9 | 38.4 |
Spatial Reasoning
Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.
See all 161 papers