Original title: GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
Junqing Du, Fernando Ropero, Erkin Turkoz, Yanfeng Zhang +1 · September 2026
Today's multimodal large language models struggle to understand 3D physical spaces. They fail to measure precise geometry, switch between first-person and top-down viewpoints, and connect detailed visual appearances to objects. Usually, fixing this requires costly extra training or specialized hardware models. To solve this, researchers built GraFT, a method that requires no extra training. GraFT gives the model a 3D scene graph, which maps out objects and their connections. Using this map, GraFT offers three tools: symbolic tools that calculate exact measurements, bird's-eye-view images showing the overall room layout, and first-person camera frames showing specific visual details. In tests, GraFT boosted scores across the board, lifting one benchmark's CIDEr metric by 27% and improving frozen models on another test by up to 65%.
Imagine you're trying to describe the layout of your house to someone over the phone. Without a floor plan, you might mix up directions or forget exact room sizes, leading to confusion. In this paper, GraFT gives AI models a 3D scene graph that acts like a virtual floor plan, providing a top-down view for layout, precise tools for distances, and specific photos to identify objects.
3D spatial reasoning is essential for systems like robots and augmented reality that interact with the physical world. Current AI models struggle with this, but GraFT improves them without needing expensive retraining, making reliable spatial understanding more accessible.
The abstract doesn't explicitly list limitations, but it focuses on improvements on ScanQA and VSI-Bench, suggesting it may not address all spatial reasoning challenges or scenarios where 3D scene graphs are unavailable.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs — Junqing Du, Fernando Ropero, Erkin Turkoz et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.