Original title: MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT
Chenguang Zheng, Le Xue, Yichi Zhang, Wenbo Zhang +7 · September 2026
Doctors often combine two whole-body scans to make a diagnosis: CT scans, which reveal body structures, and PET scans, which track metabolic function, or chemical activity inside organs. Most medical artificial intelligence tools look only at regional CT scans of single body parts, not full-body combinations. To solve this, researchers built MetaStructAtlas, a large dataset linking whole-body PET and CT images with labeled organs and medical text. They also built MetaStructVQA, a question-answering benchmark. This tool ties diagnostic questions directly to visual evidence across both scans, checking anatomical structure, shape, and metabolic signals. Finally, the team evaluated modern medical vision models on this benchmark to establish a foundation for AI reasoning across the entire body.
Imagine planning a hike where you need both a trail map and a weather forecast to stay safe. If you only use the map, you might walk into a storm; if only the forecast, you don't know the paths. You must combine them to decide which route to take. In this paper, PET scans are like the weather forecast showing metabolic activity, and CT scans are like the map showing structure. This dataset lets AI learn to use both together for better diagnosis.
Right now, medical AI only handles scans of small body regions, but whole-body PET/CT is crucial for diagnosing diseases like cancer. This dataset gives AI the data it needs to learn from full-body scans, helping nuclear medicine doctors make more accurate decisions.
The abstract doesn't mention specific limitations, but it positions the work as a foundation for future research rather than a clinical solution. It doesn't claim immediate improvements in diagnosis or compare performance against existing benchmarks, focusing on providing data and tests for others to build upon.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT — Chenguang Zheng, Le Xue, Yichi Zhang et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.