Original title: InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models
Chao Shen, Xinyuan Li, Yunfan Zhou, Jianguo Yao +3 · September 2026
Humans can easily read industrial gauges, but advanced AI vision models struggle with continuous measurements. Most current tests check these models in simple settings that ignore real-world industrial environments. To address this, researchers created InSituMeasure, a test featuring real factory scenes. They tested models by checking if the AI could accurately read measurements and units within specific error limits, while also testing if the AI would reject impossible or fake questions. The models were evaluated across realistic conditions like strange camera angles, blocked views, and distracting surroundings. The results showed a huge gap between general AI vision skills and reliable physical measurement. The highest-performing model correctly got both the measurement number and unit right only 25.7% of the time.
Imagine you're in a busy kitchen trying to read a digital thermometer on a pot while multiple chefs are shouting and steam is rising. It's hard to see clearly and focus because of the chaos. Here, AI models face similar distractions when reading gauges in noisy industrial scenes, leading to frequent mistakes in getting the exact number and unit right.
Industries rely on accurate gauge readings for safety and efficiency. If AI can't do this reliably, it limits its use in automating monitoring tasks, potentially affecting operations and human workers.
The benchmark focuses on industrial scenes and may not generalize to other measurement contexts. It evaluates existing models but does not propose solutions to improve their accuracy. The study is limited to the specific instruments and noise conditions included in the dataset.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models — Chao Shen, Xinyuan Li, Yunfan Zhou et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.