AI explanations for heart rate estimation from video don't reliably signal accuracy.

Computers can measure your pulse just by analyzing video of your face. This technique is called remote photoplethysmographyremote photoplethysmographyA technique that estimates pulse or heart rate from video recordings of the skin, often the face.. Researchers often look at visual heatmapsheatmapsVisual representations that highlight areas of an image that an AI model focuses on. to guess which facial areas the computer uses to detect blood flow. However, people rarely test these visual explanations with hard numbers. Researchers tested an AI model named RhythmFormer across different video datasets and conditions like talking, head turning, cycling, and changing light levels. They measured whether the model actually looked at bare skin and whether its internal focus stayed faithful to the task. They found that a method called Beyond IntuitionBeyond IntuitionA specific explanation method used to attribute model decisions to input features. explained the model best across datasets. But surprisingly, checking where the model looked did not tell them if its heart-rate guess was correct for a specific video clip. Even when the AI focused directly on skin, its pulse calculations could still fail. Looking at the right place does not guarantee an accurate pulse measurement.

Why it matters

This matters for doctors and patients using AI health tools, because if explanations don't match real accuracy, decisions could be based on misleading information.

Questions answered