Original title: Statistically grounded evaluation of demographic bias in deep facial representations using FairFace and UTKFace
Andisani Nemavhola, Serestina Viriri, Colin Chibaya · September 2026
Facial recognition systems can achieve high overall accuracy while still performing poorly for certain demographic groups. Researchers created a testing framework to measure this unfairness. They tested three computer vision models trained on images to see how well their learned representations handle faces. They ran tests on two face datasets: one balanced across groups and one unconstrained. To see how faces are organized inside the models, they looked at raw nearest neighbors and also trained simple linear classifiers to separate demographic traits. They found that simple classifiers consistently beat nearest-neighbor sorting, showing that models naturally encode demographic traits in clean, separable patterns. ResNet50 had the best overall accuracy, but MobileNetV3 gave more consistent results across different subgroups. Looking at race and gender together proved that deep disparities remain even when overall accuracy looks strong.
Imagine you are a teacher grading a class test. You look at the average score and see it is 85%, so you think the test was fair. But then you check each student's score and find that students from some neighborhoods scored much lower than others. Worse, when you look at subgroups like girls from a certain neighborhood, the difference is even larger. In this paper, the class average is the overall accuracy, the neighborhoods are demographic groups like race, and the subgroups are combinations like race and gender. The researchers show that even if the average is good, there can still be unfair differences that need to be examined.
Facial recognition systems are used in security, hiring, and policing. If models are less accurate for certain races or genders, it can lead to unfair treatment. This paper provides a way to measure bias statistically, so developers and regulators can identify disparities that average accuracy metrics miss.
This work evaluates only ImageNet-pretrained models on two datasets; results may not generalize to models trained from scratch or other face datasets. The framework relies on 1-NN and Logistic Probe, which may not capture all forms of bias. The study does not address debiasing methods, only evaluation. Statistical significance tests are used but the specific tests are not detailed in the abstract.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Statistically grounded evaluation of demographic bias in deep facial representations using FairFace and UTKFace — Andisani Nemavhola, Serestina Viriri, Colin Chibaya — Scientific Reports — doi:10.1038/s41598-026-71795-6
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.