Abstract

Here, we ask whether the most common approaches used to identify demographic biases in artificial intelligence (AI) are also biased. We conducted a Scoping Review of papers indexed in Scopus and WoS on biases in a particular AI application (face recognition). Fourteen original articles met our inclusion criteria. Of these, the vast majority (13) used an a priori approach to identify bias, i.e., they started from a known background in which social groups were subject to low accuracy by the algorithms. Only one study found bias a posteriori, i.e., they examined the results without underlying assumptions about the discriminated groups. Remarkably, this single article identified that it was workers who suffered the negative effects of face recognition, a social segment not analyzed by any study using an aprioristic approach. Of the aprioristic studies, 79% examined skin color and ethnicity, 50% analyzed gender, and two (14%) studied age. Only two articles analyzed bias on-the-ground, while most focused on experiments. We argue that the almost exclusive use of the common approach (aprioristic and experimental designs) to identify systematic errors is a methodological bias. This precludes knowledge of other discriminated social groups or even biases towards humanity as a whole that have never been identified (deep-rooted biases), since their awareness depends on the historical context. To better describe AI models, we believe that eXplainable Artificial Intelligence (xAI) tools should work together with a posteriori bias identification strategies and the measurement of their direct effects on citizens' lives.