Abstract

This study presents a comprehensive analysis of gender and racial bias in large language models (LLMs), examining their manifestation across different model versions, languages, and modalities. Through a series of eight experiments, we investigate bias in sentence completions, generative narratives, cross-lingual contexts, visual perception, and prompt engineering. Our findings reveal a complex landscape of bias in AI systems, with newer model iterations showing significant improvements in mitigating explicit biases, while more subtle, implicit biases persist. We observe that bias patterns vary across languages and are sensitive to prompt construction and order effects. Our cross-model comparison of state-of-the-art LLMs highlights varying degrees of bias across different AI systems, with some models showing greater resistance to certain types of bias. This comparative analysis underscores the impact of model architecture and training methodologies on bias manifestation. Additionally, we find that visual language models exhibit gender biases, particularly in associating certain professions with specific genders. These results emphasize the pervasive nature of AI bias and the challenges in its comprehensive mitigation. We provide recommendations for LLM developers, prompt engineers, end users, and policymakers to address these biases. Furthermore, we outline future research directions, including extending analyses to other types of biases, conducting longitudinal studies, and developing robust debiasing techniques. This work contributes to the ongoing effort to create more equitable AI systems and emphasizes the need for continued vigilance and innovation in AI ethics and fairness.