Bias

Introduction

With the rapid development of new language models like ChatGPT and image generation models like DALL-E, artificial intelligence poses more untapped potential than ever before. However, as these models have been developed and utilized further, the potential for bias has also drastically increased. Today, one of the biggest problems plaguing artificial intelligence is not only how to build well-performing models, but how to build well-performing models that are free from bias. Whether it be through resume tracking softwares that deny equally qualified applicants of economic opportunities or healthcare models that deny patients of life-saving care, bias in AI poses a significant risk to millions. In a recent study that I led, our group set out to measure race and gender bias prevalence in AI text-to-image models, namely on the popular model Stable Diffusion from Stability AI.

Text-To-Image Models

Text-to-image models have been gaining significant ground in the field of artificial intelligence in the past few years. Namely, Stable Diffusion, the model we used in our study, is trained on more than 2.3 billion images and uses a technique known as “latent diffusion.” Through this technique, the model can take any input of text and generate an original image in mere seconds. Despite the huge technological feats these models present, biases are often hard to identify, as many artificial intelligence models are a “blackbox,” with their inner workings almost impossible to decipher. However, by generating thousands of images with prompts designed to taunt out racial and gender stereotypes, we were able to properly identify bias that is deep rooted within these text-to-image models.

The Experiment

We begin the experiment with generating two sets of 50 individual prompts, designed to coax potential biases within our text-to-image models, one set for action-based prompts and one for profession-based prompts. Our action-based set included prompts like “playing basketball” and “teaching a class,” while our profession-based set included prompts ranging from “CEO” to “caretaker.” Next, we passed these prompts into our Stable Diffusion model and generated a dataset of more than 2000 unique images. From there, we were able to use OpenFlamingo, a multimodal language model, to tag and identify the race and bias of the person that each unique image generated by the model contained. Finally, using tools like Pandas and Seaborn and data from the Bureau of Labor Statistics, we were able to compare Stable Diffusion’s gender and race distribution to real-world data and visualize any persistent biases we identified across the dataset.

The Results

Our results ultimately proved that racial and gender biases were persistent across the dataset. In our dataset, men made up more than 80% of post-bachelor higher education jobs, while women were the majority represented in jobs classified as requiring a high school diploma and bachelor’s degree. The model also tended to generate more faces classified as “Asian” for jobs corresponding with higher education levels, while faces classified as “Black” were more often associated with lower-education jobs. Across income levels, women were overrepresented in low income levels at 59%, while men made up 63% and 62% of mid-to-high income level jobs, respectively. Compared to real data from the Bureau of Labor Statistics, we found the distributions of gender and race from the model were comparable to their real-world distributions, with some gender biases magnified in certain occupations like “nurse” and “caretaker.”

Conclusions

Ultimately, we found that bias did exist within Stable Diffusion, as the model often portrayed certain stereotypes against gender and race. For example, the model tended to overrepresent women in nurturing roles like “caretaker” and “nurse,” while overrepresenting men in leadership roles like “CEO” and “politician.” Moreover,  Asians tended to be overrepresented in higher-income, higher education jobs and actions, while other POC, such as Native Americans and Pacific Islanders, were significantly underrepresented, if at all, across the entire dataset. While it is undeniable that significant headway has been made in the field of artificial intelligence in just the past couple of years, it is important to ensure that the models that we continue to develop and deploy in the real world do not exhibit and accentuate harmful biases to the people they are meant to help.