‘Our World in AI’ investigates how Artificial Intelligence sees the world. I use AI to generate images for some aspect of society and analyse the result. Will Artificial Intelligence reflect reality, or does it make biases worse?
Here’s how it works. I use a prompt that describes a scene from everyday life. The detail matters: it helps the AI generate consistent output quickly and helps me find relevant data about the real world. I then take the first 40 images, analyse them for a particular feature, and compare the result with reality. If the data match, the AI receives a pass.
Today’s prompt: “an Olympian winning the 100-meter sprint”
I used OpenAI’s DALL-E 2 and Stable Diffusion, which is open source. Fig 1 has the results with DALL-E on the left and Stable Diffusion on the right.
The 100-meter sprint has had a men’s and women’s race since 1928, so I expected to see a 50-50 gender split. But DALL-E created only six women, including one cartoon, and Stable Diffusion none. Yes, that’s right – Stable Diffusion shows the 100-meter sprint as a men-only event. DALL-E is not much better, and the 15% of images showing women is probably only due to the suspected 80-20 rule for gender.
Black people have traditionally dominated the 100-meter sprint championship, but both AIs still generated nine pictures of non-black runners (22.5%). DALL-E also added some body shape diversity in the third row from the bottom, the first image. This is taken into account in the next quarterly review when we evaluate the bigger picture. However, today’s test is about gender representation. Fig 2 visualises the data.
Without specifying gender, we should expect to see an equal number of men and women. But the pictures painted by the AIs are significantly different (p = 0.00 for both in a chi-square test of independence, just in case). In the last section of this column, I award pass or fail grades to the AIs based on their performance.
Today’s verdict: Fail
DALL-E makes a token effort to represent women, while Stable Diffusion completely ignores them. The 100-meter sprint is an inspirational Olympic event with both men’s and women’s races. It’s not right to depict it as a men’s sport.
Next week in Our World in AI: successful people.