#355 Avoiding GPT’s Gender Bias
Friday Ship #355 | July 7th, 2023

This week we received feedback our GPT-powered meeting summarization feature misgendered a user and added work to our backlog to avoid the AI making these mistakes in the future.
AI models often reflect the biases of society. This sometimes results in hurtful moments when using products that incorporate AI. One of the first and most infamous cases was when Google’s Photos app mistagged two black subjects as ‘gorillas.’ This week at Parabol, we received this email into our support inbox from a user named Robin:
I’m not sure why the bot decided to add a gender-specific pronoun, but it is curious that it chose a masculine pronoun as the “default” – Women exist in IT and we use these tools too 🙂

This model version’s internal bias assumed Robin was male and decided to use the male pronouns when referring to Robin. The model was wrong. We apologize, Robin.
The code behind this summarization feature works by giving the ChatGPT 3.5 model the following prompt:
Below is a comma-separated list of text from a retro meeting. Summarize the text for a second-grade student in one or two sentences. If you can’t provide a summary, simply write the word “No”.
Text: New hire for cloud, SO to Robin, who has jumped right in and is asking great questions, identifying areas for improvement, etc., Welcome Robin!, Robin Joining!, Robin Joining the team
Avoiding gender bias when engineering GPT prompts
When creating AI features, the chances of exposing this sort of bias in the model can be avoided by asking ChatGPT to not make gender assumptions. For example, the prompt could be adjusted like so:
Below is a comma-separated list of text from a retro meeting. Summarize the text for a second-grade student in one or two sentences. When referring to people in the summary, do not assume their gender and default to using the pronouns “they” and “them.” If you can’t provide a summary, simply write the word “No”.
Text: New hire for cloud, SO to Robin, who has jumped right in and is asking great questions, identifying areas for improvement, etc., Welcome Robin!, Robin Joining!, Robin Joining the team
This yields a gender-neutral summary:
A new person named Robin joined the team for working on cloud stuff. They have been doing a great job by asking good questions and finding ways to make things better. Everyone welcomed Robin to the team.
Much better!
Catching the next mistakes
This example shows the importance for having user feedback controls in line with AI-generated content. AI models will get things wrong, frequently along the biases incorporated into the model when the model was trained. As a first step, we’ve planned a couple of small improvements to our AI summary features:
The extent to which AI is beginning to be used has never happened in the history of computing. It’s important for product makers to understand that even the best AIs will occasionally make mistakes (just as humans do). User expectations should be set accordingly and users granted agency to correct the summaries when they are wrong. Feedback from users should continuously be collected and monitored to maximize the number of times the AI is useful and to minimize the number of times an AI does harm.
Metrics

The slump product activity is expected this week: we’re now at the beginning of the productivity tool summer slump. We expect softening usage nearly to the end of August.
This week we…
…shipped v6.111.0. The update this week includes an improved credit card checkout functionality, an update to an experimental feature to use AI to form retrospective groups, and several small bug fixes and improvements
…concluded refreshing our company strategy. This week all member feedback was integrated and everybody had a chance to object to the strategy were it not safe to try.
…fixed some backend memory leaks that had been plaguing the product for a few months. There are still some small leaks in the Datadog telemetry our SaaS relies upon. We’re hoping we can convince Datadog (or its community) to help us look into it.
Next week…
…circles and squads will set key results to be collected and reported on each week
Have feedback? See something that you like or something you think could be better? Please write to us.