#355 Avoiding GPT’s Gender Bias

This week we received feedback our GPT-powered meeting summarization feature misgendered a user and added work to our backlog to avoid the AI making these mistakes in the future.

AI models often reflect the biases of society. This sometimes results in hurtful moments when using products that incorporate AI. One of the first and most infamous cases was when Google’s Photos app mistagged two black subjects as ‘gorillas.’ This week at Parabol, we received this email into our support inbox from a user named Robin:

This model version’s internal bias assumed Robin was male and decided to use the male pronouns when referring to Robin. The model was wrong. We apologize, Robin.

The code behind this summarization feature works by giving the ChatGPT 3.5 model the following prompt:

When creating AI features, the chances of exposing this sort of bias in the model can be avoided by asking ChatGPT to not make gender assumptions. For example, the prompt could be adjusted like so:

This yields a gender-neutral summary:

Much better!

This example shows the importance for having user feedback controls in line with AI-generated content. AI models will get things wrong, frequently along the biases incorporated into the model when the model was trained. As a first step, we’ve planned a couple of small improvements to our AI summary features:

The extent to which AI is beginning to be used has never happened in the history of computing. It’s important for product makers to understand that even the best AIs will occasionally make mistakes (just as humans do). User expectations should be set accordingly and users granted agency to correct the summaries when they are wrong. Feedback from users should continuously be collected and monitored to maximize the number of times the AI is useful and to minimize the number of times an AI does harm.

The slump product activity is expected this week: we’re now at the beginning of the productivity tool summer slump. We expect softening usage nearly to the end of August.

…shipped v6.111.0. The update this week includes an improved credit card checkout functionality, an update to an experimental feature to use AI to form retrospective groups, and several small bug fixes and improvements

…concluded refreshing our company strategy. This week all member feedback was integrated and everybody had a chance to object to the strategy were it not safe to try.

…fixed some backend memory leaks that had been plaguing the product for a few months. There are still some small leaks in the Datadog telemetry our SaaS relies upon. We’re hoping we can convince Datadog (or its community) to help us look into it.

…circles and squads will set key results to be collected and reported on each week

Have feedback? See something that you like or something you think could be better? Please write to us.