Week 5 · Lesson 1 · 50-60 minutes
Audit a Dataset for Fairness
Learn how uneven examples create bias, read group-level results to see who a system works well for, and audit a dataset for fairness.
- Materials
- This lesson in a web browser · Paper and pencil, or a notes app
- This lesson
- In progress
Auto-saves on this device
What you'll be able to do
- Define bias and representation and explain how they are connected.
- Read a group-level result to compare how well a system works for different groups.
- Explain what fairness means when an AI treats groups differently.
- Audit a described dataset and name one way its examples are unbalanced.
Think about it
A voice assistant understands one friend almost every time but keeps mishearing another friend who has a different accent. The software is the same for both. Why might it work so differently for the two of them?
Keep your first guess in mind — you will test it against real group-level numbers later in the lesson.
Make a prediction
Predict: if a face-recognition system learned mostly from photos of adults, do you think it will work equally well on photos of children?
Vocabulary
See the full course vocabulary- Bias
- When a system regularly works better or worse for some people or things than others, usually because of the examples it learned from.
- Representation
- How much a group shows up in a dataset. A group is well represented if it has many examples, and underrepresented if it has few.
- Group-level result
- A score measured separately for each group, instead of one number for everyone, so you can see who a system works well for.
- Fairness
- Treating groups in a way people would agree is even and reasonable, so one group is not left worse off without a good reason.
Bias comes from the examples, not from the computer 'wanting' anything
A computer does not have opinions or feelings. When we say an AI is biased, we mean it regularly works better for some people or things than others. Almost always, that happens because of the examples it learned from.
If a system learns from examples that are uneven, it becomes good at the common cases and weaker at the rare ones. The software is doing exactly what it was trained to do — the unevenness came from the data people gave it.
- A speech system trained mostly on one accent understands that accent best.
- A photo classifier trained mostly on daytime pictures struggles at night.
- A translation tool trained on formal writing handles slang poorly.
Representation: who is in the examples, and how much
Representation is about how much each group shows up in the data. A group is well represented when it has many examples and underrepresented when it has few. A dataset can have thousands of examples and still leave some groups with almost none.
Underrepresentation is one of the biggest sources of bias. A model can only learn patterns it has seen. If a group barely appears in the examples, the model has little to learn from, so it tends to make more mistakes for that group.
- A dataset of 10,000 dog photos with only 20 photos of one breed.
- A medical dataset collected mostly from adults, with few children.
- A handwriting dataset with lots of print but very little cursive.
Group-level results show what one overall score hides
It is tempting to judge a system by a single number, like 'it is 92 percent accurate.' But one number is an average across everyone. It can look great while still failing badly for a smaller group whose mistakes get buried in the average.
A group-level result measures accuracy separately for each group. When you break the score apart, you can see who the system works well for and who it works poorly for. Fairness questions almost always need group-level results, not just one overall number.
- Overall 92 percent, but only 74 percent for one accent group.
- Overall high accuracy, but far more errors on night photos than day photos.
- A test that passes on average yet fails often for left-handed writers.
Fairness: is anyone left worse off without a good reason?
Fairness means treating groups in a way people would agree is even and reasonable. It does not always require identical results, but it does ask whether one group is being left worse off, and whether there is a good reason for the gap.
Fairness is a human judgment, not something the computer decides. People have to look at the group-level results, decide whether a gap is acceptable, and take responsibility for fixing it — often by collecting more balanced examples.
- Deciding a 74 percent versus 92 percent accuracy gap is too large to ship.
- Choosing to gather more examples from underrepresented groups.
- Asking who is harmed if the system makes a mistake for a particular group.
Worked example
Auditing a smile-detector dataset
- Name the task. A camera app tries to snap a photo when everyone is smiling. Input: a face. Output: 'smiling' or 'not smiling'.
- Look at representation. The training set has 8,000 photos of adults and only 500 photos of young children.
- Predict the effect. With so few child examples, the model has little to learn about how children smile, so it will likely make more mistakes on children.
- Check the group-level result, not just the average. Overall accuracy is 90 percent, but split by group it is 94 percent for adults and 71 percent for children.
- Make the fairness call. The gap is large and children are worse off with no good reason, so the fair fix is to collect more child examples and re-test before shipping.
Takeaway: Audit a dataset by asking who is underrepresented, then check group-level results to see whether that unevenness turned into unfair mistakes.
Values shown in % accuracy.
A bar chart of a speech-recognition system's accuracy measured separately for four accent groups, in percent. The overall average is 88 percent, but the groups are uneven: Accent group A 94 percent, Accent group B 91 percent, Accent group C 82 percent, and Accent group D 74 percent. Groups C and D, which had the fewest training examples, score far below the average, so the single overall number hides a real gap of about 20 points between the best and worst groups.
Break the average apart, and the system clearly works better for some groups than others.
| Accent group | Training examples | Accuracy |
|---|---|---|
| A | 5,000 | 94% |
| B | 4,200 | 91% |
| C | 900 | 82% |
| D | 400 | 74% |
A table linking how many training examples each accent group had to its accuracy. Group A: 5,000 examples, 94 percent accuracy. Group B: 4,200 examples, 91 percent. Group C: 900 examples, 82 percent. Group D: 400 examples, 74 percent. The groups with the fewest examples have the lowest accuracy, showing how underrepresentation turns into bias.
Activity
Fairness Audit
Audit a fictional school's STEM-program recommender: see how an overall accuracy hides a weak result for a smaller group, then fix it by limiting a proxy feature and adding under-represented examples.
- Read the overall accuracy — then look at the accuracy for each group. They are not the same.
- The model leans on a misleading proxy feature. Lower its weight, and add examples for the under-represented group.
- Rerun the audit and compare with the flawed baseline. Improving fairness is progress, not a guarantee.
Fictional scenario: a school recommends after-school STEM programs. Every student, group, and neighborhood here is made up. Two groups: Hillside (larger, near campus) and Riverside (smaller, farther away).
Overall accuracy
63%
5 of 8 test students predicted correctly.
Accuracy by group
False approvals: 0 · Missed good-fit students: 0 · Training examples: 10
False approvals: 0 · Missed good-fit students: 3 · Training examples: 6
The overall number can look fine while one group does much worse — always read the group-level results.
Before vs. after
| Measure | Flawed baseline | Your version |
|---|---|---|
| Overall accuracy | 63% | 63% |
| Hillside accuracy | 100% | 100% |
| Riverside accuracy | 25% | 25% |
Lowering the proxy weight or adding Riverside examples raises Riverside's accuracy. Note that a change can also shift another group's result — improving representation is real progress, but it does not guarantee perfect fairness.
Check your understanding
Answer these to check that you can spot bias, read group-level results, and judge fairness.
Question 1. Where does bias in an AI system most often come from?
Question 2. Decide if the statement is true or false.
A single overall accuracy number is enough to prove that a system is fair to every group.
Question 3. Based on the group-level results, what is the most reasonable next step?
A photo app reports 90 percent accuracy overall. Broken down by group, it is 95 percent for adults and 70 percent for children. The training set had very few photos of children.
0 of 3 answered
Try it yourself
Design a fairer dataset
Pick a simple AI task and plan a dataset that would be fair across groups before any model is trained.
- Choose a task, such as reading handwriting or recognizing pets.
- List the groups the system should work well for.
- Decide roughly how many examples each group needs so none is underrepresented.
- Write how you would check the result with a group-level accuracy, not just an overall average.
Success looks like
- At least three groups the system should serve are named.
- A plan gives each group enough examples to avoid underrepresentation.
- The plan checks fairness using group-level results.
Reflection
Saved on this device as you type
Please don't include private information. Your answers are saved only in this browser and are never sent anywhere.
Recap
Bias usually comes from uneven representation, and only group-level results reveal whether a system is fair to every group.
- Bias means a system regularly works better for some groups than others, usually because of its examples.
- Underrepresented groups tend to get more mistakes because the model saw fewer of them.
- A single overall score can hide a real gap; group-level results show who the system works for.
- Fairness is a human judgment about whether a gap is acceptable and how to fix it.
Grades 7–8 extension
When equal accuracy still is not fair
Sometimes two groups have the same accuracy, but the mistakes matter more for one of them. Imagine a system that flags library books as 'overdue.' A wrong flag might be a minor annoyance for one group but block another group from borrowing at all.
Pick a decision an AI might make, and describe a case where equal accuracy across groups still leads to an unfair outcome because the cost of a mistake is not equal. What extra information would you need to notice that?
Finish this lesson
- Attempt the knowledge check — not done
- Save a reflection (recommended) — not done
Attempt the knowledge check to finish this lesson.