Week 2 · Lesson 3 · 45-60 minutes
Repair the Dataset
Play data detective: hunt down duplicates, incorrect labels, and unbalanced categories, then fix them so a model can learn a fair, accurate pattern.
- Materials
- This lesson in a web browser · Paper and pencil, or a notes app · The built-in messy dataset shown in this lesson
- This lesson
- In progress
Auto-saves on this device
What you'll be able to do
- Spot common data problems: duplicates, incorrect labels, and unbalanced categories.
- Explain how each problem can hurt a model's accuracy or fairness.
- Repair a small dataset so its examples are correct and its categories are balanced.
Think about it
A student is teaching a model to tell apples from bananas. Their table has 18 apples and only 2 bananas, one banana is copied three times, and one apple is labeled 'banana' by mistake. What could go wrong when the model learns from this?
By the end of the lesson you will be able to name and fix all three problems.
Make a prediction
Predict: if a dataset has 18 apples and only 2 bananas, which category do you think the model will learn better, and why?
Vocabulary
See the full course vocabulary- Balanced data
- A dataset where each category has a fair, roughly equal number of examples, so the model learns every category well.
- Duplicate
- An example that appears more than once in a dataset, which can make the model over-focus on that repeated case.
- Incorrect label
- An example given the wrong answer, like a banana labeled 'apple', which teaches the model a false pattern.
- Accuracy
- The share of examples a model labels correctly; bad data usually lowers it.
- Generalization
- How well a model works on new examples; clean, balanced data helps a model generalize.
Duplicates: the same example counted twice
A duplicate is an example that shows up more than once. It might look harmless, but a duplicated row makes the model treat that one case as if it were many, giving it too much weight.
If one exact banana appears three times, the model may over-focus on that specific banana instead of learning what bananas look like in general. Removing duplicates keeps each real example counting once.
- The identical banana row copied three times in the table
- The same photo pasted into a dataset twice
Incorrect labels: the wrong answer teaches the wrong pattern
An incorrect label is an example marked with the wrong category — like an apple accidentally labeled 'banana'. Because a model trusts its labels as the truth, a wrong label teaches it a false pattern.
Even a few incorrect labels can lower accuracy, because the model tries to satisfy answers that were never right. Fixing a label means changing it to the correct category (or removing the example if you can't tell).
- A round red fruit labeled 'banana'
- A cat photo labeled 'dog' in an animal dataset
Balanced data: give every category a fair share
Balanced data means each category has a fair, roughly equal number of examples. If a dataset has 18 apples but only 2 bananas, the model sees far more apples and may barely learn what a banana looks like — so it guesses 'apple' too often.
Balancing the categories — by collecting more banana examples, or trimming down the apples — helps the model learn every category and generalize fairly. Clean, correct, balanced data is the foundation of an accurate model.
- 10 apples and 10 bananas is more balanced than 18 apples and 2 bananas
- A model trained mostly on one category tends to over-predict that category
Worked example
Repairing a messy fruit table step by step
- Start with a messy table: 18 apples and 2 bananas, one banana row copied three times, and one apple labeled 'banana'.
- Find duplicates: the identical banana appears three times, so remove two copies and keep one.
- Fix incorrect labels: the round red fruit labeled 'banana' is really an apple, so change its label to 'apple'.
- Check balance: now the categories are very lopsided, so add more real banana examples until apples and bananas are roughly even.
- Result: a clean, correctly labeled, balanced dataset the model can learn a fair pattern from.
Takeaway: Repair a dataset by removing duplicates, fixing wrong labels, and balancing the categories before a model learns from it.
Values shown in examples.
A bar chart of how many examples each category has, counted in examples. Before repair: Apple 18, Banana 2 — very unbalanced. After repair: Apple 10, Banana 10 — balanced. Balancing the categories gives the model a fair chance to learn both fruits instead of over-predicting apple.
| Problem | How to repair | Why it matters |
|---|---|---|
| Duplicate | Remove the extra copies | One case gets too much weight |
| Incorrect label | Change it to the correct category | A wrong answer teaches a false pattern |
| Unbalanced categories | Add to the small category or trim the large one | The model under-learns the rare category |
A table matching each data problem to its fix and why it matters. Duplicate: an example repeated; fix by removing extra copies; matters because it over-weights one case. Incorrect label: wrong answer on an example; fix by correcting the label; matters because it teaches a false pattern. Unbalanced categories: one category has far more examples; fix by adding to the smaller category or trimming the larger; matters because the model under-learns the rare category.
Activity
Repair the Flawed Dataset
Fix a messy space-fruit dataset — remove duplicates, correct wrong labels, and improve balance — then re-run the model and compare before and after.
- Inspect the flawed dataset. Remove duplicate rows and fix any labels that look wrong.
- Improve the balance by adding more examples of the under-represented category.
- The testing fruit stays hidden and unchanged. Check and re-run the model to compare before and after.
Rows
18
Safe / Not safe
14 / 4
Duplicate rows
4
Wrong labels
?
Duplicate rows are detected automatically (identical features). Wrong labels stay hidden until you check — investigate first. The 8 testing fruit are held back and never edited here.
| Fruit | Color | Shape | Texture | Grows in | Seeds | Sweetness | Glow level | Size | Label / remove |
|---|---|---|---|---|---|---|---|---|---|
| DawnberryDuplicate of Dawnberry (copy) | crimson | round | smooth | canopy | 6 | 8 | 0 | 10 cm | |
| Sandplum | amber | oval | bumpy | dune | 4 | 7 | 1 | 8 cm | |
| Tidefruit | teal | round | fuzzy | reef | 8 | 9 | 0 | 12 cm | |
| Emberpear | crimson | oval | smooth | canopy | 5 | 6 | 2 | 9 cm | |
| Dunedrop | amber | round | bumpy | dune | 3 | 7 | 1 | 7 cm | |
| Canopyfig | teal | oval | fuzzy | canopy | 7 | 8 | 0 | 11 cm | |
| Redgourd | crimson | round | smooth | dune | 6 | 5 | 2 | 10 cm | |
| Reefplum | amber | oval | fuzzy | reef | 4 | 9 | 1 | 8 cm | |
| Greenpod | teal | round | bumpy | canopy | 9 | 6 | 0 | 13 cm | |
| Amberbud | amber | round | fuzzy | canopy | 2 | 7 | 2 | 6 cm | |
| StarflareDuplicate of Starflare (copy) | violet | star | spiky | crater | 2 | 3 | 4 | 5 cm | |
| Glowcoil | azure | spiral | bumpy | cavern | 1 | 2 | 5 | 4 cm | |
| Dawnberry (copy)Duplicate of Dawnberry | crimson | round | smooth | canopy | 6 | 8 | 0 | 10 cm | |
| Starflare (copy)Duplicate of Starflare | violet | star | spiky | crater | 2 | 3 | 4 | 5 cm | |
| Voidstar | violet | spiral | smooth | crater | 0 | 3 | 5 | 5 cm | |
| Thornstar | azure | star | spiky | cavern | 3 | 2 | 2 | 5 cm | |
| Cavernglow | azure | star | bumpy | cavern | 2 | 1 | 4 | 7 cm | |
| Sweetreef | teal | oval | smooth | reef | 6 | 9 | 0 | 10 cm |
Dawnberry
Duplicate of Dawnberry (copy)- Color
- crimson
- Shape
- round
- Texture
- smooth
- Grows in
- canopy
- Seeds
- 6
- Sweetness
- 8
- Glow level
- 0
- Size
- 10 cm
Sandplum
- Color
- amber
- Shape
- oval
- Texture
- bumpy
- Grows in
- dune
- Seeds
- 4
- Sweetness
- 7
- Glow level
- 1
- Size
- 8 cm
Tidefruit
- Color
- teal
- Shape
- round
- Texture
- fuzzy
- Grows in
- reef
- Seeds
- 8
- Sweetness
- 9
- Glow level
- 0
- Size
- 12 cm
Emberpear
- Color
- crimson
- Shape
- oval
- Texture
- smooth
- Grows in
- canopy
- Seeds
- 5
- Sweetness
- 6
- Glow level
- 2
- Size
- 9 cm
Dunedrop
- Color
- amber
- Shape
- round
- Texture
- bumpy
- Grows in
- dune
- Seeds
- 3
- Sweetness
- 7
- Glow level
- 1
- Size
- 7 cm
Canopyfig
- Color
- teal
- Shape
- oval
- Texture
- fuzzy
- Grows in
- canopy
- Seeds
- 7
- Sweetness
- 8
- Glow level
- 0
- Size
- 11 cm
Redgourd
- Color
- crimson
- Shape
- round
- Texture
- smooth
- Grows in
- dune
- Seeds
- 6
- Sweetness
- 5
- Glow level
- 2
- Size
- 10 cm
Reefplum
- Color
- amber
- Shape
- oval
- Texture
- fuzzy
- Grows in
- reef
- Seeds
- 4
- Sweetness
- 9
- Glow level
- 1
- Size
- 8 cm
Greenpod
- Color
- teal
- Shape
- round
- Texture
- bumpy
- Grows in
- canopy
- Seeds
- 9
- Sweetness
- 6
- Glow level
- 0
- Size
- 13 cm
Amberbud
- Color
- amber
- Shape
- round
- Texture
- fuzzy
- Grows in
- canopy
- Seeds
- 2
- Sweetness
- 7
- Glow level
- 2
- Size
- 6 cm
Starflare
Duplicate of Starflare (copy)- Color
- violet
- Shape
- star
- Texture
- spiky
- Grows in
- crater
- Seeds
- 2
- Sweetness
- 3
- Glow level
- 4
- Size
- 5 cm
Glowcoil
- Color
- azure
- Shape
- spiral
- Texture
- bumpy
- Grows in
- cavern
- Seeds
- 1
- Sweetness
- 2
- Glow level
- 5
- Size
- 4 cm
Dawnberry (copy)
Duplicate of Dawnberry- Color
- crimson
- Shape
- round
- Texture
- smooth
- Grows in
- canopy
- Seeds
- 6
- Sweetness
- 8
- Glow level
- 0
- Size
- 10 cm
Starflare (copy)
Duplicate of Starflare- Color
- violet
- Shape
- star
- Texture
- spiky
- Grows in
- crater
- Seeds
- 2
- Sweetness
- 3
- Glow level
- 4
- Size
- 5 cm
Voidstar
- Color
- violet
- Shape
- spiral
- Texture
- smooth
- Grows in
- crater
- Seeds
- 0
- Sweetness
- 3
- Glow level
- 5
- Size
- 5 cm
Thornstar
- Color
- azure
- Shape
- star
- Texture
- spiky
- Grows in
- cavern
- Seeds
- 3
- Sweetness
- 2
- Glow level
- 2
- Size
- 5 cm
Cavernglow
- Color
- azure
- Shape
- star
- Texture
- bumpy
- Grows in
- cavern
- Seeds
- 2
- Sweetness
- 1
- Glow level
- 4
- Size
- 7 cm
Sweetreef
- Color
- teal
- Shape
- oval
- Texture
- smooth
- Grows in
- reef
- Seeds
- 6
- Sweetness
- 9
- Glow level
- 0
- Size
- 10 cm
Add examples to improve balance
These extra, correctly-labeled fruits are available to add. Adding more of the rare category helps the model learn it.
- Sparkstar — Not safe to eat. A 6 cm violet star fruit with smooth skin and 1 seeds, sweetness 4 of 10, glow level 3 of 5, growing in a crater.
- Coilspine — Not safe to eat. A 4 cm azure spiral fruit with spiky skin and 2 seeds, sweetness 2 of 10, glow level 1 of 5, growing in a cavern.
- Brightcoil — Not safe to eat. A 5 cm azure spiral fruit with bumpy skin and 1 seeds, sweetness 1 of 10, glow level 4 of 5, growing in a cavern.
- Thornflare — Not safe to eat. A 6 cm violet star fruit with spiky skin and 3 seeds, sweetness 3 of 10, glow level 2 of 5, growing in a crater.
- Duskstar — Not safe to eat. A 5 cm violet star fruit with fuzzy skin and 0 seeds, sweetness 4 of 10, glow level 3 of 5, growing in a crater.
- Nightcoil — Not safe to eat. A 4 cm azure spiral fruit with spiky skin and 2 seeds, sweetness 2 of 10, glow level 2 of 5, growing in a cavern.
- Meadowfig — Safe to eat. A 11 cm teal oval fruit with fuzzy skin and 7 seeds, sweetness 8 of 10, glow level 1 of 5, growing in a canopy.
- Sunplum — Safe to eat. A 9 cm amber oval fruit with smooth skin and 4 seeds, sweetness 8 of 10, glow level 2 of 5, growing in a dune.
The made-up classroom rule: A space fruit is Not safe to eat if it glows brightly (glow level 3 or more) OR it is spiky. Otherwise it is Safe to eat. “Wrong label” means the label does not match this rule — not a real-world fact.
Check your understanding
Answer these to check that you can spot and fix data problems.
Question 1. Which of these are problems you should fix before training a model? Choose all that apply.
Select all that apply.
Question 2. Decide if the statement is true or false.
A dataset with 18 apples and only 2 bananas is well balanced.
Question 3. Put the dataset-repair steps in a sensible order.
- 1.Inspect the dataset to find problems
- 2.Remove duplicate rows and correct wrong labels
- 3.Balance the categories
- 4.Retrain the model and check its accuracy
0 of 3 answered
Try it yourself
Break it, then fix it
Take a clean tiny dataset, add each of the three problems on purpose, then repair it and describe the effect.
- Start with a small, clean, balanced dataset of two categories.
- Add a duplicate row, change one label to be incorrect, and remove examples so one category is much larger.
- Now repair all three problems and explain, for each, how the fix should help the model's accuracy or fairness.
Success looks like
- All three problems are added on purpose and clearly marked.
- All three are repaired.
- Each repair has a reason tied to accuracy or fairness.
Reflection
Saved on this device as you type
Please don't include private information. Your answers are saved only in this browser and are never sent anywhere.
Recap
Duplicates, incorrect labels, and unbalanced categories all hurt a model; repairing them produces clean, balanced data that improves accuracy and helps the model generalize fairly.
- Remove duplicates so no single example is over-weighted.
- Correct incorrect labels so the model does not learn a false pattern.
- Balance the categories so the model learns each one and generalizes fairly.
Grades 7–8 extension
When you can't collect more data
Balancing by adding more examples of the rare category is ideal, but sometimes you cannot easily collect more. Another option is to trim the larger category down so the counts match — though that means throwing away some real examples.
For the messy fruit table, argue which approach you would choose — add more bananas or remove some apples — and explain the trade-off each choice makes for the model's accuracy and fairness.
Finish this lesson
- Attempt the knowledge check — not done
- Save a reflection (recommended) — not done
Attempt the knowledge check to finish this lesson.