Data and Model Drift Detection
Introduction
ML models degrade over time. The world changes, user behavior shifts, and the patterns
the model learned become stale. Drift detection identifies when this happens so you
can retrain before performance degrades too much.
There are two types of drift:
- Data drift: input feature distributions change
- Concept drift: the relationship between features and target changes
Why This Matters
Unlike traditional software that breaks obviously (errors, crashes), ML models fail
silently — they keep making predictions, just bad ones.
Without drift detection:
- Model accuracy degrades gradually
- By the time someone notices, damage is done
- You don't know when to retrain
With drift detection:
- You get early warning when distributions shift
- You can trigger retraining automatically
- You have evidence for why a model needs updating
Tasks
Hint: Compare production feature distributions to training distributions