Feature drift and concept drift are the two dominant failure modes in production ML, and they are frequently conflated because they can look similar in their symptom: model predictions start diverging from reality. The confusion matters because they require different detection approaches and — critically — different remediation strategies. Responding to concept drift as if it were feature drift wastes a retrain cycle without fixing the underlying problem.
The definitions that actually distinguish them
Feature drift (also called covariate shift or input drift) is a change in the distribution of your model's input features. The relationship between features and outcomes that your model learned during training is still valid — but the distribution of inputs your model now sees in production has shifted away from the distribution it was trained on. A credit risk model trained on borrowers with median monthly income of $4,200 that now serves a population with median income of $5,800 is experiencing feature drift.
Concept drift is a change in the underlying relationship between inputs and outputs — the conditional probability P(Y|X) has changed, not just P(X). The real world that your model was built to predict has structurally changed. A fraud detection model trained before a specific type of fraud attack became prevalent will experience concept drift even if the input feature distributions haven't shifted — the inputs look the same, but what constitutes fraud has changed.
Feature drift: P(X) changed. Concept drift: P(Y|X) changed. This is the cleanest way to hold the distinction.
Detection: what to measure for each
Feature drift is detectable without ground truth labels. Because you're measuring the distribution of inputs, you can compute drift metrics (PSI, KL divergence, Wasserstein) directly on your feature vectors using the training distribution as the baseline. No knowledge of actual outcomes is required. This makes feature drift the more immediately detectable of the two: you can compute it in near-real-time on your prediction traffic.
Concept drift requires ground truth labels to detect — or a proxy for them. If you have delayed label feedback (actual outcomes arriving hours, days, or weeks after predictions are made), you can compute model accuracy metrics on labeled data and compare them against the training-period baseline. A consistent downward trend in accuracy or F1 on labeled data, without a corresponding feature distribution shift, is a strong signal of concept drift.
When delayed labels aren't available (which is common — ground truth for a credit default might not arrive for 18 months), concept drift detection relies on proxy signals: prediction confidence distribution shifts (the model starts returning predictions it's less certain about), output distribution shifts (the distribution of predicted scores changes), or population-level outcome proxies (external signals that the real-world phenomenon has changed).
The confounding case: both happening simultaneously
In practice, feature drift and concept drift often co-occur, and distinguishing them when they're entangled is genuinely difficult. Consider an e-commerce recommendation model experiencing an unusual holiday season that brings a new demographic of buyers with different purchasing patterns. The input features are drifting (new user cohort with different browsing history distributions) and the concept is potentially drifting (what constitutes a "good" recommendation for this cohort differs from the training population). The model's accuracy drops, but the root cause is ambiguous.
The practical diagnostic is to compare the two signals: if PSI on input features is high and accuracy on labeled samples is dropping proportionally to what the input shift would predict, feature drift is the primary explanation and a retrain on the new distribution should resolve it. If PSI on input features is modest but accuracy drop is disproportionately large — meaning the model is performing worse than the input shift alone would explain — concept drift is the more likely explanation and a retrain requires investigation into whether the training data adequately represents the new relationship between inputs and outcomes.
Remediation: why they require different responses
Feature drift remediation: collect new training data from the current input distribution and retrain. The model architecture, feature engineering, and training objective can stay largely the same — you just need the model to re-learn the same relationship on the new distribution of inputs. If you have a rolling retrain policy with drift triggers, a feature-drift-triggered retrain is the expected response and the policy should be tuned to the historical drift velocity.
Concept drift remediation is structurally harder. Retraining on new data may help if you have labeled examples from the new real-world regime, but it requires examining whether your feature set is still adequate for predicting the new real-world relationship. Sometimes concept drift signals that you need new features — signals from the real world that your model wasn't using but that now carry significant predictive information.
We're not saying feature drift is trivial and concept drift is catastrophic — both require response. We're saying that treating concept drift as a simple retrain problem can lead to a retrain loop where you retrain repeatedly on new data without recovering model performance, because the real problem is a missing feature or a fundamentally changed relationship that more data on the old feature set won't fix.
Governance implications: different drift types need different policy responses
A retrain policy that treats all drift signals as equivalent doesn't distinguish between "the input distribution shifted, retrain the model" and "the world has changed in a way that may require reconsidering the model design." The former is a routine engineering response. The latter should trigger a review that includes the ML Lead and potentially the product owner, not just an automated retrain pipeline execution.
One way to encode this distinction in policy: tier your drift signals by type and required response. PSI-triggered feature drift fires a standard retrain proposal that goes through the normal approval chain. Concept drift signals — accuracy drops on labeled data that are disproportionate to input drift — fire a "design review" proposal that requires ML Lead plus product owner involvement and a documented decision about whether retraining, feature engineering changes, or model redesign is the appropriate response.
This tiering prevents the governance failure mode where a concept drift event goes unexamined because it was categorized as a routine retrain trigger, gets processed through the standard approval chain, and produces a retrained model that still underperforms because nobody stopped to ask whether the training objective was still the right one.
When ground truth is sparse: early warning systems
For models where ground truth is delayed or expensive to obtain, an early warning system for concept drift uses prediction confidence distributions as a proxy. A model that was consistently producing predictions with confidence above 0.85 that begins producing a higher proportion of low-confidence predictions (0.55–0.70) on identical input types is a signal worth investigating, even if labeled outcomes haven't confirmed a performance drop yet.
This is not a reliable direct measure of concept drift — it's a proxy that can generate false positives. But for long feedback-loop models, it's often the only real-time signal available. The governance response to a confidence distribution alert should be human investigation, not automated retraining — precisely because the signal is ambiguous and the appropriate response depends on domain knowledge about whether the world has actually changed in a relevant way.