Drift Monitoring Configuration

Inferpathio supports three statistical drift detection methods. Each can be tuned independently per model and per feature group.

Detection methods

Population Stability Index (PSI)

PSI measures the shift in a feature's distribution relative to a baseline window. Threshold guidance:

• PSI < 0.10 — stable, no action needed
• PSI 0.10–0.20 — moderate shift, review recommended
• PSI > 0.20 — significant drift, trigger retraining

Kolmogorov-Smirnov (KS) test

The KS test compares CDFs of two distributions. Lower p-values indicate greater divergence. Recommended threshold: p < 0.05 for drift detection in low-cardinality features.

Wasserstein distance

Also known as Earth Mover's Distance. Useful for continuous numerical features. Thresholds are feature-scale-dependent; start at 5% of feature range and tune from there.

YAML configuration reference

monitor:
  model: fraud-detector
  baseline_window: 30d
  evaluation_window: 7d
  methods:
    psi:
      enabled: true
      threshold: 0.20
    ks_test:
      enabled: true
      p_value_threshold: 0.05
    wasserstein:
      enabled: false
  feature_groups:
    - name: transaction_features
      features: [amount, merchant_category, hour_of_day]
      method_override: psi
    - name: user_features
      features: [account_age_days, past_30d_txn_count]
      method_override: ks_test
  alerts:
    - channel: slack
      webhook: https://hooks.slack.com/services/...
      on: [drift_detected, threshold_crossed]
    - channel: pagerduty
      routing_key: ${PD_ROUTING_KEY}
      on: [threshold_crossed]

Baseline window

The baseline_window defines how far back Inferpathio looks to compute the reference distribution. Longer windows are more stable but slower to reflect legitimate concept drift. Common values:

• 7d — fast-moving seasonal data
• 30d — typical production models
• 90d — slow-changing enterprise datasets

Setting baseline programmatically

m = ifp.monitor("fraud-detector")
m.set_baseline(
    X_train,
    window_label="2025-Q4-training",
    freeze=True   # prevent automatic rolling updates
)

When freeze=True, the baseline stays fixed until you explicitly call m.set_baseline() again. Useful when you want drift scores relative to a specific training cohort.