Reliable and Interpretable Drift Detection in Streams of Short Texts

  • 2023-05-28 16:14:54
  • Ella Rabinovich, Matan Vetzler, Samuel Ackerman, Ateret Anaby-Tavor
  • 0


Data drift is the change in model input data that is one of the key factorsleading to machine learning models performance degradation over time.Monitoring drift helps detecting these issues and preventing their harmfulconsequences. Meaningful drift interpretation is a fundamental step towardseffective re-training of the model. In this study we propose an end-to-endframework for reliable model-agnostic change-point detection and interpretationin large task-oriented dialog systems, proven effective in multiple customerdeployments. We evaluate our approach and demonstrate its benefits with a novelvariant of intent classification training dataset, simulating customer requeststo a dialog system. We make the data publicly available.


Quick Read (beta)

loading the full paper ...