Monitoring Model Drift & 6 Different Methodologies
The objective of this article is to discuss ways in which we can monitor and report drift. As we discussed in the monitoring the drift as it occurs is a crucial step in addressing the same. We will go through in detail six different ways in which we can monitor the drift. Visual analysis on data distribution Automated detection through hypothesis testing Study of the correlation matrix Study of variable importance in the model Monitoring through model accuracy metrics Dissecting the monitoring of accuracy metrics 1. Visual analysis of data distribution As we discussed in part of the earlier article on the model drift, change in data distribution is a very common type of drift that can occur in any environment. As can be seen in the picture above, comparing the distribution of data across training and scoring datasets is a straightforward way of spotting and monitoring data drift. This is a common functionality provided by many tools to compare and understand the drift. Fol...