Stage 4 – Integration and testing
After model training, it is integrated into the client’s management system. This integration is simplified by the fact that the model realises only one function – it predicts or recommends. In this regard, the interface is very simple and the integration is reduced to data transfer and displaying those recommendations.
Practical tests are carried out with client’s side experts participating. During testing, it is necessary to measure the accuracy of the model, and achieved economic effect. In the service designed for MMK, the board of experts involved in the testing waived any responsibility from the operator if he did not quite agree with the recommendation made. This allowed us to estimate the effect of the model use and to determine the necessary changes for its use in production. During testing, model showed a decrease in ferroalloy use of about 5%, and the projected savings comprised $4.3 million per year.
Stage 5 – Model monitoring
The last stage is the production use of the service that successfully passed testing. The production use requires constant monitoring of the model quality and its regular additional training on newly collected data. Unfortunately, completely self-trained machine learning systems are in the research stage now, and their application for business and industry use is not possible.
About the author:
Alexander Khaytin is Chief Operating Officer of Yandex Data Factory - https://yandexdatafactory.com