A theoretical model for evaluating the reliability of machine learning systems.

For a machine learning system comprising multiple machine learning models and input data, researchers at the University of Tsukuba developed a theoretical model to evaluate the effect of diversity in machine learning models and input data used in a system. of machine learning on the reliability of its output.

The developed model can be used to explore appropriate configurations of the machine learning system. The study is published in IEEE Transactions on Emerging Topics in Computer Science.

Machine learning systems for autonomous driving, medical imaging, and other applications require reliable and safe results. One such system design is the version N machine learning system. In this system, multiple machine learning models and input data are combined to prevent inference errors in the machine learning models from directly affecting the final output of the system. .

However, although the diversity of machine learning models and input data is empirically known to affect output reliability, a theoretical model has not yet been developed to explain this.

In this study, the researchers introduced the diversity metrics for the machine learning models and input data with respect to the inference errors of the machine learning models and built a theoretical model to evaluate the reliability of the output of the machine learning system. machine learning.

The results showed that a configuration method that uses the diversity of machine learning models and input data is the most stable method to improve the reliability of a machine learning system in generally assumed situations.

The overhead and cost of performing multiple inference processes are other challenges in practical system design. Researchers will continue to research and develop methods to achieve high reliability in N version machine learning systems while reducing cost, power consumption, and overhead from theoretical and experimental perspectives.