A Machine Learning Approach for Data Unification and Its Application in Asset Performance Management

Abstract

The amount of data is growing fast with the advance of data capturing and management technologies. However, data from different source are often isolated and not ready to be analyzed together as one data set. The effort of connecting pieces of isolated data into a unified data set is time consuming and often costly in terms of cognitive load and programming time. To address this problem, here we proposed an approach using machine learning to augment human intelligence in the data unification process, especially complex categorical data value unification. Many aspects of useful information are extracted from supervised machine learning models, then used to amplify intelligence of human experts in various aspects of the data unification process. An empirical study is performed applying the proposed methodology to the field of Asset Performance Management, specifically focus only on the performance of equipment asset. The experiments show that machine learning helps experts in the unification standard generation, unified value suggestion, batch data unification. We conclude that machine learning models contain valuable information that can facilitate the data unification process.