A Systems Theoretic Framework for Online Machine Learning with an Empirical Application

Abstract

Online (machine) learning is an active field of research which has been widely explored in terms of statistical learning theory, convex optimization theory and game theory, however, little to no frameworks exist for the design and application of online learning systems, both in theory and in practice. This work presents a unique, general framework for the modeling of online learning in general systems theoretic principles, which are not specific to any solution methods. Herein, online learning is defined as a system; its hierarchical relationship with machine learning is captured and deepened; its performance, properties and applications are re-defined in system theoretic terminology to discover alternative categorization and characterization of these systems; and its dynamic relationship with concept drift mathematically captured and explored. Subsequently, this work developed an unprecedented practical methodology to evaluate the testability of deployed online learning systems with – an unexplored, yet vital property for learning systems in real-world applications. In conclusion, this research developed an original systems theoretic framework and performance evaluation methodology for online learning to establish a foundation for the design, operation and analysis of online learning systems and their properties, in an effort to engineer safe and reliable real-world deployment of artificial intelligence.