Self-Organizing Democratized Learning: Toward Large-Scale Distributed Learning Systems

Simple item page
dc.contributor.authorNguyen, Minh N. H.en
dc.contributor.authorPandey, Shashi Rajen
dc.contributor.authorTri Nguyen Dangen
dc.contributor.authorEui-Nam Huhen
dc.contributor.authorTran, Nguyen H.en
dc.contributor.authorSaad, Waliden
dc.contributor.authorHong, Choong Seonen
dc.date.accessioned2022-06-14T12:49:47Zen
dc.date.available2022-06-14T12:49:47Zen
dc.date.issued2022-05-10en
dc.description.abstractEmerging cross-device artificial intelligence (AI) applications require a transition from conventional centralized learning systems toward large-scale distributed AI systems that can collaboratively perform complex learning tasks. In this regard, democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. The outlined principles are meant to study a generalization in distributed learning systems that go beyond existing mechanisms such as federated learning (FL). Moreover, such learning systems rely on hierarchical self-organization of well-connected distributed learning agents who have limited and highly personalized data and can evolve and regulate themselves based on the underlying duality of specialized and generalized processes. Inspired by Dem-AI philosophy, a novel distributed learning approach is proposed in this article. The approach consists of a self-organizing hierarchical structuring mechanism based on agglomerative clustering, hierarchical generalization, and corresponding learning mechanism. Subsequently, hierarchical generalized learning problems in recursive forms are formulated and shown to be approximately solved using the solutions of distributed personalized learning problems and hierarchical update mechanisms. To that end, a distributed learning algorithm, namely DemLearn, is proposed. Extensive experiments on benchmark MNIST, Fashion-MNIST, FE-MNIST, and CIFAR-10 datasets show that the proposed algorithm demonstrates better results in the generalization performance of learning models in agents compared to the conventional FL algorithms. The detailed analysis provides useful observations to further handle both the generalization and specialization performance of the learning models in Dem-AI systems.en
dc.description.notesThis work was supported in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant 2020R1A4A1018607, in part by the Institute of Information & Communications Technology Planning & Evaluation (IITP) Grant funded by the Korean Government [Ministry of Science and ICT (MSIT)] (Evolvable Deep Learning Model Generation Platform for Edge Computing) under Grant 2019-0-01287, and in part by the IITP Grant funded by the Korean Government (MSIT) (Artificial Intelligence Innovation Hub) under Grant 2021-0-02068.en
dc.description.sponsorshipNational Research Foundation of Korea (NRF) - Korean Government (MSIT) [2020R1A4A1018607]; Institute of Information & Communications Technology Planning & Evaluation (IITP) - Korean Government [Ministry of Science and ICT (MSIT)] [2019-0-01287]; IITP - Korean Government (MSIT) (Artificial Intelligence Innovation Hub) [2021-0-02068]en
dc.description.versionPublished versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.doihttps://doi.org/10.1109/TNNLS.2022.3170872en
dc.identifier.eissn2162-2388en
dc.identifier.issn2162-237Xen
dc.identifier.pmid35536803en
dc.identifier.urihttp://hdl.handle.net/10919/110774en
dc.language.isoenen
dc.publisherIEEEen
dc.rightsCreative Commons Attribution 4.0 Internationalen
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/en
dc.subjectLearning systemsen
dc.subjectDistance learningen
dc.subjectComputer aided instructionen
dc.subjectTask analysisen
dc.subjectComputational modelingen
dc.subjectComputer scienceen
dc.subjectPhilosophical considerationsen
dc.subjectDemocratized learningen
dc.subjectdistributed artificial intelligences (AIs)en
dc.subjecthierarchical learningen
dc.subjectself-organizationen
dc.titleSelf-Organizing Democratized Learning: Toward Large-Scale Distributed Learning Systemsen
dc.title.serialIEEE Transactions on Neural Networks and Learning Systemsen
dc.typeArticle - Refereeden
dc.type.dcmitypeTexten

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
NguyenSelfOrganizing2022.pdf
Size:
3.14 MB
Format:
Adobe Portable Document Format
Description:
Published version
Download

Collections

Scholarly Works, Electrical and Computer Engineering