Utilizing Machine Learning Methods for Usability Evaluation in Learning Management Systems

Simple item page
dc.contributor.authorTorres Molina, Richard Andresen
dc.contributor.committeechairSeyam, Mohammed Saad Mohamed Elmahdyen
dc.contributor.committeememberFox, Edward A.en
dc.contributor.committeememberMcCrickard, Donald Scotten
dc.contributor.departmentComputer Science and#38; Applicationsen
dc.date.accessioned2024-05-15T08:00:34Zen
dc.date.available2024-05-15T08:00:34Zen
dc.date.issued2024-05-14en
dc.description.abstractThe concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs.en
dc.description.abstractgeneralInstructors and students have used online platforms known as Learning Management Systems (LMSs) to improve learning and satisfaction. Students need to achieve their learning goals by interacting with these systems. To achieve these goals, usability evaluation involves ensuring that LMSs attain effectiveness (task completion), efficiency (time measurement), and satisfaction (positive attitude). Usability evaluation usually follows questionnaires, user testing of the LMS, and expert reviews. Although these methods are widely used due to several benefits, they face challenges related to trying these software systems multiple times until the system satisfies student needs and human subjectivity perception. To face these challenges, promote student engagement with the system, and create a better design in the LMS courses, we propose a hybrid approach based on data, questionnaire answers, and machine learning algorithms to predict usability scores. We evaluated this approach through a case study with data collected from undergraduate students at Virginia Tech. The results showed different advantages and drawbacks of machine learning performance. The approach contributes to the engineering and computing education field by providing a reliable predictive tool for usability scores to improve the student learning experience and the features of the LMS.en
dc.description.degreeMaster of Scienceen
dc.format.mediumETDen
dc.identifier.othervt_gsexam:40067en
dc.identifier.urihttps://hdl.handle.net/10919/118974en
dc.language.isoenen
dc.publisherVirginia Techen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subjectusability evaluationen
dc.subjectmachine learningen
dc.subjectlearning management systemsen
dc.titleUtilizing Machine Learning Methods for Usability Evaluation in Learning Management Systemsen
dc.typeThesisen
thesis.degree.disciplineComputer Science & Applicationsen
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen
thesis.degree.levelmastersen
thesis.degree.nameMaster of Scienceen

Files

Original bundle
Now showing 1 - 1 of 1
Name:
Torres_Molina_RA_T_2024.pdf
Size:
5.86 MB
Format:
Adobe Portable Document Format
Download

Collections

Masters Theses