A Knowledge Map-Centric Feedback-Based Approach to Information Modeling and Academic Assessment

The structure of education has changed dramatically in the last few decades. Despite major changes in how students are learning, there has not been as dramatic of a shift in how student learning is assessed. Standard letter grades are still the paradigm for evaluating a student's mastery of course content and the grade point average is still one of the largest determining factors in judging a graduate's academic aptitude.

This research presents a modern approach to modeling knowledge and evaluating students. Based upon the model of a closed-loop feedback controller it considers education as a system with an instructor determining the set of knowledge he or she wishes to impart to students, the instruction method as a transfer function, and evaluation methods serving as sensors to provide feedback determining the subset of the information students have learned. This method uses comprehensive concept maps to depict all of the concepts and relationships an educator intends to cover and student maps to depict the subset of knowledge that students have mastered. Concept inventories are used as an assessment tool to determine, at the conceptual level, what students have learned. Each question in the concept inventory is coupled with one or more components of a comprehensive concept map and based upon the answers students give to concept inventory questions those components may or may not appear in a student's knowledge map. The level of knowledge a student demonstrates of each concept and relationship is presented in his or her student map using a color scheme tied to the levels of learning in Bloom's taxonomy.

Topological principles are used to establish metrics to quantify the distance between two students' knowledge maps and the distance between a student's knowledge map and the corresponding comprehensive concept map. A method is also developed for forming aggregate maps representative of the knowledge of a group of students. Aggregate maps can be formed for entire classes of students or based upon various demographics including race and gender.

XML schemas have been used throughout this research to encapsulate the information in both comprehensive maps and student maps and to store correlations between concept inventory questions and corresponding comprehensive map components. Three software packages have been developed to store concept inventories into an XML Schema, to process student responses to concept inventory questions and generate student maps as a result, and to generate aggregate maps. The methods presented herein have been applied to two learning units that are part of two freshman engineering courses at Virginia Tech. Example student maps and aggregate maps are included for these course units.