Intended Use Evaluation Approach for Information Visualization

TR Number
Date
2006-11-28
Journal Title
Journal ISSN
Volume Title
Publisher
Virginia Tech
Abstract

Information visualization is applied in many fields to gain faster insights with lighter user cognitive loads in analyzing large sets of data. As more products are being introduced each year, how can one select the most effective tool or representation form for the task? There are a number of information visualization evaluation methods currently available. However, these evaluation methods are often limited by the appropriateness of the tool for a given domain since they are not evaluating according to tools' intended use. Current methods conduct evaluations in a laboratory environment with "benchmark" tasks and often with field data sets not aligned with the intended use of the tools. The absence of realistic data sets and routine tests reduces the effectiveness of the evaluation in terms of the appropriateness of the tool for a given domain. Intended use evaluation approach captures the key activities that will use the visual technology to calibrate the evaluation criteria toward these first-order needs. This research thesis presents the results from an investigation into an intended use evaluation approach and its effectiveness of measuring domain specific information visualization tools.

In investigating the evaluation approach, criteria for the intelligence analysis community have been developed for demonstration purposes. While the observations from this research are compelling for the intelligence community, the principles of the evaluation approach should apply to a wider range of visualization technologies. All the design rationale and processes were captured in this thesis. This thesis presents a design process of developing criteria and measuring five intelligence analysis visual analytic tools. The study suggests that in selecting and/or evaluating visual analytic tools, a little up front effort to analyze key activities regarding the domain field will be beneficial. Such analysis can substantially reduce evaluation time and necessary effort throughout a longer period of time.

Description
Keywords
Information Visualization, Evaluation, Intended Use, Domain evaluation, Intelligence Analysis, Visualization
Citation
Collections