Usability and Reliability of the User Action Framework: A Theoretical Foundation for Usability Engineering Activities

Abstract

Various methods exist for performing usability evaluations, but there is no systematic framework for guiding and structuring assessment and reporting activities (Andre et al., 2000). Researchers at Virginia Tech have developed a theoretical foundation called the User Action Framework (UAF), which is an adaptation and extension of Norman's action model (1986). The main objective of developing the User Action Framework was to provide usability practitioners with a reliable and structured tool set for usability engineering support activities like classifying and reporting usability problems. In practice, the tool set has a web-based interface, with the User Action Framework serving as an underlying foundation. To be an effective classification and reporting tool, the UAF should be usable and reliable. This work addressed two important research activities to help determine the usability and reliability of the User Action Framework. First, we conducted a formative evaluation of the UAF Explorer, a component of the UAF, and its content. This led a re-design effort to fix these problems and to provide an interface that resulted in a more efficient and satisfying user experience. Another purpose of this research was to conduct a reliability study to determine if the User Action Framework showed significantly better than chance agreement when usability practitioners classified a given set of usability problem descriptions according to the structure of the UAF. The User Action Framework showed higher agreement scores compared to previous work using the tool.