A Software Framework for Out-of-turn Interaction in a Multimodal Web Interface

Multimodal interfaces are becoming increasingly important with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. This thesis investigates systems support for web browsing in a multimodal interface. Specifically, we outline the design and implementation of a software framework that integrates hyperlink and voice interaction. This enables the user to engage in out-of-turn interactions to personalize access at an information site. For the developer, the framework enables the creation of sites that adapt to the needs of users, yet permits fine-grained control over what interactions to support. Design methodology, implementation details, and two case studies are presented.