Departmental Papers (CS)

Toward a more accurate view of when and how people seek help with computer applications

David G. Novick et al. — Tue, 18 Aug 2009 08:17:59 PDT

Based on 40 interviews and 11 on-site workplace observations of people using computer applications at work, we confirm that use of printed and on-line help is very low and find that providing greater detail of categories solution methods can present a more realistic picture of users’ behaviors. Observed study participants encountered a usability problem on average about once every 75 minutes and typically spent about a minute looking for a solution. Participants consumed much more time when they were unaware of a direct way of doing something and instead used less effective methods. Comparison of results from different data-collection methods suggests that interviews, and probably surveys, provide less reliable views of users’ problem-solving behaviors than do participatory evaluation and direct observation.

Usability inspection methods after 15 years of research and practice

David G. Novick et al. — Tue, 18 Aug 2009 08:17:57 PDT

Usability inspection methods, such as heuristic evaluation, the cognitive walkthrough, formal usability inspections, and the pluralistic usability walkthrough, were introduced fifteen years ago. Since then, these methods, analyses of their comparative effectiveness, and their use have evolved in different ways. In this paper, we track the fortunes of the methods and analyses, looking at which led to use and to further research, and which led to relative methodological dead ends. Heuristic evaluation and the cognitive walkthrough appear to be the most actively used and researched techniques. The pluralistic walkthrough remains a recognized technique, although not the subject of significant further study. Formal usability inspections appear to have been incorporated into other techniques or largely abandoned in practice. We conclude with lessons for practitioners and suggestions for future research.

Why don't people read the manual?

David G. Novick et al. — Tue, 18 Aug 2009 08:17:56 PDT

Few users of computer applications seek help from the documentation. This paper reports the results of an empirical study of why this is so and examines how, in real work, users solve their usability problems. Based on in-depth interviews with 25 subjects representing a varied cross-section of users, we find that users do avoid using both paper and online help systems. Few users have paper manuals for the most heavily used applications, but none complained about their lack. Online help is more likely to be consulted than paper manuals, but users are equally likely to report that they solve their problem by asking a colleague or experimenting on their own. Users cite difficulties in navigating the help systems, particularly difficulties in finding useful search terms, and disappointment in the level of explanation found.

What users say they want in documentation

David G. Novick et al. — Tue, 18 Aug 2009 08:17:56 PDT

While earlier work provided a partial view of users’ preferences about manuals, for most users in most work contexts the important question remains open: What do users want in documentation? This paper presents the results of a study in which a diverse cross-section of 25 users was interviewed in depth about their needs and preferences with respect to software help systems, whether printed or on-line, that they use at work. The study’s participants indicated that they preferred documentation, whether online or printed, that is easy to navigate, provides explanations at an appropriate level of technical detail, enables finding as well as solving problems through examples and scenarios, and is complete and correct. These preferences give rise to difficult issues, including a possibly inherent tension between coverage and precision, and variation among users with respect to desired level of technical complexity of explanation.

Co-generation of text and graphics

David G. Novick et al. — Tue, 18 Aug 2009 08:17:54 PDT

To reduce potential discrepancies between textual and graphical content in documentation, it is possible to produce both text and graphics from a single common source. One approach to co-generation of text and graphics uses a single logical specification; a second approach starts with CAD-based representation and produces a corresponding textual account. This paper explores these two different approaches, reports the results of using prototypes embodying the approaches to represent simple figures, and discusses issues that were identified through use of the prototypes. While it appears feasible to co-generate text and graphics automatically, the process raises deep issues of design of communications, including the intent of the producer of the documentation.

Usability over time

Valerie Mendoza et al. — Tue, 18 Aug 2009 08:17:53 PDT

Testing of usability could perhaps be more accurately described as testing of learnability. We know more about the problems of novice users than we know of the problems of experienced users. To understand how these problems differ, and to understand how usability problems change as users change from novice to experienced, we conducted a longitudinal study of usability among middle-school teachers creating Web sites. The study looked at the use both the use of documentation and the underlying software, tracking the causes and extent of user frustration over eight weeks. We validated a categorization scheme for frustration episodes. We found that over the eight weeks the level of frustration dropped, the distribution of causes of frustration changed, and the users’ responses to frustration episodes changed. These results suggest that the sorts of errors that are most prominently featured in conventional usability testing

Root causes of lost time and user stress in a simple dialog system

Nigel G. Ward et al. — Tue, 18 Aug 2009 08:17:52 PDT

As a priority-setting exercise, we compared interactions between users and a simple spoken dialog system to interactions between users and a human operator. We observed usability events, places in which system behavior differed from human behavior, and for each we noted the impact, root causes, and prospects for improvement. We suggest some priority issues for research, involving not only such core areas as speech recognition and synthesis and language understanding and generation, but also less-studied topics such as adaptive or flexible timeouts, turn-taking and speaking rate.

A computational model of culture-specific conversational behavior

Dusan Jan et al. — Tue, 18 Aug 2009 08:17:51 PDT

This paper presents a model for simulating cultural differences in the conversational behavior of virtual agents. The model provides parameters for differences in proxemics, gaze and overlap in turn taking. We present a review of literature on these factors and show results of a study where native speakers of North American English, Mexican Spanish and Arabic were asked to rate the realism of the simulations generated based on different cultural parameters with respect to their culture.

"Conversational" Dialogues in Direct-Manipulation Interfaces

David G. Novick — Thu, 10 Feb 2005 08:55:30 PST

This paper reports ongoing research in extending direct-manipulation interfaces by incorporating, via the direct-manipulation modality itself, interaction techniques that add kinds of language features associated with spoken conversation. The paper proposes means of implementing ways for a user of a direct-manipulation system to define new kinds of relations among objects in the interface.

Users and Uses of Synchronous Business Communications Software

David G. Novick et al. — Mon, 07 Feb 2005 14:30:11 PST

To help designers and authors understand users' intentions and work practices for synchronous business communications in a systematic way, we used ethnographic and task-analytic techniques to collect, analyze and classify evidence of the activities of potential users as they conducted their work lives. The interactions we observed among our users took place through a variety of modalities. We found eight categories of tasks for the collaborative or interactive work in which our subjects engaged. Based on these data, we were able to classify roles of potential users of synchronous business communications software into a set of "archetypes" that characterize their use: Thinkers, Producers, Authors, Networkers and Diplomats. Issues raised by our work include questions about the nature of user archetypes, user tasks, and their modalities.

Accounting for Domain Context in Evaluation

Meriem Chater et al. — Mon, 07 Feb 2005 13:52:06 PST

Work is situated activity. Taking into account human factors in evaluation involves considering not only users but also their contexts of use. Consequently, the evaluation of systems — from video-games to safetycritical interfaces — requires analysis of context to understand not only the effect of context on usability but also the impact of artifacts' usability on users' environments. In the case of safety-critical systems (SCS), errors (by users or designers) may threaten human lives.
To assess the degree to which interface evaluation methods currently account for context, we have used the research strategy taxonomy of McGrath as a framework for classifying existing evaluation methods of aviation domain and general HCI interactive systems. This framework enabled us to describe common grounds and key differences of methods used in HCI and SCS, and to highlight aspects of context that could be analyzed using each strategy.
For instance, characteristics of SCS, such as time-criticality, unpredictability and dynamics, emphasize the leading role of operational context on the remaining work context including physical or technical constraints defined by organizational, social, cultural and technical contexts which is not the case for general HCI.

Extending Direct Manipulation in a Text Editor

David G. Novick et al. — Mon, 07 Feb 2005 13:05:51 PST

This paper describes the implementation of a prototype text editor that incorporates conversation-like features through the direct-manipulation modality. In this way, traditional direct-manipulation interaction techniques such as direct reference via pointing can be extended to include techniques more commonly associated with human conversation, such as negotiation of reference. The paper illustrates the use of the prototype with an extended example, and discusses research issues raised by the implementation.

Hands-Free Documentation

Karen Ward et al. — Mon, 07 Feb 2005 12:33:44 PST

In this paper, we introduce an analysis of the requirements and design choices for hands-free documentation. Hands-busy tasks such as cooking or car repair may require substantial interruption of the task: moving the pan off the burner and wiping hands, or crawling out from underneath the car. We review the need for hands-free documentation and explore the role of task in the use of documentation. Our central analysis examines the roles and characteristics of input and output modalities of hands-free documentation. In particular, we review the use of speech as an input modality, and then visual means and speech as possible output modalities. Finally, we discuss the implications of our analysis for the design of hands-free documentation and suggest future work. The design implications include issues of navigating through the documentation, determining the user’s task and taskstep, establishing mutual understanding of the state of the task, and determining when to start conveying information to the user.

An Interaction Initiative Model for Documentation

David G. Novick et al. — Mon, 07 Feb 2005 12:16:38 PST

In this paper we propose a model of creation and use of documentation based on the concept of mixed-initiative interaction. In our model, successful single-initiative interaction is characterized by grounding of contributions, and successful mixed-initiative interaction is characterized by both grounding and agreement. Just as in spoken conversation, achievement of actual agreement depends on the intentions of both parties; agreement is achieved when the reader follows the documentation’s instructions. In fact, readers are not obligated to—and often do not—act according to the author’s intentions. By making these dynamics explicit, the model can aid authors in developing effective documentation. The paper describes the model and its antecedents, explains the application of the model to documentation, discusses implications such as effects of printed versus electronic forms of documentation, and outlines future work that includes empirical testing of the model.

Assessing Effectiveness of Personality Style in Documentation

Kenneth Sayles et al. — Mon, 07 Feb 2005 10:36:26 PST

This paper extends previous work by other researchers that indicated that users of computers preferred a computer with a personality that was similar to theirs. We conducted a similar experiment, but looking beyond preference to see if the personality of documentation would make a difference in the user’s performance. Our data suggest did not indicate that personality match affects performance; and if such a relationship exists it is likely to be weak. We discuss the related research, describe our methodology, present our results, and describe their implications and limitations.

Post-Training Support for Learning Technology

Sam Snoddy Jr. et al. — Mon, 07 Feb 2005 10:14:17 PST

To examine the effects of post-training support, we studied the introduction of new gradebook software in a public high school. The school's 108 faculty members received training on the software, and approximately half of the faculty received posttraining support for eight weeks. The study measured the faculty's current computer usage, usage of earlier versions of the software, and their perceived skill levels in using the software. The data suggest that the faculty members who received post-training support maintained and raised their skill levels, while unsupported faculty had their skill levels decline.