Eye-based human-computer interaction (HCI) goes back at least to the early 1990s. Controlling a computer using the eyes traditionally meant extracting information from the gaze—that is, what a person was looking at. In an early work, Robert Jacob investigated gaze as an input modality for desktop computing.1 He discussed some of the human factors and technical aspects of performing common tasks such as pointing, moving screen objects, and menu selection. Since then, eye-based HCI has matured considerably. Today, eye tracking is used successfully as a measurement technique not only in the laboratory but also in commercial applications, such as marketing research and automotive usability studies.
With heavy competition between iPhone games, proper
playtesting is vital in making an easy to use, fun game. Eye
tracking can give valuable insights in player behavior but
current handheld eye tracking set-ups suffer technologial
limitations, inhibiting normal play. This study aims to
identify the merits and shortcomings of a new handheld
eyetracking set-up for qualitative user research. It is part of
a series of ongoing tests to improve the set-up. In this
study, seven participants played an iPhone puzzle game
using the new set-up. Results indicated the set-up was
suited for simple tasks like browsing, but interfered with
normal gaming too much for most players. Factors
contributing to interference were: Lack of depth perception,
unnatural handling, uncomfortable posture and enlarged
display of hands. Solutions for improvement are discussed:
With longer practice for players and with tweaks to the setup,
interference can be reduced or partly removed.
Accurate depth perception remains a challenge, however.
Projector phones, handheld game consoles and many other
mobile devices increasingly include more than one display,
and therefore present a new breed of mobile Multi-Display
Environments (MDEs) to users. Existing studies illustrate
the effects of visual separation between displays in MDEs
and suggest interaction techniques that mitigate these
effects. Currently, mobile devices with heterogeneous
displays such as projector phones are often designed
without reference to visual separation issues; therefore it is
critical to establish whether concerns and opportunities
raised in the existing MDE literature apply to the emerging
category of Mobile MDEs (MMDEs). This paper
investigates the effects of visual separation in the context of
MMDEs and contrasts these with fixed MDE results, and
explores design factors for Mobile MDEs. Our study uses a
novel eye-tracking methodology for measuring switches in
visual context between displays and identifies that MMDEs
offer increased design flexibility over traditional MDEs in
terms of visual separation. We discuss these results and
identify several design implications.
Interactive Evolutionary Computation (IEC) community
aims at reducing user's fatigue during an optimization task
involving subjective criteria: a set of graphic potential
solutions are simultaneously shown to a user which task is
to identify most interesting solutions to the problem he had
to solve. Evolutionary operators are applied to user choices
expecting to produce better solutions. As traditional IEC
ask the user to give a mark to each solution or to explicitly
choose bests solutions with a mouse, we propose a new
framework that uses in real time gaze information to predict
which parts of a screen is more significant for a user. We
can therefore avoid the user to explicitly choose which
solutions are interesting for him. In this paper, we mainly
focus on automatically ordering solutions shown on a
screen given a gaze path obtained by an eye-tracker. We
applied several supervised learning methods (SVM, neural
networks…) on two different experiments. We obtain a
formula that predict with 85% user choices. We
demonstrate that decisive criterion is time spent on one
solution and we show the independency between this
formula and the experiment.
This study investigates dynamic information acquisition strategies during decision making. The authors conduct an eye-tracking experiment to trace consumers‘ moment-to-moment decision process on comparison websites. A new hierarchical Hidden Markov Model is developed to analyze the eye-movement data. It consists of three connected hierarchical layers: a lower layer that describes the eye-movements, a middle layer that captures product-based and attribute-based information acquisition strategies, and an upper layer that enables us to analyze the time course of switching between these information acquisition strategies. In the experiment on the effects of presentation formats of comparison websites for laptop computers, the authors quantify the usage of information acquisition strategies, identify switching patterns, and investigate the impact that strategy switching has on evaluation of the choice process. Consumers switch frequently between information acquisition strategies: around 50 to 60 times for the average decision. The contiguity of presented information and the row-column presentation format influence information strategy usage and product choice. These findings support our recommendations for the rapidly growing comparison website industry.
The use of e-book readers (e-readers or electronic-readers) has become increasingly widespread. An e-reader should meet two important requirements: adequate legibility and good usability. In our study, we investigated these two requirements of e-reader design. Within the framework of a multifunctional approach, we combined eye tracking with other usability testing methods. We tested five electronic reading devices and one classic paper book. The results suggested that e-readers with e-ink technology provided legibility that was comparable to classic paper books. However, our study also showed that the current e-reader generation has large deficits with respect to usability. Users were unable to use e-readers intuitively and without problems. We found significant differences between the different brands of e-book readers. Interestingly, we found dissociations between objective eye-tracking data and subjective user data, stressing the importance of multi-method approaches.
We analysed the eye-tracking data of 147 participants as they used a total of 15 separate website navigation menus to complete key activities. The hypotheses for this study were that (a) the psychological phenomenon of the order effect would manifest in that items at either end of a menu would be located more quickly than those in the middle and (b) that the items that were relevant to completing the user‘s tasks would be located more quickly through peripheral visual identification of these items. Although items relevant to the user‘s task were acquired 1.8 seconds faster on average, both of the hypotheses were rejected as no statistically significant patterns were found. It was concluded that each user was likely to have his or her own searching behaviour and this could be affected by other factors such as the graphic design of the menu.
An experiment was conducted to test the efficacy of a new intelligent hypermedia system, MetaTutor, which is intended to prompt and scaffold the use of self-regulated learning (SRL) processes during learning about a human body system. Sixty-eight (N=68) undergraduate students learned about the human circulatory system under one of three conditions: prompt and feedback (PF), prompt-only (PO), and control (C) condition. The PF condition received timely prompts from animated pedagogical agents to engage in planning processes, monitoring processes, and learning strategies and also received immediate directive feedback from the agents concerning the deployment of the processes. The PO condition received the same timely prompts, but did not receive any feedback following the deployment of the processes. Finally, the control condition learned without any assistance from the agents during the learning session. All participants had two hours to learn using a 41-page hypermedia environment which included texts describing and static diagrams depicting various topics concerning the human circulatory system. Results indicate that the PF condition had significantly higher learning efficiency scores, when compared to the control condition. There were no significant differences between the PF and PO conditions. These results are discussed in the context of development of a fully-adaptive hypermedia learning system intended to scaffold self-regulated learning.
The affective component has been acknowledged as critical to understand information search behavior and user–computer interactions. There is a lack of studies that analyze the emotions that the user feels when searching for information about products with search engines. The present study analyzes the emotional outcomes of the online search process, taking into account the user’s (a) perceptions of success and effort exerted on the search process, (b) initial affective state, and (c) emotions felt during the search process. In addition, we identify profiles of online searchers based on the emotional outcomes of the search process, which allow us to differentiate the emotional processes and behavioral patterns that lead to such emotions. The results of the study stress the importance of the affective component of the online search behavior, given that these emotional outcomes are likely to influence all the subsequent actions that users perform on the Web.
Web sites need fast and effective navigation systems. An eye tracking laboratory study with n = 120 participants was conducted to compare the influence of different navigation designs (vertical versus dynamic menus) and task complexity (simple versus complex navigation tasks) on user performance, navigation strategy, and subjective preference. With vertical menus, users needed less eye fixations, were faster and more successful. We conclude that, firstly, vertical menus fit better to perception and cognition than dynamic menus, where the navigation items are hidden and must be accessed by an additional mouse click. Secondly, navigation systems should be extended with different kinds of navigation items adapted to the complexity of the users’ navigation tasks, because users tend to switch their navigation strategy when confronted with complex tasks.
This paper investigates the eye movement sequences of users visiting web pages repeatedly. We are interested in potential habituation due to repeated exposure. The scanpath theory posits that every person learns an idiosyncratic gaze sequence on first exposure to a stimulus and re-applies it on subsequent exposures. Josephson and Holmes (2002) tested the applicability of this hypothesis to web page revisitation but results were inconclusive. With a recurrent temporal pattern detection technique, we examine additional aspects and expose scanpaths. Results do not suggest direct applicability of the scanpath theory. While repetitive scan patterns occurred and were individually distinctive, their occurrence was variable, there were often several different patterns per person, and patterns were not primarily formed on the first exposure. However, extensive patterning occurred for some participants yet not for others which deserves further study into its determinants.
Visual analytics is often based on the intuition that highly interactive and dynamic depictions of complex and multivariate databases amplify human capabilities for inference and decision-making, as they facilitate cognitive tasks such as pattern recognition, association, and analytical reasoning (Thomas and Cook 2005). But how do we know whether visual analytics really works? This article offers a generic evaluation approach combining theory- and data-driven methods based on sequence similarity analysis. The approach systematically studies users' visual interaction strategies when using highly interactive interfaces. We specifically ask whether the efficiency (i.e., speed) of users can be characterized by specific display interaction event sequences, and whether studying user strategies could be employed to improve the (interaction) design of the dynamic displays. We showcase our approach using a very large, fine-grained spatiotemporal dataset of eye movement recordings collected during a controlled human subject experiment with dynamic visual analytics displays. With this methodological approach based on empirical evidence, we hope to contribute to a deeper understanding of how people make inferences and decisions with highly interactive visualization tools and complex displays.
In order to improve human-computer interaction, eyetracking, physiological measures and task recognition are used to assess the content and quality of interaction in an ecommerce application with User Generated Content. Results are analyzed in conjunction with users’ verbalization and their subjective assessment of attitudes
toward the product and their wish to contribute.
Guidelines for designing information charts often state that the presentation should reduce ‗chart junk‘ – visual embellishments that are not essential to understanding the data. In contrast, some popular chart designers wrap the presented data in detailed and elaborate imagery, raising the questions of whether this imagery is really as detrimental to understanding as has been proposed, and whether the visual embellishment may have other benefits. To investigate these issues, we conducted an experiment that compared embellished charts with plain ones, and measured both interpretation accuracy and long-term recall. We found that people‘s accuracy in describing the embellished charts was no worse than for plain charts, and that their recall after a two-to-three-week gap was significantly better. Although we are cautious about recommending that all charts be produced in this style, our results question some of the premises of the minimalist approach to chart design.
Recommender systems have emerged as an effective decision tool to help users more easily and quickly find products that they prefer, especially in e-commerce environments. However, few studies have tried to understand how this technology has influenced the way users search for products and make purchase decisions. Our current research aims at examining the impact of recommenders by understanding how recommendation tools integrate the classical economic schemes and how they modify product search patterns. We report our work in employing an eye tracking system and collecting users' interaction behaviors as they browsed and selected products to buy from an online product retail website offering over 3,500 items. This in-depth user study has enabled us to collect over 48,000 fixation data points and 7,720 areas of interest from eighteen users, each spending more than one hour on our site. Our study shows that while users still use traditional product search tools to examine alternatives, recommenders definitely provide users with new opportunities in their decision process. More specifically, users actively click and gaze at products recommended to them, up to 40% of the time. In addition, recommendation areas are highly attractive, drawing users to add 50% more items to their baskets as a traditional tool does. Observing that users consult the recommendation area more as they are close to the end of their search process, it seems that recommenders enhance users' decision confidence by satisfying their need for diversity. Based on these results, we derive several interaction design guidelines that can significantly improve users' satisfaction and perception of product recommenders.
With the advent of a digital economy, an emphasis on digital products and services has emerged. Those who are not using current technologies will become excluded, however, from this revolution. Older adults represent one such group in danger of exclusion. In some cases, older adults have been disinterested in new technologies. In other cases, however, the technologies fail to take into consideration the strengths and weaknesses of older users that would promote this usability. This paper examines components of information search by younger and older adults. These are considered in terms of long-term implications of designing for older users, with current problems viewed as foreshadowing future trends.
Multimodal interaction in everyday life seems so effortless. However, a closer look reveals that such interaction is indeed complex and comprises multiple levels of coordination, from high-level linguistic exchanges to low-level couplings of momentary bodily movements both within an agent and across multiple interacting agents. A better understanding of how these multimodal behaviors are coordinated can provide insightful principles to guide the development of intelligent multimodal interfaces. In light of this, we propose and implement a research framework in which human participants interact with a virtual agent in a virtual environment. Our platform allows the virtual agent to keep track of the user’s gaze and hand movements in real time, and adjust his own behaviors accordingly. An experiment is designed and conducted to investigate adaptive user behaviors in a human-agent joint attention task. Multimodal data streams are collected in the study including speech, eye gaze, hand and head movements from both the human user and the virtual agent, which are then analyzed to discover various behavioral patterns. Those patterns show that human participants are highly sensitive to momentary multimodal behaviors generated by the virtual agent and they rapidly adapt their behaviors accordingly. Our results suggest the importance of studying and understanding real-time adaptive behaviors in human-computer multimodal interactions.
Children with dyslexia and attention deficit disorders often have problems in short term memory, yet can benefit from learning strategies for remembering. In this paper, we describe the design of a multimedia educational game called Memory Challenge to help children with Specific Learning Difficulties (SpLDs) learn strategies for memory and to develop their cognitive skills. We focus in our approach on the involvement of children with SpLDs and domain specialists and practitioners in the design process. Involving various
participants from our target population (native Arabic-speaking users) in different stages of our design process was effective in obtaining an insight into the needs of people with SpLDs and has contributed to the design with actionable implications.
Europeana’s priority as it moves towards a fully operational service is to provide access to Europe’s heritage in ways that engage and satisfy users.
A principal objective of Europeana.eu is to engage young people, both in the course of their learning experience and for personal enrichment. In the swift current of online innovation, theirs are the needs and expectations that change most rapidly. Consequently, in order to define the user requirements for the fully operational service, Europeana focused on detailed qualitative analyses of user behaviour, paying particular attention to students.
Six focus groups were convened, comprising a total of 77 participants in four European countries. Two of the focus groups took place in an international school in Amsterdam, the Netherlands; in Sofia, Bulgaria they were held in a secondary school and a school of applied arts. There was also one for university students in Fermo, Italy and one for university library and teaching staff with representatives of the general public in Glasgow, Scotland.
Studies were also run in Media Labs. These tests used eye-tracking and close observation of 12 subjects to derive empirical evidence of their response to Europeana’s navigation and usability. This is one of the first studies published in the digital library context in which eye tracking combined with analysis of user behaviour and feedback have been used to refine the vision of what users want.
The results of the studies inform the design and functionality of the operational Europeana. In addition, and of value to the marketing and communications initiatives, the studies have helped define the benefits sought by primary target segments, what promotional messages they would respond to, and how these should be delivered to them.
A better understanding of the human user’s expectations and sensitivities to the real-time behavior generated by virtual agents can provide insightful empirical data and infer useful principles to guide the design of intelligent virtual agents. In light of this, we propose and implement a research framework to systematically study and evaluate different important aspects of multimodal real-time interactions between humans and virtual agents. Our platform allows the virtual agent to keep track of the user’s gaze and hand movements in real time, and adjust his own behaviors accordingly. Multimodal data streams are collected in human-avatar interactions including speech, eye gaze, hand and head movements from both the human user and the virtual agent, which are then used to discover fine-grained behavioral patterns in human-agent interactions. We present a pilot study based on the proposed framework as an example of the kinds of research questions that can be rigorously addressed and answered. This first study investigating human-agent joint attention reveals promising results about the role and functioning of joint attention in human-avatar interactions.