Eye tracking used to compare tactile and visual reading strategies

Researchers from the Netherlands used Tobii Pro Glasses and an infrared motion-capturing system to understand the reading strategies of both sighted and braille-dependent students when reading algebraic expressions.

The background

The limitations of current notational systems and available software make it difficult for braille-dependent students to participate in mathematic studies. As a result, braille-dependent students seldom enter, or make progress in, science-oriented tracks in upper secondary education. The aim of this research was to understand how both sighted and braille-dependent students read and comprehend algebraic expressions to identify ways to better support braille-dependent students in manipulating mathematical expressions. Obtaining insight in tactile and visual-reading strategies will help to design guidelines to help braille-dependent students and their teachers.

This project is a unique collaboration between three Dutch institutes: the Royal Dutch Visio in Huizen, the Freudenthal Institute for Science and Mathematics Education at Utrecht University, and the Department of Neuroscience at Erasmus University Medical Center Rotterdam. The work was led by Annemiek van Leendert, Michiel Doorman, Paul Drijvers, Johan Pel, and Hans van der Steen as a part of the research program, "Improving the Mathematical Abilities of Braille-dependent Students." This study was financed by the Netherlands Organization for Scientific Research (NWO).

The method

Six sighted students participated in the eye tracking study. Their eye movement patterns were recorded using Tobii Pro Glasses, in combination with a laptop and a video camera, and recorded at 50 Hz. At the Royal Dutch Visio school for the visually impaired, three braille-dependent students participated in a finger-tracking study. Their finger movement patterns were recorded using a combination of a laptop with a refreshable braille display, an infrared motion-capturing system (Vicon, Oxford, United Kingdom), and a video camera. Specific hand and index finger movements were recorded using four reflective markers on each student's hand. Motion-tracking software was used to record (at 200 Hz sampling rate) these movements and analyze the trajectories of each marker separately. The video camera recorded all handling, instructions, and verbal responses. Using verbal instructions given before the task began, the students all solved the same expressions with one or two variables (substitutions). The sighted students read the expression from a PowerPoint slide. The braille-dependent students read the expression with the braille-display from a Word document.

The following information was collected: eye movement, finger tracking, along with both audio and video. The collected data were analyzed using self-written Matlab software.

The results   

Gaze patterns enabled the researchers to quantify equation reading and solving strategies, in terms of exploration time, number of (re)fixations, and switching between local and global characters. 

This graph shows an impression of the gaze patterns of a sighted student. A first exploration of the gaze data reveals that the student starts to explore the equation for about seven seconds. The student calculates '3 + 2,' and then '1 – 5'. Finally, the equation is solved correctly by calculating '-4 + 4 = 0'. 

This graph shows the movements of the index fingers over the cells of the braille-display (left finger is represented with light green). In the exploration phase, both index fingers are active. During the first seconds, the left index finger initially scans '4 +' and, after three seconds, starts to explore the rest of the expression. Before starting to talk, the subject starts over and says the words "four plus" after about 13 seconds. In the next 20 seconds, they continue solving the expression correctly.

The conclusion

Gaze patterns obtained with Tobii Pro Glasses enabled researchers to quantify reading and solving strategies and made it possible to compare the outcome with tactile strategies of braille-dependent students. The insight obtained will be used to design guidelines to better support braille-dependent students and their teachers.