Driven by Caravaggio: eye tracking study reveals the artist’s intentions

Michelangelo Merisi da Caravaggio was one of the most influential painters in the 17th century, and he is especially well known for his dramatic lighting effects. Caravaggio was active at a time when optical studies were progressing, and the works of artists were created to be displayed in a specific location.

Researchers at Scienza Nuova in Italy used eye tracking to test the hypothesis that Caravaggio understood how we perceive images and that he, while creating his art, took into account how the environment would affect the viewer's visual experience. The study revealed that, by moving the art from its original environment, the intended viewer experience decreases. These results can help improve how art is displayed in museums and guide visitors how to view the elements of a painting for the best experience.

The background

Since ancient times, we have wondered how the human brain acquires and processes images from the outside world. Cognitive psychology explains the phenomenon through embodied simulation: the ability to build a representation of the outside world to which our visual experiences are related. The act of viewing art is so complex that the cognitive disciplines have begun to use eye tracking to observe both ocular and brain behavior.

The goal of this study is to identify the formal elements (line, shape, form, tone, texture, pattern, color, and composition) that guide the viewer through the works of the painter in order to apply the techniques of Michelangelo Merisi da Caravaggio to other visual pieces. The intent is to increase the effectiveness of images in many fields, improve the museum experience, and increase the appreciation of artwork.

The interaction of art and the brain has been the focus of numerous studies, but this is the first time that the study of this process is being used to improve the guest-fulfillment experience.

The method

A Tobii Pro Glasses wearable eye tracker, was used for the data collection in this study to facilitate a natural viewing experience for the participants. This is the first-time eye tracking research has been conducted on art in its natural environment. The Pro Glasses Analyzer software allowed the team to easily overlap the visual scan path with the corresponding stimulus.

The experiment consisted of two parts, where attention data was collected from participants looking at Caravaggio's paintings in two different locations.

The initial study took place over three days at the church of Pio Monte della Misericordia in Naples. Caravaggio's painting, Sette Opere di Misericordia, is preserved in its original condition there.

The second phase of the research took place at the Capodimonte Museum in Naples. This is where another Caravaggio's painting, La Flagellazione di Cristo, is on exhibit. 
The researchers randomly selected participants from the visitors to the church and museum. After individual calibration, they were allowed to naturally walk through the locations while wearing Pro Glasses for three minutes.

The following eye tracking metrics were used for the analysis: number of gazes, number of fixations, time to first gaze, time to first fixation, areas of interest, and visit count.

The results  

In the first phase, the visual pathway of the subjects was defined. It was discovered that each participant focused their attention on the same area of interest. In fact, the aggregated data allowed researchers to identify five areas of interest based on the fixation count. These were visualized by the Pro Glasses Analyzer in a heatmap. 

The time to first fixation, which corresponded to the precise moment when the visitor actively observed each portion of the painting, demonstrated not only when the gaze was resting on each of the scenes of the composition but also when the visitor started the cognitive process of understanding. 

The Pro Glasses Analyzer's gazeplot visualizations confirmed that the visual pathway of the participants was repetitive, and two recurring visual patterns were found. 

In the second phase of the study, it was not possible to find a common pathway among the 22 participants in the experiment. In fact, the researchers indentified 20 different patterns. To compare the variability of pathways in the two different scenes, a Pathway.Variability Index (PVI) was defined: 

PVI = number of distinct pathways / number of subjects. 

This PVI tends to be 0 when there are very few distinct pathways (i.e. low variability with several subjects using the same pathway), and it tends to be 1 when the number of distinct pathways increases with the number of the subjects (i.e. high variability with each subject performing a different pathway). A PVI of 0.35 was obtained in the first scenario, and 0.90 was the PVI in the second one - much higher than the previous case. 

The conclusion   

There is growing evidence that Caravaggio was aware of the phenomenon of the perception of images since optical studies were popular in the 16th century. It is possible that he used this knowledge to direct the construction of his paintings so that his methods still control how his work is consumed by viewers after several centuries have passed. 

Eye tracking allowed the researchers to validate this hypothesis by observing the visual pathway of the visitors to Pio Monte della Misericordia, where Caravaggio's painting, Sette Opere di Misericordia, is preserved in its original condition. Data collected from a group of 40 visitors showed that people follow a consistent pattern when observing this work of art. 

In contrast, there was no common pattern found among the visitors at the Capodimonte Museum for Caravaggio's painting, La Flagellazione di Cristo, which was on exhibit in different physical conditions from those originally intended when the work was created. Although the areas of interest were shared among most visitors, the order (visual pathway) was different for each of the participants.