When words and graphs move the eyes:The processing of multimodal causal relations

Parodi, Giovanni; Julio, Cristóbal; Recio, Inés

Keywords: Eye movement, eye tracking, causalrelations, discourse marker, multisemiotic texts, graphs, first pass reading time, second pass reading time, integrative transitions

Abstract

Research on causal relations in multisemiotic texts constituted by words and graphs has been scarce with only a few exceptions. In the current study, eye movement behavior was studied in seventy-six Chilean high school students, who read a set of twelve causally-related economics texts in Spanish in four experimental conditions. The objective is two-fold. We aimed, on the one hand, to observe the main effects of the causal discourse mark-er (DM)por tanto and the statistical causal graph (G), as well as the interaction effect of both variables on different eye tracking measures.On the other, we seek to observe the effects of the DM on the same eye tracking measures for the graph system (GS) area of interest(AOI).The findings showed that the conjoint presence of the DM and the G did not positively influence the processing of selected AOIs.Analyses also reveal no significant effects on the GS AOI.Thus, the results indicate that the DM tend to decrease processing times, while the G increases them.Additional analyses conducted on the integrative transitions between the verbal system and the graph system reveal that more transitions were identified between the consequence segment and the graph system, thereby confirming that the consequence segment is crucial for the integration of both semiotic systems.

Más información

Título de la Revista: JOURNAL OF EYE MOVEMENT RESEARCH
Volumen: 11
Número: 1
Editorial: INT GROUP EYE MOVEMENT RESEARCH
Fecha de publicación: 2018
Página de inicio: 1
Página final: 18
Idioma: English
Notas: WOS