STEM education of the future

Prof. Dr. Jochen Kuhn explains how modern education technology facilitates teaching and learning STEM subjects, and how eye tracking enables studying its effects.

Prof. Dr. Jochen Kuhn is the chair of physics education at LMU Munich, Germany. His research interests currently focus on learning with multiple representations in STEM education and physics education through multimedia-based learning environments and technologies (smartphones, tablets, AR/VR, etc.). This includes STEM teacher training, learning with and about artificial intelligence (AI) in schools and universities, and eye tracking based examination of learning and problem-solving processes.

What is the overarching vision of your research?

Physics is still mostly perceived by many students today as artificial and not very relevant to the everyday world. The content is considered "dry" and abstract, with many relationships and processes purely theoretical and invisible.

We develop and investigate approaches to bridge these tensions between abstract and phenomenological, as well as every day and discipline-specific, through the use of multimedia learning environments. By using multimedia visualizations, so-called multiple external representations, students learn with and about cognitive tools that they also need in everyday life (e.g., interpretations of diagrams and graphs) to enable them to build these bridges themselves. To this end, we also implement modern technologies in learning environments to enhance students’ (and teachers’) education with and about such media. The reason for this is that we can expect modern technologies to be used in our everyday lives in the future.

Of course, we want to find out how and with which representations students successfully learn physics, design experiments, or solve physical problems. So, we are not only interested in the outcome of learning or problem solving but in the process itself because it allows us to better understand how we can support, encourage, or challenge students to learn successfully.

This learning support must be individualized and personalized, so different students can access the same learning content through different types of representations, with different strategies, and in different ways. Therefore, predicting learning success through machine learning techniques is also an essential aspect of our research with representations and cutting-edge learning technologies.

What has inspired you to embark on this journey, and what keeps you motivated to carry on?

There are many ways to design good physics lessons. However, learning with multiple representations offers comprehensive, static and dynamic options through the use of multimedia, and innovative learning technologies. These options will continue to increase - for education and research - especially in the wake of digitization.

Moreover, through this kind of development and research of innovative learning technologies with content-related concepts, the current and coming generation can also acquire socially relevant competencies, such as data literacy or AI literacy, and thus change society itself.

So, the multiple options and the importance for social development were causes for the choice of learning with multiple representations using innovative, cutting-edge learning technologies.

What would you highlight as the main finding of your work so far?

There is no one important result; rather, it is the realization that education is in danger of lagging behind societal needs and requirements in some areas due to the high dynamics of digitization. This means that education must receive the opportunity to keep pace with societal developments.

When you consider that it took more than a decade, especially in Germany, for an everyday medium like a tablet to find its way more or less systematically into the classroom, we need new opportunities for future developments.

Especially in educational research, we must try harder to anticipate and investigate early on which next everyday technologies could have a similar effect to smartphones and tablets. Then, valid and practicable learning concepts should be developed and studied with them, so that by the time the next generation of learning technologies is introduced, we already have empirically tested successful concepts for teaching and do not have to wait until then to start developing them.

And politics need to create frameworks and programs to enable such cutting-edge developments and research of them in close cooperation with the schools inside the classrooms.

On the one hand, it is important to involve all institutions focused on education and research in such a process. On the other hand, teachers also need to be trained to implement new approaches and technologies in their classrooms in a meaningful and targeted manner.

How has your work benefited from employing eye tracking in your experiments?

In addition to head-mounted-displays (HMDs) for augmented reality (AR) and virtual reality (VR), we estimate eye-tracking-based systems, among others, will be part of or implemented in such next-generation educational technologies (EDTech).

For example, when learning, problem solving, or experimenting with multiple representations, we use eye tracking systems of different types (stationary, mobile, integrated into VR/AR HMD) to investigate learners' visual strategies to distinguish between successful and unsuccessful learning strategies.

In addition, we can use gaze data to train AI algorithms to predict whether a problem will be solved successfully or not, using visual strategies of the learners already during the learning process. This allows us to provide personalized support before they would choose an incorrect strategy or solution - and without any additional assessments, just based on the visual strategy and completely customized.

Again, it is important to train and develop teachers to use such gaze-based systems.

From your current perspective, and extensive experience with eye tracking, what would you advise those considering adopting it in their research?

First, one should comprehensively consider what role eye tracking analyses should play in one's research.

If eye tracking is not planned to be the primary research method, I would advise to investigate the related questions rather in cooperation with experienced research partners to gain experience rather than investing in equipment when the benefit is unclear.

However, if eye tracking methods are to be established as a standard repertoire of the research group, I would suggest investing sufficient material and personnel resources exclusively for this line of research from the beginning.

Related information

Here is a selection of recent publications reporting on the work which employed the eye tracking technology:

Becker, S., Küchemann, S., Klein, P., Lichtenberger, A. & Kuhn, J. (2022). Gaze patterns enhance response prediction: More than correct or incorrect. Physical Review Physics Education Research, 18(020107).

Dzsotjan, D., Ludwig-Petsch, K., Mukhametov, S., Ishimaru, S., Küchemann, S., & Kuhn, J. (2021). The Predictive Power of Eye-Tracking Data in an Interactive AR Learning Environment. Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers, September 2021, 467–471

Klein, P., Becker, S., Küchemann, S., & Kuhn, J. (2021). Test of understanding graphs in kinematics: Item objectives confirmed by clustering eye movement transitions. Physical Review Physics Education Research, 17(1), 013102.

Kumari. N., Ruf, V., Mukhametov, S., Schmidt, A., Kuhn, J., & Küchemann, S. (2021). Mobile Eye-Tracking Data Analysis Using Object Detection via YOLOv4. Sensors, 21(22), 7668. MDPI AG.

For more on Prof. Dr. Kuhn’s lab, please visit their webpage.