Investigating multimodal interactions for the design of learning environments: a case study in science learning

Abstract

This thesis focuses on multimodal interactions for the design of a learning environment. The process of designing such systems involves studying the benefits of multimodal interactions in learning. Therefore, it analyses the structure of the interactive space between the learner and the content to be learnt, and introduces and tests a framework to structure it. It proposes that multimodal interactions can encourage rhythmic cycles of engagement and reflection that enhance learners’ meaning construction in science concepts, such as ‘forces and motion’. The framework was the outcome of an iterative process of analysis and synthesis between existing theories and three studies with learners of different ages. Through these theory-informed studies, the significance of physical manipulation of objects and symbols through the employment of multiple modalities was emphasised as a way to facilitate learners’ meaning construction, engagement and reflection.