With the widespread commercial availability of kinesthetically-interactive computational environments, the use of gesture-based technology in education for various subjects and grade levels has been gaining momentum worldwide. The premise of natural gesture-based learning is that the use of natural gestures enhances the absorption, retention, and engagement of students in the learning process. However, the abstract nature of mathematics has historically posed a challenge in effectively leveraging visual and kinesthetic components that complement traditional learning mediums. The convergence of advancements in affordable 3D-sensing technologies, machine vision, and gaming systems offers an unprecedented ability to infuse math coursework with visually and kinesthetically interactive content, potentially providing an innovative means of addressing math proficiency in countries like the United States. This research examines the impact of kinesthetically-enhanced, game-based environments in undergraduate mathematics education at a research university in the United States. More specifically, we are completing experiments to analyze the utility of gesture-based games when the content, gesturing, and delivery are all designed synergistically for effective transfer of calculus and statistics course concepts.

The design of the natural gesture framework in development is extensible to other science, technology, engineering, and mathematics (STEM) subjects, and supports built-in customization, like the generation of gestures for the illustration and demonstration of complex mathematical concepts. The framework utilizes Omek Interactive’s Gesture Authoring Tool (GAT), and the Beckon Motion Toolkit for integration into the Unity3D game engine. These tools permit the creation of simple, content-specific, customized gestures. Conventional classroom learning materials, i.e. lecture and supplemental notes and texts, comprise the content adapted for the lesson modules embedded throughout the interactive game environment. So far, this work has focused on the game development in order to ensure consistency and accuracy of the game’s content when compared to the traditional lecture materials for the calculus and statistics courses being studied. The evolution of the natural gesture-based game prototype as an innovative educational system constitutes the key product of this research effort to-date. Development of the experimental design for the impact analysis of the novel, innovative educational platform is currently underway. A summary of the baseline historical math course grades and preliminary results comparing the natural gesture-based game environment to a conventional learning environment for a single math lesson will further be described as the results of this ongoing research study.