NAIRR: National Artificial Intelligence Research Resource Pilot
Multimodal AI Socratic Instructor for Smartphone
Over the past three years our NSF RETTL project team has developed a free LiDAR-based smartphone app called LiDAR Motion Visualizer. This app enables students to see real-time representations (position-time and velocity-time graphs) of their motion as they move back and forth in front of flat surfaces (e.g., walls or trees).
This research has been extended through NAIRR by creating a phone-based AI ‘Socratic instructor’ that can prompt and guide users to make sense of, and to respond appropriately to the instantaneously-generated feedback. What the users see now on the phone screens are position-time and velocity-time visual graphs updated every 400 milliseconds as the users locomote and move during a series of embodied learning tasks. AI will enable us to make the feedback targeted, personalized, multi-modal just-in-time, and based on best pedagogical practices.
Project Team:
- PI Colleen Megowan-Romanowicz (Pl)
- coPI Mina Johnson-Glenberg (coPI )
- Rebecca Vieyra (coPI )
- Chrystian Vieyra (coPI )
The MAISIS (Multimodal AI Socratic Instructor for Smartphone) uses verbal, affective, and embodied motion inputs as well as information on the student’s self-generated attempts to correctly match motion graphs from the motion app to prompt
and guide thinking and learning in a way that foregrounds the construction of a causal conceptual model of motion. The AI will construct a multi-dimensional cognitive model of where the learner is in understanding kinematics andl provide nuanced feedback that will predict, explain, and respond to the progressively more complex challenges that the app presents, based on how the user is performing in real-time. The need for this AI model has emerged from our studies on the app use with middle school and high school students and this is where we expect it to have the greatest impact.
Research on Data Visualization with Physics Toolbox
NSF Award of $382,000
The National Science Foundation (NSF), one of the worldwide leaders in science and science education funding, has awarded $382,689 to the American Modeling Teachers Association in coordination with Vieyra Software and a team of educational researchers, science educators, and scientists to advance data visualization tools and research their use in student learning. In this project, Vieyra Software will aim to use novel augmented reality (AR) technology to develop a sensor-based magnetic field visualization tool to address common student struggles in 3-D visualization. This Cyberlearning grant is intended to advance both technology and our understanding of teaching and learning with technology. Our research and development team includes the following:
- Colleen Megowan Romanowicz (PI), physics education researcher *AMTA
- Mina Johnson (co-PI), cognitive psychologist *ASU, Embodied Games LLC
- Chrystian Vieyra (co-PI), software engineer *Vieyra Software
- Rebecca Vieyra (co-PI), project manager, physics educator *Vieyra Software
- Ramon Lopez, professor of physics, heliophysicist *UT-Arlington
- Heidi Baumgartner Komkov, doctoral student in electrical and computer eng. *University of Maryland
- Benjamin Xu, undergraduate student in computer science, *UT-Austin
Our work is supported by a diverse advisory group, including the following:
- Colleen Countryman, professor of physics *Ithaca College
- Alex Hornstein, computational engineer and data visualization expert *Looking Glass Factory
- Eric Klopfer, educational technology researcher *Massachusetts Institute of Technology
- Arturo Marti, professor of physics *Universidad de la República (Uruguay)
- Martin Monteiro, professor of physics *Universidad ORT Uruguay (Uruguay)
- Ann-Marie Pendrill, professor of physics *University of Gothenburg / Lund University (Sweden)
- Patricia White, high school physics teacher, mechanical engineer *Manchester Township High School
See the full announcement from the NSF Award Search below:
“Magnetic field visualizations are essential to many branches of the technology workforce. While fields are a core concept in K-14 STEM education, the field model of magnetic interactions is often poorly conveyed and regularly misunderstood by K-14 teachers and students alike. This project will significantly advance the augmented reality (AR) capabilities of personal mobile devices to help learners conceptualize magnetic fields. The proposed technology will enable users to use smartphones or tablets to visualize in real time and 3-D, the shape, magnitude and direction of magnetic fields around everyday objects. Few affordable technological solutions exist to address students’ difficulties with fields or to support instructional improvements with 3-D visualization. This innovative technology development and educational research aims to address this issue by providing a broadly accessible, easy to-use, low cost magnetic field visualization tool that will enable both guided and unguided exploration, and support the learner’s ability to make sense of magnetic field interactions in their environment. Ultimately, this project will produce low-cost visualization tools that will be available to smartphone users for use anytime, anywhere to visualize the magnetic fields in their environment or workplace.
Researchers and educational software developers will design and create a new software educational tool as well as developer resources for data-based visualization. The software will utilize the new capabilities of recently-introduced AR frameworks to determine physical location of the mobile device based upon both the inertial sensor unit including accelerometer and gyroscope, and the camera. It then overlays the screen with field information based upon the magnetometer. Magnetic field data will be collected as the user “sweeps” through space surrounding a field source by simply moving the mobile device around the physical area. The vast majority of pre-existing AR software resources utilize tools to superimpose images onto a primarily 2-D world, require the use of an image target, and focus on entertainment as the main goal, not education. The development of the proposed application for teaching and learning will work toward addressing technological challenges for developers in motion tracking, environmental understanding, and light estimation, as well as challenges associated with field data acquisition and 3-D visualization. The researchers will study how this new tool will support the teaching and learning of fields, a notoriously difficult topic for introductory learners given its reliance upon theoretical understanding and 3-D visualizations. User testing and observations in both high school and college physics classrooms will be carried out during the development process. A Magnetic Field Concept Inventory will be developed and used to measure student gains in conceptual coherence.
This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.”
Conference Report
Advancing Interdisciplinary Integration of Computational Thinking in Science
Would you like to read? Click here – Final Report