Laboratory for the Study of Metacognition and Advanced Learning Technologies

The main objective of our research lab is to examine the role of cognitive, metacognitive, affective, and motivational self-regulatory processes during learning with advanced learning technologies (e.g., intelligent tutoring systems, hypermedia, simulations). More specifically, we aim to understand the complex interactions between humans and intelligent learning systems by using interdisciplinary methods to measure cognitive, metacognitive, affective, and motivational processes and their impact on learning and transfer. To accomplish this goal, we conduct laboratory, classroom, and in-situ (e.g., medical simulator) studies and collect traditional data (e.g., learning outcomes, self-reports) as well as rich, multi-channel trace data (e.g., eye tracking, facial expressions of emotions, physiological arousal, log files) to develop models of human-computer interaction; examine the nature of temporally unfolding self- and other-regulatory processes (e.g., human-human and human-artificial agents); and, design intelligent learning and training systems to detect, track, model, and foster humans' self-regulatory processes.

People

Michelle Taub
Post-Doctoral Scholar

Nicholas V. Mudrick
Ph.D. Student


Amanda E. Bradbury
Ph.D. Student

Megan J. Price
Ph.D. Student

Elizabeth B. Cloude
Ph.D. Student


Click here for a list of past SMART Lab members.

Recent Publications

Journal Articles

  • Harley, J.M., Carter, C.K., Papaionnou, N., Bouchet, F., Azevedo, R., Landis, R. L., & Karabachian, L. (2016). Examining the predictive relationship between personality and emotion traits and students’ agent-directed emotions: Towards emotionally-adaptive agent-based learning environments. User Modeling and User-Adapted Interaction, 26, 177-219.
  • Trevors, G., Feyzi-Behnagh, R., Azevedo, R., & Bouchet, F. (2016). Self-regulated learning processes vary as a function of epistemic beliefs and contexts: Evidence from eye tracking and concurrent and retrospective reports. Learning and Instruction, 42, 31-46.

Book Chapters

  • Taub, M., Martin, S. A., Azevedo, R., & Mudrick, N. V. (2016). The role of pedagogical agents on learning: Issues and trends. In F. Neto, R. Souza, & A. Gomes (Eds.), Handbook of research on 3-D virtual environments and hypermedia for ubiquitous learning (pp. 362-386). Hershey, PA: IGI Global.
  • Azevedo, R., Taub, M., Mudrick, N., Farnsworth, J., & Martin, S. A. (2016). Interdisciplinary research methods used to investigate emotions with advanced learning technologies. In M. Zembylas & P. Schutz (Eds.), Methodological advances in research on emotion and education (pp. 231-243). Amsterdam, The Netherlands: Springer.

Refereed Conference Proceedings

  • Lallé, S., Mudrick, N. V., Taub, M., Grafsgaard, J. F., Conati, C. & Azevedo, R. (2016). Impact of individual differences on affective reactions to pedagogical agents scaffolding. In D. Traum et al. (Eds.), Proceedings of the 16th International Conference on Intelligent Virtual Agents—Lecture Notes in Computer Science 10011 (pp. 269-282). The Netherlands: Springer.
    [Winner of the Best Conference Paper Award]
  • Azevedo, R., Martin, S. A., Taub, M., Mudrick, N. V., Millar, G. C., & Grafsgaard, J. F. (2016). Are pedagogical agents’ external regulation effective in fostering learning with intelligent tutoring systems? In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th International Conference on Intelligent Tutoring Systems—Lecture Notes in Computer Science 9684 (pp. 197-207). The Netherlands: Springer.
    [Winner of the Best Conference Paper Award]
  • Bouchet, F., Harley, J., & Azevedo, R. (2016). Can adaptive pedagogical agents’ prompting strategies improve students’ learning and self-regulation? In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th International Conference on Intelligent Tutoring Systems—Lecture Notes in Computer Science 9684 (pp. 368-374). The Netherlands: Springer.
  • Martin, S. A., Azevedo, R., Taub, M., Mudrick, N., Millar, G., & Grafsgaard, J. (2016). Are there benefits of using multiple pedagogical agents to support and foster self-regulated learning in an intelligent tutoring system? In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th International Conference on Intelligent Tutoring Systems—Lecture Notes in Computer Science 9684 (pp. 273-279). The Netherlands: Springer.
  • Taub, M., & Azevedo, R. (2016). Using eye-tracking to determine the impact of prior knowledge on self-regulated learning with an adaptive hypermedia- learning environment? In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th International Conference on Intelligent Tutoring Systems—Lecture Notes in Computer Science 9684 (pp. 34-47). The Netherlands: Springer.
  • Taub, M., Mudrick, N., Azevedo, R., Millar, G. Rowe, J., & Lester, J. (2016).Using multi-level modeling with eye-tracking data to predict metacognitive monitoring and self-regulated learning with Crystal Island. In A. Micarelli, J. Stamper, & K. Panourgia (Eds.), Proceedings of the 13th International Conference on Intelligent Tutoring Systems—Lecture Notes in Computer Science 9684 (pp. 240-246). The Netherlands: Springer.

Conference Presentations

  • Azevedo, R., Taub, M., Mudrick, N., Martin, S. A., & Grafsgaard, J. (2016, August). Measuring and supporting the dynamic interplay between self- and externally-regulated learning with advanced learning technologies. Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Azevedo, R., Taub, M., Mudrick, N., Martin, S. A., & Grafsgaard, J. (2016, August). Using adaptive scaffolding by animated pedagogical agents to improve self-regulation during complex learning: Evidence from multi-modal trace data. Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Martin, S. A., Mudrick, N., Taub, M., & Azevedo, R. (2016, August). The importance of regulatory flexibility in learning with advanced learning technologies. Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Mudrick, N., Taub, M., & Azevedo, R. (2016, August). Multimedia discrepancies and their influence on metacomprehension during multimedia learning. Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Mudrick, N., Taub, M., & Azevedo, R. (2016, August). Using eye-movements to understand metacomprehension during learning with multimedia discrepancies. Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Taub, M., Mudrick, N., & Azevedo, R. (2016, August). Using multi-level models to predict how metacognitive monitoring predicts performance assessment with MetaTutor. Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Wortha, F., Azevedo, R., Taub, M., Mudrick, N., Martin, S. A., & Millar, G. C., & Narciss, S. (2016, August). Judgements of learning during learning with hypermedia: How do they affect study time allocation and study behaviors? Paper to be presented at the biennial meeting of the European Association for Research on Learning and Instruction (EARLI) Metacognition SIG, Nijmegen, The Netherlands.
  • Azevedo, R., Martin, S. A., Taub, M., Mudrick, N., Millar, G., & Grafsgaard, J. (2016, June). Are pedagogical agents’ external regulation effective in fostering learning with intelligent tutoring systems? Paper presented at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Azevedo. R., Mudrick, N. V., Taub, M., Martin, S., Wortha, F., & Millar, G. (2016, June). The coupling between metacognition and emotions during STEM learning with advanced learning technologies: A critical analysis and implications for future research. Paper presented at the 2nd International Workshop on Affect, Meta-Affect, Data and Learning (AMADL 2016) at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Bouchet, F., Harley, J., & Azevedo, R. (2016, June). Can adaptive pedagogical agents’ prompting strategies improve students’ learning and self-regulation? Paper presented at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Martin, S. A., Azevedo, R., Taub, M., Mudrick, N., Millar, G., & Grafsgaard, J. (2016, June). Are there benefits of using multiple pedagogical agents to support and foster self-regulated learning in an intelligent tutoring system? Paper presented at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Martin, S., Grafsgaard, J., Mudrick, N. V., Taub, M., & Azevedo. R. (2016, June). On the feasibility of providing real-time adaptive support for motivation and emotion in intelligent tutoring systems. Paper presented at the 2nd International Workshop on Affect, Meta-Affect, Data and Learning (AMADL 2016) at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Taub, M., & Azevedo, R. (2016, June). Using eye-tracking to determine the impact of prior knowledge on self-regulated learning with an adaptive hypermedia- learning environment? Paper presented at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Taub, M., Mudrick, N., Azevedo, R., Millar, G. Rowe, J., & Lester, J. (2016, June). Using multi-level modeling with eye-tracking data to predict metacognitive monitoring and self-regulated learning with Crystal Island. Paper presented at the 13th International Conference on Intelligent Tutoring Systems (ITS 2016), Zagreb, Croatia.
  • Azevedo, R., (2016, April). Multimodal data tracking, alignment, and analyses of Metacognitive processes: Measurement issues and challenges in learner modeling. Paper presented at the annual Learning Environments Across Disciplines (LEADS) workshop, Washington, DC.
  • Taub, M., Azevedo, R., Martin, S. A., Millar, G. C., & Wortha, F. (2016, April). Aligning log-file and facial expression data to validate assumptions linking SRL, metacognitive monitoring, and emotions during learning with a multi-agent hypermedia-learning environment. Structured poster presented at the annual meeting of the American Educational Research Association, Washington, DC.
  • Taub, M., Mudrick, N. V., Azevedo, R., Markhelyuk, M., & Powell, G. S. (2016, April). Assessing middle school students' use of a metacognitive monitoring tool during learning with SimSelf. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.
  • Feyzi-Behnagh, R, Azevedo, R., Bouchet, F, & Tian, Y. (2016, April). The role of an open learner model and immediate feedback on metacognitive calibration in MetaTutor. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.
  • Wortha, F., Azevedo, R., Taub, M., Mudrick, N. V., Martin, S. A., Millar, G. C., & Narciss, S. (2016, April). Emotion profiles: The importance of emotions during learning with a multi-agent hypermedia-learning environment. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.

Projects

​​

Crystal Island
The Lost Investigation

Crystal Island is a Game-Based Learning Environment (GBLE) where college students are tasked with finding the source of a mysterious illness that has stricken the inhabitants of an island. This system has been designed based on the collaboration between the SMART Lab and the Intellimedia Group. Through educational books and posters, interactions with characters, and clues left from previous investigators, students learn as much as they can about microbiology and epidemiology to solve the mystery and save the characters on the island. Our goal is to examine the relationships between affect, agency, and STEM-learning with Crystal Island. Data is collected from multiple channels, including measures of galvanic skin response, facial expressions of emotion, and gaze behavior through eye-tracking.


Research Team

Roger Azevedo, Principal Investigator
Researcher Profile

Michelle Taub, Postdoctoral Research Scholar
Researcher Profile

Nicholas Mudrick, Ph.D. Student
Researcher Profile

Amanda E. Bradbury, Ph.D. Student
Researcher Profile

Megan J. Price, Ph.D. Student
Researcher Profile

Collaborators

James Lester, Co-Principal Investigator

Kirby Culbertson, Digital Artist

Jonathan Rowe, Research Scientist

Robert Sawyer, Graduate Assistant

Robert Taylor, Senior Research Engineer

Bradford Mott, Senior Research Scientist


Funding Source

McGill University and the Social Sciences and Humanities Research Council of Canada (SSHRC)

​​

MetaTutor
Study 5

MetaTutor is an Intelligent Tutoring System (ITS) designed to foster and scaffold college students’ Self-Regulated Learning (SRL) processes during learning about the human circulatory system. Real-time analyses of SRL data is used to provide intelligent, adaptive scaffolding by using three Pedagogical Agents (PA) embedded in MetaTutor. These pedagogical agents prompt and assist the learner using 13 SRL processes during learning. MetaTutor data is collected from multiple channels, including think-aloud protocols, screen and audio recordings, gaze behavior through eye tracking, and identification of emotional states through camera recordings and physiological sensors. The benefits of this study include a greater understanding of SRL as well as the impact of ITSs on students’ cognitive, affective, metacognitive, and motivational SRL processes.


Research Team

Roger Azevedo, Principal Investigator
Researcher Profile

Michelle Taub, Postdoctoral Research Scholar
Researcher Profile

Nicholas Mudrick, Ph.D. Student
Researcher Profile

Megan J. Price, Ph.D. Student
Researcher Profile

Elizabeth B. Cloude, Ph.D. Student
Researcher Profile


Funding Source

National Science Foundation (NSF)

Social Sciences and Humanities Research Council of Canada (SSHRC)


Collaborators

Gautam Biswas, Vanderbilt University

Eunice Jang, University of Toronto

Anila Asghar, McGill University

Cristina Conati, University of British Columbia

Inge Molenaar, Radboud University

Rebeca Cerezo, Universidad de Oviedo

Francois Bouchet, University of Paris 6

Jason M. Harley, University of Alberta

Reza Feyzi-Behnagh, SUNY-Albany

Virtual Reality in STEM
Lucid and Cary Academy

This project is based on a new partnership between North Carolina State University, a local private school (Cary Academy), and an industrial partner (LUCID DREAM). The overall goal is to induce, detect, measure, and analyze high school students’ metacognitive knowledge and regulatory skills in real-time while solving complex STEM (e.g., computer science, math, science) problems that require the use of specific computational thinking skills (e.g., abstractions and pattern generalization; systematic processing of information, etc.) using virtual reality both as a research and learning tool.

CI REFLECT
Intellimedia Group

This project is based on a collaboration between the departments of computer science and psychology. We will collect multichannel multimodal streams of data (e.g., logfiles marking activities within the GBLE, eye-tracking, facial expressions of emotions, galvanic skin response, etc.) along with traditional pre-post content knowledge and self-report tests, then analyze these data sources by combining traditional statistics with techniques such as data mining, machine learning, multilevel-modeling, and sequence mining. The overall goal is to investigate the effectiveness of theoretically grounded, adaptive reflection tools to scaffold students’ cognitive and metacognitive reflection during learning in a game-based learning environment.

​​

MetaTutor IVH


MetaTutor IVH (Intelligent Virtual Human) uses Intelligent Virtual Humans in order to address the issues that students have when monitoring their learning. Using metacognitive strategies can help students determine whether or not the content that they read while learning is actually relevant to their learning goals. However, research has shown that students are not always successful with this task, thus introducing the importance of IVHs as externally-regulating agents to foster the effective use of metacognitive Self-Regulated Learning (SRL) strategies. IVHs are computer-generated characters designed to look and behave like real people, who are able to replicate the social effects in human-human interaction. The SMART Lab aims to study the impact that these affect-rich IVHs have on students’ content evaluation and metacognitive monitoring strategies, thereby assessing how these may influence students’ responses to a series of biology questions about different human body systems.