Projects

The Institute for Student-AI Teaming will: (1) develop the theories, Artificial Intelligence (AI) technologies, and know-how for creating next-generation collaborative learning environments composed of diverse students, teachers, and AI; (2) grow a diverse workforce of future AI researchers and practitioners by engaging 5,000 middle/high school students in innovative AI education through AI-enabled pedagogies; (3) serve as a national nexus point for empowering diverse stakeholders to engage in responsible co-design of student-AI collaborative technologies. This national institute brings together a geographically distributed team of researchers from nine universities with partners from academic, K-12 school districts, and industry to address the central challenge of how to promote deep conceptual learning via rich socio-collaborative learning experiences for all students. Read more here, here, and here.

Our aim is to gain fundamental knowledge on how the eye and brain dynamically coordinate to construct mental representations during complex learning from texts. The project will use behavioral (eye tracking and reading times) and non-invasive brain measures (high-density functional near-infrared spectroscopy and electroencephalography) to identify neurobehavioral markers of core processes involved in reading comprehension (e.g., focused attention vs. mind wandering, inference generation and elaboration) and the extent to which these measures can predict learning outcomes (comprehension, knowledge, and transfer) assessed immediately after learning and a week later. The project will also leverage advances in deep (machine) learning to build computational neurobehavioral models of the process and outcome variables in a manner that is sensitive to local reading context (e.g., textual difficulty, cohesion) and to individual differences (e.g., prior knowledge, interest). This project is in collaboration with Leanne Hirshfield. [National Science Foundation].

The project utilizes sensor technologies for tracking team behavior in information workplaces. The first aim is to develop computational models of critical team states and processes (e.g., team cohesion, team coordination, team mood/affect), based on unobtrusive, continual, longitudinal sensing of physiology, behavior, and communication in a real-world context to understand factors that lead to team effectiveness. The second aim is to use those insights to develop and test an intelligent (AI-based) team facilitator. The results will contribute to a new understanding on how 21st century teams can manage complexity, how team heterogeneity can lead to team effectiveness, and will identify successful strategies for team adaptability. This work is in collaboration with Gloria Mark from UC Irvine and Aaron Striegel from Notre Dame. Learn more here. [National Science Foundation].

The project investigates disruptions to teamwork and how teams adapt during the COVID-19 crisis and in the ensuing recovery period. We study 30 real-world teams over a four-month period to understand how teams respond to changing contexts, how teams support each other, how conflict is managed, and how teams develop, adapt, and sustain the rhythms of teamwork during COVID-19. This foundational research will be essential to help organizations establish team structures and collaborative processes that enable them to more successfully address disruptions in the current and in future crises. This work is in collaboration with Aaron Striegel from Notre Dame. [National Science Foundation].

Collaborative problem solving (CPS) is an essential 21st century skill in our increasingly connected and globalized world. . One goal of this project is to discover how interpersonal interactions arise and influence CPS processes and outcomes in digital STEM learning environments. A second goal is to model dynamic CPS processes using nonlinear time series analyses and multimodal deep recurrent neural networks. Increasing basic understanding and computational modeling of emergent interpersonal processes is a critical step towards designing next-generation STEM learning environments that aim to make CPS more enjoyable, engaging, and effective. This project is in collaboration with Nick Duran from Arizona State U and Val Shute from Florida State U. [National Science Foundation].

Personnel selection is a foundational challenge for all organizations given they share an interest in hiring individuals who have the necessary attributes for job performance. To reduce the burden of applicant screening and selection, many large organizations are turning to machine learning (ML) methods to augment humans in making selection decisions. Machine learning-based tools can lead to substantial savings in personnel selection. Yet, these systems can inadvertently perpetuate bias. Our goals are to: (1) Understand gender differences in expressed behaviors and interviewer ratings (trained and untrained interviewers) using ML techniques; (2) Use these insights to reduce predictive discrepancies between men and women by accounting for it in the models. This work is in collaboration with Louis Tay and Sang Eun Woo from Purdue University. [National Science Foundation].

Our goal is to provide automated feedback on teacher discourse towards improving teaching effectiveness and student achievement. We design hardware/software interfaces for efficient, flexible, scalable audio data collection by teachers. The data is used to computationally model dimensions of effective discourse by combining linguistic, discursive, acoustic, and contextual analysis of audio with supervised and semi-supervised deep recurrent neural networks. The models are incorporated into an interactive analytic/visualization platform to promote data-driven reflective practice. Collaborators include Sean Kelly and Amanda Godley (University of Pittsburgh) and Patrick Donnelly (Cal State Chico). [National Science Foundation].

The purpose of this project is to develop a framework to assess and identify collaborative problem-solving (CPS) skills in computer-based educational environments and to explore the relationship between collaborative problem solving and student learning outcomes. This research aims to address three key questions: (1) what is the CPS construct in general and in the paradigm of computerized learning environments, and what student actions and behaviors constitute evidence; (2) what is the nature of the association between CPS and student learning outcomes, particularly in computerized educational environments; and (3) how can a model be developed to analyze and automate assessment of collaborative problem solving skills. This project is in collaboration with Jessica Andrews and Tanner Jackson from ETS and Greg Chung from UCLA. [Institute for Education Sciences]

In collaboration with Val Shute at Florida State University and Ryan Baker at University of Pennsylvania, we will study theoretically-guided learning supports and elements of their design as malleable factors that can improve both the learning experience and outcomes in STEM learning games. Our goal is to enhance understanding of the types of cognitive and affective supports that promote formal STEM learning, enhance science interest, and improve the learning experience. The knowledge gained will contribute to the design of next-generation learning games that blur the distinction between assessment and learning. [Institute for Education Sciences]

The R&D Virtual Learning Center is a collaboration between the University of Florida, CU Boulder, and StudyEdge. It will be at the forefront of an emerging field known as “precision education,” using prior students’ data to support the learning opportunities for future students. It will address the challenge of improving algebra outcomes for low-achieving students using Algebra Nation. The research team will personalize Algebra Nation for students by analyzing data from prior users, develop indicators of engagement during learning, and design professional development to help teachers use learning analytics to differentiate instruction. Visit project page here. [Institute for Education Sciences]

From carefully crafted messages to flippant remarks, warm expressions to unfriendly tones, teachers’ behaviors set the tone, expectations, and attitudes of the classroom. Importantly, teacher behaviors are not perceived in the same way by all students; rather, students’ background plays a key role in how these behaviors are interpreted. Using videos from 6th to 9th grade math classes, student self-report questionnaires, and achievement data from the Measures of Effective Teaching (MET) project, we aim to use human coding as well as automated speech and language processing to identify teacher verbal and nonverbal behaviors that are related to students’ psychological and academic outcomes, particularly for students with stigmatized identities. Collaborators include Sean Kelly and Stephanie Wormington. [Mindset Scholars Network] via a grant from the Bill & Melinda Gates Foundation.

We study how high-school seniors’ mindsets about intelligence and effort, adduced from large volumes of biographical data (biodata), predict college success, and how aspects of the family, home, and neighborhood environment influence thee relationships. Our project leverages a large six-year longitudinal dataset from a nationally representative sample of U.S. undergraduate students who enrolled in college in 2009. Its sheer size affords a “big biodata” approach to mindset science, featuring machine learning, multivariate clustering, and an emphasis on moderation and generalizability. This project is in collaboration with Angela Duckworth from University of Pennsylvania and Margo Gardner from Columbia University. [Mindset Scholars Network]

Working with Jim Brockmole, we aim to blend basic research focused on why, when, and how minds wander with advances in eye tracking, mental state estimation, and conversational learning technologies to advance a new genre of attention-aware learning technologies that automatically detect and combat wandering minds. Our exemplary technological innovation leverages emerging consumer-grade eye tracking and recent advances in mental state estimation to add an attention-aware computing layer to Guru, a cyberlearning technology for high-school biology. The research will be conducted in 9th grade biology classrooms in Northern Indiana. Learn more here. [National Science Foundation]

This project is part of the Multimodal Objective Sensing to Assess Individuals with Context (MOSAIC) program. Our teamm called Tesserae, consists of researchers from U of Notre Dame (lead), Carnegie Mellon University, Dartmouth College, Georgia Institute of Technology, Ohio State University, University of California Irvine, University of Colorado-Boulder, University of Washington, and the University of Texas. During the study, 750 participants will wear an activity tracker that is paired with a smartphone app to gauge biomarkers like heart rate, sleep, physical activity and stress, as well as daily patterns — things people normally track for their own personal health. All of these factors contribute to overall well-being and workplace performance. Passive sensors will also collect information about the workplace, such as ambient noise and light levels, to contextualize participant activity. [Intelligence Advanced Research Projects Activity (IARPA)]

We leverage findings suggesting that language and thought are inextricably coupled to investigate: (1) how mindsets and motivation elucidated from manual coding of students’ open-ended descriptions about their real-world extracurricular and work experiences (Ex/Wk) mediate the relationship between sustained Ex/Wk participation and college success; (2) how natural language processing and machine learning techniques can be used to automatically measure these constructs at scale; and (3) how the family, school, and neighborhood environment moderate the relationship between Ex/Wk experiences, language-derived mindset and motivation, and college success.&nbsp This project is in collaboration with Angela Duckworth from University of Pennsylvania and Margo Gardner from Columbia University. [Mindset Scholars Network]

Our aim is to create a suite of reliable, valid, cost-effective, and easy-to-administer tasks which middle and high school students can complete during a typical school period. Because they do not rely upon the subjective judgments of students (or teachers or parents), performance tasks offer a potential solution to the reference bias problem. That is, because student behavior on our tasks will be assayed directly, they obviate subjective judgments about behavior that are necessarily influenced by standards of comparison that may vary across schools (or countries or even across time as individuals develop new standards for what “average” looks like). As an exploratory aim, we will investigate alternatives to performance tasks for assessing grit and self-control, including “big data” techniques (e.g., analyzing patterns of use of Kahn Academy). This project is in collaboration with Angela Duckworth from the University of Pennsylvania and Louis Tay from Purdue University. [Walton Family Foundation]

For over a century, research has documented the dominant configuration of lecture, recitation, and seatwork in American schools. Recent research looking at the role of classroom discourse, i.e., interactions between teachers and students, has confirmed, as an alternative to this configuration, the importance of open discussions prompted by open-ended teacher questions ("authentic teacher questions") in reading and literature instruction. The goal of our project, using cutting-edge research in speech recognition, discourse classification, and natural language understanding (NLU), is to develop CLASS 5.0, a computer program that will autonomously code classroom interactions between teachers and their students. Collaborators include Martin Nystrand (University of Wisconsin-Madison), Andrew Olney and Art Graesser (University of Memphis), and Sean Kelly (University of Pittsburgh). Visit project page here. [Institute of Education Sciences]

The foundational research question is how engagement emerges from complex three-way interactions among the learners themselves (i.e., individual differences), the instructional materials (i.e., text difficulty), and the learning activities (i.e., task control and task value). A distinctive goal is to track the dynamics of emergent engagement trajectories via state-of-the-art technologies and methods from affective computing, eye tracking, and nonlinear dynamical systems. The possibility of promoting engagement and learning will also be considered by developing predictive software that selects activities and materials in a manner that is sensitive to the traits, needs, and styles of individual learners.

[This project is funded by the National Science Foundation]

We develop and validate a suite of performance tasks of academic diligence in middle school children. Our approach is grounded in the deliberate practice framework, which posits that skill is the consequence of sustained effort on challenging practice activities, repeated over time, with feedback and guidance. We hypothesize that diligence may distinguish students who persist in tedious tasks from those who quickly disengage by switching to less beneficial but more enjoyable alternatives. The primary outcome of this project is computer software for the psychometrically validated behavioral measure of diligence. The software will be designed to be scalable to run on any computer or mobile device, and extensible to allow other researchers to customize it as needed. This project is in collaboration with Angela Duckworth from the University of Pennsylvania. [John Templeton Foundation]

The overall goal of this project led by Angela Duckworth from the University of Pennsylvania is to provide new insight into student factors that predict college persistence and develop strategies to cultivate them via school-based interventions. The project entails three complementary components: (1) a longitudinal study of urban high school seniors through their first year of college; (2) an in-depth, multi-method study of urban high school seniors who have demonstrated exceptional ­learning trajectories; (3) a series of double-blind randomized intervention experiments with urban high school seniors aimed at improving their mindsets about their academic potential as well as the intellectual and social meaning of critical feedback from college professors. [Bill & Melinda Gates Foundation]

In collaboration with David Yeager and Marlone Henderson at UT Austin, this project aims to (1) create and validate three behavioral measures of “academic perseverance” and (2) develop and experimentally test in urban public middle schools a) a student-targeted “purpose” intervention designed to encourage adolescents to tie academic pursuits to higher-order goals, imbuing them with a sense of purpose around academic work and b) teacher practices designed to further reinforce in students a sense that their academic work serves a larger purpose. Each intervention in (2) above is predicted to improve outcomes as assessed by the perseverance measures described in (1) above, in addition to raising overall grades. [John Templeton Foundation]

In collaboration with Ryan Baker from Teacher's College Columbia and Valerie Shute from Florida State University, this project focuses on building automated detectors of students emotions while they learn physics by playing a fun and engaging educational game called Newton's Playground. The project combines multimodal assessment of student affect and engagement from automated facial feature analysis and interaction patterns with stealth assessment of conscientiousness and conceptual physics understanding. The goal is to figure out the ways that specific affective states disengaged behaviors, and conscientiousness interact and ultimately influence learning. [Bill & Melinda Gates Foundation]

We are collaborating with QUASAR USA to develop a computerized tutoring platform that adapts its teaching strategy to students in real-time by monitoring their brain activity. As part of this program, we will conduct trials with high school students using its dry electrode headsets to measure EEG in a classroom environment. EEG has the potential to enhance instruction by giving the tutoring system unique insight into the student's cognitive workload and engagement. [National Science Foundation]

This project focuses on the affective state of confusion with an emphasis on the following research questions: (1) What are the appraisals that lead to confusion? (2) How is confusion expressed in the face, speech, body, physiology, language, and context? (3) What are the temporal dynamics of confusion,? (4) How is confusion effectively regulated? (5) When is confusion beneficial for learning? We have developed computerized interventions that induce, track, and regulate confusion to test the hypothesis that there might be some benefits to productively confusing learners. [National Science Foundation]

Working in collaboration with researchers at Sandia National Laboratories, the University of Memphis, and the University of Notre Dame, we attempt to identify skills that may differentially affect performance of individuals in cognitive tasks relevant to flying airplanes and communicating with team members. The project attempts to identify or develop measures to quantify individual ability with respect to each identified skill, particularly the ability to flexibly switch strategies in response to dynamically changing task constraints. [Sandia National Laboratories]

The goal of this research is to build and test learning environments that coordinate complex learning and learner emotions. The project augments an existing intelligent tutoring system (AutoTutor) that helps learners construct explanations by interacting with them in natural language and helping them use simulation environments. The tutorial dialogue of AutoTutor will be enhanced in the proposed research by incorporating signal processing algorithms and sensing devices that classify various facial patterns and affective states of learners. [National Science Foundation]

This project, in collaboration with Andrew Olney, investigates expert tutoring mechanisms at multiple levels including models, modes, and moves. We are developing broad computational models of expert tutors that encompasses their pedagogical and motivational strategies, dialogue, language, affective responses, and gestures. The overall goal of the project is to develop a computer tutor for high school biology based on strategies and dialogue of expert human tutors. The tutor could have a big impact on Memphis City Schools because it seeks to improve educational outcomes on the Tennessee Gateway Science Test, which high school students must pass in order to receive a diploma.[Read More] [Institute of Education Sciences]

MIKI is a three-dimensional directory assistance-type digital persona displayed on a prominently-positioned 50 inch plasma unit housed at the FedEx Institute of Technology at the University of Memphis. MIKI, which stands for Memphis Intelligent Kiosk Initiative, guides students, faculty and visitors through the Institute’s maze of classrooms, labs, lecture halls and offices through graphically-rich, multidimensional, interactive, touch and voice sensitive digital content. MIKI differs from other intelligent kiosk systems by its advanced natural language understanding capabilities that provide it with the ability to answer informal verbal queries without the need for rigorous phraseology. [FedEx Institute of Technology]

This project focused on characterizing the performance of RFID tags in a GHz Transverse Electromagnetic (GTEM) cell. Performance of four commercially available RFID tags manufactured by different vendors was characterized on the basis of horizontal directivity,vertical directivity, sensitivity, and frequency characteristics. With these baseline characteristics determined, we moved two of the four tags through a real world environment in three dimensions using an industrial robotic system to determine the effect of asset position in relation to the reader on tag readability.

Led by Stan Franklin, LIDA is a cognitive architecture that aspires to model several facets of human and animal cognition. LIDA incorporates sophisticated action selection, a centrally important attention mechanism, and multimodal instructionalist and selectionist learning mechanisms. Empirically grounded in cognitive science and neuroscience, the architecture is strictly neither symbolic nor connectionist, but blends crucial features of each. LIDA is a successor of IDA, an agent that helps the Navy by assigning sailors to jobs. IDA is a very complex agent that perceives e-mails from sailors, deliberates on the right jobs for the sailor and negotiates with the sailor in the context of sailor's preferences and Navy's policies.