Publications

Recommended Citation (APA):
Gobert, J., Dickler, R., & Adair, A. (2024). Using an AI-based dashboard to help teachers support students’ learning progressions for science practices. In H. Jin, D. Yan, & J. Krajcik (Eds.), Handbook of research on science learning progressions. Taylor & Francis Group.

Abstract:
Inq-Blotter, a teacher dashboard, provides teachers with formative assessment data and actionable alerts on students’ competencies at science practices. Assessment data on students’ competencies in science practices and their respective sub-components are generated in realtime by patented algorithms as students conduct an inquiry in Inq-ITS (Inquiry Intelligent Tutoring System). This chapter begins with a description of how Inq-ITS and Inq-Blotter are used for both assessment and instruction. Following this, it discusses a study that used Epistemic Network Analysis to characterize teachers’ discourse, elicited by Inq-Blotter’s alerts, that led to robust improvement in science practices that students could apply over a series of Inq-ITS formative assessment labs.

Recommended Citation (APA):
Segan, E., Gobert, J., Pedro, M. S., Adair, A., & Owens, J. A. (2024). Using AI-based assessment and scaffolds to identify student difficulties with plotting data and modeling in virtual science labs. In R. Lindgren, T. I. Asino, E. A. Kyza, C. K. Looi, D. T. Keifert, & E. Su ́arez (Eds.), Proceedings of the 18th International Conference of the Learning Sciences - ICLS 2024 (pp. 1466-1469). International Society of the Learning Sciences

Abstract:
Developing proficiency in science practices, including using mathematics, outlined in the Next Generation Science Standards is essential for success in STEM courses and future careers. However, students often struggle with developing mathematical models, which limits their ability to understand scientific phenomena. To improve students’ learning and teachers’ assessment, we extended Inq-ITS to automatically assess and scaffold students’ competencies in developing mathematical models of scientific phenomena. We analyzed student data from six virtual science labs in Inq-ITS at both the practice level and the sub-practice level to determine if they maintained their mathematical competencies with scaffolding. By operationalizing and analyzing data at the sub-practice level, the results provide valuable formative data regarding the challenges students face when developing mathematical models about scientific phenomena, which in turn, can inform future scaffolds across science domains.

Recommended Citation (APA):
Adair, A., Sao Pedro, M., Gobert, J., & Segan, E. (2023). Real-time AI-driven assessment and scaffolding that improves students’ mathematical modeling during science investigations. In N. Wang, G. Rebolledo-Mendez, N. Matsuda, O. C. Santos, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 202-216). Springer, Cham.

Abstract:
Developing models and using mathematics are two key practices in internationally recognized science education standards, such as the Next Generation Science Standards (NGSS). However, students often struggle at the intersection of these practices, i.e., developing mathematical models about scientific phenomena. In this paper, we present the design and initial classroom test of AI-scaffolded virtual labs that help students practice these competencies. The labs automatically assess fine-grained sub-components of students’ mathematical modeling competencies based on the actions they take to build their mathematical models within the labs. We describe how we leveraged underlying machine-learned and knowledge-engineered algorithms to trigger scaffolds, delivered proactively by a pedagogical agent, that address students’ individual difficulties as they work. Results show that students who received automated scaffolds for a given practice on their first virtual lab improved on that practice for the next virtual lab on the same science topic in a different scenario (a near-transfer task). These findings suggest that real-time automated scaffolds based on fine-grained assessment data can help students improve on mathematical modeling.

Recommended Citation (APA):
Owens, J. A., Adair, A., Segan, E., & Gobert, J. (2023, June). Automated analyses of students’ difficulties with explanations in science inquiry [Conference presentation]. Annual Meeting of the Society for Text and Discourse, Olso, Norway.

Abstract:
Students have difficulties constructing explanations and arguments in scientific inquiry. The current study made use of McNeill et al.’s (2006) claim, evidence, and reasoning framework to elicit students’ competencies with constructing scientific explanations and engaging in argumentation and natural language processing algorithms to score students’ Claim-Evidence-Reasoning statements at a fine-grained level. The analysis showed that although students would benefit from greater support in all three components, they struggled the most with particular aspects of Reasoning.

Recommended Citation (APA):
Adair, A., Sao Pedro, M., Gobert, J., & Owens, J. A. (2023). Assessing students’ competencies with mathematical models in virtual science inquiry investigations. In P. Blikstein, J. Van Aalst, R. Kizito, & K. Brennan (Eds.), 17th International Conference of the Learning Sciences (pp. 914-917). International Society of the Learning Sciences.

Abstract:
Developing models and using mathematics are two key practices in internationally recognized science education standards such as the Next Generation Science Standards (NGSS, 2013). In this paper, we used a virtual performance-based formative assessment to capture students’ competencies at both developing and evaluating mathematical models in science inquiry contexts aligned with the NGSS (2013). Results show that model development and evaluation competencies are correlated, but students who demonstrate proficiency with model development often struggle with evaluation. Nuanced data illustrate how components of modeling competencies differ and how they may be related.

Recommended Citation (APA):
Adair, A., Dickler, R. F., Gobert, J., Sao Pedro, M., Olsen, J., Owens, J. A., & Lott, C. (2023, April). Examining students' mathematical evidence in CER explanations during science inquiry contexts [Symposium poster]. American Educational Research Association (AERA) Annual Meeting.

Abstract:
In the present study, we qualitatively examined students’ explanations about mathematical relationships between physical science variables written in a Claim, Evidence, Reasoning (CER) format. Results showed that the quality of students’ responses varied, illustrating the need for fine-grained, evidence-based rubrics that can identify precisely how students are struggling at CER so that teachers can best support students on their scientific explanation writing.

Recommended Citation (APA):
Li, H., Adair, A., Li, G., Dickler, R. F., & Gobert, J. (2023, April). Evaluation of automated scoring methods for students’ claim, evidence, reasoning responses in science [Symposium poster]. American Educational Research Association (AERA) Annual Meeting.

Abstract:
Given the importance of developing students’ competencies on NGSS practices, the present study evaluated two different automated scoring methods for written claims, evidence, and reasoning (CER): the RegEx method and the WordDistance method. Results showed that both methods performed moderately well in scoring students’ CER responses. These findings have implications for implementing automated assessments and facilitating personalized scaffolding to support students’ CER writing.

Recommended Citation (APA):
Adair, A. (2023). Teaching and learning with AI: How artificial intelligence is transforming the future of education. XRDS: Crossroads, The ACM Magazine for Students, 29(3), 7-9.

Recommended Citation (APA):
Adair, A., & Koh, J. J. (2023). Making speech recognition work for children: An interview with Amelia Kelly. XRDS: Crossroads, The ACM Magazine for Students, 29(3), 26-29.

Recommended Citation (APA):
Sao Pedro, M., Gobert, J., Adair, A., Dickler, R., & Betts, C. (2022, November). Using mathematics to deepen understanding of scientific phenomena [Conference presentation]. Kentucky Science Teachers Association Annual Conference, Richmond, KY, United States.

Abstract:
The Next Generation Science Standards (NGSS) emphasize the importance of using mathematical representations to model, make predictions about, and deepen understanding of science phenomena. However, students often struggle at integrating mathematics and science in a myriad of ways. Join us to learn how graphical reasoning can help students bridge the STEM gap and achieve proficiency as envisioned by the NGSS. In this interactive session, you will learn how self-grading virtual labs paired with an Artificial Intelligence-based alerting dashboard can help your students become proficient at understanding graphs, making predictions, and using mathematics to deepen content understanding. Key takeaways include: (1) deepened understanding of the design tradeoffs necessary to create rich, formative assessment activities capable of yielding fine-grained data about students’ difficulties; (2) techniques to help students who lack algebraic fluency bridge the gap between science and mathematics; and (3) research-backed ways to provide real-time feedback to students that have been shown to improve their learning. All attendees will leave with demo access to our materials that they can use immediately with students for authentic assessment and instruction of NGSS practices, specifically using mathematics (NGSS Strand 5).

Recommended Citation (APA):
Olsen, J., Adair, A., Gobert, J., Sao Pedro, M., & O’Brien, M. (2022). Using log data to validate performance assessments of mathematical modeling practices. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 488-491). Springer, Cham.

Abstract:
Many national science frameworks (e.g., Next Generation Science Standards) argue that developing mathematical modeling competencies is critical for students’ deep understanding of science. However, science teachers may be unprepared to assess these competencies. We are addressing this need by developing virtual lab performance assessments that assess these competencies in science inquiry contexts. Through our design processes, we developed a method for validating the assessments that takes advantage of the unique opportunities afforded by collecting log data. Here, we describe this method and demonstrate its utility by analyzing students’ competencies with one example sub-practice of mathematical modeling, plotting controlled data generated from a simulation.

Recommended Citation (APA):
Herrick, K. S., Nachman, L., Montilus, K. D., Runyon, K. R. M., Adair, A., & Ferrara, L. (2022). An integrated approach to learning solutions: UCD + LS&D + AIEd. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 94-98). Springer, Cham.

Abstract:
User-centered design (UCD), learning science and instructional design (LS&D), and AI in education (AIEd) can be powerful, yet siloed practices when developing educational products. This paper describes our Agile ways of working in the ETS® AI Labs™ and how we are taking an integrated approach to educational product development. We discuss lessons learned and ways we can bridge the gap between learning theories and practices, artificial intelligence, and user experience/research to craft effective learning solutions.

Recommended Citation (APA):
Adair, A., Owens, J., & Gobert, J. (2022). Using epistemic network analysis to explore discourse patterns across design iterations of a teacher dashboard. In C. Chinn, E. Tan, C. Chan, & Y. Kali (Eds.), 16th International Conference of the Learning Sciences (pp. 297-304). International Society of the Learning Sciences.

Abstract:
Providing high-level support to students on NGSS inquiry practices can be challenging; however, teacher dashboards can help teachers provide just-in-time instruction to students, both in-person and online. Prior work has shown some success with a dashboard that alerts teachers in real time on students’ science inquiry difficulties, but teachers differed in their use of the alerts. To further support teachers, we designed a second iteration, in which the alerts included actionable, evidence-based Teacher Inquiry Practice Supports (TIPS), a series of suggested scaffolds that teachers can use to support students on the practices with which they are struggling. In this study, we investigate how the discursive support patterns from one teacher differed when using the dashboard alerts without TIPS followed by with TIPS. Findings suggest that TIPS influenced how the teacher incorporated different types of support for her students, and further, that the support given varied across different virtual lab stages.

Recommended Citation (APA):
Sao Pedro, M., Gobert, J., Adair, A., Olsen, J., Dickler, R., & Betts, C. (2022, March). Using mathematics to deepen understanding of scientific phenomena [Conference presentation]. National Science Teaching Association’s National Conference on Science Education, Houston, TX, United States.

Recommended Citation (APA):
Sao Pedro, M., Gobert, J., & Adair, A. (2021, November). Strengthening students’ inquiry competencies via progress monitoring [Conference presentation]. Science Teachers Association of New York State (STANYS) Conference, Rochester, NY, United States.

Recommended Citation (APA):
Lee, J., Adair, A., Gobert, J., & Dickler, R. (2021, August). Can text features of investigative questions in science predict students’ inquiry competencies? [Conference presentation]. Annual Meeting of the Society for Text and Discourse.

Abstract:
To fully realize the vision of the Next Generation Science Standards, students must be able to make sense of the scientific text that guides their inquiry and understanding of science phenomena. In this study, we analyze the text features of several investigative questions in the Inquiry Intelligent Tutoring System (Inq-ITS) to understand the ways in which the difficulty of the investigative questions relates to students’ competencies with science inquiry practices. Results show that certain word-level features (e.g., familiarity, concreteness) of the investigative questions can explain students’ performances on science inquiry, but that these features differ across science topics.

Recommended Citation (APA):
Dickler, R., Gobert, J., Adair, A., & Olsen, J. (2021, August). Using a teacher dashboard to support students remotely on science inquiry [Conference presentation]. Annual Meeting of the Society for Text and Discourse.

Abstract:
In this study, we examined teachers’ remote use of a science dashboard, Inq-Blotter, through discourse analyses and students’ corresponding inquiry performance in Inq-ITS. Specifically, Epistemic Network Analyses were applied to compare patterns of support elicited by Inq-Blotter when students improved versus did not improve on inquiry in Inq-ITS. Analyses revealed significant differences in teacher support patterns in relation to student improvement. These results demonstrate how dashboards can support science discourse and learning during remote instruction.

Recommended Citation (APA):
Dickler, R., Adair, A., Gobert, J., Hussain-Abidi, H., Olsen, J., O’Brien, M., & Sao Pedro, M. (2021). Examining the use of a teacher alerting dashboard during remote learning. In I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 134-138). Springer, Cham.

Abstract:
Remote learning in response to the COVID-19 pandemic has introduced many challenges for educators. It is important to consider how AI technologies can be leveraged to support educators and, in turn, help students learn in remote settings. In this paper, we present the results of a mixed-methods study that examined how teachers used a dashboard with real-time alerts during remote learning. Specifically, three high school teachers held remote synchronous classes and received alerts in the dashboard about students’ difficulties on scientific inquiry practices while students conducted virtual lab investigations in an intelligent tutoring system. Quantitative analyses revealed that students significantly improved across a majority of inquiry practices during remote use of the technologies. Additionally, through qualitative analyses of the transcribed audio data, we identified five trends related to dashboard use in a remote setting, including three reflecting effective implementations of dashboard features and two reflecting the limitations of dashboard use. Implications regarding the design of dashboards for use across varying contexts are discussed.

Recommended Citation (APA):
Dickler, R., Sao Pedro, M., Adair, A., Gobert, J., Olsen, J., Kleban, J., Betts, C., Staudenraus, C., & Roughan, P. (2021). Supporting students remotely: Integrating mathematics and sciences in virtual labs. In E. de Vries, Y. Hod, & J. Ahn (Eds.), 15th International Conference of the Learning Sciences (pp.1013-1014). International Society of the Learning Sciences.

Abstract:
Tools that automatically assess and support students are important during remote instruction due to COVID-19 because students do not have direct access to teacher support. We present results of the remote use of virtual labs in an Inquiry Intelligent Tutoring System (InqITS), which captures student performance on practices at the intersection of mathematics and science. Implications are discussed for the development of scaffolding and design of labs to support remote instruction.

Recommended Citation (APA):
Dickler, R. & Adair, A. (2021, April). Using the Inq-Blotter dashboard to support teachers and students on science practices [Invited talk]. Rutgers Graduate School of Education Brown Bag Lecture Series.

Abstract:
Technologies such as dashboards can support teachers’ real-time monitoring and assessment of their students’ learning. While most dashboards rely on coarse multiple-choice assessment of learning, Inq-Blotter’s data is based on educational data-mined and knowledge engineered algorithms of students’ fine-grained competencies at science practices, which are presented in actionable alerts. In this presentation, we will describe the design of Inq-Blotter and demo how it works in tandem with the intelligent tutoring system, Inq-ITS. We will also discuss our recent work on teachers’ use of Inq-Blotter to support science inquiry learning while their students engaged in real time inquiry. Specifically, we will describe results about how teacher support elicited by Inq-Blotter fostered student improvement on inquiry practices in Inq-ITS. We will also describe how epistemic network analysis was used to examine the patterns in the discursive supports provided to students by teachers. Lastly, we will describe our newest development, namely, Teacher Inquiry Practice Supports (TIPS) to further guide teachers in providing real time support to students.

Recommended Citation (APA):
Dickler, R., O’Brien, M., Gobert, J., Olsen, J., Adair, A., & Hussain-Abidi, H. (2021, April). Analyzing student-teacher discourse prompted by a real-time alerting dashboard for science inquiry practices [Roundtable paper presentation]. American Educational Research Association (AERA) Annual Meeting.

Abstract:
The present study used an analysis of teacher-student discourse to examine how conversations prompted by fine-grained alerts on science inquiry practices in the state-of-the-art teacher dashboard, Inq-Blotter, related to changes in student inquiry performance within virtual labs in the intelligent tutoring system, Inq-ITS.

Recommended Citation (APA):
Adair, A., Dickler, R., Gobert, J. & Lee, J. (2021, April). Inq-ITS supports students maintaining their science inquiry competencies during remote learning due to COVID-19 [Poster presentation]. American Educational Research Association (AERA) Annual Meeting.

Abstract:
Although prior research on remote instruction exists (Brinson, 2015; Heradio et al., 2016), the COVID-19 crisis provides a unique context to investigate how an unexpected switch to remote instruction affects learning in difficult domains, such as science inquiry. The present study explores students’ science inquiry practice performances in Inq-ITS virtual labs prior to and during remote learning due to the COVID-19 pandemic. Specifically, we examined how a sophisticated educational platform that assesses students’ inquiry at a fine-grained level and scaffolds them in real time may support students even under extreme circumstances.

Recommended Citation (APA):
Adair, A., Dickler, R., & Gobert, J. (2021, March). Developing and implementing teacher inquiry practice supports for remote and in-person instruction [Poster presentation]. Rutgers Graduate School of Education Annual Poster Session.

Recommended Citation (APA):
Adair, A., Dickler, R., & Gobert, J. (2020). Supporting teachers supporting students: Iterative development of TIPS in a teacher dashboard. In M. Gresalfi & I. S. Horn (Eds.), 14th International Conference of the Learning Sciences, Volume 3 (pp. 1769-1770). International Society of the Learning Sciences.

Abstract:
When implementing the Next Generation Science Standards, it is challenging for teachers to support students on inquiry practices; technological tools are a good solution to help inform teachers’ pedagogical practices. In this study, we developed actionable, evidence-based Teacher Inquiry Practice Supports (TIPS) that are presented as fine-grained real-time alerts within the teacher dashboard Inq-Blotter. These TIPS aid teachers in providing detailed support to students in order to scaffold students’ specific inquiry difficulties on the practices.