Peer-Reviewed Conference Proceedings
Using AI-based assessment and scaffolds to identify student difficulties with plotting data and modeling in virtual science labs
Ellie Segan, Janice Gobert, Michael Sao Pedro, Amy Adair, Jessica A. Owens
International Conference of the Learning Sciences
Recommended Citation (APA):
Segan, E., Gobert, J., Sao Pedro, M., Adair, A., & Owens, J. A. (2024). Using AI-based assessment and scaffolds to identify student difficulties with plotting data and modeling in virtual science labs. In R. Lindgren, T. I. Asino, E. A. Kyza, C. K. Looi, D. T. Keifert, & E. Su ́arez (Eds.), Proceedings of the 18th International Conference of the Learning Sciences - ICLS 2024 (pp. 1466-1469). International Society of the Learning Sciences
Abstract:
Developing proficiency in science practices, including using mathematics, outlined in the Next Generation Science Standards is essential for success in STEM courses and future careers. However, students often struggle with developing mathematical models, which limits their ability to understand scientific phenomena. To improve students’ learning and teachers’ assessment, we extended Inq-ITS to automatically assess and scaffold students’ competencies in developing mathematical models of scientific phenomena. We analyzed student data from six virtual science labs in Inq-ITS at both the practice level and the sub-practice level to determine if they maintained their mathematical competencies with scaffolding. By operationalizing and analyzing data at the sub-practice level, the results provide valuable formative data regarding the challenges students face when developing mathematical models about scientific phenomena, which in turn, can inform future scaffolds across science domains.
2024
Peer-Reviewed Conference Proceedings
Real-time AI-driven assessment and scaffolding that improves students’ mathematical modeling during science investigations
Amy Adair, Michael Sao Pedro, Janice Gobert, Ellie Segan
International Conference on Artificial Intelligence in Education
Recommended Citation (APA):
Adair, A., Sao Pedro, M., Gobert, J., & Segan, E. (2023). Real-time AI-driven assessment and scaffolding that improves students’ mathematical modeling during science investigations. In N. Wang, G. Rebolledo-Mendez, N. Matsuda, O. C. Santos, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 202-216). Springer, Cham.
Abstract:
Developing models and using mathematics are two key practices in internationally recognized science education standards, such as the Next Generation Science Standards (NGSS). However, students often struggle at the intersection of these practices, i.e., developing mathematical models about scientific phenomena. In this paper, we present the design and initial classroom test of AI-scaffolded virtual labs that help students practice these competencies. The labs automatically assess fine-grained sub-components of students’ mathematical modeling competencies based on the actions they take to build their mathematical models within the labs. We describe how we leveraged underlying machine-learned and knowledge-engineered algorithms to trigger scaffolds, delivered proactively by a pedagogical agent, that address students’ individual difficulties as they work. Results show that students who received automated scaffolds for a given practice on their first virtual lab improved on that practice for the next virtual lab on the same science topic in a different scenario (a near-transfer task). These findings suggest that real-time automated scaffolds based on fine-grained assessment data can help students improve on mathematical modeling.
2023
Peer-Reviewed Conference Presentations
Automated analyses of students’ difficulties with explanations in science inquiry
Jessica A. Owens, Amy Adair, Ellie Segan, Janice Gobert
Annual Meeting of the Society for Text and Discourse
Recommended Citation (APA):
Owens, J. A., Adair, A., Segan, E., & Gobert, J. (2023, June). Automated analyses of students’ difficulties with explanations in science inquiry [Conference presentation]. Annual Meeting of the Society for Text and Discourse, Olso, Norway.
Abstract:
Students have difficulties constructing explanations and arguments in scientific inquiry. The current study made use of McNeill et al.’s (2006) claim, evidence, and reasoning framework to elicit students’ competencies with constructing scientific explanations and engaging in argumentation and natural language processing algorithms to score students’ Claim-Evidence-Reasoning statements at a fine-grained level. The analysis showed that although students would benefit from greater support in all three components, they struggled the most with particular aspects of Reasoning.
2023
Peer-Reviewed Conference Proceedings
Assessing students’ competencies with mathematical models in virtual science inquiry investigations
Amy Adair, Michael Sao Pedro, Janice Gobert, Jessica A. Owens
International Conference of the Learning Sciences
Recommended Citation (APA):
Adair, A., Sao Pedro, M., Gobert, J., & Owens, J. A. (2023). Assessing students’ competencies with mathematical models in virtual science inquiry investigations. In P. Blikstein, J. Van Aalst, R. Kizito, & K. Brennan (Eds.), 17th International Conference of the Learning Sciences (pp. 914-917). International Society of the Learning Sciences.
Abstract:
Developing models and using mathematics are two key practices in internationally recognized science education standards such as the Next Generation Science Standards (NGSS, 2013). In this paper, we used a virtual performance-based formative assessment to capture students’ competencies at both developing and evaluating mathematical models in science inquiry contexts aligned with the NGSS (2013). Results show that model development and evaluation competencies are correlated, but students who demonstrate proficiency with model development often struggle with evaluation. Nuanced data illustrate how components of modeling competencies differ and how they may be related.
2023
Peer-Reviewed Conference Presentations
Examining students’ mathematical evidence in CER explanations during science inquiry contexts
Amy Adair, Rachel F. Dickler, Janice Gobert, Michael Sao Pedro, Joe Olsen, Jessica A. Owens, Christine Lott
American Educational Research Association (AERA) Annual Meeting
Recommended Citation (APA):
Adair, A., Dickler, R. F., Gobert, J., Sao Pedro, M., Olsen, J., Owens, J. A., & Lott, C. (2023, April). Examining students' mathematical evidence in CER explanations during science inquiry contexts [Symposium poster]. American Educational Research Association (AERA) Annual Meeting.
Abstract:
In the present study, we qualitatively examined students’ explanations about mathematical relationships between physical science variables written in a Claim, Evidence, Reasoning (CER) format. Results showed that the quality of students’ responses varied, illustrating the need for fine-grained, evidence-based rubrics that can identify precisely how students are struggling at CER so that teachers can best support students on their scientific explanation writing.
2023
Peer-Reviewed Conference Presentations
Evaluation of automated scoring methods for students’ claim, evidence, reasoning responses in science
Haiying Li, Amy Adair, Grace Li, Rachel F. Dickler, Janice Gobert
American Educational Research Association (AERA) Annual Meeting
Recommended Citation (APA):
Li, H., Adair, A., Li, G., Dickler, R. F., & Gobert, J. (2023, April). Evaluation of automated scoring methods for students’ claim, evidence, reasoning responses in science [Symposium poster]. American Educational Research Association (AERA) Annual Meeting.
Abstract:
Given the importance of developing students’ competencies on NGSS practices, the present study evaluated two different automated scoring methods for written claims, evidence, and reasoning (CER): the RegEx method and the WordDistance method. Results showed that both methods performed moderately well in scoring students’ CER responses. These findings have implications for implementing automated assessments and facilitating personalized scaffolding to support students’ CER writing.
2023
Teacher Conference Presentations
Using mathematics to deepen understanding of scientific phenomena
Michael Sao Pedro, Janice Gobert, Amy Adair, Rachel Dickler, Cameron Betts
Kentucky Science Teachers Association Annual Conference
Recommended Citation (APA):
Sao Pedro, M., Gobert, J., Adair, A., Dickler, R., & Betts, C. (2022, November). Using mathematics to deepen understanding of scientific phenomena [Conference presentation]. Kentucky Science Teachers Association Annual Conference, Richmond, KY, United States.
Abstract:
The Next Generation Science Standards (NGSS) emphasize the importance of using mathematical representations to model, make predictions about, and deepen understanding of science phenomena. However, students often struggle at integrating mathematics and science in a myriad of ways. Join us to learn how graphical reasoning can help students bridge the STEM gap and achieve proficiency as envisioned by the NGSS. In this interactive session, you will learn how self-grading virtual labs paired with an Artificial Intelligence-based alerting dashboard can help your students become proficient at understanding graphs, making predictions, and using mathematics to deepen content understanding. Key takeaways include: (1) deepened understanding of the design tradeoffs necessary to create rich, formative assessment activities capable of yielding fine-grained data about students’ difficulties; (2) techniques to help students who lack algebraic fluency bridge the gap between science and mathematics; and (3) research-backed ways to provide real-time feedback to students that have been shown to improve their learning. All attendees will leave with demo access to our materials that they can use immediately with students for authentic assessment and instruction of NGSS practices, specifically using mathematics (NGSS Strand 5).
2022
Peer-Reviewed Conference Proceedings
Using log data to validate performance assessments of mathematical modeling practices
Joe Olsen, Amy Adair, Janice Gobert, Michael Sao Pedro, Mariel O’Brien
International Conference on Artificial Intelligence in Education
Recommended Citation (APA):
Olsen, J., Adair, A., Gobert, J., Sao Pedro, M., & O’Brien, M. (2022). Using log data to validate performance assessments of mathematical modeling practices. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 488-491). Springer, Cham.
Abstract:
Many national science frameworks (e.g., Next Generation Science Standards) argue that developing mathematical modeling competencies is critical for students’ deep understanding of science. However, science teachers may be unprepared to assess these competencies. We are addressing this need by developing virtual lab performance assessments that assess these competencies in science inquiry contexts. Through our design processes, we developed a method for validating the assessments that takes advantage of the unique opportunities afforded by collecting log data. Here, we describe this method and demonstrate its utility by analyzing students’ competencies with one example sub-practice of mathematical modeling, plotting controlled data generated from a simulation.
2022
Peer-Reviewed Conference Proceedings
An integrated approach to learning solutions: UCD + LS&D + AIEd
Kristen S. Herrick, Larisa Nachman, Kinta D. Montilus, K. Rebecca Marsh Runyon, Amy Adair, Lisa Ferrara
International Conference on Artificial Intelligence in Education
Recommended Citation (APA):
Herrick, K. S., Nachman, L., Montilus, K. D., Runyon, K. R. M., Adair, A., & Ferrara, L. (2022). An integrated approach to learning solutions: UCD + LS&D + AIEd. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 94-98). Springer, Cham.
Abstract:
User-centered design (UCD), learning science and instructional design (LS&D), and AI in education (AIEd) can be powerful, yet siloed practices when developing educational products. This paper describes our Agile ways of working in the ETS® AI Labs™ and how we are taking an integrated approach to educational product development. We discuss lessons learned and ways we can bridge the gap between learning theories and practices, artificial intelligence, and user experience/research to craft effective learning solutions.
2022
Peer-Reviewed Conference Proceedings
Using epistemic network analysis to explore discourse patterns across design iterations of a teacher dashboard
Amy Adair, Jessica Owens, Janice Gobert
International Conference of the Learning Sciences
Recommended Citation (APA):
Adair, A., Owens, J., & Gobert, J. (2022). Using epistemic network analysis to explore discourse patterns across design iterations of a teacher dashboard. In C. Chinn, E. Tan, C. Chan, & Y. Kali (Eds.), 16th International Conference of the Learning Sciences (pp. 297-304). International Society of the Learning Sciences.
Abstract:
Providing high-level support to students on NGSS inquiry practices can be challenging; however, teacher dashboards can help teachers provide just-in-time instruction to students, both in-person and online. Prior work has shown some success with a dashboard that alerts teachers in real time on students’ science inquiry difficulties, but teachers differed in their use of the alerts. To further support teachers, we designed a second iteration, in which the alerts included actionable, evidence-based Teacher Inquiry Practice Supports (TIPS), a series of suggested scaffolds that teachers can use to support students on the practices with which they are struggling. In this study, we investigate how the discursive support patterns from one teacher differed when using the dashboard alerts without TIPS followed by with TIPS. Findings suggest that TIPS influenced how the teacher incorporated different types of support for her students, and further, that the support given varied across different virtual lab stages.
2022
Teacher Conference Presentations
Using mathematics to deepen understanding of scientific phenomena
Michael Sao Pedro, Janice Gobert, Amy Adair, Joe Olsen, Rachel Dickler, Cameron Betts
National Science Teaching Association’s National Conference on Science Education
Recommended Citation (APA):
Sao Pedro, M., Gobert, J., Adair, A., Olsen, J., Dickler, R., & Betts, C. (2022, March). Using mathematics to deepen understanding of scientific phenomena [Conference presentation]. National Science Teaching Association’s National Conference on Science Education, Houston, TX, United States.
2022
Teacher Conference Presentations
Strengthening students’ inquiry competencies via progress monitoring
Michael Sao Pedro, Janice Gobert, Amy Adair
Science Teachers Association of New York State (STANYS) Conference
Recommended Citation (APA):
Sao Pedro, M., Gobert, J., & Adair, A. (2021, November). Strengthening students’ inquiry competencies via progress monitoring [Conference presentation]. Science Teachers Association of New York State (STANYS) Conference, Rochester, NY, United States.
2021
Peer-Reviewed Conference Presentations
Can text features of investigative questions in science predict students’ inquiry competencies?
Jeremy Lee, Amy Adair, Janice Gobert, Rachel Dickler
Annual Meeting of the Society for Text and Discourse
Recommended Citation (APA):
Lee, J., Adair, A., Gobert, J., & Dickler, R. (2021, August). Can text features of investigative questions in science predict students’ inquiry competencies? [Conference presentation]. Annual Meeting of the Society for Text and Discourse.
Abstract:
To fully realize the vision of the Next Generation Science Standards, students must be able to make sense of the scientific text that guides their inquiry and understanding of science phenomena. In this study, we analyze the text features of several investigative questions in the Inquiry Intelligent Tutoring System (Inq-ITS) to understand the ways in which the difficulty of the investigative questions relates to students’ competencies with science inquiry practices. Results show that certain word-level features (e.g., familiarity, concreteness) of the investigative questions can explain students’ performances on science inquiry, but that these features differ across science topics.
2021
Peer-Reviewed Conference Presentations
Using a teacher dashboard to support students remotely on science inquiry
Rachel Dickler, Janice Gobert, Amy Adair, Joe Olsen
Annual Meeting of the Society for Text and Discourse
Recommended Citation (APA):
Dickler, R., Gobert, J., Adair, A., & Olsen, J. (2021, August). Using a teacher dashboard to support students remotely on science inquiry [Conference presentation]. Annual Meeting of the Society for Text and Discourse.
Abstract:
In this study, we examined teachers’ remote use of a science dashboard, Inq-Blotter, through discourse analyses and students’ corresponding inquiry performance in Inq-ITS. Specifically, Epistemic Network Analyses were applied to compare patterns of support elicited by Inq-Blotter when students improved versus did not improve on inquiry in Inq-ITS. Analyses revealed significant differences in teacher support patterns in relation to student improvement. These results demonstrate how dashboards can support science discourse and learning during remote instruction.
2021
Peer-Reviewed Conference Proceedings
Examining the use of a teacher alerting dashboard during remote learning
Rachel Dickler, Amy Adair, Janice Gobert, Huma Hussain-Abidi, Joe Olsen, Mariel O’Brien, Michael Sao Pedro
International Conference on Artificial Intelligence in Education
Recommended Citation (APA):
Dickler, R., Adair, A., Gobert, J., Hussain-Abidi, H., Olsen, J., O’Brien, M., & Sao Pedro, M. (2021). Examining the use of a teacher alerting dashboard during remote learning. In I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, & V. Dimitrova (Eds.), International Conference on Artificial Intelligence in Education (pp. 134-138). Springer, Cham.
Abstract:
Remote learning in response to the COVID-19 pandemic has introduced many challenges for educators. It is important to consider how AI technologies can be leveraged to support educators and, in turn, help students learn in remote settings. In this paper, we present the results of a mixed-methods study that examined how teachers used a dashboard with real-time alerts during remote learning. Specifically, three high school teachers held remote synchronous classes and received alerts in the dashboard about students’ difficulties on scientific inquiry practices while students conducted virtual lab investigations in an intelligent tutoring system. Quantitative analyses revealed that students significantly improved across a majority of inquiry practices during remote use of the technologies. Additionally, through qualitative analyses of the transcribed audio data, we identified five trends related to dashboard use in a remote setting, including three reflecting effective implementations of dashboard features and two reflecting the limitations of dashboard use. Implications regarding the design of dashboards for use across varying contexts are discussed.
2021
Peer-Reviewed Conference Proceedings
Supporting students remotely: Integrating mathematics and sciences in virtual labs
Rachel Dickler, Michael Sao Pedro, Amy Adair, Janice Gobert, Joe Olse, Jason Kleban, Cameron Betts, Charity Staudenraus, Patrick Roughan
International Conference of the Learning Sciences
Recommended Citation (APA):
Dickler, R., Sao Pedro, M., Adair, A., Gobert, J., Olsen, J., Kleban, J., Betts, C., Staudenraus, C., & Roughan, P. (2021). Supporting students remotely: Integrating mathematics and sciences in virtual labs. In E. de Vries, Y. Hod, & J. Ahn (Eds.), 15th International Conference of the Learning Sciences (pp.1013-1014). International Society of the Learning Sciences.
Abstract:
Tools that automatically assess and support students are important during remote instruction due to COVID-19 because students do not have direct access to teacher support. We present results of the remote use of virtual labs in an Inquiry Intelligent Tutoring System (InqITS), which captures student performance on practices at the intersection of mathematics and science. Implications are discussed for the development of scaffolding and design of labs to support remote instruction.
2021
Peer-Reviewed Conference Presentations
Analyzing student-teacher discourse prompted by a real-time alerting dashboard for science inquiry practices
Rachel Dickler, Mariel O’Brien, Janice Gobert, Joe Olsen, Amy Adair, Huma Hussain-Abidi
American Educational Research Association (AERA) Annual Meeting
Recommended Citation (APA):
Dickler, R., O’Brien, M., Gobert, J., Olsen, J., Adair, A., & Hussain-Abidi, H. (2021, April). Analyzing student-teacher discourse prompted by a real-time alerting dashboard for science inquiry practices [Roundtable paper presentation]. American Educational Research Association (AERA) Annual Meeting.
Abstract:
The present study used an analysis of teacher-student discourse to examine how conversations prompted by fine-grained alerts on science inquiry practices in the state-of-the-art teacher dashboard, Inq-Blotter, related to changes in student inquiry performance within virtual labs in the intelligent tutoring system, Inq-ITS.
2021
Peer-Reviewed Conference Presentations
Inq-ITS supports students maintaining their science inquiry competencies during remote learning due to COVID-19
Amy Adair, Rachel Dickler, Janice Gober, Jeremy Lee
American Educational Research Association (AERA) Annual Meeting
Recommended Citation (APA):
Adair, A., Dickler, R., Gobert, J. & Lee, J. (2021, April). Inq-ITS supports students maintaining their science inquiry competencies during remote learning due to COVID-19 [Poster presentation]. American Educational Research Association (AERA) Annual Meeting.
Abstract:
Although prior research on remote instruction exists (Brinson, 2015; Heradio et al., 2016), the COVID-19 crisis provides a unique context to investigate how an unexpected switch to remote instruction affects learning in difficult domains, such as science inquiry. The present study explores students’ science inquiry practice performances in Inq-ITS virtual labs prior to and during remote learning due to the COVID-19 pandemic. Specifically, we examined how a sophisticated educational platform that assesses students’ inquiry at a fine-grained level and scaffolds them in real time may support students even under extreme circumstances.
2021
Peer-Reviewed Conference Proceedings
Supporting teachers supporting students: Iterative development of TIPS in a teacher dashboard
Amy Adair, Rachel Dickler, Janice Gobert
International Conference of the Learning Sciences
Recommended Citation (APA):
Adair, A., Dickler, R., & Gobert, J. (2020). Supporting teachers supporting students: Iterative development of TIPS in a teacher dashboard. In M. Gresalfi & I. S. Horn (Eds.), 14th International Conference of the Learning Sciences, Volume 3 (pp. 1769-1770). International Society of the Learning Sciences.
Abstract:
When implementing the Next Generation Science Standards, it is challenging for teachers to support students on inquiry practices; technological tools are a good solution to help inform teachers’ pedagogical practices. In this study, we developed actionable, evidence-based Teacher Inquiry Practice Supports (TIPS) that are presented as fine-grained real-time alerts within the teacher dashboard Inq-Blotter. These TIPS aid teachers in providing detailed support to students in order to scaffold students’ specific inquiry difficulties on the practices.
2020