CategoriesJournal article

From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case Study

How can we provide students with highly informative feedback on their essays using natural language processing?

Check out our new paper, led by Sebastian Gombert, where we present a case study on using GBERT and T5 models to generate feedback for educational psychology students.

In this paper:

➡ We implemented a two-step pipeline that segments the essays and predicts codes from the segments. The codes are used to generate feedback texts that inform the students about the correctness of their solutions and the content areas they need to improve.

➡ We used 689 manually labelled essays as training data for our models. We compared GBERT, T5, and bag-of-words baselines for scoring the segments and the codes. The results showed that the transformer-based models outperformed the baselines in both steps.

➡ We evaluated the feedback using a randomised controlled trial. The control group received essential feedback, while the treatment group received highly informative feedback based on our pipeline. We used a six-item survey to measure the perception of feedback.

➡ We found that highly informative feedback had positive effects on helpfulness and reflection. The students in the treatment group reported higher levels of satisfaction, usefulness, and learning than the students in the control group.

➡ Our paper demonstrates the potential of natural language processing for providing highly informative feedback on student essays. We hope that our work will inspire more research and practice in this area.

You can read the full paper here.

Published by Daniele Di Mitri

Daniele Di Mitri is a research group leader at the DIPF - Leibniz Institute for Research and Information in Education and a lecturer at the Goethe University of Frankfurt, Germany. Daniele received his PhD entitled "The Multimodal Tutor" at the Open University of The Netherlands (2020) in Learning Analytics and wearable sensor support. His research focuses on collecting and analysing multimodal data during physical interactions for automatic feedback and human behaviour analysis. Daniele's current research focuses on designing responsible Artificial Intelligence applications for education and human support. He is a "Johanna Quandt Young Academy" fellow and was elected "AI Newcomer 2021" at the KI Camp by the German Informatics Society. He is a member of the editorial board of Frontiers in Artificial Intelligence journal, a member of the CrossMMLA, a special interest group of the Society of Learning Analytics Research, and chair of the Learning Analytics Hackathon (LAKathon) series.

Leave a Reply

Your email address will not be published. Required fields are marked *