Category: Results

CategoriesConference article

New paper: A Human-centric Approach to Explain Evolving Data

Posted on 06/30/2024by Daniele Di Mitri

A recent study led by my colleague Gabriella Casalino at the University of Bari highlights the importance of transparency and explainability in Machine Learning models used in educational environments. 

As we embrace this technological shift driven by AI in education, it is imperative to address the ethical considerations surrounding AI applications in educational settings. A recent study has underscored the critical importance of transparency and explainability in machine learning models utilized in educational environments.

At the forefront of this study is the introduction of DISSFCM, a dynamic incremental classification algorithm that harnesses the power of fuzzy logic to analyze and interpret students' interactions within learning platforms; by offering human-centric explanations, the research endeavours to deepen stakeholders' understanding of how AI models arrive at decisions in educational contexts.

One of the key strengths of the DISSFCM algorithm lies in its adaptability. It dynamically adjusts its model in response to changes in data, ensuring resilience and reliability in educational data analytics. This adaptability enhances the algorithm's performance and instills confidence in the insights derived from educational data.

Transparency and ethical standards are paramount in AI practices, particularly in educational settings. We can build trust and ensure fairness in deploying educational technologies by upholding these principles. The study sheds light on the evolving landscape of AI integration in education and emphasizes the pivotal role of explainable AI in fostering trust and understanding among stakeholders.

As we navigate the intersection of AI and education, prioritizing transparency and explainability will be instrumental in shaping a future where technology enhances learning experiences while upholding ethical standards. By embracing these principles, we can pave the way for a more transparent and accountable educational ecosystem powered by AI.

Reference to the article: 

G. Casalino, G. Castellano, D. Di Mitri, K. Kaczmarek-Majer and G. Zaza, "A Human-centric Approach to Explain Evolving Data: A Case Study on Education," 2024 IEEE International Conference on Evolving and Adaptive Intelligent Systems (EAIS), Madrid, Spain, 2024, pp. 1-8, doi: 10.1109/EAIS58494.2024.10569098.

The paper also got an award at the EAIS conference. 


No alt text provided for this imageNo alt text provided for this image

 

CategoriesPresentations

Invited talk at the University of The Philippines

Posted on 06/24/2024by Daniele Di Mitri

 

On June 19th, I was invited to give an online talk at the University of the Philippines. The title of my talk was "Intelligent Tutors, Learning Analytics, and Multimodal Technologies," and it served as the kickoff guest lecture for the webinar series hosted by the Intelligent Systems Center of the University of the Philippines. At its peak, the lecture had over 170 participants connected online.

During the talk, I discussed how learners in the twenty-first century need continuous instruction and timely feedback to develop their competencies. In situations where human experts are not readily available, Artificial Intelligence (AI) systems can offer automatic, personalized, and real-time feedback to learners in distance learning settings. This allows learners to practice at their own pace while receiving continuous feedback. Moreover, AI feedback can extend beyond traditional cognitive tasks to provide input on physical learning tasks by integrating with immersive and multimodal technologies such as Augmented and Virtual Reality (AR/VR) or sensor-based systems.

I summarized the main insights of my research in AI in education and Multimodal Learning Analytics (MMLA), introducing the concept of "Multimodal Tutors". I demonstrated how MMLA can support distance teaching and learning with personalized feedback and adaptation. Through relevant use cases, I illustrated how AI and immersive technologies can be used to enhance feedback. Finally, I presented my research agenda for augmenting feedback with AI and how it can provide personalized and adaptive support to learners and teachers.

CategoriesJournal article

From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case Study

Posted on 01/26/2024by Daniele Di Mitri

How can we provide students with highly informative feedback on their essays using natural language processing?

Check out our new paper, led by Sebastian Gombert, where we present a case study on using GBERT and T5 models to generate feedback for educational psychology students.

In this paper:

➡ We implemented a two-step pipeline that segments the essays and predicts codes from the segments. The codes are used to generate feedback texts that inform the students about the correctness of their solutions and the content areas they need to improve.

➡ We used 689 manually labelled essays as training data for our models. We compared GBERT, T5, and bag-of-words baselines for scoring the segments and the codes. The results showed that the transformer-based models outperformed the baselines in both steps.

➡ We evaluated the feedback using a randomised controlled trial. The control group received essential feedback, while the treatment group received highly informative feedback based on our pipeline. We used a six-item survey to measure the perception of feedback.

➡ We found that highly informative feedback had positive effects on helpfulness and reflection. The students in the treatment group reported higher levels of satisfaction, usefulness, and learning than the students in the control group.

➡ Our paper demonstrates the potential of natural language processing for providing highly informative feedback on student essays. We hope that our work will inspire more research and practice in this area.

You can read the full paper here.

https://link.springer.com/article/10.1007/s40593-023-00387-6

CategoriesJournal article

How to improve Knowledge Tracing with hybrid machine learning techniques

Posted on 01/04/2024by Daniele Di Mitri

 

Knowledge Tracing is a well-known problem in AI for Education. It consists of monitoring how the student's knowledge changes during the learning process and accurately predicting their performance in future exercises. But how can we improve the current methods and overcome heir limitations?

In recent years, many advances have been made thanks to various machine learning and deep learning techniques. However, they have some pitfalls, such as modelling one skill at a time, ignoring the relationships between different skills, or inconsistent predictions, i.e. sudden spikes and falls across time steps.

In our recently published systematic literature review, we aim to illustrate the state of the art in this field. Specifically, we want to identify the potential and the frontiers in integrating prior knowledge sources in the traditional machine learning pipeline to supplement the normally considered data. We propose a taxonomy with three dimensions: knowledge source, knowledge representation, and knowledge integration. We also conduct a quantitative analysis to detect the most common approaches and their advantages and disadvantages.

Our work provides a comprehensive overview of the hybrid machine-learning techniques for Knowledge Tracing and highlights the benefits of incorporating prior knowledge sources in the learning process. We believe this can lead to more accurate and robust predictions of student performance and help design more effective and personalized learning interventions. However, we also acknowledge that many challenges and open questions still need to be addressed, such as how to select the most relevant and reliable knowledge sources, how to represent and integrate them in a meaningful way, and how to evaluate their impact on the learning outcomes.

We hope that our work can inspire more research and innovation in the field of Knowledge Tracing and AI for Education.

Zanellati, A., Di Mitri, D., Gabbrielli, M., & Levrini, O. (2023). Hybrid Models for Knowledge Tracing: A Systematic Literature Review. IEEE Transactions on Learning Technologies, 1–16. doi: 10.1109/TLT.2023.3348690

https://ieeexplore.ieee.org/document/10379123