A Pilot Study on Machine Learning Models for Predicting Student Pass/Fail Outcomes Using LMS Activity Logs and Interactive Dashboard Analytics

Pratiwi Rachmadi

doi:10.56705/ijodas.v7i2.415

Authors

Pratiwi Rachmadi Perbanas Institute Jakarta

DOI:

https://doi.org/10.56705/ijodas.v7i2.415

Keywords:

Machine Learning, Pass/Fail Prediction, Learning Management System, Educational Data Mining, Learning Analytics Dashboard, KDD Framework

Abstract

Introduction: Learning Management Systems (LMS) generate extensive behavioral data that can support early identification of students at risk of academic failure, yet such data are often underutilized for predictive and intervention purposes. Method: This exploratory pilot study developed a machine learning pipeline following the Knowledge Discovery in Databases (KDD) framework using LMS activity logs from 36 students enrolled in a Discrete Mathematics course. Student-level behavioral features were extracted and modeled using Logistic Regression, Random Forest, and XGBoost. Model performance was evaluated using repeated 5-fold cross-validation, while an interactive learning analytics dashboard was designed to translate predictions into risk alerts and actionable recommendations for lecturers. Results and Discussion: Random Forest achieved the best overall performance with 87.0% accuracy, 0.88 precision, 0.94 recall, a macro-averaged F1-score of 0.92, and an AUC of 0.89, outperforming XGBoost and Logistic Regression. Assignment regularity and average quiz scores were identified as the most influential predictors. Qualitative feedback from two lecturers indicated that integrating risk information within a familiar LMS interface could support more practical and timelier student intervention. However, the small single-course sample limits generalizability. Conclusion: The proposed approach demonstrates the feasibility of integrating machine learning prediction and learning analytics dashboards as an early-warning mechanism, while further validation using larger, multi-course, and multi-institutional datasets is required.

Downloads

Download data is not yet available.

References

[1] S. Boujmiraz et al., "Predicting student performance: A comprehensive review of machine learning and deep learning applications in educational data mining," Computers and Education: Artificial Intelligence, vol. 6, p. 100210, 2024.

[2] J. Wang, "Machine learning approach to student performance prediction of online learning," PLoS ONE, vol. 20, no. 1, e0299018, 2025.

[3] M. Khan, H. AlAqrabi, and R. Hill, "Utilizing Machine Learning Models to Predict Student Performance From LMS Activity Logs," IEEE Access, vol. 11, pp. 86953-86962, 2023.

[4] K. Hubbard, "Feature Engineering on LMS Data to Optimize Student Performance Prediction," arXiv preprint arXiv:2504.02916, 2025.

[5] A. Al-Ameri et al., "Student Academic Success Prediction Using Learning Management Multimedia Data With Convoluted Features and Ensemble Model," Journal of Data and Information Quality, vol. 17, no. 3, pp. 1-18, 2025.

[6] V. Ganeshan and S. Umarani, "Beyond The Gradebook: Machine Learning and LMS Data for True Student Performance," Journal of Machine and Computing, vol. 5, no. 1, pp. 539-546, 2025.

[7] A. Al-Gahmi et al., "On Time-based Exploration of LMS Data and Prediction of Student Performance," in ASEE Annual Conference Exposition Peer Review, pp. 12-25, 2022.

[8] R. Santos and M. Castro, "Enhanced predictive performance: A comparative analysis of ML and DL models using augmented LMS interaction data," Contemporary Educational Technology, vol. 17, no. 2, ep495, 2025.

[9] A. Turkmenbayev, "The application of machine learning in predicting student performance in engineering education," Frontiers in Education, vol. 10, p. 1562586, 2025.

[10] E. Ahmed et al., "Student Performance Prediction Using Machine Learning Algorithms," Applied Computational Intelligence and Soft Computing, vol. 2024, ID 4067721, 2024.

[11] F. E. Arévalo-Cordovilla et al., "Comparative Analysis of Machine Learning Models for Predicting Student Success in Online Programming Courses," Mathematics, vol. 12, no. 20, p. 3272, 2024.

[12] T. Althaqafi et al., "The role of class imbalance handling in machine learning models for student performance prediction," Information Retrieval Journal, vol. 28, no. 1, pp. 45-67, 2025.

[13] M. Riestra-González et al., "Predicting academic performance of university students from LMS data," Journal of Learning Analytics, vol. 8, no. 1, pp. 118-135, 2021.

[14] G. Ramaswami et al., "Deep learning methods for student performance prediction using LMS logs," Education and Information Technologies, vol. 28, no. 9, pp. 11205-11231, 2023.

[15] K. Alalawi et al., "Predictive Analytics Dashboards in Higher Education: A Systematic Review," IEEE Access, vol. 13, pp. 4452-4470, 2025.

[16] I. Perać, "Designing Prescriptive Learning Analytics Dashboards for Higher Education Instructors," Computers & Education, vol. 224, p. 105120, 2025.

[17] L. Harrison and S. Cross, "The impact of socio-demographic features on educational predictive models," Journal of Computing in Higher Education, vol. 37, no. 2, pp. 201-224, 2024.

[18] J. R. Romero et al., "Educational Data Mining: A review of status and trends," Computers and Education: Artificial Intelligence, vol. 5, p. 100150, 2023.

[19] T. M. Ferguson, "Learning analytics dashboards: The gap between descriptive indicators and prescriptive interventions," Journal of Learning Analytics, vol. 10, no. 3, pp. 14-29, 2023.

[20] N. Ammar and H. Shahrour, "Mitigating overfitting in small-sample educational data mining via tree-based regularization," Education and Information Technologies, vol. 29, no. 4, pp. 4811-4833, 2024.

[21] R. Baker, "Stupid tutoring systems, intelligent analytics: A data leakage warning," International Journal of Artificial Intelligence in Education, vol. 34, no. 2, pp. 310-325, 2024.

[22] H. Choi and Y. Kim, "Cross-validation pitfalls in student modeling with temporal features," IEEE Transactions on Learning Technologies, vol. 17, pp. 88-99, 2024.

[23] S. Joksimović et al., "Portability of predictive models in learning analytics: A multi-institutional study," Computers & Education, vol. 194, p. 104697, 2023.

[24] M. Gasevic et al., "Empirical evaluation of algorithmic transparency in learning analytics dashboards," IEEE Transactions on Learning Technologies, vol. 18, pp. 112-125, 2025.

[25] A. Pardo et al., "Using learning analytics to scale timely personalized interventions," Journal of Computer Assisted Learning, vol. 40, no. 1, pp. 152-168, 2024.

[26] L. P. Macfadyen and S. Dawson, "Numbers Are Not Enough: Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan," Educational Technology & Society, vol. 26, no. 3, pp. 114-126, 2023.

[27] Y. Zhao et al., "Evaluation of a Moodle-integrated early warning system based on aggregated behavioral traces," Interactive Learning Environments, vol. 32, no. 5, pp. 2045-2063, 2024.