Evaluating Class Imbalance Handling in Student Dropout Prediction Using Enrollment Information and First Semester Performance
Keywords:
student dropout, educational data mining, class imbalance, SMOTENC, class weightingAbstract
Class imbalance treatments can improve educational classification while changing the decisions and probabilities produced by a model. This retrospective study evaluates enrollment information and first semester performance in the original UCI dataset of 4,424 students, preserving Dropout, Enrolled, and Graduate as separate outcomes. Twenty configurations combine two predictor sets, five training treatments, and logistic regression or random forest. Development-only nested cross-validation selects the model, and 885 untouched records support paired bootstrap intervals and ten planned contrasts with Holm correction. Random undersampling with first semester logistic regression was selected and achieved macro-F1 64.82% (95% interval 61.71%–67.97%), accuracy 68.70%, and dropout recall 63.03%. Adding first semester features increased natural-frequency macro-F1 by 15.35 percentage points for logistic regression and 13.98 points for random forest. Class weighting improved macro-F1 for both classifiers after multiplicity adjustment, and random oversampling improved random forest. Gains primarily reflected better recognition of Enrolled students, accompanied by lower dropout recall. All balancing treatments increased multiclass Brier error within each classifier, although dropout-specific error did not uniformly increase. A resolved-outcome binary analysis showed small classification gains and inflated mean dropout probabilities under balancing. The results support evaluating treatments against class-specific objectives and probability reliability, rather than assuming that balancing necessarily improves dropout identification. Missing event timestamps prevent interpreting this benchmark as prospective prediction after semester one.
Downloads
References
Olaf Zawacki-Richter, Victoria I. Marín, Melissa Bond, Franziska Gouverneur. Systematic review of research on artificial intelligence applications in higher education – where are the educators?. International Journal of Educational Technology in Higher Education. 2019;16(1):39. https://doi.org/10.1186/s41239-019-0171-0
Sarah Alturki, Ioana Hulpuș, Heiner Stuckenschmidt. Predicting Academic Outcomes: A Survey from 2007 Till 2018. Technology, Knowledge and Learning. 2022;27(1):275–307. https://doi.org/10.1007/s10758-020-09476-0
Lidya R. Pelima, Yuda Sukmana, Yusep Rosmansyah. Predicting University Student Graduation Using Academic Performance and Machine Learning: A Systematic Literature Review. IEEE Access. 2024;12:23451–23465. https://doi.org/10.1109/ACCESS.2024.3361479
Daniel Andrade-Girón, Juana Sandivar-Rosas, William Marín-Rodriguez, Edgar Susanibar-Ramirez, Eliseo Toro-Dextre, Jose Ausejo-Sanchez, et al.. Predicting Student Dropout based on Machine Learning and Deep Learning: A Systematic Review. ICST Transactions on Scalable Information Systems. 2023. https://doi.org/10.4108/eetsis.3586
Alberto Fernandez, Salvador Garcia, Francisco Herrera, Nitesh V. Chawla. SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary. Journal of Artificial Intelligence Research. 2018;61:863–905. https://doi.org/10.1613/jair.1.11192
Miriam Seoane Santos, Pedro Henriques Abreu, Nathalie Japkowicz, Alberto Fernández, Carlos Soares, Szymon Wilk, et al.. On the joint-effect of class imbalance and overlap: a critical review. Artificial Intelligence Review. 2022;55(8):6207–6275. https://doi.org/10.1007/s10462-022-10150-3
Valentim Realinho, Jorge Machado, Luís Baptista, Mónica V. Martins. Predicting Student Dropout and Academic Success. Data. 2022;7(11):146. https://doi.org/10.3390/data7110146
Valentim Realinho, Mónica Vieira Martins, Jorge Machado, Luís Baptista. Predict Students' Dropout and Academic Success. [Dataset]. UCI Machine Learning Repository. 2021. https://doi.org/10.24432/C5MC89
Mónica V. Martins, Daniel Tolledo, Jorge Machado, Luís M. T. Baptista, Valentim Realinho. Early Prediction of student’s Performance in Higher Education: A Case Study. In Trends and Applications in Information Systems and Technologies. Springer. 2021:166–175. https://doi.org/10.1007/978-3-030-72657-7_16
Khalid Oqaidi, Sarah Aouhassi, Khalifa Mansouri. Towards a Students’ Dropout Prediction Model in Higher Education Institutions Using Machine Learning Algorithms. International Journal of Emerging Technologies in Learning (iJET). 2022;17(18):103–117. https://doi.org/10.3991/ijet.v17i18.25567
Matti Vaarma, Hongxiu Li. Predicting student dropouts with machine learning: An empirical study in Finnish higher education. Technology in Society. 2024;76:102474. https://doi.org/10.1016/j.techsoc.2024.102474
Jovial Niyogisubizo, Lyuchao Liao, Eric Nziyumva, Evariste Murwanashyaka, Pierre Claver Nshimyumukiza. Predicting student's dropout in university classes using two-layer ensemble machine learning approach: A novel stacked generalization. Computers and Education: Artificial Intelligence. 2022;3:100066. https://doi.org/10.1016/j.caeai.2022.100066
Stephan Romero, Xiyue Liao. Statistical and machine learning models for predicting university dropout and scholarship impact. PLOS One. 2025;20(6):e0325047. https://doi.org/10.1371/journal.pone.0325047
Eissa Alzabidi, Önder Yakut, Sabah Saad, Oğuz Fındık. SOD-FE: a supervised outlier detection and feature engineering approach for student dropout prediction. Scientific Reports. 2026;16(1):25906. https://doi.org/10.1038/s41598-026-56591-6
Ahmad S. Tarawneh, Ahmad B. Hassanat, Ghada Awad Altarawneh, Abdullah Almuhaimeed. Stop Oversampling for Class Imbalance Learning: A Review. IEEE Access. 2022;10:47643–47660. https://doi.org/10.1109/ACCESS.2022.3169512
Ruben van den Goorbergh, Maarten van Smeden, Dirk Timmerman, Ben Van Calster. The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regression. Journal of the American Medical Informatics Association. 2022;29(9):1525–1534. https://doi.org/10.1093/jamia/ocac093
Alex Carriero, Kim Luijken, Anne de Hond, Karel G. M. Moons, Ben van Calster, Maarten van Smeden. The Harms of Class Imbalance Corrections for Machine Learning Based Prediction Models: A Simulation Study. Statistics in Medicine. 2025;44(3-4):e10320. https://doi.org/10.1002/sim.10320
Ben Van Calster, David J. McLernon, Maarten van Smeden, Laure Wynants, Ewout W. Steyerberg. Calibration: the Achilles heel of predictive analytics. BMC Medicine. 2019;17(1):230. https://doi.org/10.1186/s12916-019-1466-7
Suhan Wu, Jingyi Duan, Min Luo. Calibrated early-warning models with fairness auditing and selective prediction for course withdrawal risk: Evidence from OULAD. PLOS One. 2026;21(7):e0352867. https://doi.org/10.1371/journal.pone.0352867
Sayash Kapoor, Arvind Narayanan. Leakage and the reproducibility crisis in machine-learning-based science. Patterns. 2023;4(9):100804. https://doi.org/10.1016/j.patter.2023.100804
Andrius Vabalas, Emma Gowen, Ellen Poliakoff, Alexander J. Casson. Machine learning algorithm validation with a limited sample size. PLOS ONE. 2019;14(11):e0224365. https://doi.org/10.1371/journal.pone.0224365
Stephen Bates, Trevor Hastie, Robert Tibshirani. Cross-Validation: What Does It Estimate and How Well Does It Do It?. Journal of the American Statistical Association. 2024;119(546):1434–1445. https://doi.org/10.1080/01621459.2023.2197686
Bevan I. Smith, Charles Chimedza, Jacoba H. Bührmann. Individualized help for at-risk students using model-agnostic and counterfactual explanations. Education and Information Technologies. 2022;27(2):1539–1558. https://doi.org/10.1007/s10639-021-10661-6
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence. 2019;1(5):206–215. https://doi.org/10.1038/s42256-019-0048-x
Ryan S. Baker, Aaron Hawn. Algorithmic Bias in Education. International Journal of Artificial Intelligence in Education. 2022;32(4):1052–1092. https://doi.org/10.1007/s40593-021-00285-9
Efthyvoulos Drousiotis, Panagiotis Pentaliotis, Lei Shi, Alexandra I. Cristea. Capturing Fairness and Uncertainty in Student Dropout Prediction – A Comparison Study. In Artificial Intelligence in Education. Springer. 2021:139–144. https://doi.org/10.1007/978-3-030-78270-2_25
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, et al.. Datasheets for datasets. Communications of the ACM. 2021;64(12):86–92. https://doi.org/10.1145/3458723
Davide Chicco, Giuseppe Jurman. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics. 2020;21(1):6. https://doi.org/10.1186/s12864-019-6413-7
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Riki Purnama, Hendra Apriawan

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.









