The Performance of the XGBOOST-LSTM and CNN-LSTM Algorithms in the Analysis of Stock Price Prediction Models for the Indonesian Banking Sector

M. Zainal Arifin; Filbert Chaitra Bessel Kristianto; Fadia Irsania Putri; Agung Bella Putra Utama

doi:10.56705/ijodas.v7i2.459

Authors

M. Zainal Arifin Universitas Negeri Malang
Filbert Chaitra Bessel Kristianto Universitas Negeri Malang
Fadia Irsania Putri Universitas Negeri Malang
Agung Bella Putra Utama Universitas Negeri Malang

DOI:

https://doi.org/10.56705/ijodas.v7i2.459

Keywords:

Prediction, XGBoost, LSTM, CNN

Abstract

Introduction: Accurate stock price forecasting is important for investors and financial institutions because stock movements reflect dynamic market conditions and influence investment decision-making. This study compares CNN-LSTM and XGBoost-LSTM models for predicting stock prices in the Indonesian banking sector. Method: Historical daily closing prices of PT Bank Central Asia Tbk (BBCA), PT Bank Rakyat Indonesia (Persero) Tbk (BBRI), and PT Bank Mandiri (Persero) Tbk (BMRI) were collected from Yahoo Finance, with 1,000 observations for each stock. Data were normalized using Min-Max scaling, transformed using a four-day sliding window to predict the following day, and chronologically divided into 80% training and 20% testing sets. Both models were evaluated using RMSE, MAE, R², and MAPE, with each experiment repeated ten times. Results and Discussion: CNN-LSTM consistently outperformed XGBoost-LSTM on all three test datasets. For BBCA, BBRI, and BMRI, CNN-LSTM achieved R² values of 0.8589, 0.7746, and 0.7409, respectively, compared with 0.8048, 0.7509, and 0.6260 for XGBoost-LSTM. CNN-LSTM also produced lower RMSE, MAE, and MAPE values across all test sets, indicating stronger and more stable generalization. Conclusion: CNN-LSTM provides more reliable predictive performance than XGBoost-LSTM for the evaluated Indonesian banking stocks, demonstrating the effectiveness of combining local feature extraction with temporal dependency learning for stock price forecasting

Downloads

Download data is not yet available.

References

[1] M. A. Khan, H. Ali, H. Shabbir, F. Noor, and M. D. Majid, “Impact of Macroeconomic Indicators on Stock Market Predictions : A Cross Country Analysis,” vol. 08, no. 01, 2024, doi: https://doi.org/10.56979/801/2024.

[2] P. Meher and R. K. Mishra, “Assessing the influence of factors affecting stock market: an ISM approach,” 2026, doi: https://doi.org/10.1108/QRFM-05-2024-0135.

[3] R. Islam and Z. Ikbal, “Impact Of Predictive Data Modeling On Business Decision-Making : A Review Of Studies Across Retail , Finance , And Logistics,” vol. 02, no. 02, pp. 33–62, 2022, doi: https://doi.org/10.63125/8hfbkt70.

[4] A. Roosmawarni, D. Fatihudin, and N. Mauliddah, “Market Capitalization and Financial Performance : Evidence from Banking Listed Company in Indonesia,” vol. 20, no. 2, pp. 124–136, 2022, doi: https://doi.org/10.31603/bisnisekonomi.v20i2.7835.

[5] G. Sonkavde, D. S. Dharrao, A. M. Bongale, and S. T. Deokate, “Forecasting Stock Market Prices Using Machine Learning and Deep Learning Models : A Systematic Review , Performance Analysis and Discussion of Implications,” 2023, doi: https://doi.org/10.3390/ijfs11030094.

[6] B. Ramadevi, “Chaotic Time Series Forecasting Approaches Using Machine Learning Techniques: A Review,” 2022, doi: https://doi.org/10.3390/sym14050955.

[7] J. Shah, D. Vaidya, and M. Shah, “Intelligent Systems with Applications A comprehensive review on multiple hybrid deep learning approaches for stock prediction,” Intell. Syst. with Appl., vol. 16, no. July, p. 200111, 2022, doi: https://doi.org/10.1016/j.iswa.2022.200111.

[8] H. Song, “Forecasting Stock Market Indices Using the Recurrent Neural Network Based Hybrid Models: CNN-LSTM, GRU-CNN, and Ensemble Models,” 2023, doi: https://doi.org/10.3390/app13074644.

[9] J. Zhang, L. Ye, and Y. Lai, “Stock Price Prediction Using CNN-BiLSTM-Attention Model,” pp. 1–18, 2023, doi: https://doi.org/10.3390/math11091985.

[10] Z. Shi, Y. Hu, G. Mo, and J. Wu, “Attention-based CNN-LSTM and XGBoost hybrid model for stock prediction,” vol. 14, no. 8, pp. 1–7, 2021.

[11] A. Mandal, Y. Shao, and X. Liu, “Automatic Historical Stock Price Dataset Generation Using Python,” pp. 1–6.

[12] J. M. Sahombu, “The Effect of Dividend Policy and Market Ratio on Stock Returns of Blue Chip Companies Listed on the Indonesia Stock,” vol. 5, no. 06, pp. 1810–1817, 2025, doi: https://doi.org/10.58471/jms.v5i06.

[13] H. Zheng, J. Wu, R. Song, L. Guo, and Z. Xu, “Predicting Financial Enterprise Stocks and Economic Data Trends Using Machine Learning Time Series Analysis,” 2024, doi: https://doi.org/10.20944/preprints202407.0895.v1.

[14] Y. Zhao and G. Yang, “Deep Learning-based Integrated Framework for stock price movement prediction,” Appl. Soft Comput., vol. 133, p. 109921, 2023, doi: https://doi.org/10.1016/j.asoc.2022.109921.

[15] D. Singh and B. Singh, “Feature wise normalization : An effective way of normalizing data ✩,” Pattern Recognit., vol. 122, p. 108307, 2022, doi: https://doi.org/10.1016/j.patcog.2021.108307.

[16] L. Liao, “An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on the Performance Properties of,” vol. 31, no. 3, 2022, doi: https://doi.org/10.1145/3506695.

[17] Z. Tao, Q. Xu, X. Liu, and J. Liu, “An integrated approach implementing sliding window and DTW distance for time series forecasting tasks,” pp. 20614–20625, 2023.

[18] S. Zaheer et al., “A Multi Parameter Forecasting for Stock Time Series Data Using LSTM and Deep Learning Model,” Mathematics, vol. 11, no. 3, pp. 1–24, 2023, doi: https://doi.org/10.3390/math11030590.

[19] Z. Deng and O. Russakovsky, “Remember the Past : Distilling Datasets into Addressable Memories for Neural Networks,” no. NeurIPS, pp. 1–14, 2022.

[20] Z. Zikai, “Enhancing Distributed Machine Learning through Data Shuffling : Techniques , Challenges , and Implications,” vol. 03018, pp. 1–8, 2025.

[21] S. Albelali and M. Ahmed, “Hidden Leaks in Time Series Forecasting: How Data Leakage Affects LSTM Evaluation Across Configurations and Validation Strategies”.

[22] K. Ullah et al., “Short-Term Load Forecasting : A Comprehensive Review and Simulation Study With CNN-LSTM Hybrids Approach,” IEEE Access, vol. 12, no. August, pp. 111858–111881, 2024, doi: https://doi.org/10.1109/ACCESS.2024.3440631.

[23] S. Lin, Y. Wang, H. Wei, X. Wang, and Z. Wang, “Hybrid Method for Oil Price Prediction Based on Feature Selection and XGBOOST-LSTM,” 2025, doi: https://doi.org/10.3390/en18092246.

[24] M. Ahmadzadeh and S. M. Zahrai, “An integrated deep neural network model combining 1D CNN and LSTM for structural health monitoring utilizing multisensor time-series data,” no. 3, 2025, doi: https://doi.org/10.1177/14759217241239041.

[25] J. Wu, K. Xu, X. Chen, S. Li, and J. Zhao, “Price graphs : Utilizing the structural information of financial time series for stock prediction,” vol. 588, pp. 405–424, 2022, doi: https://doi.org/10.1016/j.ins.2021.12.089.

[26] X. Wang, H. Ren, and A. Wang, “Smish : A Novel Activation Function for Deep Learning Methods,” 2022, doi: https://doi.org/10.3390/electronics11040540.

[27] A. Zafar et al., “A Comparison of Pooling Methods for Convolutional Neural Networks,” pp. 1–21, 2022, doi: https://doi.org/10.3390/app12178643.

[28] S. Abed and M. H. Alshayeji, “Hybrid XGBoost-LSTM model for oil well production forecasting using the volve field dataset,” Results Eng., vol. 30, no. March, p. 110128, 2026, doi: https://doi.org/10.1016/j.rineng.2026.110128.

[29] H. Oukhouya, H. Kadiri, K. El Himdi, and R. Guerbaz, “Forecasting International Stock Market Trends : XGBoost , LSTM , LSTM-XGBoost , And Backtesting XGBoost Models,” vol. 12, no. January, pp. 200–209, 2024, doi: https://doi.org/10.19139/soic-2310-5070-1822.

[30] K. S. Cabuk et al., “Chasing the objective upper eyelid symmetry formula,” Int. Ophthalmol., vol. 44, no. 1, pp. 1–9, 2024, doi: https://doi.org/10.1007/s10792-024-03157-y.

[31] T. O. Hodson, “Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not,” Geosci. Model Dev., vol. 15, no. 14, pp. 5481–5487, 2022, doi: https://doi.org/10.5194/gmd-15-5481-2022.

[32] J. Gao, “R-Squared ( R ) – How much variation is explained ?,” 2024, doi: https://doi.org/10.1177/26320843231186398.

[33] G. J. Exerc, S. Res, and E. Physiology, “What the mean absolute percentage error ( MAPE ) should adopt from Bland – Altman analyses,” 2025, doi: https://doi.org/10.1007/s12662-025-01084-3.