Sentiment Classification Of TikTok Comments on The Free Nutritious Meal Program Using Multinomial Naïve Bayes

Salom Sefanya Onibala; Rosmasari; Kezia Arum Sary

doi:10.56705/ijodas.v7i2.448

Authors

Salom Sefanya Onibala Universitas Mulawarman
Rosmasari Universitas Mulawarman
Kezia Arum Sary Universitas Mulawarman

DOI:

https://doi.org/10.56705/ijodas.v7i2.448

Keywords:

Sentiment Classification, Multinomial Naive Bayes, IndoBERT, Free Nutritious Meal Program, TikTok, TF-IDF

Abstract

TikTok has become an important platform for public discussion of government programs, including the Free Nutritious Meals (MBG) Program. The large volume of user comments makes manual analysis inefficient and motivates the use of automated sentiment classification. This study investigates public sentiment toward the MBG Program through comments collected from the official TikTok account of the National Nutrition Agency (@badangizinasional.ri). The study applies a pipeline consisting of text preprocessing, automatic sentiment labeling using the IndoRoBERTa sentiment-classification model, undersampling, TF-IDF feature extraction, and Multinomial Naïve Bayes classification. A total of 1,048 comments were collected, of which 1,028 remained after preprocessing. The IndoRoBERTa-based pseudo-labeling process assigned 333 comments (32.39%) to the positive class and 695 comments (67.61%) to the negative class. Because the class distribution was imbalanced, undersampling was applied, resulting in 666 comments consisting of 333 comments per class. Hyperparameter optimization using GridSearchCV with 5-fold cross-validation selected alpha = 0.5, max_df = 0.8, min_df = 3, ngram_range = (1,3), and sublinear_tf = False. On the 20% stratified test set, Multinomial Naïve Bayes achieved 80.60% accuracy, 81.04% precision, 80.60% recall, and 80.53% F1-score when evaluated against the IndoRoBERTa-generated labels. These results should be interpreted as agreement with automatically generated pseudo-labels rather than direct evidence of performance against human-verified ground truth. Therefore, the observed negative sentiment distribution reflects the original post-preprocessing dataset before undersampling, while the classifier was trained and tested on a balanced subset. Manual validation of the pseudo-labels and comparison with alternative classifiers are recommended for future work

Downloads

Download data is not yet available.

References

[1] W. F. Satrya, R. Aprilliyani, and E. H. Yossy, “Sentiment analysis of Indonesian police chief using multi-level ensemble model,” Procedia Comput. Sci., vol. 216, no. 2022, pp. 620–629, 2022, doi: https://doi.org/10.1016/j.procs.2022.12.177.

[2] M. Rodríguez-Ibánez, A. Casánez-Ventura, F. Castejón-Mateos, and P. M. Cuenca-Jiménez, “A review on sentiment analysis from social media platforms,” Expert Syst. Appl., vol. 223, no. August 2022, 2023, doi: https://doi.org/10.1016/j.eswa.2023.119862.

[3] Z. Yao, J. Yang, J. Liu, M. Keith, and C. H. Guan, “Comparing tweet sentiments in megacities using machine learning techniques: In the midst of COVID-19,” Cities, vol. 116, no. July 2020, p. 103273, 2021, doi: https://doi.org/10.1016/j.cities.2021.103273.

[4] F. Guo, Z. Liu, Q. Lu, S. Ji, and C. Zhang, “Public Opinion About COVID-19 on a Microblog Platform in China: Topic Modeling and Multidimensional Sentiment Analysis of Social Media,” J. Med. Internet Res., vol. 26, no. 1, 2024, doi: https://doi.org/10.2196/47508.

[5] M. Isnan, G. N. Elwirehardja, and B. Pardamean, “Sentiment Analysis for TikTok Review Using VADER Sentiment and SVM Model,” Procedia Comput. Sci., vol. 227, pp. 168–175, 2023, doi: https://doi.org/10.1016/j.procs.2023.

[6] D. F. M. Dina, T. Haryanti, and M. A. Haq, “Analisis Sentimen Terhadap Komentar Pada Media Sosial Tiktok Yang Berpotensi Menyebabkan Depresi Menggunakan Metode Naive Bayes,” Comput. Insight J. Comput. Sci., vol. 7, no. 1, pp. 1–9, 2025, doi: https://doi.org/10.30651/comp_insight.v7i1.26327.

[7] R. Haque, N. Islam, M. Tasneem, and A. K. Das, “Multi-class sentiment classification on Bengali social media comments using machine learning,” Int. J. Cogn. Comput. Eng., vol. 4, no. December 2021, pp. 21–35, 2023, doi: https://doi.org/10.1016/j.ijcce.2023.01.001.

[8] N. Puspitasari, R. Rosmasari, F. W. Pratama, and H. Sulastri, “Quality Classification of Palm Oil Varieties Using Naive Bayes Classifier,” Digit. Zo. J. Teknol. Inf. dan Komun., vol. 13, no. 1, pp. 11–23, 2022, doi: https://doi.org/10.31849/digitalzone.v13i1.9773.

[9] D. G. Pradana, M. L. Alghifari, M. F. Juna, and D. Palaguna, “Klasifikasi Penyakit Jantung Menggunakan Metode Artificial Neural Network,” Indones. J. Data Sci., vol. 3, no. 2, pp. 55–60, 2022, doi: https://doi.org/10.56705/ijodas.v3i2.35.

[10] M. R. Ananda et al., “Implementasi Support Vector Machine Dalam Analisis Sentimen Netizen Terhadap Program Makan Bergizi Gratis,” J. Mhs. Tek. Inform., vol. 10, no. 2, pp. 3594–3601, 2026.

[11] W. Maharani, H. Daud, N. Muhammad, and E. A. Kadir, “Leveraging Social Media Data for Forest Fires Sentiment Classification: A Data-Driven Method,” J. Inf. Syst. Eng. Bus. Intell., vol. 10, no. 3, pp. 392–407, 2024, doi: https://doi.org/10.20473/jisebi.10.3.392-407.

[12] W. N. Mergianti, N. Khudin, A. Afandi, N. A. Shofah, and D. C. R. Novitasari, “Analisis Sentimen Pengguna Aplikasi X Terhadap Pelaksanaan Makan Bergizi Gratis Menggunakan Metode Adaptive Neuro-Fuzzy Inference System,” Sains Data J. Stud. Mat. dan Teknol., vol. 1, pp. 24–33, 2026, doi: https://doi.org/10.10.52620/sainsdata.v4i1.339 ABSTRAK.

[13] Z. P. Tiararosa, I. S. Wijaya, and E. Rohaini, “Perbandingan SVM dan Naive Bayes pada Analisis Sentimen MBG di Platform TikTok dan Instagram,” J. Inform. dan Rekayasa Komput., vol. 6, no. April, pp. 1983–1993, 2026, doi: https://doi.org/10.33998/jakakom.v6i1.

[14] W. B. Zulfikar, A. R. Atmadja, and S. F. Pratama, “Sentiment Analysis on Social Media Against Public Policy Using Multinomial Naive Bayes,” Sci. J. Informatics, vol. 10, no. 1, pp. 25–34, 2023, doi: https://doi.org/10.15294/sji.v10i1.39952.

[15] D. Bor, B. S. Lee, and E. J. Oughton, “Quantifying polarization across political groups on key policy issues using sentiment analysis,” pp. 1–31, 2021.

[16] R. W. Harwenda, M. D. Angelo, I. Budi, A. B. Santoso, and P. K. Putra, “Sentiment Analysis on Government Public Policies: A Systematic Literature Review,” Dinasti Int. J. Educ. Manag. Soc. Sci., vol. 6, no. 5, pp. 4192–4211, 2025, doi: https://doi.org/10.38035/dijemss.v6i5.4699.

[17] Herlawati, D. B. Srisulistiowati, S. C. Agustin, P. H. Syafina, N. Rachmatin, and S. Setiawati, “Metode Naïve Bayes dan Support Vector Machine untuk Mengolah Sentimen Ulasan dan Komentar di Platform Digital,” J. Students‘ Res. Comput. Sci., vol. 5, no. 2, pp. 197–212, 2024, doi: https://doi.org/10.31599/dby15h32.

[18] J. A. Zulqornain, Indriati, and P. P. Adikara, “Analisis Sentimen Tanggapan Masyarakat Aplikasi Tiktok Menggunakan Metode Naïve Bayes dan Categorial Propotional Difference (CPD),” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 7, pp. 2886–2890, 2021.

[19] J. A. Widians, U. Hairah, and T. Fadila, “Sistem Pakar Diagnosis Penyakit Mioma Uteri Menggunakan Teorema Bayes,” J. Rekayasa Teknol. Inf., vol. 8, no. 2, p. 174, 2024, doi: https://doi.org/10.30872/jurti.v8i2.1446.

[20] S. K. Q. Asit, C. M. Nahumury, J. Feninlambir, W. ode N. Hasiyati, and F. S. Tomia, “Analisis Sistem Informasi Akuntansi Siklus Pendapatan Menggunakan Dfd Dan Flowchart Pada Bengkel Latansa Walding,” J. Tagalaya Pengabdi. Kpd. Masy., vol. 1, no. 2, pp. 193–200, 2024, doi: https://doi.org/10.71315/jtpkm.v1i2.30.

[21] N. Salsabilah, Irawati, and L. N. Hayati, “Performance Analysis of Convolutional Neural Networks and Naive Bayes Methods for Disease Classification in Tomato Plant Leaves,” Indones. J. Data Sci., vol. 6, no. 3, pp. 445–453, 2025, doi: https://doi.org/10.56705/ijodas.v6i3.255.

[22] C. Hill, F. Irshaidat, M. Johnson, and J. Fresneda, “An analytical assessment of sentiment analysis trends and methods through systematic review and topic modeling,” Decis. Anal. J., vol. 17, no. October, p. 100644, 2025, doi: https://doi.org/10.1016/j.dajour.2025.100644.

[23] N. P. Yusal, A. Erfina, and C. Warman, “Sarcasm and Irony Detection in Lazada App Reviews Using IndoBERT,” vol. 6, no. 3, pp. 618–625, 2025.

[24] C. Shaw, P. LaCasse, and L. Champagne, “Exploring emotion classification of indonesian tweets using large scale transfer learning via IndoBERT,” Soc. Netw. Anal. Min., vol. 15, no. 1, pp. 1–12, 2025, doi: https://doi.org/10.1007/s13278-025-01439-6.

[25] M. A. Imbiri, L. Y. Baisa, and J. J. A. Limbong, “Sentiment Classification and Influential Actor Detection on Twitter ( Case Study : The Raja Ampat Mining Conflict ),” vol. 7, no. 1, pp. 12–28, 2026.

[26] N. N. A. S. Wahyuni, I. G. I. Sudipa, N. N. A. J. Sastaparamitha, A. G. Willdahlia, and I. G. A. A. M. Aristamy, “Public Response on X to the Revocation of Indonesia’s 3-Kg LPG Retail Ban: A Support Vector Machine Study,” Indones. J. Data Sci., vol. 6, no. 3, pp. 412–425, 2025, doi: https://doi.org/10.56705/ijodas.v6i3.349.

[27] N. Raveendhran and N. Krishnan, “A novel hybrid SMOTE oversampling approach for balancing class distribution on social media text,” Bull. Electr. Eng. Informatics, vol. 14, no. 1, pp. 638–646, 2025, doi: https://doi.org/10.11591/eei.v14i1.8380.

[28] P. Guleria, J. Frnda, and P. N. Srinivasu, “NLP based text classification using TF-IDF enabled fine-tuned long short-term memory: An empirical analysis,” Array, vol. 27, no. August, p. 100467, 2025, doi: https://doi.org/10.1016/j.array.2025.100467.

[29] L. Zhang, “Features extraction based on Naive Bayes algorithm and TF-IDF for news classification,” PLoS One, vol. 20, no. 7 July, pp. 1–17, 2025, doi: https://doi.org/10.1371/journal.pone.0327347.

[30] N. Kewsuwun and S. Kajornkasirat, “A sentiment analysis model of agritech startup on Facebook comments using naive Bayes classifier,” Int. J. Electr. Comput. Eng., vol. 12, no. 3, pp. 2829–2838, 2022, doi: https://doi.org/https://doi.org/10.1016/j.rineng.2024.102148

[31] V. A. Fitri, R. Andreswari, and M. A. Hasibuan, “Sentiment analysis of social media Twitter with case of Anti-LGBT campaign in Indonesia using Naïve Bayes, decision tree, and random forest algorithm,” Procedia Comput. Sci., vol. 161, pp. 765–772, 2021, doi: https://doi.org/10.1016/j.procs.2019.11.181.

[32] M. A. K. Raiaan et al., “A systematic review of hyperparameter optimization techniques in Convolutional Neural Networks,” Decis. Anal. J., vol. 11, no. April, p. 100470, 2024, doi: https://doi.org/10.1016/j.dajour.2024.100470.

[33] Y. N. Fuadah, M. A. Pramudito, and K. M. Lim, “An Optimal Approach for Heart Sound Classification Using Grid Search in Hyperparameter Optimization of Machine Learning,” Bioengineering, vol. 10, no. 1, 2023, doi: https://doi.org/10.3390/bioengineering10010045.

[34] S. I. G. Iwan, M. Habibi, E. S. Jullev Atmadji, and I. Arfiani, “Predictive Modeling of Air Quality Levels Using Decision Tree Classification: Insights from Environmental and Demographic Factors,” Indones. J. Data Sci., vol. 5, no. 3, pp. 251–258, 2024, doi: https://doi.org/10.56705/ijodas.v5i3.201.

[35] C. Dewi, R. C. Chen, H. J. Christanto, and F. Cauteruccio, “Multinomial Naïve Bayes Classifier for Sentiment Analysis of Internet Movie Database,” Vietnam J. Comput. Sci., vol. 10, no. 4, pp. 485–498, 2023, doi: https://doi.org/10.1142/S2196888823500100.

[36] M. Yusrizal and T. B. Sasongko, “Analisis Sentimen Masyarakat Terhadap Presiden dan Calon Presiden Terpilih 2024 Menggunakan Naïve Bayes,” J. Media Inform. Budidarma, vol. 8, no. 3, p. 1673, 2024, doi: https://doi.org/10.30865/mib.v8i3.7882.