Comparison of Naïve Bayes and SVM in Sentiment Analysis of ChatGPT for Learning on X and YouTube

Ni Putu Eka Swari; Ni Wayan Jeri Kusuma Dewi; Ni Ketut Utami Nilawati; Aniek Suryanti Kusuma; Ni Luh Wiwik Sri Rahayu Ginantra

doi:10.56705/ijodas.v7i1.382

Authors

Ni Putu Eka Swari Institut Bisnis Dan Teknologi Indonesia
Ni Wayan Jeri Kusuma Dewi Institut Bisnis Dan Teknologi Indonesia
Ni Ketut Utami Nilawati Institut Bisnis Dan Teknologi Indonesia
Aniek Suryanti Kusuma Institut Bisnis Dan Teknologi Indonesia
Ni Luh Wiwik Sri Rahayu Ginantra Institut Bisnis Dan Teknologi Indonesia

DOI:

https://doi.org/10.56705/ijodas.v7i1.382

Keywords:

Naive Bayes, Support Vector Machine(SVM) , Social Media, Sentiment Analysis, Chat GPT

Abstract

The rapid development of artificial intelligence technology has encouraged users to actively express opinions on social media platforms such as X and YouTube, including discussions on the use of ChatGPT as a learning support tool. This study aims to analyze public sentiment toward the use of ChatGPT in learning contexts by comparing the performance of the Naïve Bayes and Support Vector Machine (SVM) classification methods. A total of 5,500 comments from platform X and 5,543 comments from YouTube were collected through a crawling process using relevant keywords during the period from January 2023 to December 2025. The data were preprocessed and labeled into three sentiment classes (positive, negative, and neutral) using a lexicon-based approach with the INSET Lexicon. Feature extraction was conducted using the Term Frequency–Inverse Document Frequency (TF-IDF) method, and the dataset was divided into training and testing sets with an 80:20 ratio. Model performance was evaluated using accuracy, precision, recall, and F1-score. The results show that the SVM classifier consistently outperformed the Naïve Bayes method on both platforms. On platform X, SVM achieved an accuracy of 76.67%, while Naïve Bayes obtained 74.60%. On YouTube, SVM achieved an accuracy of 73.10%, significantly higher than Naïve Bayes at 62.04%. These findings indicate that SVM is more effective for sentiment analysis of social media data related to the use of ChatGPT in learning environments

Downloads

Download data is not yet available.

References

[1] M. F. Saleh and R. Imanda, “Public Sentiment Analysis of the Free Meal Program: A Comparison of Naive Bayes and Support Vector Machine Methods on the Twitter (X) Social Media Platform,” J. Appl. Informatics Comput., vol. 9, no. 1, pp. 131–139, 2025, doi: 10.30871/jaic.v9i1.8895.

[2] S. Ngarifatul Khofiyah and P. Subarkah, “Comparison of Naive Bayes and SVM in Public Opinion Sentiment Analysis on Platform X,” J. Teknol. Inf. Univ. Lambung Mangkurat, pp. 125–138, 2025, doi: 10.20527/jtiulm.v10i2.478.

[3] M. Alawida, S. Mejri, A. Mehmood, B. Chikhaoui, and O. Isaac Abiodun, “A Comprehensive Study of ChatGPT: Advancements, Limitations, and Ethical Considerations in Natural Language Processing and Cybersecurity,” Inf., vol. 14, no. 8, 2023, doi: 10.3390/info14080462.

[4] R. F. P. Pratama and W. Maharani, “Comparative Analysis of Naive Bayes and SVM for Improved Emotion Classification on Social Media,” Edumatic J. Pendidik. Inform., vol. 9, no. 1, pp. 11–20, 2025, doi: 10.29408/edumatic.v9i1.29087.

[5] A. Göçen, M. M. Ibrahim, and A. U. I. Khan, “Public attitudes toward higher education using sentiment analysis and topic modeling,” Discov. Artif. Intell., vol. 4, no. 1, 2024, doi: 10.1007/s44163-024-00195-4.

[6] Y. Mao, Q. Liu, and Y. Zhang, “Sentiment analysis methods, applications, and challenges: A systematic literature review,” J. King Saud Univ. - Comput. Inf. Sci., vol. 36, no. 4, p. 102048, 2024, doi: 10.1016/j.jksuci.2024.102048.

[7] S. Yang, Y. Dong, and Z. G. Yu, “ChatGPT in Education: Ethical Considerations and Sentiment Analysis,” Int. J. Inf. Commun. Technol. Educ., vol. 20, no. 1, pp. 1–19, 2024, doi: 10.4018/IJICTE.346826.

[8] N. F. B. Casillano, “Education in the ChatGPT Era: A Sentiment Analysis of Public Discourse on the Role of Language Models in Education,” J. Eval. Educ., vol. 5, no. 4, pp. 144–154, 2024, doi: 10.37251/jee.v5i4.1151.

[9] L. Lu et al., “Healthcare professionals and the public sentiment analysis of ChatGPT in clinical practice,” Sci. Rep., vol. 15, no. 1, pp. 1–11, 2025, doi: 10.1038/s41598-024-84512-y.

[10] D. Smith-Mutegi, Y. Mamo, J. Kim, H. Crompton, and M. McConnell, “Perceptions of STEM education and artificial intelligence: a Twitter (X) sentiment analysis,” Int. J. STEM Educ., vol. 12, no. 1, 2025, doi: 10.1186/s40594-025-00527-5.

[11] M. A. Palomino and F. Aider, “Evaluating the Effectiveness of Text Pre-Processing in Sentiment Analysis,” Appl. Sci., vol. 12, no. 17, 2022, doi: 10.3390/app12178765.

[12] M. Siino, I. Tinnirello, and M. La Cascia, “Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiers,” Inf. Syst., vol. 121, no. March 2023, p. 102342, 2024, doi: 10.1016/j.is.2023.102342.

[13] A. F. Aufar, Mochamad Alfan Rosid, A. Eviyanti, and I. R. I. Astutik, “Optimizing Text Preprocessing for Accurate Sentiment Analysis on E-Wallet Reviews,” JICTE (Journal Inf. Comput. Technol. Educ., vol. 7, no. 2, pp. 42–50, 2023, doi: 10.21070/jicte.v7i2.1650.

[14] X. Ding, B. Liu, and P. S. Yu, “A holistic lexicon-based approach to opinion mining,” WSDM’08 - Proc. 2008 Int. Conf. Web Search Data Min., no. February 2008, pp. 231–239, 2008, doi: 10.1145/1341531.1341561.

[15] A. Nurkasanah and M. Hayaty, “Feature Extraction using Lexicon on the Emotion Recognition Dataset of Indonesian Text,” Ultim. J. Tek. Inform., vol. 14, no. 1, pp. 20–27, 2022, doi: 10.31937/ti.v14i1.2540.

[16] Y. Fauziah, B. Yuwono, and A. S. Aribowo, “Lexicon Based Sentiment Analysis in Indonesia Languages : A Systematic Literature Review,” RSF Conf. Ser. Eng. Technol., vol. 1, no. 1, pp. 363–367, 2021, doi: 10.31098/cset.v1i1.397.

[17] G. Popoola, K. K. Abdullah, G. S. Fuhnwi, and J. Agbaje, “Sentiment Analysis of Financial News Data using TF-IDF and Machine Learning Algorithms,” 2024 IEEE 3rd Int. Conf. AI Cybersecurity, ICAIC 2024, 2024, doi: 10.1109/ICAIC60265.2024.10433843.

[18] H. Barus, I. N. Fajri, and Y. Pristyanto, “Sentiment Classification Analysis of Tokopedia Reviews Using TF-IDF, SMOTE, and Traditional Machine Learning Models,” J. Appl. Informatics Comput., vol. 9, no. 5, pp. 2552–2561, 2025, doi: 10.30871/jaic.v9i5.10524.

[19] Andi Riswawan, “Implementing TF-IDF and Logistic Regression for Sentiment Analysis of YouTube Comments on the iPhone 16,” J. Teknol. Dan Open Source, vol. 7, no. 2, pp. 277–284, 2024, doi: 10.36378/jtos.v7i2.4753.

[20] F. Fitriana and H. Setiawan, “Performance Analysis of SVM In Emotion Classification: A Comparative Study Of TF-IDF and Countvectorizer,” J. Embed. Syst. Secur. Intell. Syst., vol. 6, no. 2, pp. 133–145, 2025, doi: 10.59562/jessi.v6i2.8396.

[21] H. Guo, J. Sun, J. Luo, Y. Peng, and C. Ye, “Thickness-Related Fault Diagnosis of Steel Strip Based on W-KPLS Method Considering Mechanism Weight Optimization,” Appl. Sci., vol. 12, no. 9, 2022, doi: 10.3390/app12094491.

[22] D. Shalikha and A. Alamsyah, “Improved Accuracy of Naïve Bayes Algorithm and Support Vector Machine Using Particle Swarm Optimization for Menstrual Cup Sentiment Analysis on Twitter,” J. Adv. Inf. Syst. Technol., vol. 4, no. 2, pp. 139–148, 2023, doi: 10.15294/jaist.v4i2.59561.

[23] C. Dewi, R. C. Chen, H. J. Christanto, and F. Cauteruccio, “Multinomial Naïve Bayes Classifier for Sentiment Analysis of Internet Movie Database,” Vietnam J. Comput. Sci., vol. 10, no. 4, pp. 485–498, 2023, doi: 10.1142/S2196888823500100.

[24] Heri Suroyo and E. J. Pratama, “Comparison of Text Representation Methods for Sentiment Analysis Using Support Vector Machine,” J. Adv. Inf. Ind. Technol., vol. 7, no. 1, pp. 21–30, 2025, doi: 10.52435/jaiit.v7i1.610.

[25] T. Ahmed Khan, R. Sadiq, Z. Shahid, M. M. Alam, and M. Mohd Su’ud, “Sentiment Analysis using Support Vector Machine and Random Forest,” J. Informatics Web Eng., vol. 3, no. 1, pp. 67–75, 2024, doi: 10.33093/jiwe.2024.3.1.5.

[26] G. Mutanov, V. Karyukin, and Z. Mamykova, “Multi-class sentiment analysis of social media data with machine learning algorithms,” Comput. Mater. Contin., vol. 69, no. 1, pp. 913–930, 2021, doi: 10.32604/cmc.2021.017827.

[27] J. Opitz, “A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice,” Trans. Assoc. Comput. Linguist., vol. 12, no. 2018, pp. 820–836, 2024, doi: 10.1162/tacl_a_00675.

[28] X. Chen, D. Esserman, and F. Li, “Competing risks regression for clustered survival data via the marginal additive subdistribution hazards model,” Stat. Neerl., vol. 78, no. 2, pp. 281–301, 2024, doi: 10.1111/stan.12317.

[29] E. Kahya Özyirmidokuz, B. Molu Elmas, and E. A. Stoica, “AI-Based Sentiment Analysis of E-Commerce Customer Feedback: A Bilingual Parallel Study on the Fast Food Industry in Turkish and English,” J. Theor. Appl. Electron. Commer. Res., vol. 20, no. 4, p. 294, 2025, doi: 10.3390/jtaer20040294.

[30] M. M. Danyal, S. S. Khan, M. Khan, M. B. Ghaffar, B. Khan, and M. Arshad, “Sentiment Analysis Based on Performance of Linear Support Vector Machine and Multinomial Naïve Bayes Using Movie Reviews with Baseline Techniques,” J. Big Data, vol. 5, pp. 1–18, 2023, doi: 10.32604/jbd.2023.041319.

[31] H. Setyawan, L. M. Azizah, and A. Y. Pradani, “Sentiment Analysis of Public Responses on Indonesia Government Using Naïve Bayes and Support Vector Machine,” Emerg. Inf. Sci. Technol., vol. 4, no. 1, pp. 1–7, 2023, doi: 10.18196/eist.v4i1.18681.

[32] N. Z. B. Jannah and K. Kusnawi, “Comparison of Naïve Bayes and SVM in Sentiment Analysis of Product Reviews on Marketplaces,” Sinkron, vol. 8, no. 2, pp. 727–733, 2024, doi: 10.33395/sinkron.v8i2.13559.

[33] M. Hamad and V. Prevelakis, “SAVTA: A hybrid vehicular threat model: Overview and case study,” Inf., vol. 11, no. 5, pp. 1–22, 2020, doi: 10.3390/INFO11050273.

[34] J. Nabila, Ida Ayu Devian Branitasandhini Putra, and Heri Wijayanto, “Sentiment Analysis on Starbucks Reviews: Implementation of K-Nearest Neighbors and Support Vector Machine,” Infact Int. J. Comput., vol. 9, no. 02, pp. 79–86, 2025, doi: 10.61179/infact.v9i02.761.