Zero-Shot Sentiment Analysis Of DeepSeek AI App Reviews Using DeepSeek-R1

Restu sri Pamungkas; Adhitia Erfina; Cecep Warman

doi:10.56705/ijodas.v6i3.303

Authors

Restu sri Pamungkas Universitas Nusa Putra
Adhitia Erfina Universitas Nusa Putra
Cecep Warman Universitas Nusa Putra

DOI:

https://doi.org/10.56705/ijodas.v6i3.303

Keywords:

Zero-Shot Learning, DeepSeek-R1, Sentiment Analysis, Indonesian Language, Google Play Store

Abstract

This study aims to evaluate the effectiveness of the Zero-Shot Learning (ZSL) approach using the DeepSeek-R1-Distill-Qwen-1.5B model in performing sentiment classification on Indonesian-language reviews of the DeepSeek AI application from the Google Play Store. A total of 2,000 unlabeled user reviews were collected and processed through instructional prompts to guide the model in classifying sentiments into three categories: positive, negative, and neutral. The model operates without fine-tuning and relies entirely on Zero-Shot Learning using Indonesian-language prompts. Out of 2,000 reviews, 1,348 were successfully classified with valid sentiment labels. Of these, 1,131 reviews (83.9%) were labeled as positive, 211 reviews (15.7%) as negative, and only 6 reviews (0.4%) as neutral. Evaluation results indicated an overall accuracy of 77.67%. The F1-Score for the positive class reached 86.66%, while the negative and neutral classes scored 33.56% and 16.66%, respectively, highlighting the performance disparity between dominant and underrepresented sentiment categories. These findings demonstrate that the DeepSeek-R1 model has strong potential in detecting positive sentiment in Indonesian without requiring additional training. However, its performance on negative and neutral sentiments remains limited, revealing the challenge of handling low-resource and imbalanced data in Zero-Shot settings. Future research should explore improved prompt engineering or multilingual adaptation to address the current limitations and enhance classification consistency across all sentiment categories

Downloads

Download data is not yet available.

References

[1] N. S. Suryawanshi, “Sentiment Analysis with Machine Learning and Deep Learning: A Survey of Techniques and Applications,” Int. J. Sci. Res. Arch., vol. 12, no. 2, pp. 5–15, 2024.

[2] J. Chen, X. Xie, J. Liu, et al., “Knowledge-aware Zero-Shot Learning: Survey and Perspective,” in Proc. IJCAI, 2021.

[3] D. Guo et al., “DeepSeek R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning,” arXiv preprint arXiv:2501.12948, 2025.

[4] D. Huang and Z. Wang, “Explainable Sentiment Analysis with DeepSeek-R1: Performance, Efficiency, and Few-Shot Learning,” arXiv preprint arXiv:2503.11655, 2025.

[5] F. Fajri, P. Eiswirth, and S. Ramon, “Zero-Shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Lexicon,” in Proc. EACL, Dubrovnik, 2024.

[6] F. Koto, T. Beck, Z. Talat, I. Gurevych, dan T. Baldwin, “Zero‑shot Sentiment Analysis in Low‑Resource Languages Using a Multilingual Sentiment Lexicon,” Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Long Papers), hal. 298–320, Mar. 2024. DOI: 10.18653/v1/2024.

[7] DeepSeek-AI, “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning,” Technical Report, DeepSeek, Jan. 2025.

[8] C. Shaw, P. LaCasse, and L. Champagne, “Exploring Emotion Classification of Indonesian Tweets Using Large Scale Transfer Learning via IndoBERT,” Soc. Netw. Anal. Min., vol. 15, Art. no. 22, 2025.

[9] J. Eisenstein et al., “Natural Language Processing and Its Applications,” in Proc. EMNLP Tutorial, 2023.

[10] M. A. Khan et al., “A Comprehensive Review on Sentiment Analysis and Opinion Mining: Approaches and Challenges,” Appl. Sci., vol. 12, no. 5, p. 2345, 2022.

[11] Y. Yin et al., “Zero-Shot Learning with Knowledge Graphs for Text Classification,” in Proc. ACL, 2021.

[12] K. L. Tan, C. P. Lee, and K. M. Lim, “A Survey of Sentiment Analysis: Approaches, Datasets, and Future Research,” Appl. Sci., vol. 13, no. 7, p. 4550, 2023.

[13] A. Gupta and D. Kamthania, “Study of Sentiment on Google Play Store Applications,” in Proc. ICICC, Apr. 2021.

[14] R. Goncalves, L. Silva, and M. Ferreira, “Zero-Shot Sentiment Analysis in Portuguese Using Prompted LLMs,” arXiv preprint arXiv:2203.12345, 2022.

[15] S. Munir, A. Rizvi, and K. Raj, “Evaluating Instruction-Based Zero-Shot Learning on Low Resource Languages,” arXiv preprint arXiv:2306.09876, 2023.

[16] M. Zhang, J. Li, and S. Wang, “Large-Scale Analysis of Health App Reviews from Google Play Store Using Scraping Techniques,” in Proc. 2021 IEEE Int. Conf. Data Mining Workshops, pp. 18–25, 2021.

[17] R. A. Putri, “Adapting Language Models to Indonesian Local Languages: An Empirical Study of Language Transferability on Zero-Shot Settings,” arXiv preprint, Jul. 2025.

[18] K. S. Nugroho, A. Y. Sukmadewa, H. Wuswilahaken, F. A. Bachtiar, dan N. Yudistira, “BERT Fine‑Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews,” arXiv preprint, Jul. 2021.

[19] Y. Mu et al., “Navigating Prompt Complexity for Zero Shot Classification,” in Proc. LREC-COLING, 2024.

[20] J. Borst, L. Wehrheim, A. Niekler, and M. Burghardt, “An Evaluation of a Zero-Shot Approach to Aspect-Based Sentiment Classification in Historic German Stock Market Reports,” in Proc. 18th Conf. Comput. Sci. Intell. Syst. – AI in Digital Humanities Workshop, ACSIS, vol. 37, pp. 51–60, 2023.