Weakly Supervised Sentiment Analysis of Gold Price Discussions Using Conventional Machine Learning and IndoBERT

Abstract

Gold price movements are closely monitored by investors because gold functions as both a safe-haven asset and a hedging instrument. With the rapid growth of social media, public discussions provide valuable insights into market sentiment. This study proposes a weakly supervised sentiment classification framework for analyzing Indonesian public sentiment toward gold prices on Platform X. A total of 7.283 Indonesian-language tweets were collected using the keyword “harga emas” during the 2023–2025 period. After preprocessing, 4.429 tweets were retained for sentiment labeling and model development. Sentiment labels were automatically generated using a domain-specific lexicon and subsequently validated through manual annotation to assess label reliability before model training. Four classification models, namely Naïve Bayes, K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and IndoBERT, were evaluated. Manual validation produced a Cohen's Kappa coefficient of 0.8718, indicating almost perfect agreement between annotators, while the lexicon-based labels achieved an accuracy of 70.62% against the manually annotated gold-standard dataset. Among the evaluated models, IndoBERT achieved the best performance with an accuracy of 98.31% and a Macro F1-score of 98.16%, outperforming SVM, Naïve Bayes, and KNN. However, evaluation using the manually annotated dataset yielded an accuracy of 69.49%, indicating that downstream classification performance remains dependent on the quality of weakly supervised labels. These findings demonstrate that weak supervision is an efficient strategy for large-scale sentiment annotation while emphasizing the importance of label validation for reliable sentiment classification in Indonesian financial social media