Confidence-Aware Depression Severity Detection in Low-Resource Urdu Social Media Text: A Multilingual Machine Learning Approach

Ahmad Naswin; Yuli Praptomo Pamungkas Hari Sungkowo

doi:10.56705/62qxjt74

Authors

Ahmad Naswin Universitas Megarezky
Yuli Praptomo Pamungkas Hari Sungkowo STIMIK El Rahma Yogyakarta

DOI:

https://doi.org/10.56705/62qxjt74

Keywords:

Depression Severity Classification, Urdu Social Media Text, Multilingual NLP, Confidence-Aware Learning, Mental Health Informatic, Machine Learning, TF-IDF

Abstract

Depression is a major mental health concern that requires early identification and timely intervention. Social media has become an important source of user-generated text that may reflect emotional distress, hopelessness, social withdrawal, and suicidal ideation. However, most existing depression detection studies focus on English or high-resource languages, while research on low-resource languages such as Urdu remains limited. This study investigates depression severity classification in Urdu social media text using multilingual and confidence-aware natural language processing approaches. The dataset consists of 4,000 Twitter/X posts collected between January 2024 and April 2025, annotated into four severity classes: none, mild, moderate, and severe. Each post is represented in three parallel textual forms: native Urdu script, Roman Urdu transliteration, and English translation. The dataset also includes label confidence scores, human verification indicators, cultural markers, and depression-related keywords. Several text representation scenarios were evaluated, including Urdu text, Roman Urdu text, English text, and combined multilingual features. Baseline machine learning models were developed using TF-IDF features with Logistic Regression, Linear Support Vector Machine, and Multinomial Naive Bayes. Confidence-aware learning was examined by incorporating label confidence scores as sample weights and by evaluating a high-confidence subset. The experimental results showed that all baseline models achieved perfect classification performance, with accuracy, macro F1-score, weighted F1-score, and Cohen’s Kappa values of 1.000 across the evaluated scenarios. These results indicate that the dataset contains highly separable linguistic patterns among depression severity classes. However, further inspection suggests that repeated or highly similar textual patterns may contribute to overly optimistic performance. Therefore, stricter validation using duplicate-free splitting, external datasets, and transformer-based models is recommended for future work. This study provides a preliminary benchmark for multilingual depression severity classification in low-resource Urdu text and highlights the potential of AI-driven mental health informatics as a supportive early-warning tool rather than a clinical diagnostic system

References

[1] D. Phiri, F. Makowa, V. L. Amelia, Y. V. A. Phiri, L. P. Dlamini, and M.-H. Chung, “Text-Based Depression Prediction on Social Media Using Machine Learning: Systematic Review and Meta-Analysis,” J. Med. Internet Res., vol. 27, p. e59002, Apr. 2025, doi: 10.2196/59002.

[2] W. B. Tahir, S. Khalid, S. Almutairi, M. Abohashrh, S. A. Memon, and J. Khan, “Depression Detection in Social Media: A Comprehensive Review of Machine Learning and Deep Learning Techniques,” IEEE Access, vol. 13, pp. 12789–12818, 2025, doi: 10.1109/ACCESS.2025.3530862.

[3] A. Khan and R. Ali, “Unraveling minds in the digital era: a review on mapping mental health disorders through machine learning techniques using online social media,” Soc. Netw. Anal. Min., vol. 14, no. 1, p. 78, Apr. 2024, doi: 10.1007/s13278-024-01205-0.

[4] M. Omar and I. Levkovich, “Exploring the efficacy and potential of large language models for depression: A systematic review,” J. Affect. Disord., vol. 371, pp. 234–244, Feb. 2025, doi: 10.1016/j.jad.2024.11.052.

[5] H. Fisher et al., “Language-based detection of depression with machine learning: systematic review and meta-analysis,” Npj Digit. Med., vol. 9, no. 1, p. 273, Feb. 2026, doi: 10.1038/s41746-026-02448-1.

[6] D. William and D. Suhartono, “Text-based Depression Detection on Social Media Posts: A Systematic Literature Review,” Procedia Comput. Sci., vol. 179, pp. 582–589, 2021, doi: 10.1016/j.procs.2021.01.043.

[7] R. Chiong, G. S. Budhi, S. Dhakal, and F. Chiong, “A textual-based featuring approach for depression detection using machine learning classifiers and social media texts,” Comput. Biol. Med., vol. 135, p. 104499, Aug. 2021, doi: 10.1016/j.compbiomed.2021.104499.

[8] M. Garg, C. Saxena, S. Saha, V. Krishnan, R. Joshi, and V. Mago, “CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Posts,” presented at the Thirteenth Language Resources and Evaluation Conference, Marseille, France, Jun. 2022, pp. 6387–6396. doi: 10.63317/3tbejaye7i8s.

[9] S. Zanwar, D. Wiechmann, Y. Qiao, and E. Kerz, “SMHD-GER: A Large-Scale Benchmark Dataset for Automatic Mental Health Detection from Social Media in German,” in Findings of the Association for Computational Linguistics: EACL 2023, Dubrovnik, Croatia: Association for Computational Linguistics, 2023, pp. 1526–1541. doi: 10.18653/v1/2023.findings-eacl.113.

[10] K. Jawad, M. Ahmad, M. Alvi, and M. B. Alvi, “RUSAS: Roman Urdu Sentiment Analysis System,” Comput. Mater. Contin., vol. 79, no. 1, pp. 1463–1480, 2024, doi: 10.32604/cmc.2024.047466.

[11] M. Ahmad, P. Basile, F. Ullah, I. Batyrshin, and G. Sidorov, “RUDA-2025: Depression Severity Detection Using Pre-Trained Transformers on Social Media Data,” AI, vol. 6, no. 8, p. 191, Aug. 2025, doi: 10.3390/ai6080191.

[12] A. Qasim, G. Mehak, N. Hussain, A. Gelbukh, and G. Sidorov, “Detection of Depression Severity in Social Media Text Using Transformer-Based Models,” Information, vol. 16, no. 2, p. 114, Feb. 2025, doi: 10.3390/info16020114.

[13] M. Kabir et al., “DEPTWEET: A typology for social media texts to detect depression severities,” Comput. Hum. Behav., vol. 139, p. 107503, Feb. 2023, doi: 10.1016/j.chb.2022.107503.

[14] R. Mohmand, U. Habib, M. Usman, J. Baili, and Y. Nam, “A Deep Learning Approach for Automated Depression Assessment Using Roman Urdu,” IEEE Access, vol. 12, pp. 193387–193401, 2024, doi: 10.1109/ACCESS.2024.3519264.

[15] L. Ilias, S. Mouzakitis, and D. Askounis, “Calibration of Transformer-Based Models for Identifying Stress and Depression in Social Media,” IEEE Trans. Comput. Soc. Syst., vol. 11, no. 2, pp. 1979–1990, Apr. 2024, doi: 10.1109/TCSS.2023.3283009.

[16] B. G. Bokolo and Q. Liu, “Deep Learning-Based Depression Detection from Social Media: Comparative Evaluation of ML and Transformer Techniques,” Electronics, vol. 12, no. 21, p. 4396, Oct. 2023, doi: 10.3390/electronics12214396.

[17] C. Chen, F. Li, H. Chen, and Y. Lin, “Heterogeneous subgraph network with prompt learning for interpretable depression detection on social media,” Knowl.-Based Syst., vol. 315, p. 113215, Apr. 2025, doi: 10.1016/j.knosys.2025.113215.

[18] A. Majeed, M. O. Beg, U. Arshad, and H. Mujtaba, “Deep-EmoRU: mining emotions from roman urdu text using deep learning ensemble,” Multimed. Tools Appl., vol. 81, no. 30, pp. 43163–43188, Dec. 2022, doi: 10.1007/s11042-022-13147-w.

[19] S. Ghosh and T. Anwar, “Depression Intensity Estimation via Social Media: A Deep Learning Approach,” IEEE Trans. Comput. Soc. Syst., vol. 8, no. 6, pp. 1465–1474, Dec. 2021, doi: 10.1109/TCSS.2021.3084154.

[20] Z. N. Vasha, B. Sharma, I. J. Esha, J. Al Nahian, and J. A. Polin, “Depression detection in social media comments data using machine learning algorithms,” Bull. Electr. Eng. Inform., vol. 12, no. 2, pp. 987–996, Apr. 2023, doi: 10.11591/eei.v12i2.4182.

[21] S. Hameed, M. Nauman, N. Akhtar, M. A. B. Fayyaz, and R. Nawaz, “Explainable AI-driven depression detection from social media using natural language processing and black box machine learning models,” Front. Artif. Intell., vol. 8, p. 1627078, Sep. 2025, doi: 10.3389/frai.2025.1627078.