Performance Analysis of IndoBERT Fine-Tuning for Indonesian Text-Based Hoax Detection
Keywords:
IndoBERT, IndoBERTweet, fine-tuning, hoax detection, social media, natural language, processing (NLP), text cllasification
Abstract
Abstract
The spread of misinformation (hoaxes) on social media, particularly on the X platform, has become a significant challenge that can influence public understanding and perceptions of various issues, including those related to education. This study aims to analyze and improve the performance of an Indonesian-language hoax detection model through a fine-tuning approach using the IndoBERT model. The dataset used in this research was collected through web scraping from the X platform and classified into two categories: hoax and non-hoax. Prior to model training, the data underwent several preprocessing stages, including data cleansing, normalization, tokenization, stopword removal, and stemming.
This study utilized the pre-trained IndoBERTweet model with two data-splitting scenarios, namely 70:30 and 80:20, along with various combinations of learning rates and epochs to determine the optimal configuration. Model performance was evaluated using accuracy, precision, recall, and F1-score metrics. The experimental results indicate that the best performance was achieved under the 70:30 data-splitting scenario with a learning rate of 2e-5 and 20 epochs. On the testing dataset, the model achieved an accuracy of 80%, precision of 83%, recall of 78%, and an F1-score of 80%.
The findings demonstrate that the fine-tuning approach applied to IndoBERT is effective in enhancing the performance of Indonesian-language hoax classification and is capable of maintaining strong performance when evaluated on previously unseen data.
Keywords: IndoBERT, IndoBERTweet, fine-tuning, hoax detection, social media, natural language processing (NLP), text classification.
References
REFERENCES
Alkhodair, S. A., Ding, S. H. H., Fung, B. C. M., & Liu, J. (2020). Detecting fake news in social media networks. Computers in Human Behavior, 104.
Antariksa, M. D. S., Sugiharto, A., & Surarso, B. (2025). BERT model fine-tuned for scientific document classification and recommendation. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 9(4), 754–764. https://doi.org/10.29207/resti.v9i4.6789
Ardakani, A., Haan, A., Tan, S., Popovici, D. T., Cheung, A., Iancu, C., & Sen, K. (2023). SlimFit: Memory-efficient fine-tuning of transformer-based models using training dynamics.
Basahel, A. M., Giriyappa, S. H., Alam, F., Alnazzawi, T. S. M., Qamar, S., & Abi Sen, A. A. (2026a). A resource-efficient approach to fine-tuning a BERT-base model for sentiment analysis. Computers, 15(3), 159. https://doi.org/10.3390/computers15030159
Basahel, A. M., Giriyappa, S. H., Alam, F., Alnazzawi, T. S. M., Qamar, S., & Abi Sen, A. A. (2026b). A resource-efficient approach to fine-tuning a BERT-base model for sentiment analysis. Computers, 15(3), 159. https://doi.org/10.3390/computers15030159
De Diego, I. M., Redondo, A. R., Fernández, R. R., Navarro, J., & Moguerza, J. M. (2022). General performance score for classification problems. Applied Intelligence, 52(10), 12049–12063. https://doi.org/10.1007/s10489-021-03041-7
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding.
Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., Yi, J., Zhao, W., Wang, X., Liu, Z., Zheng, H.-T., Chen, J., Liu, Y., Tang, J., Li, J., & Sun, M. (2023). Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3), 220–235. https://doi.org/10.1038/s42256-023-00626-4
Düntsch, I., & Gediga, G. (2019). Confusion matrices and rough set data analysis. Journal of Physics: Conference Series, 1229(1), 012055. https://doi.org/10.1088/1742-6596/1229/1/012055
Gorunescu, F. (2011). Data mining. Springer.
Hua, H., Li, X., Dou, D., Xu, C., & Luo, J. (2021). Noise stability regularization for improving BERT fine-tuning. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 3229–3241. https://doi.org/10.18653/v1/2021.naacl-main.258
Isadora, F. R., Hanggara, B. T., & Mursityo, Y. T. (2021). User experience design for the HomeCare mobile application of Semen Gresik Hospital using the Design Thinking method. Jurnal Teknologi Informasi dan Ilmu Komputer, 8(5), 1057–1066. https://doi.org/10.25126/jtiik.2021844550
Jocelynne, C., Tobing, L., Wijayakusuma, I. L., Harini, L. P., & Harini, I. (2025). Detection of political hoax news using fine-tuning IndoBERT. Journal of Applied Informatics and Computing (JAIC), 9(2).
Kiran, S., Kumar, A. A., Reddy, R. P. K., & Reddy, D. S. (2022). Analysis of in-place matrix rotation of square matrix for information security applications. Procedia Computer Science, 215, 131–139. https://doi.org/10.1016/j.procs.2022.12.015
Koto, F., Rahimi, A., Lau, J. H., & Baldwin, T. (2020). IndoLEM and IndoBERT: A benchmark dataset and pre-trained language model for Indonesian NLP. Proceedings of the 28th International Conference on Computational Linguistics, 757–770. https://doi.org/10.18653/v1/2020.coling-main.66
Koto, F., Lau, J. H., & Baldwin, T. (n.d.). IndoBERTweet: A pretrained language model for Indonesian Twitter with effective domain-specific vocabulary initialization. https://huggingface.co/huseinzol05/
Li, H., Chaudhari, P., Yang, H., Lam, M., Ravichandran, A., Bhotika, R., & Soatto, S. (2020). Rethinking the hyperparameters for fine-tuning.
Lin, T., Wang, Y., Liu, X., & Qiu, X. (2022). A survey of transformers. AI Open, 3, 111–132. https://doi.org/10.1016/j.aiopen.2022.10.001
Lu, W., Luu, R. K., & Buehler, M. J. (2025). Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities. NPJ Computational Materials, 11(1), 84. https://doi.org/10.1038/s41524-025-01564-y
Mahendra, M., & Triandi, R. (n.d.). Analisa dan deteksi konten hoax pada media berita Indonesia menggunakan machine learning.
Mosbach, M., Andriushchenko, M., & Klakow, D. (2021a). On the stability of fine-tuning BERT: Misconceptions, explanations, and strong baselines.
Mosbach, M., Andriushchenko, M., & Klakow, D. (2021b). On the stability of fine-tuning BERT: Misconceptions, explanations, and strong baselines.
Nuryanto, N. (2025). Prosiding Seminar Nasional SI & Teknik Informatika (pp. 134–140).
Oktaviana Yamin, V., Tenriawaru, A., Saidi, L. O., Rahman, G. A., et al. (2023). Penerapan Naïve Bayes classifier dengan algoritma Nazief dan Adriani untuk deteksi hoaks. Journal of Information Technology and Computer Science, 1(1).
Pranesti, D., & Arifin. (2019). Perlindungan korban dalam kasus penyebaran berita hoax di media sosial di Indonesia. Jurnal Hukum Media Bhakti, 3(1), 8–17.
Priyanka Arora, & Arora, P. (2019). Mining Twitter data for depression detection. IEEE, 186–189.
Purohit, H., Hampton, A., Shalin, V. L., Sheth, A. P., Flach, J., & Bhatt, S. (2013). What kind of #conversation is Twitter? Mining psycholinguistic cues for emergency coordination. Computers in Human Behavior, 29.
Raghu, D., Gupta, N., & Mausam. (2021). Unsupervised learning of KB queries in task-oriented dialogs. Transactions of the Association for Computational Linguistics, 9, 374–390. https://doi.org/10.1162/tacl_a_00372
Rahayu, R. N. (2020). Analisis berita hoax COVID-19 di media sosial di Indonesia. Jurnal Ekonomi, Sosial & Humaniora, 1(9), 60–73.
Raza, M. N. (n.d.). Sistem deteksi berita hoax menggunakan algoritma Naïve Bayes dan Random Forest pada machine learning.
Riyaddulloh, R., & Romadhony, A. (2021). Normalisasi teks bahasa Indonesia berbasis kamus slang: Studi kasus tweet produk gadget pada Twitter. Jurnal Teknologi Informasi, 8(4).
Sinapoy, M. I. K., Sibaroni, Y., & Prasetyowati, S. S. (2023). Comparison of LSTM and IndoBERT method in identifying hoax on Twitter. Jurnal RESTI, 7(3), 657–662. https://doi.org/10.29207/resti.v7i3.4830
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023). Efficient transformers: A survey. ACM Computing Surveys, 55(6), 1–28. https://doi.org/10.1145/3530811
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359, 1146–1151.
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025). Parameter-efficient fine-tuning in large language models: A survey of methodologies. Artificial Intelligence Review, 58(8), 227. https://doi.org/10.1007/s10462-025-11236-4
Wang, W., & Pan, S. J. (2021). Variational deep logic network for joint inference of entities and relations. Computational Linguistics, 47(4), 775–812. https://doi.org/10.1162/coli_a_00415
Zaidan, M. N., Sibaroni, Y., & Prasetyowati, S. S. (2024a). Learning rate and epoch optimization in the fine-tuning process for IndoBERT's performance on sentiment analysis of MyTelkomsel app reviews. Jurnal Teknik Informatika (JUTIF), 5(5), 1443–1450. https://doi.org/10.52436/1.jutif.2024.5.5.2396
Zaidan, M. N., Sibaroni, Y., & Prasetyowati, S. S. (2024b). Learning rate and epoch optimization in the fine-tuning process for IndoBERT's performance on sentiment analysis of MyTelkomsel app reviews. Jurnal Teknik Informatika (JUTIF), 5(5), 1443–1450. https://doi.org/10.52436/1.jutif.2024.5.5.2396
Zhang, C., Liu, Z., & Sun, M. (2018). Enhancing Naïve Bayes with contextual embeddings for text classification tasks. Proceedings of the Association for Computational Linguistics (ACL), 10(2), 145–153.
Alkhodair, S. A., Ding, S. H. H., Fung, B. C. M., & Liu, J. (2020). Detecting fake news in social media networks. Computers in Human Behavior, 104.
Antariksa, M. D. S., Sugiharto, A., & Surarso, B. (2025). BERT model fine-tuned for scientific document classification and recommendation. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 9(4), 754–764. https://doi.org/10.29207/resti.v9i4.6789
Ardakani, A., Haan, A., Tan, S., Popovici, D. T., Cheung, A., Iancu, C., & Sen, K. (2023). SlimFit: Memory-efficient fine-tuning of transformer-based models using training dynamics.
Basahel, A. M., Giriyappa, S. H., Alam, F., Alnazzawi, T. S. M., Qamar, S., & Abi Sen, A. A. (2026a). A resource-efficient approach to fine-tuning a BERT-base model for sentiment analysis. Computers, 15(3), 159. https://doi.org/10.3390/computers15030159
Basahel, A. M., Giriyappa, S. H., Alam, F., Alnazzawi, T. S. M., Qamar, S., & Abi Sen, A. A. (2026b). A resource-efficient approach to fine-tuning a BERT-base model for sentiment analysis. Computers, 15(3), 159. https://doi.org/10.3390/computers15030159
De Diego, I. M., Redondo, A. R., Fernández, R. R., Navarro, J., & Moguerza, J. M. (2022). General performance score for classification problems. Applied Intelligence, 52(10), 12049–12063. https://doi.org/10.1007/s10489-021-03041-7
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding.
Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., Yi, J., Zhao, W., Wang, X., Liu, Z., Zheng, H.-T., Chen, J., Liu, Y., Tang, J., Li, J., & Sun, M. (2023). Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3), 220–235. https://doi.org/10.1038/s42256-023-00626-4
Düntsch, I., & Gediga, G. (2019). Confusion matrices and rough set data analysis. Journal of Physics: Conference Series, 1229(1), 012055. https://doi.org/10.1088/1742-6596/1229/1/012055
Gorunescu, F. (2011). Data mining. Springer.
Hua, H., Li, X., Dou, D., Xu, C., & Luo, J. (2021). Noise stability regularization for improving BERT fine-tuning. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 3229–3241. https://doi.org/10.18653/v1/2021.naacl-main.258
Isadora, F. R., Hanggara, B. T., & Mursityo, Y. T. (2021). User experience design for the HomeCare mobile application of Semen Gresik Hospital using the Design Thinking method. Jurnal Teknologi Informasi dan Ilmu Komputer, 8(5), 1057–1066. https://doi.org/10.25126/jtiik.2021844550
Jocelynne, C., Tobing, L., Wijayakusuma, I. L., Harini, L. P., & Harini, I. (2025). Detection of political hoax news using fine-tuning IndoBERT. Journal of Applied Informatics and Computing (JAIC), 9(2).
Kiran, S., Kumar, A. A., Reddy, R. P. K., & Reddy, D. S. (2022). Analysis of in-place matrix rotation of square matrix for information security applications. Procedia Computer Science, 215, 131–139. https://doi.org/10.1016/j.procs.2022.12.015
Koto, F., Rahimi, A., Lau, J. H., & Baldwin, T. (2020). IndoLEM and IndoBERT: A benchmark dataset and pre-trained language model for Indonesian NLP. Proceedings of the 28th International Conference on Computational Linguistics, 757–770. https://doi.org/10.18653/v1/2020.coling-main.66
Koto, F., Lau, J. H., & Baldwin, T. (n.d.). IndoBERTweet: A pretrained language model for Indonesian Twitter with effective domain-specific vocabulary initialization. https://huggingface.co/huseinzol05/
Li, H., Chaudhari, P., Yang, H., Lam, M., Ravichandran, A., Bhotika, R., & Soatto, S. (2020). Rethinking the hyperparameters for fine-tuning.
Lin, T., Wang, Y., Liu, X., & Qiu, X. (2022). A survey of transformers. AI Open, 3, 111–132. https://doi.org/10.1016/j.aiopen.2022.10.001
Lu, W., Luu, R. K., & Buehler, M. J. (2025). Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities. NPJ Computational Materials, 11(1), 84. https://doi.org/10.1038/s41524-025-01564-y
Mahendra, M., & Triandi, R. (n.d.). Analisa dan deteksi konten hoax pada media berita Indonesia menggunakan machine learning.
Mosbach, M., Andriushchenko, M., & Klakow, D. (2021a). On the stability of fine-tuning BERT: Misconceptions, explanations, and strong baselines.
Mosbach, M., Andriushchenko, M., & Klakow, D. (2021b). On the stability of fine-tuning BERT: Misconceptions, explanations, and strong baselines.
Nuryanto, N. (2025). Prosiding Seminar Nasional SI & Teknik Informatika (pp. 134–140).
Oktaviana Yamin, V., Tenriawaru, A., Saidi, L. O., Rahman, G. A., et al. (2023). Penerapan Naïve Bayes classifier dengan algoritma Nazief dan Adriani untuk deteksi hoaks. Journal of Information Technology and Computer Science, 1(1).
Pranesti, D., & Arifin. (2019). Perlindungan korban dalam kasus penyebaran berita hoax di media sosial di Indonesia. Jurnal Hukum Media Bhakti, 3(1), 8–17.
Priyanka Arora, & Arora, P. (2019). Mining Twitter data for depression detection. IEEE, 186–189.
Purohit, H., Hampton, A., Shalin, V. L., Sheth, A. P., Flach, J., & Bhatt, S. (2013). What kind of #conversation is Twitter? Mining psycholinguistic cues for emergency coordination. Computers in Human Behavior, 29.
Raghu, D., Gupta, N., & Mausam. (2021). Unsupervised learning of KB queries in task-oriented dialogs. Transactions of the Association for Computational Linguistics, 9, 374–390. https://doi.org/10.1162/tacl_a_00372
Rahayu, R. N. (2020). Analisis berita hoax COVID-19 di media sosial di Indonesia. Jurnal Ekonomi, Sosial & Humaniora, 1(9), 60–73.
Raza, M. N. (n.d.). Sistem deteksi berita hoax menggunakan algoritma Naïve Bayes dan Random Forest pada machine learning.
Riyaddulloh, R., & Romadhony, A. (2021). Normalisasi teks bahasa Indonesia berbasis kamus slang: Studi kasus tweet produk gadget pada Twitter. Jurnal Teknologi Informasi, 8(4).
Sinapoy, M. I. K., Sibaroni, Y., & Prasetyowati, S. S. (2023). Comparison of LSTM and IndoBERT method in identifying hoax on Twitter. Jurnal RESTI, 7(3), 657–662. https://doi.org/10.29207/resti.v7i3.4830
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023). Efficient transformers: A survey. ACM Computing Surveys, 55(6), 1–28. https://doi.org/10.1145/3530811
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359, 1146–1151.
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025). Parameter-efficient fine-tuning in large language models: A survey of methodologies. Artificial Intelligence Review, 58(8), 227. https://doi.org/10.1007/s10462-025-11236-4
Wang, W., & Pan, S. J. (2021). Variational deep logic network for joint inference of entities and relations. Computational Linguistics, 47(4), 775–812. https://doi.org/10.1162/coli_a_00415
Zaidan, M. N., Sibaroni, Y., & Prasetyowati, S. S. (2024a). Learning rate and epoch optimization in the fine-tuning process for IndoBERT's performance on sentiment analysis of MyTelkomsel app reviews. Jurnal Teknik Informatika (JUTIF), 5(5), 1443–1450. https://doi.org/10.52436/1.jutif.2024.5.5.2396
Zaidan, M. N., Sibaroni, Y., & Prasetyowati, S. S. (2024b). Learning rate and epoch optimization in the fine-tuning process for IndoBERT's performance on sentiment analysis of MyTelkomsel app reviews. Jurnal Teknik Informatika (JUTIF), 5(5), 1443–1450. https://doi.org/10.52436/1.jutif.2024.5.5.2396
Zhang, C., Liu, Z., & Sun, M. (2018). Enhancing Naïve Bayes with contextual embeddings for text classification tasks. Proceedings of the Association for Computational Linguistics (ACL), 10(2), 145–153.
Published
2026-08-04
How to Cite
Siti, T., & Muzakir, A. (2026). Performance Analysis of IndoBERT Fine-Tuning for Indonesian Text-Based Hoax Detection. International Journal of Education, Vocational and Social Science , 5(03), 6910-6922. https://doi.org/10.63922/ijevss.v5i03.5764
Contact Person +6282139474255
Email : ijevss@citakonsultindo.or.id / editor.ijevss@gmail.com
