When Does Artificial Intelligence Improve Academic Achievement? The Moderating Role of Academic Self-Efficacy: Evidence from Hierarchical and Moderated Regression Analyses Using R
Keywords:
Artificial intelligence, Academic self-efficacy, Academic achievement
Abstract
The rapid integration of Artificial Intelligence (AI) in higher education presents new learning opportunities, yet its direct impact on academic achievement remains inconclusive. This study investigates the effect of AI utilization on students' academic achievement and examines the moderating role of academic self-efficacy. Employing a quantitative survey design, data were collected from 260 undergraduate students in Indonesia. Hierarchical and moderated regression analyses revealed that AI utilization does not exert a significant direct effect on academic achievement. Conversely, academic self-efficacy emerged as a robust positive predictor. Crucially, academic self-efficacy significantly moderated the relationship between AI use and academic achievement. Simple slope analysis demonstrated that AI utilization positively predicts academic achievement exclusively among students with low levels of academic self-efficacy, functioning as a compensatory support mechanism. For students with high self-efficacy, the marginal benefits of AI were non-significant. These findings indicate that the academic benefits of AI are highly conditional and dependent on learners' psychological characteristics. The study underscores the necessity of fostering academic self-efficacy alongside technological integration to optimize learning outcomes in the AI era.
References
Alam, M. N., Islam, M. A., Babiker, M. O. A., Siddiqui, M. S., Amin, M. Bin, & Oláh, J. (2026). AI-assisted learning tools and student learning outcomes: A cognitive load theory perspective. Computers in Human Behavior Reports, 21, 100986. https://doi.org/10.1016/J.CHBR.2026.100986
Chan, C. K. Y., & Hu, W. (2024). Students’ Voices on Generative AI: Perceptions, Benefits, and Challenges in Higher Education. Computers and Education: Artificial Intelligence, 7, 100270. https://doi.org/10.1016/j.caeai.2024.100270
Crompton, H., & Burke, D. (2023). Artificial Intelligence in Higher Education: The State of the Field. International Journal of Educational Technology in Higher Education, 20(22), 1–22. https://doi.org/10.1186/s41239-023-00392-8
Hassen, M. Z. (2025). The Impact of AI on Students’ Reading, Critical Thinking, and Problem-Solving Skills. American Journal of Education and Information Technology 2025, Volume 9, Page 82, 9(2), 82–90. https://doi.org/10.11648/J.AJEIT.20250902.12
Huang, A. Y. Q., Lu, O. H. T., & Yang, S. J. H. (2023). Effects of Artificial Intelligence-Enabled Personalized Recommendations on Learners’ Engagement, Motivation, and Outcomes. Computers & Education, 194, 104684. https://doi.org/10.1016/j.compedu.2022.104684
Huang, X., Chang, T.-W., & Chen, N.-S. (2025). Generative Artificial Intelligence in Education: Opportunities, Challenges, and Future Directions. Educational Technology & Society, 28(1), 1–15.
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeiffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & others. (2023). ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Khalil, H., Alfazari, S., Eid, M., Hassan, A. M., Kamal, A. A., & Khatiry, A. (2026). Human and AI support in higher education: a hybrid model for an enhanced learning experience. Frontiers in Education, 11, 1772646. https://doi.org/10.3389/FEDUC.2026.1772646/TEXT
Kizilcec, R. F. (2023). To Advance AI Use in Education, Focus on Understanding Educators. International Journal of Artificial Intelligence in Education, 34(1), 12–19. https://doi.org/10.1007/s40593-023-00326-5
Kizilcec, R. F. (2024). Reimagining Learning with AI: Opportunities, Challenges, and Implications for Education. Nature Human Behaviour, 8(1), 12–18. https://doi.org/10.1038/s41562-023-01742-1
Krisnaresanti, A., Suwatno, S., Mulyadi, H., & Utama, D. H. (2026). The Power of Belief: How Self-Efficacy Shapes Teacher Readiness through Technological Pedagogical Content Knowledge-Based Dynamic Capabilities. Munaddhomah: Jurnal Manajemen Pendidikan Islam, 7(1), 125–140. https://doi.org/10.31538/MUNADDHOMAH.V7I1.2186
Leite, H., Luna-Nevarez, C., Baruca, A., & Joubert, A. M. (2026). “When Gen Z meets GenAI”: The perfect storm in higher education. Technological Forecasting and Social Change, 229, 124692. https://doi.org/10.1016/J.TECHFORE.2026.124692
Maslakhah, C. L., Wardani, F. D., Nahdiyant, I., & Reggia, S. H. D. (2023). The Role of Self-Efficacy on Student Academic Achievement. International Conference on Psychology and Education (ICPE), 2(1), 18–23. https://proceeding.unesa.ac.id/index.php/icpe/article/view/5786
Miao, H., Guo, R., & Li, M. (2025). The influence of research self-efficacy and learning engagement on Ed.D students’ academic achievement. Frontiers in Psychology, 16, 1562354. https://doi.org/10.3389/FPSYG.2025.1562354/FULL
Navarro, R., Vega, V., Bayona, H., Bernal, V., & Garcia, A. (2023). Relationship between technology acceptance model, self-regulation strategies, and academic self-efficacy with academic performance and perceived learning among college students during remote education. Frontiers in Psychology, 14, 1227956. https://doi.org/10.3389/FPSYG.2023.1227956/TEXT
Nzenwata, N., Okoye, K., Nwafor, C., & Adeyemi, A. (2025). Artificial Intelligence Dependency and Student Learning Outcomes in Higher Education: A Systematic Review. Computers and Education: Artificial Intelligence, 8, 100312. https://doi.org/10.1016/j.caeai.2025.100312
Panadero, E., Broadbent, J., Boud, D., & Lodge, J. M. (2022). Using Formative Assessment to Influence Self-Regulated Learning: A Systematic Review. Educational Psychology Review, 34(2), 847–875. https://doi.org/10.1007/s10648-021-09630-w
Rosiyanti, H., Priatna, N., & Turmudi. (2025). Self-Efficacy in the context of learning strategies: A systematic literature review. Indonesian Research Journal in Education |IRJE|, 9(01), 347–361. https://doi.org/10.22437/irje.v9i01.44097
Schunk, D. H., & DiBenedetto, M. K. (2020). Motivation and Social Cognitive Theory. Contemporary Educational Psychology, 60, 101832. https://doi.org/10.1016/j.cedpsych.2019.101832
Strzelecki, A. (2024). Students’ Acceptance of ChatGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology. Innovative Higher Education, 49(2), 223–245. https://doi.org/10.1007/s10755-023-09686-1
Sui, C. J., Yen, M. H., & Chang, C. Y. (2023). Investigating effects of perceived technology-enhanced environment on self-regulated learning. Education and Information Technologies 2023 29:1, 29(1), 161–183. https://doi.org/10.1007/S10639-023-12270-X
Sultana, N., Saeed, A., Fatima, N., & Jahangir, A. (2025). AI Enhanced Learning System and their Effect on Students Psychological Health Academic Confidence and Social Engagement. Review of Applied Management and Social Sciences, 8(4), 1577–1589. https://doi.org/10.47067/RAMSS.V8I4.589
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the Devil is My Guardian Angel: ChatGPT as a Case Study of Using Chatbots in Education. Smart Learning Environments, 10(15), 1–24. https://doi.org/10.1186/s40561-023-00237-x
Vieriu, A. M., & Petrea, G. (2025). The Impact of Artificial Intelligence (AI) on Students’ Academic Development. Education Sciences 2025, Vol. 15, Page 343, 15(3), 343. https://doi.org/10.3390/EDUCSCI15030343
Wang, S., Wang, F., Zhu, Z., Wang, J., Tran, T., & Du, Z. (2024). Artificial Intelligence in Education: A Systematic Literature Review. Expert Systems with Applications, 252, 124167. https://doi.org/10.1016/j.eswa.2024.124167
Zahid, H., Danish, M., Ali, S., Huo, C., & Khalid, N. (2026). Generative AI chatbots in higher education: addiction, cognitive offloading, and the role of AI literacy. Interactive Learning Environments. https://doi.org/10.1080/10494820.2026.2677727
Zhang, J., Wang, C. Z., & Liu, M. (2026). Generative AI use profiles and academic outcomes in higher education: the role of institutional support. Interactive Learning Environments. https://doi.org/10.1080/10494820.2026.2668610
Zhao, Y., Yue, Y., Sun, Z., Jiang, Q., & Li, G. (2025). Does Generative Artificial Intelligence Improve Students’ Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments. Journal of Intelligence 2025, Vol. 13, Page 160, 13(12), 160. https://doi.org/10.3390/JINTELLIGENCE13120160
Chan, C. K. Y., & Hu, W. (2024). Students’ Voices on Generative AI: Perceptions, Benefits, and Challenges in Higher Education. Computers and Education: Artificial Intelligence, 7, 100270. https://doi.org/10.1016/j.caeai.2024.100270
Crompton, H., & Burke, D. (2023). Artificial Intelligence in Higher Education: The State of the Field. International Journal of Educational Technology in Higher Education, 20(22), 1–22. https://doi.org/10.1186/s41239-023-00392-8
Hassen, M. Z. (2025). The Impact of AI on Students’ Reading, Critical Thinking, and Problem-Solving Skills. American Journal of Education and Information Technology 2025, Volume 9, Page 82, 9(2), 82–90. https://doi.org/10.11648/J.AJEIT.20250902.12
Huang, A. Y. Q., Lu, O. H. T., & Yang, S. J. H. (2023). Effects of Artificial Intelligence-Enabled Personalized Recommendations on Learners’ Engagement, Motivation, and Outcomes. Computers & Education, 194, 104684. https://doi.org/10.1016/j.compedu.2022.104684
Huang, X., Chang, T.-W., & Chen, N.-S. (2025). Generative Artificial Intelligence in Education: Opportunities, Challenges, and Future Directions. Educational Technology & Society, 28(1), 1–15.
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeiffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & others. (2023). ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Khalil, H., Alfazari, S., Eid, M., Hassan, A. M., Kamal, A. A., & Khatiry, A. (2026). Human and AI support in higher education: a hybrid model for an enhanced learning experience. Frontiers in Education, 11, 1772646. https://doi.org/10.3389/FEDUC.2026.1772646/TEXT
Kizilcec, R. F. (2023). To Advance AI Use in Education, Focus on Understanding Educators. International Journal of Artificial Intelligence in Education, 34(1), 12–19. https://doi.org/10.1007/s40593-023-00326-5
Kizilcec, R. F. (2024). Reimagining Learning with AI: Opportunities, Challenges, and Implications for Education. Nature Human Behaviour, 8(1), 12–18. https://doi.org/10.1038/s41562-023-01742-1
Krisnaresanti, A., Suwatno, S., Mulyadi, H., & Utama, D. H. (2026). The Power of Belief: How Self-Efficacy Shapes Teacher Readiness through Technological Pedagogical Content Knowledge-Based Dynamic Capabilities. Munaddhomah: Jurnal Manajemen Pendidikan Islam, 7(1), 125–140. https://doi.org/10.31538/MUNADDHOMAH.V7I1.2186
Leite, H., Luna-Nevarez, C., Baruca, A., & Joubert, A. M. (2026). “When Gen Z meets GenAI”: The perfect storm in higher education. Technological Forecasting and Social Change, 229, 124692. https://doi.org/10.1016/J.TECHFORE.2026.124692
Maslakhah, C. L., Wardani, F. D., Nahdiyant, I., & Reggia, S. H. D. (2023). The Role of Self-Efficacy on Student Academic Achievement. International Conference on Psychology and Education (ICPE), 2(1), 18–23. https://proceeding.unesa.ac.id/index.php/icpe/article/view/5786
Miao, H., Guo, R., & Li, M. (2025). The influence of research self-efficacy and learning engagement on Ed.D students’ academic achievement. Frontiers in Psychology, 16, 1562354. https://doi.org/10.3389/FPSYG.2025.1562354/FULL
Navarro, R., Vega, V., Bayona, H., Bernal, V., & Garcia, A. (2023). Relationship between technology acceptance model, self-regulation strategies, and academic self-efficacy with academic performance and perceived learning among college students during remote education. Frontiers in Psychology, 14, 1227956. https://doi.org/10.3389/FPSYG.2023.1227956/TEXT
Nzenwata, N., Okoye, K., Nwafor, C., & Adeyemi, A. (2025). Artificial Intelligence Dependency and Student Learning Outcomes in Higher Education: A Systematic Review. Computers and Education: Artificial Intelligence, 8, 100312. https://doi.org/10.1016/j.caeai.2025.100312
Panadero, E., Broadbent, J., Boud, D., & Lodge, J. M. (2022). Using Formative Assessment to Influence Self-Regulated Learning: A Systematic Review. Educational Psychology Review, 34(2), 847–875. https://doi.org/10.1007/s10648-021-09630-w
Rosiyanti, H., Priatna, N., & Turmudi. (2025). Self-Efficacy in the context of learning strategies: A systematic literature review. Indonesian Research Journal in Education |IRJE|, 9(01), 347–361. https://doi.org/10.22437/irje.v9i01.44097
Schunk, D. H., & DiBenedetto, M. K. (2020). Motivation and Social Cognitive Theory. Contemporary Educational Psychology, 60, 101832. https://doi.org/10.1016/j.cedpsych.2019.101832
Strzelecki, A. (2024). Students’ Acceptance of ChatGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology. Innovative Higher Education, 49(2), 223–245. https://doi.org/10.1007/s10755-023-09686-1
Sui, C. J., Yen, M. H., & Chang, C. Y. (2023). Investigating effects of perceived technology-enhanced environment on self-regulated learning. Education and Information Technologies 2023 29:1, 29(1), 161–183. https://doi.org/10.1007/S10639-023-12270-X
Sultana, N., Saeed, A., Fatima, N., & Jahangir, A. (2025). AI Enhanced Learning System and their Effect on Students Psychological Health Academic Confidence and Social Engagement. Review of Applied Management and Social Sciences, 8(4), 1577–1589. https://doi.org/10.47067/RAMSS.V8I4.589
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the Devil is My Guardian Angel: ChatGPT as a Case Study of Using Chatbots in Education. Smart Learning Environments, 10(15), 1–24. https://doi.org/10.1186/s40561-023-00237-x
Vieriu, A. M., & Petrea, G. (2025). The Impact of Artificial Intelligence (AI) on Students’ Academic Development. Education Sciences 2025, Vol. 15, Page 343, 15(3), 343. https://doi.org/10.3390/EDUCSCI15030343
Wang, S., Wang, F., Zhu, Z., Wang, J., Tran, T., & Du, Z. (2024). Artificial Intelligence in Education: A Systematic Literature Review. Expert Systems with Applications, 252, 124167. https://doi.org/10.1016/j.eswa.2024.124167
Zahid, H., Danish, M., Ali, S., Huo, C., & Khalid, N. (2026). Generative AI chatbots in higher education: addiction, cognitive offloading, and the role of AI literacy. Interactive Learning Environments. https://doi.org/10.1080/10494820.2026.2677727
Zhang, J., Wang, C. Z., & Liu, M. (2026). Generative AI use profiles and academic outcomes in higher education: the role of institutional support. Interactive Learning Environments. https://doi.org/10.1080/10494820.2026.2668610
Zhao, Y., Yue, Y., Sun, Z., Jiang, Q., & Li, G. (2025). Does Generative Artificial Intelligence Improve Students’ Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments. Journal of Intelligence 2025, Vol. 13, Page 160, 13(12), 160. https://doi.org/10.3390/JINTELLIGENCE13120160
Published
2026-07-15
How to Cite
Inayah Dzil Izzati Hartono, Syukrul Hamdi, & Andina Nurul Wahidah. (2026). When Does Artificial Intelligence Improve Academic Achievement? The Moderating Role of Academic Self-Efficacy: Evidence from Hierarchical and Moderated Regression Analyses Using R. International Journal of Education, Vocational and Social Science , 5(03), 4897-4913. https://doi.org/10.63922/ijevss.v5i03.5208
Contact Person +6282139474255
Email : ijevss@citakonsultindo.or.id / editor.ijevss@gmail.com
